AI Governance: LLMs, Fraud and Solutions with Daniel Turner

In this episode, with Daniel Turner of Financial Crime Dynamics (FCD), we explore some varying definitions and facets of artificial intelligence to include LLMs, generative AI, in addition to challenges and solutions related to fraud. We provide a dynamic and broad survey of new regulations and public policy issues. We finish up with a specific example of how synthetic data and data democratization come to the rescue in the domain of financial crime. 

Transcript

Paul Starrett: Hello, and welcome to another podcast of PrivacyLabs. My name is Paul Starrett. I am Co-Founder of PrivacyLabs. And like other podcasts before, we are going to be talking about a topic that is near and dear to the hearts of data scientists and those involved in artificial intelligence. And that has to do with synthetic data and similar topics. As we get into our podcast, this will be fleshed out more. 

I’m honored to have with us as a guest today, Daniel Turner, who is a Co-Founder and Chief Product Officer of Financial Crime Dynamics. Very briefly, I have known Financial Crime Dynamics for a while back when they were called Ealax. And I’ve also got a podcast with one of the co-founders, Edgar Lopez, which you will find in our podcast series. 

So, without further ado, what we’re going to do basically is just kind of go through a high level treatment, if you will, of what is artificial intelligence. We all have some notion of that. If you’re listening to this, you probably have some notion of what we’re talking about. But with the recent advent of ChatGPT and generative AI and the like and the AI governance which is going on, the topic of governance and being able to find solutions to help people get closer to a place of compliance or a lowered risk is very much involved. And that is why we have Daniel here today to talk about this topic. 

So, we’re loosely going to go through roughly what’s artificial intelligence, then we’ll look at some regulations, then at a standard called CRISP-DM. I’ll flesh that out more when we get there. Then we will look at some specific applications that FCD, Financial Crime Dynamics has, to help us understand specifically how the technologies can be applied in specific verticals. So, without further ado, Daniel, welcome. Thank you so much. It’s good to talk to you again. Could we just start by having you explain your role, your background, and then we’ll get to the meat of it. 

Daniel Turner: Thank you for having me on the podcast, Paul. My background is quite diverse. I originally started in the life sciences in terms of pharmacology then moved on to neuroscience. So, a couple of years ago, I was looking at the organized little molecular signals in the brain. Then for the last three years, I moved on to financial services within Fin Crime Dynamics and I look at financial networks and capturing the criminals that like to ride those. At the end of the day, it is data. Data is always our manifest. 

My background is primarily in data science and also product. And both cases of the life sciences and financial services are quite close to my heart. We suffer the same problems within these industries. We want to provide value for the patient or for the end customer for them to have a smooth experience, for them to have a better quality of life. However, to achieve that, we need a lot of data to make the right decisions for our models. But as you know, in both these industries, data is very difficult to get, very difficult to use and also restrictive. So, what I wanted to do with Fin Crime Dynamics is change that and create a smooth experience to get better models out, to get a better quality of life for the end user. 

Paul Starrett: Yes, well stated and thank you for your introduction. You sound like a very smart person and I know you would be that. 

Daniel Turner: No, I hope I am. The years have treated me well.

Paul Starrett: Well, I think you’ve really done a great job with what you’ve done with FinCrime Dynamics, but that’s another podcast, I suppose. But I think really the overarching question is machine learning, and I’m going to state the obvious here, is machine learning either uses abnormalities as a means of determining what should be looked at as a potential issue, which can include fraud, they can include threat detection, or it’s using historical data -- this learned from called supervised learning, and it uses that learning to then try and figure out how it might be able to predict the future or to classify something as A, B, or C. And as you alluded to, the overall arching issue has to do with availability of data. 

First, it’s in very different formats, it’s in different locations, that may bring with it data protection rules where certain jurisdictions may not allow you to remove it at all. Or it may contain private data, and the private data may be a part of what makes machine learning smart. And if you have to remove the private data because the law requires it, you then lose some accuracy or some of the performance of the model. So, does that sound right, Daniel? I want to make sure that I’m kind of putting that the right way. Do you have anything to add to that? 

Daniel Turner: I like it, I like it, Paul. I’m going to have to add one little bit. So, the way I always like to describe machine learning, so focusing on machine learning for now, we can look at artificial intelligence after. If you take the example unsupervised and supervised, what you’re trying to find is signals. So, for the machine to read different signals, what we do is we take two different features. 

So, for example, it could be the time that something has happened, the balance of an account, the type of transaction it is, we can combine them to create better signals to represent what we want to see. And these signals we train the machine on in supervised learning are called labels. The more labels we have, the better the signal. The more we make our features robust, the more the machine can learn the stronger the signal. And that’s all it comes down to. We’re playing around with creating better signals for the machine to learn from. And then when it has to make a decision, it’s going to, I know this signal, that’s my signal. I’m going to mobilize on it. 

Paul Starrett: Got it. And I think a good way maybe to segue into the topic is that I happen to know from personal experience. Here at PrivacyLabs, we had a project we were working on and we actually built a tool for this. But suffice it to say, there’s a Kaggle competition that occurred some years ago, where they used a 285,000 row spreadsheet with something like 25 columns, which are labels effectively. And they were labeled as fraud and non-fraud. And so the issue was that of those 285,000 rows, only 500 were fraudulent. And so the issue becomes in that case, what’s called class imbalance. There’s so, so few fraud examples and so many vulnerable legitimate transactions. It’s very difficult to tell them apart. They get conflated. It’s very difficult to classify. 

So, there are some approaches in synthetic data. One’s called ADASYN, I think, and the other one’s called SMOTE. I don’t remember what that means, but those are synthetic data in theory, so that you can accommodate for that class imbalance. 

Paul Starrett: So, Daniel, what else are you seeing as issues that need to be solved? And we’ll get into regulations a little bit here, but I just wanted before we get to that, maybe give you a chance to flesh that a little bit more. 

Daniel Turner: So, probably have to gather this hot topic. Like everyone’s heard it, everyone’s seen it. It’s a fun ChatGPT, plus many other of its friends, it’s the foundation models, the large language models. I think there’s definitely right now in the state of play at the front of everyone’s mind, but I think it’d be very interesting if we cover a bit of that.

Paul Starrett: Yes. And I think if we then, let’s say, move to the regulations, I think it can be said, I think many people have understood because ChatGPT has put AI so much in the forefront of everyone’s thinking that there are some issues that we can glean as a general matter for artificial intelligence. Because ChatGPT is based on essentially natural language processing, but there’s really three buckets. There’s fraud, which is tabular, columns and rows, is typical relational database. There’s image processing for computer vision and things like that. But then we’re not really discussing that specifically, the image processing, but the LLMs. We’ve heard about how they hallucinate. 

There were some attorneys who asked ChatGPT to create an answer to a court proceeding or into a court proceeding, and it was wholly fake. The ChatGPT made everything up. The court was real, the cites were real, but everything was made up and they got in very big trouble. So, these models, they hallucinate, they might leak private data, they might give you incorrect data, they might defame somebody. If you ask it to create something, a way of committing a crime, it might generate that. So, there are guardrails that have to be put into these things. And this really becomes the hallmark of the regulations, is to make sure that we don’t step on other people’s rights and that we aren’t providing something to the public that’s going to cause something to go south on them. Dan, I just want to sort of get your thoughts on that.

Daniel Turner: Definitely agree on that one, Paul. I think it’s interesting in retrospect to look at the timeline. LLMs foundation models, they were always around, they were always being developed, but it wasn’t until much of it got released to the general public with the boom of ChatGPT that it lit a fire under the regulators because people were able to use it in their day-to-day life and [inaudible]. And as you’ve highlighted, it’s quite dangerous because it can hallucinate, it can provide you the wrong information. And if people cannot apply critical thinking to something like the internet sometimes, then creating a model which is based on a natural language where a human might infer it as more than just, a probabilistic way of putting the next words out is quite dangerous if there’s no regulations there. 

I think the other interesting aspect is it is a very generalized model and people are going to use it for specific applications. So, for example, what is stopping me from asking it medical advice? Sure, it’s probably going to give a disclaimer saying that, oh, you should consult a real doctor. But how many people that we know seek medical advice on forums or different websites and don’t read the fine print of, go see your doctor. Your doctor is best suited to do that. 

Paul Starrett: Yes, yes. I’m going to move this to what I think is the next phase of the way we’re looking at this. And I think all the regulations, the ones that I’ve seen, by the way, I have a law firm, starrettlaw.com, a little plug there, where we specialize in AI governance. But I think what I have seen is that explainability is really at the forefront, understanding how the model works. Because at that point, you can tell yourself, what is it doing right and wrong? Where could it go off the rails and what risks could it produce? 

The problem though, is that the more explainable you make it, it has to be more simple. And sometimes things like deep learning and neural networks and things like that, the explainability suffers significantly. But the model performance is higher. So, you have this kind of an inverse relationship between accuracy and performance of the model and how explainable it is and how compliant, if you will, it is. 

And so I think that when it comes to technologies that Financial Crime Dynamics uses, is being able to understand the model as much as possible and then being able to understand how it’s being used in the real world, allows you to generate data that can give you the best possible data that you can get. And thereby giving you more control over how the data can be used to do what you want to do in the real world, but also protect yourself. 

Daniel Turner: Exactly, Paul. I think what would be really interesting for us is to take a step back and have a look at some of the potential regulations that have been coming out around these. Because these regulations were building even before it became [inaudible]. I think the regulations are just getting a bit tighter, getting a bit more robust. And the big player I’d probably like to discuss is the European Regulation Act. That one is definitely at the front and center. 

It’s also interesting because America right now is having a slightly different approach where they are applying current regulations to the use of AI. Whereas Europe is trying to draft its own thing. And the UK is following quite similar to Europe, but they’re applying it in a slightly different way, doing more sector specific. But I think it’s always good to look at the big dog. Everyone’s going to look at what they’re doing. They’re going to look at their reactions and they’ll probably inform a lot of regulations outside of that. 

So, if we break it down into different sections, I think one of the best breakdowns I’ve seen of some of the requirements from the European Act were on a group that designed a sort of benchmark called the Holistic Evaluation of Languaging. This was by Stanford. Essentially what this helm does, as you say, is it tries to benchmark different language models against different sorts of aspects, and then going into stuff like computer vision, etc., and other more complex foundation models. 

Now, what they’ve done is they’ve collected a couple of requirements from the Upcoming European Regulations Act. And I think going through each of these categories and keywords that I use would be very compelling for us. 

Paul Starrett: Interesting. You said keywords. 

Daniel Turner: Yes. They break it down by category and keyword. So, for example, the four main categories they’ve highlighted out of the requirements is data, compute, model, and deployment. 

Daniel Turner: And I think it would be great if we deep dive into a couple of these. Keep it very high level, but it’s going to come back to what we want to discuss later quite nicely. 

Paul Starrett: Got it. I did want to sneak in, though, the CRISP-DM. It’s the cross… I should have this down. But you know what? Why don’t we stay away from CRISP-DM? Is that okay? 

Daniel Turner: Yes. I think with CRISP-DM we can cover that later, because it’s going to go really nicely into these regulations. 

Paul Starrett: Okay. Got it. 

Daniel Turner: So, I think just looking at what we have in terms of the core requirements and breaking them down. So, on the data number one, you have the data sources. So, pretty simple. Describe the data sources to train the model. Well, simple to put it in words, but when you actually have to go back and understand the complexity and the amount of data that’s flowing through these, documenting that alone is going to be very difficult, especially if it involves any data mining, data scraping of live scale, which foundation models and these LLMs tend to use. And what are your thoughts on this, Paul? Like in terms of data sources, where would you say these certain terms are your experience? 

Paul Starrett: Well, again, it does depend on the vertical or the domain like health care, financial services if I take that way of breaking it down. But I would say that it’s a matter of getting permission from those who hold the data. That’s the first problem. That’s not always easy. There are people who are afraid of what might be in the data, personal data, and so forth. And then I think laws that affect the ability to use the data as I’ve alluded to or to move it. And ultimately, as you take the data in, you have to decide whether that data is robust enough for you to build a predictive model or other type of machine learning model that’s useful. Because you go in with some purpose to say, well, I want it to do this. I want it to detect fraud or I want it to help me decide where I should point a customer to make a purchase or what’s the weather doing, what’s going to happen tomorrow? Is it going to be hot or rainy or what have you. 

So, I think that’s really this sort of process of understanding what data you need to make a model that is going to be useful and then getting that from its sources. And of course, you come into things like bad data. There’s missing values. There are outliers that you may or may not have to remove. Doing some data clustering to learn about the data. So, I think all of those things sort of come to a head around how data becomes a challenge for this process.

Daniel Turner: Definitely. And it’s one of the most important areas of any operation to understand what data do you actually need because that’s where a lot of the risk comes from, especially from the perspective of GDPR and many other data governance laws. Why do you need this data? Is this necessary for good model performance to achieve what you want to achieve? It’s also interesting to factor in, okay, well, this data, am I introducing any bias? Is it suitable for my use case? Have I applied the appropriate mitigation controls to make sure the data is safe? 

Paul Starrett: Yes. I’m sorry, continue. 

Daniel Turner: No, sorry. There’s just so much within that particular area that you can carve out and very much de-risk yourself at the onset. However, some of it does have to flow downstream to the pipeline so you understand the decisions that are made. 

Paul Starrett: Yes. And I think one thing that I have seen with some other areas is that you can start developing the model before you get all the data. You can wait for data to come in. And as it comes in, you can then start to build and make decisions. So, you don’t necessarily have to be in a place where it’s a boil the ocean or do everything at once. And I think also there’s synthetic data approaches to help with this too to speed up that process to get the data in sooner. But yes, I think really to round this out and the gist is that I think there’s an adage that 80% of machine learning is data collection and preparation and understanding. The model training itself is 20%. The rest of it is 20%. But this is really where the expense and the tedium comes in. And I think this is where one of the biggest challenges is, not only from a model performance standpoint, but also from a regulatory standpoint. 

Daniel Turner: Definitely agree there, Paul. The amount of time I spend on data cleaning and just feature engineering [inaudible] mental because that’s where most of the impact tends to happen. 

Paul Starrett: Yes, the next one was compute, right? Yeah. Okay, great. Thank you. I think that sort of covers the data piece. The next one I think you had on your list before was compute. 

Daniel Turner: Compute is a very interesting one, especially from a European perspective. It can cover a couple of different elements. So, energy consumption. The Europeans are very, very for environmental regulation and this would definitely cover it, especially with the large compute necessary to build a lot of these models. And number two, for me, this one is definitely going to be more about the competitor analysis and a bit on the fairness side. Because there’s only so much you can do with the limited compute if you’re a smaller company versus a bigger company. So, my assumption here is they want you to disclose this just so they understand more about the state of play. Because the amount of compute you throw at a model can infer a lot about it as well when we’re talking about these big scale foundational models. 

Paul Starrett: They’re very expensive. 

Daniel Turner: Exactly. But the good thing is that as long as the core infrastructure is built, a lot of the open source community can take this and build horizontal models, which are more specialized for their use case. Whereas the core architecture and the core learning still stays within the original model that was built. 

Paul Starrett: Got it. There’s an interesting issue that I’ll raise later, because I want to get through our four pieces here. Did you want to move then on to the third of the four? 

Daniel Turner: Definitely. So, the next one is model. The points they cover here is capabilities limitations. So, what is the capability of this model? What is the limitation? A lot of this can be within the decision making process itself. Is it capable to do the task it has? And most importantly, limitation. What can it not do? Where does it fall back? And then you go into the next one, which is risks and mitigations where you have to understand, okay, what are the foreseeable risks of this model? How can it actually impact it before you deploy it in a production environment? And this is really important, because once it’s there in a production environment, it’s already affecting the outside world. It’s already making decisions that can influence people’s lives. So, this one is really important in terms of what you can do here. 

Paul Starrett: Yes. I just want to sneak out that for those who don’t know, production environment, sometimes referred to as deployed or deployment is when you take your model from your sandbox, as it were, where you were testing it and doing research and development, and you put it out into the actual real world, into the enterprise infrastructure where it’s doing its job. It’s actually in the real world doing what it’s supposed to. I’m sorry, I just wanted to leave that. Please continue. 

Daniel Turner: Exactly. And that’s always really nice -- the next two key words, which are evaluations and testing. So, as it goes, evaluations to benchmark it against something else. The tricky part here is it’s really difficult to properly benchmark models of this caliber. HELM is actually quite nice in terms of what they’ve managed to achieve with the benchmarks they are creating under a lot of different verticals, what is particularly in the LLM space. But in terms of testing, if it’s like a very domain specific model, this is something you [inaudible] will have to put a lot of controls in your place to understand what it does affect, what does it not affect; what is its core value at the end? 

Paul Starrett: Got it. Could you expand on HELM, what that is? 

Daniel Turner: No problem. So, as mentioned earlier, it’s essentially holistic evaluation of language models. So, it’s a, as I described, living benchmark created by Stanford. And it assesses all these different LLMs that are released if they are publicly, if they are not on that against a lot of different criteria. Now, it’ll take us a long time to go through all the criteria, but one of the ones I found quite interesting is the prompt criteria. So, it checks, it does a bit of a gap analysis in terms of how a model can respond to different prompts. And there’s another one which looks at toxicity. So, essentially prompts which are negative prompts, which may have a negative influence. But as the word goes, it’s a living benchmark because a model that is so complex as it evolves, as people interact with it, we’re going to find new areas to actually improve on. And I’m really excited to see where this goes as we start growing it out. 

Paul Starrett: Yes. And I think one thing I’d like to just add in there is that just with LLMs in particular, this can certainly be ported over to the other areas of tabular data and computer vision. But the commercial LLMs, large language models, do not give you access to the underlying documents or data that they were used to train on. And so it makes explainability more tough, more difficult, right? So, if all you’re being given is the model features and weights and you want a prediction, you can’t do that. 

If you’re building an LLM from scratch yourself, not using ChatGPT or Llama or what have you, you have access to those and you can have their computer book, libraries that you can use in Python and so forth to tell you, okay, I sent in a prompt. That is, I submitted a question or request to a model, LLM, and it gave me back this answer. What you can do when you own that whole process is you can determine which documents it used and how it uses those documents to make the decision, which makes it very explainable. So, I think there’s an issue for me with the explainability of these commercially available LLMs. 

And so to your point, where I was going with that is then it makes it that much more difficult to kind of reverse engineer, if you will, in a legal way, I’m just saying, to the extent you understand the model, how’s it making this decision? Just for a quick example, if I go to ChatGPT, using their API and I send in a question, it will generate a response. I can click regenerate, everything is the same, it will give you a completely different answer. I do it again, give me another answer. So, getting back from the same model, same question, three different responses. So, how is it coming up with that? So, I see your point, the permutations, the decision process becomes almost incalculable. 

Daniel Turner: Exactly. I did an interesting experiment once in terms of trying to see how quickly I can expedite certain research using ChatGPT, for example. And I have a good understanding of the particular area I was looking at and I kept asking it questions about things that may or may not be possible. And then I realized it started creating experiments which aren’t feasible, which don’t make sense to me as if they were real, but it communicated it in such a way where it’s as if they actually did happen, similar to the legal argument here. 

And this is where it’s quite dangerous because if you don’t have knowledge to question it, you may consider this the correct answer. It’s like I mentioned earlier, with using the internet, using the internet, you have access to so much content, so many pages, but it’s your critical thinking that actually infers what you’re going to get out of it. And sometimes you have to follow a lot of different breadcrumbs to find that solution you want.

Paul Starrett: I like that analogy. I would note that a lot of these commercial LLMs are based on what’s on the internet. And if someone told me that my data source is the internet, there’s a lot of junk out there. 

Daniel Turner: Exactly. 

Paul Starrett: And the other thing is that these things, these pieces move. Whenever I do put in something into ChatGPT now it says this data is only accurate up to 2021. I know if it is exactly that, but it was trained up until 2021, until they have a chance to update the model to more recent data. So, there’s yet one more dimension that makes this really like a beehive, a bee. It’s untenable almost. 

Daniel Turner: Sometimes I consider it a bit of a hornet’s nest even. If we [inaudible] political about it, everyone has some type of political alignment. True neutrality is very difficult, if not impossible to achieve. And whoever is setting the stage for this model and what it’s trained on, they can say, use this information and not that information; will influence its output. Hence why having a single commercial LLM to roll them on is very dangerous. 

Paul Starrett: Yes. So, I think what we’ve done here is we kind of, in a reflexive way, brought up issues that we’ve seen. But sort of bringing this back into the regulatory analysis, I think we’re really starting to see just by anecdote and by our own thought processes and what we’ve seen, where the issues really lie and how the regulations come back in. So, from there, where would you think would be that the next place in our regulatory exploration here? 

Daniel Turner: So, what we’ve actually covered is really good to deal with the last category and that’s deployment. So, the three key words they use there, machine generated content, member states and downstream documentation. So, the machine generated content is essentially a disclaimer telling their user that this is generated by AI, take this over a bag of salt, I’d say, not just a grain of salt. And the next one is member states, which is telling where this model will be in the Europe market. It’s very like European specific, but essentially there’s a lot of laws within Europe, within its own member states that the model might need to adhere to, and you might need to have certain criteria or check marks; as we saw recently with it being banned in Italy for a small while. 

And the last one is downstream documentation. This one’s really important and flags a bit of the issue we discussed earlier, that there’s not enough explainability and transparency to it. Because what this rule essentially says, and what is a requirement is you need to provide sufficient technical compliance or downstream compliance with the European AI Act. So, anyone who’s going to use your architecture downstream of its original use case will need documentation to be able to mobilize on that. 

Paul Starrett: Got it. That’s well said, actually. I think that covers a lot in a few sentences or paragraphs. 

Daniel Turner: I mean, what we’re seeing here is a really interesting regulation. It’s still in the drafting stages, but it’s taking shape. It’s taking shape. Well, there is a very concerning item here. And that’s the state of play of many different countries and regulations. Because if we talk about these large commercial bodies or even the open source community, you are looking to develop a product, a product that needs to have a function. And if you have different compliance directives in each separate country, nation, etc., that’s going to be a nightmare to actually operationalize. 

Paul Starrett: Yes, it is. I can say no more because it almost seems to me like the regulatory world could just significantly stifle progress and development, and it becomes a public policy question then. How much privacy leakage are you willing to live with, for example? And then you have to also ask the question of the specific entity that’s creating the model. Not only does it have to worry about where that model may wind up in deployment, meaning out in the real world, but also they have their own risk appetite, their own set of reasons for using machine learning, the risks that they might take for use of that. So, that all really makes it very kaleidoscope, if you will, of issues. 

But I wouldn’t say, though, that that is a barrier or even something that’s not necessarily fairly straightforward. I think in most cases, you have to decide if machine learning is going to be the answer for you. Because I did have one client at one time, I won’t say who, but the cost to be compliant was so much that they would have had to shut down business. So, they decided not to use machine learning. And so you get to those threshold questions. It’s just getting to the point where you feel it is a net gain to proceed. 

Daniel Turner: Well said, well said. And I have to say as well that you don’t need machine learning for everything. You don’t need artificial intelligence for everything. It has to be like a honed knife for its use case, because applying it [inaudible] is just going to lose you a lot of money. 

Paul Starrett: Yes, yes. And I think that one of the admonishments I have in my little when I’m on the -- when I’m pontificating is don’t use machine learning unless you have a reason to. And it’s really going to be based on your data. There’s compute and other issues too, but ultimately it’s going to… Data science, machine learning, it’s math, right? It’s like a calculator, it’s not wrong. The data is wrong. So, it’s really the data it really does come back. That’s a big piece of this. I think that can’t be understated. 

Daniel Turner: Indeed, you’d be surprised how many things can just be solved with an application of linear regression.

Paul Starrett: Yes, yes. There are various different things for explainability. There’s a whole rabbit hole there, a whole area of study. We actually have some training videos on our website at privacylabs.ai about explainability to just discuss this. But anyway, so what would be the next step then? What would you think is the -- as far as continue our regulatory path here? 

Daniel Turner: I think there’s one topic I wanted to cover and that’s essentially on the operationalization of a lot of this, because what you’re going to see is the need for many experts to come in outside of just your typical data scientists who are running these models. You’re going to need a lot of human in the loop, as I like to say, people that can add the necessary compliance to something that is quite a complex topic for most. So, I think in the future, depending on how these regulations go, you might see a slightly different organizational structure for your typical data science team. 

But this is mostly for these really, really large models. I think the more applied machine learning models are still going to be fairly safe in terms of their deployment and the general goings on. But definitely when the complexity of the problem rises with what you need to create as a model, you’re going to need a more robust series of experts to come in and make sure that this is all pretty crystal clear. 

Paul Starrett: Yes. Yes. And I think you’re really harkening back to what’s going on. I don’t know if this may be a good time to take the CRISP-DM, which is Cross Industry Standard Process for Data Mining, in case anyone doesn’t know. That is when I was in my master’s program at Northwestern in Data Science, every assignment had to be completed by going through this six stage process that they have. The reason I’m taking that out, what might be considered to be out of turn here, is because the first part of the CRISP-DM process is to have a business understanding first. 

And the idea is who needs to be a part of the decision process for what the model is going to be doing. And if you have someone from, let’s say, data science and software engineering, maybe somebody from your legal team and someone else from the domain itself, maybe some sort of analyst or subject matter expert, they all have to understand what’s going on. So, keeping it simple is key, starting there, which is to say it’s explainable. It’s something that each person understands in their own language, such that they can evenly and adequately provide an informed contribution into the decisions that have been made jointly. And so I hope I’m not taking that out of turn. 

Just for our audience, this conversation wasn’t meant to be a very highly ordered discussion. It’s sort of meant to be a bit reflexive and free form. So, I just felt that was a good time to do that. The CRISP-DM does go on to discuss much of what we’ve already talked about with understanding your data and learning about it, training the model and then monitoring it in deployment and so forth. That standard, which you have to follow. It’s not being pedantic. If something goes wrong upstream, if the business decision process isn’t good, everything that follows in the sequential process of the model development and deployment suffers. And in the second phase, if there’s something wrong there, everything downstream suffers. And I think that given that, many of these regulations follow that very process because they have to. 

Daniel Turner: 100% agree on that one, Paul. I mean, CRISP-DM is a pretty classic approach to data science. Then this original purpose was to save cost in terms of before you put your model into deployment, you’re exactly sure that it’s going to be worthwhile instead of going through the whole process and then putting it into production, which is costly, and you have to recalibrate, etc. and all that type of stuff. Whereas this focuses on the business understanding, the data understanding and loops it through the process into the evaluation. So, before you go to the deployment, you can double check, am I sure that this is the right model for the job? 

Of course, sometimes you will have to deploy that model. That’s the only way to understand new events and how your model controls the shadow model, for example, which is actually a model that runs in the background and just collects information on its decision making process. But this speeds all of this up before you need to get this model out. And I think in terms of what we’ve discussed before, this is a really good way to also put this more auditable schematic into it. Because within your business understanding, data understanding and your evaluation, you can put some extra layers of compliance in there to make sure that your model is doing what you want it to do. And you can describe exactly why this model is trustworthy and robust. 

Paul Starrett: Yes, yes. And that is kind of touching on a few other topics. I don’t want to digress too much. Trustworthy, is it in the public policy area, is it biased against a certain group of people, is it fair? Does it treat people equally? And then robustness, if there are actually tools, this area called adversarial robustness training, where you can test your model for how robust it is to attack by hackers, hackers who use machine learning to defeat machine learning. So, all of those things come in to become a part of accommodating risk and so forth. And I think… 

Daniel Turner: A question for you here. When are we going to see machines fight machines again? 

Paul Starrett: I don’t know. That’s a good one. I know that we’ve seen that thing on TV where you have these little robots that once they hammer and the other is screw or something and they try to destroy the other. But… 

Daniel Turner: Would it be a battle of compute or would we see a David and Goliath situation? 

Paul Starrett: I don’t know if I want to know. But yeah. Did you want to move into another area of regulation? 

Daniel Turner: I think we’ve covered the dryness of regulation extensively enough for our dear listeners. We can go on something more interesting. 

Paul Starrett: Yes, yes. I might like to just round out that a lot of this is common sense, a lot of this is if you think of the CRISP-DM, you can Google it, CRISP-DM and it’s an easy read. And I would tell you that a vast majority of these regulations will follow that whole process on some level. 

Paul Starrett: Okay. So, Daniel, we’ve gone through the regulatory process roughly. And now let’s get into some fun things to see this applied in the real world. I know Financial Crime Dynamics has a very valuable tool for assisting with everything we’ve mentioned. So, take it away. 

Daniel Turner: Thank you, Paul. We covered a really good state of play there in terms of what is AI regulation, what is AI trustworthiness and where we need to take it. I want to cover a bit more down the machine learning angle and essentially pose the question of if you haven’t seen it before, if you haven’t seen something, how do you know it’s going to be trustworthy? How do you know how your model is going to react? And that’s where FinCrime Dynamics comes in, because what we are is an intelligent sharing platform that tries to simulate what we’ve seen in other areas that maybe you have not encountered before. 

A good example of this is financial crime, essentially the industry I sit in. You have a lot of people doing transactions, etc. within the financial system, criminals come along and they cook up different schemes. Some of them can be very simple, some of them can be very complex. Now, the question is, what if a criminal creates a scheme that you’ve never seen before but another institution has? You wouldn’t have the labels for that, because it takes a very long time sometimes to identify a new financial crime behavior, then create a label that you can train your model on so it can then detect it in the future. And this is where the robustness is quite difficult for machine learning, because there’s a lot of events that you haven’t encountered and your model may not be able to detect or you may not be able to understand how your model can react to it. 

And what we want to do is to try to simulate these events and understand how the model would perform before the event actually happens. But for an event to happen, we need to understand that it exists. Hence the intelligent sharing approach. We look at what other institutions are suffering from in terms of the financial crime. We look at what potential regulations may change, how the economy might change. And we try to make some educated forecasting, as I call it, in terms of how the criminals will evolve and that way you’re able to test a lot of complex behaviors on your models that you might have never seen or by the time your model sees it, you have something totally different in your production environment that is making those decisions. And then it’s going to be a very long process to re-label and re-evaluate a new model that you want to understand how it works. We just want to expedite that process. 

Paul Starrett: That’s very interesting. I do want to maybe just touch on the technology and I don’t know if we’re getting too deep here, and I don’t know if… I’m going to use this term loosely, but a topic called agent based modeling where essentially you create these agents and then you release them out into an environment, if you will. This is all done with programming and simulation where it goes off and it behaves. And by bouncing into things and trying different permutations, you find new data, new threats, new ways in which the system will actually occur. Rather than having it happen in the real world the simulation simulates it and gives you some advanced insight or knowledge or foresight into what’s going on. 

Very briefly, that’s how they do some of these spread of disease where, say for COVID, they would say, well, here’s a human. This one goes to work at this time and they go to the grocery store and so forth. And they cut these people loose in a phony society and they let them bounce around and they’re just little agents. They’re just little computer programs. But they learn, well, this person went to the store and they bumped into this number of people. Well, when they did, those people went off and they bumped into other people at work or at home and they can then figure out how fast the virus might spread or other issues. So, by using the simulation, you could then met out, if you will, or extrapolate out other behavior. And I know that I’m touching on this fairly coarsely grained level, but is that sort of maybe a good way to start with approaching this from a more layperson standpoint? 

Daniel Turner: Yeah, definitely. Think of an agent as an account, a customer. What do you do with your account? You send money, you receive money, you have different networks of people you interact with, and that’s what we’re trying to simulate. And then we just drill into how the money goes in, how the money goes out and who the money gets sent to or the money that flows in. And with that, you can model a lot of complex behaviors, because if we talk about money laundering, that network is going to explode. The money going in and out will be quite anomalous. It might be quicker, it might be very calculated. Accounts that in the past may have not done anything start to activate, start to change their behaviors. So, a lot of really fun ways to experiment with this and create behaviors which are quite sophisticated. 

Paul Starrett: Got it. And I found this area fascinating. And again, I don’t know if I stated this, but I was an advisor to Financial Crime Dynamics in 2021, 22, I think. But because of my background, it was fascinating and it was wonderful to learn from you, Daniel and from Edgar, about how this works. 

Paul Starrett: Great. Okay. So, I think, Dan, you’re going to segue now into one of our last topics here, which has to do with synthetic data. Can you expand on that sort of in a more detailed way? 

Daniel Turner: No problem whatsoever. Also, as we covered before, what we do is simulations. So, we’ve talked about the agents, how the agents like to run around and send money. Well, how is that compiled? What is the output of that? Well, the output is synthetic data. So, it’s a row by row segment of transactions by all these different accounts. And where we use this is to test different machine learning models. So, this is very much supervised classification where we create a data set and we put some financial crime into it. If you look at it row by row, there’s going to be accounts which are performing criminal transactions. Then we test this against the model to understand can this model identify these transactions? Is it able to pick out the signals? 

Now, what we can do as well, which is super fascinating to me, is we can use the same stuff to train the model so the model can actually learn from what we are simulating. And then when this event appears, it can actually detect it. And that’s where the real intelligence sharing comes up, because we can take a behavior we’ve seen in a different institution, simulate it, allow a model to be trained on this and act as a proactive defense. And as you did mention, that very nice analogy of viruses, I want to use something similar here. 

So, think of financial crime as a virus, viruses evolve [inaudible]. We try to treat viruses with vaccines. Vaccines are our essential proactive defense. The problem is we have to keep updating the vaccines because viruses mutate and evolve just like the criminals. What our simulations do is they basically democratize intelligence sharing and [inaudible] a vaccine to keep it as updated as possible for everyone and keep every institution healthy, creating this nice, proactive defense and stopping the spread of these criminals/viruses. 

Paul Starrett: Yeah, Daniel, I did want to point out when you said that you simulate, you take the data that you see from the simulation and then your machine learning model can train on that. Just to make that a little bit more clear, when we think about training data, it’s the data that it learns from from the real world and that your machine learning model then gets tuned and refined on that data. What you’re doing is you’re adding new data to that data that is synthetic, right? And now it’s training on a larger, broader, more robust set of data that the machine learning model learns from and therefore benefits from.

Daniel Turner: Exactly.

Paul Starrett: Got it. Yes, see, this stuff isn’t really that hard to understand. I mean, I’m living proof that anyone can understand this stuff, I think. Okay. Did you feel you’ve given that enough justice as far as the application and what our purpose was in covering the topic? 

Daniel Turner: Yeah, definitely. We share intelligence and stop viruses from spreading. 

Paul Starrett: Yes. Okay. Well, I think we’re running into an hour here, which is on the upper end. But I still think we haven’t done this quite enough justice. Certainly very good treatment here, that’s for sure. But we can go on all day. But what I would like to do then… What I ask all of my podcast visitors, which we thank you, by the way, is there anything that we have not discussed that you think our audience should know about? Anything? 

Daniel Turner: So, I guess a bit of a company plug here. We are currently tying up a grant of trustworthy AI and building a consortium that is all about intelligence sharing to stop financial crime. And what I described before is the methodology we want to use to create our sort of defenses. And the consortium is going to be a fundamental part of that. So, stay tuned on updates about the consortium, how it goes and all the trials and tribulations that comes with running something of that magnitude.

Paul Starrett:So Daniel, thank you again. That was a pleasure to have you on this.On this podcast. Where can people get in touch with you if If they need to, they can find out more about the consortium.

Daniel Turner: So, I'll provide you a couple of links you can attach to the podcast and if people wanna reach out to me, feel free to add me on LinkedIn. Okay. I'm always happy to the podcast and if people wanna reach out to me, feel free to add me on LinkedIn.I'm always happy to discuss all these different from AI items, be it from the financial service perspective or from a life science perspective. Oh, for me it's gonna be a stimulating discussion regardless.

Paul Starrett: Okay, great. So what we'll do is, we are, there's a transcript here. I will put at the base or at the base of the cons, transcript, some of the links you, you provided and some other ways of reaching you if people need to. 

Daniel Turner: Thank you for having me, Paul. 

To contact Daniel for questions, links to discussed efforts or other information, visit: 

FCD’s website at: https://fincrimedynamics.com

Daniel’s Linkedin profile at https://www.linkedin.com/in/dtsfcd/