IBS Intelligence Global FinTech Interviews
Go one-on-one with the innovators, disruptors, leaders, and decision-makers driving change in FinTech and financial services. IBS Intelligence delivers exclusive global interviews that uncover strategies, challenges, and the ideas powering the next wave of financial technology.
IBS Intelligence Global FinTech Interviews
EP1028: AI Testing Boosts Trust in BFSI
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
This interview explores how AI-powered testing frameworks are revolutionising the financial services sector by enhancing regulatory compliance and data sovereignty. It highlights the shift from traditional manual processes to agentic automation, which allows for the creation of testing scripts before software development is even completed. The text emphasises that cybersecurity and functional testing are now merging, requiring organisations to use artificial intelligence to defend against increasingly sophisticated digital threats. Furthermore, the role of Quality Assurance professionals is evolving from routine execution to strategic orchestration and validation, necessitating deeper domain expertise. Ultimately, the source advocates for a human-in-the-loop approach to ensure the accuracy and ethical deployment of these emerging autonomous technologies.
Imagine it's uh two o'clock in the morning on a Sunday.
SPEAKER_00Okay, setting the scene.
SPEAKER_02Yeah, and a major bank's automated system just approves a fifty million dollar commercial loan transfer.
SPEAKER_01Oh wow.
SPEAKER_02Nobody is awake to double check the paperwork, right? Uh-huh. The system did exactly what it was supposed to do, except for this one like hidden detail.
SPEAKER_00Let me guess, something changed under the hood.
SPEAKER_02Exactly. Over the past three months, the artificial intelligence model running the bank's risk assessment quietly drifted.
SPEAKER_00Ah, the dreaded model drift.
SPEAKER_02Right. It learned from a recent batch of uh slightly anomalous market data, subtly rewrote its own internal logic, and suddenly a high-risk financial profile looked perfectly safe.
SPEAKER_00Which is terrifying.
SPEAKER_02Millions of dollars just gone. Right. Simply because the machine, you know, changed its mind.
SPEAKER_00Aaron Powell And in the banking, financial services, and insurance sector, like the BFSI space, that is the absolute nightmare scenario. For sure. A crashed social media app is an inconvenience, right? But a hallucination in an algorithmic trade or um an automated insurance payout, that's an economic disaster.
SPEAKER_02Aaron Powell Yeah, the margin for error isn't just small.
SPEAKER_00It is effectively zero.
SPEAKER_02Which is exactly why we're looking at a fascinating interview today that tackles this exact problem head on.
SPEAKER_00We really need to get into this one.
SPEAKER_02Welcome to today's deep dive. We are exploring how artificial intelligence is being forced to rewrite the rules of its own quality assurance.
SPEAKER_00And we have some incredible source material for this.
SPEAKER_02We do. We are diving into an article from the IBS FinTech Journal. It features Madhav Bhadra, the founder and CEO of AQM Technologies.
SPEAKER_01The piece is titled AI Testing Boosts Trust in BFSI.
SPEAKER_02Right. Yes. And I have to say, the visual they paired with the article sets the tone brilliantly.
SPEAKER_01I notice that too.
SPEAKER_02It shows this robotic handshake, but it's layered over a really dense glowing background of candlestick charts, banking icons, and you know, data analytics.
SPEAKER_00It perfectly illustrates the core tension here. Yeah. It's this forced, slightly uncomfortable marriage between the totally unpredictable, generative frontier of machine intelligence and an industry that is built entirely on strict, heavily regulated trust.
SPEAKER_02Aaron Powell Okay, let's unpack this. Because before we can even start talking about AI making workflows faster, or like writing code, we first have to establish why trust is the ultimate hurdle.
SPEAKER_00Aaron Powell It's not just a corporate buzzword here.
SPEAKER_02Aaron Ross Powell No, it is a structural mechanical requirement.
SPEAKER_00Aaron Powell Exactly. And Badra provides a very rigid framework for how financial organizations must view AI confidence.
SPEAKER_02Aaron Powell You can't just install a large language model and hope for the best.
SPEAKER_00No, you absolutely cannot just install an LLM, cross your fingers, and hope it naturally follows federal law. Aaron Powell Right.
SPEAKER_02That's a fast track to getting shut down.
SPEAKER_00Aaron Powell So he breaks the deployment of AI down into three specific angles that just have to be secured.
SPEAKER_02Aaron Powell Okay, what's angle one?
SPEAKER_00The first angle is the foundational reliability of the models themselves.
SPEAKER_02Aaron Powell So the actual brain of the AI.
SPEAKER_00Aaron Ross Powell Exactly. This deals directly with accuracy and that concept you mentioned earlier, model drift.
SPEAKER_02Aaron Powell Let's hover on drift for a second, because I think um people assume an AI is like a traditional software program.
SPEAKER_00Aaron Powell A lot of people make that mistake.
SPEAKER_02Aaron Powell Right. Like if you write a calculator app, two plus two always equals four forever. Why doesn't an AI just stay locked in place once you deploy it?
SPEAKER_00Aaron Powell Because traditional code is static, but generative models are probabilistic and they're adaptive.
SPEAKER_02Aaron Powell Even if you tell it not to learn anymore.
SPEAKER_00Even if you freeze an LLM's weights, which means you stop it from formally training on new data, the context it encounters in the real world constantly shifts.
SPEAKER_02Aaron Powell So it's reacting to a changing environment.
SPEAKER_00Right. If a banking AI is analyzing thousands of daily transactions to flag fraud, well, the nature of fraud evolves. Trevor Burrus, Jr.
SPEAKER_02Tankers change their tactics.
SPEAKER_00Exactly. And the model starts interpreting those new nuanced patterns. Over time, its baseline for what constitutes normal shifts.
SPEAKER_02And that shift is drift.
SPEAKER_00That degradation of its original baseline is drift. Catching it requires constant automated recalibration.
SPEAKER_02So the first angle is securing the model's brain. What is the second?
SPEAKER_00Aaron Powell The second angle focuses on the agentic platforms.
SPEAKER_02The agents doing the work.
SPEAKER_00Aaron Ross Powell Right. These are the frameworks that actually execute tasks. It is one thing for an AI model to just read a document and summarize the risk. Sure. But it is an entirely different operational threat for an AI agent to be given the authority to actively move money or uh send emails or approve a claim based on that summary.
SPEAKER_02Aaron Powell Because agency implies action.
SPEAKER_00Aaron Ross Powell Correct. And that brings us to the third angle, which is the governance and the guardrails.
SPEAKER_02Aaron Ross Powell The rules of the road.
SPEAKER_00These are the hard-coded, non-negotiable rules wrapped around that agent.
SPEAKER_02Aaron Powell It's like deploying a self-driving armored car for a bank.
SPEAKER_00Oh, I like that analogy. Aaron Powell Yeah.
SPEAKER_02So angle one is making sure the vehicle's internal GPS and logic don't suddenly drift and get lost in a bad neighborhood. Right. Angle two is the actual driving mechanism. So the agenic platform, physically turning the steering wheel and hitting the brakes.
SPEAKER_00Oh, okay.
SPEAKER_02And angle three is the digital governor on the engine, making sure that the car rigidly obeys all traffic laws, speed limits, and you know, banking regulations, no matter what route the AI wants to take.
SPEAKER_00Aaron Powell If any one of those fails, you have a catastrophe.
SPEAKER_02Exactly.
SPEAKER_00That is a great way to visualize the dependencies. But Padra notes a foundational element that underpins all three of those angles. Which is contextual data availability and structuring.
SPEAKER_02Uh the data itself.
SPEAKER_00Right. Going back to your armored car, the GPS, the steering, and the traffic laws are all completely useless if the car doesn't have an accurate structured map of the roads.
SPEAKER_02Aaron Powell That makes total sense.
SPEAKER_00Aaron Powell If your bank's historical data is just a disorganized mess of PDFs and legacy mainframes.
SPEAKER_02The AI is going to crash.
SPEAKER_00The AI cannot perform reliably, regardless of how sophisticated the model is.
SPEAKER_02But you know, locking down those internal guardrails and organizing your data doesn't matter much if the perimeter of the building is breached.
SPEAKER_00No, it doesn't.
SPEAKER_02Which brings us to a much more aggressive part of this equation. The source material shifts from internal compliance to external threats. We have to talk about cybersecurity.
SPEAKER_00The paradigm of cybersecurity in finance is currently undergoing a pretty violent shift. Historically, businesses adopted AI purely for productivity, right? Like how do we process mortgages faster?
SPEAKER_02Yeah. Efficiency.
SPEAKER_00But now functionality and security are entirely converging. Exactly. Hackers are not manually probing firewalls anymore. They are using their own offensive AI to find new vulnerabilities. And they launch exploits at a scale and speed that human IT teams literally cannot perceive, let alone react to.
SPEAKER_02I have to push back on the novelty of this though.
SPEAKER_00Okay, go ahead.
SPEAKER_02I mean, we've had automated cybersecurity tools for decades. Hackers have always used scripts to ping servers looking for open ports. And defenders have always used automated firewalls to block them. Wait, so we are essentially entering an endless automated arms race. Right. Why does throwing an LLM into the mix suddenly mean we have to throw out the old playbook?
SPEAKER_00That's a fair question. The difference is that traditional automation only looks for known vulnerabilities. Right. A script checks if you left the back door unlocked. But an AI generates novel, polymorphic attacks.
SPEAKER_02Aaron Ross Powell Polymorphic, meaning it changes its shape.
SPEAKER_00Yes. It can analyze your bank's specific API documentation, deduce a logical flaw in how two systems talk to each other, and just write a custom exploit in milliseconds.
SPEAKER_02Oh wow.
SPEAKER_00It is creating new keys for locks that didn't even know they had keyholes.
SPEAKER_02Which means human defenders are practically obsolete at the perimeter.
SPEAKER_00Entirely. This is why organizations must adopt a roadmap of using AI for securing AI.
SPEAKER_02Fighting fire with fire.
SPEAKER_00Exactly. Now the industry is trying to regulate this. We have major benchmarks right now. Like what? There's the AWASP Top 10 for LLMs, the NIST AI Risk Management Framework, ISO IE SORTY 2001, and of course the massive EU AI Act.
SPEAKER_02But Bajer points out a pretty sobering flaw with relying on those benchmarks, doesn't he?
SPEAKER_00He does. He emphasizes that frameworks are inherently static documents.
SPEAKER_02Right. They take years to write.
SPEAKER_00They take years to draft, debate, and pass into law. But hackers update their offensive AI models daily.
SPEAKER_02So by the time the law is passed, it's already outdated.
SPEAKER_00Precisely. So while those frameworks are essential baselines, relying on them as a shield is dangerous.
SPEAKER_02You need something active.
SPEAKER_00Your internal AI security agents have to be agile enough to recognize threats that the EU AI Act hasn't even conceptualized yet.
SPEAKER_02So if our defense strategy requires our own AI to constantly prowl through our internal networks, which it does. Running tests, analyzing customer accounts, and looking for vulnerabilities before the hackers find them, we hit an immediate wall.
SPEAKER_01The privacy wall.
SPEAKER_02Because if the AI is digging through customer accounts, the AI now has access to all the customer accounts.
SPEAKER_01Yes, it does.
SPEAKER_02And banking exposing raw, unmasked client financial data to a cloud-based AI model is a federal compliance nightmare.
SPEAKER_00What's fascinating here is how Padra addresses the mechanics of that exact conundrum.
SPEAKER_02How do you fix that?
SPEAKER_00The architecture of these agentic platforms must be explicitly designed to minimize data exposure. You have to use data masking.
SPEAKER_02So you hide the real info.
SPEAKER_00You scramble the names, the account numbers, the social security data. So if the AI somehow leaks it, it is useless.
SPEAKER_02Makes sense.
SPEAKER_00But doing that introduces a massive secondary problem. Which is the tension between data privacy and the financial cost of computing.
SPEAKER_02The financial cost. Why does protecting privacy make the AI more expensive to run?
SPEAKER_00It comes down to how AI processes language and information, which is measured in tokens.
SPEAKER_02Right, tokens.
SPEAKER_00You pay the AI provider, like an open AI or anthropic, based on how many tokens the model has to process.
SPEAKER_02Okay, so more reading equals a higher build.
SPEAKER_00Exactly. When you mask data, you strip away the natural context. A masked database is incredibly confusing to read.
SPEAKER_02Because instead of John Doe, it just says user 4857.
SPEAKER_00Right. To compensate, you have to feed the AI massive amounts of supplementary context and complex instructions so it understands what it is looking at and how to test it.
SPEAKER_02And more context equals more tokens.
SPEAKER_00More tokens equals a much higher computing bill.
SPEAKER_02So it's a trade-off.
SPEAKER_00Yes. These platforms are stuck in a balancing act. They have to calculate what data is benign enough to expose in plain text to keep token costs down.
SPEAKER_02Aaron Ross Powell Versus what has to be hidden.
SPEAKER_00Aaron Powell Right. Versus what data absolutely must be heavily masked and protected to maintain data sovereignty, regardless of the astronomical processing cost.
SPEAKER_02Aaron Powell So how do they solve it? The article mentions a technical solution involving vectorization and uh inverted referencing, or argin retrieval augmented generation.
SPEAKER_00Yes, argin is key here.
SPEAKER_02Aaron Powell How does that actually work under the hood to bypass the masking problem?
SPEAKER_00Aaron Powell Instead of handing the AI a giant mask spreadsheet and asking it to find anomalies, the platform vectorizes the database.
SPEAKER_02Aaron Powell What does vectorizing do?
SPEAKER_00It means it runs the data through an algorithm that translates every piece of information into a mathematical coordinate of vector in a high-dimensional space.
SPEAKER_02Okay, a high dimensional space.
SPEAKER_00Words and numbers are replaced by proximity and mathematical relationships.
SPEAKER_02Aaron Powell To make sure I'm visualizing this right, the AI is no longer reading English text or raw financial number.
SPEAKER_01No.
SPEAKER_02It is operating entirely in a math space, looking at the distance between data points to find the answers.
SPEAKER_00Precisely. Through ARG, the AI queries that vectorized index.
SPEAKER_02So it only pulls what it needs.
SPEAKER_00It pulls only the specific mathematical relationships it needs to answer a prompt or run a security test.
SPEAKER_02Aaron Powell That's incredibly smart.
SPEAKER_00It processes the logic without ever actually reading the raw master copy of the data set.
SPEAKER_02So you get the insight without the exposure.
SPEAKER_00You maintain total control and privacy over your data while dramatically reducing the amount of raw text tokens the AI has to process. It solves both the compliance and the cost issues simultaneously.
SPEAKER_02That is an incredibly elegant mechanical solution.
SPEAKER_00It really is.
SPEAKER_02But let's look at the other side of the coin. We've established how to secure the data, build the guardrails, and fight off the external threats.
SPEAKER_00The defensive side.
SPEAKER_02Right. How does all of this trusted AI actually change the day-to-day reality of a software tech team? Let's move from the defense of cybersecurity to the offense of software development.
SPEAKER_00This is where we see the most tangible revolution for quality assurance teams.
SPEAKER_02The QA testers.
SPEAKER_00Yeah. Audra points out a historical limitation of traditional test automation. In the past, if a QA engineer wanted to write an automated script to test a new login screen, that login screen code had to exist first.
SPEAKER_02Because the testing tool needs something to grab onto.
SPEAKER_00Exactly. Traditional automation tools work by looking for specific elements in the code, like a button ID.
SPEAKER_02Which means the QA team is inherently bottlenecked.
SPEAKER_00Highly bottlenecked.
SPEAKER_02They cannot do their job until the software developers finish coding the feature.
SPEAKER_00In the industry, they call this the N1 Sprint.
SPEAKER_02N minus one, break that down.
SPEAKER_00Development is organized into blocks of time called sprints. The developers are always working on the current Sprint N. The QA team is perpetually stuck writing automated tests for the features built in the previous Sprint N1.
SPEAKER_02So they are always lagging behind.
SPEAKER_00It forces a heavy reliance on slow manual testing for brand new features simply because the automation scripts can't be written until the code is baked.
SPEAKER_02Here's where it gets really interesting.
SPEAKER_00Yeah.
SPEAKER_02Because agentic AI completely shatters that timeline, doesn't it?
SPEAKER_00It eradicates the N1 Sprint entirely.
SPEAKER_02Just wipes it out.
SPEAKER_00The article specifically highlights a platform called testblend.ai, which utilizes this new agentic approach.
SPEAKER_02And how does that fix the lag?
SPEAKER_00QA engineers no longer have to wait for the developer to write the code. They can generate full, complex automation scripts directly from the initial design files.
SPEAKER_02Wait, how is that mechanically possible? If the code doesn't exist, what is the AI testing?
SPEAKER_00It uses multimodal LLMs and computer vision.
SPEAKER_02Okay, so it can see images.
SPEAKER_00You feed the AI a Figma file, which is essentially just a visual mock-up or a picture of what the app will eventually look like.
SPEAKER_02Right, just a picture.
SPEAKER_00The AI understands the visual hierarchy, it knows what a submit button looks like, it knows what a username field is, and it predicts the underlying code structure.
SPEAKER_02Oh wow.
SPEAKER_00It then writes the automated testing scripts based on that prediction before the developer has even started typing.
SPEAKER_02It's like um a film critic writing a fully accurate, publishable review of a movie just by looking at the director's initial pencil storyboards.
SPEAKER_00That's a great way to put it.
SPEAKER_02Rather than waiting for the movie to be filmed, edited, and released.
SPEAKER_00That is the perfect analogy. Bodra calls this day zero automation.
SPEAKER_02Day zero.
SPEAKER_00It completely flips the traditional software lifecycle. You are building the testing infrastructure simultaneously with the design phase.
SPEAKER_02That must speed things up immensely.
SPEAKER_00It drastically accelerates how fast a bank or an insurance company can securely release a new digital product.
SPEAKER_02But this naturally begs a rather large, glaring question.
SPEAKER_00I think I know where you're going with this.
SPEAKER_02We established earlier that AI can drift. It can hallucinate.
SPEAKER_00Right.
SPEAKER_02If an AI is writing the critical software tests for a banking app purely by looking at a picture, who is making sure the AI isn't making a mistake?
SPEAKER_00And that brings us to one of the most futuristic concepts in the entire interview: ATAs.
SPEAKER_02ATAs.
SPEAKER_00Agent testing agents.
SPEAKER_02We are literally deploying AI to test the AI.
SPEAKER_00We have to. A traditional software testing tool cannot test a generative AI agent.
SPEAKER_02Why not?
SPEAKER_00Traditional tools test binary outcomes, pass or fail, yes or no. But AI agents operate using natural language and probabilistic logic. Trevor Burrus, Jr.
SPEAKER_02So they're too unpredictable for standard tools.
SPEAKER_00Exactly. So organizations have to deploy a secondary layer of AI, the ATA, to monitor and interrogate the primary AI.
SPEAKER_02Aaron Powell What exactly is the ATA interrogating? Like what are the mechanics of an AI auditing another AI?
SPEAKER_00Aaron Powell It runs holistic stress tests that go way beyond just clicking buttons.
SPEAKER_02Give me an example.
SPEAKER_00For example, it tests for multi-turn consistency.
SPEAKER_02Aaron Powell What's a multi-turn?
SPEAKER_00A multi-turn is a conversation or a process that requires multiple back and forth steps.
SPEAKER_02Okay.
SPEAKER_00Generative AI is notoriously bad at maintaining context over long interactions. Its context window decays.
SPEAKER_02It forgets what it was doing.
SPEAKER_00Exactly, it forgets what it was doing in step one by the time it reaches step ten.
SPEAKER_02Aaron Powell So the ATA is checking its memory.
SPEAKER_00The ATA actively tries to trap the primary AI in logic loops during these multi-turns to see if it breaks.
SPEAKER_02That makes sense. It's testing the model's stamina.
SPEAKER_00But the most critical thing the ATA looks for in financial setting is information leakage during tool calling.
SPEAKER_02Break down tool calling for me. What does that mean?
SPEAKER_00Let's say a bank's AI assistant needs to calculate the interest on a customer's loan. Okay. The LLM itself doesn't do the math. It reaches out or calls a tool, which in this case is the bank's back-end calculator API.
SPEAKER_02Right. It outsources the math.
SPEAKER_00The danger is what the AI decides to send in that API request. If the primary AI gets confused, it might accidentally bundle the customer's unmasked social security number into the raw text of that tool call.
SPEAKER_02And they exposes it.
SPEAKER_00Exposes it to internal server logs that aren't secure.
SPEAKER_02Oh, that's bad.
SPEAKER_00The ATA's job is to sit in the middle intercepting those tool calls and making sure the primary AI isn't leaking protected data when it talks to other software.
SPEAKER_02Aaron Powell So with all of these layers of automation AI predicting the code, AI writing the tests, and ATAs testing the primary AI, where does the human being fit into this? That's the big question. Do the human QA testers just pack up their desks?
SPEAKER_00If we connect this to the bigger picture, the human role doesn't disappear, but it fundamentally transforms.
SPEAKER_02How so?
SPEAKER_00The human in the loop remains vital, but they move from being an executor to an orchestrator.
SPEAKER_02Explain the difference.
SPEAKER_00Execution is writing the Python script to test the login button. The AI does that now.
SPEAKER_02Right.
SPEAKER_00Orchestration is knowing what the login button represents to the business. You see, platforms like testblend.ai are codeless.
SPEAKER_02You don't need to write code.
SPEAKER_00You don't need to know programming languages to use them. Therefore, the future QA worker doesn't need to be a coder.
SPEAKER_02What do they need to be?
SPEAKER_00They need deep analytical skills and profound domain knowledge.
SPEAKER_02They need to understand the banking regulations, the business logic, and the financial architecture.
SPEAKER_00Right. There is a definitive quote from Bhadra that summarizes this. He says agentic automation is redefining QA from execution to orchestration. The focus is shifting toward validating intelligence, not just testing software.
SPEAKER_02Validating intelligence.
SPEAKER_00The AI accuracy depends heavily on context. Humans are needed to sanitize the inputs, design the complex prompt architectures, and verify the outputs to correct model drift.
SPEAKER_02So the human manages the machine that does the clicking. Let's pull all of this together. We started with the foundational three pillars of AI trust model reliability, agentic execution, and rigid governance.
SPEAKER_00All anchored by structured data.
SPEAKER_02Right. We explored the mechanics of the cybersecurity arms race, where static defense frameworks fail against polymorphic AI attacks, forcing this AI versus AI perimeter.
SPEAKER_00An endless battle.
SPEAKER_02We broke down the immense tension between data sovereignty and token costs and how vectorization and R allow AI to analyze data mathematically without ever reading the raw, masked text.
SPEAKER_00A brilliant workaround.
SPEAKER_02We looked at how multimodal models are killing the N1 sprint by writing day zero tests directly from design files.
SPEAKER_00Changing the whole timeline.
SPEAKER_02And finally, we arrived at the rise of the agent testing agent and the human's evolution from a manual coder to an orchestrator validating intelligence.
SPEAKER_00That pretty much covers it.
SPEAKER_02So what does this actually mean for you?
SPEAKER_00A lot, honestly.
SPEAKER_02Whether you are in finance, healthcare, or logistics, the underlying mechanics of your industry are changing. The future of your career won't be defined by your ability to execute repetitive manual tasks.
SPEAKER_00No, the machines have that covered.
SPEAKER_02The machines are infinitely faster at execution. Your future value lies entirely in your domain expertise. Your deep, nuanced understanding of why a system works will be infinitely more valuable than knowing how to code it.
SPEAKER_00That is the essential takeaway.
SPEAKER_02Yeah.
SPEAKER_00But it also leaves us with a paradox, something that builds on everything Padra outlined.
SPEAKER_02Lay it on us.
SPEAKER_00If an AI agent is predicting and writing the code, and another AI agent is testing that code, and an ATA is actively testing the tester, at what point does human domain knowledge just become a rubber stamp? Oof. As we delegate the actual mechanical understanding of our architecture to these codeless agentic platforms, how do we ensure that we as humans don't eventually lose the fundamental understanding of how our own critical financial systems actually operate under the hood?
SPEAKER_02That is a brilliant, slightly unsettling paradox to leave you with. If we outsource the blueprints and the construction, do we eventually forget how the building stands up? Keep that in mind the next time you log into your banking app and hit transfer. Thanks for joining us on this deep dive, and we'll catch you next time.