Why Platform Maturity Determines Agentic AI Success

AI is getting more capable, but it is also getting more expensive and that changes everything. We sit down with Jason Perry, a cloud and platform engineer at BBD Software, to talk about agentic AI that actually survives contact with real enterprise workflows: budgets, governance, security, compliance, and the uncomfortable question of responsibility when autonomous agents act on their own. 

We dig into why “strategic deployment” beats chasing the largest model, and how rising token costs are pushing organisations to prove ROI in concrete terms. Jason breaks down what makes AI platform engineering different from simply using off-the-shelf productivity tools, then walks through the biggest blockers teams hit today. We cover cost control guardrails, observability, the EU AI Act and other evolving regulations, and why agent identity and access management is often the most technically complex piece of scaling agentic AI safely. 

We also explore practical ways to reduce waste while improving quality: minimising tokens per correct answer, narrowing context, limiting tools, and using techniques like semantic routing and model response caching through an AI gateway. Finally, we look ahead to the shift toward small language models and how that can reshape cloud versus on-prem decisions around data sovereignty and governance. 

If you’re building an AI strategy, an AI platform, or autonomous agents you can trust, subscribe, share this with a teammate, and leave a review. What is the one workflow you would automate with agentic AI first?

Transcript

Welcome And Meet Jason

SPEAKER_01 0:20

Tech Unboxed with BBD.

SPEAKER_00 0:24

Welcome to another BBD Tech Unboxed. And today I'm with Jason. And Jason is a real human being. Ladies and gentlemen, he will enlighten us about Agentic AI. Jason, can you please introduce yourself?

SPEAKER_02 0:39

Yeah, sure. So, like Quinn mentioned, I'm Jason, Jason Perry. I am a cloud and platform engineer for BBD software. I work out of the Netherlands and I've been on a number of our clients dealing with things like container orchestration platforms, serverless platforms, data platforms, and now with all of the hype around us and what we're discussing today, AI platforms.

Strategic Deployment Over Bigger Models

SPEAKER_00 1:02

And you recently did an article in Tech UK, which is a very nice magazine. And I I was reading your article, and what struck me was less about size, more about strategic deployment. So can you light us a bit more what you meant by this phrase?

SPEAKER_02 1:21

Yeah, definitely. And I think it's something that will become probably a bit of a common theme throughout our discussion. Cost is becoming a big factor. It doesn't really make sense to throw a tech that's becoming increasingly more expensive at problems where there's deterministic options or basically where AI isn't suitable. And what we're seeing really is that a lot of enterprises know that they want to use AI, but and they're investing in it heavily, to be clear, but they're struggling to identify suitable concrete use cases where AI really makes sense. And that's kind of what I'm referring to here is um strategic deployment focuses on taking AI where it's really going to be beneficial for an enterprise, deploying it there, and then quantifying the benefits that they get out of it.

SPEAKER_00 2:11

I was also surprised because recently there has been some press coverage around uh some of the larger organizations that pull away from AI and actually go back to human employees.

SPEAKER_02 2:24

Yeah, um I kind of think that might have been a bit of an overinvestment from some companies. You know, some jumped the gun too early during a time where uh AI was still very much an emerging technology, although it still is even today. And during a time where token costs were quite heavily subsidized. Uh so basically what we saw was some companies going all in on a technology that hadn't yet been proven to be cost effective, and now we're seeing a bit of a correction for that in some companies. But overall, I would say there's still a very positive outlook on where AI is going. We just have to be cautious about how it's implemented.

Why AI Is Not A Five-Minute Job

SPEAKER_00 2:59

Well, it's funny that we we talk about that side of the equation, but there's also the other side of the equation, which is it's very simple to set up, it's uh it's a five-minute job. Um but I think it in reality it's much more complicated, isn't it?

SPEAKER_02 3:15

Uh it definitely is. Um and I think it's kind of important to highlight the difference here between people using existing or pre-built offerings for things like productivity versus actually implementing AI or agent solutions into business workflows. Um and if you look at the ecosystem, it's becoming very broad. So what you would typically expect, uh you've got everything, I mean, let's just ignore training and hosting large language models, right? Because that's a completely different ball game. You still have to deal with all of the other concerns. You've got governance, cost, observability, uh, security and access control, all of these different factors come into play. And it's quite rare to have the skill, uh, the knowledge, and the experience in-house to cater for all of these different topics. And even if you do have that, it's pretty rare for organizations to have platforms at a level of maturity where you can immediately uh build AI solutions on top of your platform.

The Real Blockers Cost Compliance Security

SPEAKER_00 4:15

So so you mentioned uh security, compliance, costs, and I think an important one is personal identification.

SPEAKER_02 4:23

Definitely.

SPEAKER_00 4:24

But if you look at all of them and and you would look at projects that you have been running for some of your clients, what would you say is the the biggest offender and how would you rank them?

SPEAKER_02 4:36

I think that's a bit of a tough one because it's changing over time. Um so I would say it's hard to rank them, but just to go through them. Cost, like I mentioned, wasn't as much of a concern initially. Uh, but now we're seeing even massive AI budgets getting consumed pretty rapidly uh due to the changes in subsidization and token costs. We also see now that compliance is catching up. You've got things like the EU AI Act, um, and these are really important because they can be showstoppers, right? They can end entire projects if you're not compliant. And then security is typically the kind of thing that people don't think about as much as they should until something goes wrong. Uh so they're all very important factors, but I think right now the thing that's in focus a lot is cost.

Agent Identity And Audit Trails

SPEAKER_00 5:22

Uh what about the um identification part?

SPEAKER_02 5:25

What people typically are not thinking about at the moment, I would say, is that agents need to act autonomously. Right. Of course, where possible, you want a human in the loop to make sure nothing has gone crazily wrong. But what we need to do is make sure that agents have their own identity, that they can get access to the systems that they need when they need it, that they only have access to the things that they need access to, and that they lose it when they no longer need it. And that we can scale this across large organizations where you're going to have at some point many agents running simultaneously, doing all sorts of things without human oversight, but you still need an order trail to understand what's happened and potentially where something has gone wrong.

SPEAKER_00 6:06

Now I'll take you back on the question of sort of ranking them and providing me the biggest offender. Which one would that then be?

SPEAKER_02 6:17

Sure, you bring in uh a tough question. Yeah, let me say I'll go for cost being the biggest offender at the moment because I think that uh budgeting is becoming super crucial. And I'll say that um identity is probably the most technically complex one to deal with.

Where AI Regulation Is Heading

SPEAKER_00 6:35

Looking back at compliance, um specifically in Europe, you already mentioned the EU AI Act. There's also another one in the UK, there's another one in uh in the US, that's multiple versions across the world. The these frameworks are evolving, but where do you see this going? And and how do you see that being useful on your day-to-day job?

SPEAKER_02 6:56

Like I mentioned, they're kind of catching up with all of the rapid development that's been happening over the past few years in the tech space. And compliance is becoming increasingly a serious concern that a lot of projects have to take into account. Um, where we see it going, well, we obviously try to engage with the parties working on these acts and frameworks. In some ways, this is to give them guidance from what we're seeing out in industry, and in other ways, it's so that we can get an idea of where things are heading to make sure that we're not building something that's going to be non-compliant at some point. What we're kind of seeing is that everyone is positive about the use of AI, but very cautious about what and when you can do things with it. Um the thing I think societally that we haven't really solved is how to make or how to give responsibility to an AI agent. And when I say that, what I really mean is if something goes wrong, usually up until now you've been able to blame a human. And that was great, right? You could fire Bob if he did a bad job. But if you've got an AI agent that's made a crucial mistake, somebody's lost money, even worse, somebody has potentially died in serious situations. There's no real way, legally speaking, to deal with that. And that's kind of a problem that's going to be difficult to take care of uh in terms of legislation and compliance, and something that we're really going to have to think about from a technical perspective.

SPEAKER_00 8:18

Well,

Measuring ROI When Tokens Are Opaque

SPEAKER_00 8:19

we also talked about um sort of the benefit that it brings. Um how do you think that today benefit has been measured within organizations?

SPEAKER_02 8:28

Yeah, that question is on a lot of people's minds. Things are a little bit opaque at the moment in terms of measuring token costs and benefits. Uh the problem that we're seeing really is that if you're taking some of the existing offerings, let's start off with the like subscription-based offerings, it's often opaque or uh ambiguous as to how many tokens or how much usage you're getting for a certain like amount of requests. So what people are typically doing is they're going towards AI-based uh bully, which is more predictable. But then even when you do that, it's hard to quantify how much benefit you're getting out of a specific, let's say, amount of tokens. And I think there needs to be a lot more focus on minimizing the amount of tokens per correct answer. Up till now, there's been a lot of focus on decreasing cost and or tokens in general, but that doesn't really help if the output is not useful. Um, so it's kind of important here to focus on picking, like I mentioned earlier, use cases where you can demonstrate and quantify the benefit AI has, and then you can tie that back into the cost per token that you're actually uh seeing.

SPEAKER_00 9:41

I like that notion that you bring in play. It's almost like the return on investment. You have the amount of tokens and you try to get the most out of it. Are there any other ways companies can prepare on this idea that the the actual token costs would be rippled down to their actual invoices?

Guardrails For Budgets And Spend

SPEAKER_02 10:01

Yeah, definitely. Definitely. Um I would say uh you can kind of rank them from more simple things you can do to more technically complex things you can do. So number one, I keep going on about this, but really picking use cases where AI makes sense. Right? Don't throw it at just any problem, be very intentional about where you're using it. Number two, building in platform guardrails to avoid overspending. So this is everything from budgets per uh, let's call it API key, um, to actual budgets per project, cost monitoring, alerting, all of the typical things that you would have seen up till now in broad enterprise platforms, but specifically apply to agentic AI solutions. And then you can really get into some technically complex things that we're seeing these days for intelligent cost control. That's things like semantic routine, model response caching, and these often go through uh tools like an AI gateway.

Policy Guardrails Built Into Platforms

SPEAKER_00 10:59

It was something that also struck my eye when I was reading your article that you mentioned uh it's easy to include policy guardrails uh within the actual prompts and within the actual agentic flows. Is that exactly what you meant?

SPEAKER_02 11:13

It's important to make a distinction here between what's happening from the user side and what's happening from the platform side because the focus of the article is really from a platform engineering perspective. And I think uh the policy guardrails that we typically see and implement coming from the platform side, where we try to build things in so that consumers of the platform or users of the platform get those security and compliance uh by default. So you don't really want to be in a situation where a user of the platform has to be responsible for making sure that they do this in a secure and compliant manner. We want to make sure from a platform engineering perspective that we give users the peace of mind that when they do something within our ecosystems or our environments that we build up for them, that they won't run into a situation where they can do something irresponsible or insecure.

Shared Responsibility Starts With Platform

SPEAKER_00 12:08

Well, it's an interesting um perspective that you put uh people against the platform. So who do in your in your perspective, who is then most responsible? Is it the platform engineering or is it actually the user to ensure that you comply with uh being within the costrails, being within the policy guardrails and so on?

SPEAKER_02 12:26

It's a bit of a shared responsibility model, like we often see in tech, but I think at the foundation of it, it starts off at the platform, because the platform engineers are the ones who are expected to have the knowledge, the experience, and the responsibility for ensuring that these types of things can't happen. Of course, from a user perspective, you should always be doing things like trying to make sure you're being secure, right? Try not to use tools that could be potentially compromised, try to make sure that you stay within the cost guardrails, be intentional about where and when you do things. But ultimately, the goal of platform engineering is to try and remove that burden, that cognitive overhead from consumers of the platform. Platform engineers want to take that responsibility and want to make sure that their platforms are easy to use so that consumers don't have to worry about all of these types of concerns?

SPEAKER_00 13:15

More of a personal question because you call it platform, but in your explanation, it sounds to me a lot like a good architecture. Are there like advice that you can give in order to help companies to deal with the situation better?

Build Platform Maturity Before Scaling

SPEAKER_02 13:33

Yeah, I can definitely try, and to your point, your platform architecture is probably the number one determining factor of whether you get this right or wrong. And uh I kind of alluded to this in the article as well. I think the level of platform maturity that an organization has is directly correlated with how easily they can deploy agentic AI solutions. So I can unpack that a little bit. The thing that we're seeing is organizations that don't have a level of platform maturity tend not to be able to do all of the things that I was just discussing, right? They don't have a way to automatically enforce policy guardrails. They don't have a self-service mechanism so that engineering teams can consume these types of capabilities, which means that every single engineering team then gets that responsibility for making sure that they're secure and within the guardrails and all of those different kinds of concerns that we've been discussing. So, my recommendation for organizations would be to start focusing on your platform foundation before you get to the point where you're trying to scale agentic AI solutions across your business. Because the platform engineering foundation is what gives you the ability to automatically apply those guardrails, those cost uh budgets, all of those types of things. And then once you have that foundation, you can look at the specific AI relevant topics. Like I said, maybe you're implementing an AI gateway, maybe you're creating custom policies that are tailored towards AI or agentec solutions as opposed to your more broad workload uh policies. But yeah, so my recommendation is definitely just focus on a strong platform foundation to enable you to build your AI solutions on top of that.

SPEAKER_00 15:11

What

The Shift Toward Small Language Models

SPEAKER_00 15:12

you also talked about in the article was um it's not about uh being like always right on the the largest model, but it's more the challenge about uh choosing the right model. So LLM becomes small LM, right? SLM. So what's your take on that?

SPEAKER_02 15:33

Yeah, I think something that we're seeing more and more is a shift in perspective. LLMs give you what we typically call general intelligence, right? But if you're trying to use it for a specific problem, let's say trying to um assist with coding, it doesn't really matter that a large language model can help you with a medical problem, right? That's got no relevance or no bearing on the problem at hand. So what we're seeing or what we think, and this is a prediction, right? Where it's not really broadly implemented.

SPEAKER_00 16:03

We like predictions in this podcast.

SPEAKER_02 16:05

I'll get to you it's then I'll give you my hot take at the moment. Yes, please. I think there's going to be a large shift towards small language models that are very focused on specific problems. I think number one, the cost factor that we spoke about will become huge here because the hardware resources that you need to run a small language model versus a large language model it's vastly different. And um just so that everyone knows, essentially a small language model, you take an existing smaller model, so not the massive amount of parameters that you would expect in your frontier models like Fable, uh GPT 5.6, you would take one of those models.

SPEAKER_00 16:45

Immediately taking the big ones.

SPEAKER_02 16:46

Yeah, yeah. So if you if you eliminate those large language models, right, and you move towards the the smaller ones, for example, your Kimi K2.5 and now K3 has just been released, you take those types of models or even Quen and you're training them or fine-tuning them on a specific topic. That yields a small language model, and then what you want to do is benchmark that against the performance of a large language model. And what we're often seeing, um at least in certain situations, like I said, this hasn't been broadly adopted yet, is that for specific topics, you can get similar performance with a small language model for that topic to what you would get from a large language model. And um, I think one of the best kind of ways or one of the best examples of this is Cursor and their Composer 2.5 model. It's a small, relatively small language model compared to the large language models. Its performance for software engineering and for coding is excellent, but it definitely won't give you that general level of intelligence that you expect from the uh larger language models.

Cloud Versus On-Prem Data Sovereignty

SPEAKER_00 17:51

Are you not also sort of moving into a discussion where you actually have the discussion cloud on-prem or on device?

SPEAKER_02 18:00

Yeah, um, I'll say that's something that we're definitely discussing in a lot of different clients and internally at BBD at the moment. The cost trade-off, but also the compliance, governance, data sovereignty. I mean, I think there's probably been a lot of discussion at the moment about EU sovereignty from a cloud perspective. People are really thinking about where they should be running their workloads. And small language models make that thought process a lot easier because you're not worrying about installing massive uh GPU-based instances on your own infrastructure, right? So I'm not saying people should necessarily go in that direction, but it gives you the option if that's where it makes sense for your business. You don't have to rely on these big frontier model providers or the large cloud uh providers for resources that you may not have internally. So it makes that discussion a lot easier to have.

Better Prompts Less Context Fewer Tools

SPEAKER_00 18:55

Well during this conversation and also in the article, you talk about uh different things. You talk about better prompting, uh limit the tools, make them the right tools. You talk also about memory um management, something that we didn't talk about here. Um Which of those uh tend to give you the biggest lever if you need to get started in an organization?

SPEAKER_02 19:22

Yeah, that's a really good question. I would think it's important to mention here that um some of these can be complex things and it doesn't always make sense for an organization to start off with these from the get-go. Right. So what we're seeing is in some cases um it makes sense to actually build your own agency harnesses, and that's for really complex workflows, complex use cases where you need this type of benefit. But if you're looking at the more general use case, ones that aren't uh specifically complex, I would say very much so trying to limit the amount of context that goes into your model is one of the big kind of gains that we're seeing. Because the more you put into these models, the more attention you lose, and the more reliability you lose in turn. So I would say things like your prompt, making sure that they're streamlined, making sure that the context that you give is narrow. So try not to give it an entire folder, try to give it the specific files that make sense for the problem that you're trying to solve. Um, that's also where semantic search comes in that a lot of these models have. They're searching for the specific context that makes sense for the problem. And then your tools, right? Try not to install just any old tool and skill that you see. Often what it does is it bloats your context in cases where the agent is never going to use it. So for each kind of problem that you're trying to tackle, really focus on getting the context, the tools, the skills in place that are relevant for the thing you're trying to solve.

The Core Takeaway On Maturity

SPEAKER_00 20:52

Well, Jason, we're almost at the end here, and uh I just want to make um uh a summary statement here, and you tell me whether I'm I'm right and if I understood well what you uh you're saying. So you say a company AI platform maturity is directly correlated to the success of deployments of Argentic AI and also related to the cost management to it. Is that correct?

SPEAKER_02 21:19

Yeah, I would definitely agree with that. Um like I said, you need a foundation to build and scale out these solutions, right? And I'm not talking about small, little individual implementations, I'm talking about broad enterprise adoption. You need that platform foundation to build in the guardrails, make sure that you're ready to adopt and scale AI solutions across your business, and I think that's where a lot of focus should be placed.

SPEAKER_00 21:46

And why should you be ready looking forward in time? Do you expect every company to be at the high level of maturity?

SPEAKER_02 21:52

Or right now I think many aren't. And I think for many, it doesn't make sense to build that capability internally. Um, we're seeing a lot of providers coming out with offerings. Let's look at Versal and Cloudflare, for example. I think that a lot of companies, especially smaller ones, it would make sense to just adopt those types of offerings. But for large enterprises, I definitely foresee a large focus on building out that platform maturity internally to enable this broad AI adoption.

How To Reach Us For Help

SPEAKER_00 22:25

Now, most of the companies that will listen will probably reflect and they'll think, Jason, come and help me. So how can they reach out to you?

SPEAKER_02 22:33

Yep. Well, if you need help with setting up um agentic AI platforms, we do have experience with this. You can reach out to us. Um, we're at bbdsoftware.com, or uh you can reach out to us on LinkedIn, contact me personally, and I can refer you to the correct people.

SPEAKER_00 22:46

Well, thank you very much for joining us today, Jason, and thank you also to the listeners.

Closing And What Is Next

SPEAKER_00 22:51

Please stay tuned as we're gonna do more deep dives here on Tech Unboxed.