Managing Token Costs: Agentic AI's Invisible Expense

Agentic AI is transforming Global Business Services: but are you budgeting for the hidden cost behind every AI decision?

In this episode, Sally Fletcher sits down with Hypatos CEO Uli Erxleben to explore the growing impact of AI token costs. As organizations scale AI agents across finance and shared services, token consumption can quickly become one of the biggest drivers of operating costs.

Discover why falling token prices don't necessarily mean lower AI bills, how token economics should influence build-versus-buy decisions, and what leading organizations are doing to control costs through smarter deployment and outcome-based pricing.

Whether you're leading GBS, finance transformation, or enterprise AI initiatives, this conversation will change how you think about the true cost of Agentic AI.

Read more about managing token costs here. 

Register for our next webinar on Securing Agentic AI at Scale. 

Transcript

SPEAKER_00 0:02

If you start to build more complicated things where you have multiple agents, it's not so easy to understand the actual token consumption that a end-to-end transaction is gonna have. And if you go for something that is super high volume with lots and lots of tokens, there is specific tasks that are incredibly hard to do for you internally, right? Because you need to optimize the workflow, so how the agents work together to make sure that they are optimized for token usage, that they are not processing stuff twice, right? That they use work results from before to minimize the token consumption in a workflow, right? That's one thing that needs to be done. And then there is an additional complication, and that's something that most companies do not put at all any attention to, but it's critical.

SPEAKER_02 1:04

AI won't politely integrate into your GBS model. It will either transform it or expose its limits. Welcome to GBS Rewired with me, Sally Fletcher, head of Thought Leadership and Community at Hypertus. This podcast is for GPS leaders who know incremental change isn't enough. In each episode, I sit down with the executives redefining how global business services operate in the age of AI. From operating model, we designed a fully agentic GBS. We challenge the big questions. Is your current GPS model built for automation or autonomy? What does an agentic GPS really look like when AI agents make decisions and not just process tasks? Where is AI delivering genuine enterprise value and where is it just hype? And are today's GPS leaders equipped for what's coming next? If you're ready to move beyond pilots and into real transformation, you're in the right place. GBS Rewired, turning AI ambition into GBS reality. And that topic is token costs. For years, the economics of shared services were relatively easy to model. We had headcount, we had seat costs, software licenses, costs per FDE. These costs were fairly easy to plan for and were relatively predictable. Agentic AI is completely changing that equation. The fact that it's an incredibly valuable tool is undisputed, and rightly, the vast majority of GBS organizations are already implementing it in some way. However, done in the wrong way, it can result in an unexpected bill, and that's an unexpectedly large bill. This new variable is the token, which, as the podcast goes into, is a basic unit of cost behind every AI agent's reasoning, every retrieval, and every multi-step workflow. Unlike a fixed software license, which many organizations are used to, token consumption scales with complexity. An AI agent that reasons through five steps instead of one, that calls out multiple tools, that retries when it gets something wrong, all of this adds up invisibly at the infrastructure layer. For GBS organizations that are building their own AI agents and are moving from pilots to scaling agentic deployments, that lack of cost transparency is a real risk, especially for a model that is usually built on cost arbitrage, which many GBS organizations are. What look like a regular automation investment when you started can behave much more like a variable cost electricity bill. So getting a shock after Christmas. And few finance or operations teams are set up to forecast in this way at the moment. In this episode, I asked Hypertiss's CEO Wooly Erksleben to explain to us what this shift actually means for GBS, how token economics should factor into that build versus buy decision, and how to mitigate rising token costs. We also look at where all of this is heading because although cost per token is falling, the total number of tokens being consumed is exploding as organizations deploy them with greater and greater numbers. I hope you enjoy this conversation. So I really wanted to get your perspective, Uli, on token costs and how you see them impacting GBS strategy, business decisions, and so on. So I'll dive straight into our first question. Really, the fundamentals. You know, give me a brief overview of token costs and why everyone is talking about them right now.

SPEAKER_00 5:42

Yeah, uh it's it's a super topic and it should be top of mind, actually. So tokens are the currency in which you pay for using large language model technology. I don't want to go too much into the technicalities, but just very briefly what it is. So the way that large language models work, right? They're using a technology that's called transformer models, it's an AI technology, what have you. Um but essentially, uh, in order for an AI to understand human language, they take words and turn them into numerical representations of those words and place them in vector rooms, right? I know it's very technical to say this. Um but this is a token, right? A word is transformed into a token. And you send tokens to large language models so that they understand, and then they provide you the answers also in tokens. And you pay for those tokens, right? It's the currency in which you pay for large language models. So and if you are using large language models in a relatively simple way, right? Chatting with them, it's just very few tokens that this is going to cost. So using assistance, not a big deal, nothing you need to necessarily worry about. But if you start using large language models in agentic workflows, where you have many different agents taking care of all different things in end-to-end multi-step workflows, using lots of work instruction inputs, dealing with very big documents that have lots of words. Think about contracts, for example, and then digging into your context, your master data, your PO information, your chart of accounts, your handbooks, your procedures. There is a lot of words. And all those words are consuming then tokens that need to be paid for. And those tokens cost different amounts of money depending on what type of models you're using. So different model versions have different token costs and models that can reason and that have high performance, right? The latest installments of those frontier labs, OpenAI, Google, what have you, that can become super, super expensive. And it's something that is not so easy to track and understand. And there are companies that have been toying around with large language models and the genetic workflow orchestration that come to realize that this gets so expensive that the business cases are not working out. But they also have little control of what's actually happening there when stuff just runs through those agents and then they wake up and spend crazy amounts of money on tokens. And that's a big headache and complication. I mean, there's horror stories out there that companies have been spending millions in tokens and not being really in control of it. So it's something that is to be planned for and considered. That's what tokens are, right? And they can become a big headache and and are hard to budget in if you build something on your own or if you work with an orchestration platform.

SPEAKER_02 9:06

Yeah, for sure.

SPEAKER_01 9:06

And I think it's almost like if I didn't know better, I'd say these LLM companies have made it difficult on purpose to understand.

SPEAKER_02 9:15

Which is possibly. But yeah, you've touched on a really good point there, this whole build versus buy um discussion that we've had ongoing, we've seen in a lot of conferences that we've attended. Um, is there anything else that you want to expand on here? Because you know, how does this factor into the financial position to implement agentic AI?

SPEAKER_00 9:36

Yeah. So if you build and if you start to build more complicated things where you have multiple agents and agents can have different models, it's not so easy to understand the actual token consumption that a end-to-end transaction is gonna have. And what I strongly believe is that if you go for something that is super high volume with lots and lots of tokens, there is specific tasks that are incredibly hard to do for you internally, right? Because you need to optimize the workflows, so how the agents work together to make sure that they are optimized for token usage, that they are not processing stuff twice, right? That they use work results from before to minimize the token consumption in a workflow, right? That's one thing that needs to be done. And then there is an additional complication, and that's something that most companies do not put at all any attention to, but it's critical because the there's always new versions of large language models coming out, and you need to constantly monitor those new versions and you need to understand whether they can do a better job, get you to better automation. But they also cost differently. So the token prices are starkly different, right? I mean, that there can be 20x difference in how much a token is per model. So whenever there is new models coming out, you need to assess how well they perform and what the token cost impact is. And you can also not just rely and say, hey, I have this model here and I've optimized already a little bit, it's okay, because those older models also are not supported anymore at some point in time. So you need to go through that process evaluating models for performance, evaluating models for token consumptions, and then decide on updates of models. And now just think about a world where you have an end-to-end workflow where you have, I don't know, 50 different agents working, and every agent is using different large language models. This is incredibly hard to do and hard to measure. And now to make things even worse, there is one very important complication. Even if you would go from one model version to the next generation, from a GPT 5.5 to 5.6, what have you, those new models also may understand the work instructions, the prompts, and also your data, your context differently. So the output may be different. So you need to also regression test. So there is a lot of engineering AI tasks to be done in order to stay in control and up to date with those models. And you don't want to do this, especially for those mighty-step complicated, complex workflows. And this is where a buy decision should really be considered, right? Because vendors like us, that's our job to do this. The way that you pay for an out-of-the-box agent system for those big multi-step processes that we like so much, is you pay by outcome. So the way that we think about this is there is different agents, agents take care of different tasks that have been done by humans before, and those tasks fall into different complexities, right? I mean, there's low complexity things like data extraction from an invoice, what have you, right? I mean, this is the lowest capability that is required to get that done, and that's super cheap, right? Costs a cent, right? Because also requires little tokens and little work instructions. But then there's things like taxation, right? Tax compliance, tax coding, where you need to do a lot of expert things, right? That would be a high complexity capability that is much more token intensive, and that task would cost more. So the way that V price is that you pay per executed task and the outcome. And this you can use over the span of the contract, three, five years, what have you, always with the same price. And of course, you also always agree on limiting the uh cost expansion if you go into renewal and so on. So you will have full budget control, and it's our job to make sure that the token consumption is optimized, that the right models are chosen, that the regressions are done. So this gives you lots of peace of mind in how you budget and how you build your business cases. And it's something that should be done by a specialist and and not by your teams because it's hard, right? And and this is, I think, one of the strongest arguments actually, to go for um standard solutions, out-of-the-box solutions from vendors with full cost control and just outsourcing the maintenance of those models to specialists.

SPEAKER_02 14:54

Yeah, I totally agree. I think it's obviously the cost is always a huge consideration factor, especially for global business services, who traditionally started as a cost arbitrage play, but particularly the fluctuation of token cost and the unpredictability, you can't forecast them necessarily. It's something that you can't put in your budget for a year or two years, three years time because you're not quite sure where they're going to go. We've never been in this future before. Um, so I think it's also kind of this unknown factor. Um, when you take on that risk yourself, um, is is very scary to think about. So, yeah, it makes a lot of sense what you're saying to focus with these providers like Hypertus, obviously, that have the outcome-based pricing. The other question that has come up a few times, um, just to play devil's advocate, and the question was if token costs rise, won't agentic AI providers like Hypertus put the prices up? What's your hot take on this?

SPEAKER_00 15:55

Yeah, no, it's it's it's also top of mind concern. Not necessarily the price increase. Um let me put it all into perspective real quick. So, I mean token prices have been coming down consistently. There's quite a bit of competition uh between the um frontier labs and BFC a continuous decrease in in token costs, right? That's a good effect. Question is a bit if this is continuous, right? Given that there is limited um uh uh data center resources, uh limited energy, right? And uh the more tokens are used, the more large language models come into everyday life. Um, of course, there is maybe a supply-side shock or issue that is going to lead to raises in token costs. One thing that could happen. And then there's another concern, right? I mean, as you all are aware, there have been there has been quite recently a situation where, and and most of those frontier labs are of course sort of US-based, where there was a decision by the US government to not provide certain large language models to companies outside of the US, right? So they they just cut the access to those large language models. So this is all factors that that need to be considered. So our approach here is that we would always use a efficient, well-priced commercial large language model from one of those frontier labs in those different ways, how you can consume them on a hyperscaler. But what we also do, just to be prepared for a situation where there might be a price shock or accessibility is restricted, is that we also offer self-hosted, so self-hosted by us, but there's also options that clients can self-host if they choose. That's usually more applicable for highly regulated industry, defense, what have you, public administration. Um, but usually we would be running open source large language models. That's large language models that are often distilled from those frontier lab models. They're lacking a couple of months, but they are still performing very strongly. And especially for most of those high-volume document-based transactional things, they're absolutely adequate to do the job. That we can run on our own servers and completely control then access and cost. So we can also always just fall back to those options if such a shock occurs. And that's something that a company cannot just do, because the infrastructure requirements to running those large language models, also the open source ones at scale, is very high, right? You need GPUs and all that stuff. But but we can do this, right? So we can very easily just switch over to those self-hosting options and switch between all different types of commercial large language models, because we have access and contracts with all of those providers and all those different hosting uh forms so that we can easily mitigate. And right, it hasn't been a big topic yet, but it is a potential risk. And I think it should also be factored in your decisions, because if you build a dependency there, it can of course terribly backfire. No access anymore, then your systems are down. And right, we are very much focused on those mission critical always on uh transactional things, right? They must run. There's no alternative, right? If they're not running and you cannot pay your vendors, you you have a really big problem. So contingency and having the options to switch around, go into self-hosting flexibly. Um that's one of the big value propositions, right, that that we are providing there.

SPEAKER_01 19:58

Brilliant. Um, sadly that's what we've got time for, but thank you very much, Ellie, for speaking about token costs with us.

SPEAKER_02 20:08

Thanks for joining us on GBS Rewired. The future of Global Business Services is being shaped right now by leaders willing to question, experiment, and evolve. If today's conversation spark new ideas, share it with your team or a fellow GBS leader and keep the dialogue going inside your organization. To stay connected beyond the podcast, join the HyperS community on LinkedIn or add me personally, Sally Fletcher, and we will continue the conversation, share insights and spotlight practical examples of AI in action. Subscribe to the show so you don't miss the next episode and join us each fortnight as we explore what's possible when AI and GBS come together with clarity and purpose. GBS Rewired turning AI ambition into GBS reality.