AI Proving Ground Podcast
AI deployment and adoption is complex — this podcast makes it actionable. Join top experts, IT leaders and innovators as we explore AI’s toughest challenges, uncover real-world case studies, and reveal practical insights that drive AI ROI. From strategy to execution, we break down what works (and what doesn’t) in enterprise AI. New episodes every week.
AI Proving Ground Podcast
AI Agents: Scaling Your Digital Workforce
Enterprise adoption of AI agents is spiking, with industry surveys showing most large enterprises are already piloting AI agents. But there's a lot of hype out there. In this episode of the AI Proving Ground Podcast, Technical Solutions Architect Ina Poecher and Senior Director of WWT's AI Practice expose widespread "agent-washing," highlight practical but ROI-heavy use cases you can pursue now. Tune in to discover how agentic architecture turns generative AI hype into secure, measurable enterprise AI outcomes — and what's next for multi-agent ecosystems.
Support for this episode provided by: Island
More about this week's guests:
Ina Poecher is a Technical Solutions Architect at World Wide Technology (WWT) and collaborates with customers and internal teams to design and validate innovative technology solutions. Working within WWT's Advanced Technology Center, she leverages extensive experience across IT infrastructure, cloud, networking and automation to develop and test complex architectures that drive business outcomes and support strategic initiatives.
Ina's top pick: Hack the Future: AI Meets Security
Jason Campagna leads AI solution strategy at World Wide Technology, helping enterprises adopt intelligent systems from AI assistants to autonomous agents. A technologist since 2001, he brings deep expertise in cloud, automation and platform architecture. Jason is known for bridging disruptive tech with real-world execution and building agile, diverse teams to drive innovation at scale.
Jason's top pick: WWT at ONUG AI Networking Summit—Dallas 2025
The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions.
Learn more about WWT's AI Proving Ground.
The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.
Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments.
From Worldwide Technology. This is the AI Proving Ground podcast Today AI agents what they are, how to use them and where the value comes from. Ai agents are being hyped as the brain of the cognitive enterprise, but our guests, data scientists Ina Posher and Jason Campagna, a senior director of WWT's AI practice, say the reality is even louder A deafening onslaught of buzzword bingo that leaders must cut through to find real value. Jason and Ina will explore how focused, task-specific agents can unblock data problems. Why start small is the mantra for every first deployment, and what trust but verified guardrails look like when agents are making decisions on our behalf. By the end of this episode, you should be able to identify which early win use cases to chase, how to measure them and the one question to ask before letting any digital teammate touch production.
Speaker 2:Let's get to it, ina welcome back to the Ad Proving Girl podcast. How's it going? It's great to be here. I'm doing great today.
Speaker 1:Really excited to continue chatting with you and with our other guests that keep coming in. For sure, jason, first timer, welcome to the show.
Speaker 3:Thanks for having me. I love being here, and especially with our recurring co-star here.
Speaker 1:Absolutely Well, today we are talking about AI agents and it's interesting. Ai agents feels like a term that simultaneously is like at the top of the hype cycle but very much can deliver on the promise of driving transformational results within the enterprise. You know the World Economic Forum, just you know, reading right here, recently called AI agents the brain of the cognitive enterprise. Jason, just to start us off, we'll get into some context around. You know what agents are, but you know the brain of the cognitive enterprise is that hype, is it? You know?
Speaker 3:But, you know, the brain of the cognitive enterprise. Is that hype? Is it? Is it, you know? Is the hybrid human agent organization a real thing? What are we seeing right now in terms of AI adoption with, as it relates to agents, we're all headed with an absolute crazy future for all of us. I think there's the dream that we all discuss and experience and so forth, and there's certainly some truth to that, but also the amount of agent washing, to use a buzzword, is definitely very, very aggressive in the market today. Any product, any SaaS provider, any ecosystem partner out there seems to be adding AI to the front or the back or the middle of their company name, product name or something similar, and it's just a deafening onslaught of buzzword bingo for our customers and, of course, for all of us to sort through to figure out what really does work, what really is going to provide the outcomes we're looking for related to agents and agentic architecture.
Speaker 1:Yeah, and Ina, maybe add on there. Filter out for us what you think you know, filter out the hype and what is the practical usage right now for AI agents, whether it's now or maybe the back half of 2025?
Speaker 2:Yeah, I mean, I think agents just takes it's an evolution of what we can do with AI and it really employs software development practices of calling a function, calling a specialized function that can do a job. So in this case, that function being an agent, that is very specialized and it's one thing, so you know it will do it right and it's not also in charge of 30 other tasks and it might get messed up in between.
Speaker 1:Yeah, well, you mentioned function calling here. So just for context, for listeners that may not be familiar with what AI agents are, jason, can you walk us through the evolution that we're seeing here, maybe from legacy AI systems to AI-powered chatbots, rag models and now onto AI agents? What is the difference between all these, or are they building on one another?
Speaker 3:Yeah, I think building on one another is the best way to look at it and an important starting place and the thing I don't hear often and we talk about it our AI days quite a bit is agents are not new. They have been around for literally decades. The question is, is what level of agency, what level of ability to make decisions, take that goal and, like Ina mentioned, take action in the real world? And for me, the idea of agentic or the idea of what we call an agent today, is really really hinged against that. I'm going to do something, I'm going to take action. Funny enough, I could also take action with a legacy, decade-old agent. It would be a deterministic workflow and would be more of a predictive AI model versus a generative AI model, and I think that kind of evolution we're seeing you know, you look at early chatbots versus AI assistance, versus this idea of agents. It's really the fundamental term there.
Speaker 3:For me is agency Like how do I actually think about or how do I really act independently and make decisions? You know, agency is this philosophical, psychological and social construct or concept that's usually discussed in terms of like, freedom and ability for people to shape their own lives and behaviors and environments the human condition, if you will. You know, and as we see agency evolve, or as we see AI models evolve to be able to do more and more on their own and we're able to trust the output we get from them, I think we see more and more advanced agents and more and more ability to actually take action against predetermined goals in a very deliberate way into the future of AI.
Speaker 1:Yeah, I think that's really what makes an agent an agent is its ability to have enterprise companies with 1000 employees or more are adopting agentic AI, which that number seemed high. But then another KPMG survey 33% of organizations are now deploying AI agents, which is, you know, a substantial increase from the previous year. I just throw those stats out there to show that it seems like adoption of AI agents is accelerating within their enterprise. So what changed over the last six to 12 months that enabled this?
Speaker 2:you know this curve up it is one massive piece, but also, I think really, what they're getting at here is leveraging these generative workflows where you're again having a little bit more agency within these agents, so it's allowing these tools to have a greater effect within the environment that they're operating. Within. A word, to use like what Jason said, they're less deterministic and more generative, and that is the change that we are really seeing. And the fact that generative AI has come and allowed there to be these stronger, larger changes in these environments is making companies see more value in agents themselves, even though they have been using them for a long time. These are AI agents and really taking on more of the generative side of AI rather than the deterministic models, machine learning models that we've seen for so long.
Speaker 3:It's almost like the word became cool again. You know, you look at the old school predictive, rules-based models. Well, I could know what the answer was going to be. In a much more deliberate way, generative AI came along and suddenly it was a lot harder to get those results. I actually think agentic and the buzz around agents is really about the ROI and the outcome that we get from AI. You know, look at an enterprise trying to deploy generative AI. That's pretty hard and it's only so much value in production of different things, whether it's images or text or whatever. You know, generative AI basics versus your ability to act.
Speaker 1:You know I do want to go back to. I want to keep going with the ROI conversation here in a second Jason. But, ina, you said what changed was verbiage. Are you just talking about Gen AI's ability to understand and talk in plain language, or what do you mean by verbiage? Is that a data thing?
Speaker 2:No, the humans using the word agent to describe what they're doing. I had a conversation with a coworker about this probably six months ago and he was asking well, what are agents? How are they actually like? What is it? It seems like this is something that I've done before and I was like it is exactly something you've done before. You've been using this for a very long time. This is just the newest word to describe with what you're doing and how you're going about doing it. Okay, and then it's a very technical guy. I just explained it in software engineering terms of what was going on and he's like well, why is it so fancy then? It's already existed for so long. Why are people so excited? Well, it's a new word.
Speaker 1:It's a word to describe how we can leverage AI to do these things for us and, as Jason was saying, see ROI for that agentic structure in months or years past that we just took for granted. Maybe that it's an AI agent or like an agent working on our behalf.
Speaker 3:Well, I think that's where we do get into some differences in where the future is. While agents had been around for a long time, agentic is really a newer term and you know that's more of an architectural term and actually to me, and I think certainly others out there, agentic has become more of a transformational term for me, much less the architectural word. It's like well, how am I evolving with AI, especially with generative AI, and not only but particularly LLMs and remember there's different models in LLMs and I do think that that agentic transformation, that idea of how am I really going to capture this marketplace hype that I hear about with AI and we hear from Jensen and others on the NVIDIA side, you know that wave, that evolution to a GenTech and then eventually to physical AI I do think it really represents that the tangible transformation in large organizations and small towards useful application of AI and what they're trying to get done.
Speaker 1:Yeah, well, you know sticking with you, jason here. Where are we seeing the best use cases for AI agents, whether it's right now or kind of what's on the immediate horizon? Where are we going to see the biggest ROI, the biggest bang for the buck, on an AI agent use case?
Speaker 3:I think the short term we are definitely seeing a drift towards more fundamental or foundational use cases that feel a lot more like my past around automation. I'm an automation individual at heart. I kind of live and breathe words like toil. And how do I automate things out of my life, my work's life, you know, customers' lives that will allow them to free up time to do more valuable work. To do more valuable work, and so I think we're seeing quite a bit around like AI incident response or agentic incident response to AIOps, itops.
Speaker 3:Days of the past that was really hard. We had to manually write a Python script or Ansible playbook or something similar, whereas now we can get 90% of the way there with a coding assistant. Oh hey, by the way, I can take all of the scripts that I've written and bring them into an archive and speak to that archive and build the next ones faster. There are things that we can do with agents in today's world that are very automation-centric. I think the other big pattern we're seeing is that foundational AI assistant, that knowledge management assistant, seeing the idea of our Atom, our Adam AI chatbot. Many others have a branded kind of front end that all the employees that digitized knowledge may rely on the output of you know, so to speak, the backend of an organization to really deliver next generation customer experiences, next generation employee experiences, and then, of course, a whole variety of industry vertical specific things, depending upon you know where our customer is coming from.
Speaker 1:Yeah, well, you know, as a data scientist, you kind of sit on the side of like what is actually possible right now You're looking at the data. So, based on what Jason said and based on your experience with some of the client organizations that we interact with, where does the market sit in terms of being able to actually deploy some of this technology and use it right now to drive ROI?
Speaker 2:Yeah, so I mean, the reason we're seeing so many chatbots pop up and similar to our Atom is that your data has to get ready. If you're wanting to do these types of use cases, if you want to have really strong agentic AI that does meaningful work and does it correctly, your data needs to be in the right order. And a really good way to test out to see if your data is ready for that is to implement a RAG-based model or some sort of chatbot model so that it can access your data and give you answers from across an organization. So we do this for quite a few customers because that's where they want to start. It's really hard to quantify the ROI on that, typically, because it would involve me logging the amount of time it took me to email a co-worker to get an answer versus asking our internal chatbot Adam, and that's that's not really an easy quantifiable number.
Speaker 2:But you can also just not create agents for the sake of having agents if your data is not ready for it. So I think that's why we're seeing so many of these chatbots and now that it's been six to six to twelve months where everybody had that hype cycle and now you see almost every website has a chatbot for either customers or for internal folks to use. Now, that's why you're hearing the agent buzzword pop up and come up so often is because people are more ready for it. They can start implementing an agent at a time rather than getting their data in order. So they know like, ok, maybe our, our sales data is the cleanest and most ready. Let's build an agent for that, rather than let's build 12 agents for every single one of our data sources. They'll start targeting the ones that are the most primed and ready to go primed and ready to go.
Speaker 3:Yeah, yeah, I've heard you mention at our AI days the. You know the conversation there of working through data and kind of learning how all these pieces fit together and work. It's it's it's not easy and starting is more important than perfect and it's bringing to evolve that data in some deliberate way. It's almost like we finally have a pressure point of sorts to really solve or work that data problem that that didn't ever exist before.
Speaker 2:Exactly, and people will complain when internal chatbots don't work well. People don't respond well. Either they don't use the tool or they complain about the tool, and those are really good driving factors and really help data scientists understand where improvements need to be made. And that's how you get that iterative improvement with our internal chatbot. Again, we iterated upon it so many times. It needed improvements because the first time we went about deploying it, we got a lot of feedback that there was an improvement in the responses, in the data, in hallucinations, and that allows you then to feel the pressure as a scientist of where there need to be improvements and where our users actually care to see those improvements. And with a lot of traditional models, you don't always get that feedback so instantly. People are very, very open to giving very strong feedback when it comes to chatbots.
Speaker 3:Yeah, I know, I certainly have. I feel like in those early Adam days where we we did, we went after the page I think many customers start with which is okay, I'm going to put this AI assistant out there, and it's going to be the all's end, all be all for all things. And then, funny enough, and this is the very early days even a Vigintic, which is kind of funny because it's like last year. That doesn't feel like the feel like a year.
Speaker 3:But we quickly started to kind of cut up the data problem into. Well, like you mentioned, for Salesforce data or for sales data. All right, well, let's go have an agent that speaks over there and we just brought online internally that capabilities agent that can speak to our sales team and what we can do. And oh, by the way, it goes over here and talks to platform data and co-pilot information, talks to platform data and co-pilot information. That focus, if you will, into those agents and making sure their purpose built and can understand that data seems to be a really key precipice or control point or what have you. And building agentic architectures at scale with success.
Speaker 2:And we had to do it because if we wouldn't have started, even with our data not being necessarily in the best shape, we wouldn't have been able to upskill ourselves, we wouldn't have been able to learn about where those improvements needed to be made and how to go about making them. So we took that jump and we definitely had to iterate because of it. We had to learn, but it improved the skills of our internal developers and scientists, which then allows us to help our customers at the end of the day, not make those mistakes, and get a little bit faster time to value and time to seeing where the data can be manipulated, massaged to be the best that it can be.
Speaker 1:Well, I'm glad the two of you have mentioned Adam a number of times. Just for those out there listening or watching, adam AI is WWT's AI-powered chatbot. It does have AI agents working on its behalf. In fact, you could go out right now this is now and say how should my organization be using AI agents, and it would return a load of good information as well as sources that you could go and discover on WWTcom. Jason, I'm hoping you can describe for me how, like who's building I mentioned, adam AI has agentic features. Who's building those agents and who's determining which agents we go after first? What? How does that prioritization work?
Speaker 3:Well, we certainly have an internal team that is a cross-organizational group, but also a core development team working on Adam. I do think it's based on need as well as readiness of the data to a certain degree, and I think there's a lot of back and forth. I know, at least in our internal capability heatmats our talk show do, as we call it, internally in on what does the data need to look like to be properly ingested? Similar with that Salesforce style agent or even Copilot style agent, and to say that we have it all perfect and figured out would really, I think, undermine the value we have in walking together with our customers on this journey. That is agentic. This is a multi-month, multi-year approach that needs to be iterated on.
Speaker 3:I think AI is really quite different. The mental shift of the output's not black or white, it's gray and it's shades of gray that you're going to get. You got to think about it differently. You've got to build in feedback loops very assertively into the architecture and also realize that there simply are different architectures depending upon what you've got to do. I think it's a major element of why agentic is so confusing.
Speaker 3:I think it's something that actually, in talking with Ina in particular, I had pulled into our AI to keynote around. This is a sample architecture, this is a way of approaching things. As a part of that, I think we've also adopted that with Adam, where, okay, for this piece we're going to do it like this and for this other piece, we're going to do a little differently because of the needs of the data, because of the needs of the platform it's interacting with. You know, the API call or depending upon if it depended upon, the use case and NCP server. That's of course we're speaking to. But I think how that evolves is really critical and as we add or snowball if you will, more and more use cases, in that it's still under the banner of Atom, but Atom gets smarter and smarter as it has friends that are different agents that are highly tuned for the particular need they have, and it can pull all that together into a response for the end user and drive that experience for the end user and drive that experience.
Speaker 2:Yeah, I think Adam, you can think about as the Adam is the orchestrator and the agents are the ones actually doing the action on the environment, if we're talking about that. So the agents do the actions, they bring back the information to Adam and then Adam takes all the information that it got from its friends and or agents and then puts that into a very nice little blurb of text for you, the user, to understand what happened in the background. So adam is the orchestrator, the leader, the, the delegator, um, in this whole system, but we do refer to the conglomerate as adam ai well, that's a great analogy.
Speaker 1:What about the concept of like a citizen citizen developer who's able to perhaps make their own agents? I'm thinking that that would just create kind of chaos here. Are we going to see an explosion of agents from citizen developers, or am I just out in left field there?
Speaker 2:You're on, you're pretty on track there. Vibe coding leads to a lot of agents that do a lot of things and they might not do it very securely but and they might also not do anything, but they, they are being developed. Uh, while a little chaotic, that is a good way for people to learn. They have the opportunity to do so, and those agents don't have to be made public to every to everybody. Not everybody has to have that sent out to their instance of Atom. I think we had like seven or eight agents that everybody has access to within Worldwide. And then there are agents that other people have developed for their own use cases, their own applications, but it is not available. Say for myself, I don't have access to that because that is not something that was deemed necessary for the entirety of the worldwide staff.
Speaker 1:Yeah, Well, Jason, I think it's a fast yeah, go ahead.
Speaker 3:Yeah, I think it's a fascinating dynamic because you know for the for the first time and I think there's kind of a running joke, at least in around here is you know what's the most popular coding language going to be in five years? And it's it's your question, because it's English, at least many of us believe it's English that idea of vibe coding or citizen developer. There's a whole lot more I can do that I'm not a coder by nature, but my 10 year old and I built a Tetris, the jeweled mashup or candy crush mashup as a game, or, and, and that that innate ability to do some really interesting things organically without knowing coding in a team or is a small subcomponent of an enterprise, I suddenly can get a lot of stuff done. You know copilot or copilot studios are great example of that.
Speaker 3:You know, I think we're seeing some patterns here for low code, no code, a lot of different platforms that are out there that different teams are using. That is sitting right next to well, wait a minute, I could go buy an agent to do this. Or hey, wait a minute. This SAS or the SAS application I have in my enterprise is now coming towards me and saying, hey, we've got several agents or an agent platform of some kind, or you can buy this agent and there's this weird crossbreed between those and you go a little further and suddenly there's more of a developer slant to that platform. Think like to stay on the Microsoft bandwagon there. Azure AI Foundry is a more development oriented feeding ground care and feeding platform for agents and for AI models in general that I can develop from the ground up.
Speaker 3:I think there's a redevelopment of software or rehash of software here. In the broader sense. It's really interesting to me with a cloud background, microservices architectures came along for the ride and now suddenly it's like well, let's swap out one of the microservices with an agent. We still have APIs all over the place. That's not necessarily going away, but there are certain things we built a lot of code around that suddenly I need I have a lot of value in training, evolving, put a feedback loop in place, provide a goal for an agent or an agent architecture that really enhances what that application is able to do for the enterprise.
Speaker 4:This episode is supported by Island. What if the enterprise had complete control over the browser? Island provides a single space where everything organizations need to work safely and productively is built in Island the enterprise browser.
Speaker 1:Yeah. So I mean, as more agents enter the fray, whether they're kind of official enterprise initiatives or they're coming from citizen coders or vibe coders and things of that nature how, from an architectural standpoint, jason, how would enterprises create a platform that can handle all those intelligent agents, like at scale? What needs to be considered there?
Speaker 3:I think it's actually multiple platforms. It reminds me a lot of Enos said it perfectly there's an orchestrator that sits at the top of this that you know. There's like quote, unquote the main agent or the main interface for a lot of people that hides things that are underneath, but that doesn't mean it doesn't need to be built. It also doesn't mean that there aren't sub components of, like, a platform that's over here that's managing some parts, some of those agents, and there's another one over here that is doing that, depending upon it's a purpose and fit for purpose, if you will.
Speaker 3:I think this is the challenge that many of our organizations have, and it reminds me a lot of the early automation cloud days, where I want to have infrastructure as a service. I want to have something, something as a service that evolved into function and platform and other layers, if you will, of cloud service capabilities. Well, now you have GPU as a service, and there's even jokes about AI as a service. Well, what does that even mean practically these days, because there's the definitions all over the place. The net, though, is I'm now going to have hundreds thousands of agents, some of which are very small and tight and fit for purpose, which, by the way, may not even be a generative model. It may be a smaller model of some kind. It may be boring old machine learning of some kind that is doing something highly, highly specific.
Speaker 3:Is this an image of a cat or a dog, as the joke has gone for decades at this point? But it's part of a larger structure that is aimed at an output and aimed at an outcome that the organization's after, which may be as high level as employee experience and saving a ton of time not finding things. You know. That kind of thing I think you know. You know. I'm curious to your, your thoughts here on, like you know, how do, how do we get a handle on that? Or is it just the sprawl that we're all dealing with that is going to evolve and there's going to be a lot of plays.
Speaker 2:Yeah well, I mean the biggest problem or the biggest hurdle to overcome is going to be making it repeatable and easy to integrate any agent into any system, meaning there needs to be a little bit of standardization in what uh is expected as an input to a model, what's expected as an output to a model.
Speaker 2:So I mean we've heard now I think we released an episode on mcp, uh and different protocols like that. So what that relates to for traditional networking folks, it's your HTTPS or TCP protocols, it's what is expected when you have an incoming message or an incoming request and what is expected for your output, and how are you going to communicate that. So when you have all these agents and, as Jason said, it starts to sprawl pretty quickly when it comes to these different tasks you need them all to be standardized in how they accept information and how they respond. And once you have that in place and there's an agreed upon methodology to do so, then you can have that sprawl and know that the agents will respond in that standardized way and you can use that information and you don't have to change something every single time for each specific use case. Instead, you just keep using the same framework every time, no need to rebuild the wheel and you save yourself a lot of time. And then it becomes more.
Speaker 1:It becomes less daunting to have such a or to create such a scalable architecture if we have this influx of agents, let's just, um, you know, let's think, as if you know, we've, we've considered mcp and agent to agent, which you know, you're correct, we did have an episode um, there's probably a couple episodes prior to this one touching on that matter. But if we have this fleet of autonomous AI agents working kind of behind the scenes, how do we as humans, or as the business, know that they're working to drive everything forward? Like, what level of trust do we give agents right out of the gate, versus how much human in the loop do we need to have? How do we know it's going to be working for us in a positive way and we're not just taking it for granted?
Speaker 2:A lot of checks and balances. To begin with, I think trust has to be gained. People like my older relatives are definitely more apprehensive to the idea of letting an orchestrator orchestrate a bunch of agents to go do tasks. If I could tell my grandma that, she would, I think, lose her mind. So you're going to need to build that trust up. Show the thought process of a lot of models, which there are quite a few of the very basic large-enders models are doing that now explaining how they come to their reasoning or explaining their reasoning, and in doing so you start to build that trust.
Speaker 2:And when you have agents that do more specific tasks and they do those tasks correctly, that trust is built and then you can have more agents that work together or call upon sub-agents. But it's iterative, as is everything in AI. It's iterative and it needs to evolve, and as does the trust in that technology. But, jason, I'm very curious on your thoughts on this as well.
Speaker 3:Did trust but verify. As the saying goes, people respect what you inspect, including your guy agents. You know, I feel like that built in feedback loop, like you have to design an agentic architecture from day one with a very clear feedback loop. The other thing, you know, obviously that's a, that's you know. They all seen I as is like a joke, it's not that one slide I always use. I feel like there's also another component here that is is often kind of skipped over in because of all the rhetoric and the hype in the marketplace, which is lots of agents is a good thing, which sounds counterintuitive because it would create sprawl. But the focus of those agents and the ability to tune them and and you know I joked about the cat versus dog thing, whatever it that is a sub component of what the agentic architecture needs to do. Cutting that up makes things easier, not harder. It's very, very, very difficult to deal with one big, giant trillion parameter model and have it do everything, and I think that in conversation I've heard that from from customers at times because they heard from a particular vendor like I got the solution here, what's what you're going to use and it's going to be awesome, which not to undermine. That may be true to a certain degree, but it doesn't necessarily work when you get into broad scale and the ability to make sure that that output is reliable and highly trustworthy. And so I think we see a drift towards that idea of like, well, what about a small language model here? And what about a different model over here? And what about training the models and the agents to begin with to raise their hand and say, hey, I need a human, I need a human to look at this thing? I'm not really sure. And here's why, and describe the rationale, that idea of human in a loop.
Speaker 3:I believe that the agentic architecture is innately with a human in the loop. It starts with a here's a human, here's a human, here's a human, here's a human. And how are we taking the non-valuable tasks that are required from the organization? Let's put an agent over here for that, an agent over here for that, and suddenly, before you know it, that's more of an integrated approach, which I think also reflects the idea of a human agent versus an AI agent and the crazy hype rhetoric out there around is AI going to replace all of our jobs? And the general saying of, well, no, not really, because, if I am able to use AI as a human, I can be much more productive and it's really more of an accelerant to those that know how to use it.
Speaker 1:Yeah, jason, I love the trust, but verify, as a former journalist. That reminds me of the common saying if your mother says she loves you, go and check it out. So I'm happy that that just rang in my head there, based on what you said. But I'll give you a scenario here. What have agents running around on our behalf? What if an agent starts negotiating with a vendor or a supplier late at night or something, and it's unbeknownst to anybody, and they make a decision on our behalf and it's not the right decision, whether it's a hallucination or data poisoning or something that caused it to make the wrong decision? Who owns that in the value chain? Is it an executive sponsor? Is it the developer of that agent? Who has responsibility here?
Speaker 3:I don't know yet.
Speaker 2:AI of security, Data scientists working with those who actually secure the environment in which the models are living in. There needs to be cross-functionality and right now we're getting better at it. In the past there has been very little, and data scientists aren't known for being necessarily the best communicators. I've been working on it really hard myself Cyber security specialists not known for being the best communicators, so you try and put those two groups in one room and have them work together. It can be difficult and getting that cross functionality is so important so that everybody is aligned and we're designing AI from the beginning, or AI models, models in general and the systems in which they live in a secure fashion, so that there is accountability and you know where these decisions were made, how they were made, why they were made. It really does come down to the security and the policies that you have surrounding your AI practice.
Speaker 1:Yeah, I mean, we've talked about you know years past. You know least privilege, zero trust, you know, so on and so forth. Access control are the before. We were talking about humans, but are we applying those to agents now too?
Speaker 3:yeah, I, I would say so, and and, and, uh, you know I'm gonna steal your line. I think I've done this a couple of times now with uh, with conversations like a model is only as valuable as the decisions that it helps drive something to that effect. I, I think that's that's where okay, well, trust, but verify comes in and as a part of the deployment of an agentic architecture, especially out of the gate, you're not gonna let it just go out of the gate. You're not going to let it just go out and make decisions like that. You're going to confirm, just like you would with a new employee you bring someone new into the organization, a new agent in the organization, and you're training it with its surroundings, its context, its world, what the output needs to be, what that decision needs to look like and oh hey, why did you make that decision?
Speaker 3:Looking at that reasoning, like Ina mentioned before, I do think that's a key part of the deployment methodology and process. You're not going to just unleash the agent out of the gate. It doesn't really start like that and, especially if it's critical impact automation, like an incident response, there are certain actions that it's going to be more of a hey, I saw this, I saw this. It pulls that needle out of the haystack that it needs to act on and says here's what I believe the action is and that is shuttled as a package to a human. That then actually hits go and while the agent may hit an automation framework and, of course, actually take that action, the confirmation of that decision and making sure the model's decision, the agent's decision, is as valuable or as on point as we want to be, I think, is tangibly how that really looks in the real world yeah, and also to add on to that, we're not giving every agent access to every single data point within an organization zero trust.
Speaker 2:yes, the agent will have access to the data that it within an organization Zero trust. Yes, the agent will have access to the data that it needs to make the decisions that it is in charge of. A Salesforce agent will have nothing to do with HR data and should never have anything to do with HR data. So they will get access to the data that is required for their task and not more.
Speaker 3:Yeah, goes to the tuning of those agents and it also, you know, perhaps also relates to the. You know the whole concept of guardrails. What guardrails are put in place for all of the agents or specific agents like the HR agent, may be highly restricted to your point, whereas another one, that's I'm trying to see everything in Office 365. This is a fun one. We hear about all the time with customers because they barely know what's going on over there half the time. It's quite a challenge, so adopting the access controls of that other platform and need to trust that that works, and then, of course, you got to go fix that if that's a problem as a part of the agent's response. You know that idea of guardrails, I think, is a fundamental thing and there's a whole lot of platform related. You know interactions there with NVIDIA and others to ensure those guardrails are in place with these agents.
Speaker 1:Yeah, it's interesting. We're talking about a certain level of human responsibilities here. As we start to integrate and adopt more AI agents, is there a re-skilling that needs to happen for you know just call it the organization in general to be able to mold into like a supervisor of multiple or dozens of agents? Very, very much so.
Speaker 3:Yes, I need to learn. I actually think it starts and there's a discipline forming here around context engineering prompt engineering, of course, is the word that's been around for a long time. It starts with simply talking to AI. You know, I love your reference to you know, like friends and family kind of a thing. It's like I was at my stepfather's 70th birthday on Friday and just got buried with AI related questions and so forth from all friends and so forth, and there is a conversation of listen, just speaking to AI as a starting place and encouraging everyone out there listen to this.
Speaker 3:And, as we do with our customers, it really does start with simply talking to AI.
Speaker 3:And, funny enough, though, when you're doing that, you're learning how to prompt. You're learning how to provide that AI model with context, a persona, an understanding of the output you're looking for. The specificity necessary there at times is more than I think folks realize. You know you often get that context in a human to human conversation and you ask questions and you go back and forth, maybe similar to how we're doing in this podcast. You've got to load that all in to the AI to begin with and think through that response, or tell the AI to ask you questions. To begin with, that's a skill set that evolves into system prompting versus user prompting pretty quickly, when I need to provide the agent with a goal and an understanding of what its output needs to be, and the specificity and focus of that becomes really critical. And you look at that maturity of how to work with AI, that adoption curve, I think, is a really significant challenge for our customers and, I think, something that we've really got to focus on. I mean, ina, what have you been seeing?
Speaker 2:Yeah, this really resonates with, I don't know, probably three or four years ago, when we were using a lot of reinforcement learning algorithms. When we still use them. They're just used a little bit differently now. But getting the engineers that are using your tool and who are supposed to provide that feedback was really hard. They didn't necessarily want to take that extra step to provide feedback to the algorithm as that human in the loop to help improve it, and that's just no longer a choice.
Speaker 2:If you want these generative models, these AI models, to work with you and work with you well, you need to provide that feedback and inherently you do it when you ask it. To rephrase an email, maybe a little bit nicer and less sassy. Or if you say well, please write this from the perspective of a data scientist, I'm writing to a project manager, use language that they will understand. So you're inherently providing that feedback with how you're talking to these models in natural English, but still providing the feedback like within. Again, adam, our tool we have a thumbs up and a thumbs down can really help those on the back end who are developing the model, while your feedback to the model itself helps the model's internal understanding of the context in which it's operating, the actual feedback of thumbs up, thumbs down. Why was this not helpful? Why was this helpful? Did it hallucinate? It's super helpful for the scientists on the back end that are trying to improve the architecture of the model itself.
Speaker 3:You mentioned data before or earlier.
Speaker 3:One of the more interesting things that I think we keep running into is the interaction between the data and the data quality or readiness and the prompt itself.
Speaker 3:It's like at times it doesn't give you the right response because there wasn't enough context in the prompt or enough understanding in the prompt, even though the data might have been there. And there's this, like to your point on like adoption with enterprises, like it's actually a little bit of both. You gotta get the data right, but you also have to be able to speak to AI or better, you may have to train one agent to talk to the other agents correctly, to pull that information and tune that accordingly. There's such an interesting interaction where the data may actually be okay but your prompting system prompting user prompting may not be where it needs to be, and perhaps it even relates to the sharing of oh, I built this agent. Well, yes, but you built a bunch of context into it and you share it with your team. That helped that happen so they could ask it very simplified questions and it figures that out on its behalf. It's a really crazy interaction of transformation for me.
Speaker 2:Yeah, I think we'll see a lot more of that, as the humans using these models are able to upskill themselves and their prompts become better, and therefore we're able to have more complex interactions between models themselves.
Speaker 1:Yeah, it's almost as if you know you talk about businesses succeeding or failing based on culture, because it enables collaboration and so on and so forth. It's almost like you need an HR program just to facilitate that collaboration and how agents talk to one another. Or am I already granting AI too much humanity?
Speaker 3:We're seeing folks in the marketplace head that direction. There are there's there's a couple out there that have actually merged IT and HR for this very reason.
Speaker 1:Yeah, well, a little bit forward facing. You know, what are we going to see? Maybe not over the next six months, but, like you know, 12 to 18 months as we get into like 2026 or even the back half of that year. Are we going to see a situation like that common situation where it's one person is controlling dozens of agents and they've come up with a billion dollar company and they're running it with one employee? Or how are we going to start to see agents mature and be utilized within the enterprise?
Speaker 2:Okay, I'm going to go first.
Speaker 3:I'm leaving the time there.
Speaker 2:There for you so I I think, as jason mentioned, right now we're seeing a lot of automation, the automating of tasks that are easily repeatable and aren't necessarily so complex that they need a human there to do it. And where we're going in my mind is to where you're able to say something to a model like I would love to go to Ontario for the weekend, please plan my trip for me. And that general of a statement revolt or then kicks off this process of the model saying okay, my, my prompt is to book a trip to ontario. For this I need to get um transportation. I most likely know, based on my content or context, where the user is living. So, all right, I know also from past requests they like to fly, they don't like trains, so I'm going to book a flight for this upcoming weekend to Ontario. Okay, great, maybe.
Speaker 2:My follow-up question then is all right, what type of lodging are you looking for? So then you get a question back from the model what's your lodging? Hotel, airbnb, are you looking for a villa? I don't know. Those types of questions you answer provide a little bit more feedback. Great Model goes out, finds a couple options within different budgets, comes back to you, gives you what you're looking for Just says hey, here are three options. Which one would you like me to book? This one, great. It goes out and books it and then has all those pieces ready for final approval and then you get to have it all done. So, rather than myself going to Airbnbcom, going to Delta and booking all these things, it goes through and does that. That is my ideal next date in the next 18 months, so that my booking of travel becomes significantly easier, significantly easier.
Speaker 3:But that type of transformation where we go from automation to really just giving a general prompt and then working together to solve it no-transcript learning of the concepts and topics we're talking about here, because to do it at scale, you, you, you got to dig in and you got to realize the transformation, the fact that it's a marathon not a sprint. Uh, and, and like I think you know you hinted on, like the idea of the model understanding when it doesn't have enough context. I think we're going to start to see that become more and more aggressive or pervasive or whatever you'd like to use. And, and you know, context engineering is a particular uh passion of mine lately in terms of learning. It's pretty new. It's like how do you engineer the context across agents? How do you pull the context across agents? How do you pull in my personal context into an enterprise agent where you may not have access to that? I speak to chat GPT on personal subjects, have projects built which effectively act like agents at times, especially with tool usage, and it's like well, there's a bunch of context there for my travel plans If I go into the corporate side that none of it doesn't pick that up, and so like, how does that start to integrate?
Speaker 3:How do we do more of that? That, for me, unfolds this, this crazy ecosystem of different platforms, different agents, different layers of platforms low code, no code, the citizen developer thing versus developer. Or I've always been really passionate about those orchestrator or orchestrator style frameworks or platforms and then all the parts that they manage underneath them in the different ecosystems. That evolution towards an ecosystem of agents and ecosystem of platforms and me coming in as a user and having my personal and corporate context in line with that, as I ask it questions and then it saying, hey, I don't have enough context in order to do what you're asking me, here's what I need. Or it goes and asks you know, based on the initial ask, it goes and asks other agents and then comes back or what have you? Or even multiple LLMs, the same question and then rolls that up into a response to get real crazy for a second, rolls that up into a response to get real crazy for a second.
Speaker 3:That idea of synthesizing across multiple LLMs to get a better decision or a better low of agency, I do think is the key feature we're going to see in agentic architectures, including across platforms, to be super crazy about it. And it heads real fast to hey, that's all cool, what do I get out of it? If I'm a business, how do I drive productivity, that the employee experience. How do I evolve that in such a way that becomes highly impactful so that those employees can do more, creative, more? Call it higher value work for end employee or partner related experience for that organizations and business and it gets into fantasy land fast. I think anything beyond the 18-month window is stretching with what it really is or where it really goes. But concrete usage today, like we're seeing in many of our customers and many projects we've been involved with, I think, continue to evolve in some really magical ways. I'll use that word to get all buzzed worried about it.
Speaker 1:Yeah, that's a great look forward on how we could use AI agents or how we should be thinking about them. Uh, real quick, I know we are running up on time here. Uh, jason and Ina, both of you. I'll start with you, jason. What are you know? Maybe one or two things. Um, you know organizational leaders need to do today to work towards that fantasy land tomorrow. Land, uh, concept could be technical, could be change, management could be across the board.
Speaker 3:Step away from perfect, focus on a highly iterative, feedback-driven approach. You know, thinking about data models and platforms, infrastructure and especially foundational security. Figuring out those things against a use case or something that's impactful for the business is definitely our practical approach. In a nutshell, and realizing you're not going to get it right, fail fast, fail a lot. Feel free, because the value we've seen, we've seen customers see, comes from that learning that you really can only experience by doing. You can't go read 4,000 ways to build an agent. You got to go and actually tactically do it. In my opinion, you got to go and actually tactically do it, in my opinion.
Speaker 2:Yeah, you do. I would add to that start small. You don't need to do everything in one go. It won't work. So start small and start, and then get everybody's buy-in and make sure that your teams are working with one another. Again, speaking to your data scientists and security engineers and developers, they all need to work together to make this the most successful version of itself. So really push for them to work together and overcome those barriers that might exist at this point in time.
Speaker 1:Yeah, excellent insights. Well, to the two of you, thank you so much for taking time out of your day to talk about an issue that I know is at the top of a lot of leaders' mind these days AI agents. Ina, I'm sure we'll see you soon. You seem to be popping up here every couple of weeks and, jason, you know, you absolutely nailed it here today, so hopefully we'll have you back on soon as well. So thanks again.
Speaker 3:Happy to have plenty of time. Thanks for having me.
Speaker 1:Thank you.
Speaker 2:Okay, thank you.
Speaker 1:Okay, that's a wrap. Three key takeaways stand out. First, start small, learn fast. Deploy one narrowly scoped agent to expose gaps and then iterate as quickly as possible. Pick a single data set, set up a retrieval augmented chatbot and schedule weekly feedback reviews. Second, architect for trust Agents should raise their hands when confidence drops, so you need to log every decision and live behind least privilege access controls. Remember Ina's mantra trust has to be gained and guardrails are non-negotiable. So embed human in the loop checkpoints. Use protocols like MCP and A2A for standardization and enforce zero trust credentials per agent. And finally, focus on measurable ROI. Early wins come from automating minutia things like expense approvals, incident response and sales knowledge. The goal isn't novelty here. It's freeing people for higher value work. Put those three pieces together and you'll move from pilot to production with confidence, turning agentic AI from buzzword to business outcome.
Speaker 1:If you liked this episode of the AI Proving Ground podcast, please consider leaving us a rating or review on whichever platform you're listening on, and you can always find more episodes and other great video content on WWTcom. This episode was co-produced by Nas Baker, cara Kuhn, mallory Schaffran and Ginny Van Berkham. Our audio and video engineer is John Knobloch. My name is Brian Felt. We'll see you next time.
