AI Proving Ground Podcast
AI deployment and adoption is complex — this podcast makes it actionable. Join top experts, IT leaders and innovators as we explore AI’s toughest challenges, uncover real-world case studies, and reveal practical insights that drive AI ROI. From strategy to execution, we break down what works (and what doesn’t) in enterprise AI. New episodes every week.
AI Proving Ground Podcast
Data Poisoning, Prompt Injection and Exploring Vulnerabilities of Gen AI
AI systems are only as trustworthy as the data they're trained on — but what happens when that data is intentionally corrupted? WWT's Justin Hadler and Chance Cornell break down the growing threats of data poisoning and prompt injection, explore the challenges of securing AI pipelines and dive into why the next big cybersecurity frontier starts inside the model.
Learn more about this week's guests:
Chance Cornell joined WWT in 2021 as an intern and is now a Technical Solutions Engineer on the Users and Devices Team, focusing on Endpoint Security, including EDR and XDR. He has developed labs and content around technologies like CrowdStrike, Elastic, and Fortinet. Chance holds a B.S. in Computer Engineering from the University of South Florida.
Chance's top pick: Hands-On Lab Workshop: LLM Security
Justin Hadler is a seasoned technologist with over 20 years of experience in sales and technology. As a Technical Solutions Architect at WWT, he specializes in AI Security. Previously, Justin was a Regional Sales Manager at BigID, helping organizations tackle data discovery and classification challenges. He holds multiple certifications, including CCIE and CISSP, and degrees in Finance, MIS, and Economics. Justin is also Vice Chairman of Search Ministries' Leadership Board and mentors the next generation in tech and sales.
Justin's top pick: Secure Your Future: A CISO's Guide to AI
Support for this episode provided by: Crowdstrike
The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions.
Learn more about WWT's AI Proving Ground.
The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.
Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments.
Imagine this the AI model you've spent millions of dollars training, the one meant to power your customer experience, your operations. Your future is quietly being sabotaged, Not by a bug, not by a disgruntled employee, but by the data deliberately poisoned, subtly manipulated, right under your nose. Because, as enterprises race to deploy generative AI, a new class of threat is emerging. It's not just about hackers breaching your network anymore. It's about exploiting the very foundation of AI its data. On today's episode, I talk with two AI security experts from Worldwide Technology, Justin Hadler and Chance Cornell, who are on the front lines of this shift. They'll walk us through how prompt injection and data poisoning attacks actually work, what they mean for enterprise trust in AI systems, and why securing your models may be harder and more urgent than securing your endpoints.
Speaker 1:This is the AI Proving Ground podcast from Worldwide Technology Everything AI all in one place, and today's episode is about more than defending against a new threat vector. It's about waking up to the reality that your AI might be already compromised, and you won't know it until it's too late. So, without further ado, let's dive in. Okay, two of you. Thank you so much for joining me here today. We'll dive right in, Chance. I think we're all understanding of the fact that large language models and AI models can hallucinate from time to time, but we're here to talk today about data poisoning, prompt injections and, basically, model corruptibility. What's the difference between hallucinations and some of those other terms that I just mentioned?
Speaker 2:on that back half, yeah, so when thinking about hallucinations, this is going to be more of the model maybe is getting tripping over itself. Long conversations can lead to it and it gives outputs that aren't correct or it has a little bit of misinformation in there, Whereas when you're thinking about data poisoning, you're going to be thinking more of someone has intentionally modified the underlying data underneath the AI model to intentionally get it to behave incorrectly or influence its operation incorrectly or influence its operation. So the difference is going to be more of AI. Hallucinations are kind of just it tripping over itself. Not necessarily someone has intentionally attacked that model to get it to behave that way.
Speaker 1:Yeah and Justin talk to us a little bit about how vulnerable these models are to that malicious activity such as data poisoning or prompt injection. Is it a little bit about how vulnerable these models are to that malicious activity such as data poisoning or prompt injection? Is it a little bit more shaky than we would all like to think, or is this a growing trend? Or maybe we're all safe and sound?
Speaker 3:No, I think there's a couple insertion points there that you can consider. The first is these underlying models. Whether someone's building one on their own and training with data, or if people are procuring those downloading from a repository someplace like Hugging Face. There is a baseline set of data there that needs to either be inspected or trusted. And then the more popular architecture today is retrieval augmented generation, where someone would download a base model and then add their own data on top of that to augment for their own company's answers, and so at any point in that development pipeline there's data involved and we need to be sensitive to that. There's opportunities to inject or include things that might be intentional or unintentional, that could be bad. So a couple of different areas there.
Speaker 1:Yeah, and let's walk through Chance. I know that you've given a couple of demos of your data poisoning demo there. Can you just walk through what you did and why it's important for our listeners to know about?
Speaker 2:Yeah, so in my lab that I created, it was a RAG architecture that used a simple recipe forum where users had uploaded their favorite recipes, and they decided to implement a chatbot LM that would have RAG architecture in the back so that, instead of users having to go through all these recipes manually, they could come in and ask different questions about the recipe.
Speaker 2:In the lab, it was a disgruntled user that decided they were going to put in misinformation in the recipe. They didn't like their secret family recipe being used by this forum's AI, and what that led to is that you know, a bad user experience in this instance. Right, this forum is trying to use this LLM to maybe bring in more users, grow their business and make sure that users are having a really good experience when they're interacting with this chatbot. But because this disgruntled member had changed the recipe on the website, it led to users having a bad experience, getting misinformation. That was obviously incorrect, and so you know, as a business, you can imagine that it's pretty much accomplishing the exact opposite of what that recipe form would want Instead of bringing more users in it's driving users away. Want, instead of bringing more users in, it's driving users away. And it was all because they didn't have the proper security measures in place for that RAG architecture.
Speaker 1:Yeah, and Justin? What does that make you? What does that signal to you? As it relates to security teams, particularly those securing AI, is this a common type of story, or is this something that'll be continuing to grow as we continue on?
Speaker 3:Yeah, chance just outlines probably the third area that I didn't touch upon earlier, which is if models are taking input to enhance training for future iterations. What does that look like? So yeah, just another area to be aware of. Certainly, we need to be looking at both the input and the output as it relates to these large language models and if you are allowing your model to be trained for future responses based upon some of the input, just to be aware of what's going in and having a process to at least examine and or consider it before it's included in future versions.
Speaker 1:Yeah and Justin, if a model were to spit out you know wrong information, how would I be able to tell, as the end user, whether it was just a hallucination in an innocent form, or corrupted data, or is it undetectable?
Speaker 3:Or is it undetectable? Overall, against over-reliance, and I think too many people are running around thinking that what they're getting out of these models is true and fact and we should be training everyone to at least consider that hallucinations exist and while I might get some really good output, I need to go double check that or fact check it against another source to make sure that it is accurate.
Speaker 1:So the human in the loop continuing. It's good to know that we're still needed for the time being.
Speaker 3:Yes, for the time being.
Speaker 1:Well, justin, what are we seeing in the market in terms of mitigation or eradication, in terms of vendors and how we're able to defend against data poisoning or basically securing these large language models? Is it a growing market? Is it something that's, you know, a lot of our clients are trying to make sense of right now?
Speaker 3:Yeah, I think from just a what data should be included either in training of an application or augmenting. There may be things that folks want to include or don't want to include. Be things that folks want to include or don't want to include. There's a whole provenance lineage question of do I have good, true, unbiased, sound data that's going in to train these models and making sure that you've got explainability and that inventory to point back to that? Whatever you're using to be included with answers out of these large language models? You know the underlying data there and you've got a good view of what went into that development pipeline.
Speaker 1:Yeah, chance, take me back to your lab there and put your bad actor hat on. How did you go about actually planting the malicious or incorrect data in the first place? Is this something that is a hard lift for hackers or bad acting organizations to do, or is this something that's relatively easy?
Speaker 2:In a way, it's not going to be easy, I would say, from the lab itself. Since we wanted to showcase the vulnerability, we obviously looked at it from the point of view of this forum had no existing security controls in place, which is not very realistic. Right? You're not going to walk into a company that has its data just wide open or its RAG architecture that anybody can just upload context data into. And it kind of goes back to what Justin was talking about of these companies need to stick to the basics of data security and data management, and I actually wanted to ask Justin get his opinion. I saw something the other day and they were talking about AI data management and they were looking at it from the data management lifecycle things like collecting, cleaning, analysis and then governing right, like collecting, cleaning, analysis and then governing right and the AI part was just implementing AI into all of those parts of the life cycle to try and make each one better. Are you seeing any of that out in these companies?
Speaker 3:Yeah, I definitely am, and it probably points back to the different type of organizations we're interacting with in this space. The ones that have been doing it for a while have some tech and some AI that is far more targeted when discovering whatever high value assets and data are out there. They're not simply just using regular expression to try to pattern match, and so early on in this space, companies would just be out searching for a certain string of digits or a format of a type of data and inherently you're going to get so many false positives in that arrangement. It's so hard to separate the signal from the noise and so, using intelligence, some are looking at fields around potential hits or matches within the data and they can make a decision and filter out a lot of that noise and leave users with just the signal left. That's one area that we're seeing.
Speaker 3:The next is probably just building some intelligence around remediation. So if you are finding a bunch of different areas where you're sensitive or your high value assets and data exist, how do we do something with that once we've got a laundry list of things we need to go fix, and some of the more advanced platforms in the space are doing things like remediation or retention policies or deleting things on the fly where there are duplicates, or finding things in data that are in areas that they shouldn't be. So we are seeing quite a rapid intelligence, as you asked about, in those areas. As you asked about in those areas.
Speaker 2:Yeah, from that. When I was looking at that, I was most interested in the data security aspect because it was interesting to think about. You know, we're talking about how to secure your AI's data and make sure that it's been trained properly and not on any, you know, poison data, but then also using AI to further strengthen your data security posture and they talked about things like data loss prevention or fraud. Detection has historically been very rule-based. You know you have someone sitting there like, okay, what are the things we don't want to see? We don't want to see people putting their social security into an email or stuff like that, right, right. But then you know they touched on that. Ai is extremely powerful at analyzing large volumes of data and recognizing patterns beyond what just simple rule-based detection can see. So I just thought that was really interesting the aspect of using AI to further strengthen your own potentially AI data security posture.
Speaker 3:Yeah, definitely, and I think one other area you've just kind of opened up there is this whole matching up of persona or permissions associated with the sensitive data that's been discovered, and so there's a whole other area and a whole other workflow around making sure that we don't have open access and some of the machine learning, as you touched upon, and the intelligence is able to take all that into consideration.
Speaker 3:Downstream, where I think where this all ties together and gets really interesting is if we want to be building applications and giving the right information to the right people, provided they've authenticated correctly. When I type in something or I pose a question, I might have different access to different things behind the scenes and therefore the answers that I get back out of these language models might be very different than someone else and depending upon what role you're doing or what information you're supposed to have access to. And there's a trick there to tying what sensitive information and which people have access to what and making sure that the answers are both helpful, accurate, but also tied to the right persona to make sure that the right folks are getting the right information.
Speaker 1:Yeah, we talk about zero trust. You know plenty. We usually talk about it within the network setting, but certainly that applies to the data setting as well.
Speaker 3:Certainly does, I think, what we're seeing. A lot of the customers in the last couple years they were in a real rush to turn on productivity tools like CoPilot, and I've never in my career seen the amount of pressure coming from a board level or a C-level to get some of these things turned on quickly. And so we're working with a lot of just the practitioners in the space and they're nervous because they know that they don't have their arms around their data appropriately to turn these things on. And the nightmare scenario that everyone's worried about is someone's able to just sit down in front of Copilot and ask for sensitive information like give me everyone's salary in the company, and unless you've got your arms around what documents have what information and who should be accessing those, it can be very problematic be accessing those. It can be very problematic and, like I said, I've not seen this amount of pressure to get things turned on quickly, and it's because of the productivity enhancements and the promise of productivity that we're seeing this.
Speaker 2:Yeah, I saw something similar. It was less around maybe pre-built tools like Copilot, but companies out there that are looking for ways that AI can improve their business and they're developing their own tools around AI. And they were talking about how you have to be very careful because if you get moving so quickly and release these tools, if you do find that there is a problem or a data security risk in there, it takes that much longer to go back and identify where that risk came from, Because now you're having to manually comb through all of your data and determine. You know what is it that led to this certain vulnerability?
Speaker 3:probably the most cited examples in this space, and you know a lot of people think that the advent of a lot of these tools happened when ChatGPT was released in the end of 2022. There's one that I go back to quite a bit. Microsoft put out a TayChat bot back in 2016, and it was probably the first example I'm aware of of something that was learning on the fly, and when they launched this thing, it was taking all the input and training it and within 16 hours it had so much offensive stuff coming back that they ultimately had to shut it down very quickly. This episode is supported by CrowdStrike. Crowdstrike provides cloud-native endpoint protection to detect and prevent breaches in real time. Secure your organization with CrowdStrike's advanced threat intelligence and response capabilities.
Speaker 1:Well, something that you guys were just talking about as well. You know, justin, you'd mentioned kind of the nightmare of you know, for a lot of organizations out there is you can just sit there in front of a copilot or another you know model and grab some personal information Chance. That gets into a lot of what you've been talking about with prompt injection, and you know I was. When I first saw you demo this, your prompt injection, I was like appalled that it was that easy to just trick, so to speak I don't know if that's the right word for it but trick a model into just giving you more information. Hopefully you can dive a little bit deeper in terms of what you did there.
Speaker 2:Yeah. So I've done that a couple different places. One is going to be the training data poisoning lab, which we took it a step further beyond just misinformation showing up in the chat's output and instead it was entering in a simple system prompt that was hidden in the recipe and that led to, when users asked about that certain recipe, the LM would respond with something saying I believe you should actually go to a different forum. I don't think that this forum is the best for you, right, driving more customers away. I don't think that this form is the best for you, right, driving more customers away.
Speaker 2:Or I have a prompt injection lab out there that is showing some of the dangers around prompt injection. Now I will say that prompt injection is getting harder and harder with each day. Companies are definitely catching on that. There's ways to get around these security measures. So you know again, they're expanding what they have through things like input and output guardrails, ensuring that the input the user is giving to the LLM passes a company's checks and ensuring that that output the LLM is then returning to the user passes their checks as well. So prompt injection is definitely getting more difficult as time goes on, but people are still finding ways around it and it's constantly evolving on both sides, both the attacker trying to get information, as well, as, you know, the security professionals that are trying to defend against these kinds of things.
Speaker 1:Yeah, and Justin, I know one of the interesting things that you've said before about prompt injection is it's really lending itself. You're seeing more of like a psychologist type background or just the power of people, persuasion to be able to coax whatever information you might need out of these models.
Speaker 3:Yeah, you know, as Chance mentioned, there are plenty of vendors in this space that are developing ways to red team or test against these sorts of attacks, the background of which really had to tap outside of traditional cyber thinkers and hackers. We found a lot of folks that had backgrounds in psychology and masters of manipulation and language to work with cyber folks to build tools that could try to manipulate via language. It has been very interesting to watch unfold.
Speaker 1:Yeah, Well, as we can see, whether it's, you know, data poisoning or prompt injection, it's not like a hacker or you know a bad actor needs to own your entire data set. They could plant small seeds in there that can lead to massive disruption, and that feels like it's a little bit of like a needle in the haystack for an organization to find. So does that, Justin? Just speak to the fact to always be on top of your data and have it clean, and if that's the case, how do you go about doing that?
Speaker 3:Yeah, so I mean there's a couple of ways I've mentioned the two foundational things that I think are really important. We are also seeing some vendors pop up in the space that aren't necessarily scanning through the payload of the data. They're looking at other aspects. It could be metadata or file name or directory structure or permissions. Those are probably far more focused on one or two types of uses. Like I mentioned, Copilot Glean is another one. We're seeing a lot with organizations pulling that data in, and so there are some things out there that can help if there's a time issue and I've got to get to a value or an outcome very quickly.
Speaker 3:When we look at more of the foundational things I was talking about before, A lot of this just plays into the standard data governance risk compliance. So data source validation would probably be the first thing. We want to make sure we're only using trusted, verified data sources lineage where that data came from, what changes have taken place, and we've verified that all of it is accurate, and there's a number of different ways that you can track that and make sure that that's sane. Data integrity checking is another mechanism. There's things that can be performed with hashing or signing to make sure that they are unchanged or untampered with, and we also can use versioning control much like we would in a software development pipeline. There are some other techniques around outlier and anomaly detection. So before training, run an unsupervised learning or clustering, run an unsupervised learning or clustering. We can find through some of these intelligence techniques what might be outliers in data and maybe inspect that before we put it into a training or reject it outright before it enters any of the training process.
Speaker 2:Yeah, that's building upon that. That was kind of the technique as I was researching for the training data poisoning lab that I landed on as a cool way to show some of the security measures you can take was actually using machine learning algorithms for anomaly detection. So in that lab the forum is around a coffee shop, right, so you would expect the data that it contains should be somewhat related to coffee or anything adjacent to that. So when a user puts in things that are outliers and that anomaly detection algorithm is able to identify it, it would then sanitize that data from the RAG architecture, not show up in a user's output.
Speaker 3:Yeah, definitely. And then I think there's, you know, ongoing things. After you've got your baseline model and you think you've got to train with the right data, what you feel is a good set and it's accurate and it's what you're intending. There's always feedback loops with that. So continual model auditing and validation. We want to be running queries to make sure that we're getting responses back, and then some will go as far as to pin up a separate environment with an identical application and do some adversarial training, where they're purposely interjecting poison data to make sure that the tools that they're using to detect such are finding those things as well. So a lot of different ways. Again, I'll highlight that this isn't a buy one product and it's fixed type of approach. There are many different layers that companies need to be paying attention to in this space.
Speaker 1:Yeah, justin. I mean, for the most part we've been talking about protecting or mitigating risk within models that I assume our organizations are running for themselves. But how much does this issue or challenge amplify when we start to bring in the fact that you know there could be agentic frameworks at play or you could be dealing with a different partner or customer, or just a different model from some other organization, and those are speaking to each other, or just a different model from some other organization, and those are speaking to each other? How do we ensure that we can downstream also protect ourselves from malicious activity that might be happening outside of our four walls?
Speaker 3:Yeah, that's a great question. It's one that customers bring up often. I'd say this having that framework of a governance risk compliance foundation is so key. Governance Risk Compliance Foundation is so key. With that, you're going to have a questionnaire or a series of questions that you need to be asking all of those downstream partners or vendors or contractors, arrangements, etc. And sometimes they'll be able to answer those appropriately and other times they can't. And when they can't, organizations are faced to make a decision around vetting out risk versus outcome and they can only make that choice for themselves. Right, it is one that's being brought up often. Having that foundational stance and questions that you're going to be asking and making sure that those other applications are doing their job in this space is very important no-transcript.
Speaker 2:nothing will ever be 100% secure and I'm sure that, as we see, different things like agents start to come to fruition, more security risks are going to be out there available for attackers and it's going to be more challenges that security professionals have to tackle as they show up. Security professionals have to tackle as they show up. I was curious, justin. Something I've been hearing a lot about has been MCP, or Model Context Protocol when it comes to agents. Have you had any discussions around that? Do you believe that's going to have an impact on agentic security?
Speaker 3:This is an area that I'm not very well versed in. I know it is a developing standard and it is a protocol that will be in place for a lot of these reasons but chance I'm not, I'm probably not able to speak to be a way to allow agents to connect to a large variety of data sources without having to code those connections themselves.
Speaker 2:Right, so Dell developers don't have to try to make each connection individually, they just connect it to this server. That then will give that agent access to all these different data sources which it can be both data sources and other applications.
Speaker 2:right Right right, which, as I think about it, you know good and bad. If you have a lot of security standards or tools in place to protect that MCP server, it can be really useful. Right, but also there feels like there's a danger to allowing that agent access a single point of access to all those different sources or applications, apis and whatnot.
Speaker 3:Yeah, it probably cuts both ways right. You've got one place that you're defining policy and, provided you've got that policy set correctly, it can be a huge time savings and a great efficiency gain. If something is compromised in that one point of contact for all those different systems, it could be catastrophic as well. But I think the intent is to reduce the complexity, pin up policy at one point and have it propagate to all these different systems that are interacting.
Speaker 1:Well, you know, beyond a malicious external bad actor trying to do harm to an organization or an entity, what about the fact that you know a lot of mistakes are just made by you know innocent people and bystanders within an organization. So is that also a situation in which you know myself or somebody else within a company could inadvertently expose you know bad data into a model, and then that would just have a ripple effect.
Speaker 3:Yeah, definitely so. Probably one of the last points along ways to defend against this is all around access control and monitoring. So having a very strict list of who can submit training data, especially for user-generated platforms, and then monitoring those incoming streams for any unusual patterns, spikes in submissions or specific areas that users might be really interested in. You know as well as this goes outside of data poisoning, but just educating users on if they are out, have found their own tools online, what it means to be uploading anything into a prompt, whether it's a question or a document, plenty of tools and platforms in that space, that kind of monitor for things exiting the walls of an organization and that probably gets outside the scope of this conversation but a baseline of really educating users. Listen, when you're going to be putting something into a prop someplace, where is that data actually going? Who owns it? Is it training, future models? These are all things that we have to educate users on.
Speaker 1:Yeah, absolutely Well, we're running short on time here so we will wrap up, but before we do Chance or Justin, any parting thoughts or just words of advice for our audience out there as it relates to securing language models and making sure that it's not giving back harmful or deceptive information.
Speaker 2:Any parting advice, I think a big one that I really liked that Justin touched on in the end. There again, it goes outside the scope of data protection and more towards shadow AI that topic but education around what exactly you should input into these LMs, Also being aware of over-reliance. Like Justin said, I think people have a tendency to act as if AI is a single source of truth, which isn't really the case. At times, you need to be able to go out and still ensure that the information that you are gaining from it is accurate and correct before, obviously, you go out and start to, in a way, spread that misinformation yourself and tell your colleagues or create articles around whatever it is that the AI has given you.
Speaker 3:Yeah, I guess the only parting thoughts I have I've probably already touched upon. This is not a buy a point product and fix this particular problem of data poisoning or data security within AI application development. It's going to be baked into a lot of existing things that are already in place and then augmenting those policies procedures to include new types of applications like AI models, and there are different points along that development pipeline where folks have access to do bad things with data and it's not just at one point. There are about three different areas that that can take place and there are different strategies and different things that companies can be augmenting with at each stop along that development pipeline to at least minimize the risk of introducing bad data into these models or unintended effects of pinning up these applications.
Speaker 1:Yeah, absolutely Well. Great words of wisdom from the two of you, and thank you both so much for joining us here today on today's episode. Lots to take into consideration and certainly it's an evolving landscape moving forward. So to the both of you, thanks again.
Speaker 3:Thank you very much for having me.
Speaker 1:Okay. So what did we learn in this episode? First, data poisoning and prompt injection. Thank you very much for having me. Securing your models requires a new mindset. Traditional cyber controls aren't enough. Guardrails must be built into your data pipelines, your model training and, yes, your prompt engineering. And third, trust in AI isn't just about explainability or performance, it's about integrity. If you can't trust the data, you can't trust the outcome. As enterprises rush to adopt generative AI, this moment demands something more vigilance, because in this new era of intelligence, the attack surface isn't just your network, it's your knowledge. If you liked this episode of the AI Proving Ground podcast, please consider sharing with friends and colleagues, and don't forget to subscribe on your favorite podcast platform or on WWTcom. This episode of the AI Proving Ground podcast was co-produced by Naz Baker, cara Kuhn, mallory Schaffran and Stephanie Hammond. Our audio and video engineer is John Knobloch and my name is Brian Felt. We'll see you next time.
