The AI Ethics Episode: Whose Job Is It?

Somewhere in your organization, an AI decision is sitting on someone’s desk right now. Who owns it? In most mid-market companies, nobody does — or rather, it’s landed on the IT leader who was already doing three other jobs.

In this episode of The Catalyst, we follow Jeremy Wight, CTO of CareMessage — a patient engagement platform serving 22 million low-income patients across the US — who had to write his organization’s AI policy himself. No committee. No playbook. Just the weight of getting it right for some of the most vulnerable people in the healthcare system.

Alongside Jeremy, we hear from Reid Blackman, author of The Ethical Nightmare Challenge and founder of Virtue, who argues that the standard policy-first approach to AI governance is already broken — and offers a framework any team can implement in weeks, not years. Olivia Gambelin, AI ethicist and author of Responsible AI, reframes the vendor selection question entirely: it’s not about auditing their product, it’s about whether their values align with yours. And Anthony Vinci, former intelligence officer and author of The Fourth Intelligence Revolution, draws an unexpected parallel — between the integrity required of a spy with no rulebook, and the integrity required of an IT leader doing the same.

====

This episode is brought to you by HPE.

From AI to data center and network modernization, HPE delivers a cloud-like experience right on your own infrastructure — the full portfolio, from one partner. softchoice.com/technology-partners/hewlett-packard-enterprise 

====

In this episode:

• Why the policy-first approach to AI governance is broken — and what to do instead
• A practical three-question framework any team can implement this week
• How to evaluate AI vendors by values alignment, not just product capability
• What it actually looks like when one IT leader has to make these calls alone — with 22 million patients on the line

Featured guests: Jeremy Wight (CTO, CareMessage) • Reid Blackman (Founder/CEO, Virtue) • Olivia Gambelin (AI Ethicist & Author) • Anthony Vinci (CEO, VICO) • Craig McQueen (VP Microsoft Practice, Softchoice)

#AIEthics #ResponsibleAI #ITLeadership #AIGovernance #TheCatalyst #Softchoice #MidMarket #HPE

===

Show Notes & Resources

Guests

• Jeremy Wight, CTO — CareMessage: caremessage.org
• Reid Blackman, Founder/CEO — Virtue: reidblackman.com • The Ethical Nightmare Challenge (book, April 2025) • Ethical Machines (HBR Press, 2022)
• Olivia Gambelin, AI Ethicist: oliviagambelin.com • Responsible AI: Implement an Ethical Approach in Your Organization • Values Canvas framework — free download at oliviagambelin.com
• Anthony Vinci, CEO — VICO: anthonyvinci.com • The Fourth Intelligence Revolution (Henry Holt, 2025) • VICO forecasting platform: vico.ai
• Craig McQueen, VP Microsoft Practice — Softchoice, a World Wide Technology Company

Sponsor

• HPE via Softchoice: softchoice.com/technology-partners/hewlett-packard-enterprise
• Softchoice AI & Ethics resources: softchoice.com/EAS

The Catalyst by Softchoice is the podcast dedicated to exploring the intersection of humans and technology. 

Transcript

Producer 0:02

This episode of The Catalyst is brought to you by HPE. From AI to data center and network modernization, HPE delivers a cloud-like experience right from your own infrastructure. The full portfolio from one partner. Click the link in the description to learn more. When I was an intelligence officer, a lot of people think that you're selected as an intelligence officer based on, I don't know, being sneaky or clever or smart or, you know, whatever it is, ruthless, maybe. Actually, the number one thing that you're selected for is integrity. Because you have to be trusted to do the right thing when nobody's looking. Because you're given these powerful tools, right? You're trained to be a spy, to break the law. You are essentially trained to break the law. That's Anthony Vinci. He's a former intelligence officer, former CTO of the National Geospatial Intelligence Agency, and the author of one of the Financial Times' best books of 2025. And he is not, it turns out, talking about spycraft. He's talking about you. Or more precisely, he's talking about what happens when someone is handed a powerful tool with real consequences, and the rules haven't quite caught up yet to tell them what to do with it. That description fits a lot of intelligence officers. And right now, it also fits every IT leader who's been handed an AI tool, a vendor pitch, and a deadline, with no policy, no committee, and no one else in the room to make the call. I'm Katie Tikasing. This season, we're making audio documentaries, real stories from the front lines of IT, exploring the challenges of small teams chasing big dreams. Today's episode, what responsible AI actually looks like, and why the old approach might be broken. We're calling it the AI Ethics Episode. Act one The Job You Didn't Apply For. Jeremy Wright has thought about this question more concretely than most. He's the CTO of CareMessage, and the 22 million low-income patients his organization serves couldn't afford for him to get it wrong. CareMessage is a patient engagement platform and the kind of tool that sends appointment reminders, care instructions, and health guidance to people who often have no other reliable connection to the healthcare system. Low-income patients, people served by free clinics and federally qualified health centers. The kind of population where a missed appointment isn't just an inconvenience. It can mean a health condition that goes unmanaged for months. We support specifically federally qualified health centers, free and charitable clinics, and other safety net organizations like food banks in engaging with their patient populations to help drive better health outcomes for low-income patients. So we've reached over 22 million low-income individuals in the United States. 22 million people, and the AI tools that could help reach them more effectively. The same tools that work beautifully for a commercial health app serving middle-class patients are trained on public data. Data that doesn't always look like care messages patient population. Safety and ethics and privacy is super important, I think, in every technology context, but even more so when you're operating in the healthcare space. When you think about the patient populations that we serve that are already marginalized and face worse outcomes in a large different way, we are even more sensitive to how we think about managing and protecting the data of individuals that we serve. In 2023, Jeremy started seeing something his team was already doing, whether he had a policy for it or not. The first was a few years ago when OpenAI released uh GPT 3.5 in particular, kind of really accelerated, you know, within the zeitgeist of the way that people were using technology, but that really captured the imagination of people and we're always looking for the best tools to amplify our work and be effective. So one was this catalyst of hey, people are going to be using these tools, whether we give them guidance or not, whether we create the ecosystem or not. They're going to naturally gravitate towards the tools that are going to give them the most leverage. That was the first spark. The second was different. It wasn't about risk, it was about what it could mean to do nothing. We also look at AI and say there is an opportunity here, and there is an inverse opportunity if we do not leverage this technology for this patient population, that it's going to have an adverse impact on them because the opportunities that AI unlocks are significant. And we see that playing out already now, in particular in kind of consumer health tech applications and those things that are available to higher income individuals. The question becomes how do you do that in a safe and ethical manner? How do you make sure that you're doing no harm with this new technology, which is trained on public data? And oftentimes that public data set is not completely representative of the patient population that we serve. So Jeremy was holding two things at once: the risk of moving forward carelessly and the risk of not moving at all. The tension, most IT leaders we talk to said it sounds familiar. Craig McQueen is VP of the Microsoft Practice at SoftChoice. He talks to mid-market IT leaders every single day. What he hears isn't panic, it's something quieter than that. Yeah, it's a big topic, and I don't think they've been able to think about it too much. I think it's very much on their mind, but they don't have a framework or approach to consider ethically what I need to consider about AI. When anybody struggles with something, whether it's innovation or ethics, they're distracted by the technology. For organizations that have tried to get ahead of this, the usual answer has been policy. Write the policy, define your values, stand up the committee. Reed Blackman has spent six years helping organizations do exactly that. He now thinks he was helping them do the wrong thing. I've been in the AI ethics space, the A responsible AI or AI governance space for over six years. And I think that approach, it worked really well for traditional machine learning or narrow AI. And then it started to wobble with generative, and then with agentic AI, it totally falls apart. So the technology is getting way too complex for this approach to handle, and it's changing way too fast for this approach to handle. The biggest issue is that policy takes a long time to create. You know, for a large organization, Fortune 500, if you want a board-approved enterprise-wide policy, it's at least a year. You finish the policy for narrow AI. Great, nothing's implemented yet. It's just a piece of you know digital paper. Now you got to implement. Implementing takes months, if not years. The AI landscape keeps changing, and it's extremely fast. It's months to months at this point. And the policies can't possibly be anything but out of date. So the standard answer doesn't work, and the technology isn't waiting. And your people are already using it. And somewhere in your organization, there's a decision sitting on someone's desk right now that nobody told them was yours to make. That's where Jeremy Wright found himself in 2023. And what he did next is what this episode is all about. Jeremy's first move was the simplest thing he could think of. Acknowledge that his team was already using these tools and give them a map. If we're not giving clear guidance, our team is still going to use this. They're just not going to use it in a way that we have as many controls around. And so from that, uh, I worked with some others in my team, but kind of drafted a essentially a policy around what is permissible, what types of uh data can we use, in what cases, under what circumstances. For us, we leverage cloud provider and we use uh Google Cloud. And uh with Google Cloud, we have what's called a BAA, a business associates agreement, which allows us to process protected health information due to this agreement that we both maintain certain security protocols and whatnot. So we first isolated that everything that is dealing with HIPAA and PHI, in particular, one, it has to have very clear approval mechanisms, like it has to be very explicit. We want to highly limit the things that we do with that type of data that contains PHI. In plain terms, a tiered system, protected health information at the top, strict rules, specific pathway, explicit approval required, general business information in the middle, some controls, context-dependent, public-facing or non-sensitive content at the bottom, largely open. Three tiers, one framework, not a novel. The second part of Jeremy's approach was about the product itself. Within the platform, when we uh started implementing AI, one of the first things was how do we actually evaluate what's happening, right? Because it's very easy to throw some information and say, you know, give it a prompt and say, give me an output, right? But how do you mean evaluate whether it's actually accurate? And then how do you maintain that over time? So there's two things in AI evaluation, precision and recall. So you have to have a human in the loop to say, we have validated these outputs and we're going to then give it these inputs and say, does it match? We actually manually labeled a data set of thousands of records, set a minimum threshold of accuracy, right? If it doesn't meet this minimum threshold, we're not going to release it. The framework in its simplest form, know what your data is. Know if the AI is working, and monitor both over time. Jeremy says if you want to stress test whether you actually need this, there's a useful question to ask yourself. If you just start to think about if this is happening a thousand times a day, how do I know if it's working or not? And start to work backwards from there. One of the questions that comes up constantly when we talk to IT leaders about AI ethics is whose job is this? Legal? HR? IT? Jeremy had a direct answer. All of them at the table with IT leading the conversation. So as the CTO and the one kind of closest to the technology, I initially was like, hey, we need to do something here. And so I created some drafts. And then I started to put that out. So we had a few different individuals. Yes, our people team uh was also involved, but I think the primary individuals, right? Chief strategy officer, our compliance manager, and myself in developing this. I also have a technology advisory board that supports me, that I go to when I'm, you know, seeking input. And a couple of those individuals are very senior people at uh well-known enterprise technology companies who are dealing with this exact thing, right? AI, safety, and ethics. It's not just me like, here's this thing, you know, do it. Because the whole point is is like we have to have reasonable guidance that's grounded in like what the technology can do, but that's also grounded in safety and compliance simultaneously. And we do that best by bringing those parties together and sometimes having hard conversations. Jeremy's framework addresses what happens inside his organization. But there's a question that comes before that. One that every IT leader faces before the internal governance even begins. Which vendors do you say yes to in the first place? Olivia Gamblin has spent a decade working at the intersection of philosophy and AI. She thinks most organizations are asking the wrong question when it comes to evaluating AI vendors. A lot of companies are defaulting towards we're gonna buy instead of build. Building your own AI is quite expensive. So naturally, we're going towards buy. Think of it this way: you make friends based off of shared values. Your friends that you look at have similar values to you. Uh, you get along well because you share similar values. That's an important part of keeping that friendship strong. Now, turning that into the conversation around vendor procurement, you want to make sure that the vendor, whatever company you are procuring from, shares those similar values that your company does. The technology moves too fast. So you can't necessarily run an ethical audit on a system today because two weeks from now it could be completely different. Instead, what you want to do in that vendor procurement process is understand what are the values, the operating system of the vendor, because that's the operating system that's going to influence any developments in AI. If you have alignment on those foundational values, then you can trust, okay, this vendor is going to make decisions similar in alignment to how we would make decisions if we were building this technology. She's not talking about good vendors and bad vendors. She's talking about aligned and misaligned. Ethics is notoriously sits within within the middle zone. There are stronger practices. So this is where we kick back over into responsible AI. There are good business practices. These are ones that will keep the machine running forward. But when it comes to ethics, you're really more just asking questions. Do your values, do your value sets that you are making decisions off of align with my value set that I am making decisions off of? It's not really that question of good or bad, it's a question of alignment. There's a dimension to all of this that no framework quite captures because frameworks assume that someone sat down and wrote them. What happens in the moment before that? When you're the only person in the room and the decision is right now. Anthony Vinci has thought about this from an unusual vantage point. He led AI integration at the National Geospatial Intelligence Agency. He's invested in AI companies. He now runs one. And he says the question of who to trust with powerful tools, with consequences that are hard to see and hard to reverse, is actually an old question. Running an AI company, it can feel like the Wild West. We have these tools now with AI that you can make things that were literally impossible even a few years ago. There's no real regulation around it, and there's no obvious rules of the road or even guidelines. So I think it comes down to the individual who's building these products, the CTO, the CEO, and people like that to decide what you believe is a good product for the world to have and what's not. When I was an intelligence officer, a lot of people think that you're selected as an intelligence officer based on, I don't know, being sneaky or clever or smart or whatever it is, ruthless maybe. Actually, the number one thing that you're selected for is integrity because you have to be trusted to do the right thing when nobody's looking. Because you're given these powerful tools, right? You're trained to be a spy to break the law. You are essentially trained to break the law. And the idea is you're going to break the law maybe in another country by doing espionage. But you have to have integrity. And I think that's how it is when you're a CEO or CTO of an AI company right now. Anthony's point isn't that IT leaders are spies. It's that they've been handed something powerful and the rules haven't caught up, which means the thing that fills the gap isn't policy. It's character. How are you using it with your customers? What are the ramifications for what the product that you're gonna build or the product you have out there that's using AI? That is a better place where you have control and where you can focus on it. Three different angles on the same challenge: a framework for your data, a values test for your vendors, and a question about what kind of leader you want to be when nobody's watching. The question now is what to actually do with all of that. Here's something we're hearing a lot from IT leaders right now. The ground is shifting under them faster than it ever has. AI is moving from a roadmap to right now. Security threats are unfolding in real time, and the data center and network you built for the last decade may not be ready for the next one. That's a lot of modernization, and most teams are trying to do it with the vendors and the budget they already have. That's where HPE comes in. Through SoftChoice, a worldwide technology company, HPE offers something most vendors can't: one full portfolio that spans AI, security, compute, storage, and networking all from a single partner. So instead of stitching together solutions from five different places, you get a cloud-like experience on your own infrastructure built to handle what's coming next. It's the latest in networking, compute, and storage designed to help you accelerate your digital transformation without rebuilding everything from scratch. If your team's weighing what's next for AI, security, or your data center, this is worth a look. Click the link in the description to learn more. Act 3. Monday morning. Reed Blackman's critique of the policy first approach comes with an alternative. He spent the last year building it, and unlike most governance frameworks, this one is designed to work for a team of five just as well as a team of 5,000. What you need is, to my mind, at a bare minimum, is something that can be rapidly implemented and that could be dynamically changed as the technology changes. And you need something that gives you a clear sense of what success looks like and what failure looks like. Which is to say it needs to be outcome-oriented, not process-oriented. My solution at an extremely high level is answer three questions. What are the ethical nightmares that pertain to AI that are relevant to your organization? What resources will you build to avoid those nightmares? And how will you train your people to effectively use those resources? Nightmares. Resources. Training, not values, not principles, not a mission statement. The point is to name the specific bad outcomes you're trying to prevent and then work backwards from there. It's wishy-washy, it's it's very open to interpretation. Like, are we following the value? Are we following our Nobody even, no one knows. And you can always find a rationale. You get so excited about the good stuff. But okay, tell me the really bad stuff that can happen and tell me what you've done to avoid that bad stuff. Those are clear directives, and if you can't answer that question, that's a problem. And the team that answers those questions doesn't need to be a standing committee. It can start this week. You can create the teams whatever you want, and you can have them up and running in a matter of weeks, not months and years. So it's rapidly implementable. And you only create the teams where you need them. So if you've got no AI going on, don't create the team. If you've got it going on over here, but not over there, then create an ENC team over here, but not over there. Jeremy White built his framework before a crisis forced him to. When our producers asked him how he'd advise an IT leader who's been putting this off, he started with a question that Reed Blackman would recognize. In any case, right, you start to create scenarios and think about the failure modes ahead of time. If you're not considering what might go wrong here, pulling your team in to be able to expand the aperture of potential failure modes so that you can again go back to how can you start to create frameworks of evaluation that can catch those failure modes ahead of time. And if you're not doing that, then I would say, in some way, shape, or form, you're not being very safe with this non-deterministic or probabilistic technology where a response comes out that you don't have control over what's going to occur. And on the question of urgency, on why now specifically, Jeremy was very direct. If if you're a leader and you haven't yet thought about AI safety, then you're way behind the curve because your people are using it, whether you know it or not. And so if you're not giving them guidance and clarity around what is allowed, what's not allowed, and in what context, then you're you've got a big area of exposure that you're not recognizing fully. Your people are using it. That that's like the simplest way I can say it. If your platform or your product has any impact on people, then you need to be thinking about how can I validate and make sure that this is not going off the rails because it can go off the rails and making sure that that's not is really important. Craig McQueen has one more piece of guidance for the IT leader who's figuring out where to start. And it has to do with who else needs to be in the conversation. It's a business strategy that incorporates AI as an enabler of that business strategy. And I feel it's the same way with ethics. Each group that understands how ethics impacts that part of the business would have the responsibility of updating it, reflecting what AI can do. Now they need to be educated on what can AI can do. And I think IT has an opportunity there. If they become an expert on what AI technology can do, they could sit down with their partners who have the corporate policies and help them understand the technology better so that they can help them update their policies in each department. And on where to actually begin, Craig has a suggestion that lands somewhere between practical and a little bit funny. Do you know one of the best ways to update your policies for AI? You ask AI. So it's you can say, hey, here's my HR policy, here's the values of our organization. I'd like to make sure this incorporates the use of LLMs and interactions, provide me ideas on how I should be updating this HR policy to reflect that. And it provides a, you know, it would provide a really good starting point. Of course, humans then need to decide whether they include it or not. Here's what we kept coming back to across every conversation we had for this episode. Nobody in mid-market IT signed up to be their organization's AI ethics officer. There's no job description for it, no training program, no committee to hand it off to. But the decisions are very real. The data is real. The patients, the employees, the customers on the other side of those systems. They're all real. Jeremy White built a framework in 2023 because he was holding 22 million people's health information and the AI tools that could help those people also had the ability to harm them if he didn't think clearly about what he was doing. He didn't need a year. He needed a clear-eyed look at what his data was actually doing. A system for knowing whether the AI was doing what he thought it was doing, and a team willing to have the hard conversations. Reed Blackman would say, name your nightmares first. Anthony Vinci would say, remember that in the absence of rules, your integrity is the rule. Olivia Gamblin would say, start with the vendor's values, not their product. And Craig McQueen would say, IT's job isn't to carry this alone. It's to make sure that everyone else shows up to carry this with you. The technology isn't waiting. Your people are already using it. And the organization that figures this out isn't the one with the best policy. It's the one with the leader who decided this was their job before anyone told them it was. If this episode sent you down a rabbit hole, here's where to go next. Read Blackman is the author of two books on AI ethics, including The Ethical Nightmare Challenge. Find him at readblackman.com. Olivia Gamblin is author of Responsible AI: Implement an Ethical Approach in Your Organization. Her Values Canvas Framework is a free download on her website. Find her at oliviagamblin.com. Anthony Vinci is the author of The Fourth Intelligence Revolution: The Future of Espionage and the Battle to Save America. Named one of the Financial Times best books of 2025. Find him at AnthonyVinci.com. The Catalyst is a podcast from SoftChoice, a worldwide technology company. If you like what you hear, subscribe now or share with a friend. Special thanks to Jeremy White, Reid Blackman, Olivia Gamblin, Anthony Vinci, and Craig McQueen for sharing their expertise and stories. Thanks again to HPE for sponsoring today's episode. If you're ready to accelerate your digital transformation with one partner, one portfolio from AI to infrastructure, visit SoftChoice.com to learn more or click the link in the description.