AI in Healthcare: Protecting Patient Data Without Falling Behind

Your doctor's office knows more about you than almost anyone. So what happens when AI gets its hands on all of it?

That's Sandy Dunn's whole job as a healthcare CISO, and her answer is calmer than you'd expect, because the threats aren't new, it's the speed and the blast radius that changed. Which is where she and Mo pick it up, getting into HIPAA as a checkbox that protects no one, why synthetic data beats guarding data you never needed, and what AI does to trust in a field that runs on it.

🔗 Podcast: https://alice.io/podcast

Follow the show so you don’t miss the next episode.
New episodes every two weeks. Stay curious.

Transcript

SPEAKER_02 0:00

Every organization, every cybersecurity team out there right now don't want to be too far behind because then you're at a disadvantage. Your adversaries have better tools. But on the other hand, you don't want to just turn a blind eye and then have something cause a huge impact within your organization. Because that's really what's changed is just the speed and the blast radius with AI. I mean, we had that before. What we're scared of hasn't changed that much. But now it's just the speed and the blast radius that it is hard to keep up with. And I don't think that's unique to healthcare. I think that's really across the board. If you listen to people who really understand the problem, everyone sounds a little nervous about it.

SPEAKER_00 0:43

If AI has ever made you stop and think, wait, what is happening? You're not alone. I'm Mo, and I'm a security researcher asking the same questions. On Curiouser and Curiouser, we're having open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how people inside the work are thinking about it as it happens. So join us and listen in as the conversation takes shape. And uh while this Sandy is not from the Bikini Bottom, she is still very cool. Sandy Dunn, great friend of mine, and just an awesome person to know in general. Sandy, I let guests introduce themselves because I always skewer, and you have such a cool history. So please go ahead and introduce yourself. What do you do? Who are you? Yeah.

SPEAKER_02 1:36

Um, so I'm Sandy Dunn, 20-plus year career in cybersecurity. Um, much longer in the computer industry. I can remember when we were encouraging people to upgrade from one gig drive to two gig drives. So I was on the phone selling computers when we were telling people you need 32 megabytes RAM in your system. So um it's been amazing. I got into cybersecurity actually because of a phone call. I had a customer actually call ordering a laptop when I was at MyCron PC, and he started telling me about carnivore, which was like just a really fancy sniffer that I think it was the NSA had put, encouraged that uh a lot of people to put at the telephone companies. And I was just absolutely fascinated. And that was kind of my first dip into oh my gosh, there's this whole other world that I want to know more about. So um I got started before it was cool, you know, before anyone was really talking about. We didn't even call it cybersecurity, we just called it security. And there was a, you know, a few passionate people that I knew here locally that were into hacking and doing that kind of thing. And that turned into a a career within the cybersecurity team at HP and um became a CISO. So now I'm a CISO. I I've had several opportunities in different types of businesses to be a CISO. So I have experience in startups, but um have done uh quite a bit of CISOing in healthcare.

SPEAKER_00 3:09

Awesome. Cool. I mean, you do a ton. I mean, I remember I think our first time meeting, you were doing a lot of work with OASP on like some of the standards writing. Um, you had like the Compass project, which was really fascinating. There was also the AI incident response uh process, which I think was really big at the time. Still is, right? Because I don't think anyone really has a great handle on AI security incidents, and they keep changing almost every week. It seems like there's not only a new one, but there's always a little bit of variance, just enough where it's like we need a new process for this. So I think that was really cool work that you did in kind of uh spearheading these projects and really pioneering some of this work.

SPEAKER_02 3:50

Well, so with OWASP top 10, you know, I kept joining those calls and I mostly listened for a long time and they kept talking about all of the different things that could happen with the AI. And I kept looking at it through my Cecil lens, which is okay, now what? So how do I defend myself? Where should I be, what should I be doing right now? And so you're right, Mo. I went to Steve and I said, you know, let's let's actually create something that helps people get ready for this. And we came up with the checklist, um, published that. It was really popular. And as you said, I mean, it kind of um ballooned into a whole bunch of other different projects, but that's been awesome to be a part of.

SPEAKER_00 4:31

Yeah, again, OASP is doing lots of cool work. I mean, just around everything. I mean, we're working with OASP right now on uh on like the agentics skills report. So that should be coming out like by Black Hat. But um, depending on when you're watching this, it's already come out or it's about to come out. So sorry if if for a spoiler or sorry for being late. Um, but that's another really cool report that's coming out with skills and how things have kind of changed and in the supply chain with AI. Um, but you know, you spent, as you said, over two decades in not just security, but in just the computering, the computing industry. And again, as a CISO at a healthcare company, you've likely seen healthcare data from almost every angle. And, you know, you have to govern HIPAA stuff, you have to deal with TII. Um, and then you have to run all those incidents around them, like when a breach hits or even when builders are creating things with AI, uh especially in your organization. So, with all of that kind

Balancing AI Benefits and Security in Healthcare

SPEAKER_00 5:29

of diversity that you've got and all the new kind of things we're seeing in AI from note takers and agentic systems to like LLMs that are like really being trained on your patient's data, how do you feel the balance between protecting patients uh in AI has changed versus like the benefits of enabling AI for their data?

SPEAKER_02 5:52

I think every organization, every cybersecurity team out there right now is trying to find that middle ground, which is you know, you don't want to be too far behind because then you're at a disadvantage. Your adversaries have better tools, and but on the other hand, you don't want to just turn a blind eye and then have something you know cause a huge impact within your organization. Because that's really what's changed is is just the speed and the blast radius with AI. I mean, we had that before, like really what what we're scared of hasn't changed that much. But now it's just the speed and the blast radius that is hard to keep up with. And I don't I don't think that's unique to healthcare. I think that's really across the board. Um, you know, if you listen to people who really understand the problem, I think everyone sounds a little nervous about it.

SPEAKER_00 6:50

Yeah. I think when we think about healthcare, right? It's just like one of those really sensitive categories. And it's easy to assume that there's a lot of sensitive data there. So, like, for example, I'm a big gamer, right? I don't think that like these game companies have all this like really interesting data on me. But you know, sometimes they take in the location of your device, which is very accurate, right? So like I don't really ever think about it from a perspective of like every time I start up a game, like, oh, they know exactly where I am. But like when I go to a doctor's office, I for sure know that they have like all this information on me, like from my blood tests to uh like my pre-existing conditions to concerns that are typically covered by doctors in like our very private uh and sensitive notes, right? And these note fields in these apps, they're usually just general text fields. You know, there's nothing special, so it's like not protected in any way other than like encryption at rest. So I'm wondering like when we start to look at solutions that are specifically using AI over all of this very sensitive data. People know it's sensitive, right? And we think about like the ways that it can be used to make a consumer's life better. What does that risk analysis kind of look like when you do that specifically, like I guess for your industry?

Protecting Patient Data with AI and Synthetic Data

SPEAKER_02 8:06

Yeah, it's a great question, though. And you know, keep in mind, like an organization like where I'm at right now, like we never get any of the notes. We would never get that level of detail. And you're exactly right. I think that's if we were to kind of pull back and understand how do we actually improve the HIPAA law, it would be to take out any of the, you know, you walk up and the, you know, you go to the desk and the nurse says, here's our HIPAA laws. And um, I always say, Well, what are those for? What do you why do I have to sign this? Why do I, you know, and the the poor administrator doesn't even know what it is. I'm sure she's sat through training and I'm sure, but it's become just a checkbox. It doesn't reduce the risk. And so um, if you look at the 18 different identifiers, it's your name, it's your address, it's these things that are already public. Um, but what people really care about is the things that you mentioned, which is I don't want somebody, if I've gone and and had some sensitive thing at a doctor, I don't necessarily want that to be public. And so maybe a revamping of it would be, you know, actually identifying what data, you know, is we could have sensitive data and super sensitive. And and, you know, there is what is considered very sensitive um PHI, which is anything that you go in for mental health or addiction or anything like that. But right now, you know, there's cases where if you have a breach and it's just a code, like what you went in to have surgery for is a code and that gets leaked, that's a breach. Well, for most of us, like that doesn't really matter. And so I don't know about you, but I get probably two or three letters a year to the point where, you know, it doesn't really have any um value to me. It doesn't, yeah, you so you gave me credit monitoring for the data that's already out there. Um, and that's the other thing, well, which is uh I don't know if you saw that Doge, they're accusing Doge, Elon Musk. Um apparently there was some big database that had every citizen's social security number in it that got you know released. And so I maybe that's part of it too, which is is how much of this data is already out there. You know, not to say that that's right, but you know, I think that we keep trying to boil the ocean and you know obviously we're not very effective at it.

SPEAKER_00 10:43

Yeah, and I think you hit something like really square on the head with HIPAA reporting, right? Like HIPAA reporting is very sensitive. So as you said, just the surgery code is one, just the name is another one. Any of that gets leaked or is assumed to be leaked or is proven to be leaked, it's just like boom, you've got you've got an hour report, right? Um and the thing is, uh, we can also talk about the the case from Doge, but like when you have that much data that is kind of breached again, right? Like you only you can only lose this data once, right? And then after that, it's just out there. And it's very difficult to like understand, I guess, the risk every time as a consumer. But as an organization, this is like something you you have to really care about a lot. Um, you know, that's why I think whenever we see those, like, oh, do you have credit monitoring? Do you have like I've got it all enabled? Um, but I also assume that everything's breached and I take a lot of like the stuff on the consumer side really seriously uh because it's so easy to get access to to this information now. I'm wondering, like, uh, you know, from a CISO's perspective, you know that this is already a sensitive requirement, and it basically gets triggered at a sneeze, you know? So like how does that kind of uh change your sensitivity to these types of uh frameworks and like exactly which ones you pay attention to, which ones you implement, uh maybe like which ones actually matter, right? Especially if you've got to deal with, you know, hip hop basically getting triggered every time you you do something new and potentially get breached. Yeah.

SPEAKER_02 12:17

It's a great question. And so we talk about it all the time where I'm at, which is um, you know, how do we reduce our attack surface? And so one of the things that we've looked at is using synthetic data, you know, and we've got you know some great um success stories around using synthetic data. And the data, we're not even using the patient dating. We're actually creating synthetic data from the patterns of the data. So, I mean, it's not it's not um PHI at all. So I think that would be my answer is you know, understand the problem that you're trying to solve and where AI can help you, and there's um probably a better way to skin the cat. And there and AI gives that too. I mean, that's one of the really exciting parts about AI is being able to actually do things um in a better way, more quickly, um, that meet the business objectives and may not necessarily, it doesn't mean you're just training every AI model with, you know, sensitive data. There's other ways to solve the problem.

SPEAKER_00 13:24

So I guess maybe to flip this a little bit around, right? Um again, synthetic data is a really good way to kind of get around um using real data and to avoid like having systems just propagate real data everywhere. So then you kind of limit your attack surface. Uh let's just say we had to do like a little thought exercise here. And if you could go and like rewrite some of these frameworks around reporting and like the HIPAA kind of stuff, right? Like if you had to modify HIPAA in order to be maybe more effective, more impactful, right? And give you more like signal to something's actually wrong here. Um, what would you what would you kind of change in these kind of frameworks to make them less noisy, so to speak?

SPEAKER_02 14:09

Yeah, take it away from being a checklist into something that's really effective. And I think that's the frustrating part sometimes as a CISO is you spend too much time trying to um pass a framework or you know, a SOC to a high trust, whatever. And it and takes you away from doing what you know, I certainly consider real security. And it's a balance. You know, if you're doing security really well, passing those types of audit questions should be really easy. And they're really the bottom bar of what you we want to be doing as security professionals. It's the you know, HIPAA's the base. I mean, it's not that's not security. And you've heard that argument before, compliance isn't security. But I want to say that compliance is really important. If you if you turn it around and think about restaurants, you know, a health inspector going into a restaurant and they come in, you know, every six months to make sure that you don't have too much grease and that you're not trying to poison people with, you know, uh old cheese and old meat and stuff. Well, it's a point in time. That health inspector coming in is just a point in time, but ideally, that's the bottom. Ideally, he never catches you when you aren't doing actually really good uh you know, managing your restaurant in a clean and healthy way. And I think that that we want to look at the SOC 2 and everything like that, too, which is that's a bottom. Like they're just checking, making sure you're not poisoning and giving people, you know, some sort of uh food illness. Um we it and it is good to have the check and balance because a lot of times um security is you know, nobody has time for it. Everyone's going too fast, and there's just not enough time in the day. But the other thing I will tell you is really modernizing. I mean, we have an opportunity, especially with AI, you know, with the CICD, the Terraform infrastructure is code. A lot of your um auditors are not up to speed. You know, they just don't run into environments that are actually deploying in a modern way. And so making sure that your policy, you know, you don't have to respond to those age-old questions of, you know, tell me about your change management process where you submit a ticket. And you no one, I don't know anyone who's done that for the last little while on how we build software anymore. So making sure that your policy and procedures uh actually reflect what you're doing and then automating it, right? I mean, nobody wants to do my manual stuff anymore.

SPEAKER_00 17:17

That's entirely fair. Um yeah, I don't I don't really think any security team wants to go through like now that we have all these tools that do such good work at reducing the amount you have to view. Like uh even this morning, I was uh watching a live stream. I wish I could tell you the organization that was hosting it. I just don't remember off the top of my head, it was like 6 a.m. Um But it was interesting. They were talking about like how application security has changed with AI, and they were talking about the program that they were building, and the new focus was around bad patterns

AI's Role in Application Security and Risk Assessment

SPEAKER_00 17:52

made in the design to look at the design decisions that are made and then identify any bad patterns that happened after that design decision was made, right? So, like, you know, you can kind of trace back these baseline security requirements that you need to establish earlier on. I mean, versus how security has been done traditionally, because you had to use people and you had to use manual time, right? You couldn't track down a bad pattern across your entire stack so fast. You would just have to do it at a point in time, you'd have to review the application and go forward. But now we have like an ability to just review signal at scale uh from an incident response perspective. From an application security perspective, you can now really work alongside developers in a way you couldn't before at scale. You know, you can basically build uh your 3,000-person application security team with agents and have really good signals sent back to the core team. So I'm wondering, uh, now that we have all this signal, all this noise, and we understand all of this, um, we go back to these frameworks around risk and risk-based frameworks and really being able to grow and build the security program around risk. But risk has had so many different changes to the language and how it's being framed. Now, we all like to say that there's one easy way to understand risk and it's the same. But with AI introducing new vulnerabilities, I think people get confused and they consistently mistaken new vulnerabilities with new risk. So I guess do you think that AI is actually introducing new classes of risk that we need to be building our security programs for? Or do you think they're introducing new vulnerabilities that can just be rolled up into a type of risk we're already familiar with, therefore being dealt with by techniques we already have?

SPEAKER_02 19:50

I mean, some of it is unique, you know, the non-deterministic nature. I mean, how do you test for something that's non-deterministic? You know, what's a pass? If you get a close to the same answer after five runs, after 10 runs, you know, like it there is a lot of what's changed with AI. Um, the fact that prompt injection is unsolvable. I mean, we could, you know, it's a system architecture issue that until we redesign the entire how the systems are deployed. So that's that's basically whack-a-mole until we figure out a better way to do it. Um, so that it is different. There are a number, you know, the the the model poisoning. You know, I remember when Poison Ivy came out, and you know, people don't talk about model poisoning as much as they used to, but it's a real thing, you know, like how easy it is to actually impact the response in a model. And if someone had malicious intent, how would they do that? So there are new threats, but risk is risk, right? It always comes down to dollars and business, you know, what you're trying to accomplish as a business. And so um I think that we have a great opportunity to actually have a better conversation. AI has helped security teams actually come to the table because people recognize that there is, it's not risk-free. And, you know, a security person should probably be in the room when you're you're rushing to deploy these kind of things. But I still find a lot of it imbalanced. You know, for instance, um, note takers. Okay, everyone has a note taker. And you know, I hear people talk about, you know, especially in healthcare, not wanting to have um note takers internally because somebody might mention some PHI. But no one ever talks about a policy to prevent anyone from joining a meeting that has a note taker, you know, that's not in their control. Uh Verizon just came out with their impact breach study, which I think it's the first year they've actually done an impact report, which is based on insurance claims. And I think to actually have a good risk conversation, you have to be able to quantify it. You know, I think that was what we struggled with as cybersecurity professionals forever was saying, hey, something bad could happen, something bad could happen. And then, you know, everyone kind of did their thing, and then the big thing happened and it took a while. And it was and no one wanted to listen to the person with the tinfoil on their head that said, you know, I told you it was gonna happen. But that's true of anything, right? That's true of there's always someone warning you about, you know, some danger out there. And so how do we actually quantify it? You know, and this goes back to um the Bruce Schneier. You know, he's talked about this for years, the security theater. We all do security theater because that's the only way that we can move the needle. You know, we um come in, you do your slides, you always talk about the worst thing that could possibly happen. Um trying to get people to actually slow down and implement the security that you're you know that the organization needs. So short answer, that was not a short answer. I think risk is risk. It's always about dollars. You know, how much is this gonna cost the business? What's at play? What's the the risk appetite? What's the risk threshold? Let's have that conversation. And I think AI is as is bringing CISOs to the table to have that conversation. Um, your your entire IT should be in the enterprise risk register. I still find cases where people are fighting to get out from underneath some uh other category within the enterprise risk register. No, IT should be one of the top categories, cybersecurity should be one of the top categories in in the risk register, and I can prove that with dollars. You know, I can show you the impact. Um so uh make sure you quantify it. And is the threat slightly different? Yes, but it there's a lot of similarities too.

SPEAKER_00 24:12

Going back a little bit, you brought up like model poisoning, and uh as soon as I like hear about that, there's you know, like right now, I think we're in a really interesting space, or at least time in the space, where open source models have gotten a lot of really good attention, right? There's been this trend where open source has kind of followed flagship uh for the longest time. And most recently, the flagship models have gotten really, really good, like really good, you know? Like you look at Fable and it's like, wow, this is incredible. You look at the newest releases for Chat GPT

Open Source Models and Biases

SPEAKER_00 24:45

and you're like, oh man, these are insane, right? And then you go and you look at like GLM 5.2, and you're like, okay, this is like really good, but also it doesn't cost you anything to run, right? Especially if you can host this locally. But these open source open weight models, right? If you look today, there was the the hugging bed that was released, right? Which is gonna basically be a P2P network of hosted models, right? These things are huge, massive, difficult to download, and yes, you can get them off of um hugging face, right? But there's now this other place where you can get them, right? So now you have to wonder, okay, well, like what type of biases are kind of preloaded here? You don't know how people are modifying their models, right?

SPEAKER_02 25:30

Like not yeah, and it's impossible. Like it's it's mathematically impossible to actually know what it what went in there. Are you, you know, you could even, I mean, hopefully they have a system card, but how do you verify it? And I know there's been a lot of discussion around that. Um, but you know, with the Chinese models, um I was just reading a threat report this morning and they found all sorts of backdoor and some networking equipment that you know was well hidden. So, you know, yeah, it's a tough problem because uh I wouldn't be comfortable writing a Chinese model just because I know, you know, I've just seen too many bad things.

SPEAKER_00 26:07

Yeah, and like we said, that you know, every model, uh all open source models have their own kind of issues. And we know that models coming from outside of the US, they have a bias already baked in. I think this is where things get difficult because these models are so cheap, so useful to run, right? And we do our best to remove the biases in these models to make them more usable in an internal environment, right? In your specific field, and I cannot think of a couple of other ones, but specifically with patient history um and populations. We never do.

SPEAKER_02 26:39

Yeah, we knew we would never touch that data into it. We would never put that in it.

SPEAKER_00 26:45

Yeah, and is it too much of a liability risk? So you think that's like the real main issue there?

SPEAKER_02 26:50

Um, it's just not useful to us. To your point, and I, you know, I'm glad I'm not at an organization that is wrestling with this. A, it would be very difficult. Like, I don't know where you got the data, but if you most organizations are, you know, they're very strict about what you do with their data. You know, if and it's there's these very long, you know, contracts that tell you what you can and can't do. To the point that, you know, even if you train a model, they they say that they own, you know, the synthetic data that you created with that model. Now, again, we don't, that's not how we do it. Um, but I'm glad that I don't have to worry about that. But you know, think about that, Mo. Look at all of the models that we use, Claude, you know, ChatGBT, they trained on your data, on my data, on everybody's data, you know, before anyone really knew what they were doing, that they were just scraping all of this. Like, maybe it doesn't even matter what data goes into a model because they have it all anyway. And I know that I don't want to be flipping about it, but and you see where people are like, well, you can't have any of our data. Like you can't use any, and you're like, Well, we're actually trying to you'll you benefit. Here's the thing, you know, we are this is still very new, you know, back when was it that was it 22 that Chat GPT? The years.

SPEAKER_00 28:16

Yeah, 22 was the mainstream moment, yeah.

SPEAKER_02 28:18

And um, I I can't believe it's been four years, but you know nobody looks at your Google Maps and says, hey, I you know, how do I know that this AI is trustworthy and stuff? I mean, they did in the beginning, remember when people were like driving off bridges and going into lights and stuff? Yeah. So there is it, it's possible that as we get more comfortable with the technology and it becomes less artificial to us, it just becomes intelligence that there's less scrutiny around it, like there's more trust. You know, I've never really thought about it. You know, how do they do the Google Maps? Why, like, why do we trust that now?

SPEAKER_00 28:58

Yeah, some of it is just adoption through use, right? You see a lot of people in the crowd use it. And um, I don't hear people actually say, oh, I like Googled this anymore. It's like, oh yeah, I asked ChatGPT, or oh, I asked Claude. Um, like that's like the new, that's the new thing, right? The the more people that adopt it at scale, uh, the more assumptions that we make as consumers, like, oh yeah, well, if a lot of people are using this, then it must be good. And that's, you know, one of like the big things that we like look at in our day-to-day is like, okay, well, like how can you trust your AI when we know that not all data out there is good. Like, for example, uh, there was like this funny anecdote that I was uh listening to on a podcast, and they were talking about how um there were like 4chan forums and um and threads on like what an AI apocalypse would look like. And the only reason why AI knows how to be like you know, violent or how it would take over the world is because we've literally written scenarios about AI taking over the world, right? And if you've got this super intelligent model that is trained on all of human writing, guess what? It's gonna have these edge case scenarios and all these crazy things in there as well. So, you know, that whole um part where we talk about training data and diffusion and like you know, bias at scale um continuously goes down. Like you see less bias as you include more data. Um, I would actually think that a lot of I mean that's like how you you would kind of mitigate uh certain risks when it's associated with LLMs. So, for example, if you were doing like a user study and you're using a certain LLM, you might actually want to include more data for more populations, or you'd want to use various LLMs to see the results from multiple places.

SPEAKER_02 30:47

Um have you ever read any of Brian Christensen's books on the alignment problem?

SPEAKER_01 30:54

Tell me more.

SPEAKER_02 30:55

Or Melanie Mitchell is another good one. Um I mean, the funny thing is, is everyone's an AI expert, you know, and I always feel bad for people like Brian and Melanie who like toiled in the darkness and nobody cared about what they thought for decades, and now everybody else is an expert. But Brian's written some really, really good books uh about you know some of the challenges, but things like you know, a lot of the medical data they use white men, and so women are underrepresented and people of color are underrepresented. So I, you know, is adding more data to that is that gonna solve that problem?

SPEAKER_00 31:36

It I don't in a theoretical sense it should, but if we think about how it historically data has been collected, right? Um, you know, like as someone who's black, like one of the big things in the black community is that we've seen like health data be very inaccurate when it comes to our population, right? So LLMs are completely trained on this type of data. And we had a really great episode with Tanisha Martin, and it was really like we talk about like how data inherently the way that it's been collected is kind of biased, right? So we are left with results that tend to be biased. However, if we have biased data that we know is already in the population, how do you kind of mitigate that, right? You kind of have to validate that data, make sure it's actually true, right? But then how do you know that the um validation source is able to do it correctly? Unfortunately, the bias problem is so hard to solve at scale. Um, right now, I think most of the solutions have been just scale it up. And eventually, over a time set, you either understand what the bias is exactly, or there's so much data that you can start to make assumptions that it's less biased than it was before because you've collected it from enough different sources, or you're using enough different LLMs to kind of process it. So like it's essentially the bias of every single LLM, and you can't really tell by the end, right? Um, all the solutions to bias are very difficult to handle. And even even the ones that are the best ones that are doing validation on the end of the data, it's still very expensive to do, you know.

SPEAKER_02 33:13

When in Dali, one of my first experiments was Dolly, is I had it create an image of a white, blonde college professor with a white horse. And then I added, and it came out, you know, kind of a cute blonde lady with a white horse, and I added one word, cybersecurity, and she came out looking like a witch. And I thought, oh, that's interesting, you know, just by adding that one word. So I um I think we're we're violently agreeing that we're just starting to understand the complexity of this problem. And and trust is, I mean,

Understanding AI Trust and Power

SPEAKER_02 33:49

it's it's so it's very powerful. You know, it's just like a stick of dynamite. You know, if you need to clear pores, get a stick of dynamite. But you gotta be very careful because you can do a lot of damage with it. And I think that's how we have to think about AI, which is you better understand what you're doing when you're holding the dynamite. Yeah, you can do some really cool stuff and move a mountain, but you want to throw it the right direction.

SPEAKER_00 34:15

Yeah. And um, you know, the the big thing about trust, especially when we when we look at like the fields that are most sensitive and most highly regulated, especially healthcare, um, I think that trust is actually the biggest thing in healthcare. Whereas we already like kind of don't trust banks, just as a society. You know, Equifax has trained us really well to just not trust financial institutions. Uh, you know, that that whole hack is, you know, the, I think the big moment. Um, but when we think about like a doctor's office, like I walk into my doctor's office and I talk about everything freely because I trust my doctor, right? So there's this big element of human connection that exists in the healthcare industry where I don't I don't really think about my doctor using AI. But I remember one time I did go in and my doctor asked ChatGTT something, right? So it's like you want, and again, I'm in the Bay Area, so I feel like over here it's normalized. It may not happen everywhere. But like you think about that, and it's like, okay, well, like what happens to the human, the human connection when we start including AI and like these sensitive workflows, um, specifically in places where like the human connection is highly valued, like a doctor-patient relationship, you know, how does that kind of change how we think about it and maybe like even where AI is being implemented in healthcare?

SPEAKER_02 35:36

Yeah, I it's a great question, Mo. I I think if you talk to doctors and nurses, I actually have two nurses, both my daughters just graduated, so I get a little inside perspective on this, but um they're they're so burdened by paperwork that they they're not having fun either. Like most of those people, I am so impressed with the healthcare industry, how they train individuals who come into it. So I will I will absolutely say my experience watching my both my daughters go through it, I I wish we had something similar in cybersecurity. Like it was the mentoring, you know, the job exposure, like they uh really uh do a fantastic job of bringing young people into careers within the medical community. But we all know that people get burnt out. Well, what I don't think people get burned out taking care of people, they get burned out with friction. And it's the same thing that happens with us. You know, we most of us love what we do in cybersecurity. We love what we do. What burns us out is if we can't solve the problem or you know, there's too much red tape and everybody keeps slowing us down. That's what wears a person out. And I think it's the same in healthcare, which is we've made that a very difficult job, mostly because of things like HIPAA, you know, mostly because of hardware, um, because systems don't talk to each other. It's so hard to tra um to trade data. Um, they're burdened by you know, the insurance stuff. Um, I actually was just going through and trying to figure out um that my different options for a dentist. I read the material and I could not figure it out. And I threw it into Claude and Claude explained it to me. And I was like, so why did I have to take that extra step? Why did I have to go to Claude to get some clarity on this? Why can't they just write this so a human can read it? Um, and so I I guess, you know, I'm Sandy Sunshine, so I'm always optimistic. I I think we're in, we're definitely in the frontier. Everyone's experimenting, um, everyone's trying to figure out where this AI um helps us be more effective. And my my real hope is that it does free up people's time. That, you know, and I'm sure this has happened to you. Like I've had nights where I stayed up all night. I was so into what I was building, you know, whether it be um cursor or claw, whatever, like literally stayed up all night because I was just so into whatever I was building. Um, that's pretty fun. You know, that's fun to be that enthusiastic about something. So uh I think, you know, it's definitely a stick of dynamite. Um the next few years will be interesting. We talk about frameworks and governance. I think that's a really challenging problem because uh our legal system was never set up to move at the pace of cybersecurity, you know, at plain networking, at the at the pace of technology, our legal and regulatory system was not not set up to keep pace with it. And now we add AI on top of it. So when people start talking to me about AI governance, I just go, I don't know. I don't know. So uh uh maybe that changes. I it but I think the next decade is going to be absolutely compelling.

SPEAKER_00 39:08

Yeah, there's a lot that I can say about that. Like, you know, um the last CISO I worked for basically invested a lot in GRC because he was like, GRC is where we're going to see the most change in cybersecurity. GRC teams are usually, you know, the business connectors for the security department, right? If you can enable them at scale with to be as proficient in application security as your access person, right, you enable them to communicate technical risk a lot better at higher levels. So the GRC engineering function that has kind of emerged in the last two years has been kind of insane to see grow. Um now I'm hearing about GRC engineering teams almost at every conference where these GRC teams are becoming really technical at how they're able to assess AI solutions, how they're able to like very quickly understand risk, how they kind of build their own internal frameworks for dealing with new AI products that are coming in, with new things that are being built, how they kind of enable application security teams and infrastructure security teams, just the security organization in general to operate a lot faster. Um they don't, you know, they don't take the place of like a your AppSec team or your cloud security team, right? They're not specialist. However, AI was always made for a generalist, in my opinion. And if you are good enough at understanding the frameworks and you have just enough information about all the different areas of security to be, I would say, good enough to have a conversation with, you can use AI to fill in some of your knowledge gaps and use experts around your team to kind of grow really fast and get ahead of governance before it becomes a problem. And I think that's kind of where we have to, we really have to grow into. How do we enable the business at scale to adopt AI? And as it's adopting AI, how do we understand these risks at scale in context to our business? So um when we think about like, or at least when I think about like uh some of the healthcare risks, right? A lot of it just comes back to the human factor. Where it's like, okay, like you said, we need to reduce the amount of friction that it takes clinicians to actually treat humans because that's the work that they get done that they do the best, right? Um, and the initial place is triage and customer service and first line of kind of uh the first point of contact for a customer. Like imagine a gentic AI system that is used to do that like first line of contact that misreads someone and says, oh, you're fine when they're not, right? And I think this is where like I get a little bit worried. Um, because again, this is like an AI solution that we deployed, that we aren't sure about. We have to like monitor all these things, but this outcome is the exact one that's dangerous to humans, right?

SPEAKER_02 42:14

You know, I would take it in a heartbeat. I hate going to the doctor. You go in and burn so much time. They treat you like you're they're doing you a favor, even though then they're charging you for it. Like somewhere along the way, the whole system got broken. Um, they just released a uh Dr. Kiosk in China. Did you see that?

SPEAKER_01 42:34

I'd go in a heartbeat.

SPEAKER_02 42:36

If you if I had the choice between a regular doctor or a robot doctor, I'd go to a robot doctor in a heartbeat. I would rather go to a robot doctor.

SPEAKER_00 42:46

I mean, it's it's an interesting thing. I think China also has a very interesting way that they manage the population's data, right? So they by default have a more consolidated data infrastructure and are able to collect data about the entire population faster. So I think those types of health systems are really well informed. And they will likely be able to um give better results. Again, this mainly comes from a data consolidation point with when it comes to China. So it's not necessarily like it's not, I think, something that like we can get to today in like the US or like the even the EU.

SPEAKER_02 43:26

Look at what happened with 23andMe, like all the DNA testing and everything. So, you know, initially it was sold to people as hey, wouldn't you like to know a little bit more about your family tree? Well, next thing we know that these people are, you know, they're understanding their databases and their family trees. And um, the one the Golden State Killer got caught because I think it was a great aunt submitted her data and then they were able to match it. So, yes, do we want the Golden State Killer off the road? Absolutely. So basically everyone in the United States today, it can be identified because the database is so large. So if you leave your DNA anywhere, you know, I didn't sign up for that. And think about photographs. Um, you know, anywhere you go, think about all of the license plate readers that are out there. Like there is a lot of decisions being made for us, kind of subversively without full visibility, um, that we all don't understand. Understand the the long-term consequences. Um you know, China was very much into pre-crime analysis. And that becomes if you're any kind of a weirdo, you know, and I label myself that way, if you're different, you know, all of a sudden I don't fit the pattern, and I'm getting someone's accusing me of you know being different than everybody else, and I get put in a different bucket. Like that's terrifying. And I think that's what we're scared of. We're not scared about, you know, somebody finding out what my blood type is. We're scared about what they do with the data. What's the long-term goal with the data and how could it be abused and misused against us as individuals?

SPEAKER_00 45:10

Yeah, I think that's gonna be it's gonna be difficult to see. And it really just depends on how the space grows.

Data Privacy, Surveillance, and Long-term Risks

SPEAKER_00 45:16

Like we're seeing a lot of you know new startups coming along every day. Um, a lot of them are very, very small and agile, right? And they're just they're filling spaces, they're filling gaps where incumbents haven't been able to traditionally move. And I think it will change ultimately how like software is built, um, it will likely change how people start to interact with different um technologies and you know, in different industries. So, you know, like lovable is a great example, right? Where people were using Squarespace or they were using like GoDaddy to go and build websites. But, you know, even to build the website was tough. You needed to use WordPress, you need to go through all these different frameworks, right? And now you have like this one-stop shop where everyone is building apps. I think Will Iam, the rapper, is uh, you know, he's the number one user of Lovable. So yeah, isn't that crazy?

SPEAKER_02 46:13

I'm building stuff I never thought I could build. And and that is super fun.

SPEAKER_00 46:19

And it's gonna enable us to use data in different in ways that we didn't think that we could. And, you know, you can ask Claude to go run analysis on different types of data, right? You can basically say, I don't understand any of these patterns. Go find a pattern for me. And don't find a pattern. And then you can go down that rabbit hole and hope that like at some point it's not hallucinating and you're not accidentally seeing patterns you shouldn't. But you likely will find something that is interesting enough and you decide to say, you know what, I think there's something here. And then you can go build it in the same day, you know. When we talk about like how data is going to be used, I think it really just depends, right? It really does depend on the time of day and what's already out. Um, and you know, for an organization that is testing these things every single day, especially whether you're in healthcare or finance, you're going to find new emergent use cases that are so specific to your industry and so niche that, you know, it might scare somebody. It'll scare somebody somewhat everywhere. There are a couple of uh, like, you know, right now there's a lot of financial products coming out that do like kind of stock picking for you.

SPEAKER_01 47:23

Right.

SPEAKER_00 47:23

So if you say, I'm really interested in, you know, the electric industry or you know, solar, it'll go and it'll put together a small portfolio. Oh, this is your risk tolerance. Perfect. Well, look at all these companies. You know, these are all the trends and patterns. Maybe this will be interesting over a 30, 30-year horizon. So who knows when the next thing comes out and it's just like, oh, well, like this is what you're doing today with your health. Well, you're gonna die in like 20 years. Like, I don't think I want to know like what I'm estimated to die based on my health patterns, but who knows what someone's gonna do with that? And they're gonna say, you know what, you should probably work out like three times a week for 20 minutes on a treadmill, and you'll extend your lifespan a year. I mean, I wear an aura ring every day. Like, I can imagine feeding this data into Clog with like some of my blood test results and being like, hey, um, tell me more about like what my what I look like from a health perspective. How am I doing?

SPEAKER_02 48:16

And I think it's it gets scary, yeah. I I've I've probably told you this story before. Um, my good friend Jay Radcliffe always talked about a story when the Apple Watch first came out, they gave it to a bunch of senior citizens because they wanted to track their heart rates. And and to me, this is a classic example of what we need to be worried about. It's the stuff that's not obvious. So they gave it to a bunch of senior citizens, and all of a sudden, about 9:30 every night, they saw everyone's heart rates. You know, there's patterns of all these heart rates spiking up, and they didn't know if it was a problem with the watch, if that was what they were feeding them. You know, it typically wasn't every night, it was sporadic, and they were trying to trace and understand what was happening. Well, of course, you know, these older people, so they were getting busy. And, you know, if they would have been so horrified if they knew that a security researcher or a researcher could pull all of that data off of the watch that they were wearing. And I think that's the question we want to ask ourselves is what are we giving up about ourselves that we aren't thinking about? And you know, there's if you watch true crime at all, I mean, it's all about where they go and pull the data from. And they pull it from watch, you know, somebody tries to to burn down their own house and he's wearing a pacemaker, they're pulling the pacemaker out and and tracking patterns, you know, if the if you're wearing any kind of a tracking device. So all of you know, all of it's we're being tracked every day. So you know, if the question is, is should we be worried about how much data as individuals that we have about ourselves that could be abused? Yes. You know, do we have any way to control that? I would say we don't have a good handle.

SPEAKER_00 50:03

I think that's a good place to to kind of end. Um maybe. Well, yeah, I mean we're gonna have to, but uh it's it's just a silly topic to end on, but it's it's also a serious one. Okay, so Sandy, where can the good people find you? Uh, where are you gonna be next? Are you gonna do anything funny?

SPEAKER_02 50:23

I am I am heads down working. I mean, I haven't even you can if you would like to reach out to me, if I've said anything that you're interested in hearing more about, probably the best place is to um find me on LinkedIn, send me an invite.

SPEAKER_00 50:37

Well, cool. Then I guess we'll see you next time. And um, I will be at Black Hat and I will be at DEF CON. We're having like a we're doing like a podcast thing, we're doing a live podcast recording. So if you're interested in that and you want to show up, let me know. I guess that's it then. All right. Sandy, thanks for watching. Thanks for having. Yeah, thanks for coming through. So stay curious, everybody, and have a great and safe rest of your day. If this episode helped cut through the noise, like or subscribe so you don't miss what's next. Thanks for spending time with us. Until next time, stay curious.