VanRein Compliance Podcast

Why AI Auditing Matters: NIST AI RMF vs ISO42001

• Rob & Dawn Van Buskirk

Share episode

Send us a text

Thank You for Listening to the VRC Podcast!
Visit us at VanRein Compliance
You can Book a 15min Call with a Guide
Follow us on LinkedIn
Follow us on X
Follow us on Facebook


Rob:

Hello and welcome back to the VanRein Compliance Podcast with your host, Rob.

Dawn:

And I'm Dawn.

Rob:

Hey Dawn, we are here this week to deep dive into the world of compliance, cybersecurity and the evolving technology landscapes. Isn't that right?

Dawn:

Ooh, yes, that is right.

Rob:

Now you're more excited than that, right?

Dawn:

Yes.

Rob:

Yes, folks, of course I am.

Dawn:

Compliance is very exciting. We make it exciting.

Rob:

It is, it's fun, it's exciting. Well, this week, what are we going to dive into? What's the special topic?

Dawn:

AI you may have heard of it. I'm not sure if you have, but why AI matters and how NIST AI compares to ISO 42001.

Rob:

Yeah, yeah, we're getting a lot of questions about this. Yes, we are. What are the two differences? And we're going to break each of these down for you this week so that you can make an informed decision on what you want for your business or for your organization. So, whether you know, whether you're a you're a tech founder, compliance professional or someone trying to just future your business or understand what's going on in the AI industry, this episode is for you guys, so let's go ahead and get going. Shall we Dawn?

Dawn:

Let's do it.

Rob:

All right. Well, first of all, I have to kind of set the stage right. This is kind of take a step back and at 30,000 foot level and take a look down and kind of see what's going on. Um, because you know, ai is no longer emerging technology. It's out there, it's around, it's, it's in everything we do, everything is trying to use it. Uh, and it's deeply embedded in how businesses operate. You know everything from service bots to answering phones, to clinical decisions, to financial fraud detectors and all of that. Our phones try to write messages in AI and it's always bad. Then Alexa tries to order dog food all the time. But I think that's a different AI. What other areas do you see AI in your world? Dawn?

Dawn:

Just writing a document, writing a Google Doc? There's Gemini right there. Hey, let me help you. It's really in everything. It's in the Delta app. You know the servicing it's in on websites. The bot, a chat bot. It'll say do you want to? You know, talk to a bot or do you want to talk to a person? And they give you the option now. So this is something we we all live with now. Yep, it is.

Rob:

And it is. But there's four key areas that we need to focus on is bias, security, vulnerabilities, privacy breaches and compliance violations. What I'm finding is and I've been to a couple of AI conferences lately and it is the new hot topic, as we all know, it is the Wild West, as we all know, but the problem with that is there's really no governance, there's no guardrails, so even in organizations, because what happens is some execs will say, oh, I need this. I saw this while I was flying. I talked to a friend that they're doing this or something of that nature, and what I've noticed is that nobody has a good idea on what to do for security and actual compliance.

Rob:

Now, some of the larger folks you know you've mentioned the Geminis and you know, obviously, the Googles and the Microsofts, the Amazons, you know there's Dialpad, there's DeepJet, there's from others out there that have actually done really well and they've dove into compliance, but majority of folks aren't. So today we're going to kind of unlock that AI can create real good tour, reparational and financial disasters for your business. We don't want the disasters. Those are the bad things. That's what we're trying to make sure we don't have. So the two areas that we're going to focus on this week is the NIST AI, RMF and the ISO 42001. So let's first start with AI auditing. So, Dawn, since you're our certified internal ISO auditor, why don't you go ahead and show us what is AI auditing and why does it matter?

Dawn:

Well, it's basically a health check for your AI systems, evaluates whether your AI is operating ethically, securely, fairly and in compliance with the standards and regulations of your business. It's just going to look at lots of different pieces of that your security posture, your AI's decision making. You know, is your personal data, the data that you have that you're utilizing the AI for. Is it protected in that AI bot application?

Dawn:

whatever you want to call it, you know, is it monitoring? You know issues happening in the AI, you know app or that type of thing. And then the quality is it actually been trained? Is it actually can answer questions? How many of you and I'm going to raise my hand have asked a bot something and it says I'm sorry, I can't answer that? I don't know what you're talking about. And you keep doing it and doing it and it just keeps saying the same thing. That's because that AI bot was not trained.

Dawn:

Probably a company and I can't remember which company it was probably just threw it out there, thinking we've got one out there. Well, you have to train it, you have to. You have to, you want it to have you know FAQs that are pertinent to your business. And if you just throw something out there and it can't answer a simple question, then that's not good, obviously. But really, really, the AI auditing is really more for the, you know, just privacy. Security is making sure that it is a secure application and that it is, you know, is trustworthy as far as, obviously, what it's shooting back to you and that the information you put in there is staying secure what it's shooting back to you and that the information you put in there is staying secure.

Rob:

Yeah, trustworthiness is key because a couple of conferences I've been to, I'll talk to people about their compliance postures and they'll try to go to their website and they don't have an ISO or a SOC or Nest AI or anything, or even a high trust. They're just sure. Whatever. And what's going on is your clients, your partners, your investors, regulatory regulatories, regulators there we go. They're the ones that want to make sure that you're actually following the law right. You're actually doing the best you can to create a great, great AI experience, but also being secure with the data. And don't just throw something into the process thinking that you know this is going to fix everything. You need to vet the AI. Throw something into the process thinking that you know this is going to fix everything. You need to vet the AI. You need to dive in deeper and make sure it is a world-class solution for the problem that you're trying to solve. Those are the big areas.

Dawn:

And we also have to take a step back, because last year we forgot to mention you kind of briefly mentioned it the EU AI Act, and it's the world's first comprehensive legal framework for AI, for regulating AI. Of course, the EU is typically first in all. This, I mean GDPR is a very refined it's a very good laws around that Framework 27,001 and all the ISO standards are European standards and regulated through those standards. You know, this was enforced just late last year and this has been really good. We've got some EU customers that already adhere to it, but this is something that the United States does not have yet.

Dawn:

There is a lot of things that have been written, have been, you know, written and discussed, but nothing is is the law yet, and so that's why each state has taken it upon itself to do. You know different kinds of laws, but I have not seen a state that's done an AI, that they're more privacy, privacy laws and that type of thing. So this is something that we'll we'll keep tabs on and see if our government, our federal government, issues something like this. But in the meantime, if you do handle you know EU resident data and you have AI, you're an AI company or you use AI you'll want to. You know this is something you'll want to look at as well as, obviously, GDPR.

Rob:

So I just wanted to bring that up really quick is obviously GDPR, so I just wanted to bring that up really quick as well. That's a good point. That's a very good point. Now, what we do have here in the States is we do have the NIST AI RMF framework. So the biggest thing right off the bat is there are two. There is one, well, two standards one certified, certifiable, and one is a framework. So the first one is our own NIST AI RMF, which is a framework. We're going to go through that. And then ISO 40 2001,. Which is the only AI certification and that did come out of the regulatory need. The regulatory frameworks came out of the EU for June 1st AI framework requirement this year.

Rob:

So, first up, the NIST AI RMF is published by the US National. You know is published by NIST. Now, for the people that don't know, nist it's the National Institute for Standards and Technology. That's the key. This is voluntary, it is not a certification right, it is not a law, but it is a framework that we use to audit and you can use to actually make good, solid business decisions, which is key. It's governed and focuses on governance map, measure and manage AI risk. These are the four frameworks, if you will, with the Nest AI RMF, and this is what we use to audit, because we do perform AI audits for clients that decide hey, we want to bring AI into the environment.

Rob:

What do we need to do? What do we need to focus on? What is the risk to the rest of my business? Where is the data housed? Where is it processed? All of those natures. There's that area we really focus more on the risk management identifying, assessing and mitigating AI risk. Like what is it going to do to the environment? Like when you do a remodel to a home or anything of that nature, what's it going to look like? And I tear this wall down, is there plumbing in there? Is there electrical outlets and stuff we've got to do, but those are things and changes to your business and it's really just a great internal improvements, especially US-based companies. So this is a great way to get going. Once again, it's not a certification which, dawn, you're going to chat about ISO 42001 here in a second but it is a great framework that we can actually audit against and get you secure and dialed in Yep. Now what about ISO?

Dawn:

In the ISO 42001, that is a. It's a standard, it's a certifiable standard and this is focused on building a full AI management system across the organization. So it's going to emphasize governance, obviously, something certified and want formal proof of utilizing AI. So you want to be certified. Hold that certification there. So you can do either we can provide, either we would do the readiness for the 42001 and then have our external auditor do piece and then put that certification stamp of approval on that if you pass, and then the NIST AI RMF.

Dawn:

That is something that we can do and a lot of times clients start with that because it's something that is a little bit. It's a framework, so it kind of gives them the idea so they can kind of figure out what best practices based on it are and kind of get things set and then you can move into ISO 42001 or whichever you want to do. But you can do either. But yes, there is a big difference. One is a certification, one is just a framework. You know we adhere to these best practices. That's great. But if someone asks you if you're certified, you can certified. You can't say you are unless you have the 42001.

Rob:

You can say you've been audited against right or comply with these NIST standards and ISOs and actual certifications. So it depends on where do you want your business to be and if you have investors or clients that require certification or just require evidence that you comply to the NIST. Those are the two differences.

Dawn:

The other thing I want to touch on is policies and procedures, and also if you're, if you're, if you're not ready, you're like whoa AI. I've just started to use it in my organization. The best thing you can do out the door and we've done this for a number of clients already is create a. We've created AI policies around how you use it.

Rob:

So do you let?

Dawn:

your staff use it? Is it just a C-level management type of thing? What are you using it for? We do have some clinics that are actually toying with using, like ambient listening, ai. Those are, those are crazy. Those are the HIPAA compliant. Those are the really expensive ones that you know because they're you know EPHI is flowing through there, obviously chat GBT. If you're really expensive ones that you know because they're you know ePHI is flowing through there, obviously chat GBT.

Dawn:

If you're going to use that, you never put any PII or ePHI in there. It is pretty much just hey, help me write an email, help me do some research. You know that type of thing. But it's always best practice for a business or organization to say how they are going to use whatever AI cloud, chat, gbt, gemini, whatever and to have it in a document, a policy, a procedure. So then the staff also knows can they use it, can they not? What can they use it for? What can't they use it for? So it's very, it's a very good idea to set those standards, those guardrails with your organization, because people will go crazy with it and we want to make sure that they're utilizing it correctly and how you want them to use it.

Rob:

Yeah, and really to look at both of these. You know NIST is more flexible, right when ISO is extremely rigid, and you've done numerous ISO 27,000 one audits here, don and I've I've been involved as well, but your name has to go on the paperwork because you're the lead auditor and you're certified. So what, what? What that really brings is structure and a perspective into the environment and it's a huge competitive differentiator to have that 42001 certification. Personally I would just go right to 4001. But hey, if you're small, you're just starting out. At least start with the NIST pieces. Yep, yep. So let's kind of, let's kind of talk kind of some of the best practices and and I'm going to go back to the HIPAA a little bit, because everything always goes back to the HIPAA, because that's kind of what I.

Rob:

That's what I stand on, right You're, you're fun and fancy with the ISOs and the SOCs and stuff. You know I'll be hanging over here with the healthcare geeks with me, right? I'd recommend do both Seriously. Start with the NIST, AI, RMF and mature your internal practices. So what we do a lot of now is we do a HIPAA plus SOC 2, or we've got a couple of clients that'll do HIPAA plus ISO.

Rob:

So HIPAA is also based off of the NIST standards and then what that does is it gives you a foundation of auditing, it gives you a foundation of policies and procedures and training and all of that. And then you build on top of that your ISO or your SOCs and people start to mature their organization so they know what it's like and can expect the right thing, what to expect and how to do things, and they build towards that 42,001 external certification and validation, because it is investment in time and money. The people that that complain about it either had the money or not the time, or vice versa. They have the time and not the money, so it's you have to commit that.

Dawn:

So Yep, and you might as well start now. If you're using AI, you might as well start. Let's do an assessment, let's see where you're at, let's see what pieces you're missing, because it's only going to get more, it's only going to be utilized more and more and it's only going to it's going to be integrated in all the software you use, whether you want it or not. It's there. Yeah, I mean, you see it. It's like anything you use. Oh, here's an AI, here's this. I mean even even our accounting software. It's in there too. It's like, okay, so it's, it's, it's. You might as well just get started. You're going to use it. Let's just start. You can contact us. We're happy to add that onto your, your suite of compliance services, if you will, and happy to help you with that, to navigate the setup and what best practices need to be instilled in your organization.

Rob:

Yeah, yep, those are some key pieces and how that overlays, how it maps everything out which is important. Now, we know things are moving fast, things are going to continue to go fast, but here's kind of some of the things that I'm seeing in the space and I know, don, the things that I'm seeing in the space and I know, dawn, you may see the same or see things differently. Maybe is really regulators or regulatory. I did it again. I said regulators, regulators, Regulators.

Dawn:

Regulators Okay, there we go.

Rob:

They're going to demand it. Your insurance and companies are going to demand a certification or a framework. Your clients, most importantly, are going to demand what are you doing? They're going to demand I need evidence of proof that you're protecting that data, not that you're thinking about it. Do you have a certification, or that you have been verified or audited against that Nest AI RMF standard, which is really key. And the other thing too, here is just the brand trust. You know brand is big. It's big with us here at Van Ryn, but your business is. Is your brand, it's been built, yeah, and you have an expectation to, to maintain that and to be, uh, to be solid and be. You have integrity on how you handle your client's data.

Dawn:

The other thing, too, is to remember that AI is something that we have to live with. I mean, it's kind of like we all live with our phones now. It's something that's there it's not going away, but we also need to be cautious. There's a lot of different.

Rob:

AIs out there. It's like a mom.

Dawn:

I know there's a lot of different AIs out there no-transcript and so it's amazing to see them evolve. But we also have to be careful. So I wouldn't just use one, I would have a couple. If you're using as resource, maybe use that. There's Claude out there, but there's a whole bunch. I mean, obviously, and even in the chat GBT there's a million different chat GBTs like different things it'll do for you, creates images, I mean it. It there's so much it can do and it starts learning when you're asking it questions. It starts learning about you. I think, rob, you did a test the other day. You went into chat GBT and and asked like said tell me about myself or something, and it like spewed out all this stuff and you're, you know, ceo of Enron. I mean it totally.

Rob:

It was kind of creepy actually.

Dawn:

Creepy. I say I guess it's smart and creepy, creepy, smart, creepy, smart. But you know. And then there's things like Grok in X you can like have conversations with, and that's super that that one actually is creepier than I think, cause it's like it'll have conversations with you and it's crazy what's out there right now. So, again, be mindful of what you're using, what information you're giving it. But also, like, double check it, double check it, double check. You know, is it? Is it saying the right thing? Don't just take its word for it. And they're definitely getting smarter, which is kind of scary. But kind of fun too. Scary and creepy, no.

Rob:

Scary and creepy. There's Dawn's words of wisdom, yeah. Don't sum that, no, that's not my sum of the day. Don't get the creepy bot. Yeah the creeps.

Dawn:

It's a very good resource, though it's very helpful. Yeah, it is.

Rob:

And it you know, it'll continue to get smarter. Like you just mentioned, it's gotten smarter.

Rob:

They actually expanded the memory so it remembers more about who you are and what you've asked it and it'll start putting together. You know it'll get responses that it knows about you and then you can even do pictures and all kinds of random stuff. You can turn people into pirates, all kinds of fun things, craziness. But to kind of bring it back to really kind of focus on what are the competitive advantages right now, I would say either one of them is a competitive advantage a NIST, ai, rmf or the ISO 42001. So I would first start with the NIST because it is less time consuming. It also is less costly.

Rob:

Just get something going and then say now I'm ready for the ISO, now I'm ready to take that next step and our auditors can take you through both of those and see which one gives you the most, the best competitive advantage. Because, just like SOC and other and 27001 and HITRUST and everything else, having a certification I said that today too, like regulatories right Having a certification really is going to set you apart and it's really going to make you very valuable in the market and in the space. And it's going to become to a point, I guarantee you in the next couple of years that if you don't have a certification, you're going to, you're going to lose probably about 30% more business or 40% more business, because nobody's going to do business with you if you don't have the right frameworks in place. Yep.

Dawn:

It's all. It's all trustworthy. Trustworthiness and, as you can see, a lot of the big players. They have a trust center at the bottom of their page. If you scroll way down to the footer of any zoom I mean any of the big players you're going to see trust centers and you're going to see they will lay out all the certifications that they have, and that is becoming very important. And also, when you're doing an ISO or SOC 2 audit, your vendors need to be also. We need to know their certification. So it's going down all the lines. It's not just you yourself, but you're using vendor XYZ. What certifications do they have? Because your data is flowing through them. So it's all about. We just got to look at everything here and make sure that you're working, always working with vendors that are trustworthy. But this is going to be something yes, that Rob's right that we're going to start seeing as people is having these additional certifications as AI gets bigger and bigger.

Rob:

Yep, yep, that's what we're seeing. So what are some takeaways Dawn for the listeners? What can they take back to their organizations? What are the first? What are? What are some takeaways done for the listeners? What can they take back to their organizations? What are the first things they need to do?

Dawn:

Well, the first thing, if you are using AI whatever AI it is identify how you're using it and create a policy and procedure around it for for your staff, for your team, for your whole business. Is it just part? Is just a departmental, departmental thing? Make sure everyone knows how they can use it and how they cannot use it. So that would be the first thing policy procedure on on your AI bot software, whatever you're using it for.

Dawn:

The second thing is, if you're, if you're using a lot of it and you're kind of unsure about if you're, you know, with the information and you're kind of unsure how to do it, you've got a bot, you've got an AI bot on your website, you've got different things going on, then let's do a NIST AI framework audit for you. Then after that, if you're like whoa, I'm really using a lot of AI, I really really need this to be, you know, concrete certification, then let's go to the 42001. So you do have steps, but I would start with policy procedure. What are you using? Let's get some policy procedure around it to identify how you're using it, what information is flowing through it.

Rob:

So Bingo.

Dawn:

There you go. That's it, and we're happy to help.

Rob:

We are. We're here to help. Yeah, I mean, if there's any questions you have, you can obviously email us at hello at vanrykecompliancecom, or put it in the chat section here on the comment section, and we'll just have a conversation. That'd be great. Absolutely, and if there's anyone that you know that's diving into the AI world or trying to figure out how they are going to be secure and compliant for this podcast and you know we grow when you grow we're trying to figure out how they are going to be secure and compliant for this podcast, and you know we grow when you grow, so we're excited to just get the information out there for people to learn and go from there. All right, dawn, I think that's the pod this week. I think we are AI'd out.

Dawn:

That's it.

Rob:

Alrighty Well, until next week. This is Rob.

Dawn:

And this is Dawn.

Rob:

Alrighty, we'll see you all next week.