Welcome to the CISSP Cybertraining Podcast. We provide you training and tools you need to pass the CISP exam. Hi, my name is Shon Gerber. I'm your host of active for the podcast. Join me each week as I provide the information you need to pass the CISSP exam and grow your cyber sector in knowledge. All right, let's get started.
Hey all Shon Gerber with CISSP Cyber Training, and hope you all are having a beautifully blessed day today. Today's Monday. And Monday we get into various aspects related to the CISSP exam. And today is no different. So we have there's three articles I'm gonna bring up that are gonna be highlighting a lot of what we're gonna talk about relating to governance. And that's the key topic of today is related to governance. But the overall aspect that we're getting into is related to AI models and how governance is an important part of this. So one thing that came up, the first article I want to bring up is the OpenAI
article that this is from PBS News, but there's many other articles out there about this. And in this article, it talks about how OpenAI basically hacked a company, an AI startup called Hugging Face. Quite the name, but yes, it's Hugging Face and it hacked them. Now, there's a lot of great details out there, and I don't want to go into too much depth because of the fact that there's more to discuss, but the bottom line is that supposedly this OpenAI had the ability to hack into this AI startup because it stole credentials from them to do this type of activity. So it comes into is really what it comes down to is open source versus closed source. How do you want to deal with it? And then it really comes into the governance associated with it. Now, the part that I think is quite interesting is the fact that these companies, as you are looking to implement AI within your company, do you have a strong governance process in place, especially as you're allowing third parties to connect into your organization? So now, in the case of you have an AI environment, an AI agent of some kind, and it goes out and steals credentials from your organization to do an operation, uh, can you blame it? Really? I guess the question comes down to is maybe, maybe not. This is where governance is going to be a huge factor in any organization that decides to deploy AI within their country or within their company. So, and within your country as well, that's gonna be a big factor. But when it comes right down to it, is you need to understand how governance works. And so I recommend you go out and check out this article related to open AI and its potentially hacking of this company called Hugging Face. But that wasn't the main article I wanted to get into.
The first one that I really wanted, or the second one I want to get into, is a credit union. So this is Synaptic AI or Cyanaptic AI. I can't say these words, but what is Cynaptic AI? Well, July 20th, Communications Federal Credit Union went live on an AI-powered lending platform from a company called Cynaptic AI. So this is what's happening, right? I want to be clear. This is what's actually happening because this matters what happens next. This isn't a pilot. This isn't a press release about some exploring AI sometime in the next year or two. This is a credit union with real people that's automating real lending decisions starting now, this week. All right, it's in place, happening as we speak. So this platform is handling credit decisioning, which means it's involved in the process of describing who gets approved for a loan, on what terms, and how fast. Now, I I'm not against this, to be honest with you. I think this is actually this has some merit, right? I don't know if all of you ever dealt with loans and loan approvals, it can be a challenge. Most definitely. Especially if you're a business owner, just to be clear, if you're a business owner, getting loans is not an easy process. So I would highly recommend that if you become a business owner, consultant, any of those on your s on your own, have a good plan because once you go that route, getting it getting it a loan can be a bit of a challenge. But what the vendor is basically saying is that it will speed up loan approvals and expands access to credits for members. And quote, it does this while you're ensuring fairness, inclusivity, and compliance with regulatory standards. Yeah, that's great. All right, that's the claim. Now let's sit with that for a second and see. So this isn't happening in isolation, by the way. Credit unions as a whole are moving to this place fast, really fast. And recent industry research has found that credit unions are planning to nearly triple their AI-powered services over the next few years. So Communication Federal isn't the outlier. It isn't the one that's sitting out there on their own doing this. There are all types of credit unions that are moving in this space. It's a leaning edge where the whole industry is heading to. So the question I want you to be sitting with as a security professional, not as a member of the public, reading the press release, how are you going to deal with this? So if you are on that credit unions risk committee, how would you actually verify that claim? How would you make sure that these guys say that? Because we all know bumper sticker claims from people can is a time a dozen, right? We've got all of our politicians do it all the time, and they don't back half of it up. I would say probably even a quarter. So again, not that do you trust the vendor, not does the marketing page say compliant? How would you personally with your CISSP go out and produce evidence that this is true? Would you ask for bias testing results? Would you ask how the model explains an individual denial to the loan officer who's explained it to its member? Would you ask what happens when the model gets retained six months from now? And does someone retest it? Oh, that's a big one. Once it how often is it where they do set it and forget it? Right? We talk about that all the time. Or does everyone just assume it's gonna be fine? Yeah, just turn it on. It's all good. Just that happened like with Skynet, right? Yeah. Yeah. So if you've if you haven't heard of Skynet, you're listening to this, go watch Terminator. Yeah, that's where we're headed. Okay, but hold on, because a story two is what happens when nobody in the building is asking these questions. So here is story number two.
Okay, so back in May, a community bank in Pennsylvania disclosed a cybersecurity incident, which, right, they're supposed to do, right? If you're in the financial industry, that's something you need to do. Well, here's the twist: it wasn't a hacker. Nobody broke in from the outside. There was no exploited vulnerability and no phishing email. None of the usual suspects. It's actually a really good movie. Go watch it. You'll like it. An employee took customer data and uploaded it into an AI chat book, okay, which we all know if you have a good governance program in place, you need to have this defined. And this comes down to AI governance policies. They put it in this AI chat book that the bank had never approved or for its use at all. And as far as everyone could tell, they were probably just trying to save some time summarizing something, drafting something, or using the tool in a way millions of people use these tools every day without even thinking about it. And we know that this happens, right? They copy, paste, because the ability of AI to help our jobs make work so much better is incredible, right? So if that is, what are you doing with it? Well, the bank says they caught it fast enough. They reached to the app's vendor before the data could actually be used to train a model, right? A model of the vendors. So this is the narrow sense they got lucky. But here's what I want you to catch. Because this is the real lesson, the damage was already done because of what one action triggered, regardless of whether the data ever got used, that single mistake, one employee, one chatbot, one paste, they triggered three separate legal obligations all at once, practically overnight. Right? So an SEC disclosure, because it was material enough to require one, a 36-hour breach notification to their prudential banking regulator, and a customer notification requirement under the Graham Leach Bliley Act. So three of those things all happened because somebody went copy paste, right? We all know this is important. So you all that are in the financial industries, you need to be truly thinking about this. I mean, everybody needs to. I had a situation when I was working with my large multinational and dealing with intellectual property. How do you deal with that? And you have to have very strict guidelines on what you will allow people to use. So again, one paste in a chat window, three regulators inside of two days. And here's the detail that you should really get your attention. If you're the one who's going to be responsible to prevent this in your own institution, you need to have a really good understanding about this. This bank had already approved sanctioned AI tools that employees were supposed to use. But did they use them? They did the responsible thing on paper, right? We talk about that. Did they have the policies? Yeah. They stood up with the policy, they had that, they gave the people that hey, we're safe sanctioned options, so they would have someplace to go and they wouldn't go rogue. But what do employees do? Yes, employees have a tendency to go do what's easy for them. So this is where training and governance aspects and putting in controls to ensure that this doesn't happen to your organization are imperative. So again, the employees still reach for a tool that the bank didn't clear. So something really to think about. This is shadow AI in its purest form. Having a policy is not the same as having a way to know when someone goes around what you have in place. So if we talk about Synaptic AI or Synaptic T AI, whatever that is with that bank, right? Do you have the governance around it to protect yourself in the event the employees are doing things? And in this case here, what if you have customers and they start putting things in there? And where does it go? You need to truly understand this because there's AI companies standing up on every street corner. You can't throw a cat around without hitting one. And yeah, I'm sorry if I offend you about cats. I'm not a big fan of cats. Cats are great, just not in my house. Actually, I have a cat, unfortunately, and I'm not a big fan of it, but it just sits there and knows that I'm not a big fan, so it just likes to torment me. Yes, I know. Okay, we're moving on, digressing away from cats. So this isn't a one-off, right? It's a pattern. I don't want you walking away thinking this was a freak accident or an unlucky bank. It isn't. It's a documented industry-wide pattern. Recent insider research now names Shadow AI as the number one driver of negligent insider incidents. And we deal with insiders all the time, and the negligent aspects is hey, I like this AI tool. Let's try this. This is ranked ahead of unmonitored file sharing, ahead of personal webmail, ahead of things that security teams will be worrying about for the last decade. Roughly one in five AI-related breaches last year traced back to Shadow AI specifically. Tools nobody approved, being quietly used outside of any governance program. So when IBM studied the actual dollar cost of these incidents, they found out that Shadow AI adds roughly $670,000 to the average cost of a breach. Okay. So let's kind of break that down, guys. We know that a breach minimum is $250k. Minimum, right? If you got any sort of size organization, you've looked at any of these articles around this and the breach reports that about it's you're talking in most breaches are what cost you north of a million dollars. So let's just say hypothetically you got the one that's 250. You got off lucky. But in this case here, say it was dealt with AI, you're now pushing a million dollars. If you're a small company, can you afford a million-dollar breach issue? Probably not. No, that's that's not gonna happen. I know with somebody like me in a very small business, yeah, you're going away, baby. You're going bye-bye. So this is not a technical failure. Technology worked exactly as design, it's a governance failure. So here's where the two stories leave us, right? Sitting side by side, right? Story one, communication of the Federal Credit Union is responsible for AI adoption, right? They supposed to look like, at least on paper. And according to the press release, this is where the industry is moving forward. But then you got story two where you got Pennsylvania Community Bank, and this is when they have bad things happen when it isn't adopted well. So even at an institution that you thought had already done the responsible thing, it didn't. So both banks and credit unions have access to the same AI tools. The gap is governance. Who's watching, who's accountable, who actually knows what's happening with AI inside the walls of your institution? It's huge. It's beyond the vendor's marketing page. And we all know that anybody, especially with AI today, can make a beautiful marketing page that sounds amazing. But do they back it up? So we're gonna get into this next part around governance related to AI and why it is important.
Okay, so as you all know, you are listening to this podcast or you're watching these videos based on getting the CISSP exam. So we're gonna be going over AI governance. And as you listen to the last bit of this when some of those different articles that are out there and those different news blips, you know that there is a lot to deal with with AI. Now, I want to give you a little quick insight. A lot of this is coming up from me working with my cohort. Now, I'm gonna throw a little plug out there, shameful plug for CISSP cyber training. I've got a cohort that's going on right now, and my guys that are in this cohort, the folks that are there, are loving it. And it's doing very well. I have good I have learned a lot out of this cohort along with the folks that are in this. It is eight weeks. It is, I will give you, it's intense, right? There's a lot there, but it's incredible where you have eight weeks and you have accountability to make sure that you are ready to take the CISSP exam in that period. I highly recommend that if you are interested in getting your CISSP and you want to get it done within the next 10 weeks, eighth, well, actually, be starting in September is my next cohort. So 10 weeks from the time that September shows up, September 8th, this is the time to do it. I will tell you, going into the holidays, get it done. This is going to give you all kinds of aspects that you're gonna be able to get ready and study for the exam. I've got a diagnostic exams, I've got lots of questions that can be asked of you. All of my training is there. But the part that makes this beyond what everybody else is, is that we meet once a week and we go over the content. We also, in addition to going over the content, we have accountability amongst ourselves. Are you getting the content done? Do you have questions in our community? Do you have anything that you need help with? That is this cohort. I can't stress it enough. They have the early bird sign up, is now, let's be honest, it's right now the cohort is like $497. I'll say it's gonna go up in price in the future. It is $497 for my early bird is a steal. It truly is. It's gonna go up to be a $597 after that period is over. So if you are interested in doing it, go check it out. I can't tell you enough that you can study for hours on end and hope you pass. At least when you sit in the cohort, you're gonna be done with your cohort and you're gonna have a good feeling on whether you feel solid and confident about it. And I'm gonna use this, make this comment and this statement. If you go through this cohort and you go through all the content that is there and you're a part, an active part in this process, I you can't help but pass the exam. It truly is, because it is there. You will pass that exam. Okay, so let's get into what we're gonna talk about today. So this is AI governance through the CISSP lens, and I want to go what credit unions and their examiners are talking about. Now you're probably asking yourself, what is the point of this? Governance is a big factor in any sort of audit. And if you're taking the CISSP, you are dealing with a higher regulatory lens than you've ever dealt with before. CMMC level one, actually, I heard they just cancel level two aspects for the CMMC. I canceled or postponed it, but it doesn't matter. There's more and more regulatory requirements that are being forced upon you as security professionals. And this is an example of how AI and the credit unions are going to be a big factor. So let's kind of walk through this.
So it's no longer a watch item for examiners, right? AI is named explicitly in 2026 the focus of NCUA's supervisory priorities letter. So this came out again in 2026. It was released, I think, January of this year. So it's coming out for specifically for credit unions in general, but it doesn't matter. Now, if you all are listening to this and you're wondering what is a credit union? If you're in another country, it's a small bank. It's a it's a group of members that actually own a bank. You you put in money and your money as a member, you have can receive dividends from it. So it's designed to take over some of the small-term banking or small-size banking. And so it's a great way for community-based. The one we mentioned in the articles uh is based out of Oklahoma City. They have 23 branches, around 1,300 members. So again, 1,300 people, not a big MT type size bank or a type JP Morgan Chase, any of those, but it's a good it serves a really good community. The key around these small banks, though, is this if you are dealing with a small bank, do you truly understand what you're actually doing from a technology standpoint? A friend of mine owns a couple small banks here in the Wichita area. Super smart, they're definitely on top of their IT and they understand it well. However, their IT folks are a very small group of people. Do they truly understand everything that goes into this aspect? I would beg they probably don't. And so what should they do? They should be getting some advice from other people. But realistically, this is moving at breakneck speed. To the next part of this. How many of the people that are in these organizations understand what generative AI is? I can only guess, but I can tell you that when I talk to people all the time, they all kind of go, yeah, it's Chat GPT. Okay, then what? So it's always interesting. Then you look at the number, there is zero single rule books that the examiners will lead on to compare when they're looking at and evaluating AI. They don't have anything. There's nothing right now because it's so new and they're developing this rule book. So what are they using? They're using the NIST AI uh framework that's there, and plus COSO. So all of those are aspects that they're using in place. And this came out of the GIO GAO report uh back in October of 2025. Again, actually was released in January of 2025. So they at that time did not have a single rule book or um way for them to be able to lean on to understand how to even examine this. So there's no single checklist that you can follow. January 25, the GAO report founded that NCUAs lack comprehensive model risk management guidance. What does that mean? That means they don't truly know. If I go to big corporations and they are struggling with it, can you imagine small banks or small businesses? NCUA also lacks the direct examination authority over third party AI vendors. What did we just talk about? So in this case here, we had community bank, right? And then community bank was using synaptic AI as a third-party vendor. But the NCUA does not have direct examination authority over them. This is where your contractual stuff comes into play, and you really need to have a good understanding. So if you're bringing in a third party that does AI, have you basically gone through everything they've got? And I would. I would not just, especially if it's your business. So back at that company, 85 years in business, right? They're gonna go ahead and they're gonna lob over the fence this synaptic AI. And I'm not talking bad, not talking shade, as they say. I'm not talking bad about what they're doing at all. I'm just saying, have they done the due diligence to look and make sure that whatever they're doing, they've invested 85 years of their company, and it could be gone overnight if they haven't done this. So the result is examiners fall back on NIST, AI, RMF, and existing third-party oversight rules. So you need to know what those rules are and you need to be able to understand them. So basically, what does this come down to? We bought a compliance tool. It's not a defense, governance is, right? So you get a compliance tool, yeah, checkbox, checkbox. Do you really understand what you're checking the box for? And that's a key factor. So this isn't a compliance footnote, it's personal exposure. Exam findings escalate fast. I don't know if you ever dealt with this. I was with a bank going through some different things, and they had some findings that were showed up, and they were relying, they there's a lot of pressure on them to deal with these findings. So an unaddressed AI governance gap can move from passing comments to a documented matter requiring attention on your watch very, very quickly. Are you able to deal with that? And once you start having that, that goes all the way up to the board, baby, and everybody has the eye of Sauron looking down your neck on what's actually happening. Accountability is naming names. Boards want a designated owner for AI risk. So you can talk about it all you want, but if you don't have an owner, someone who actually owns it, that is where your security or your compliance leader in the room is going to end up owning. So if you are your CISO or you are the compliance leader leading your organization, odds are high it's gonna be you, baby. If not, here's what I would do: I would take it, own it. Just own it. Go after it and take it for your own. Because if you do that, then at least at a minimum, you have some sort of ownership over it and you can control the narrative. The fair lending liability is real. So algorithmic bias in lending isn't hypothetical, right? It's a it's an ECOA reg B exposure, okay, with members and regulatory consequences based on that happening. So you gotta make sure that your the models are reporting what they're supposed to be reporting. And then this moves faster than any annual review cycle. Now, if you're in the financial industry, you know there are annual review cycles that you go through these things. But these vendors are updating these models continuously, especially if you're dealing with the model in the case of like the synaptic, synaptic AI. Are their models changing as they're making products? I mean, I'm doing this cohort, right? The cohort materials are done, but I've got feedback from some of my students and they're going, hey, this would be nice. And I'm like, Oh, so what are we doing? Making changes as we go. We're enhancing it. We're making it better for the next group. And the next group will make it better for the next group. Same thing happens with these different models. They're making them better for their customers. And as they're doing that, how often are their models being looked at? Okay. Are they being assessed? Are the same questions they answered the first time around actually the answers of what they are six months later? Odds are high, they're probably not. So one regulatory shift for CISSP domains that are tied to this. You have security and risk management, right? Governance frameworks, board-approved policy all falls within under domain one. Security architecture. You've got explainability by design, not bolted on. Do you have open cloud open environments or closed environments? That is your security architecture. And is there monitoring in place for this? Assessment and testing. Are you doing bias testing? Are you have audit trails behind this? And is there continuous validation related to it? And then third-party risk, which focuses on domain one and domain seven. All of these are aspects related to vendor AI due diligence and your governance gaps. You need to understand all of those. So as you can see, as you're studying for your CISSP, there are many different domains that we'll cover just in this one area related to AI governance.
So the board approved policy question. Examiners are going to ask you this specifically. And I've heard this, I haven't been dealt with an examiner other than talking to some folks that have been on the receiving end of these examiners. Does your credit union have a board-approved policy addressing artificial intelligence? Now I want to start that just a minute. I have been on the receiving end of this, just not in the financial industry. Mine we're dealing with government regulatory entities. And we didn't, at that time, our artificial intelligence was still so new. But in this case here, where the examiners for financial institutions are asking these credit unions, do you have a board-approved policy addressing artificial intelligence? Now, that can come down to is you do. You have a governance policy, you have a policy that's set in place related to AI and governance. That's awesome, right? That's what you should have. But this is the interesting part is does your board actually understand the language in which you provided that information? Or did you just grab some boilerplate language that you could reached out to the AI and said, hey, I need a policy of AI governance, make it for me. And it made it for you. And then you put it forward and everybody's like, oh, this is awesome. But the words in it you don't truly understand. And it's dolphin and you talk shark. Yeah, that doesn't work so well. So you need to make sure that you're providing the same kind of level of understanding into your board that you understand. That's another part that's going to go into this. With any of these policies, especially with stuff that's been around for a while, the board is they're very adroit. They know what's going on. However, when it comes to these new things, you are going to have to train them. Now, I'm not saying they're not going to get it because they're super smart people, totally smart, but they talk in a different language. And so because they talk in a different language, they may not truly understand everything that you have in the documentation. You have to be prepared to be able to talk to them about it. So the follow-up questions examiners will actually ask you. So how do you vet them? AI-specific due diligence. What data does the vendor have access to? Who are their subcontractors? How many of these vendors are actually just like paper? And behind them, there's multiple subcontractors that are actually creating this front. So what protects the member data contractually? How is that tied? You need to make sure you understand that. There has to be contractual aspects related to it. And the reason I'm saying that, I'm bringing this up as a big point. And I was in a multi-billion dollar multinational, okay, global, all over. And our contracts people were extremely good. But were they always connected with everything that was going on in the cyberspace? No. Did I have to work through them with them on this? Yes, routinely. So let's take a step back and let's say you now are part of a credit union, okay, and you are a small shop. And let's just say, in the case of maybe you've got a billion dollars in assets or maybe a couple hundred million in assets, do you have the people that are dealing with your contracts that are tied to this? Do you understand what to do with this related to your contracts? Maybe, maybe not. I would say be willing to bet you probably don't, or at least there's enough people out there that don't. There probably are people that do, but I would say there's probably my vast majority do not have something in place like this. So, how do you know they're using AI? So you run IAData call to surface to look for any sort of shadow AI. Vendors often will add AI features to tools you already use with no formal adoption decision ever made. What does that mean? So let's just use Copilot for an example. Microsoft Copilot. You deploy it to your organization and you've got Microsoft 365. Do you have a copilot instance that is local or is your copilot instance global, right? Is it out to the web? Did you actually turn it on because that's what happened when it came out of the box and you didn't even really realize that that's what you were doing? Those types of activities can happen. If you have copilot, are there little buttons that you've clicked that are allowing it to share more data than you want? Important parts you need to ask. What's regulating all this? Okay, so there is no separate AI rule book. It's mapped to BSA, AML, right? It's fair lending, vendor management, and ERM. All of these inventories are there specifically where it's mapped to, but there is no rule box rule book specifically focused around AI. What does that mean? It means you're gonna have to develop it. You're gonna have to come up with something using the BSA and the AML, as well as your NIST AI framework, all of these aspects you're gonna have to come and do. So a vendor questionnaire is not just a form. As a cyber professional in the room, your job is to understand how AI actually gets across to your organization. How does it get out to your people? How does it leave your organization and go to other places? You need to know this. So the questionnaire asks the right questions, but you need to go and dig deeper on that. So the questioner goes, yep, check, got it. But if you don't ask the sub-level questions, that's a challenge. Now I will say there's a great financial framework out there called CRI. CRI is a great framework that's dealing with this, the Cyber Risk Institute. And we've talked about that on CISP Cyber Training a couple times. Highly recommend them. Very, very good. So what do you recommend to your credit union? You're a credit union, you are you're in a financial administration and you are a cyber person with them, or maybe just you lead IT. If you don't, do you need an AI officer? Well, if you're a small organization, no, you don't need that. And NCUA governance, they say that you don't need this based on the size and complexity. There's no mandate for a dedicated AI person. So no FTE specifically that. However, you do need to have a named and accountable owner. And we talk about this a lot with CISSP cyber training when it comes to ownership, whether it's for data, assets, or in this case AI in general, which deals with the assets and the data, if you don't have an owner, that's a really bad place to be. So if you don't own it and you don't know who else owns it, what that tells you is nobody owns it. So even some of the smallest shops out there, they need to have assign AI governance to an existing role. Could be your CISO, IT director, compliance officer, somebody. If you don't have one, one of you all, just grab it. Take it, own it, right? Plant that flag in the SAM. So the more complexity that's growing, again, you need to have small cross-functional AI governance committees in place. And you need to walk through this because you need to understand it and educate your people. Now, if there's no in-house bandwidth, you got nobody, right? You got, I got my CISO or I got my IT director. I don't know. Here's what I would recommend. Bring in a fractional or virtual CISO that can own your AI governance specifically until it's justified a dedicated role. Yeah, you may you may think, oh my gosh, CISOs are expensive. They're they're not terribly expensive in the fact of if especially if you're dealing with a breach. They can be some of your most inexpensive work and they can come in with the knowledge and skills you need to help you in this environment. So I highly recommend that if you don't have the skills inside, you don't want to build the skills inside, or maybe you just don't have the time right now, no bandwidth, look at a CISO. Look at a virtual CISO, get them on the books, go talk. You can feel free to reach out to me. I can point you in the right direction, either myself or other people that can help you with this. But and this is obviously a plug, right? For me, I'd love to have you do that. But honestly, I just want you to find someone that can help you get meet your needs because realistically, I don't want you to be in a situation where things just go sideways. So when it comes right down to it, is that the things you need to, your board needs to understand the risk literacy behind this. The owners need to be accountable for inventory policy and the vendor oversight. And then your staff needs to understand acceptable use and be aware of that. Back to the article we talked about where they had an employee that decided to go and use an AI platform that wasn't part of their acceptable use platform or their acceptable use policies. Yeah, then that just goes bad. So you need to have a good plan around that.
So AI vendor risk, run through your TPRM lifecycle. Now, if you don't have a TPRM policy or lifecycle set up or program set up, again, also reach out to me. I can get you in contact with some people that can help you with that. But it comes down to this: you got your inventory and classify. You need to surface every AI touch point and you need to understand the vendors and what are they using in this. You need to have a due diligence package associated with this for your SOC2 report, which comes down to explainability documentation, bias testing of your evidence, and then model change notification terms. All of that needs to be done in this package to talk about AI to your leaders. If you have a SOC 2 type 2 that comes in, they're gonna want to ask you this question. They are not gonna want to, they will ask you this question. They're also gonna talk about contractual language. You need to understand the right to audit clauses. What that means is with your third-party vendor, do you have the right to audit them in the event that something were to that you want to? I did this with people that maintained and stored our intellectual property. I had a right to audit clause. You need to understand what yours is. You also need to have around incident notification and the SLA that goes with that. Because when if something were to happen, how fast can they respond to you? Because guess what? Especially if you're in the financial industry, you will have to, your own people you have to respond to. So contractual language, big, big nugget there. Ongoing monitoring, you need to have periodic reassessments and drift tracking on your AI models. You need to understand those and you need to be in contact with them of going, what's going on? Do not treat your AI vendors as a set it and forget it type of activity. If you do, you will wake up one morning dealing with a whole lot of headache. And you still may have the headache, even if you are more in-depth and invested with them, but at least you'll see that train wreck coming before it actually happens. So when we're dealing with domain three and six, architecture and testing aren't optional add-ons. So domain three, security architecture. You need to have explainability as an architectural decision. It needs to be made before you deploy. You need to understand what you're doing. This should not be bolted on afterwards. So in practice, if a vendor can't explain why loan application was flagged, that's an architectural gap, not a documentation gap, you can paper over that later. It's an architectural gap. So you need to make they need to be able to explain that to you. They also need to understand bias testing. This comes down to the assessment and testing piece of this. They have to have bias testing understanding that will pass at launch that doesn't cover a model that has been retrained twice since. So if you see that situation that's occurred, the revalidation has to be triggered by change, not the calendar. So what they're saying is that as they see change, how is the re-evaluation happening? If a change happens in the model, then it needs to be reevaluated, not going, okay, it's Monday and it's been a week, I'm now going to do it again on Monday. So important part for you to understand when it comes to these models. And so these are how the different CISSP domains kind of flow into that. Domain two, we're dealing with data classification as a governance control, not paperwork. So classification only matters if it's enforced. How often I have seen it where classification is deployed, people say it's there. There's a great binder that says, hey, we have data classification, but nobody does anything with it. So a DLP rule that actually blocks something is something that's useful. A DLP rule that just kind of flags and everybody just kind of ignores it, not so much. So you need to have classification that matters and it needs to be enforced. Ownership, again, domain two, baby. We gotta come down here and it has to be a named role, not a vendor's job. So when a communications FCU platform, we talked about this, process a loan file. Who is accountable for the data owner approving that flow? So just like we mentioned before, they're having this synaptic that's gonna be doing this, and it's amazing who's approving that flow. Not just letting the custodian handle it. You're just gonna let it run. What is your thing? Who is the named person that's going to own it? Okay, you gotta have somebody who owns it. I think I think I've beaten that horse pretty much to death at this point. So the model validation lifecycle. This is what it should be in practice. You have pre-deployment testing. Bias and fairness testing should be completed before, underline, before you go live, not discovered after somebody complains about it. Now, this is where in testing is important. Now, this you also can have a lot of pressure, especially if you're trying to roll something out that's new. Whether you're a startup or a new capability, there's pressure to get this thing out. Dates have been set, people are having are counting on it. So what happens? Sometimes people don't go through all the details. Model behavior tracked over time. A model that passed testing on January can drift by June. It can. It could drift by noon, but it'll basically drift by June for sure, six months later. So you need to be able to understand the continuous drift monitoring that's going on with these models. There also needs to be a documentation trail. Audit logs, decision rationale, and model version history all needs to be understood. The evidence examiners are actually going to ask for is this documented? A bunch of uh auditors I dealt with, they said, you know what, when you have your exam or you have your audit, make sure it's over documented. And it can't have this enough. But it has to be more than just documented on a piece of paper. You have to do it in practice. And then trigger-based rev revalidations. Again, based on model updates or material changes, that is when it needs to happen. Not a date on the calendar. The one-year annual reval. No, it needs to happen when it sees changes.
Okay, so let's get into a couple questions. So, one, your credit union board approved the vendor AI lending platform based on vendor marketing claim. Okay, that's not good. That the tool is bias-free and fully compliant. The vendor SOC 2 type 2 report is current and unqualified. What should you recommend to the board to do first validation step before you actually go live with this product? A. Accept the vendor's marketing claim since it constitutes a binding representation. B. Approve the deployment. The SOC 2 type 2 report confirms the vendor's compliance posture. C. Require independent bias testing results and explainability documentation specific to the lending model, or D. Delay deployment indefinitely until the federal AI legislation is finalized. Okay, there's a trap in here, and that you will go through, and we talk about that and the cohort cohort a lot is their traps, the various traps that ISC Squared is going to be asking you related to these different questions. And they're going to want you to try to bite off on it. So let's kind of look at what you think. Accept the vendor's marketing claim, approve the deployment, require independent testing, or delay the deployment. What do you all think it is? Well, the answer is C, right? Require independent bias testing and explainability documentation. So the one that can bite you, right, is the B. So we go back and we look at this, we go, hey, it's approved deployment. SOC 2 type 2 report confirms a vendor's compliance posture. That would be the easy way for you to go chase that rabbit. However, when you've got two real credible situations, you need to really look at the one that is the most important and how you would expect to treat them and how you'd want to go. So again, when you're dealing with B and C, SOC 2 attests to security, availability, and processing integrity controls at the vendor. It says nothing about whether the AI model itself is free of bias or can explain its decisions. Only independent bias testing and explainability documentation are the only way you can do this. And that was what will actually answer the question the board is really asking. Again, you gotta dig deeper in this. So why the other claims are wrong, right? Whatever's wrong. Accept the vendor's marketing claim. Kind of goes without saying, right? Anyone can put anything together when it comes to this. They can say whatever they want. Site 2 type 2 confirms compliance, right? It attests to, like we talked about, the integrity of the controls of the vendor, but it was not designed for AI models or bias and explainability. And then delay until federal legislation is finalized. Well, that really doesn't work because what's going to happen? Your organization is going to want to keep moving forward. They're not going to wait. So you need to make sure that you have a plan to deal with that. Okay, the next question. AI lending model passed bias testing at deployment in January. The vendor has since pushed two silent updates at the model to improve approval speed. Your institution's policy requires model recertification annually. It is now June, and the examiner asks what triggered your last revalidation. What is the primary gap you need to address? A. Recertification should be triggered by material model changes, not just the calendar date. B, the model is still within the actual recertification window, so no gap exists. C. This should be reported to NCUA immediately as a compliance violation. Or D, the vendor is contractually responsible for testing its own updates, so the institution has no exposure. So now you all listen to this. You'll realize pretty quickly which one it is, but especially since we've talked about trigger-based revalidations. So the answer is A, right? Recertification should be triggered by material model changes. Anytime that happens, it should be changed. Anytime there's no change to the model, it should trigger revalidation. When you're dealing with the fact that if you don't, right, compliant with a letter of the policy is the same thing as the model has actually been validated. That's not true, right? That's a trap. Two silent updates occurred inside that window, unnoticed. The policy underlying assumption was violated, right? You know that when you deal with two silent updates, they're telling you that, well, okay, it didn't have anything to say that these silent updates were actually dealt with a problem and there was some sort of resolve to that. They were just pushed out. And then another one, the correct answer isn't about breaking the policy, it's recognizing the policy's trigger condition is inadequate. So trigger revalidation. Okay, so the last question Shadow AI in a core vendor. During routine vendor risk review, you discovered that your core processing vendor quietly added, quietly is the key term, added an AI-powered fraud detection feature to its platform. Yay for it six months ago. No one at the credit union formally approved this addition, and it was never part of the original contract scope. What is your first step? Now this can happen, guess it show sure can happen, and it happens frequently. A. Since the feature improves fraud detection, no further actions needed. Yes, a win. B, wait for the annual review cycle to formally address it. Yes, waiting again for the review cycle, checkbox. C, immediately terminate the vendor contract for unauthorized scope change. That's possible, right? Maybe something you want to do. That's a little bit draconian, but maybe. Or D. Add the feature to your AI inventory and open a due diligence review. Hmm. So two of those, which one is the right one, right? A and B, eh, probably throw those out, right? But C and D, maybe. So the answer is D, right? Add a feature to your AI inventory and open a due diligence review. Now it depends how mad you are. If you're totally ticked and this has been something going on with your vendor, well then maybe you do fire them. But in this case here, it's your add this AI feature to your open due diligence review. So the scenario offers two emotionally satisfying extremes. Terminate the contract, yeah, baby, or do nothing because the outcome sounds much more beneficial. But again, both skip the step that actually reduces the risk. Finding out what the feature does, the data it touches, and whether it meets your standards is an important part. And that's where the due diligence review will come into play. You wait for the annual cycle. Yeah, that's always awesome. Just wait. Uh, but don't do it. Just don't do that. You need to get it reviewed, reviewed. So the correct answer is the least dramatic option on the list, right? Exactly why it's the easiest to underrate under time pressure. All right, so to pass forward, where do
you go from here? What do we do? So again, this comes down to AI governance, an important part of what we're doing. You build it for yourself. You now have the framework. Use the NIST AI RMF mapping, you can use TPRM workflows, you have validation lifecycles, all these things you can find, right? You can get all of this stuff online for you. You can find it through Chat GPT, you can get it in Claude. It'll step you through all of these different aspects related to what you may need. Or if you don't have the time for that, and in most cases, you probably don't have the manpower for it, you can build it with us, right? I can CISSP Cyber Training and Reduce Cyber Risk are there to help you with this overall plan. I can get you to find and help you with this. I'm developing an AI governance program that's going to be helping credit union security and compliance professionals. It's going to be in the beginning throes of it. It'll be coming out soon. But the bottom line is I'm here to help you at CISSP Cyber Training, or if I can't help you, I know quite a few people who can, and that we can definitely help you get done what you need to be done. So thanks again for joining me today on CISSP Cyber Training. I hope you got a lot out of this, a lot of great stuff when it comes to AI governance and the importance of AI governance within your organization. Again, showing you the articles in which how important it is, how that dealing with the CISSP and the different aspects that you're going to have to know as a cybersecurity professional when you understand AI governance, these are all things that you deal with on a day-to-day basis. And this is why the ISC Squared CISSP is so important. And this is why it's an important part for you to get this certification for you and your future career. Thanks again for joining me, and we'll catch you on the flip side.
SpeakerSee ya.
Speaker 1Thanks so much for joining me today on my podcast. If you like what you heard, please leave a review on iTunes as I would greatly appreciate your feedback. Also, check out my videos that are on YouTube and just head to my channel at CISSP Cyber Training, and you'll find a plethora or a conocopia of content to help you pass the CISSP exam the first time. Lastly, head to CISSP Cyber Training and sign up for 360 free CISSP questions to help you in your CISSP journey. Thanks again for listening.