The Rook Ep. 002: Your Compliance Program Is Not a Security Program

A clean audit doesn't tell you whether your company is secure. It tells you something much narrower, and the gap between what the audit answers and what executives read into it is where most companies are quietly carrying real risk.

In this episode, David Shaw walks through what compliance audits actually evaluate, the three places where compliance and real security pull apart inside companies (access management, detection, out-of-scope creep), what someone running a real security practice will tell the board, and the two questions every board should be putting on the agenda at the meeting after the next audit closes.

In this episode:

• What an audit actually answers, and what it doesn't
• Why the gap between the report and reality isn't a failure of the audit
• The three places compliance and real security pull apart: access, detection, scope
• What a real security practice looks like, versus a compliance program
• What someone running a real program will tell the board
• The two questions to put on the agenda after the next audit closes

Resources mentioned:

• SOC 2, ISO 27001, PCI, NIST, HIPAA frameworks

Connect with David Shaw:

• Website: corvus-cyber.com
• LinkedIn: linkedin.com/in/djshaw
• Email: david@corvus-cyber.com

The Rook · Corvus Cybersecurity · corvus-cyber.com · David Shaw, CISSP, GLEG

Transcript

The Rook Ep. 002: Your Compliance Program Is Not a Security Program 
 

[00:00:00] Hook 
 

David Shaw: The most dangerous moment in any company's security history is the one right after a clean audit. That's when the questions stop. The board moves on, and the things the audit was never looking at sit there, unseen, while everyone is congratulating each other on a report that wasn't looking for them in the first place. 

[00:00:20] Introduction 
 

David Shaw: Welcome to The Rook. I'm David Shaw. I've spent the last twenty years as a CISO across some of the most complicated and heavily regulated verticals. I created this show for the people who own cybersecurity risk but don't always know what good security looks like or how to tell when they're seeing it. 

David Shaw: This show is for anyone who's ever read a security report, a vendor proposal, or a compliance document and thought, "I don't actually know what I'm looking at." If you're new here, hit subscribe so you don't miss an episode. If you're returning, welcome back. Today we're talking about why a clean audit and a secure company are not the same thing. 

David Shaw: We'll look at what an audit actually evaluates and what it doesn't, where compliance and real security pull apart inside companies, and the two questions every board should be asking the person responsible for security after the next audit closes. If you've just paid for an audit or you're about to, or you're sitting on a board reviewing one, this one's for you. 

David Shaw: So let's get into it. 

[00:01:21] What Compliance Audits Actually Do 
 

David Shaw: Let's start with what a compliance audit actually does, because the answer is more specific than most people realize. Every framework asks a defined question, and the question depends on who's asking. Some frameworks exist because your customers want assurance. SOC 2 and ISO 27001 are the two most common. 

David Shaw: And they answer the question your customer is asking when they hand you a contract: "Are you handling our data the way you said you would?" Others exist because a regulator or contractual authority requires them. PCI applies, for example, if you touch credit card information, and the card brands set the rule. 

David Shaw: HIPAA, GLBA, and CMMC are written into law or federal contract, and they cover health information, financial information, and defense work respectively. The list goes on, and most companies of any size are subject to more than one. But none of them is asking, "Is this company secure?" That's a much bigger question, and even the audits that try to answer it run into the same wall. 

David Shaw: You see, an audit captures a point in time. It's looking at evidence from a period that already happened. Security isn't a point in time. It's real time. The audit looks backwards, and the threat is looking right at you now. Inside any framework, you and the auditor agree on the scope. What the systems are in, what processes are covered, what time period the evidence applies to, what kind of evidence is acceptable. 

David Shaw: Everything outside of that scope isn't part of the audit, by mutual agreement and by economic necessity. Auditors don't review what isn't paid for. And what you get back is a report that says something specific. It says, "Against this framework, in this scope, this company met these requirements during this period." 

David Shaw: And that statement is true. It's also narrow. most of what determines whether a company is actually secure sits outside of it. And here's where the gap surfaces. The audit is answering a defined question, and executives read it as the answer to a different one. The company may pass the audit and get the certificate, but still have serious security issues that the audit was never looking at. 

[00:04:02] Where Compliance and Security Diverge 
 

David Shaw: Now let me walk you through three places where this shows up. The first is in access. When an employee leaves a company, the off-boarding process removes their access from the systems on the standard list: email, the corporate single sign-on system, the production systems that go through that, the laptop, the VPN. 

David Shaw: An auditor will sample termination events from the audit period and look for evidence that this work happened. That's a routine check, and most companies pass it. What an audit can't easily catch is the systems that never made it onto the standard list in the first place. The Trello board that the marketing team set up two years ago with a personal email. 

David Shaw: The Stripe account that the founder created back when the company was just three people. Or the Figma seats that are provisioned by direct invitation. The shared password vaults that nobody fully inventoried. these tools accumulate over the life of a company, and they hold real access to real customer or financial information, and they typically don't show up in any off-boarding workflow because nobody has a complete picture of where all of them are. 

David Shaw: That's what people in my line of work call identity sprawl. And it's not a failure of the audit. It's part of the company the audit was never pointed at. 

David Shaw: The second is detection. Compliance frameworks require an incident response plan, and they require evidence that the plan was tested during the audit period. They don't usually test whether the system that's supposed to alert your team when something goes wrong, the system that pulls together security logs from across the company and watches for suspicious patterns, they don't tell you whether that would actually catch a real attacker. 

David Shaw: Because the system can be running and logs can be flowing, alerts can even be configured, and none of it would fire on the actual techniques most intrusions use today. The audit confirms that the system is in place. What it doesn't do is usually test whether it would have caught anyone. Now, the third is what I call out-of-scope creep. 

David Shaw: Most audits cover defined scope. It's things like your production environment, customer data systems, the corporate sign-on system. But what's typically out of scope are things like marketing standalone subscription tools, the sales team's add-on integrations or the engineering team's experimental subdomain, personal cloud accounts where developers occasionally test things. 

David Shaw: These systems can hold real customer data, real credentials, and be real attack paths, and they remain entirely invisible to the audit. But notice what these three things have in common. They aren't failures of the audit, and they aren't things the auditor missed. They're things the audit was never designed to look at in the first place. 

David Shaw: So the report is honest. It's the reading of the report that's what's off. And here's where this becomes dangerous. Because once an executive starts to accept a clean audit as evidence of security, they start asking fewer questions. The board hears fewer concerns, and budget for things outside of the framework get harder to defend. 

David Shaw: So then the compliance program becomes the security program by default rather than by design, and the things the audit was never going to surface accumulate quietly underneath. But there's a second kind of accumulation happening too, and it's just as quiet. The controls that the audit did confirm were working on the day the auditor checked. 

David Shaw: The process that ran on cadence during the audit period was running on cadence then. What happens for the eleven months after the auditor leaves is a separate question, and most companies don't have a great answer for it. Because controls degrade, processes slip, people leave, new systems get added, and by the time the next audit starts, the company is not the same company the last audit looked at. 

David Shaw: The certificate on the wall is a year old, and so is the evidence behind it. This is the difference, and it's worth saying plainly. A compliance program is about evidence at a moment in time. A security program is a living thing. A compliance program asks, "Can we show this was true when the auditor looked?" 

David Shaw: A security program asks, is this still true today? And will it still be true tomorrow? One is a snapshot, the other is a practice. And when a company stops being able to tell the difference, the snapshot is what gets maintained, and the practice is what quietly stops happening . 

[00:09:13] What a Real Security Program Looks Like 
 

David Shaw: Okay, so if the audit isn't good enough, what is? Here's the difference, and it's the whole episode in one idea. An audit is about what was defined. A security program is about what can hurt the business. Those are two different shapes of work. An audit asks, did the company meet these requirements in this scope during this period? 

David Shaw: It's a fixed question with a fixed answer. A real security program asks three different questions, and it asks them continuously. What can actually hurt this business? How would we know if it were happening, and what are we doing about it? That's the loop. What can hurt us? How would we detect it? And what do we change to deal with it? 

David Shaw: The audit confirms a slice of the loop once a year. The loop itself is the work. So let's start with the first question. What can actually hurt this business? Not what the framework asks about, what the business depends on. What would cost real money or real customers or real trust if it failed? And what an attacker would actually go after if it were trying to harm you. 

David Shaw: That requires a current picture of what you have, who has access to it, and what would happen if any of it stopped working or fell into the wrong hands. And most companies have part of the picture for the systems that were in their last audit. A real program has it for everything that matters to the business, and it keeps it current. 

David Shaw: Now, the second question: How would we know? if an attacker is operating in your environment right now, would your team see it? And how fast? If a system that handles customer data started behaving strangely, would anyone notice? If an employee's credentials were being used from somewhere they shouldn't be, would the alert reach the right person? 

David Shaw: A real program has tested the answer to that question against how attackers actually behave today, not against the framework's checklist. The audit can confirm the detection system is in place. It usually doesn't confirm it would catch anyone. Now the third question, what are we doing about it? When something goes wrong, who acts on what authority and with what information and how quickly? 

David Shaw: A real program has rehearsed this with the people who would actually be involved, with the actual contact list, the actual systems, and someone deliberately making it harder than it would be on a good day. The point isn't to prove that the plan works. The point is to find out where it doesn't before something real is happening. 

David Shaw: And when something does change, whether it's a new threat, a new system, a new acquisition, the program adjusts. The loop runs again. That's a security program, a continuous practice of identifying what can hurt the business and then building the capacity to detect it and adjusting what the company does in response. 

David Shaw: The audit doesn't replace any of that. The audit just confirms a small defined piece of it once a year. 

[00:12:53] When the Audit Becomes the Practice 
 

David Shaw: So when a program like this is in place, the audit changes shape. The controls you document for the auditor are the same controls running in your business every day. The audit confirms what is already true, and the certificate is a byproduct, not the goal. Companies that operate this way notice something at audit time. 

David Shaw: The audit gets easier every year, not harder. There's no scramble, and there's no eleven-month gap to close. The certificate arrives because the practice is real, not because the practice got rebuilt for the auditor. And just as important, you can ask different questions of the people responsible for security than the ones the audit was built to answer. 

David Shaw: Ask them what the audit didn't tell you. Ask them where it was pointed. Ask what threats wouldn't have shown up because of the kind of evidence that the framework requires. Treat the audit silence as information, not as comfort. 

[00:13:55] What Good Looks Like 
 

David Shaw: Now let me talk about what good looks like from the perspective of a person responsible for security because what you should expect to hear matters as much as what you should be asking. Someone running a real security practice will tell you uncomfortable things. They'll tell you that the incident response ex-exercise you ran for the audit was just too quick. 

David Shaw: It's probably not meaningful, that nobody could find the contact list when something actually goes wrong, and that the person, responsible for updating the playbooks hasn't been updating those for the last several months And that there's no clear chain of authority when the executive who's on call is unavailable. 

David Shaw: They'll tell you which client systems are still in use but weren't in scope for the audit, and what that means for your actual exposure. They'll tell you that your policy about employees using their own laptops and phones for work exists on paper but has no real enforcement, so you don't actually know what's connecting to your environment. 

David Shaw: Someone focused on compliance or conforming to a standard will walk you through the executive summary, they'll point to the clean opinion letter, and they'll tell you the program is in good shape. They might mention next year's scope, but they probably won't mention what wasn't in this year's or what's silently causing them concern that the framework doesn't ask about. 

David Shaw: The difference between those two responses isn't capability. It's whether the person has been given the room to surface what they actually know. 

[00:15:36] Two Questions for Your Board 
 

David Shaw: Before I close out, I wanna leave you with something usable. Two questions. I want you to put these on the agenda for your next board meeting after the next audit closes. Direct them at whoever is responsible for security at your company. That might be an internal lead, might be a third party, might be a person wearing many hats. 

David Shaw: The point isn't to test them. The point is to put them in a position to tell you what they already know. So the first question is this: If we ran our audit today, what would we have trouble with, and what are we doing to test that to make sure that it stays covered? that question forces a specific answer. 

David Shaw: Someone running a real program will name the control that's been degrading or the process nobody's been running on a cadence or the evidence they couldn't reproduce today if they had to. And critically, they'll tell you what they've been doing about it. Someone chasing compliance will tell you, "We're fine. 

David Shaw: We passed. We'll be ready for next year," or something similar. Now, the second question is this: What issue or control are you most concerned about that wasn't covered in our most recent audit? Now, that question forces a different specific answer. Someone running a real program will name the systems that weren't in scope but should've been, or the threats that don't fit cleanly into the framework that you're using, or the policies that look good on paper but have no enforcement. 

David Shaw: Someone chasing compliance will tell you, "Ah, the audit covered everything material," which it didn't, because no audit ever does So write these two questions down. Put them on the agenda and listen carefully to what comes back. When someone answers both questions with specifics and uncomfortable truths, they're running a program that's worth investing in. 

David Shaw: Someone who answers both with reassurance is doing what the company has rewarded them for doing. The most useful answer you can get is the one that makes you uncomfortable, the one where they tell you what isn't working, what isn't covered, and what's keeping them up at night. If you ask those two questions and the answer is reassuring, you likely don't have a security program. 

David Shaw: You have someone who has learned that the audit is what gets rewarded. 

[00:18:03] Outro 
 

David Shaw: Well, that's episode two of The Rook. If you take one thing away from this episode, take this. A clean audit and a secure company are not the same thing, and the difference is likely more dangerous than you think. Now if this episode was useful, the most valuable thing you can do is send it to one person on a board, an executive team member, or a deal team member who needs to hear it. 

David Shaw: That's how this show grows, and it's how the work reaches the people that it was meant for. New episodes drop every other Tuesday. I'm David Shaw. This is The Rook, and now it's your move