Fortifying Cyber Defenses: Commvault's Clean Room Recovery and the Art of Cyber Resilience
May 04, 2024
Evan Kirstel
Unlock the mysteries of cyber resilience with us on an enlightening journey featuring Commvault's Michael Stemp, as he elucidates how traditional disaster recovery methods simply don't cut it against the sophistication of cyber threats. We'll unveil the secret sauce behind Commvault's Clean Room Recovery service and why it's becoming the security blanket for businesses large and small. Learn how Michael's extensive expertise points to the necessity of a paradigm shift, from the ground up, to fortify your company's data against relentless digital dangers. It's not just about bouncing back; it's about preempting the fall.
This episode is a deep dive into the nitty-gritty of cyber recovery best practices and the latest trends shaping the industry. We scrutinize the absence of a one-size-fits-all approach in the cyber recovery playbook and how Commvault is charting the course towards setting industry standards. Our conversation with Michael Stemp, a vanguard in product and ecosystem strategy, will arm you with the knowledge to overhaul your cyber defenses, integrating the vital OODA loop and leveraging powerhouse partnerships with Palo Alto and Microsoft. Discover the pivotal role of dynamic recovery strategies within the NIST framework, and why robust cyber resilience can be your company's knight in shining armor.
Unlock the mysteries of cyber resilience with us on an enlightening journey featuring Commvault's Michael Stemp, as he elucidates how traditional disaster recovery methods simply don't cut it against the sophistication of cyber threats. We'll unveil the secret sauce behind Commvault's Clean Room Recovery service and why it's becoming the security blanket for businesses large and small. Learn how Michael's extensive expertise points to the necessity of a paradigm shift, from the ground up, to fortify your company's data against relentless digital dangers. It's not just about bouncing back; it's about preempting the fall.
This episode is a deep dive into the nitty-gritty of cyber recovery best practices and the latest trends shaping the industry. We scrutinize the absence of a one-size-fits-all approach in the cyber recovery playbook and how Commvault is charting the course towards setting industry standards. Our conversation with Michael Stemp, a vanguard in product and ecosystem strategy, will arm you with the knowledge to overhaul your cyber defenses, integrating the vital OODA loop and leveraging powerhouse partnerships with Palo Alto and Microsoft. Discover the pivotal role of dynamic recovery strategies within the NIST framework, and why robust cyber resilience can be your company's knight in shining armor.
More at https://linktr.ee/EvanKirstel
Speaker 1:
Hey everybody. Super important and informative chat today with Commvault around cyber resilience topics. Michael, how are you? Good sir, how are you doing today? I'm very well, just gearing up here at RSAC and, wow, there's a lot going on. I have lots of questions about Commvault's latest and greatest, but before that, maybe introduce yourself and your role at Commvault.
Speaker 2:
Sure. So, Michael Stemp, I am the Senior Director of Product and Ecosystem Strategy at Commvault, Been here for about 10 months. Prior to that I ran the data protection product at Sirius Computer Solutions and then CDW. Once CDW bought Computer Solutions, Sirius Computer Solutions and then CDW. Once CDW bought Serious Computer Solutions.
Speaker 1:
Wow. So you have a lot of insight to share with us today. One thing I'm particularly intrigued by is your new announcement around clean room recovery and its fundamental importance to cyber resilience cybersecurity Maybe start with some definitions for those who aren't aware who is Commvault at the 10,000-foot level. Talk about your latest news at RSAC.
Speaker 2:
Yeah, so the latest is Commvault Clean Room Recovery, which is a tool to help people actually test their recovery methodologies around a cyber event. You know, for so many years we tested disaster recovery plans, typically quarterly, yearly, whatever it may be. We did that for years, yet the funny thing was less than 1% of all customers ever actually had declared disaster recovery. Wow. And now you look at cyber recovery. You know last year it affected 66% of companies. United States had some sort of a breach 66% and yet almost no one is actually testing those cyber recovery plans.
Speaker 2:
And really the biggest reason for that is the complexity. Right Back when we used to do DR, it was typically two sites, very easy, you would cross, replicate. But nowadays you know you have AWS, you have Azure, microsoft 365, a couple on-prems, a colo. It gets very complex, very difficult. You have all these interdependencies between them. How do you even test that interdependencies between them? How do you even test that? It gets complex, it gets costly. And so Cleanroom utilizing Commvault's any-to-any data portability so I can take any workload and move it anywhere else and convert it. I take all those workloads from all over the place and I bring them up in Microsoft Azure. And what's great about that is I don't have to spend millions of dollars in infrastructure that sits in my data center waiting to be used. A couple of times a year I spin up my infrastructure I need up in Azure, I do what I need to do and then I throw it away when I'm done.
Speaker 1:
Wow, that's a pretty radical new approach. But talk about traditional recovery solutions. You know the playbooks that we're so used to for disaster recovery. I mean, what are some of the key features that differentiate? You know clean room recovery from. You know the approach that most people are deploying today.
Speaker 2:
Sure. So two things there. One, the traditional playbooks that most people use. They're just moving over their disaster recovery playbooks, and the problem with that is, you know, a hurricane is not going to encrypt your data. A tornado doesn't do malicious things to your infrastructure. It tears out a building or whatever the disaster may be, but you don't have to worry about whether your data is valid anymore or your infrastructure is valid. So it's a totally different methodology, a methodology, though, that's been defined for years and there's terminology around it, and we don't have that with cyber. So people are doing the best they can by using what's already there and they're failing miserably because of it, because you can't just be talking about speeds and feeds and getting things back as fast as possible, because all that does is lead to reinfection. So what we're really focusing on is the ability to align this directly with cyber. Bring in forensic analysis, bring in cyber insurers and look at the whole process and test it with those things in mind, testing into the fact that my network is suspect, my servers are suspect, my data is suspect, everything has to be suspect. And really, the problem with competitors and ours, the limited fashion they have.
Speaker 2:
People have what's called cyber vaults. These are always on-prem. These are using very specific hardware you have to buy from the vendor. So that's, you know, could be millions and millions of dollars that you're putting in there. Plus, it's not really taking care of the problem.
Speaker 2:
In a lot of these attacks we are seeing bad actors shut off the air conditioning system to the whole data center, right? So if your cyber vault is in the same data center, it just got powered down. We've had them threatened to set off the fire suppressant system. So, again, if it's in the same data center, it's going to be shut down. You know up to 30 percent of attacks cyber vault in a cage with biometrics and video. Then it's going to be suspect and it could be exploited during this attack. And that's one of the main reasons why we go to cloud with this. It is a brand new Microsoft tenant that we stand up when we do clean room. No one's ever been in there before. It's new passwords, it's new logins, it's just entirely clean. If the data center gets shut down with the AC or fire suppressant doesn't affect what's going on up in Azure. So it is a true clean room, separated from your production environment.
Speaker 1:
Wow, that's fantastic. You also talk a lot about the democratization of data recovery solutions. What does that mean? I thought every business now had data recovery, even my four-person business. So what does democratization mean for businesses?
Speaker 2:
Well, in order to continue on, again, you're not going to be able to stop the breaches. They're going to happen. Right, they've just got MITRE, one of the main organizations that do security, set up the methodologies for it. They're hitting everybody. So we look at it and say, all right, there's two types of companies in this world those that know they've been breached and those that don't know they've been breached. So if you're going to deal with what happens after the breach, we need to look at recoverability, and that recoverability is costly.
Speaker 2:
Right, again, most cyber vault solutions are all on-prem. They're specific hardware done in a specific way. It really the cost on this goes where most middle-sized companies just simply can't afford it and don't have the manpower, the expertise in order to actually pull off testing it to see if they're going to be able to recover in a real event. The way that we've made this available to everyone is no additional sunk in hardware costs. Right, you bring up the new tenant as you need it. If you're done testing in a day or two, that's a couple hundred dollars on Azure. You throw it away then and you're good to go.
Speaker 2:
We also don't mandate a certain level of licensing within Commvault, so we have three main licenses. Any of those three are available to use the clean room to do their testing. Because we think it's important, because, just because you're doing, you know our basic levels. Operational recovery that's oh, I lost an email or something. I still need to protect that data. I still need to know how to recover that data, and so what we're doing is we're allowing anybody that's in this journey from being a operational recovery sort of customer to a disaster recovery customer, bringing them into a cyber recovery outlook and being able to do cyber, which is going to be the primary attacks hitting them. We're helping them walk that journey through education and allowing them to use all the tools that are available within Commvault.
Speaker 1:
Wow, phenomenal approach. I also understand you're using AI within the clean room recovery platform to kind of improve the process, enhance the process. How does that work exactly?
Speaker 2:
Yeah.
Speaker 2:
So when you start looking at some terms, a term, you hear a lot out.
Speaker 2:
There is a blast radius report, right, Something that's going to tell you you know which machines were affected, what date were they affected. How do we bring this in? That could be something very easy to figure out if it was a smaller attack, but if this was a larger attack I mean we have people that have tens of thousands of VMs, let's say, a couple thousand of them are hit and they're all have to be recovered at different date and times based upon different telemetry and information. That's going to be very difficult to do and really extend this entire recovery process, and so we're utilizing AI that helps us figure out that blast radius report based upon our own anomaly and threat detection, as well as links into other systems such as like Palo Alto or Microsoft Sentinel, that will help be able to create these lists, feed them into our clean room, what's called recovery groups, our wizard that sets up the recoveries and allows us to get that clean recovery point. So the data that we're storing into the clean room is actually clean.
Speaker 1:
Phenomenal. Well, that's intriguing. Can't wait to see it in action. For so long, any best practices or baby steps you could suggest to businesses you know to upgrade, to modernize their you know their approach to cyber recovery, you know what are.
Speaker 2:
The first one, two three steps to get started on this journey. So excellent question Number one thing I will say is start utilizing the term embrace the breach. I was in the military and we had to embrace the suck and thank you, and it basically means just get used to being uncomfortable, get used to the fact that you've probably been breached, and if you go to work tomorrow with that mindset, the choices you make are going to be entirely different, right? So? So that's the first and foremost is understand that you need to have a strong security set up, but bad actors are getting through those, so be prepared for what's next. The biggest problem after that is really, though, that no one has really defined best practices or methodologies or, for that matter, even terminologies around cyber recovery. You know, for disaster recovery, we've had these terms like RTO and RPO. You can pretty much ask anybody in IT what is RTO, rpo, and they can say well, it's about business continuity, right, and maybe they can't say exactly, but they have an understanding. We don't have any of that right now, and so Commvault's actually teamed up with some major analyst groups, and you are going to see in the coming months that methodology and terminology and best practices being released, not from us, but with our sponsorship and our assistance, so people have that that they can do. And along with that is we are converting our services team into this, really this consulting group around cyber recovery, to help people walk the walk to get to that.
Speaker 2:
And it starts with some very basic things. We talked about the cyber recovery plans. Those have to change right. But then it also goes into tabletop exercises. Tabletop exercises in the last few years have kind of become a joke, right. Everybody knows it's once a quarter or whatever it might be, and you go into a room and there's donuts and there's coffee, and everybody sits down and you know exactly what's going to happen and it's all make-believe so nobody really cares. And so our way of kind of dealing with that is shaking it up a little bit. I'll walk into a room and I'll immediately throw four people out of the room. Might be the backup admin, who knows right, but you got to change it up a little bit. And then one of the things we like to do is we like to, instead of letting them, come in with a list of servers that we're going to test today. That's very well planned out. That's very DR-like.
Speaker 2:
A cyber attack is chaotic, it's unplannable, it's totally off the wall. So I'll go in, I'll write the server name on the back of each card and throw that deck of cards against the wall and whatever's faced up, that's the ones that were just hit today. So it's changing the methodology, it's changing that whole thing. And then you take that and you move it into what we call a noodle loop. It's a term that's been around for years in the military. It's observe, orient, decide and act. It's a feedback loop into testing. You won't know what you don't know until you test it. And so you take your modified cyber recovery plan and you go do a tabletop exercise and create chaos in those and then bring them into the clean room for actual testing and then go back to your cyber recovery plan with what you found worked and didn't work.
Speaker 1:
Wow. Your enthusiasm and insights there are fantastic. I can't wait to chat more at RSAC. And, speaking of RSAC, a lot of other, of course, partners and vendors that you collaborate with care to call out any of those partnerships. As we know, it takes more than a single vendor partner to protect a network these days. How do you work with other players in the industry, for example?
Speaker 2:
Yeah. So we have a whole host of them that we have been teaming with, especially over the last 12 to 18 months. I think critical to all of this is a good SIM store integration. We have those with multiple different vendors. Ours with Palo Alto. We're about to release another update to our Palo Alto integration.
Speaker 2:
Here's the thing we can't be everything to everybody, right. While we are a data security company, right, we can't be everything. We can't be a CIMSOR, we can't be a Canary, we can't be everything for everybody. So what we do is we do the things we can in support of that fifth pillar within NIST, which is recover right. So we're really making sure that our security. That we're doing means that, at the end, your last line of defense, your data protection product, will be alive and able to help you.
Speaker 2:
Other than that, we're relying on our partners. So Palo Alto, microsoft's been a huge partner with us as far as the teaming for the clean room. We also have Microsoft Defender integration. So as we bring things into the clean room, microsoft will be looking at things, because if you look at data protection, really what they do with anomaly and threat detection is they look for malware, whether that's entropy of a file or signatures or any of that stuff. We're really not looking for indicators of compromise and indicators of attack. That's a traditional security tool sort of thing. And so we're bringing in Microsoft Defender and Sentinel and Palo Alto to really augment that for us and bring in that deep security.
Speaker 2:
Here's the one thing I'll say is all of things around, a cyber recovery has to be about cleanliness. Too many times I hear people about speeds and feeds. Speeds and feeds are not what you want to talk about in a cyber event. The average recovery time for a cyber event is 24 days. Out of that, only seven days is actually recovery, so you have plenty of time. What you need to do is not reinfect your environment, and so everything we do and the partners we're choosing to partner with are all about the cleanliness of the data when we do that recovery of the data when we do that recovery.
Speaker 1:
Such an important insight. Thanks for that. So we're heading into RSAC as we speak. What are some of the trends and opportunities challenges you're personally on the lookout for this year?
Speaker 2:
Well, I think, with all the recent hits in that, I think one you have to understand you're going to be breached or possibly are breached, and so looking for that affirmation from the security experts who will be speaking and I know a couple of them that will be up there speaking on this is that you know there was actually an article I saw out there talking about how you got to nowadays MVP your security and really be looking at that cyber resiliency. That's what there's where you need to really be applying the new funds that your organization has. So I'm going to be looking at that. Ai AI is a big thing, but AI is difficult. Ai can get you in a lot of trouble depending upon how you do it.
Speaker 2:
We actually wanted to look at AI in a little bit different way. We do not want to look at what we would call data plane AI, where we are analyzing the customer's data and finding things out about it. We really want to look at that control plane. Right, how do we make what we do better? How do we make documentation easier to understand and to consume?
Speaker 2:
How do we come up with that blast radius report where it's all automated and we get rid of the false negatives and the false positives, which is the reason why so many people turn off the anomaly detection within data protection products out there. And then you know help us analyze the telemetry of a SIMSOR or other feeds to really find out what is that blast radius report? Where do I need to restore and what machines are associated to that? If I get a blast radius report and it says there's these 12 machines that were affected, machine number 10 might be part of some application group that has another 15 machines. I need some AI that's going to say, hey, don't bring back just these 12, but you also have to bring back these other machines to augment it, so you're actually bringing up the true application. So that's where we think AI is a perfect fit.
Speaker 1:
Fascinating. Well, I can't wait to stop by the Commvault booth at RSAC Check out Clean Room Recovery in action. Hope to see you there on the show floor and thanks very much for the quick chat. I know you're super busy with the team and I appreciate the insights and the time.
Speaker 2:
Michael Well, thank you for your time.
Speaker 1:
And thanks for watching everyone Check out the Clean Room Recovery Technology at RCC and I will see you there as well, on site seeing the latest and greatest. See you in San Francisco, everyone, take care.