The Lock & Key Lounge — An ArmorText Original Podcast

Out-of-Band or Out of Luck

• ArmorText • Season 1 • Episode 13

Share episode

Communicating When You Can’t Trust the Network

When adversaries can read the playbook—searching email, SharePoint, Teams/Slack, and even joining incident response calls—communication becomes the attack surface. That’s why the FBI now explicitly urges organizations to “plan to use out-of-band communications when normal channels like email and VoIP are compromised.” This episode is about preparedness: defining OOB comms, getting executive buy-in, and exercising realistic failovers so “if you can’t trust the network,” you can still run the response.

We’re joined by Aubrey Wade to explore why secure out-of-band communications have become essential infrastructure, and how organizations are building resiliency when it matters most.

Navroop Mitter:

Hello, this is Navroop Mitter, founder of ArmorText. I'm delighted to welcome you to this episode of The Lock & Key Lounge, where we bring you the smartest minds from legal, government, tech, and critical infrastructure to talk about groundbreaking ideas that you can apply now to strengthen your cybersecurity program and collectively keep us all safer. You can find all of our podcasts on our site, ArmorText.com, and listen to them on your favorite streaming channels. Be sure to give us feedback.

Navroop:

And today, we're concluding our special three-part mini-series that was inspired by the Chatham House Rule executive forum that we co-hosted with Clear and T-Mobile. We've already examined the legal and investigative angles of DPRK-linked remote worker deception. And now we turn to crisis response itself. In high-stakes scenarios, traditional networks can't always be trusted. So we're joined by Aubrey Wade to explore why secure out-of-band communications have become a central infrastructure, and to learn how organizations are building resiliency when it matters most.

This episode directly precedes our upcoming webinar Fraud, Fakes, and Foreign Threats:

Identity Verification and Secure Comms in the Age of DPRK Remote Worker Schemes. For those joining after the webinar, the recording link is available in the show notes. Aubrey, welcome to the show.

Aubrey Wade:

Thank you for having me.

Navroop:

Aubrey Wade is a certified MBCP and CRISC resilience leader whose program spanned crisis management, business continuity, disaster recovery, and operational risk. She's built and scaled enterprise resilience at firms like Regions Bank, where she's a Disaster Recovery VP; Voya Financial, where she led business resilience; and UKG, where she was a Lead DR Analyst. While at UKG, she integrated DR strategy with cloud transformations and developed measurable programs, metrics, and governance. Aubrey specializes in turning chaos into clarity, designing training exercises and communications workflows so teams aren't reinventing the process mid-crisis. Earlier in her career, she also taught and built curricula, experience that shows up in the way she operationalizes readiness for non-technical leaders and boards. And with that, a little bit more of what we're going to be discussing today—when adversaries can read the playbook (searching email, SharePoint, Teams, or Slack, and even joining incident response calls), communication becomes the attack surface. That's why the FBI now explicitly urges organizations to plan to use out-of-band communications when normal channels like email and VoIP are compromised. This episode is about preparedness, defining out-of-band communications, getting executive buy-in, and exercising realistic failovers so that if you can't trust the network, you can still run the response. And with that, let's dive right in. So, Aubrey, in practice, what does out-of-band communications mean to you? How is it different from just switch to Slack or Teams or consumer apps like Signal and WhatsApp?

Aubrey:

So I think it's great to start with the definition. We have lots of communication options available to us today through our phones, through our computers. Lots of offerings kind of on the table. But out of band is kind of a unique breed in terms of how we use it for incident response and crisis management. So when we think of out of band, we want to consider that it is completely separate from our kind of core infrastructure—Active Directory ID provider for the company. So it's something that is uniquely standalone. It's on its own trust plane with a distinct tenant. It doesn't have any ties to your infrastructure or things that bad actors could potentially hack in these types of infiltration events. So you think of things like Teams we use every day or Slack. Those things are integrated into your Active Directory. Those are through your identity provider. And those are things that, if someone has the right access and credentials, they can read, they can manipulate, and they can get ahead of you in terms of your incident response—if you were discussing remediation strategies or quarantine strategies and they're able to read those communications. Sometimes people think, well, I could just switch to WhatsApp—something I already have on my phone—or Signal. But those consumer apps lack policy control. It's hard to audit them, as they're on personal iPads or iPhones or Android phones. You have to go get privilege to people's personal devices. You can't control the roles that are in those phones, so everyone may have access to the larger discussion when you want to parse out or segregate who sees what.[00.04.40.04–00.04.59.21] You might not be able to control retention of the records being discussed on those consumer apps. And they lack federation boundaries. So those are not really ideal for this type of communication when we're talking about kind of a sensitive moment in a company's crisis response. And so it's better to have some tools like ArmorText that you can trust, that are out of band, but also have all these other capabilities that the consumer apps lack like the auditability, policy control, retention options, legal hold. So all of that said, that's why out of band is kind of a unique animal in this space. And so that's where we start, I guess, is the definition. It's also worth noting that it's better to have a solution selected, so choosing that out-of-band solution before you're in the moment. Because asking everyone to download and trust a solution they've never seen, and put confidential information or sensitive conversations into a net-new tool that you haven't had the opportunity to vet or test, is also not really the way you want to start or walk into these types of events.

Navroop:

Aubrey, I appreciate the way that you've described out-of-band communications. You've probably done a far better job than I do when I get overly technical about the topic. And so that actually takes me to my next question then, right? I often have to talk about what are the reasons an organization would want to have an out-of-band capability? Or what are the triggers for when they’ll want to use it? And I'd love to get your take on that, right, as someone who’s had to put these in place. What's the real trigger that tells an organization it needs a true out-of-band capability?

Aubrey:

So some triggers. Hopefully we're starting on our front foot rather than our back foot. So some things out there, like you mentioned at the very top, is that the federal guidance that recommends out-of-band communications for these types of remediation strategies and to bolster crisis management responses for organizations. Other standards that do these kinds of reviews for companies, like Gartner and Forrester, also have out of band as a part of their recommendation when talking about cyber and incident response. So ideally, companies are working to move toward these standards and be as resilient as possible and ready to respond. But if you were in an event and you had suspicion that your identity provider had been compromised, or there’s some discovery showing that maybe a bad actor is attempting to access email or SharePoint, or you see unusual faces on your incident response call, all of those things could be indications that your communications have been compromised. And it's not uncommon for people to see these when they're in these events—that suddenly they have concerns about the security of the communication channels that they are utilizing to triage and respond. Other tactics being used today, like MFA fatigue and executive impersonation, are also high on the list of things that would maybe make an organization think about investing in an out-of-band solution, to make sure that their cyber team is ready to switch gears and talk on a secure channel to respond to this. I think your famous tagline, which I've heard you say before in other discussions we've had, is "if you can't communicate, you can't remediate"—something like that.

Navroop:

Yes. The tagline. All right. Well, Aubrey, with that said, so an organization realizes they might need out-of-band communications, or there are at least portions of the team that have realized this might be the case. Ultimately, a lot of whether or not something gets adopted is whether or not you get the right buy-in, right. So what are the biggest internal hurdles to adopting out of band? Are they technical? Is it cultural? Is it executive buy-in? And regardless of what they are, how do you overcome them?

Aubrey:

So I think there are challenges in those three areas. And I'll talk about each of them independently. Executive buy-in, I think, is probably the first thing you need to make this move into out of band and framing it as critical infrastructure. Understanding the guidelines coming from standards and the FBI about out of band, and getting them to really get on board with why this tool is not a nice-to-have but linked to response time, crisis exposure, disclosure timing, and even some compliance pressure, especially in highly regulated industries. So kind of telling the story of why out of band is—what out of band is here to serve, what purpose it's serving for those executives—is the first hurdle. And then getting everybody else on board afterward is the second one, so that kind of cultural component. Having folks understand that, yes, you have Teams, yes, you have Slack, yes, you can talk to your friends on WhatsApp, but this is serving a different purpose and it is not yet another tool. It is not yet another thing we just bought because we want another tool. It has a very specific use case and is meant to reduce risk for threats that we know are out there today. So getting everybody else on board after the executives is kind of the second step. And then there are some technical components that you have to consider when you're implementing one. Once you've got that first initial momentum—so how do you create the break glass identity for an out-of-band solution that is not identity provider dependent? How do you get it on folks' devices so that they're independent? Again, separate trust plane for communication. And how do you provision this thing? I mean, to begin with, you may begin with a small audience to gain that culture buy-in exposure. But ultimately, during an event, you may find yourself needing a lot more bodies to help solution, quarantine, and triage, remediate than the initial rollout. So another technical hurdle that we've encountered is just how do we bring on the amount of resources we might need for remediation in an event? So I think those are the three primary hurdles I've seen. Technically, in the case of at least your tool, I haven't seen a lot of hurdles in terms of OS incompatibility or challenges from a person installing or anything like that, or the UX. All that seems fine. It's more about how do we broadly get this to our audience? How do we broadly get folks to adopt it and know when to use it and when not to?

Navroop:

Yeah, as always, that training—that muscle memory—is key, right? Investing in all that is critical. It's why we're gonna be talking so much about that. So, as you're preparing to use a secure out-of-band platform then, like an ArmorText, what policies, roles, and runbooks do you think need to exist before an incident occurs?

Aubrey:

This is a long one because there's a lot to consider, I think, when you're implementing a tool like this. So, I'd like to start with the who. So who needs to be there? From leadership—CISOs, incident response leads, crisis communication teams, marketing or PR, CEO or board liaison—these types of individuals that would be your first line of stakeholders into the tool. And then understanding the needs of each of those groups so you can actually compartmentalize the conversations and the retention and the review by subject matter or group, if necessary. So you can create a least-privilege onboarding inclusion into channels or exclusions. And so that also is one of the decisions that should be made ahead of time—of who gets to see what and who gets moved into which conversations. This has to be fluid to some extent. And it's—you can be fluid about it—but I think having a baseline of understanding of who's in the room, and who's talking about what things, and who has a visibility into what, is a good place to start. Because that's going to drive some of the preservation and retention controls. So what is going to be auditable? What can reviewers see? Reviewers are a role that is separate from the admin that has read rights to different conversations. What types of holds do you want to put on these conversations in terms of storing the data and archiving it? There's a lot to consider when we're thinking about the worst-case scenario and how we're talking about it, because at the end of the day, once all is said and done, a lot of this stuff can be reviewed and audited by regulators. So having those guidelines in place is really key, I think, to not the successful, necessarily, event triage—which we talked about before, which is communication—but surviving the aftermath. And so keeping those sensitive comms within least privilege for the different groups, and also the appropriate retention, there are also often needs to expand the audience when you're doing these types of incident responses to external third party. So having federated controls or understanding how you're going to onboard those individuals—and again, what information and what conversations you want them to be a part of—are decisions you can create policies or guidelines around. So you're not guessing in the middle of an incident. Gosh, trying to think of more things. There are so many things. Oh, there has to be a way to onboard everybody. Like I said before, there may be people that are already familiar with the tool, that have been exposed to it, but having procedures for an SSO bypass if you chose to put this on SSO. That has not been my experience. Typically, we're trying not to use the SSO provider. But using other methods to onboard folks that would get around using their primary email if those were offline. And ways to address, if someone has a lost device. So let's say you have people already involved in this tool. What is the policy for when they tell you, I don't really know where my phone is right now. How do you handle that? So having guidelines as to remediation and who's responsible, and speed of addressing those types of things, is also key. And the last thing that came up, when we met in—for the event—the panel event—a few months back was, how do you know everyone in the room is who they say they are, especially when you're onboarding a large number of people. And I know ArmorText has been partnering with Clear, and I don't know where this is, but knowing how to identi—verify the identities of those that are in the room, so that additional verification step to confirm that everyone who has logged on and is participating or observing—which participating is better—is who they say they are. And this is still a secure method of communication. So those are my my policy runbook tips for if you're going down this path of implementing out of band.

Navroop:

I appreciate that, Aubrey. I think on that last note there about the out-of-band identity verification, this is going to be the topic of the webinar. So that's a perfect tie-back to what this three-part mini-series is leading up to. It's a webinar where we'll be discussing out-of-band identity verification, especially in light of the fact that organizations are potentially going to be calling a bunch of outside parties in when an attack takes place. And you need to know that people who are supposed to be showing up from this third party that you've enlisted to help you are indeed the people who do show up, right? Or suddenly making sure that the attacker isn't impersonating one of your own people, and just leveraging the credentials or other access that they've obtained along the way to sort of join the conversation, right? And so, perfect tie in to what's coming up. But coming back to something else you were talking about when you were defining the who needs to be more involved—the GC, the CISO, PR and comms folks—one of the things you specifically mentioned was the CEO and the board. And I think by extension of that, your senior leadership. What we've often seen is that there is an adoption of ArmorText, and it then naturally moves from an IR-specific initial implementation to then also coming into security operations or threat intel sharing that involves more day-to-day use. And as a result of that, a lot of the folks who are more technical and are in security or part of the CISO's team, and sometimes even part of the legal, are using the platform far more frequently, almost day to day. And so fluency for them is—it's, well, it's second nature, right, to drop that muscle memory. But when we come back to the upper executive leadership, the senior leadership in the boards, how do you keep them fluent on how to use your out-of-band comms platform when this is likely something they're not already using day to day?

Aubrey:

So you're right. Senior leadership is busy, and they're not necessarily coming in to ArmorText just to have side conversations. In fact, we wouldn't want them to be. But having them educated on the tool is the first step. And not letting it go stale is the second. So micro exercises, mini drills too. It's kind of like a fire drill when you were a kid, right? Having them go into the tool, and having them test it out and have a conversation, confirm their understanding of it through them or their EAs. You can even put metrics to those, right? How long did it take us to get everybody in the room? How well are we adhering to the policy if we're having conversations? Is anybody actually sending an email, too? We're not supposed to be doing that. No backsliding into email. And having someone who's there to help them too—call it a concierge or that liaison with executive leadership who they're comfortable with, who can support them during their use of the tool so they don't feel like they're out of touch, don't know what's going on. Having that additional support to reset a password or guide them to the groups or have them understand what documents they could view or share. Within those exercises, maybe each exercise goes through a different component of the tool and the capabilities within it to keep it front of mind and practice. Practice makes perfect, but practice keeps us from forgetting as well what this is for, when we use it, and how important it is. You don't want to call an event a real event and have it take two hours to get the people you need for decisions on the chat or the phone. So practice leads to better time to decision-making, and thus improved recovery and remediation. So that's the thing I will preach until I'm blue in the face is practice, practice, practice.

Navroop:

I think Matt Welling, who has previously spoken about this topic, would agree with you. It all comes down to muscle memory, and that is developed by practicing. It’s like any other sport, right? So moving on, though, what does a realistic out-of-band exercise look like then? What failure modes should be simulated?

Aubrey:

We can look to activities taking place in the world to build our exercises. I think having real-world scenarios is really impactful and meaningful when we do those. So you think of email search, or a SharePoint access threat, or instances or news articles you can pull of adversaries or threat actors on these incident response calls in other organizations, if they've disclosed that. So pulling from real-world examples and talking about the potential threats or risks to email or SharePoint, I think, is a good place to start. You can also drill outages. We can make scenarios where we say, hey, we're having phone outages, the telephony is down. How are we going to communicate? Let's run that scenario specifically, or everyone is only permitted to work off of their cell phones. How does that look? What happens when our identity provider is fully offline, and how are we going to address that? How does out of band solve for that? And what other things do we need to support it? So I think realistic drills and a diverse set of drills are good ways to start thinking about your exercise portfolio for out-of-band communications, and make sure you're getting a lot of participation across the organization based on the scenario effect. In some cases, that may include different teams across your organization. In other cases, you may need key vendors or outside counsel. Maybe your cyber insurance would be involved at some point. So considering not just the core team when you run these exercises, but potentially expanding the conversation to those other groups that would realistically be involved in that scenario and that incident response.

Navroop:

That’s a great point. And it actually reminds me of something that was shared with us recently. And I'm trying to figure out the best way to say this without giving away too many details. This is something for those who are considering different tools or different types of tools for their out-of-band communications or their incident response coordination. We were alerted recently to a case where there was an IR coordination portal, and it was a browser-based tool. There's a web session, and ultimately, in that setup, the communications that parties were having could be centrally decrypted, right, anywhere. It was just simply encryption at rest, encryption in transit. Having the right credentials would open up the communications. And in the scenario that was very real, that was brought to our attention, one of these IR coordination portals experienced a situation where, through no fault of theirs, a set of outside counsel had their credentials exposed, right, during an incident where they were helping a particular client at the behest of—I think—of an insurer. And when they—when those counsel's credentials were exposed, suddenly the attackers gained access to every single other matter that the counsel was working on. So it wasn't just your matter, it was everyone else's matter. By "I say your," not literally yours, Aubrey. But the matter that they were working on for this particular client. Suddenly, all other matters that that counsel were working on were also exposed. And it might actually be an interesting thing to drill. How and when you encounter situations like that, what do you do? What are those next steps? And/or to feed that into the consideration around what kind of tools you do or do not want to adopt. Is end-to-end encryption high on your priority list as a result? If it is, does that mean that some of these browser-based tools aren't necessarily going to be high up on the stack, right, when you're looking at what you may or may not want to consider, or what you may or may not want to integrate, even if you are using an end-to-end encrypted comms tool. So, yeah, just in the absence of true end-to-end encrypted comms for incident response and tenant separation, you may actually want to practice some of those things as well. Actually, and come to think about it, we've been talking about these kind of runbooks and everything else, Aubrey, what should be in every out-of-band runbook?

Aubrey:

So if we're talking about the playbook of what do we do when this happens—not the policies for setting up the tool—but the actual incident response plan should include a couple of key things, which I don't think are going to shock anybody who's in the resiliency or cyber space or even the DR space, but I'm going to say them anyway. So activation criteria and triggers for activation. When do we switch tools? When is that decision made? Who makes the decision to move everyone from standard communication to out of band? And I think some of those triggers we already talked about—escalation paths and notification of other key members or stakeholders to make them aware of this pivot. The groups that we talked about and those kind of predefined sets of stakeholders are—should likely be in predefined channels, or those could be configured upon event launch. But having an idea of what those configurations look like, if those are not already defined and exercised, is another good thing to have written down. Who's on a need-to-know list? Guidance for alternative devices. If people need to pivot from one technology or—for out—Android to iPhone—or they have multiple phones, how do you get your alternate device online? And what's the guidance for that? Contact trace? I know, like a phone call. We're all rarely on landlines, but having those contact information for folks to get them moved over to the out-of-band communication tool and pivot off of perhaps your Microsoft Teams and Slack without putting that in Microsoft Teams and Slack is wise. There also should be probably a section of things we do and do not do when we move to out of band. So for example—and this seems silly—but if you have a conversation on your out-of-band communication tool, probably don't put that in an email recap summary or post it on a Teams channel if you're concerned that those methods of communication might be readable by threat actors, right? We don't want to be putting any more information to those primary tools if we've switched to out of band. We shouldn't be taking screenshots. That's in most companies' policies. But we could reiterate that in a playbook. And no passing out of tokens or keys. It's just like password sharing. These are kind of basic things. But again, putting them down again is always a good reminder, especially in a crisis moment when people forget themselves and are trying to work quickly and sometimes don't have that moment of, oh, I'm not supposed to do that. How can I best communicate this without using my standard communication tools? Lastly, I think one of the things that surprises folks when they're done with these incidents or they’ve—they feel like they’ve completed remediation, they’re out of the woods—is understanding the post-incident work. So having that in a runbook is also incredibly helpful. So what regulatory or legal needs need to be met post-incident? What exports of data need to be captured? How do we deprovision all of these users we might have onboarded to help support the event? And then everybody knows, after you have an event, you do a lessons learned and you put it right back into the plan to get better every time. So all those things come at the end. And once we get through the adrenaline of action, it’s good to have a checklist to remember all the other things we have to do to make sure that we leave the incident well, correctly, within compliance. Clean up this tool best we can, and then get better.

Navroop:

I love it. I think you've done a great job recapping all of the major points that were being discussed—without any of the attribution, of course, because it was a Chatham House Rule event. And so I really appreciate that. I'd like to actually ask you to put on your prognosticator hat now, right? Pull up the crystal ball.[00.28.31.04–00.28.42.07] Obviously, AI is a big discussion that everyone's having all the time now. And so I'd love to get your perspective on this. How does AI change crisis comms? Where does it help, and where does it potentially hurt?

Aubrey:

So AI is great. We use it all the time, but it does create new threats in the environment and new considerations for crisis comms. And I think of deepfakes or even the video and voice deepfake—things we can see appearing in the threat space. But it could just be a voice of someone who's an authority figure on a bridge line. They can be social engineering campaigns launched during an incident response. I think the mention of the legal firm that you referred to earlier—we're all, I guess, potential victims of a social engineering threat. And AI is making it even more challenging for humans to identify the real from the AI-generated. And I think there is this consideration for that when we talk about credentials and credential compromise, things like that. You also, depending on how your business runs—and we all love taking our notes out of a meeting or the transcript out of a meeting and summarizing it in ChatGPT—but that can lead to some unintended data exposure. And from what I've seen—and I don't know if this is true across the board—but many companies are implementing policies where they either require the AI to be their own tenant, and/or prohibit completely the use of AI for that purpose. Because there is too much potential for someone to accidentally reveal information about applications, ID provider, identity provider in those tools. So I think there are some things that open us up to threats that are not even malicious intent—that AI is providing a service, but is also creating a risk to organizations. That said, if used in the right place and the right time, it can help do that summary. If you have it contained, it can create timelines.[00.30.43.17–00.31.14.18] It can help through a discussion, build the action items, or we say Navroop agreed to do this, and by this day—isn't that convenient that I don't have to go look for what he said in the meeting? But I think those are kind of minor conveniences compared to the threat. So we need to consider it and take it a bit with a grain of salt, and the best thing I've seen organizations do is create policies around the use of AI, particularly external AI tools. And the same applies to incidents, right? It needs to be private. It needs to be vetted. And if you're not sure, we probably shouldn't be using it for incident response. Because, at the moment, there's too much risk for the maybe convenience of a summarization reward, if you will.

Navroop:

Aubrey, I could talk to you for hours. But we're coming up on our time. And one of the traditions we have here on the podcast is to end with a fun wrap-up question, and so if you've got time for one more, I've got a quick one for you.

Aubrey:

Sure.

Navroop:

All right. You've just led a flawless cut-over to out-of-band comms. You pull the team off of a noisy, compromised network, and you kept the response on track. The incident winds down. The war room empties. What's your celebratory libation of choice? And who gets the first toast?

Aubrey:

So I'm going to be a little snooty, but I think I'm going to have to go with a Rioja wine, because that's just a nice to have. And probably first toast is incident response lead, or whoever has been this me, organizing that cyber team for response remediation, though I would assume that they're probably drinking something stronger.

Navroop:

I would gladly join for the Rioja wine and/or the something stronger, which is likely a really nice whiskey. So I love that. Aubrey, we're grateful for your time and expertise. And with that, we wrap up our three-part series inspired by the Chatham House forum we hosted with Clear and T-Mobile back in June. Remember, if you can't communicate, you can't remediate. So make out of band a part of your muscle memory before you need it. Be sure to check out our upcoming webinar, Fraud, Fakes, and Foreign Threats. And if you're catching this later, the recording link is in the show notes.

Matt Calligan:

We really hope you enjoyed this episode of The Lock & Key Lounge. If you're a cybersecurity expert or you have a unique insight or point of view on the topic—and we know you do—we'd love to hear from you. Please email us at lounge@armortext.com or our website, armortext.com/podcast. I'm Matt Calligan, Director of Revenue Operations here at ArmorText, inviting you back here next time, where you'll get live, unenciphered, unfiltered, stirred—never shaken—insights into the latest cybersecurity concepts.