Cisco Manufacturing Leaders

Weakness at your boundaries

October 03, 2019 Cisco Manufacturing Leaders Season 1 Episode 4
Cisco Manufacturing Leaders
Weakness at your boundaries
Chapters
Cisco Manufacturing Leaders
Weakness at your boundaries
Oct 03, 2019 Season 1 Episode 4
Cisco Manufacturing Leaders

Visit our website >
Read this episode's blog >
Where are the boundaries for threats to your operational systems? Who should be involved when responding to a cybersecurity incident? In this episode...

  • Learn what boundaries are and why your strategy depends on them
  • Build a realistic action plan to respond to threats and prevent damage
  • Discover where the majority of attacks are happening today
  • Use your organization and processes to build an effective execution plan to mitigate increasing risks 
Show Notes Transcript

Visit our website >
Read this episode's blog >
Where are the boundaries for threats to your operational systems? Who should be involved when responding to a cybersecurity incident? In this episode...

  • Learn what boundaries are and why your strategy depends on them
  • Build a realistic action plan to respond to threats and prevent damage
  • Discover where the majority of attacks are happening today
  • Use your organization and processes to build an effective execution plan to mitigate increasing risks 
Speaker 1:

Hi everyone. Thank you for joining us today on Cisco's manufacturing leaders. This podcast is presented by Cisco and I'm your host, Caroline Hila . Cisco's manufacturing leaders is designed to help those involved in the manufacturing industry make better decisions for their business. My goal is to bring you the best industry knowledge and expertise that's available to help you understand the latest trends, best practices, and more. Most importantly, I want to help you solve your unique problems and find new ways to gain a competitive edge. Welcome to Cisco's manufacturing leaders. Today's episode is called weakness at your boundaries, a realistic cybersecurity assessment. On today's episode, we're going to talk about boundaries, what they are, why they matter, and why your security strategy depends on your ability to protect them. We'll also look at where attacks are happening today and how you can prevent them within your boundaries. And lastly, we'll explore steps that you can take today to determine who is responsible for what in your organization when an incident occurs. To start off, I really want to help you first understand what is at the heart of this problem. You know what is motivating manufacturers to improve their security strategy and really start prioritizing investments in things that can protect them and I found a lot of my answers to these questions in the Kaspersky 2019 state of the industrial cybersecurity report. By the way, I highly recommend reading this report. They do it every year. It's an annual report and it's always packed full of really good, interesting survey data from the line of business executives and they also provide a lot of new information on trends, so definitely recommend that one in this report. One of the questions asked in a survey was what are the top three concerns you have in case of a cybersecurity incident and here's the results. 63% cited environmental damage as their number one concern. 84% said injury or death and number one was 87% reported reputational damages . Their number one concern, and I find these answers pretty eye opening because I think a lot of us assume that manufacturer's concerns are only about losing intellectual property or about operations shutting down whenever it comes to and it's security incident. But what we're really seeing in the results is that these decision makers in the industry are much more focused on what happens after the incident. But the key to really preventing these incidents in the first place is by having a good understanding of any areas of vulnerability that you have in your environment. So here's another really interesting point from this report on this. Four in 10 companies surveyed stated that they have not experienced any cyber incidents within the last 12 months. This percentage is lower than the 51% recorded in 2018 and the report even points out that this appears to be a negative development, you know, but it is possible these companies were unable to identify all the incidents in 2018 so in other words, the attackers are actually just getting smarter and harder to detect. And the report says the higher use of intrusion detection solutions today may expose more cyber incidents than were visible in the past. Instead, it is more likely that they employ little or no anomaly detection to detect dangerous or suspicious network traffic. Now I'd like to bring on our distinguished guest speaker, Robert. All back to help us break this down a little further. Robert is responsible for coordinating the direction of Cisco's industrial security efforts across business units and security research teams. He joined Cisco in 2010 when he defined and delivered three network security solutions with the most recent Cisco's first industrial security appliance. Prior to joining Cisco, Robert guided the IPS management solutions for intrusion prevention pioneer tipping point. Thanks for joining us today. Robert. To start off, can you explain what type of boundaries exist in the manufacturing environment and how they relate to industrial security?

Speaker 2:

There's a lot of different boundaries and there's two that I really kind of focus on. The first one, Hey , we're a Cisco or a networking company, so we tend to look at the network topology and we often will apply this Purdue model , um , in the manufacturing space in which we've got a levels five and four up in the classic enterprise environment. And then we've got three, two, one and zero say on the factory side. Uh , typically between that enterprise and the factory, we hope to see not often enough, unfortunately a , a, a DMZ , uh , in place, which basically separates the factory environment from the rest of the world, including the , uh, the it dominated enterprise side. So that's kind of a high level organizational boundaries, set levels three on down the factory. And this should be owned by OT. That's at least traditionally with it responsible for four and five. So that's one view of boundary sets. Um, the other set , um, happens to be maybe in a network topology perspective, but we're say, do you , we've got access, which is we've got aggregation switches and eventually we have the data center and then we've got that DMZ again, and then we sort of repeat that process up to the other environment. So that's another set of boundaries where people look at things from the perspective of networking, networking alone. And then the one that I really think doesn't get enough attention or organizational boundaries. And this is where we look at different groups , um, and sometimes individuals within groups who are being held responsible for this part of the network. Or this part of the process. And this is the area that I think is contributed to some of the challenges that have arisen particularly over the last three years. So again, different kinds of boundaries, different sets of concerns and different kinds of results with regards to industrial security in the manufacturing space.

Speaker 1:

So how would you describe the relationship between operations and these organizational boundaries?

Speaker 2:

We've got a set of people who are traditional , um , operations and they are manufacturing engineers, engineers of different types of chemical engineers, perhaps , uh, industrial engineers, mechanical engineers, electrical engineers, I , and they've all sort of gone through their career ensuring that the operation, the process continues and keeps working and doing its thing so they know their process, they understand the equipment, the machines that are on place and how these things largely interact to a certain point. Um, however, how they interact is typically going to be done over a TCP IP network and possibly some UDP there as well. But that is very much a tried and true [inaudible] technology stack. Those processes are actually monitored and controlled by a very normal it stack, which has a bunch of windows boxes. And so we then get to this question of, well, who really understands those processes ? There's no doubt that at the lower level it's your manufacturing engineers have all their different stripes and such. Um, there's a lot more question when you get to the top of the factory and you're looking at something like factory talk, which is a.net application running on windows. It utilizes Microsoft sequel server and everything up there, including the network infrastructure may look identical to what's running over in the HR department on the enterprise side. So this is where we kind of get into this, you know, who's in the best position to help maintain the resiliency and continue operation those assets, including it security. Yeah, absolutely. So that kind of brings me to my next question then. When something happens, when an incident occurs who own that in the industrial environment or who said that , who should own it? So there's a lot of variety across different corporations. I've seen , um, entities at wholly say, look below this industrial firewall within the factory. Everything is wholly owned by the factory operations people and it folks, you know, thank you very much, but kindly please, you know , take care of the enterprise side. Um, that I see changing. And so about three years ago I did a survey of about 600 folks , um, associated with the , uh , manufacturing and utility space. And what we found was that things were split between a third was totally by the OT team, maybe a third by the it group. And of course, depending upon which assets we're talking about. And then a third of them were, we're still trying to figure this out. So who should own it? Um, that's going to be an internal discussion, but I've got a general guidance, which is if the technology stack is unique to say the operations team, then chances are the operations team needs to wholly own that. However, if the technology stack at play such as a whole set of windows boxes and Microsoft sequel server, there's probably a good chance that the it teams likely to have more experience. And traditional success and knowledge of the tools actually make all of that , um, resilient and secure. So really what do people do today? I think it's in transition for a number of reasons, mostly having to do with some incidents over the last three years. Um, but what has been going on is a bit of a mix and again, I think it's really in transition. That makes a lot of sense. Then if it's in transition, especially being an internal discussion. Can you kind of explain to you what's at stake here and why this is worth talking about, especially within their organizations? Well, I think what's at stake really came to light. I mean the usual security concerns of, gee, someone's going to steal my loan , but actual property, someone's going to manipulate my process and caused problems. Um , or just flat out someone's just here to cause harm. And it doesn't matter what the harm looks like or is this going to cause problems, if you will? Uh, so what's at stake traditionally has been somewhat hypothetical. We always used to really focus on the things. Here's the sensor and someone's gonna manipulate the sensor outputs. Here's an actuator, someone's going to change the actuators behavior and here's the controller, and that's the point where I'm going to manipulate all those things. Those types of attacks have happened, but they're very rare. And the tax that mostly have happened in that case , uh , at least within commercial manufacturing environments, I've actually sailed a , they tried to , uh, address some safety instrumentation systems over in Saudi Arabia. Um, and that attempt actually failed there, but it could have been incredibly devastating. The safety instrumentation system. However, what happened, and they really changed the attitudes , um, started in 2017 in which we saw worldwide massive outages and industrial systems ranging from transport with people like federal express and Maersk shipping to manufacturers such as Merck and mandolins. Uh, Honda field Reno is others who basically had their factories shut down or had performance outages. All of this happened because of incidental infections that made their way effectively from the finance department over into the factory. So factories were shut down worldwide, I think it was like $14 billion in losses that were estimated again by incidental accidental infections. And so people started asking the question, Oh my gosh, how the heck could all of this manufacturing worldwide get shut down? And the answer became, well we had a whole bunch of unpatched systems. Vulnerabilities were known, it was all on windows and we just didn't have the right people involved at the right places. And the result that we see now, I'm going to extend that a little bit, 2019 bad people notice what happened in 2017 and said, look what happens when something happens by accident. Entire factories shut down. What is someone are to actually try. And now we have ransomware attacks explicitly targeting industrial systems like hydro Norris can , aluminum manufacturer and others. And so now we see people trying to profit explicitly on this organizational boundary problem. And the question of a whole bunch of 'em unpatched windows boxes running the factory. So it sounds like they're really vulnerable in wide variety of their different boundaries within the organization. Not necessarily just one. Exactly. And so it's easy enough to say, here's a vulnerability. It's windows based and here's a piece of malware, which there's thousands of examples that could abuse a windows vulnerability. That's the technical side of it. Um, the bigger challenge in security is really more about process and that process is going to be driven by organizational decisions. So the fact that we see it not being really allowed to focus on what looks very much like a typical I T malware problem , um , is something that organizations are going to have to think twice about. And how would you then recommend aligning those skills and responsibilities so that it is able to help solve those problems? Well, there's gonna have to be some trust and some boundaries actually you're going to need to be , um , torn down. Some boundaries need to be established. And so what I mean by that is we're going to need to look at the technologies that are at play, understand what's unique about them and then what's not so special about them. And then align those with the skillset to best understand how to keep them up and running. So in the case of these windows based applications, things like factory talks from Rockwell wind CC from Siemens and Emerson Delta elevation and all these other types of applications that are out there. They're dot net applications. That means windows. Um, and the it department has been securing and maintaining that for years. That's a good fit. Now there are windows based HMI down inside the machinery, but these are very special case, a windows environments . Those probably need to be kept under the control of the operations team because we're getting down to the machines and the cells. This is the space that the it department doesn't have a lot of experience with. And placing them into that role of going in and figuring out what these things are and potentially taking actions is probably not something your normal it team is ready to tackle. So my basic advice is to look at all of this as a process. Bring in the it teams and let them know that at the top of the factory things look like typical it equipment. That's a great place for them. And it happens to align with exactly the kinds of attacks and abuses we see today. As you get closer to the factory floor in more specific non it looking pieces of equipment needed to go a lot slower. So let's the OT teams need to speak with the it groups and said, look, here's where we want your help and perhaps over time we'll bring you down to help us out in these other environments, but start at the top, start a DMZ, separate the factory from the rest of the world including the enterprise side, come help lock down these windows based controls and below that that will be the OT groups and based on how well we're working together, perhaps you're going to bring the it folks down to maybe start managing some of that more sophisticated network infrastructure and even farther, but it's going to take time and it's going to take trust and part of that trust will be built by having some early successes in places that look like technologies that the it team is used to maintaining and securing already.

Speaker 3:

That's a really great point. I really liked that you brought up to the trust with between the organizations and how important that is within the company. Um, and some that you pointed out when you're talking about these attacks, I'm curious to hear your perspective on where do you see people focusing today versus where do we actually see majority of these attacks occurring?

Speaker 2:

Um , sure. And that's really , um, part of the problem that we've , we've had this myths , you know, the sort of misalignment. Um , starting back, I hate to bring up Stuxnet, but there was Stuxnet and someone did attack and abuses on explicit controls and equipment. Um, basically on the facts , you know, and a factory for environment. Um , very dramatic. A lot of things. And people I think over-rotated, I mean they put too much focus in response to what was a very unique set of attacks and circumstances. Uh , nation States a weapons grade uranium enrichment, that's a different set of concerns versus somebody cranking out widgets and other types of things as most of the world's manufacturing space looks like. So I think a lot of tools were developed and a lot of practices were focused on that Stuxnet story. Um, we then had the incidents that took place in the Ukraine and those also had some industrial elements. Um, some , uh, analog digital converters at play, other pieces of equipment there as well. But the path to get there, the following the kill chain involved a whole lot more windows typical tools at play long before anyone ever touched or abused a OT piece of equipment. So this is the point that I think we've got that influxion now is that people are really looking at the whole kill chain, the kill chain being the process of initiating and attack, which there's all kinds of different acts that take place on different kinds of platforms. And even in the case of the attack on the safety instrumentation system , um, the individuals who did a lot of that investigation came back and said there was like a hundred units made , been anecdotal, but his point was well made. There was a hundred different compromises of different pieces of equipment along the path before they ever touched that OT unique safety instrumentation system. And his statement was of those hundred 99 of them are windows boxes, classic by T infrastructure. So by over rotating on the 1%, you miss really 99 chances to detect the behaviors that really signaled this oncoming attack and the chance to shut them down in advance. So I think we're now seeing a balance where people are realizing that it's a lot of classic it infrastructure that's going to be abused and attack using classic it attack tools , um, that really should gather people's attention , uh , before they get down to the, now let's talk explicitly about that final stage. So I think that's really the transition that we're seeing. And a , it's a positive one. I mean, balance is being restored to , uh , that cyber security , um, focal area within manufacturing.

Speaker 1:

Yeah, that's a great point. And it really sounds like this is more of a group effort that requires multiple stakeholders across the organization to not only identify the risks but also to build an execution plan for mitigating them. And for those that want to make sure their boundaries are secured and that they have the necessary processes in place, where should they start focusing?

Speaker 2:

Well I focus on the organization and let's make sure that um, there's a real listic understanding of who's responsible for what and then ensuring that the who's responsible for what, that the Watts and the hooves are set up for success. If it looks like an it set of assets, then the it groups probably your optimal environment. If it's the OT side, then chances are your OT process engineers are the best people, but have that conversation, consider cross training as well. Um, if you've got OT people who are interested in pursuing it, there really shouldn't be a lot of problem in cross training them, giving them some basic it security knowledge, that place and putting it in place. Um, a lot of, you know, my analysis was done, brought about by my niece, Katrina is going to go and study chemical engineering and we went to look at the curriculum and it turns out that there's only two hours of compute platform training within her for your engineering degree. Um, that doesn't have to be the end of education if she were to pursue a role within a process plant somewhere. And she decides that perhaps a cybersecurity for pro for OT environments is good for her. She should have the opportunity to trust correct , um , to cross train. And she needs to do that probably with someone with an it background. So to summarize, let's make some decisions about who's going to be responsible. Let's marry up the responsibilities with the skills and the assets that are at play. Take the opportunity to cross train. But the OT people have an opportunity to learn about it , security skills. But the it people have the opportunity to work with the OT groups to understand the processes. Um, but folks are going to have to work together and there's really just no way around it.

Speaker 3:

That's a really good point. And I kind of have another question to bouncing off of what you said with cross training. I think that's a really great method to helping both sides understand where they are at risk. And you know what I've been seeing, especially in the research is that it in operations having different priorities and you know OT in a sense feeling sometimes threatened by the it team coming in and needing to patch something up. Do you think that cross training helps with that so that they can also understand they're vulnerable? Sure .

Speaker 2:

Well it, it helps in a couple areas. In some areas for the it people it helps them to understand how their standard practices are potentially harmful within the operational space. The OT folks can express that and say, by the way, if we see you running and in that scan at the normal rate of scans, these particular pieces of equipment won't be able to handle that. So this is how it folks, some of your practices need to change to reflect this environment. Similarly, the OT people working with the it folks can come to learn what they have. The ability to do and how they can vary it. So back to that end map scan again , they can actually tweak the depth of the scan, they can eat and tweak the rate at which the scan takes place. And thus using a typical it tool in a fashion that works pretty darn well , um, within a , um, a limited , um, um, OT type of environment. So just working together and understanding each other what's possible, what needs to be taken into consideration. And then once everyone's working together, there's just a chance to do so much more and to create a more resilient and secure manufacturing space.

Speaker 1:

Absolutely. And at the end of the day, I think manufacturers just want to feel prepared and protected from potential security threats so they can worry less and focus more on building great products that their customers love. So Robert, thank you so much for sharing your knowledge with us today. I know our listeners are always eager to stay informed on today's industrial security landscape and I also know that you have a great resource for our listeners to continue learning about this. So can you tell us a little bit about your security newsletter?

Speaker 2:

Sure. So every month I publish a a newsletter , um , that talks about , um, some of the new collaterals and tools that Cisco has published , uh , that would relate to security and operational space. Um, we talk about some of the new protections. We have a dedicated , uh , industrial security research group inside Talos. Uh, they published new identifiers for both threats as well as , um, um, standard OT protocol activities every month. Um, we name all of the vulnerabilities that have been published through the ICFA cert with links to what those things might be. And then we talk about , um , um , vulnerability that the tallows research team has actually discovered and published as well. And then in addition, we get to news of the month. So what types of major news that , uh, is , uh, of , uh, value to understand in terms of both incidents as well as government legislation and other types of regulatory activities , um , that may be pertinent to their particular space. Um, all of this again , uh, delivered , um, um , once a month and you can peruse at your , um, discretion. Hopefully of some value.

Speaker 1:

If you'd like to receive Robert's monthly security newsletter, you can easily subscribe by going to this episodes blog and follow the newsletter instructions posted at the top of the page. Be sure to also check out our featured resources, including videos, white papers, infographics, and more. To keep learning more about this topic and at the bottom of the blog, be sure to submit a comment to let us know what you'd like to learn about in future episodes. Thanks again for listening to Cisco's manufacturing leaders [inaudible] .