What is it about Operational Technology that makes overcoming the fundamentals so difficult to achieve?

As defenders, how are we implementing controls and how do we do it better? Join Jamison Nesbitt and Stefan Liversage for this educational, thought provoking and controversial conversation.

The fundamentals are simple; patching, segmentation, malware controls, visibility – but what we are trying to achieve is no easy feat when you look to dismantle more than 20 years of ad hoc OT security measures as there is still risk during the implementation phase.

Key points to be discussed:

• Technical controls / process and people
• Poor visibility (automation)
• Organisational structure to support OT
• Manage risk faster
• Contain threats quicker
• Minimise impact
• Green field vs brown field
• Skills from the ground up
• Managed services
• Managing multiple streams of activity
• Iterative, agile process, we learn, we move forward
• Rapid threat containment

Transcript

Welcome to the CyberSenate Podcast. I'm James Nesbitt, director and founder of the CyberSenate. And we are joined here today with Stephen Liversidge, the systems engineer and OT subject matter expert at fortunate. Today we're going to talk about what it is about operational technology that makes overcoming the fundamentals so difficult to achieve. We've invited Stefan to join us, he has joined fortunate in 2021. As a system engineer and OT subject matter expert, he brings over 15 years of experience in OT and cybersecurity working across design, engineering, commissioning, and consulting disciplines. Stefan has previously worked primarily in the systems integration and end user environments across the CNI and pharma industries, bringing insights into the challenges faced by ot owners and operators. This conversation is meant to be educational, thought provoking, and potentially controversial. So hang on tight as we jump in to the fundamentals of patching segmentation, malware controls, visibility, and really a discussion on what we're trying to achieve. It's no easy feat when you look at dismantling more than 20 years of ad hoc ot security measures, as there's still a lot of risk out there during the implementation phase. So some of the key points that we want to address in this morning's podcast are technical controls process and people poor visibility, organizational structure to support oti, how we can manage risk faster, contained threats quicker, minimize impact, we need to look at Greenfield versus brownfield skills from the ground up managed services, managing multiple streams of activity,
agile processes, we learn, we move forward, and also rapid threat containment. So there's quite quite a few points there that we do want to go over.


We welcome all of our listeners. And thank you for tuning in to the CyberSenate podcast where conversations count, hope you hopefully you'll find this educational and that any point if you want to reach out to the CyberSenate or to Fortune net, to discuss some of these points further, just get in touch with us, we'd be happy to to point you in the right direction. So Stefan, welcome. Thank you, James, thank you very much for having me on the show. That's great to have you. I greatly appreciate you taking the time this morning. We, you obviously spoke on our conference recently. And I know that was a very well received presentation. So thank you very much for your insight there and fortunate support.

  
Without further ado, we're going to get stuck right into this conversation. It's there's a lot to talk about. So first and foremost, Stefan, why do organizations struggle to implement ot security controls?


Fundamentally, it's not easy to achieve. You know, as you said, in the intro, you know, we were trying to deal with 20 years of ad hoc networking implementation, I was around when ot was transitioning from serial networks, and we're just starting to embrace. Ethernet really is a common practice. And that was at the forefront of that. So, you know, I was fresh out of uni, you know, finding my way through, you know, networking and the fundamentals of networking. So, you know, through that transition, and that implementation, was everything done, right? No, was it done with security in mind? Certainly not.

So, you know, obviously now there's much more awareness of that.
  
And the breadth of what needs to be achieved, the depth that we need to go to, can often overwhelm organizations. So a lot of people get lost in that transition from effectively zero security to where they want to be. But as we know, cybersecurity is a journey is not a project.

You know, so that has impacts across an organization not only from a technology point of view, but from a people and processes point of view. It impacts not only engineering, it affects, you know, right through through management, and even stretches as far as procurement. So organizations really need to start and focus on what it is that they want to achieve, why they want to achieve that?

And then start to formulate how they're going to go to meet that. And part of my presentation, we talked about being smart, you don't have to be right. I think, you know, when when organizations and when when engineers, and OT professionals start to look at ot security, we start to think about what's right, and how can they protect their environments where maybe we need to sort of take a bigger picture of you. And think about them compensating countermeasures, that we can say, things that are maybe easier to implement don't have quite the same impact from an implementation point of view. So we can sort of provide with risk reduction measures, there was talk about honeypot type capabilities has been something that was, you know, of high value in terms of being able to rapidly detect threats from IoT network, but it's not a typical first step that organizations take.

Which is something that those out there the battling with implementing things like segmentation should should maybe consider.


So and OT was, is or was, of course, somewhat of a afterthought. Rarely does IT security normally took front front row, didn't it for years, and now we're starting to see a lot of organizations say, hey, this OT, ot security is a real threat, we need to do something about it, we need to look at how we're fundamentally structured. To address all of the vulnerabilities across the attack surface. We need a risk management plan in place. And, and of course, then we've got a myriad of other issues that are arising such as Workforce Development and the skills. So yeah, it does seem that organizations are still struggling, it is nice to see at least from CyberSenate perspective, when we're running our events that folks are coming out. And they they see the need to transition they need to see they have the awareness that they need to make some changes. And those changes are being made. And each organization being slightly different, with different budgets, different focuses.

And different staffing do seem to be making some headway here. So it's, it's quite, it's quite good to say. Another question we had for you is who holds the budget to support ot security programs? And how does that impact operational technology?

Yeah, I mean, it's a really good point. And it just sort of cross over into the previous question. Really, you know, I typically what we see is it hold the budget. And typically, what we have seen is that it become responsible for ot security.

And again, that whether that's the right direction or wrong direction depends a lot on the organizational structure. But I think again, a lot of organizations are really starting to look at what they need to be effective in implementing IoT security controls. So yeah, typically it holds a budget. I think some of the challenges that that brings is that OT, certainly from an engineering discipline, and not been greater articulating the need.

And the business value in implementing cybersecurity, and there's a strong shift that we're seeing is organizations looking to deliver value and security and OT security is typically seen as an expense. I think we, as engineers, you know, maybe that's something that we've sort of, not not exactly addressed in the way that we should, whereas really, when you look at OTC coated that should be the enabler for digitalization, which is obviously key to driving efficiencies, optimizations and you know, effectively allowing organizations to be competitive in the marketplace.

So I don't think you know, as as ot practitioners and OT security practitioners that have been positioning that very well to it. Now, obviously, for it or holding the budget, then they are much more comfortable, much more familiar and what's more, better able to articulate the need for spending it. So I think that's typically why we're seeing it budgets being you know, that much more larger than than what we've seen. OT is probably also visibility point of view as well. And typically, ot isn't represented within the board. I know, a lot of organizations are starting to address that. And that's a very encouraging thing to see. So I think, you know, it's shifting.
You know, we're still dealing with that, that legacy of it being the sole purpose holder. Interesting, isn't it? Maybe one day, we'll just have a security division or a security budget, as opposed to an AI, kind of an IT focus, budget, and then the engineering and OT community and each organization fighting or trying to raise awareness of what their requirements are, hopefully, we can see all of that converge and say, Hey, this is our organizational security budget, and we're building fundamentally all of our digitization program, off of this security. Vision. Yeah, absolutely. I think we talk about the differences between it and OT security. And there are, you know, you can't get away from that, you know, typically, you know, I find it's not necessarily really about the technology, it's more about the sort of implementation of that technology and the use of that technology. That's, that's different within two environments, but fundamentally, the two are inherently dependent on each other. You know, we talk about, you know, a theme of the CyberSenate conference in London was, you know, the biggest concern that organizations have a threat actors gaining access to the corporate network, and then looking to pivot through into OT, and obviously, that becomes more prevalent, the more we connect those systems. So we can't decouple those two environments from a risk point of view. And equally, you know, where we've not got adequate access controls in OT, that could mean that we end up introducing backdoors. So it may be easier for a threat actor to actually compromise OT and pivot through into it as well. So, you know, to your point, really, what we need to start thinking about is, you know, a security, an enterprise security approach that includes both of those two disciplines.
 
And really brings around that sort of convergence of addressing the skills gap.

I think a lot of organizations look at the skills gaps that they have in OT, but actually, if you think about the skills that they have in house, the present day, they probably have all the skills that they need. They're just in different areas and different disciplines, you need to get them in the same room talking to each other, maybe a small number of individuals that can quite happily and comfortably sort of move between IT security notice security to sort of knit it together. But I think ot engineers are very intelligent, very skilled people. And we can bring that knowledge into it. security context, really helps drive things forwards. Sure. Yeah. Makes sense. We need champions, it seems in each organization to raise awareness, because these don't always come from the sea level, do they? I mean, sometimes they do. And it's always awesome to see that. But think he each organization we've met needs that champion to raise awareness and to bring people together. And I was just thinking as you were talking, and what we really, truly need is a convergence of thinking.

Yes, how we think about OT, how we think about it, how we think about security, and digitization entirely, needs to be addressed, really, and you have to have those champions, that, that help spur that convergence of thinking, you know, internally, these organizations, so any of our listeners, maybe it'll hopefully inspire them to do to grab, you know, their colleagues, you know, we were working with one airport in Florida not long ago. And they're head of it and their head of operations. Both got together, rather, I think it was bi weekly or monthly, but they identified the fact that there was so much IoT, and there was so much connectivity throughout the, the airports that they needed to get together and they needed to have a convergence of thinking to, to make sure that they were plugging all of the vulnerability gaps that they could possibly do. And they took that initiative and I when I when I first met them, and I first heard their presentation, it gave me goosebumps, I thought, wow, this is people making the change. You know, this isn't this isn't about funding and board level strategy. These are these are two individuals that looked at the platform and said we've got a problem, and we need to come up with a solution. And then of course, that type of innovative thinking. Probably I have to check back with them but I would assume the airport probably adopted that. But you know, that was the head of operations, and the head of it that said, we've got a problem. And they came together. And I thought, wow, you know, it'd be awesome if we heard that type of a story from a lot of different organizations. So it kind of brings us to our next question stuff on how to organizations approach risk management, and OT, think it's often a bit knee jerk, really, you know, we organizations realize that there is this risk, and this threat that they need to manage. So what do they do? Well, they look to prevent that event from occurring. So you know, would go to the traditional protection type response. So, you know, we look to implement segmentation, you know, we put the firewalls in, you know, we look toward patch management, we look towards, you know, AV solutions for endpoint, etc.
And they're all the things that we should do. And when you look at a sort of typical IoT, security, maturity, sort of journey, really, no, that's often why we start, the trouble with those is, is that can often be the ones that are most challenging to implement.

You know, we see issues with vendor support for putting, you know, advanced malware solutions and capabilities on you know, OEM systems. So therefore, we lose potentially new support, or you've got to jump through some hoops to gain support for that organization start to look at the cost of implementing segmentation in terms of outage and those kinds of things.

So then, obviously, we've got to go through the justification of those controls. And
then when an organization does decide to bite the bullet, then obviously, they have to look at outages and planning. And that that can then mean that that program stretches over a number of years. So whilst it's the right thing to do, and everybody should definitely be looking at that. What can you do in the meantime, to address that risk? Because you can't just say, we've identified this risk, yeah, we're gonna address it. But it's gonna take four years until we get to the point because the trouble is, then he gets four years time, and the game has moved on once again. So again, it comes back to you know, those kinds of capabilities that allow you to do threat detection. You know, how can you identify if there is malicious or suspicious activity on your in your environment? How can you then respond to that as well? So, you know, again, a lot of organizations spin up these sort of classic waterfall approaches, you know, cybersecurity as a project, etc. Whereas really, we need to be starting to think about well, okay, I need to put some network security. 


Absolutely. What's my sort of minimum viable product? And the the conference that, you know, I asked to sort of a question of, well, if I put a firewall between it and OT, and I don't have any security policy, have I actually improved anything that
you would, again, the knee jerk reaction would be? No, if you've gotten you know, minimal security policy, then you've not really made any risk reduction? Well, actually, you have, because you've then got a point where you can start to control
traffic, you get better visibility, you get an ability to be able to respond to new or emerging threats that are in there. So again, taking that approach where you deploy a capability, or then maybe through a process of iteration, you use that capability in a much better way going forwards.


Okay, is the approach to risk management and security CRO controls different in Greenfield sites versus brownfield sites? In your opinion? Yeah, I mean, obviously, Brownfield, we're talking about a difficulty in implementing implementing things like network segmentation, etc. So, yeah, that applies to a brownfield site. If you've got a greenfield site, it's far easier, you know, you should be building and designing that that, that greenfield site, with security in mind.


From a technology point of view, yeah, that makes up sort of sense.

The only difficult becomes then really, if you think about that, from a sort of operational maintenance point of view, do you have the skills and capabilities to be able to manage a secure environment, our OT skills to be able to make those process changes be able to respond to failures in the environment? 
I don't think we're quite there yet. But again that does that mean that you shouldn't build that greenfield site with the capability to be able to deploy those controls moving forwards? Absolutely not, you should 100% Build with way where you want to be in an ideal situation. So again, you know, create those boundaries, those enforcement points doesn't necessarily mean that you have to use them as granularly as you would like to, or at least there so that you can build upon that foundation moving forwards.
  
It's interesting, because often, when we talk about Greenfield versus Brownfield, it's often it's often thought, at least on my behalf, and I think a lot of the industry experts, we talk to you that Greenfield makes more, it's easier, right? It's if we can build these things with security, by design from the get go intelligently how they should be. And of course, there's a risk management and an implementation process. And that all has to be modeled, right and, and agreed at a board level. And then of course, we have the implementation of that. But that's no easy feat, you know, that that's a massive project. And we're at the moment where we're talking with a lot of rail leaders about cybersecurity, and was just recently talking to some folks over at Crossrail. Now, when you look at that infrastructure project, one of the largest, if not the largest in Europe, how many engineering procurement contractors, Project contractors, excuse me, were involved in that. And that was basically Greenfield, for the most part, it may have some legacy elements to it, but it's quite intense to think about, wow, you know, how did they put that risk management process in place? How did they determine, you know, what connects to what how they were going to secure it? And how did they make those choices as to who was going to lead that kind of cybersecurity vision for the build of something of that size?

And, you know, I think, yeah, brownfield is is obviously very difficult to bolt on cybersecurity, to these legacy systems that were never built with digitization in mind. But also some of these larger greenfield projects. Wow. I mean,how much of that process has to go into that? And how much funding is it surely massive. So that that maintenance, you know, after the fact, you know, once it's implemented, you've still got to manage and maintain and operate in that environment, it comes back to that people process and technology point of view. Yeah, Greenfield is easier to implement the technology, but you've still got those operational, organizational challenges that you've always had. So, again, through an organization taking that big picture planning and maybe looking towards it or not, it's not something I've seen organizations really consider.

On a cybersecurity program point of view, how can they address Greenfield sites today, more than focus on on brownfield implementations and the struggles? Again, and again, that sort of comes back to that, you know, how much resource Do you plow into as an organization into deploying those protected cycles into brownfields? From experience, that's probably been the larger portion of  effort and resources have gone into that? Is that the right thing to do? Again, every organization is different, but you know, maybe not. Maybe we can look at things in a more smart way. Yeah, yeah. I agree. I agree. I mean, some of the Retrogrades and up upgrades and retrofit conversations we've had with with folks who, you know, build locomotives and these types of things is, is there was looking at how we have these legacy systems, how many more years are left of this particular asset? And should we just swap it all out now? Or we can we upgrade this and what you know, what does that look like? And what a massive challenge that is?

 
Another question for you here is, is the skill shortage in OT security being addressed? And what is the role of OEMs and educational institutes in that process?

 
So, I mean, we do now have, you know, some some very good training available for ot professionals to, you know, upskill and become much more aware and comfortable with deploying security within our team. But I still feel, you know, from educational institutes that maybe not enough.
 
You know, the typical paths that we that we see, for ot professionals, you know, the sort of education paths that they take, every single one of those should have, you know, a not insignificant portion of cybersecurity built into those, so that when, you know, graduates, etc, of coming out of university and colleges, that they go into industry, already with an awareness to everything that they do, you know, they need to consider security implications. I think that would be, you know, a massive benefit, because it gets the grassroots dying to question to ask why, you know, raise, you know, I mean, I remember, as a graduate, you come out and you've got lots of questions, you don't necessarily have the answers, or what that means is, then you're asking questions of, of those experienced professionals? And maybe, then they realize that, well, maybe I don't have the answer for this. So it's sort of, you know, grassroots feeding of, you know, definitely encourage them. So, yeah, the education institutes have a greater role to play, outside and above the dedicated security, cybersecurity qualifications that there are out there, I think there needs to be no discipline to feed into the the other disciplines as well.

And then on to the subject of OEMs.


I think this, this one's a challenging one, really, you know, they do lots of courses that train ot professionals in how to use their products and solutions. Now, we all know that OEMs are very definitely moving forwards and increasing their security capabilities within that product offerings, which is great to see that what I haven't seen, that's not to say it doesn't exist, but I haven't seen courses from OEMs that include how to use the security capabilities, are there solutions on what sort of best practices would be.


So I think, again, you know, that's not going to be the be all and end all, you know, OEMs can deliver IoT security, training and feed, you know, what we need, as an industry, that's, that's definitely not the case. But it's more from an awareness piece, you know, encouraging people to use the capabilities that are there, existing and inherent within products, which, again, we're going to be seeing more and more coming downstream, which is great. But we just need to making sure that engineers using these systems day to day are equipped with at least some fundamental knowledge of those capabilities, so that you as a customer, you know, if you want to, you know, add some elementary security controls into, you know, projects that you're deploying, then, you know, the sort of systems integrators, etc, you know, have the ability to be able to provide that.

It's, it's a very complex situation, to some extent, isn't it? I mean, finding the rights, this the right workforce, the right skills. And then there's the question of mentoring. I was reading an article this morning about mentoring. And as he, as you've said, some of the students coming out of universities, if they're asking the right questions, then that shows that they have a passion that they're asking those questions. And maybe those answers aren't obvious, but at least they're asking the questions. And that's probably the that is the most important part of that particular paradigm.
 
And then we have the the mentoring element. I know, we've met a lot of folks at these asset owners, energy companies, rail companies, critical infrastructure across the board, that are very keen on training and mentoring younger, younger professionals as they come into the business. So we have that, but sadly, those folks are going to retire also. And you know, and so yeah, these these educational programs are very important. And I know at the conference, we were, you know, I asked the audience, where did you come from urine, OT, where did you come from? And they all raise their hand saying they were from it, didn't they? There was a lot of I mean, that's, that's good to see. Ya equally. But I think I think, you know, if you ask those that said that they're coming from OT and ended up in OT security, you know, that each of those journeys would be very different and very diverse.

And the theme would be that they sort of fell into it by accident, which is concerning really, you know, it's not ot security hasn't been a dedicated career path. have been sort of removed from the sort of engineering discipline for a number of years now, I'm not sure if that's changed, you know, be interesting to to gain an insight into that,