Introduction to Enterprise Blue Teaming

We’re following up our Enterprise Red Teaming episode with the logical next topic: Enterprise Blue Teaming. ETAS’s Rene Reuter, Product Manager Enterprise Security Systems, and Wolfgang Neufeld, Subject Mater Expert Red & Purple Teaming, return and are joined by Sven Ulke, Sr. Manager DFIR at SVA System Vertrieb Alexander GmbH. The trio provide a thorough look at Blue Teaming – what it is, who works on it, why it’s important, potential challenges and how to address them …they even touch on AI and the impact on automotive, especially regarding software-defined vehicles. 

 If you enjoyed the Red Teaming episode, you’ll definitely want to check out this one as well!

Thanks for listening!

• Email us at: contact.us@etas.com
• Learn more about ETAS on our website 
• Follow us on LinkedIn: @ETAS

Transcript

00:00:02 Voiceover

Welcome to the Empowering Tomorrow's Automotive Software Podcast, brought to you by ETAS, a single source of cutting-edge software and hardware solutions that make automotive embedded systems safe, smart, secure, and sustainable.

00:00:15 Voiceover

Each episode, we'll be joined by ETAS and industry experts to discuss how electrification, automation, and connectivity are impacting the automotive industry.

00:00:25 Voiceover

Now, sit back and enjoy the discussion.

00:00:32 Rene Reuter

Hi everybody at ETAS’s Empowering Tomorrow's Automotive Software podcast.

00:00:37 Rene Reuter

This is the second episode of our three-part series about red, blue, purple teaming.

00:00:43 Rene Reuter

If you haven't listened to our first episode about red teaming, I encourage you to do so.

00:00:48 Rene Reuter

It's very nice.

00:00:49 Rene Reuter

Today we're going to talk a little bit about a little bit on teaming part.

00:00:53 Rene Reuter

And in this episode, we will try to describe a little bit to you

00:00:57 Rene Reuter

What is blue teaming?

00:00:58 Rene Reuter

What are the persons working in it?

00:01:00 Rene Reuter

And why is it necessary?

00:01:02 Rene Reuter

My name is Rene Reuter.

00:01:04 Rene Reuter

I'm the responsible product manager for enterprise security services here at ETAS.

00:01:09 Rene Reuter

And today with me, I've invited 2 experts, 2 blue team experts, Wolfgang and Sven.

00:01:14 Rene Reuter

Hi to you.

00:01:16 Sven Ulke

Hi.

00:01:17 Wolfgang Neufeld

Then I will start to introduce myself also.

00:01:20 Wolfgang Neufeld

I'm Wolfgang Neufeld, also working at ETAS now for five years.

00:01:25 Wolfgang Neufeld

I'm

00:01:26 Wolfgang Neufeld

subject matter expert for red, blue, purple teaming at that point.

00:01:30 Wolfgang Neufeld

And I will try to describe what my views are on that part today.

00:01:36 Sven Ulke

Thanks to you guys for having me.

00:01:39 Sven Ulke

My name is Sven Ulke.

00:01:40 Sven Ulke

I'm Senior Manager for Digital Forensics and Incident Response at a German IT company that consults customers during cybersecurity incidents.

00:01:50 Sven Ulke

And it's a pleasure for me to be able to speak here because I also worked for ETAS for more than three years.

00:01:57 Sven Ulke

And we had a really great collaborationship working together.

00:02:02 Sven Ulke

So thanks for the invitation.

00:02:04 Rene Reuter

Thank you very much.

00:02:05 Rene Reuter

I'm very glad having you both here.

00:02:07 Rene Reuter

So I think we want to start with looking a little bit back in the early years, what was blue teaming about?

00:02:15 Rene Reuter

So looking at the past 20 years and from that on,

00:02:21 Rene Reuter

In the past, the blue team was more or less the administrator of the system.

00:02:25 Rene Reuter

So he was doing the blue team part as kind of a side role, and he needs to do the operations, he needs to do the monitoring and everything.

00:02:34 Rene Reuter

But today it changed a little bit, but I guess it makes more sense if we're really having a look at the history, how has this evolved over the past years?

00:02:43 Rene Reuter

I think Sven is going to give us a little bit of an introduction to that.

00:02:47 Sven Ulke

So exactly as you described it.

00:02:49 Sven Ulke

So when we think back 20 to 25 years ago, it was this one IT guy or several IT guys that did all the IT stuff, the whole infrastructure, the firewalls, everything that you had in your company.

00:03:06 Sven Ulke

And usually,

00:03:09 Sven Ulke

This kind of blue teaming didn't happen on purpose.

00:03:12 Sven Ulke

It was only if an incident occurred or if something strange happened, like systems went down or there was some troubleshooting needed because some services were really slow.

00:03:24 Sven Ulke

Then the administrator or the operator of the system started his investigation and checked what is strange on the system, what is causing this heavy load.

00:03:35 Sven Ulke

and why are some things happening that are not normal in the system.

00:03:40 Sven Ulke

And this was at a time where the IT environment was not so complex like it is today, where we have cloud services, software as a service solutions, lots of people working from home, out of the home office.

00:03:56 Sven Ulke

So over the past 20 years, IT infrastructure

00:04:02 Sven Ulke

got more and more complex and also more and more new roles were introduced.

00:04:10 Sven Ulke

And when we look at the blue team side, if you think about having just a few systems, you have to look out if they do or have some strange behavior.

00:04:21 Sven Ulke

Now you have thousands of servers, different services.

00:04:25 Sven Ulke

So now you really need experts in each of this

00:04:29 Sven Ulke

Special fields to figure out maybe if there's something wrong with your cloud instance, what might have caused this issue, and also when we switch to perspective, like you did in the first episode on the red teaming part, so...

00:04:46 Sven Ulke

20 years before, the attackers had a really small number of systems which they were able to attack when they wanted to get into a company.

00:04:55 Sven Ulke

Today they have various possibilities using cloud services, using e-mail, all that stuff.

00:05:01 Sven Ulke

So also the attack surface got much bigger.

00:05:07 Sven Ulke

And so also the blue team must defend the company at several

00:05:14 Sven Ulke

different positions at the IT infrastructure.

00:05:19 Rene Reuter

Okay, thank you.

00:05:20 Rene Reuter

Yeah, I guess this is a little bit easier for large corporations nowadays as they have more expertise and more persons working on that.

00:05:27 Rene Reuter

But I guess it's still an issue for small and medium companies.

00:05:31 Rene Reuter

as they cannot afford having this kind of specialized teams.

00:05:35 Rene Reuter

I guess this is still an issue and we want to tackle this down a little bit in today's episode.

00:05:40 Rene Reuter

But I guess it makes sense in the beginning that we have a little bit of an example.

00:05:45 Rene Reuter

What is a blue team about?

00:05:46 Rene Reuter

We thought maybe it makes sense to start with a non-technical example, better explain what's the purpose of a blue team.

00:05:53 Rene Reuter

Wolfgang is going to cover that.

00:05:56 Wolfgang Neufeld

Yeah, I think it also makes really sense to start with a non-technical example.

00:06:01 Wolfgang Neufeld

because blue teaming is often misunderstood and why you should let this blue teaming stuff be done by experts and why you need expert and expertise in that.

00:06:13 Wolfgang Neufeld

And I invented a story some years ago to explain that a little bit.

00:06:19 Wolfgang Neufeld

And from this invented story, I always tell about Paper Factory.

00:06:24 Wolfgang Neufeld

And in this paper factory, there is some careless employee just going in, smoking his cigarettes, and then not putting it out properly.

00:06:35 Wolfgang Neufeld

And yeah, as it happens, it ignites the whole warehouse.

00:06:40 Wolfgang Neufeld

Yeah, bad.

00:06:42 Wolfgang Neufeld

Huge amount of damage is caused.

00:06:44 Wolfgang Neufeld

This is bad.

00:06:45 Wolfgang Neufeld

And the CEO of that company says, okay, this is something that if this happens again, we have a really big issue.

00:06:52 Wolfgang Neufeld

We already lost a lot of money.

00:06:55 Wolfgang Neufeld

We really have to avoid this kind of problem in the future.

00:06:59 Wolfgang Neufeld

And they came up with some very easy ideas.

00:07:02 Wolfgang Neufeld

So the first thing, of course, in a paper warehouse or paper factory, the thing which should have been already obvious,

00:07:11 Wolfgang Neufeld

a smoking ban is introduced.

00:07:13 Wolfgang Neufeld

So don't smoke in a paper warehouse.

00:07:15 Wolfgang Neufeld

Okay, that's obvious.

00:07:17 Wolfgang Neufeld

Yeah.

00:07:18 Wolfgang Neufeld

So that's the first thing.

00:07:20 Wolfgang Neufeld

And so far so good.

00:07:21 Wolfgang Neufeld

And as an additional measure, a sprinkler system should be installed.

00:07:25 Wolfgang Neufeld

So whenever there is a fire, it can be put out automatically by sprinkling water.

00:07:30 Wolfgang Neufeld

And yeah, great.

00:07:32 Wolfgang Neufeld

So to celebrate after the new sprinkler system has been installed,

00:07:40 Wolfgang Neufeld

And to celebrate the new sprinkler system, they started to fire up some sausages and steaks and grilled them next to the warehouse.

00:07:50 Wolfgang Neufeld

And as expected, the fire alarm system recognized the smoke, sounds the alarm and floods the warehouse with water as ordered.

00:07:59 Wolfgang Neufeld

So another total loss, this time from water.

00:08:03 Wolfgang Neufeld

And yeah, like IT threats, this is something, it wasn't analyzed by experts.

00:08:08 Wolfgang Neufeld

So it was

00:08:10 Wolfgang Neufeld

just a shot that we said, okay, we had the problem with fire.

00:08:13 Wolfgang Neufeld

You can fight fire with water, that's obvious.

00:08:16 Wolfgang Neufeld

But the problem was completely underestimated and misjudged.

00:08:20 Wolfgang Neufeld

And in the end, it led to the same problem that was sought that has been solved once and for all already.

00:08:27 Wolfgang Neufeld

And I think that story, I told that a lot, or the story I told a lot to CEOs because they are mostly not technical, but then they understood why we need expertise also in that area.

00:08:42 Wolfgang Neufeld

And this is, I think, a good start, why, and now we are getting more and more technical after that part.

00:08:50 Wolfgang Neufeld

why we say please don't underestimate the problem and get help already by defining the problems because if you don't get the expertise you will lose a lot of money and time and this is something that you normally don't have by the attack surface and the automated attacks that you currently face out there.

00:09:11 Rene Reuter

Yes.

00:09:12 Rene Reuter

Thank you very much for this very good non-technical example.

00:09:15 Rene Reuter

So and now let's go a little bit deeper and I think we now should switch to a more technical example.

00:09:21 Rene Reuter

What are the challenges for blue teams?

00:09:24 Rene Reuter

And I guess when you can give us a more technical one.

00:09:27 Sven Ulke

Yeah, sure.

00:09:28 Sven Ulke

So to start with that, we decided to take some kind of ransomware example.

00:09:34 Sven Ulke

What is meant by that?

00:09:35 Sven Ulke

So ransomware is a kind of malicious software.

00:09:39 Sven Ulke

Then when it will get executed on systems, it uses encryption algorithms to encrypt the data.

00:09:47 Sven Ulke

And afterwards, if you want to get back to your data, you need the keys to decrypt the data.

00:09:56 Sven Ulke

And the attackers usually have these keys on their side.

00:10:01 Sven Ulke

And if you want to get access back to your data, you have to pay money for it.

00:10:05 Sven Ulke

So you get a ransom note that is stating how you can recover your data, how you can contact the attacker, and what amount of cryptocurrency or money is needed to get that data back.

00:10:19 Sven Ulke

And there are various

00:10:21 Sven Ulke

other kinds of attacks on companies.

00:10:24 Sven Ulke

But I think this is also a really good example to understand the problem that we are facing today on the blue team side.

00:10:33 Sven Ulke

So when a ransomware attack happens, especially also smaller companies that do not have this kind of monitoring systems or people that are watching at the systems 24-7, at some point they will recognize that something is wrong.

00:10:49 Sven Ulke

Because

00:10:51 Sven Ulke

After a system got encrypted, often normal services aren't working anymore, systems are going offline, you are not able to visit your own company website, for example, or you are not able to transfer money, et cetera, et cetera.

00:11:09 Sven Ulke

And so ransomware has one big positive aspect.

00:11:13 Sven Ulke

If you are tackling such kind of incident, you will definitely find it out because

00:11:20 Sven Ulke

systems stop working.

00:11:21 Sven Ulke

If we have other kind of attacks, there might be the chance that attackers are in your environment for several years before you recognize them.

00:11:30 Sven Ulke

But when we find out that systems are encrypted and data is encrypted, we have some kind of systems where the attacker run malicious software and the software somehow got not blocked by our security tools.

00:11:47 Sven Ulke

This is the first question, how this could happen.

00:11:52 Sven Ulke

But usually you want to know how was it possible that such kind of attacker was able to get access into our environment.

00:12:02 Sven Ulke

And this is also a good example that shows how complicated blue teaming is nowadays, because on this ransomware side,

00:12:14 Sven Ulke

It's not that there's just one guy sitting at home and hacking the way into the company.

00:12:20 Sven Ulke

It's like a criminal business with various jobs and roles.

00:12:25 Sven Ulke

You have people that are developing the malicious software.

00:12:29 Sven Ulke

You have people that are trying to get the initial access to the company, maybe by credentials and usernames in the darknet.

00:12:39 Sven Ulke

or use vulnerabilities that you might have in some kind of software or appliances you are using to get initial access, and so on and so on.

00:12:49 Sven Ulke

So it's a really complex business with a lot of expertise on the criminal side.

00:12:55 Sven Ulke

And so there are various ways how it was possible that some kind of attacker could get access into your environment.

00:13:03 Sven Ulke

And all this kind of

00:13:05 Sven Ulke

possible entry vectors must be found and must be closed to make sure that after, if you have backups and could restore your data from the backups, that this kind of attackers don't come back.

00:13:20 Sven Ulke

The second part is, it's also the question what the attacker did instead of just encrypting it.

00:13:29 Sven Ulke

So nowadays, also before starting the encryption,

00:13:33 Sven Ulke

the attackers steal sensitive data and exfiltrate that out of your environment.

00:13:39 Sven Ulke

So customer data, construction data, business data, personal data of employees or customers are exfiltrated and you will be forced by the attacker that this kind of sensitive data will get published if you do not pay the ransom.

00:14:02 Sven Ulke

And also you have

00:14:03 Sven Ulke

lots of legal problems with that and the attacker knows that you are forced to do some claims at various, especially in Europe, and various legal institutions.

00:14:17 Sven Ulke

So the attackers have a great chance that you on the defender side must pay, even though if you can restore the data, you are not sure which data was exfiltrated, was it sensitive data or not.

00:14:32 Sven Ulke

And so there's a great chance that the attacker has success and gets the money from such kind of incident.

00:14:40 Rene Reuter

Okay, thank you very much.

00:14:42 Rene Reuter

So I guess with those two examples,

00:14:45 Rene Reuter

I really discovered that this is a very complex topic, especially in the blue teams, as you've mentioned, with the attackers having various teams trying to get access to the company.

00:14:55 Rene Reuter

So I guess the expertise which is needed on the blue team part is also very crucial.

00:15:01 Rene Reuter

And I think it makes sense that we'll have a look at the different kind of roles and expertise which is needed for the blue team.

00:15:08 Rene Reuter

So maybe we'll start with the SOC analyst here.

00:15:11 Wolfgang Neufeld

Yeah, of course.

00:15:13 Wolfgang Neufeld

I can try to answer that.

00:15:15 Wolfgang Neufeld

Let's go to the ransomware example and try to figure out what's going on in an ideal case.

00:15:23 Wolfgang Neufeld

I mean, you got hacked, you got ransomware, so that's already not ideal.

00:15:27 Wolfgang Neufeld

But following from that, okay, what could have happened, what might have prevented then the further steps from ransomware.

00:15:36 Wolfgang Neufeld

So you have some kind of systems already there, some kind of detection, some sensors.

00:15:42 Wolfgang Neufeld

And then you have a special person there, which is the SOC analyst, which looks at some dashboards, which gets the alerts from all the systems.

00:15:52 Wolfgang Neufeld

And in an ideal case, he finds some kind of suspicious behavior in the logs and can correlate them and say, okay, look at that endpoint, something really bad happened.

00:16:04 Wolfgang Neufeld

if some malicious file has been downloaded or while it's executed, it has some suspicious behavior on it.

00:16:13 Wolfgang Neufeld

And then the SOC analyst would sound alarm and would say, hey, please, I need help.

00:16:19 Wolfgang Neufeld

I think something fishy is going on.

00:16:22 Wolfgang Neufeld

So this is the first thing that should happen in a company.

00:16:26 Wolfgang Neufeld

And if this already is not going on, then you will have a bad time with ransomware.

00:16:30 Wolfgang Neufeld

So this is the first line of defense that should work.

00:16:34 Wolfgang Neufeld

and how you can maybe detect some kind of ransomware attack.

00:16:39 Rene Reuter

You just use the abbreviation SOC.

00:16:41 Rene Reuter

Maybe you can just say what's the abbreviation for.

00:16:45 Sven Ulke

Yeah, so sure.

00:16:46 Sven Ulke

Sorry for that.

00:16:47 Sven Ulke

So SOC means Security Operations Center.

00:16:50 Sven Ulke

So it's a kind of one single source of truth where you collect all your information, log files to see how your systems are up and running.

00:17:02 Sven Ulke

and which log messages are generated from each source or from each of your sensors that you have deployed in your environment?

00:17:11 Rene Reuter

Okay, perfect.

00:17:12 Rene Reuter

Thanks.

00:17:12 Rene Reuter

So the next one in the line would be someone who is called a so-called incident responder.

00:17:19 Sven Ulke

Yeah, sure.

00:17:21 Sven Ulke

So I'm doing this for a very long time.

00:17:23 Sven Ulke

And you can imagine if the SOC analyst is at the beginning of this kind of blue team, which

00:17:32 Sven Ulke

gets the alerts or finds some malicious behavior and rings the bell and says, okay, there's something strange ongoing, the incident responder itself is most of the time at the last part.

00:17:44 Sven Ulke

So if all your prevention measures didn't work out and you have an IT or cybersecurity incident in your environment, then the incident responder is there to coordinate all the incident-related activities.

00:17:58 Sven Ulke

And what is meant by that?

00:18:00 Sven Ulke

the technical investigation part, but also the organizational investigation part.

00:18:05 Sven Ulke

Like from the technical side, you will find out maybe what the attacker did do, how the software was named or created, or what the software did on your systems.

00:18:17 Sven Ulke

But on the organizational side, he has to figure out and coordinate how you get rid of this kind of incident and how to, yeah, let's say it easy, kick the attacker out of the environment.

00:18:29 Sven Ulke

And that's

00:18:30 Sven Ulke

That's most of the, that's the whole role of the incident responder in the blue team.

00:18:37 Sven Ulke

And which is really near or really close to that.

00:18:42 Sven Ulke

And it's also part of the blue team is the so-called digital forensics analyst.

00:18:46 Sven Ulke

And what are these guys doing?

00:18:49 Sven Ulke

Like they do the technical deep investigation stuff.

00:18:52 Sven Ulke

So if you have systems

00:18:55 Sven Ulke

that look suspicious, or if you know that the attacker executed the ransomware on a specific server, they people took all the data off the server and do analysis and look for forensic evidence that can show you from which system did the attacker come, how was he able to access the system, which users and credentials did he use.

00:19:24 Sven Ulke

which kind of malware did he bring onto the system, and what steps did he do on each of the systems, and what were the steps to come to the next system.

00:19:36 Sven Ulke

And all of this stuff is done in the forensic analysts.

00:19:41 Sven Ulke

And if you want to go one step deeper on the blue team side, then you need so-called malware analysts or reverse engineers, which is a really

00:19:54 Sven Ulke

deep technical role and which have a great expertise in disassembling and decoding malware that is used, for example, the ransomware and what are they usually doing in the ransomware.

00:20:10 Sven Ulke

So in the early days of ransomware, there was a lot of issues in the ransomware itself.

00:20:15 Sven Ulke

So encryption algorithms weren't implemented correct and there were failures in it.

00:20:22 Sven Ulke

Sometimes they didn't encrypt the whole data, just parts of files or the beginning of files.

00:20:29 Sven Ulke

And so malware analysts try to figure out how does the ransomware work.

00:20:35 Sven Ulke

which kind of evidences will be left on systems where the ransomware was run on, and if there's the chance that there's some kind of issue in the coding of the malicious software that might help in recovering the data without having to pay the ransom.

00:20:54 Sven Ulke

And this is something that usually is done by a malware analyst in such kind of incident.

00:21:00 Sven Ulke

And when we

00:21:02 Sven Ulke

got called by customers that say, okay, we have the fear that we are having a cybersecurity incident which a ransomware group is involved or which might lead to ransomware.

00:21:15 Sven Ulke

We nowadays have the problem that ransomware groups are professional attackers.

00:21:21 Sven Ulke

that work exactly the same like red teamers do, and also have same techniques that also maybe state-sponsored espionage groups use.

00:21:32 Sven Ulke

So for us, it's not easy at the beginning to figure out which kind of attacker is in such kind of environment, and especially if before the

00:21:43 Sven Ulke

the encryption ransom part, when we see, okay, there's an attacker in the environment and it's active, we cannot say if this will lead to ransomware or to espionage.

00:21:54 Sven Ulke

And what we can use for that is we can use another role, which is called Cyber Threat Intelligence Analyst.

00:22:01 Sven Ulke

And what is meant by that?

00:22:03 Sven Ulke

Within this role of the blue

00:22:05 Sven Ulke

team, these guys are collecting reports and information about past compromises and attacks and correlate that.

00:22:15 Sven Ulke

So if you find a specific forensic evidence within the ransomware incident, you can ask the cyber threat intelligence analyst and say, have you seen that before?

00:22:28 Sven Ulke

Or have you any information that could tell us

00:22:33 Sven Ulke

that this is such kind of attacker or the attacker in various other cases which are related to the same or might be related to the same attacker group, the attackers did the following.

00:22:45 Sven Ulke

And this really helps on the blue team side to figure out which kind of attacker do we have for what kind of malicious software we have to look out or also to find out

00:22:58 Sven Ulke

What might could have been the initial entry vector to get access to our systems, because attackers are lazy as well.

00:23:06 Sven Ulke

And if attackers know that a special vulnerability works really well, there's a really high chance that they use the exact same vulnerability at different companies and different targets.

00:23:17 Sven Ulke

And so this is also a great resource in the blue team that could help in handling such kind of incidents.

00:23:26 Rene Reuter

Okay, thank you very much.

00:23:27 Rene Reuter

I think we covered already like 5 different roles, which there is still another additional role you might have missed.

00:23:36 Wolfgang Neufeld

I think the last one, the detection engineer, what really is so the role that is evolving right now and which brings all the pieces together.

00:23:47 Wolfgang Neufeld

He has an oversight over all the different roles.

00:23:50 Wolfgang Neufeld

He knows every interface to the others.

00:23:53 Wolfgang Neufeld

And

00:23:53 Wolfgang Neufeld

tries to identify so-called indicators of compromise, which are specific patterns that in the end could lead to specific groups.

00:24:05 Wolfgang Neufeld

And then you can tell, okay, this was some kind of ransomware group that we already know.

00:24:11 Wolfgang Neufeld

And they usually do that.

00:24:14 Wolfgang Neufeld

There's also a big help for the blue team in the end that if you know, for example, it's Black Basta Group, then they usually do the following steps.

00:24:21 Wolfgang Neufeld

They go for the DC first.

00:24:24 Wolfgang Neufeld

They use emails to come into the organization, something like that.

00:24:30 Wolfgang Neufeld

helps A lot.

00:24:31 Wolfgang Neufeld

And the detection engineer tries then to summarize all these kind of indicators of compromise and to improve the systems that you have and to shape them and make them more aware of what could go wrong or what did go wrong and try it in the end to prevent it so that it never happens again or that you can detect it in the future more easily.

00:24:59 Rene Reuter

Okay, so thank you very much.

00:25:01 Rene Reuter

So I think we covered a lot the different personas which are part of a blue team.

00:25:05 Rene Reuter

I think what we also need to mention is it's not only about personas, but it's also about tools.

00:25:10 Rene Reuter

And there's a myriad of tools around which you can buy to help the blue team basically do their work much more efficient.

00:25:19 Rene Reuter

Just speaking of SIEM tools, security incident event management tools, we have EDR tools, endpoint detection and response tools, which are the new

00:25:29 Rene Reuter

form of antivirus software, usually installed on clients and servers.

00:25:34 Rene Reuter

We have network monitoring tools.

00:25:36 Rene Reuter

We have sensors we need to install in our network to actually make detections.

00:25:42 Rene Reuter

But what we see is you can easily buy from different vendors this kind of security appliances, this kind of tools.

00:25:50 Rene Reuter

But I think what is crucial to understand is

00:25:52 Rene Reuter

The work usually starts when you're going to deploy this appliance or software into your company.

00:25:58 Rene Reuter

It's not ending there.

00:26:00 Rene Reuter

This is where you have to define your processes.

00:26:01 Rene Reuter

This is where you have to train your members of your blue team.

00:26:06 Rene Reuter

And you have to basically make use of those tools.

00:26:09 Rene Reuter

It's not ending with just installing an appliance and then saying to yourself, okay, it seems that I'm secure now.

00:26:15 Rene Reuter

Let's wait if I see an alert.

00:26:18 Rene Reuter

to basically invest a little bit of further education also inside your company for that.

00:26:25 Rene Reuter

But what kind of challenges are there besides that?

00:26:29 Rene Reuter

What do you want to see typical challenges for companies if we are looking at team?

00:26:36 Wolfgang Neufeld

I think for me, the favorite part is to

00:26:40 Wolfgang Neufeld

have the whole blue team part internally, but that is something that only works for very big companies with a huge budget also.

00:26:49 Wolfgang Neufeld

And there's also the problem of if you know what you need, many, many companies already struggle to define what kind of roles and what kind of expertise you really need to defend in the end.

00:27:03 Wolfgang Neufeld

So this is something that's very complicated and very complex.

00:27:07 Wolfgang Neufeld

And for that, you already need a big company to handle that, in my opinion.

00:27:13 Wolfgang Neufeld

And so the first thing that everyone thinks, okay, can't I buy that from outside, from a service provider?

00:27:21 Wolfgang Neufeld

And I think there's pros and cons, and I think it works for depending on the company you have.

00:27:29 Wolfgang Neufeld

And I think Sven is for the external company, for sure.

00:27:33 Wolfgang Neufeld

So I would hand over to Sven for his...

00:27:37 Wolfgang Neufeld

insight on that.

00:27:38 Wolfgang Neufeld

Yeah.

00:27:39 Sven Ulke

Sure.

00:27:40 Sven Ulke

So exactly as you said, we do a lot of this kind of roles and offer a lot of kind of the services from as an external party.

00:27:49 Sven Ulke

And we are mainly focused on small, medium business and mid-sized companies because it's exactly that.

00:27:59 Sven Ulke

Like it's really hard to find all this kind of

00:28:03 Sven Ulke

expertise on the job market, cybersecurity talent shortage is even bigger than IT security challenge, but exactly as Wolfgang said.

00:28:14 Sven Ulke

So it has pros and cons.

00:28:18 Sven Ulke

And we can bring a lot of stuff from the external side, but even though if we are involved in an incident, we always need some counterparts on the internal side.

00:28:28 Sven Ulke

So

00:28:30 Sven Ulke

If we're looking at a technical perspective, how the incident happened, and if we were talking about, okay, how can we get business up and running, and how can we get rid of the situation, we can say from a technical perspective, okay, we have to do this, we have to do this, we have to rebuild this kind of systems, but we do not have the exact knowledge about the internal processes, the applications, the priorities, the dependencies, and which kind of

00:28:58 Sven Ulke

a row, you have to restore some services that are dependent on each other.

00:29:02 Sven Ulke

For all kind of this stuff, we need internal expertise.

00:29:07 Sven Ulke

But also, we bring a lot of knowledge with us because we hope that if you look at the internal side, that you have such kind of huge incidents, hopefully just once in five years and not every year.

00:29:23 Sven Ulke

So we from the external side

00:29:25 Sven Ulke

have a huge look also at other companies of various sectors and can say, okay, this kind of attackers usually do this and that and looking out for this and that.

00:29:36 Sven Ulke

This is also knowledge that you can get from the external side as well.

00:29:41 Sven Ulke

And in the perfect world, it's some kind of hybrid setup where you have an internal

00:29:49 Sven Ulke

internal team with a few resources stuff there.

00:29:53 Sven Ulke

And if you really need it, you put additional services on it or additional experts from the external side.

00:30:00 Sven Ulke

And this is also for us the most beneficial setup to work in such kind of incidents.

00:30:07 Wolfgang Neufeld

So kind of combination.

00:30:08 Sven Ulke

Yeah.

00:30:09 Rene Reuter

Well, makes sense.

00:30:10 Wolfgang Neufeld

I think for small and medium businesses, that's exactly where the crucial part is.

00:30:16 Wolfgang Neufeld

to get as much knowledge already and as much people involved and as much sensors and detection that an external party can help you in the end.

00:30:26 Wolfgang Neufeld

So if you don't do logging, if you don't do EDRs and all that kind of stuff, the benefit from an external company can quite fast be very limited because they have to do a lot of guesswork and they can't even do the forensics afterwards and help you.

00:30:43 Wolfgang Neufeld

How the attacker

00:30:45 Wolfgang Neufeld

How did the attacker get in?

00:30:46 Wolfgang Neufeld

On what systems did the attacker maybe look around or whatever?

00:30:51 Wolfgang Neufeld

If you don't have that kind of monitoring, at least in your environment, then there's also very limited was what the external company can do then.

00:31:01 Wolfgang Neufeld

So it's at least for the small and medium businesses, you should at least try to come to a state where you can say, okay, here we have detected that we are

00:31:13 Wolfgang Neufeld

Yeah, we got hacked that we have ransomware, suspicious behavior, and we have some kind of big lock sources where you can then hand it over to an external party that then can really help you.

00:31:26 Wolfgang Neufeld

If you don't have that and not the know-how to build that up, this is something that I would advise to get at least to that point.

00:31:34 Wolfgang Neufeld

And this is already a big sum of money that you have to invest on resources.

00:31:39 Wolfgang Neufeld

which is already hard for small and medium business, but doing that completely external, this is, in my opinion, that is an illusion to be able to do that.

00:31:49 Wolfgang Neufeld

Yeah.

00:31:50 Rene Reuter

Okay.

00:31:51 Rene Reuter

And do you think maybe AI can help tackling that problem a little bit?

00:31:56 Sven Ulke

Yeah, great that you're, Vinny, you are really a salesman.

00:32:00 Sven Ulke

So especially when I talk to sales guys from various companies and resellers, so AI is the solution for everything.

00:32:09 Sven Ulke

No, so to stop joking, it's really impressive how AI involved in this short period of time.

00:32:19 Sven Ulke

And what we can say is that AI is definitely used on the attacker side and might be not in such kind of fancy ways as you might imagine.

00:32:32 Sven Ulke

Like what we are seeing is that some kind of attackers have

00:32:37 Sven Ulke

knowledge gaps in such kind of coding skills or programming skills.

00:32:42 Sven Ulke

And they use AI for that to maybe develop their malware faster or to develop new kind of ways of how malware behaves, which they would have or had to invest much of money to get this kind of knowledge.

00:32:57 Sven Ulke

So attackers use this as well, and especially they use this also for translation purposes.

00:33:03 Sven Ulke

So nowadays you have attacking groups that are usually working in South America, for example.

00:33:09 Sven Ulke

They are also now targeting Europe because AI helps them to translate their kind of malicious software also in German language or in other languages as well.

00:33:19 Sven Ulke

So this is stuff that attackers already do.

00:33:23 Sven Ulke

And also on the defender side, AI can really help

00:33:27 Sven Ulke

in speeding up your analysis, like if you're not so experienced in some tooling or maybe coding, you can also get help on that, especially if you have to walk through a huge amount of data.

00:33:40 Sven Ulke

This can be speeded up with it.

00:33:43 Sven Ulke

And also, AI can give you context information or like a kind of better Googling if you are

00:33:54 Sven Ulke

figuring out some kind of malicious behavior, and you are asked, okay, which kind of software this might be related to.

00:34:01 Sven Ulke

But there's still work to do, and I have to work for more than over 30 years, and I think until retirement, even though with AI, we will have lots of things and cases and incidents to solve.

00:34:17 Wolfgang Neufeld

Okay.

00:34:19 Wolfgang Neufeld

When we are now talking about all these kind of ransomware stuff, and we more or less had the enterprise side of that, by enterprise, I mean the office world and all the e-mail stuff, and which is quite known to be attacked.

00:34:35 Wolfgang Neufeld

So, and everyone knows ATAS is an automotive company, or at least has its source from there that it's working for automotive, and it's very strong there.

00:34:47 Wolfgang Neufeld

Do you see any of the attacks evolving also on the automotive side currently?

00:34:51 Wolfgang Neufeld

Is that a thing?

00:34:52 Wolfgang Neufeld

Do I need to invest in EDRs and stuff like that and sensors also for the automotive world?

00:34:59 Rene Reuter

Yeah, I can cover that.

00:35:02 Rene Reuter

If you look at the current development in the automotive industry, we are facing the so-called SDV, software defined vehicle.

00:35:11 Rene Reuter

So means nowadays cars,

00:35:14 Rene Reuter

They're not closed anymore, like 10 years or 15 years ago.

00:35:18 Rene Reuter

No, they're usually defined by software.

00:35:20 Rene Reuter

They have connections to various back-end systems, to cloud systems.

00:35:24 Rene Reuter

They even have connections to each other, car-by-car communication is possible.

00:35:29 Rene Reuter

So the threat landscape nowadays is much more broader than in the past.

00:35:34 Rene Reuter

So if you look back a little bit to the enterprise IT, I guess in the future, the automotive industry, and especially cars, will face nearly the same attack surface like what we have currently in the enterprise IT.

00:35:48 Rene Reuter

I mean, just look at the technologies.

00:35:50 Rene Reuter

So in the past, ECUs were highly customized.

00:35:55 Rene Reuter

developed in C with a dedicated firmware running on the ECU.

00:35:59 Rene Reuter

Nowadays, we see already embedded Linux being part of that.

00:36:03 Rene Reuter

We have Bluetooth connections.

00:36:05 Rene Reuter

We have Wi-Fi in cars nowadays.

00:36:08 Rene Reuter

And again, communication to the back end basically means there's also a communication channel from the back end towards the car possible.

00:36:15 Rene Reuter

Just look at Tesla, who's providing a huge API interface where you can basically, with a simple access token,

00:36:24 Rene Reuter

Connect to the car and ask various insights about the car, like current battery capacity, even geolocation, as possible just by using an API as simple as we are using it at the back-end system.

00:36:38 Rene Reuter

So looking a little bit at the boot teams, I guess the car of the future also needs to have sensors inside the car.

00:36:47 Rene Reuter

There needs to be a so-called vehicle SOC, vehicle security operations center, who is going to collect block data from cars driving on the street and then trying to also collect blocks from the backend system and doing a triage there to see if someone is maybe attacking the current fleet of an OEM.

00:37:07 Rene Reuter

Looking in the past 12 months, we saw a lot of attacks in the automotive industry where the attackers were basically able to take over the fleet using the backend systems.

00:37:16 Rene Reuter

Because again, what is just mentioned, they are all providing APIs which you can basically talk to a whole fleet.

00:37:22 Rene Reuter

I guess this is a little bit of a doomsday scenario here.

00:37:25 Rene Reuter

If you can take over a backend system and then basically take over the whole fleet.

00:37:29 Rene Reuter

So monitoring sensors, logging,

00:37:33 Rene Reuter

This will all come to the car in the future, like we've seen it currently in the enterprise world.

00:37:39 Rene Reuter

Yeah.

00:37:40 Wolfgang Neufeld

Just the natural evolvement, whatever brings money.

00:37:43 Wolfgang Neufeld

And then you might even have also the safety topic in the end by cars.

00:37:48 Wolfgang Neufeld

If that is not separated and maybe interfaces, so this can have serious consequences.

00:37:54 Wolfgang Neufeld

So.

00:37:55 Rene Reuter

Just referencing to an attack.

00:37:58 Rene Reuter

published last week by the PwC Automotive Group.

00:38:01 Rene Reuter

I think it was a Nissan hack where they were also able, via a Bluetooth connection, to hack into the car and basically make a persistence that we're able to control using command control server towards themselves.

00:38:15 Rene Reuter

And in the end, they were able to basically manipulate the steering wheel during the driving.

00:38:21 Rene Reuter

So this has some serious safety impacts now already.

00:38:25 Wolfgang Neufeld

That are really some doomsday scenarios that we should really prepare for in the future to detect them and prepare that this never happens.

00:38:34 Wolfgang Neufeld

And yeah.

00:38:35 Rene Reuter

Definitely.

00:38:36 Rene Reuter

So I guess we covered pretty good the history.

00:38:39 Rene Reuter

We covered a little bit the challenges which are for internal or external blue teams.

00:38:46 Rene Reuter

But despite that, do you think what are the further challenges for blue teams nowadays?

00:38:53 Sven Ulke

So you said it in the beginning, like when you buy appliances or install appliances, this is not solving the issue.

00:39:00 Sven Ulke

And that's exactly what we see.

00:39:02 Sven Ulke

So in this kind of connected world and more and more data is collected, it's really hard to finding the right balance for detection capabilities.

00:39:10 Sven Ulke

And also to, at the same stage, don't overwhelm the analysts in the security operations center.

00:39:18 Sven Ulke

So you need detection capabilities.

00:39:21 Sven Ulke

that are showing you if something is going on at a really early stage, but you really want to have a small amount of false positives, because otherwise, if a lots of false alarms come into your security operations center, there's a really high chance that your security analysts oversee really true positive alarms, because they are flooded by a huge amount of false positives,

00:39:46 Sven Ulke

And this is called alert fatigue to being so overwhelmed by false positives that you do not see the current attack ongoing.

00:39:54 Sven Ulke

This is 1 huge challenge.

00:39:56 Sven Ulke

And also, exactly as you described it in the car world, on the blue team side, you're always one step behind the attackers.

00:40:05 Sven Ulke

So even though if the blue team nowadays tries to be proactive and implement detections as Wolfgang described, attackers are evolving.

00:40:14 Sven Ulke

They are

00:40:15 Sven Ulke

finding new techniques to attack your infrastructure, they're finding new vulnerabilities, and you always have to react to that.

00:40:24 Sven Ulke

And especially what we see in the last months that is ongoing, that a lot of customers improved really their security posture, broad sensors in the network, and what are attackers nowadays doing?

00:40:36 Sven Ulke

They do not

00:40:38 Sven Ulke

in the ransomware example, encrypt on the system itself, they go to the hypervisor level and encrypt on the hypervisor stage because they knew there is no such kind of security software that is able to be run on hypervisors itself.

00:40:54 Sven Ulke

So this is such kind of what attackers do.

00:40:58 Sven Ulke

And on the blue team side, you always have to cover that, especially also when you think about this kind of complex IT environments,

00:41:06 Sven Ulke

where your on-premise infrastructure gets connected to the cloud world and all is synchronized together.

00:41:13 Sven Ulke

Maybe you have several hyperscalers or several clouds that are connected together.

00:41:18 Sven Ulke

So this is also, as you described, it makes a really huge threat landscape and attack landscape.

00:41:25 Sven Ulke

And all these kind of challenges are always compared to this, as we said in the beginning, this kind of talent shortage that we all have.

00:41:37 Sven Ulke

this is one of the major problems or challenges that on the blue team side is there for the last years and it's an ongoing problem.

00:41:48 Sven Ulke

Yeah, and maybe to add one point to the talent shortage and exactly what we said in the beginning, also on the blue team side, what is a problem that is often underestimated is

00:42:03 Sven Ulke

especially as Wolfgang described, there are lots of tools, sensors, network monitoring, EDR system, SIEM systems.

00:42:11 Sven Ulke

So on the blue team side, you have a huge tool landscape to handle and to work with.

00:42:17 Sven Ulke

And these systems are all in itself complex and most of the time, they are not really good integrated into each other.

00:42:25 Sven Ulke

So what is meant by that is that if some part of this blue teaming tools

00:42:32 Sven Ulke

throwing alerts.

00:42:34 Sven Ulke

It could be that the analyst has to jump into three, four, or five different systems or tools to figure out what really happened.

00:42:42 Sven Ulke

And it's really hard for them to bring all that together to get the overall view.

00:42:48 Sven Ulke

And this is a really, really challenging work to do.

00:42:53 Sven Ulke

And this is also one major challenge.

00:42:55 Sven Ulke

What we see

00:42:57 Sven Ulke

what where customers are currently struggling with, especially if they already have a bigger team and onboarded experts and also got some of this kind of security tools already in place.

00:43:11 Rene Reuter

Okay, thank you very much.

00:43:13 Rene Reuter

I think we covered a very good ground in trying to explain what are blue teams, what are their challenges,

00:43:20 Rene Reuter

What are the tools they need to do?

00:43:23 Rene Reuter

So hopefully you really liked that episode and we were able to highlight it a little bit to you.

00:43:29 Rene Reuter

What is the work of blue teams?

00:43:31 Rene Reuter

I want to say thank you, Sven.

00:43:33 Rene Reuter

Thank you, Wolfgang.

00:43:35 Rene Reuter

Thank you for listening.

00:43:36 Rene Reuter

I hope you enjoyed this episode.

00:43:38 Rene Reuter

Stay tuned for the last one in this series about purple teaming.

00:43:42 Rene Reuter

Thank you very much.

00:43:43 Rene Reuter

Goodbye.

00:43:47 Voiceover

Thank you for joining this episode of the Empowering Tomorrow's Automotive Software Podcast.

00:43:51 Voiceover

Please leave a comment or review with your feedback or what you'd like to hear in future episodes.

00:43:57 Voiceover

To learn more about Automotive Embedded Systems and ETAS's capabilities, visit our website at ETAS.

00:44:02 Voiceover

That's ETAS.com.