Generative AI Security in the Automotive Domain

 In this episode, host Zane Pelletier speaks with Ivan Granero from Bosch about the use and security of generative AI (GenAI) in the automotive domain. The conversation covers the transition from traditional machine learning to probabilistic GenAI models and takes a deep dive into the security challenges of large language models (LLMs) in a safety-critical environment. Zane and Ivan also talk about common threats, such as "jailbreaking" and injection attacks, and explore defense mechanisms like implementing "guardrails" and the "spotlighting method" to protect system prompts.

If you are an automotive software developer, cybersecurity professional or AI engineer working int he connected vehicle space - this episode is a must listen!

To view the tool Zane and Ivan mentioned in their discussion, click here. 

Thanks for listening!

• Email us at: contact.us@etas.com
• Learn more about ETAS on our website 
• Follow us on LinkedIn: @ETAS

Transcript

00:00:02 Voiceover

Welcome to the Empowering Tomorrow's Automotive Software Podcast, brought to you by ETAS, a single source of cutting-edge software and hardware solutions that make automotive embedded systems safe, smart, secure, and sustainable.

00:00:15 Voiceover

Each episode, we'll be joined by ETAS and industry experts to discuss how electrification, automation, and connectivity are impacting the automotive industry.

00:00:25 Voiceover

Now, sit back and enjoy the discussion.

00:00:32 Zane Pelletier

Hello, everybody, and welcome to the Empowering Tomorrow's Automotive Software Podcast.

00:00:37 Zane Pelletier

I am your host, Zane Pelletier.

00:00:39 Zane Pelletier

I am a penetration tester and reverse engineer here with ETAS.

00:00:43 Zane Pelletier

And today, I am very happy to introduce to you Ivan Granero, who is a senior security engineer at Bosch.

00:00:51 Zane Pelletier

We're going to be talking about primarily the use of generative AI in the automotive domain.

00:00:58 Zane Pelletier

We're also going to be jumping into a little bit of common threats to Gen.

00:01:02 Zane Pelletier

AI applications and other areas for which we might use different language models in the space.

00:01:09 Zane Pelletier

And I'd also like to talk about the future of Gen.

00:01:12 Zane Pelletier

AI and what this could mean for the automotive industry.

00:01:16 Zane Pelletier

My guest today, Ivan, is a cybersecurity specialist with more than 20 years working in automotive software and hardware.

00:01:23 Zane Pelletier

Currently, he specializes in red teaming and securing generative AI and large language models to advance the safety and innovation in AI technologies.

00:01:32 Zane Pelletier

Welcome, Ivan.

00:01:32 Zane Pelletier

Do you want to give us a little bit of background on yourself?

00:01:35 Zane Pelletier

And maybe as a little bit of a primer, you could tell us how you use generative AI tools in your day-to-day tasks.

00:01:43 Ivan Granero 

Thank you, Zanen.

00:01:43 Ivan Granero 

Yeah, thank you for the really good introduction.

00:01:47 Ivan Granero 

And yeah, I think, I mean, you've pretty much covered most of the things that I'm doing and working on right now.

00:01:53 Ivan Granero 

If I can add something, it's a little bit more of my background.

00:01:57 Ivan Granero 

So actually my career or background studied as an electrical engineer.

00:02:03 Ivan Granero 

That's my bachelor's.

00:02:06 Ivan Granero 

And later on, my curiosity and just as a hobby or a side activities, I continued to do a bunch of programming and that

00:02:19 Ivan Granero 

That sparked my curiosity into continuing my studies.

00:02:22 Ivan Granero 

And I did a, I decided on a master's in software engineering with a focus in cybersecurity.

00:02:28 Ivan Granero 

That was out of the University of Texas at El Paso.

00:02:31 Ivan Granero 

So that pretty much helped me into learning all of the, most of the tricks and basis for what I'm doing right now, right?

00:02:41 Ivan Granero 

And it overlaps a little bit what you do as well as a pen tester, I will say.

00:02:47 Ivan Granero 

I'm trying to, of course, know what the pen testing teams or hacking red teams are doing, and also on the blue team side, right?

00:02:56 Ivan Granero 

So I'm trying to protect the systems, adding layers of protections to our products at Bosch, you name it, right?

00:03:04 Ivan Granero 

From engine controllers to gateways, going through all the different embedded devices and even sensors nowadays.

00:03:13 Ivan Granero 

As we all know,

00:03:16 Ivan Granero 

AI is now, we all have to use it.

00:03:18 Ivan Granero 

That's my, I guess, personal opinion.

00:03:20 Ivan Granero 

So of course, it's getting into cybersecurity, both sides, blue team, red team.

00:03:26 Ivan Granero 

So it's important that at different scales, of course, we all get to use it.

00:03:32 Ivan Granero 

And yeah, I think that's, we're going to be talking about a few use cases.

00:03:37 Ivan Granero 

But yeah, pretty much that's how I started into the cybersecurity world.

00:03:41 Ivan Granero 

And

00:03:43 Ivan Granero 

for the last year into Gen.

00:03:45 Ivan Granero 

AI specifically, but I would say just into machine learning activities within cyber security.

00:03:52 Zane Pelletier

Yeah, definitely.

00:03:53 Zane Pelletier

Yeah, I mean.

00:03:55 Zane Pelletier

I definitely, I know that you're the bridge between the two sides, right?

00:03:59 Zane Pelletier

And I know we've had some discussions about this before.

00:04:02 Zane Pelletier

I guess in my mind, it seems that there are two different general applications of AI currently that I'm seeing, at least in the automotive space.

00:04:11 Zane Pelletier

So there's the utilization of, let's say, I'll call them AI tools,

00:04:17 Zane Pelletier

usually in the form of some kind of large language model that's connected to some other tooling or information source, right, to actually secure embedded applications.

00:04:27 Zane Pelletier

So I'll call this kind of the off-board side of things, the utilization of using that in the security practice of hardening, for instance, or finding security issues, right?

00:04:37 Zane Pelletier

And then I do see some discussion of using some systems

00:04:42 Zane Pelletier

most applicably like I think traditional machine learning models, right, in actual use in the vehicle or the vehicle backend, right, for ADAS, IDS, those types of things, right?

00:04:55 Zane Pelletier

So I guess my question to you kind of seeing this through and being in the space for the amount of time that you have, right, and having all the experience you have, how do you think that Gen.

00:05:04 Zane Pelletier

AI has changed and will continue to change, you know, for instance, in the automotive space?

00:05:10 Ivan Granero 

Yeah, I mean, you mentioned machine learning models, right?

00:05:13 Ivan Granero 

Deterministic machine learning models.

00:05:15 Ivan Granero 

And we've transitioned into more probabilistic now.

00:05:19 Ivan Granero 

And you're mentioning Gen.

00:05:21 Ivan Granero 

AI, right?

00:05:21 Ivan Granero 

Just to make a quick distinction there.

00:05:25 Ivan Granero 

Yeah, I mean, I think, and maybe that's my personal opinion since the 2017 with the famous paper, attention is all you need.

00:05:33 Ivan Granero 

I think that was the main transformation

00:05:38 Ivan Granero 

That paper was what made AI, I would say, evolve into what we know as Gen.

00:05:47 Ivan Granero 

AI now.

00:05:48 Ivan Granero 

And thanks to that, and also thanks to the powerful computing, especially talking about GPUs, that's what

00:05:58 Ivan Granero 

help everyone into jumping into this big ship, right?

00:06:02 Ivan Granero 

That's one of the two main things I will say.

00:06:05 Ivan Granero 

So one, the computing power, and the second one, the Gen.

00:06:10 Ivan Granero 

AI using the attention is all you need, right?

00:06:13 Ivan Granero 

For just talking about cybersecurity in general, you mentioned IDS.

00:06:18 Ivan Granero 

One other thing that I've noticing, yeah, everybody's trying to use Gen.

00:06:23 Ivan Granero 

AI, but it's important to also

00:06:27 Ivan Granero 

understand all the legacy machine learning models that are out there sometimes.

00:06:33 Ivan Granero 

It depends on your application.

00:06:34 Ivan Granero 

You don't need to use GenAI.

00:06:37 Ivan Granero 

If you are doing a phish e-mail detection or classification, maybe just a Naïve Bayes classification algorithm is good enough.

00:06:47 Ivan Granero 

So you don't need all the compute power and you don't need to spend all the money.

00:06:53 Ivan Granero 

So I just want to

00:06:55 Ivan Granero 

mentioned that it's important to continue to use all the machine learning models that are out there.

00:07:02 Ivan Granero 

I mean, Gen.

00:07:02 Ivan Granero 

AI is at the end, it's just another machine learning model, right?

00:07:06 Ivan Granero 

But with the use of the compute power that we have nowadays, we are able to multiply millions of matrices at the same time.

00:07:18 Ivan Granero 

And thanks to that, we are able to predict the next token, I will say.

00:07:22 Ivan Granero 

And luckily, we are using this powerful prediction for not only detection, but also for pen testing, I will say.

00:07:34 Zane Pelletier

Definitely, yeah.

00:07:35 Zane Pelletier

And I really liked...

00:07:36 Zane Pelletier

what you said regarding the know when to use AI.

00:07:41 Zane Pelletier

I think that that's just such a generalized term at this point, like using generative AI versus a traditional machine learning model.

00:07:47 Zane Pelletier

And there are some problems that we still solve.

00:07:50 Zane Pelletier

I think that it's better to just have some kind of deterministic solution, right?

00:07:55 Zane Pelletier

It doesn't always have to be so dynamic, right?

00:07:59 Zane Pelletier

I think that's part of framing the problem is, you know,

00:08:02 Zane Pelletier

A lot of people have kind of, I think, phrased this as a silver bullet, right?

00:08:05 Zane Pelletier

It's going to solve all the problems, right?

00:08:07 Zane Pelletier

And it's going to do it efficiently.

00:08:08 Zane Pelletier

And I think we're seeing now that isn't necessarily the case.

00:08:12 Zane Pelletier

But I do think that there are a lot of problems that some of these solutions can solve.

00:08:16 Zane Pelletier

And I do think that regardless of whether or not this is the best way to do things, people are integrating these into more traditional systems at breakneck pace right now.

00:08:26 Zane Pelletier

I mean, we're seeing it in every facet of every industry, essentially, somebody's trying to

00:08:32 Zane Pelletier

to integrate these new tools.

00:08:34 Zane Pelletier

So with that being said, the large number of integrations, let's say, for these LLM-based applications, I know that these systems are notoriously very difficult to secure, right?

00:08:47 Zane Pelletier

Even harder to ensure things like safety.

00:08:50 Zane Pelletier

You know, I think in the automotive space, we hear a lot about functional safety.

00:08:53 Zane Pelletier

And a lot of times, this is very much interwoven and connected to security and embedded security for these systems on the vehicles.

00:09:02 Zane Pelletier

So for Gen.

00:09:04 Zane Pelletier

AI applications in general, and I guess in the automotive space and maybe focusing in a little bit more, do you have any general recommendations for being able to secure things like this?

00:09:16 Zane Pelletier

And especially for those who are responsible for building out these applications, are there any guardrails or can you identify any kind of areas that typically I would say fail in some of these non-deterministic systems?

00:09:31 Ivan Granero 

Yeah, that's a very good question.

00:09:34 Ivan Granero 

And that's one of the things that we have to be cautious about, right?

00:09:38 Ivan Granero 

When we start using, especially Gen.

00:09:42 Ivan Granero 

AI with the prediction that we all know that might hallucinate.

00:09:46 Ivan Granero 

So how do we keep it safe, right?

00:09:49 Ivan Granero 

How do we keep it secure?

00:09:51 Ivan Granero 

I will backtrack a little bit just so we all understand what I'm going to try to say.

00:09:57 Ivan Granero 

So we were talking about how the

00:10:00 Ivan Granero 

attention mechanism jumped the industry, right?

00:10:03 Ivan Granero 

Reason why, because now we're able to not only understand one part of the input, we're able to correlate that one part of the input to everything around, right?

00:10:16 Ivan Granero 

So you're pretty much asking, okay, what's the relationship in between these, let's say, work compared to the other ones?

00:10:23 Ivan Granero 

And that solves a lot of the issues that legacy machine learning models had, such as long, short-term models or recurrent neural networks.

00:10:36 Ivan Granero 

So thanks to that, we're able to have the GenAI in predictions, and we're able to pretty much have a natural language processing.

00:10:43 Ivan Granero 

And after that, we have the LLMs, which are using this attention mechanism as a base

00:10:51 Ivan Granero 

for what is called the transformer, right?

00:10:54 Ivan Granero 

And basically, you are converting the input based on all the relationships in that one big, let's say, paragraph, converting it or transforming it into one new vector.

00:11:13 Ivan Granero 

So that's the main reason why it's called a transformer.

00:11:16 Ivan Granero 

Now,

00:11:18 Ivan Granero 

We are all using it, let's say, ChatGPT, right?

00:11:21 Ivan Granero 

We ask questions and it will answer to you based on the training and cutoff dates that it has.

00:11:30 Ivan Granero 

After that, we introduce the tool calling.

00:11:33 Ivan Granero 

And that's where I'm going to jump into the security of it, right?

00:11:36 Ivan Granero 

So now with tool calling, we're able to not only talk to it, I will say,

00:11:42 Ivan Granero 

and it's not only replying back to us, but it needs extra information.

00:11:46 Ivan Granero 

Let's say it needs to do a query out of a database.

00:11:51 Ivan Granero 

It needs to do a web search.

00:11:53 Ivan Granero 

So now it's telling you, okay, I need to do this.

00:11:58 Ivan Granero 

So there are different frameworks to automate this, such as LangChain, for example, or you can just create.

00:12:06 Ivan Granero 

create your own in Python, right?

00:12:08 Ivan Granero 

So you will get it back and then you will call your Python function or whatever you need to do.

00:12:14 Ivan Granero 

And that way you have tool calling, right?

00:12:17 Ivan Granero 

That's what created what now we know as agentic AI.

00:12:22 Ivan Granero 

With that, it's not only, you're not only talking to an LLM model.

00:12:28 Ivan Granero 

Now the safety or I will say the security aspect is very important.

00:12:34 Ivan Granero 

We've seen cases where

00:12:36 Ivan Granero 

We're introducing chatbots, for example, and we're given access to a database with tool calling.

00:12:44 Ivan Granero 

And maybe the red teaming or a hacker is able to get unlimited coupons or unlimited free flights based just by talking to the chatbot.

00:12:55 Ivan Granero 

So your question was, how do you protect for it, right?

00:12:59 Ivan Granero 

So there are different guardrails.

00:13:03 Ivan Granero 

for an LLM in particular in this case.

00:13:06 Ivan Granero 

And the main idea is try to aid or try to help the LLM to distinguish in between what is the system prompt and the user prompt and the rest of the things that are getting into it.

00:13:22 Ivan Granero 

Because for the LLM at the end, everything that goes into it is just tokens.

00:13:26 Ivan Granero 

It doesn't know better.

00:13:28 Ivan Granero 

It doesn't know the difference in between what is the user prompt

00:13:31 Ivan Granero 

or if it's coming from a RAG, or if it's coming from a web search, it's all inputted into the LLMS tokens.

00:13:39 Ivan Granero 

So the one I will mention, I guess, because it's really simple to understand, is basically you are highlighting the user prompt, spotlighting method.

00:13:56 Ivan Granero 

As part of the system prompt, you introduce something as, okay,

00:14:02 Ivan Granero 

Next, I'm going to give you an user prompt, and it's going to be delimiter or delimited by brackets.

00:14:08 Ivan Granero 

Or the user prompt is going to be in base 64.

00:14:12 Ivan Granero 

That way, the LLM knows, okay, what everything that is base 64 is going to be the user prompt.

00:14:19 Ivan Granero 

And it's going to make it more difficult for the, I would say, the hacker

00:14:26 Ivan Granero 

to get out of the user prompt, right?

00:14:28 Ivan Granero 

Because when you're trying to jailbreak an LLM, that's what you're trying to do.

00:14:33 Ivan Granero 

Like break out of the user prompt and get access to the system prompt or get access to the tool calling and all the tools that you are trying to reach out, right?

00:14:44 Ivan Granero 

So that's the basic idea of a guardrail.

00:14:48 Ivan Granero 

So basically helping the LLM to distinguish the difference and of course,

00:14:55 Ivan Granero 

trying to keep the user in the user front.

00:15:00 Zane Pelletier

Sure, definitely.

00:15:01 Zane Pelletier

I guess I could even kind of equate this to something like a sandbox escape, right?

00:15:06 Zane Pelletier

There's a system, there are a certain number of capabilities that a system has, right, with a traditional, like, let's say, a web browser or a virtualized machine, right?

00:15:18 Zane Pelletier

And in the overall system, there's certain things that, let's say, the host machine has access to that you don't necessarily want

00:15:25 Zane Pelletier

for your virtualized machine to get access to, for instance.

00:15:29 Zane Pelletier

But there are still, I feel like, capabilities, some of these systems, we find there are ways to escape kind of that sandboxed environment and get access to some of those features that traditionally we don't want the internal system to have access to.

00:15:44 Zane Pelletier

So if you think about it in that way, you can kind of understand the threat that you have here with a user being able to get access to

00:15:52 Zane Pelletier

Like you were saying, the tools that the model can use, or the way that the system prompt was designed, and you can find weaknesses there to kind of leverage to, I would say, get the model to perform actions that it shouldn't necessarily be performing or shouldn't be allowed to perform.

00:16:09 Zane Pelletier

So, that's a really interesting aspect of this.

00:16:12 Zane Pelletier

I think, in some ways, I have heard people compare breaking large language models or generative AI as a very, very specific intersection of...

00:16:27 Zane Pelletier

technical hacking and social engineering almost, because it's like taking advantages of the more natural language features of this type of system, kind of in the same way you would trick a person to give you information or do something that maybe they shouldn't be doing for you.

00:16:43 Zane Pelletier

And then also just the technical aspect of the fact that this is still an electronic device that has some amount of deterministic inputs, right?

00:16:50 Zane Pelletier

So that's really interesting.

00:16:53 Zane Pelletier

I guess in your experience, what would you say, if

00:16:57 Zane Pelletier

If you're taking a look at the security of the system, what is a certain technical mindset shift?

00:17:03 Zane Pelletier

I find that with different systems I look at, for instance, I need to kind of change the way that I'm framing or thinking about the intrinsic security of the device to understand what are some things I could do to try to circumvent some of the security mechanisms that are in place.

00:17:18 Zane Pelletier

I guess, how do you think about it when you're approaching a Jain AI system or a large language model-based application?

00:17:25 Ivan Granero 

I think you hit the nail on the head, right?

00:17:27 Ivan Granero 

It's totally a mix of social engineering, I will say, and the technical hacking methods that you will usually use.

00:17:36 Ivan Granero 

I mean, social engineering is one of them, right?

00:17:38 Ivan Granero 

But I will say a lot of it will be social engineering in the case of jailbreaking.

00:17:44 Ivan Granero 

And of course, I was mentioning the limiters.

00:17:47 Ivan Granero 

All of the things that we know from cybersecurity,

00:17:52 Ivan Granero 

rules still apply to LLM, for example, what we know in SQL or SQL injection.

00:17:59 Ivan Granero 

So all you're trying to do there is break out of the limiters.

00:18:04 Ivan Granero 

So the same thing applies for an LLM.

00:18:06 Ivan Granero 

If you know the limiters that are being used for that particular guardrail, you can just put it in the user prompt and the next thing is going to be out of it, right?

00:18:16 Ivan Granero 

So it's the same, some of the same concepts still apply.

00:18:22 Ivan Granero 

Because at the end, we are applying similar protection or guardrails to it, the same things that we apply on other cybersecurity guardrails, right?

00:18:34 Ivan Granero 

I mean, not only for AI, but it's important, or I guess one of the things that are different compared to, for example, when we're talking about implementing.

00:18:46 Ivan Granero 

AI in embedded devices, one of the challenges, it will be, I will say the biggest challenge, it will be computing power.

00:18:55 Ivan Granero 

That will be one, trying to quantize or optimize the model to run in an embedded device.

00:19:03 Ivan Granero 

And definitely the safety and security of it, because especially if you are trying to run an LLM, which is very probabilistic, and

00:19:15 Ivan Granero 

might hallucinate, so how do you control that?

00:19:17 Ivan Granero 

How do you make it a little bit more deterministic, right?

00:19:22 Ivan Granero 

Of course, part of the system parameters that you're able to control is not a hyperparameter, but it's a system parameter, the temperature.

00:19:31 Ivan Granero 

Of course, you set it to 0 for these kind of applications.

00:19:37 Ivan Granero 

Of course, you add layers around.

00:19:39 Ivan Granero 

So not only the guardrails, if your model is going to be talking, let's say, network inside the vehicle, then you need to add all of the authentication mechanisms that we have in regular vehicles.

00:19:56 Ivan Granero 

You have to differentiate that particular command is coming from an LLM.

00:20:04 Ivan Granero 

compared to coming from another embedded device.

00:20:08 Ivan Granero 

So that's important.

00:20:09 Ivan Granero 

And the same is being done in IT, right?

00:20:13 Ivan Granero 

Everything that the LLM is inputting and outputting, it's being logged.

00:20:19 Ivan Granero 

And that's a very important security mechanism that everyone does, because even nowadays, when you have billions of parameter,

00:20:30 Ivan Granero 

you don't really know what's going to come out of the model.

00:20:34 Ivan Granero 

Even if you give the same input, the output won't be the same, even with the temperature set at 0.

00:20:40 Ivan Granero 

So it's important to log everything.

00:20:43 Ivan Granero 

And now with authentic AI, I was just on a training on securing LLMs by Gary Lopez from Microsoft, and he even mentioned that there's been research on, okay, what if we optimize

00:20:58 Ivan Granero 

the communication in between LLMs, right?

00:21:00 Ivan Granero 

Why they might be able to talk faster because of machine to machine.

00:21:08 Ivan Granero 

However, the advice is don't keep it on a language that we understand because it's important to log it as of now.

00:21:19 Ivan Granero 

So that's, yes, it's going to be slower, but we're going to be able to log it and understand what's going on.

00:21:25 Ivan Granero 

in between the two agents, I would say.

00:21:29 Zane Pelletier

Okay, so having a certain level of traceability there is going to be necessary, which I totally agree.

00:21:35 Zane Pelletier

I mean, knowing what is happening is the first step to being able to have a full understanding of the objective security of a system, or even like safety, right, for instance.

00:21:47 Zane Pelletier

I guess that's a good segue into kind of asking you about, you know, for automotive

00:21:53 Zane Pelletier

right?

00:21:54 Zane Pelletier

There are two main standards.

00:21:57 Zane Pelletier

There are more recommendations here in the States.

00:21:59 Zane Pelletier

You know, there's UN R155 in Europe that's kind of mandating things, right?

00:22:03 Zane Pelletier

But here in the States, we have, you know, ISO SAE 21434, which is, you know, security for vehicle systems, right?

00:22:12 Zane Pelletier

And then we have ISO 26262, which is functional safety.

00:22:16 Zane Pelletier

And a lot of times these kind of have parallels and they're very interrelated.

00:22:20 Zane Pelletier

So given these kind of these strict safety standards and these strict security standards that we do have in a lot of regions, do you ever see a future where AI applications, for instance, could determine and make decisions related to the actual drivability of vehicles?

00:22:37 Zane Pelletier

I guess, you know, we see this a lot right now with Tesla, for instance.

00:22:41 Zane Pelletier

I mean, they have full self-driving and there's a lot going on in the back end.

00:22:47 Zane Pelletier

there's a lot of onboard capabilities that those vehicles have to make decisions while driving.

00:22:51 Zane Pelletier

So I guess, yeah, do you see, I guess with your experience at Bosch, a huge shift to kind of doing that?

00:22:59 Zane Pelletier

Or are a lot of other companies kind of hesitant to follow, let's say, in the footsteps of Tesla, which I would argue is kind of doing some radical things there, right?

00:23:09 Ivan Granero 

Oh, yeah, definitely.

00:23:11 Ivan Granero 

So, yeah, to answer your question, if I

00:23:14 Ivan Granero 

If I see AI being used to control, let's say, the steering wheel, control things in the vehicle, definitely yes, right?

00:23:21 Ivan Granero 

And we are seeing it with ADAS or autonomous driving.

00:23:25 Ivan Granero 

The main difference, I will say, compared to LLMs that I was referring to earlier.

00:23:32 Ivan Granero 

Personally, I will, my advice, same as this expert from Microsoft, I will say, don't trust an LLM and at the end is,

00:23:41 Ivan Granero 

It's only algorithm, right?

00:23:43 Ivan Granero 

It's machine learning.

00:23:44 Ivan Granero 

The difference is compared to ADAS, an LLM was trained with blogs, it was trained with shots, it was trained with websites, and that's how we learned the language.

00:23:58 Ivan Granero 

So my lack of trust is not really in the computer, it's on everything that is out on the internet.

00:24:07 Ivan Granero 

And that's where

00:24:08 Ivan Granero 

it might misbehave, I will say.

00:24:10 Ivan Granero 

Now, if we go to ADAS, in there we're using mainly convolutional neural networks, which are really good for detecting features in images with the pooling layers.

00:24:27 Ivan Granero 

And so they're very good at that.

00:24:30 Ivan Granero 

And there are techniques, fusion techniques, where you can overlay an image coming from a camera.

00:24:38 Ivan Granero 

together with an image coming from a LIDAR or data coming from a LIDAR or a radar.

00:24:44 Ivan Granero 

And that way you are able to input all of the data together into the same layer of neurons, right?

00:24:53 Ivan Granero 

Input, the first input of, input layer for the, in this case, for the CNN model, the convolutional neural network.

00:25:04 Ivan Granero 

And I will say pretty much most of the ADAS models that we have in the market right now are using CNN.

00:25:11 Ivan Granero 

Tesla uses them.

00:25:13 Ivan Granero 

I believe they have like, it's like a hybrid.

00:25:17 Ivan Granero 

I forgot how they call it.

00:25:19 Ivan Granero 

It's like a HydraNet.

00:25:21 Ivan Granero 

So it has different models and multi-heads.

00:25:28 Zane Pelletier

All right.

00:25:29 Zane Pelletier

So it's kind of a combination of systems, right?

00:25:32 Ivan Granero 

Exactly.

00:25:32 Zane Pelletier

With certain voting parameters, yeah.

00:25:34 Ivan Granero 

Exactly.

00:25:35 Ivan Granero 

And so it's not only one model, it's several models, and they're combining them.

00:25:41 Ivan Granero 

But yeah, the main one I will say is CNN, which it's easier to trust because it's trained on images and it's trained on, basically it's trained on the driving.

00:25:53 Ivan Granero 

So it's millions of miles probably, but they're training their algorithms.

00:25:58 Ivan Granero 

And so it's replicating that, right?

00:26:02 Ivan Granero 

So it's taking an action based on predicting the next movement based on all the driving history that it was trained with.

00:26:12 Ivan Granero 

And on top of that, you have the safety that we have in vehicles.

00:26:17 Ivan Granero 

So for, let's say, if a camera is

00:26:22 Ivan Granero 

is not working right now, then it will ask you to probably take on the steering wheel, right?

00:26:29 Ivan Granero 

Like it will tell you, hey, this camera is not working properly, so just take over.

00:26:34 Ivan Granero 

And those safety mechanisms that have to be there.

00:26:38 Zane Pelletier

Definitely, yeah.

00:26:39 Zane Pelletier

I mean, I know that we don't really see much guidance officially in terms of the standards, because the standards take a while to catch up, I think, to a lot of these things.

00:26:49 Zane Pelletier

But I think if I had to guess, I would probably, as more and more of these systems get integrated, we're probably going to see some requirements surrounding thresholds there.

00:26:59 Zane Pelletier

The ability to manually override things needs to be there to some extent in the system, which I guess if I go back to Tesla, the robotaxis at this point don't really have steering wheels.

00:27:10 Zane Pelletier

So it's a little difficult with a system like that to say, oh, well, how am I going to override steering?

00:27:15 Zane Pelletier

I don't really have a mechanism to do that, right?

00:27:17 Zane Pelletier

But

00:27:18 Zane Pelletier

Yeah, that's a very interesting area, though, that I think is, it's kind of the Wild West right now.

00:27:24 Zane Pelletier

And, I'm not seeing a lot of, official guidance come out of this.

00:27:28 Zane Pelletier

So I think talking with you about it's really interesting.

00:27:31 Zane Pelletier

So yeah, I mean, so from a security perspective, though, I guess kind of returning to, how are you utilizing Gen.

00:27:38 Zane Pelletier

AI kind of in your day-to-day, I guess, do you see a huge

00:27:44 Zane Pelletier

Because I know there's a lot of tools available out there.

00:27:46 Zane Pelletier

I'd use some of them too.

00:27:47 Zane Pelletier

I'm trying to follow, you know, agentic systems, for instance, that can help me perform more like red teaming tasks.

00:27:54 Zane Pelletier

Are you seeing that there are more and more systems that you can kind of just use that are open source that you're utilizing?

00:28:01 Zane Pelletier

Or are you having to, I would say, build a lot of these Gen.

00:28:05 Zane Pelletier

AI applications that you're utilizing kind of on your own at this point?

00:28:10 Ivan Granero 

At the beginning, yes, I was, I guess I had to create my own.

00:28:14 Ivan Granero 

Luckily, there is a lot of open source tools that are being released.

00:28:23 Ivan Granero 

And one of the things that really changed the game, I would say MCP protocol, thanks to that, a lot of people are creating the MCP servers for their tools.

00:28:36 Ivan Granero 

which makes it easier and very simple to talk to it with, talk to those tools with your LLM, right?

00:28:44 Ivan Granero 

And if I will say, yeah, it's, if you're not familiar with it, I mean, just for the audience, I know you are, to get familiar with, this is just one protocol, right?

00:28:57 Ivan Granero 

I will use it and try, if you have your own tools, try to create

00:29:03 Ivan Granero 

the MCP server for it, so you're able to use LLMs with your tool, right?

00:29:10 Ivan Granero 

And for pen testing, I will say there is a lot of tools out there, which as being in both sides, blue team and red team, it's a little bit of concerning that to see so many tools that are being deployed out there, I will say.

00:29:29 Zane Pelletier

Yeah.

00:29:30 Zane Pelletier

I guess a little bit for those listening, MCP is essentially a framework, I would say.

00:29:36 Zane Pelletier

It's a framework for building out a common interface that large language models and Gen.

00:29:43 Zane Pelletier

AI in general can use to call tools and get an expected, you know, feedback from that tool.

00:29:50 Zane Pelletier

I guess for security applications, I know I've used Ghidra MCP and Ghidra Assist, which is another one for like reverse engineering.

00:30:00 Zane Pelletier

Those are great.

00:30:01 Zane Pelletier

I believe, I think it was, it Lori Graham that made I can't remember.

00:30:06 Zane Pelletier

But yeah, there are a lot of tools out there.

00:30:09 Zane Pelletier

There's one for like those that do like web application testing.

00:30:13 Zane Pelletier

Burp Suite has an MCP server for it as well.

00:30:18 Zane Pelletier

So yeah, those are great.

00:30:20 Zane Pelletier

Being able to integrate that is very, very good.

00:30:22 Zane Pelletier

But at the same time, you know, I guess you kind of have to consider

00:30:29 Zane Pelletier

the kind of directed aspect of that as well.

00:30:33 Zane Pelletier

I mean, I guess, do you have any advice regarding, because I always have this issue when I'm using tools personally where I don't want to allow the model to have full control over a set of steps, right?

00:30:45 Zane Pelletier

and actually be fully agentic and like, I'm going to execute this, I'm going to get the feedback, and I'm going to start executing further tasks, right?

00:30:52 Zane Pelletier

So a lot of times, you'll try to implement something like human in the loop, where you actually have decisions over, yes, go in this direction, or no, you got the result back, but you interpreted this incorrectly, and I don't want you to continue executing this path of decisions that you're making.

00:31:08 Zane Pelletier

So I guess to make that a concrete question, what do you find yourself using most often?

00:31:15 Zane Pelletier

And do you use more agentic tooling that allows the LLM to take almost full control over an entire workflow?

00:31:22 Zane Pelletier

Or do you find yourself utilizing kind of a traditional LLM chat along with some MCP tools that you have integrated in?

00:31:32 Ivan Granero 

Yeah, I will say it depends on your goal, right, on the application that I'm working on.

00:31:39 Ivan Granero 

If I'm working on a capture the flight competition, I just let it run, sure, and depending, because it is a controlled environment that you don't really mind if something goes wrong, right?

00:31:53 Ivan Granero 

And I add a few breaks where it's gonna ask me for permission, right?

00:32:00 Ivan Granero 

Human in the loop, as you're saying.

00:32:03 Ivan Granero 

There are, for example, one tool that I will mention, there are, and actually there are a few of them, command lines, GPTs, where basically you just talk to it and it will give you the command line that you need to execute, right?

00:32:17 Ivan Granero 

And it asks for permission.

00:32:19 Ivan Granero 

So it doesn't execute it.

00:32:21 Ivan Granero 

It will ask, okay, this is a command line.

00:32:23 Ivan Granero 

Do you want to execute it?

00:32:24 Ivan Granero 

And you just say yes or no, and then it will execute it on your...

00:32:29 Ivan Granero 

Linux command line or DOS, whatever your environment it is for those cases, I will say yes, it's very important to have these prompts, otherwise, yeah, it will be very drastic the consequences that this LLM might do on your system.

00:32:47 Ivan Granero 

If you are creating your tool with the use of LLM, for example, I have one that I've shown to you before, the packet analyzer where, and it's out there in my GitHub, where basically I'm doing searches and executing those on my local environment.

00:33:08 Ivan Granero 

I don't really have protections there because I'm only doing searches on data.

00:33:14 Ivan Granero 

Even at access to delete or remove or drop lines in the database, it's just doing searches, right?

00:33:22 Ivan Granero 

I guess the worst that can happen is that the search wasn't what you wanted to do, so I just execute it.

00:33:29 Ivan Granero 

The user will send...

00:33:31 Ivan Granero 

natural language search, such as find everything that starts with four letters and follow up by three digits.

00:33:39 Ivan Granero 

And it will create the regs for it and it will execute it.

00:33:42 Ivan Granero 

So applications like that, yeah, I will say you can just run it.

00:33:47 Ivan Granero 

And so, yeah, I will say it depends.

00:33:49 Ivan Granero 

And your other question was on what tools?

00:33:53 Ivan Granero 

Yeah, locally now we have a lot of frameworks, I will say, for agentic AI.

00:33:59 Ivan Granero 

It depends on your experience of also what level of integration you want to do with it.

00:34:07 Ivan Granero 

If you want to go high level, I mean, n8n, that will be a very easy way to integrate or start with Agentic AI.

00:34:18 Ivan Granero 

If you know a little bit of Python programming, then you can use other frameworks such as LandChain,

00:34:26 Ivan Granero 

Or you can just do it on your own, right?

00:34:29 Ivan Granero 

You can create your own tool calling once you, I will say, with a little bit of prompt engineering and train the model to always reply in a certain way.

00:34:41 Ivan Granero 

My favorite format is JSON for when I'm doing tool calling.

00:34:50 Ivan Granero 

I guess it's because most of the LLMs out there were training a lot of JSON.

00:34:55 Ivan Granero 

data.

00:34:56 Ivan Granero 

So they are very good with JSON.

00:34:59 Ivan Granero 

And so I will say it depends on how much integration you want to have, how much control you want to have of it, and the application, how much human in the loop you want to add to it or not.

00:35:13 Zane Pelletier

Right, yeah, definitely.

00:35:15 Zane Pelletier

Yeah, I agree.

00:35:16 Zane Pelletier

I think all of those things are good to consider when you're choosing tooling like that.

00:35:21 Zane Pelletier

And I know another one I've used too, like for agents, for instance, it's called smolagents.

00:35:26 Zane Pelletier

It's from Hugging Face.

00:35:27 Zane Pelletier

But that's a really easy one to integrate just because it allows you to do tool calling, I would say, natively with them.

00:35:34 Zane Pelletier

And it's like really low amount of actual setup that's required for that.

00:35:38 Zane Pelletier

I think you can get going with about 10 lines of Python, and it'll start executing stuff pretty much immediately, which is really cool to see.

00:35:47 Zane Pelletier

But definitely, we'll have to include the packet analyzer, the description for this podcast, actually, because I've seen that and used that.

00:35:53 Zane Pelletier

And I think for network traffic, for instance, that's very, very useful.

00:35:58 Zane Pelletier

I mean, for anybody who's actually manually combed through logs, like network logs and raw information, it's so, difficult to do that manually.

00:36:09 Zane Pelletier

And there's really no reason to if you have a tool like that.

00:36:11 Zane Pelletier

So yeah.

00:36:14 Zane Pelletier

thank you for that excellent discussion, Ivan.

00:36:17 Zane Pelletier

I really appreciate you joining.

00:36:19 Zane Pelletier

I think we covered a lot today.

00:36:20 Zane Pelletier

I mean, we've talked about a lot of facets of security with regards to Gen.

00:36:25 Zane Pelletier

AI and also, you know, some applications in machine learning on vehicles.

00:36:29 Zane Pelletier

Kind of two, I would say, distinct topics, but there is a lot of overlap in terms of, you know, I would say generalized security and safety, right, as well.

00:36:38 Zane Pelletier

So, yeah, I guess, do you have any final closing thoughts for us regarding any of that?

00:36:44 Ivan Granero 

I will say for everyone listening, if you are not using AI, yeah, don't wait.

00:36:53 Ivan Granero 

Start using it.

00:36:54 Ivan Granero 

Try it out.

00:36:55 Ivan Granero 

Try to integrate it in your tools that you might have.

00:37:00 Ivan Granero 

And very important, yeah, it's to distinguish the different algorithms or models that are out there because not

00:37:09 Ivan Granero 

Using Gen.

00:37:10 Ivan Granero 

AI is not always necessary, I would say.

00:37:14 Ivan Granero 

As we were discussing at the beginning, there are some cases that just a small machine learning model might do the work, right?

00:37:23 Ivan Granero 

So it's important to don't lose track on that.

00:37:27 Ivan Granero 

And I guess one final thought, I believe even the LLMs that we're using on powerful compute, brack of GPUs nowadays,

00:37:38 Ivan Granero 

I foresee even those with methods such as quantization and the computing power increasing in vehicles.

00:37:49 Ivan Granero 

I believe we're going to see more neural networks working inside the vehicles, I will say.

00:37:56 Ivan Granero 

Not necessarily the LLMs as we know them.

00:38:00 Ivan Granero 

I mean, with all the data trained from the internet,

00:38:03 Ivan Granero 

but similar neural networks models inside the vehicle for different applications.

00:38:08 Ivan Granero 

I believe we're going to see those in the near future, and it's exciting for me.

00:38:15 Zane Pelletier

Awesome.

00:38:16 Zane Pelletier

Well, thank you again, Ivan.

00:38:17 Zane Pelletier

I really appreciate you talking with me about this today.

00:38:20 Zane Pelletier

And for everyone listening, thank you so much for tuning into this episode of Empowering Tomorrow's Automotive Software Podcast.

00:38:27 Zane Pelletier

We really hope that you found our discussion insightful today and very valuable as well.

00:38:31 Zane Pelletier

If you enjoyed today's episode, don't forget to subscribe on Spotify, Apple Music, or wherever you get your podcasts.

00:38:37 Zane Pelletier

Feel free to share this episode with your network and leave us a review.

00:38:40 Zane Pelletier

We'd love to hear your feedback.

00:38:41 Zane Pelletier

This concludes our episode.

00:38:43 Zane Pelletier

Please check back soon for a new one.

00:38:48 Voiceover

Thank you for joining this episode of the Empowering Tomorrow's Automotive Software podcast.

00:38:53 Voiceover

Please leave a comment or review with your feedback or what you'd like to hear in future episodes.

00:38:58 Voiceover

To learn more about automotive embedded systems,

00:39:00 Voiceover

and ETAS's capabilities, visit our website at ETAS.

00:39:04 Voiceover

That's ETAS.com.