What's Up with Tech?

Revolutionizing Automotive Safety: Keymakr's AI Innovations Transforming Vehicle Perception and Data Annotation in Challenging Environments

• Evan Kirstel

Share episode

Interested in being a guest? Email us at admin@evankirstel.com

Discover how the future of AI is reshaping the automotive industry with insights from of Keymakr, as we explore the critical role of human input in training AI models, particularly in computer vision. Learn why vehicles must evolve to be proactive in ensuring safety, especially in challenging environments like Boston’s unpredictable weather. Dennis highlights the ongoing debate on the best technologies—such as LiDAR, radar, and cameras—for enhancing vehicle perception and safety. Prepare to expand your understanding of active safety advancements and the continuous refinement needed for AI models to excel in diverse driving conditions.

Get ready to unravel the complexities of data creation for AI model training, especially in niche industries like automotive, agriculture, and security. Denis offers a glimpse into the balancing act between synthetic and real-world data, pointing out the limitations synthetic data faces in mimicking human behavior. Discover the iterative process of tailoring data sets to meet specific needs and the importance of minimizing subjectivity through consensus among data annotators. This episode promises valuable insights into best practices and challenges in data annotation, enhancing AI's precision and effectiveness in real-world applications.

Support the show

More at https://linktr.ee/EvanKirstel

Speaker 1:

Hey everybody. Fascinating chat today around the world of AI, data and innovation and what's next in this space with an innovator and thought leader from Keymaker Dennis. How are you?

Speaker 2:

Hey, evan, we're doing very nice. Thank you for having me today.

Speaker 1:

Well, thanks for being here and really intrigued by this topic. Before that, maybe introduce yourself and what's the big idea, the mission behind Keymaker.

Speaker 2:

So my name is Denis Sorokin and I'm a head of project management in Keymaker. And what are we exactly doing? In a nutshell, let's say we are working on the data in terms of curating it to our customer needs or creating the data if let's say the data is not there on the market. So we're basically providing human eyes, human hands, human approach to train computer vision. So where are those eyes that will guide the machine vision?

Speaker 1:

So that's interesting. Why do we need eyes? The machine vision, so that's interesting. Why do we need eyes? I thought the whole idea of AI was you know, you don't need human input, but why is annotation so important when it comes?

Speaker 2:

to the success of AI models. Well, I would say you would always need human eyes to extend depending on, let's say, what type of precision in your industry is currently required. Let's say, if you have a drone I don't know let's say flying around the tomato field, right, or, I guess, tomatoes in a warehouse or whatever, let's say a tomato field, and it will recognize with 95% precision that this tomato is ripe and this one will'll say is rotten, it's not a big deal. But if your autonomous vehicle is driving and it needs 99 plus close to 100% precision to identify if this is a pedestrian, if this is a green light or a red light and so forth, imagine, I don't know, let's say, any medical domain where you will do a operation or remote operation and so forth. So in some industries for now we're fighting or helping companies to fight this particle of a percent of preciseness where it's truly needed. So it's one of the examples.

Speaker 1:

Interesting. So, yeah, let's take automotive. That's a fascinating space right now, with robo-taxis from Waymo and Elon Musk and many others. What is the state of the industry state of play, let's say and how is it going? What's next to really reach this potential here?

Speaker 2:

It's a great question and I can actually even continue with what I've just left off. So in the automotive, I would say one of the biggest challenges or the areas that needs attention is active safety.

Speaker 2:

Well, let's say we all started with, let's say, cars basically recognizing lanes right.

Speaker 2:

Then cars were recognizing, let's say, animals crossing the roads, then it was pedestrian, then let's say motorists, people on the bikes and so forth.

Speaker 2:

But now it's more about being active, or even say proactive. So the car would not just stare ahead but it would also analyze what might happen. Let's say there is a sensor inside of the vehicle saying, uh, it will overlook your, the gaze of your eyes, your, let's say what are you doing? Maybe you're, maybe you're lighting up a cigarette, or you're playing, or you're distracted with the phone, or you're staring in the uhview window window sorry, not a window mirror for a while right, or put in a makeup or something else. So therefore, the car would be able to tell that something is definitely going on, something might happen, and, together with the car or vehicle overseeing, with all types of sensor lidars, radars and so forth, combining all this together, it might say that, hey, something is possible, so let me start braking or whatever the algorithm is.

Speaker 2:

So the car or the vehicle let's say, not only the car will try to analyze and be proactive. So what? I think this is one of the most challenging areas that we're working on for now, because you know pretty much every vehicle can say this is a stoplight, this is a deer running across of you. So it's pretty easy. But being proactive for active safety, that's the game for now. Being proactive for active safety.

Speaker 1:

that's the game for now Interesting. So I have real questions for you here about self-driving cars in Boston. If you've ever driven in Boston, we have ice, we have snow, we have the craziest, most aggressive drivers in the country extreme conditions really. How would you know? Annotation and continuous model improvement, let's say, in an area like Boston work?

Speaker 2:

It's a great question and it's good that you mentioned, let's say, boston or the US, or a particular weather conditions, because, let's say, the model that you put in your vehicle for it to be self-driving, it might completely work, let's say, in the perfect weather condition it's sunny, there is no snow, it's not slippery, the car is rolling, all is good. So this is why a company like us are on the market, because you might need to train your model with particular sets of, let's say, requirements or conditions, such as how does snow look like visually, how does melted snow look like, how blizzard looks like, look like how blizzard looks like, how does, let's say, the lane right or whatever markings or the pavements are, how do they look when they're covered with snow, and so forth. So, as mentioned earlier, you need you do need to train your data, basically for, let's say, different countries. We had a customer that we used to work on the datasets from US, from Europe, from Japan, from South Korea, because everything, like the environment, is too different. So it might work in US, but it will work less or not work at all, let's say, in Japan or in the UK, where everything is basically left-handed, right, right, yeah, great point, great insight.

Speaker 1:

And let's talk technology specifically. There's a big war on the pros and cons of LiDAR versus radar versus camera and you know each company in this space has different or differing competing in some cases approaches. What's your take on the different pros and cons of these technologies?

Speaker 2:

Well, for now, well, I'll briefly give you an insight of what is the con and pro of each and every one. So, basically, it's all down to effectiveness and performance, in fact, in its work and its economic effectiveness, right. So let's say you can put a LIDAR right, which is basically a radar that is using laser beams, right? So it will be very accurate, let's say to measure the distance. Right, it will not really be affected by lighting conditions, right? So if it's day or night, it doesn't really matter. On the other side, it's pretty short-range, right. So LiDAR, let's say again, in a basic way, let's say, driving around your neighborhood, your LiDAR will be very nice on a sunny day, but if you're doing 100 miles on a highway where the car is being like I don't know, let's say half a mile or something is in front of you, it will not really reach those. And this is where radar uh will kick in. Uh, being cheaper but being a more long range, let's say. So, uh, lidar is definitely there, is doing good job, um, for the range, uh, range finding uh, depth mattering uh. But again, it's pretty expensive. Uh, it really requires a lot of computational power.

Speaker 2:

And if we talk about, um, automotive industry. It really has limited performance in some bad weather, not even bad weather, but let's say even some rain or fog can really interfere with it. So just a lidar will not fix it. Um, radar or just camera again, they're cheap, they're nice, they're cheap, they're nice, they're everywhere, it's really affordable, it's good to capture all types of objects, but it might have harder time in terms of precision. So the camera or radar, it might have pretty low resolution, right, when radar is seeing an object, you know it's something that is moving in front of you. So you cannot really tell if it's a motorcycle or a person on a bicycle, right, and depending if what types of vehicle this is, it might be moving slower or faster.

Speaker 2:

So I would say, to summarize, for now the real industry trend is the sensor fusion. So you don't really choose either one like, let's say, radar or camera, or a lidar or something. It's really best if you combine all of them together. And this is what, again, we're, not to mention brands of those vehicles or the car's manufacturers, but let's say the biggest one, the fanciest one. They're actually combining all of those three at least, maybe more, together for the most redundancy and robustness. So sensor fusion is the future Interesting.

Speaker 1:

And what about data creation for different vehicles? I mean, obviously you have sedans and SUVs and trucks and other, you know, SUVs. How does that differ per vehicle?

Speaker 2:

You're saying refining question. You're saying how is data creation working for different types of car bodies? Let's say, yeah, for different types of car bodies. Let's say, well, to specify data creation, if we're talking about data creation, it really is. Let's say, a niche product when you're working on your model, right, and it doesn't really matter what kind of industry you are if it's automotive, agriculture, medical, I don't know, retail, security, logistics, whichever you really need data to work on. Where can you get the data? You can crawl internet, you can just collect it. Let's say you work on the face recognition, you can just crop it. Let's say you work on the face recognition, you can just crop some YouTube videos, you can download some Instagram pictures or some Facebook pictures or whatever, but you not always can find the data with a specific scenario.

Speaker 2:

If we talk about automotive and you are training your model to be able to recognize and distinguish all different types of car bodies or vehicles, right, you would need and this is something that we do you would need or you might want to create the data for yourself. Now, when we talk about data creation and it's a huge topic, it's like you know, basically we can call it Hollywood for computer vision. You can take the data basically by creating it, actually making a movie with your pre-required conditions, or you can create synthetic data, but synthetic data and we will talk about this later on, I believe, in our interview synthetic data might not really cover all of your scenarios. To make it shorter and easier, let's say you need 10 minutes of videos where a person, let's say I will describe myself a Caucasian male, I don't know let's say late 30s, wearing a cap, lighting up a cigarette, holding a phone in his hand and turning a steering wheel.

Speaker 2:

You cannot really find open source data to train a model. So this is something that you would want to create, and when you create the data, you need diversity, so we'll have people of different genders, nationalities, skin tones and so forth. So data creation is there when you need to have your data set tailored to your particular needs. Yeah, it's a big question, sorry, so I can talk for hours here?

Speaker 1:

No, I'm sure you know we only have a few minutes, but are there some? You know major drawbacks with synthetic data. It seems to be a hot topic, or is it, like you said, fusion of? You want synthetic data and real-world data, real-world annotation.

Speaker 2:

You know, it really depends, I believe, when you just start the training of your model, right, let's say, you just want to teach your vehicle to be able to distinguish when there is some obstacle, so you can totally open.

Speaker 2:

Well, again, let's say, whatever data creation module, right, you can use AI software to just generate whatever you want, and for the basic needs it would be enough. But if we talk about, let's say, some really delicate and precise autonomous vehicle driving, you really need more tailored scenarios, like I mentioned earlier. Let's say you have a busy street, it's snowy, you have a school bus in front of you, you have yeah, it's winter, but you have guys on motorcycles passing you by, right, and this you can totally create a synthetic data for it. But synthetic data would not really help you that much because you really need live person behavior. So, short answer, or to summarize, when you're just starting on your AI training, synthetic data can totally help you. But again, it works. Let's say, sooner or later you would need some real data that you can either collect, that we can do for you, or create Again based on your script, your requirements, whatever your data scientist will tell you. So synthetic data is there, but it's not one-stop shop solution.

Speaker 1:

Let's say interesting and maybe for us describe at keymaker the end-to-end process from you know new uh, project creation to deployment, uh, what does that look like and what are some of the you know barriers that you have to overcome and some of the best practices you have to deploy in a typical engagement with a client.

Speaker 2:

It's a great question so we can take. I can answer this in the example of two types of projects, like, let's say, just annotations and data creation. So let's say, in scenario one, you would give us I don't know terabytes or hours of videos right of you driving around your neighborhood, let's say, and what we would need to do is basically analyze your data based on the guidelines that you would give to us, as well as we can totally help you to tailor them. Let's say, you would like to know, on this particular second or frame of the video, what is the probability of a crash. Like, you see a car in front of you, let's say 10 feet, you're doing 25 miles an hour, and you need a human or a team of people to tell you what is the probability of you doing a crash. Is it 25%? Is it 75%, and so forth. So one of the most challenging part is us to look at something subjective and agree or to have a consensus. In other words, the most difficult part is, let's say, if you look at my sweater, you need to have about, let's say, 10 people, 20 people, 100 people saying that this is particularly navy blue color. How are you going to achieve it Not black, not dark blue, not light navy blue, whatever. This is navy blue color, right? This car is exactly 10 feet off you and it's like 70% to 85% probability of a crash. So we're training our people to have or eliminate this subjectivity. So that's number one.

Speaker 2:

We will take your guidelines. We will work with you in iterations like a basic. Any IT project will work. We will deliver data in small batches. Make sure that it fits your needs. We will receive feedback, because you know, in training the data, it's really important to do alterations all the time. It's not that you start on January 1st and then we'll see you in six months later and here's your annotations. No, you will constantly give us feedback as your model will evolve. So we're basically working iterations Working on documentation, iterations, receiving feedback, applying feedback, deployment Roughly.

Speaker 2:

This is how we work with data annotation projects and data creation, which I called earlier like Hollywood of AI. Well, that will take, I believe, a couple hours to cover, but basically, yes, it's like you're making a movie, so I will ask you what do you want? What is the script? What do you want to see? What is the length? What are the props? What is the lighting conditions? Which hardware should we use? What should be the resolution? So imagine it's like you're making a movie with exactly the same process. Well, maybe it will just be shorter in time, but basically it's like a Hollywood for AI.

Speaker 1:

Fantastic, what a great opportunity. So any big predictions on some breakthroughs we might see over this year and next? Do you think we'll see success with true self-driving, you know, level five autonomous driving? Will we see Elon Musk's robo-taxi finally be realized? What's your prediction in some of these areas?

Speaker 2:

Well, again, it's world human right. So, as mentioned earlier, we have subjectivity. So I personally think that it's already here. It's just not scaled up yet. So the robo-taxis are there. Let's say, the Gen 5 of autonomous vehicles is not there, but it's about connectivity, right.

Speaker 2:

So sooner or later, cars will not just be, let's say, analyzing what is in front of them, how to avoid it. This is what they're doing now. Uh, cars already, as mentioned earlier, are proactively analyzing you as a driver Are you tired? Maybe even there is a thermal sensors, right, maybe you have a fever. So, basically, there is proactive safety and pretty soon, I would say, the cars, the autonomous vehicles you know I keep saying cars all the time, but it's not exactly correct, but it's a vehicles they will be basically communicating to each other, saying hey, I am approaching this intersection, I'm approaching this intersection. Okay, let's make sure we don't crash. So, long story short, it's coming. How quick? Again, depending on the technology. I believe in terms of documentation, legislation is there, they have a green light, so it's coming.

Speaker 2:

I would say a year or two, maybe less, and another, I would say, addition that I think is coming next is AI being able to work with different data sources. So let's say, you can have a camera right looking at the if we're talking about vehicles, you can have a camera looking on the road conditions and trying to predict something. It can recognize the text right on the road signs. But what we're really lacking, and is just starting to evolve, is the AI model that will understand and combine different data source text, images, videos, 3d data, audio data, sensor data combine all this together and provide you a solution. So this is a next step. I would say Some people work companies, some companies work on annotating images and other companies work on annotating audio and other work on sensor data like thermal infrared, so forth. But combining all this together is the next step. It's a very small step. It's coming. We're talking months, years maybe, but pretty soon.

Speaker 1:

Wow. Well, that's very exciting and positive note. Thanks so much for sharing those insights. I couldn't be more excited about the future of autonomy and autonomous driving. It's just an incredible time to be alive, right? So thanks so much, dennis. Good luck on the journey, take care. Thank you, ivan. And thanks everyone, Thanks for listening and watching and sharing.

Speaker 2:

Have a good one.