"Danger Dan Rosenstein!" - at the Frontier of Robotics and Pinball

Embark on a riveting journey through the nexus of edge computing and robotics with my good friend and robotics virtuoso, Dan Rosenstein. From his boyhood wonderment with gears and circuits to becoming a guiding force in robotics, Dan's story is not just about the machines—it's a personal saga that intertwines with the pulse of technology. Our chat takes you through his fascinating career trajectory, where a surprising detour led him from an initial aversion to software to the forefront of robotics innovation. Imagine controlling robots with the same familiarity as flipping through TV channels; Dan's experiences promise to draw back the curtain on this swiftly advancing realm.

Strap in for a deep dive into the symbiotic dance of robotics and AI, as Dan elucidates on NVIDIA's cutting-edge simulations that are redefining robot training. We marvel at the cultural tapestry robotics has woven over time, stretching from industrial assembly lines to the friendly confines of our living rooms, with robotic vacuums and more. It's not all about the mechanics and AI algorithms; we also touch upon the profound connections we share with these intelligent machines. Plus, we'll scout the horizon for drones and robots making their mark in society, from beach rescues to the nuanced discussion around the practicality of humanoid robotics.

Feel the palpable excitement as we veer into a nostalgic yet innovatively rich discussion of pinball and its parallels with robotics. Dan's passion for pinball isn't merely a hobby; it's a testament to the intricate dance between physical and virtual worlds, a theme that resonates throughout our conversation. As we anticipate a technological 'Cambrian explosion' at the edge, Dan and I invite everyone to seize the burgeoning opportunities in the fusion of robotics, AI, and the digital-physical interface. So, tune in, get inspired, and perhaps you'll find your own intersection within this technological renaissance.

Check out Dan's amazing Pinball podcast (Pinball Innovators and Makers) at https://www.kineticist.co/pinball-promoters/pinball-innovators-and-makers-podcast

Want to scale your edge compute business and learn more? Subscribe here and visit us at https://edgecelsior.com.

Transcript

Pete: 1:04

You know, fill in the blank today whatever you're building your AI-powered dog food dish or whatever I haven't seen one, yet we actually need more of them.

Dan: 1:12

AI-powered dog food dishes.

Pete: 1:14

Yeah, yeah, there you go. That's a freebie, that's a free idea for somebody, no consulting fees required, anyway. So we're burning a lot of tape here, but I wanted to welcome Dan Rosenstein to the EDGECELSIOR Show. And, dan, thanks for joining us, it's my pleasure being here.

Dan: 1:34

Thank you for having me. We've known each other for a while and it's good to be on the show.

Pete: 1:38

That's right, full disclosure. Dan and I know each other, so this is an easy conversation. Sometimes I have guests that literally I've met them for the first time on the show and I've looked at their LinkedIn profile and it's kind of an awkward thing. But Dan and I have worked together in a number of projects over the years, so it's good to have you. So Dan is kind of an expert on a few topics that we're going to talk about today, and one of them is going to be robotics and robotics platforms and where those are heading.

Pete: 2:05

Uh, especially off the news from nvidia last week at gtc. I think that's going to be a good conversation. The other thing that dan is like kind of an expert on I'll call him an expert is around pinball, and so he has a podcast, uh, pinball podcast and uh, which we'll put in the, a link to in the comments and in the description. But so, yeah, so why don't you give us your you're at Microsoft, you've been at Microsoft a long time, and why don't you give us kind of the origin story of Dan?

Dan: 2:33

Yeah, no, Pete, I'm totally happy to, and again thank you for having me on the show. So my Microsoft origin story starts at 25 years ago. It's been 25 years I've been at Microsoft, but my yeah, this June will actually be 25 years. And my origin story, though, in robotics actually goes much farther back. So I've you know, ever since I was six, we all knew that I had the knack and I was going to do something in engineering.

Dan: 3:00

It was just a matter of you know which engineering degree, not what's I going to pursue engineering, and you know my dad was in aerospace engineering. It was just a matter of which engineering degree, not what's I going to pursue engineering, and my dad was an aerospace engineer, and he's still with us he's not practicing. He hasn't practiced for years and I've been around computers pretty much my whole life.

Dan: 3:15

My dad, started a software business for medical billing in the Midwest in Ohio, back in the early 80s, we had one of the very first PCs, and so I was like six years old or five years old, running around and typing away at the keyboard, and so tech and software have always been part of my life, and as far back as I can remember I've loved robots like Transformers, gi Joe, star Wars, you name it it. If there was a robot in some way, shape or form, I I became obsessed with it. Um, in, in fact, like anytime I can get a battle android trooper figure from gi joe I I I buy them up, um just have a small collection of them. Um, and that that all parlayed, you know, through throughout high school, throughout middle, I was always in front of a computer or always building something. And when I went off to college I decided I wanted to do automotive engineering, and so I wanted to do mechanical and electrical.

Pete: 4:14

And so.

Dan: 4:14

I went to Washington University and the thing that I didn't want to do was software. I wanted to be as far away from software as possible. I'd been around it for a very long time programming as I would have called it, and when I went off to school I couldn't take any mechanical or electrical classes, but I could take computer science 101. So I did.

Dan: 4:34

And the next thing I knew for the next three and a half years I was a TA for various different computer science classes and I switched to computer science as my major and so I came out to Microsoft. There's a good story for how I ended up with Microsoft, but the bottom line is, you know, I went through my interviews, I got my ass kicked excuse me for my language during the interviews. This was 1999. So it was nothing special, but I did have 21 job offers coming out of college. This was right before the dot-com bubble and Microsoft was not the highest paying company. But I got my rear handed to me on a silver platter during the interviews my final interviewers we like to call them the as-app, the as-appropriate.

Dan: 5:15

I was like so why do you want to come to Microsoft? And I was like I don't. They're like why did you come? I got a free trip to Seattle. I've never been out there. And he was like no, seriously, why a free trip to Seattle? I've never been out there. And he was like no, seriously, why did you go through all this? I'm like well. He's like why don't you want to come here? And I'm like well, I'm not happy with the products that Microsoft makes. And he was like so, come and fix them, make them better. And I was like, yes, yes, I want to come and make them better. And so for the next nine years or so I was a software developer in tests and then became a developer.

Dan: 5:49

My biggest claim to fame is that I was the youngest full-time engineer on the original Xbox and so did Xbox and Xbox Live shipped both of those and then I started in Windows Media. I went back to Windows Media after three years in Xbox and fast forward to 2007, 2008,. You know I was loving my career at Microsoft. I'd been there about 10 years, you know. I was, you know, upwardly mobile. I was personally satisfied. I was professionally satisfied. You know I had some rough edges to work out at the time. I probably still have some rough edges to work out.

Pete: 6:21

We all do.

Dan: 6:23

But that burning feeling of doing robotics and machines and like it never went away. And so I applied to the University of Washington for mechanical engineering for a master's PhD program while I was working at MS, and three days later I got in. And five days later I found out that we were having our first daughter, so it was a big time of change.

Speaker 3: 6:45

Lots of milestones there, yeah, so I hit 30.

Dan: 6:49

I had a kid coming.

Dan: 6:51

I was about to go to grad school and this was right when the economic crisis in 2008 was happening, so job safety all of a sudden wasn't really there. So, to make the story go a little bit faster, I went to grad school, did robotics and controls at the University of Washington. I was going to do a PhD. I was working on my PhD project thesis, which was I wanted to builda robot that played video games, and as I was working on it at my dining room table, my daughter got up, who was playing next to me, and walked across the room. I got up, put her back down, kept working on the project. She got up and walked away again. I then emailed my advisor and said I'm switching to a master's and I want to be done with this in 12 months because there's no way I'm going to be able to do it with the kids.

Dan: 7:41

So I got my master's in robotics and controls and the next portion of my career I really spent being robotics technology adjacent, working in computer vision back in 2012 on photos, doing photo deduplication, the photos application for windows, and these are big, big projects that almost billions of people use. So I understood scale and I worked on clients and services and back and forth throughout my years at the company and a lot of edge work. And then I ended up, through a series of discussions, in the IoT organization, which is where I ended up connecting with you. And as all of this was going on, I also started mentoring and coaching with FIRST for Inspiration and Recognition of Science and Technology, which is robotics programs, the most commonly known for high school, but they actually go from elementary all the way through high school. And as I became more involved in volunteering, I ended up getting on the state board for First Washington.

Dan: 8:47

I ended up being Microsoft's representative to First globally and got connected with their upper leadership there, and that helped me personally. That helped me professionally as I was working in Azure engine platform, um, on various different projects, some of which with with with you, pete. Um I ended up working on I got closer and closer to robotics work. Um, I had done a bunch of education work for a few years on um, what applications we could bring to the windows store that really, really showed the value of windows and windows applications. And you know the ability to connect physical devices and communicate with them and use the value of Windows and Windows applications. And you know the ability to connect physical devices and communicate with them.

Dan: 9:27

and use the power of the GPU and power of user interface were some showcase pieces, and so there was a bunch of robotics and robotics related tech that I brought to the table, and so the bottom line is I spent the next eight years in one way or another being very robotics adjacent.

Dan: 9:47

There was something robot going on, whether it was vision or actuation or education, and I worked with Lego during this period of time on their robots, the Mindstorms thing yeah so Mindstorms, wedo and Spike Prime, and actually you know, behind the scenes, like this stuff is supposed to just work, but like Lego needs an update to the Bluetooth driver, because our Bluetooth functionality has, you know, doesn't support this one thing or hasn't supported it yet, and you know somebody's like there's somebody who's got to go make sure that that all happens, and I was the guy.

Dan: 10:23

So you know that was my thing, like the connection between Bluetooth and windows and windows apps, and you know the, the, the architecture there, and so I worked with you know, making that happen, just as an example. Then we, you know, help Lego build their, bring, their bring, bring their product to market on the, on the windows platform, and so then you know, I got the honor and luck and privilege of helping to you know, write down what Microsoft should do in robotics. This was a couple of years ago and, through a series of people coming to Microsoft and a set of meetings, I ended up being in the strategic missions and technologies team within Microsoft. Where, you know, azure Quantum is where all of our 5G work happens at Microsoft, where all of our space work happens, and robotics is another aspect of that, and so for the last two years, I've been on this team in the office of the CTO, working on robotics related things, which is super cool.

Pete: 11:24

So you know I like to joke that it only took me 23 years to get my- yeah, exactly, I mean you finally kind of got to the destination through a, but you know, obviously part of it is with any of these career journeys, right. You pick up all these experiences and one thing sort of leads to another and now you have a rich sort of 360 degree view of all this context. You know that you bring to the table which if you had just kind of gotten into robot robotics programs 23 years ago, it would have been a different story, right?

Dan: 11:51

So, and, and you know, my, my, uh, the, the guy I worked for, tim Chung, who um came from, you know, most recently from DARPA, um, but has been in um, you know, has been in academia as well. Like you know, he's roughly the same age as me, his journey is about the same length and he went to robots straight and like we're kind of at the exact same place at the exact same time, which is awesome. Yeah, that's pretty cool.

Pete: 12:15

And so what was your take on some of the announcements recently from NVIDIA at GTC around, like Groot and the foundational model? I mean, when you look at I don't know if you've ever met Deepu Tala, but he runs the robotics and edge platforms at NVIDIA. He used to be, he used to interact with us when we were at Microsoft as well and he gave a talk last week or at the GTC and it was really interesting with Jensen's keynote too around. You know applying the they they've always had. You know Jetson based, you know robotics platforms and you know omniverse and stuff. But everything is now seems to have kicked into another gear, especially with generative AI. You know models and architectures being applied to the robotics platforms, but what was your take? I know you said you had some folks at the show. I don't think you got to make it yourself, but what was your take on on all that?

Dan: 13:04

Yeah, I, I, I didn't get to go. There were folks from from my team who were there Um a a. A couple of things. First of all, there is this like awesome point in time right now where you know, and robotics has been this cyclical thing that every few years, um, it becomes hot, it kind of cools down, it gets hot again, right, and this you know, there's always the quip around which is like why is this time different?

Dan: 13:31

And you know, I really do feel that this time is different. You keyed in a couple things that the fact that generative AI is like it's not a fad, it's here, like it's real tech, Like does it have improvement to make? Absolutely, you know that's a, that's a statement across the industry. That's not a statement about any, any specific technology, but is the? Is the kernel of what, what LLMs, LMMs, large multimodal models can do, powerful Absolutely, or is there? Is there applicability? You know, across across digital transformation? Yes, and the great thing about it is what is happening in AI today is creating this awesome resurgence in robotics and what some are calling embodied AI. You know there's been a lot of announcements about what's happening in embodied AI, which is bringing AI to physical systems. Yeah, and going to the NVIDIA announcements, like, yeah, there's a lot of great things, Like NVIDIA has done an amazing job at building up a robotics business number one.

Dan: 14:37

Number two, looking at what their absolute strengths are, specifically around simulation and Isaac Sim and the Omniverse platform, there's obviously a need for their graphics acceleration, their 3D processing pipelines, which happen to be graded matrix math, which happens to be really, really important for both artificial intelligence, but also for creating simulated 3D environments, which is where those chips originally came from, going back almost 25 years to when the Xbox used them, and so, in addition, the ability to do this at hyperscale you know, in the cloud is obviously important and critical, and so I definitely applaud NVIDIA for everything they're doing, all the way from having robotic specific controllers to cloud compute for AI, and then the ability to create simulations, train inside those simulations, deploy from those simulations to the edge. They're working on the whole round trip of data going from the cloud to the edge, right.

Dan: 15:43

Metal to cloud, to the cloud, yeah, and so from that perspective, it's awesome. You know, we had a number of companies had very good announcements, aligned with NVIDIA. We had some awesome keynotes that were done as well. There Folks can go watch them. I'm not here to push the Microsoft point, folks and go watch them. I'm not here to to to to push the Microsoft point, um. But you know, I was very proud of what, what, what, what, the team and what the extended Microsoft team.

Pete: 16:10

Yeah.

Dan: 16:11

Um, and and and and. The fact, like the fact that when AI is being talked about, it's being taught a lot, with the context of robotics actually like, warms my heart and makes me happy because you know AI is not robotics, robotics is not AI, but they're both very symbiotic.

Pete: 16:30

Yeah, it's sort of like you know robotics is where AI meets physics. You know it's sort of like the physical embodiment of AI almost. And you know you mentioned about simulation. So part of it is like how do you train robots in real world environments but do it safely and like? One of the ways to do it is in simulated environments like omniverse and other simulators right, where you can simulate the physics of a real world environment. You can map out a warehouse, you can do stuff and then train through reinforcement learning.

Pete: 16:58

You know these, these robots, um, so that they're not, like you know, knocking over stuff in in the real world over and over again a million times to like kind of learn how to do it right. So I think the simulation and the power of the simulation and there's lots of platforms out there, omniverse is just one of them and the generative AI nature of turning these robots from sort of hard-coded programmed arms into more flexible platforms and running these LLMs on non-cloud platforms right, I mean, we're seeing LLMs and SLMs running on Qualcomm, snapdragons as well as NVIDIAs and you know all sorts of things, and so there's a lot of really interesting work.

Pete: 17:43

taking that kind of that transformer architecture and putting it on some edge chips, and I think that's an interesting kind of confluence of tech happening. To sort of take robots, like you said. I mean robots have been around since you know, the first representation of a mechanic, of a human on a mechanical thing, goes back to the Egyptians like 3000 years ago, right, you know? I mean we've been talking about, you know, robot like human, human machines for for thousands of years. It's got such a cultural significance. I even think back to, like the 1927 film Metropolis, um, and if no one's seen that, I suggest you go look at it. It's an incredible silent film about sort of the future of society and you could, you know, by the, by the way, someone could just make that movie today and it would probably be pretty applicable. In fact, it probably should be remade.

Pete: 18:31

Um, so there's all this like emotional content and cultural content around robots, right? So, like you said, you said you know they come and go and they usually show up in the movies, right, danger, will Robinson and all this other stuff. I mean we can quote all these things and you know there's always that maybe part of it is they kind of suffer from a little bit of hype as terms of robots can do this and actually you know they really can't. They're very fixed and usually behind. Like you know they really can't. They're very fixed and usually behind, like you know, chain link fences, like you know, manufacturing car pieces.

Pete: 19:03

But now we're, and you know actually so the we should talk about probably the only successful robot in the consumer world has been the vacuum cleaner. Yeah Right, the iRobots and similar things. I think I use a RoboRock, I think I got a couple of those, but you know. So then when they, when they're, you know the reality of robots have never really met kind of the hype and the potential out of our imagination. But you know, and we're still years away, we're starting to see where some of this stuff is going to come together and you're going to be able to tell the robot to pick up a coffee cup and it's going to figure out how to pick up the coffee cup, given the environment that it's in and other things.

Pete: 19:43

And so that's exciting.

Dan: 19:45

Yeah, in fact, in that coffee cup example, it's a really good one. The thing to note is that you can't look at robotics as a purely software problem. You can't look at robotics as a purely software problem, like there's been plenty of problems that technology has looked at and gone. This is a software problem. We can go apply bits Exactly Very software, while software is meant to be written so that it can handle exceptional failures in exceptional ways. Hence why we call them in C++, exceptions, right. Right, when you're working with physical systems, you're not working in a binary world anymore, you're working in a. There's many values between zero and one, there's not only a zero or one. And the fact that failure happens in the physical world from things that are expected wind resistance, you know, uh, friction, um, you know, external forces, um, these all cascade to larger things that create safety envelopes, which is why you're talking about the chain link fences where the robots?

Dan: 20:53

operate, and so the, the, the so you have, you have kind of three things going on at once. The first one is the, the, the simplicity to the complexity of what the robot physically can do with the programming and the smarts that it has.

Speaker 3: 21:11

Right.

Dan: 21:11

The second is how we can do that in a way that's safe, reliable and verified in and trustworthy in the environment that it's operating in.

Dan: 21:23

And the third is how we can do that with other humans or robots that are there in cooperation. This problem of the cup that you're talking about is a really good one with that context, because for us to pick up a cup and you know, pick it up and drink from it, it's, it's super easy. You use your wrist, but the but, the reality is that for a robot, like if a robot is off by a couple degrees, or if the, the, the articulation method, um, isn't exactly proper for that cup, it's not going to get that cup and be able to pick it up.

Dan: 22:04

Even if it has the contextual knowledge of I see the cop, I know where the cop is and I have an action that I want to pick it up, and so you know. Understanding how software taking it all back, understanding how software systems need to account for the complexity and difficulty of the physical world in all those different scales that I talked about is where artificial intelligence and LLMs is actually helping us today and the work that, like NVIDIA in simulation, is doing. The simulation isn't only for training the AI, as an example.

Dan: 22:37

The simulation is also validating a route that a robot is going to take or a action that the robot is going to take, or a plan or a mission that the robot is going to take in a simulated environment, and perhaps even a digital twin, where the environment is a physical, digital, you know, combined representation. Right now, all of a sudden, the pre-decision making and, like the, the basic trustworthiness, like yeah, it has even a chance of being able to do this.

Dan: 23:06

You know, can be done through simulation and then yeah as it's, as it's executing, being able to update in real time and address faults and exceptions that happen in real time, that need to be expected rather than exceptional.

Pete: 23:19

Yeah, yeah, and people also to think about robots. You know we tend to go to the. Our imaginations go to the humanoid robot, which is kind of a hot new thing, the humanoid robot. I mean I think Jensen Wong said you know, we've built this whole world, you know, for us, but we have two arms, two legs, you know and you know, and so therefore it's kind of built for humanoid robots and that's that's kind of a good reason why that's that's kind of interesting platform. But, like, as I mentioned, there's there's the vacuum cleaner robots, there's the you know, drones would be considered to be robots as well, you know, for extending observability and things. There's the obviously a lot of the warehouse type robots out there that are, you know, moving things around in Amazon warehouses and other places, and so robotics is, you know, it's not just the, it's not just the C3PO thing going on, but it captures the imagination for sure.

Dan: 24:13

And and and you know, in addition to what you're saying and dovetailing off, that we're also seeing situations where, like you know, wildland firefighting is an example. You know, wildland fires are becoming worse and worse every single year. They're becoming a much more significant issue, both for the safety of the personnel and the firefighters who are going to fight them. You know, for property as well as personal damage. The you know for for, for, uh, property as well as per personal damage and and, unfortunately, um, uh uh, in, in certain cases, loss of life. Um, unfortunately, um, you know, robots are starting to be looked at. How can they be applied to the this problem space? And that goes from finding where the fire is to finding where hotspots are, after a, you know, after the fire has been put out but still needs to be swept to make sure there's no embers or underlying Right yeah, human safety environments, or unsafe human environments, is a great application for everyone who watched Terminator.

Pete: 25:10

As you know, the skeleton is impervious to flame, so that's important to know.

Dan: 25:17

But even like you know, there's cases where water needs to be brought from a local lake to the fire you know there's somebody flying that plane and there's a certain level of danger to that, and so being able to bring autonomy into that process, but even the ability to get the water, that can be an autonomy problem by itself. Yeah, exactly and so. But now you have, when you have something like a wildland firefight, you've got multiple different agencies, some local, some state, some federal, all working together, and you also have robots from you know now and in the future coming together. And how do, how does all that coordination happen?

Dan: 26:00

You know, and you're, you're, you're not in a structured environment at this point you know, and so how does your 5g equipment work, how does your wireless equipment work?

Dan: 26:10

If the fire goes through your, your temporary tower, your edge connectivity has gone down and what happens? And so there's a lot of the, the the really like. To me, the really cool, awesome thing about about this whole robotic space and the the the fact that AI has has come around and we're having this great resurgence in in AI as well is there's all these hard problems that bring edge computing and bring near edge and far edge and the fog and all the stuff that you talk about on this podcast, right, and all the stuff that you love all the way from the low level tiny edge and and silicon to the cloud and how that that the infrastructure there happens for all kinds of different communication but then bringing, but you also have bringing people in the mix and bringing sure robots together, and so this, this, this time that we're at, is this like renaissance, of being able to work in robotics, is the overall realization that you can't just apply software, traditional software, to robotics problems.

Dan: 27:24

You have to deal with the error conditions in ways that are part of the design. They're not just nodes on a map, if you will.

Pete: 27:32

Well, and also you have to and this is one of the things I learned, too is you have to think about these systems as complete, end-to-end systems that need to be deployed and managed. So it's not just building it and shipping it, but it's deploying it, managing it. You know life cycle of the whole things. Like you said, these things are going to work in very kind of complex interactions. One of those scenarios, speaking of drones, that I had heard about was, you know, they're starting to use drones for life-saving on beaches, where you can, you know, get the drone out there with a, with a, with a floatable, you know one of those rings or whatever, and it can find the person struggling and drop the ring there. Obviously, you can't go down there and kind of help the person yet, but yeah, so there's.

Pete: 28:13

There's all these different things and it's I think it's fascinating the confluence of our imaginations and, obviously, the engineering reality, which is hard, and setting the right expectations is always difficult, right? Because I think actually OpenAI just did a video on YouTube people should check it out called their AGI robot. It came out like two weeks ago. I don't know if you've seen it yet, but it's one of these videos that shows demos, shows, demos, and you know it's going to get people all excited and freaked out, but it takes, you know, many, many years to develop and commercialize into police kind of systems. So people also need to sort of temper their expectations a little bit.

Dan: 28:50

Yeah, to that point. You brought up the Terminator before as a joke and we talked about humanoid robots real quickly. The thing to note like they're absolutely as a joke and we talked about humanoid robots real quickly. The thing to note like there absolutely is a real, true argument around. You know that we live in a world designed for bi-legged you know humanoids and so why not Bipedals.

Dan: 29:13

Bipedals, yeah sorry, and why not replicate that with the robots? You know, I'd also make the argument like there's this long running standing desire to have a robot that can wash dishes and dry them and clean them, and like I'm sitting in my kitchen, right?

Pete: 29:30

now, oh, the dishwasher.

Dan: 29:31

Dishwasher. Exactly the one thing that the dishwasher doesn't do is doesn't take the dishes out and stack them and put them dry them and put them completely, dry them and put them away.

Dan: 29:41

So, yeah, you could potentially have a, you know, a standing robot that looks like a human to be able to go do that, or we could change the way that we store dishes in the house to you know, and so there's this, there's this, this, this, this combination and confluence of you know, bringing autonomy into the world that we live, but also structuring some of the changes in our world to be better for autonomy.

Pete: 30:07

Another one is yes.

Dan: 30:08

Yes, exactly, and I don't think it's an or I think it's an and I think both are going to happen.

Pete: 30:14

No, and I think it'll be a little bit of both. Yeah, no, and I think that'll be a little bit of a little bit of both. Speaking of mechanical things, just to shift gears a little bit, you know we're going to get to the topic that I know that you love. So there's nothing really more mechanical or visceral than a pinball machine. And, um, I, I've listened to your podcast a number of times and so you kind of give us a little bit of like, you, you, you're sort of a pinball fanatic. I mean, I don't want to oversell it, but your house is packed with like and your Facebook posts is always like machines that you're building or selling or buying and what. What is going on there.

Dan: 30:46

Yeah, so um it, it, it, pinball goes. I appreciate you bringing that up. So there's a couple of different ways to to to enter this. Um, let's, I'll start at the beginning of that. I'll start at the beginning and then I'll jump forward. I won't go too long. I've been in love with pinball ever since. I was in love with robots, and in my head, pinball machines are just robots under glass is the way I look at them. There's the great conversation over cocktails or over a beer, of what is a robot and what makes a robot. For me, if you're working with the physical world and you're controlling some aspect of it in the digital world, you know you're you're dealing with a robot there.

Speaker 3: 31:19

And I would even go further.

Dan: 31:21

You don't even need digital control. There's plenty of good analog control systems out there. And so you know, I, I I got my first pinball machine when I was 13. I still have that pinball machine to right now in my collection. You are, I only have one floor of my home completely decked out in pinball machines, not the whole house, only one floor.

Dan: 31:43

It's my part, but there's a couple of things. I like to joke that control systems are my hobby and if you look at it, there's truth to that. I have a 1981 DeLorean that I've pulled the ice engine out and I'm converting to electric. I've gotten two months worth of work done in three years, so it's a slow, slow burn. Um, the pinball machines. I enjoy playing, I enjoy the art, I enjoy the history.

Dan: 32:06

You talked a little bit about metropolis and the history of robotics. There's a great history of video, like the. The entire history of video games comes from pinball, like even the term one up, like when you get a one-up to get a free guy. Where does that come from? It comes from pinball. One-up, player, one-up is on. It's right there on the glass.

Dan: 32:24

The artwork, the history, the evolution of the control systems within the pinball machines. They started off fully mechanical, then they brought a little electricity, then there were electronics and it evolved from there. And so the other thing is, you know, as I was going through the journey and I talked about it, I moved up from being an IC to a manager. I've moved between being a manager and an IC through a lot of part of my individual contributor through a lot of my career and the times that I've been a manager, to keep kind of my technical skillset sharp. That's when I would come home and work on a pinball machine. I'd strip them down, take them apart, clean them, fix up the mechanical system. A lot of soldering and re-soldering. I've never actually built my own pinball machine, I've only restored. However, my podcast is around control systems as well. There's this ever emerging group of amazing folks who are building pinball machines in their garage.

Dan: 33:22

And it's you know, there's been a resurgence. There's been this like this has been going on for years. People have been taking pinball machines, taking the artwork off of them and putting their own art on it. But in the last 10 years, 10, 12 years the advent of Raspberry Pis and Arduinos and this whole maker ecosystem that evolved, that bled over into pinball, and there were specific companies that built pinball controllers that allowed you to change the sound, the wiring, the game code, the logic, the light show. Basically, the whole control system was able to be changed out and you could actually bring full computing platforms to it, including the Raspberry Pi, and you could actually bring full computing platforms to it, including the Raspberry Pi, and so people started to really have a platform that they could redo the entire aspect of the pinball machine.

Dan: 34:05

And so now there's a number of people and multiple different computing platforms to be able to create your own pinball controllers and pinball control code, and so the interesting thing is like seeing where artificial intelligence and where robotics and where technology as a whole has come, and software technology specifically, and seeing where like pinball is. You see very consistent themes, like as, first of all, the same names keep showing up, like Rockwell Automation as an example, right, um, the, the um. But the other thing is like only recently did you start getting um over the air updates of pinball code from a server at stern pinball to stern being the big, the biggest manufacturer to a machine in my house, and my machine is connected to wifi. But how long have we been working on that?

Pete: 35:00

I know we've been doing yeah exactly, exactly, but it wasn't.

Dan: 35:04

it was less than 10 years ago that I had to get a USB stick. You know, put some ROM onto the USB stick connect to the machine and and and and Pete. Like it was only 15 years ago that I had to pull a ROM off of my board, put it into a ROM burner and connect that to my PC and flash it to put it back.

Dan: 35:25

But 15 years ago we had over-the-air updates already on Windows and so did other platforms, and so it's just interesting seeing this like long lead lag of of of technology evolution. But on the other hand you you said it yourself there's these aspects of scale and there's these aspects of like full end to end system. You have to think through like, like. If there's a pinball machine at a bar, it needs to have Wi-Fi, it needs to have somebody that can access that Wi-Fi password, it needs to be able to dial back home, and then somebody has to know how to ensure that when the update comes down, the machine resets appropriately. There's this whole end-to-end. And so I love playing pinball, I love talking, I've got a bunch of friends who are into it, but really what I like is just the. I mean, what got me really into it is opening up the machine, pulling the play field up and seeing the way that that machine works. People call them kinetic works of art, but I actually think that they're control systems, work of arts, like of art.

Pete: 36:37

The way it's all connected underneath is just absolutely yeah, yeah, but can you get a humanoid robot to play a pinball machine? I guess that's the question.

Dan: 36:45

So this is it's. It's funny you should say that um, there there was um, as reinforcement learning was getting hot. Four or five years ago now, there was a microsoft build presentation where, although it wasn't a humanoid, somebody did build a full scaffolding that actually articulated With the flippers.

Dan: 37:03

Yeah, exactly, and had a camera sitting on top and did a full reinforcement learning of what was a 1992 Star Wars pinball machine and they had pretty good success to it during and they were basically doing the training in between build sessions. They would have, like how's the robot doing? And so being able to get a humanoid robot to come up and start playing. I'm actually wearing a watch that has robots playing pinball as it's visual.

Dan: 37:51

In the near future. Do you have a favorite pinball platform or? Pinball machine was originally designed in 2019, but they just did a rerun. So I've got a one of the one of the rerun machines and it's like it is the most modern of the classic pinball machines. You can have it as a screen. It's got all the assets from the show. Um, you know it's. It's just a very immersive game, like there's a, there's a uh uh, uh. There's a screen that actually comes down. There's a screen that a comes down. There's a screen that a projector projects onto and shows movie clips or video clips, but that screen actually folds down and behind is the Demogorgon.

Dan: 38:24

You shoot the ball right into the Demogorgon's mouth oh no way yeah, so it's like to me that's the most satisfying shot in all of pinball.

Pete: 38:30

Sure sure you see that ball go into the.

Dan: 38:32

Demogorgon's mouth. But then I also have another machine which is called yeah great, I have another machine called the Multimorphic P3. And the cool thing about the P3 is the whole bottom section of it is all a screen. It's not wood, it's not painted wood, and then the upper section of it is like a one and a half foot by two foot rectangle that you can pull out and place another module. So this is like modular pinball machine.

Dan: 39:01

And so underneath, uh, underneath the number of my pinball machines, I have these like modules that I hold in in in roller boxes and I can place the module into the machine and it's a whole new game, the, the screen changes to whatever the the play should be for that machine, and then I can also download a game updates where I can get new games that use the same physical modules.

Pete: 39:25

Right, right, right, so it's really a reprogrammable game.

Dan: 39:29

Exactly, and it's much more like a Nintendo or an Xbox, where you have a platform and then you have um you know games that you you buy along for it, and then you have physical accessories that go with it. And the cool thing is there's um where the screen is. It can track where the ball is because it's got um, it's got sensors along that lower, lower, one third, all over two thirds of the of the machine, and so you can do all these really cool interactive things, like I've got a game that's the old asteroids game they call it rocks R-O-C, and it's got these digital asteroids that show up on the screen and you shoot the ball over them, the ball goes over it, the asteroid explodes.

Dan: 40:08

It's a super yeah it's, it's an amazing experience. And so the technology of that machine is what got me into it and I I love, like, and the games are super fun for it. And then the third one so I talked about a pure digital machine, a pure physical machine that has a little digital aspect. We talked about a machine that's a real hybrid between the two. And then my third one is I have a virtual pinball machine. It's a full scale pinball machine but it's got just a screen. No, no physical ball, but I've got you know 2000 games in there. And what's cool about it is just like MAME, where you have arcade machines and their actual ROMs that are emulated in software. This has pin MAME in it, so the ROMs that are being used are the actual, exact same ROMs that are in some of my machines.

Dan: 40:53

But then people have gone through and created physical replicas of the of the machine themselves and so you can actually play a digital version. But I've got, you know, it's got solenoids in it, it's got a fake pop bumper in it, it's got an accelerometer so it knows when when I shake it. It's got real pinball buttons and so I mean, once you get in the zone and you're playing, you feel like you're playing a real pinball machine, in fact fact I have a 1990 Simpsons machine that sits right next to it and I can load up the Simpsons machine on this machine.

Dan: 41:23

You can play one, you can go directly to the other and play it. But here's kind of the beauty. That brings it all back to robotics, the control system. There is a computer. At the end of the day it's a computer with an NVIDIA chip taking this or with an NVIDIA processor coming back to the NVIDIA announcements but this idea of being able to do something in simulation, learn and then go and do it in the physical world, this is a really great example. I can learn the rule set and the timing of the flips.

Pete: 41:53

Right and the physicality when the shots are.

Dan: 41:55

Exactly On the digital machine and then I can literally go to the physical machine and although I'm not going to be absolutely perfect my first time, I'm very, very close.

Dan: 42:05

I know the rules, I know right the, the, the shot map, um, I know kind of where the ball is going to go, how it's going to move, and so the, the, the, you know, the, the ability to go between SIM and real and then real to SIM, is one of these places where companies like Nvidia are really pushing the envelope, and that's that's showing up in in in consumer solutions like like the pinball machines, and so that that, that, that. That's why I joke that that control systems are really, really my hobby, like it's it's what I honestly love.

Pete: 42:36

No, that's pretty awesome. I mean I remember playing Asteroids back when I grew up in New Jersey in the 70s, hanging out at the 7-Eleven and playing the Asteroids machine for hours on end. But the pinball machines are a whole other beast and, yeah, you're right, They've been around for a long time. I mean remember Microsoft Pinball. That was a pretty popular piece of software for a long time, right.

Dan: 42:58

In fact get this Okay. It's funny you should say that Microsoft has it's had different names. There's a hackathon, there's fixhacklearn, but you know, microsoft offers time that folks can just like go and explore something that might be interesting to them that's technologically related and carves out the formal time where a lot of people can do this together, whether it's a company level or team level?

Dan: 43:24

um, it was either last year or two years ago. A group of folks got together and took that microsoft pinball and created a physical version of it and there's a real pinball machine with a play field from that game now. It totally blows my mind.

Pete: 43:40

That's kind of really interesting. Yeah, god, well, you know it's quite the imagination. But well, dan, I think this has been a fantastic conversation. I mean, I think kind of bringing it back to what's happening with control systems, robotics, ai, simulation. It feels like I mentioned this about what I saw last week.

Pete: 44:02

It felt like kind of a Cambrian explosion of AI and edge and cloud and lots of ways these things are going to come together and, like we've been talking about the notion of robots in these forms, especially humanoid forms, but also non-humanoid it really is. It's kind of capturing a lot of people's imaginations and I think it's going to pull a lot of new people into the field. You know, you and I have been kind of working on the tech stuff for a little while, but there's a lot of new folks maybe listening to this podcast, thinking about careers in tech and this intersection of the physical and the virtual is really an interesting space and there's a lot of upside in terms of the scenarios, like you mentioned, around going into dangerous areas and helping people and cleaning dishes. I guess that's good too.

Dan: 44:51

Pete. On that note, the one thing that's worth at least just touching really quickly as we close is you know folks are looking at LLMs and this modern generative AI as this way to bridge the human-robot interaction with like speech et cetera, which is great.

Dan: 45:16

Deeper connection, which is once you start talking about multiple robots and how they need to work together to build a product at an auto assembly line or to deliver goods from one part of a warehouse to another you start talking about these fleets of robots and what we're finding with LLMs and LMMs and generative AI as a whole, is the ability to sequence them and the ability, like there's a control system, of control systems, that's coming together, that the other limbs are capable of doing, which is pretty powerful, and so if folks are looking for kind of the, the, the emerging, and like what's what's interesting in the space and what makes it different, that's a good place, like for those listening to the podcast.

Dan: 45:54

You want to go learn more there's. There's a lot of really cool things happening.

Pete: 45:57

There's an example link chaining yeah, yeah, no, and I think we're sort of on the cusp of discovering a whole bunch of new work. Uh, that needs to get done to make these things real. So, uh, but, dan, I really appreciate the time, as always always a fascinating conversation. Uh, we'll link to your podcast in the in the show notes as well, so I encourage people to. If you want to really get into it with pinball, that's a really good show to listen to.

Pete: 46:21

So thanks again Dan thank you, thanks for joining us today on the edge seltzer show. Please subscribe and stay tuned for more and check us out online about how you can scale your edge compute business. © BF-WATCH TV 2021.

Speaker 3: 47:55

¶¶, ¶¶. ©. Bf-watch TV 2021.