EDGE AI POD
Discover the cutting-edge world of energy-efficient machine learning, edge AI, hardware accelerators, software algorithms, and real-world use cases with this podcast feed from all things in the world's largest EDGE AI community.
These are shows like EDGE AI TALKS, EDGE AI BLUEPRINTS as well as EDGE AI FOUNDATION event talks on a range of research, product and business topics.
Join us to stay informed and inspired!
EDGE AI POD
EDGE AI Partner: David Aronchick of Expanso
The digital landscape is rapidly evolving beyond centralized cloud computing. In this illuminating conversation with David Aronchik, co-founder of Expanso, we explore the growing necessity of processing data right where it's generated—at the edge.
Drawing from his impressive background as the first non-founding PM for Kubernetes at Google and his leadership in open AI strategy at Microsoft, David reveals how these experiences led him to tackle a persistent challenge: how do you leverage container technologies and ML models outside traditional data centers? While cloud platforms excel at centralized workloads, businesses increasingly need computing power in retail locations, manufacturing facilities, and smart city infrastructure.
Expanso's elegantly named Bacalhau project (Portuguese for cod, a clever nod to "Compute Over Data") offers a solution by providing reliable orchestration of workloads across distributed locations. Their lightweight Go binary runs on virtually anything from Raspberry Pis to sophisticated edge servers, managing the delivery and execution of jobs while gracefully handling connectivity disruptions that would cause traditional systems to fail.
David makes a compelling case for edge computing with a simple physical reality: even 100,000 years from now, the speed of light will still impose a 45-millisecond latency between LA and Boston. This unchangeable constraint, combined with data transfer costs and regulatory requirements, makes local processing increasingly essential. For organizations struggling with high telemetry bills, Expanso confidently promises at least 25% cost reduction—or they work for free.
Whether you're managing satellite networks, underwater cameras for aquaculture, or thousands of retail locations, this conversation illuminates how the future of computing involves bringing intelligence to where data lives rather than constantly shipping bytes across networks. Join us to discover how this paradigm shift is making AI more effective in the physical world.
Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
Okay, well, thanks everyone for coming back to our Partner Edge broadcast here. I'm here with David Aronchik from Expanso. Thanks, david, for joining me.
Speaker 2:My pleasure.
Speaker 1:And a little known factoid. David and I are going to have coffee later this afternoon, so we're actually living near each other, one of the many conveniences of being in Seattle.
Speaker 2:I know.
Speaker 1:It's kind of it's a strange world. I was actually talking to someone from Sony the other day who I've been talking to for a long time and I thought he was in Japan, but he was in Mercer Island. I was like you're like 15 minutes away from me, but anyway it. You're like 15 minutes away from me, but anyway it's true world. It is a small world but welcome, yeah. So so want to talk about expanso and and kind of your origin story here and, um, kind of introduce yourself to the to the edge ai community? Why don't you kind of give us a what's the backstory here?
Speaker 2:yeah. So it's a. It's a really interesting problem space, everything. So my most recent kind of like arc of the world was I was the first non-founding PM for Kubernetes and I led that for a bunch of years for Google and I co-founded the Kubeflow project also while I was at Google. Then I went over to Microsoft to lead open AI strategy and worked in the office of the CTO and did some of their early open AI work and things like that.
Speaker 2:So you know, I had kind of bounced around a bunch of different cloud native and declarative platforms over the past few years and one thing I kept hearing over and over again was hey, this Kubernetes thing is great. These workloads, containers are great, machine learning is great. How do we take advantage of it? But not in a central data center sitting in Oregon or Iowa or Paris or Tokyo. Like turns out that lots of our businesses are, you know, in retail, in manufacturing, in transportation, you know, in buildings, you know so on and so forth, and compute is growing everywhere. And I was hearing this for years and years. And don't get me wrong, there is absolutely nothing wrong with these platforms of Kubernetes and Spark and Hadoop and Snowflake and.
Speaker 2:Redshift. They're all great. They are all great, but their stories all begin at the moment. You move everything, like I said, into a giant rectangular building somewhere generally in the middle.
Speaker 1:Somewhere else.
Speaker 2:Yeah, somewhere else. So all of that together was hey, you know, I've had some experience starting things. Let's try and start a new thing where we give people a similar kind of power around rolling these things out, but do it in a distributed way, where it's not in a single data center. You can say hey, you know what, I have 250 stores across the Southwest and I want to roll out the same job to every one of them. Or I have five zones, each of which have regulations in them, so I can't centralize the data.
Speaker 2:How do I take my ML model and run it over those remote models? Or, hey, you know what? I'm a smart city and I want to do processing of video at each street corner to count the number of people crossing and crosswalks, right, like, how do I actually get a model there? How do I update that model? How do I change the configuration of people crossing and crosswalks? Right, like, how do I actually get a model there? How do I update that model? How do I change the configuration of that model in a reliable declarative, kubernetes or container or whatever like way? And that's what we built. So we co-founded this project called Backlyow. It means COD in Portuguese, because we call it compute over data right.
Speaker 1:So COD compute over data. I thought maybe you had sort of a Portuguese, I don't know sort of origin story there too Exactly. I was like oh, I've had Macau, I've had that before.
Speaker 2:Exactly. So we were there, we happened to have been in Portugal when we came, came up with it and we kept abbreviating it cod, uh, because we're just texting each other and some smart alec was like, oh, you know, I'm gonna, let's call it back, yeah. I was like, oh, it's kind of cute name, you know. Like, uh, I'd never heard of it, uh. And then you know, if you get there, um, it is just everywhere I. You can't walk three feet without seeing it in Lisbon.
Speaker 1:Yeah, I know, I mean it's, every restaurant has it.
Speaker 2:Anyhow, we liked it. We got the domain name. We thought it was worth it and that's what we provide, and so we started the company about a year and a half ago now, officially kicked it off and got funded by Samsung and General Catalyst and others, and our mission really is you know, it's to make compute over data a thing. Now, to be clear, it's already a thing, right, people already do this with cron jobs and ssh and you know all these various things. What we want to do is make it much more efficient and reliable. Um and and in edge ai. That is all about, like you know, how do I use ai, edge on the edge, and again, I I'm a much more generous person when it comes to edge. Edge for me is anything outside of that big box in.
Speaker 1:Yeah, no, I think that's a, that's a reasonable definition.
Speaker 2:Exactly so. That's edge. So anything outside of that is where you're going to do some data processing, and even though I'm talking to the AI foundation, it could just be running a Python script or bash script script or grep or whatever, or it could be full on. You know, like I said, that video example that I mentioned earlier. You know people want to transfer video in, but video is way too big to send in raw. So if you had a very simple object detection model, you know, on the edge that simply said hey, you know what, I'm only going to send back video when there's a human being in it. Nothing sophisticated, just a snippet that sends that back across. You know magic right.
Speaker 1:Yes, hot dog, not hot dog right.
Speaker 2:Whether or not it's pull the stuff in or push the stuff out, we help in either direction.
Speaker 1:Right, right and stuff out. We help in either direction right, right and and um, yeah, we've seen so many use cases where, especially in video you're talking about ai vision I mean processing it at the point of impact when you're capturing the data makes total sense and, yeah, makes nonsense to actually start moving that video data yeah, exactly but like the, the edge, you know, as we know, is like, notoriously heterogeneous absolutely and so how do you, how are you tackling that Cause?
Speaker 1:there's like the Docker container and then there's Kubernetes and like. So what are your dependencies on the wild and woolly edge out there?
Speaker 2:Well, you know that's something that's kind of crazy. Right Like in my day because I'm very old you would have to have your own completely dedicated build. You know tools and chain and everything to in order to get your jobs out there. Now it's way easier, right, Go Rust, so on. They all cross compile, so all you do is you name your thing and you take it out and run. What the way we are architected is, we're Go binary and Go supports. You know everything. We haven't run into anyone that has a problem. You know running. You know, with our Go binary it's not incredibly small. I think it's 150 megabytes, so it's not teeny-tiny, but it's not gigabytes either.
Speaker 2:It runs on here's my visual aid. This is a Raspberry Pi, right here.
Speaker 1:Okay.
Speaker 2:Runs great on that and, to be clear, what we do is not the job right.
Speaker 1:Yeah, you don't run the job Exactly. You're like the FedEx envelope.
Speaker 2:Exactly, exactly, right. So we orchestrate the job. If you take whatever Lama 405 billion, that's not going to run on that Raspberry Pi. Sorry, right, but that's not our responsibility.
Speaker 2:That's your responsibility to make a decision on. We don don't care, we could run it there if it has the power, but at the end of the day, it's pretty straightforward. Now what I'll say is that you do end up getting into this interesting topology discussion with a lot of our customers and users where they're like hey, you know, I'd like to run this over. You know these cameras, but the cameras are too locked down and so, as a result, we don't run on the cameras themselves. We'll create a nearby node and those cameras will feed a small subset of the whatever 15,000 cameras will feed into that nearby node, and that's where you know a 2U box or something sitting in a you know office building somewhere.
Speaker 1:And that's like a classic Brownfield deployment right when you have a bunch of cameras in the parking garage. Who knows when they were installed, Nobody's going to touch them.
Speaker 2:Exactly right.
Speaker 1:It's all CapEx spent a long, long time ago and so, and so now it's like, well, can we add some smarts into these things?
Speaker 2:and yeah, that's 100% route them into some sort of box, and then you can control the workload on that and in truth I think you just see that over and over again, even on very lightweight scenarios right like imagine you know I don't, uh, this is my house. So like you don't see it, but like if I was in an office building you'd probably have a thermostat, you'd probably have a fire detector, a smoke alarm, whatever.
Speaker 2:You have a variety of different things in the room itself. That's all brownfield too right Now. A lot of times that'll be wired into a smart panel in the building that can be more sophisticated, but even then, you know, like, there's this idea that like, oh no, the only way to do this stuff you know anything is to push it off into that data center. And the fact is that's just never going to work. And not because, like you know, I'm deeply incented for it not to work. The reason is is because the speed of light in 100,000 years time, the ping rate between LA and Boston will still be 45 seconds, 45 milliseconds, excuse me, right, there's no way you can't get it any shorter than that. That is the speed of light. So, for every byte, for every exchange, for everything where you need to do some form of interaction back and forth, do you really want to be adding you know that minimum latency on top of it, or do you want to be doing it nearby?
Speaker 2:And more and more people will want to do it nearby.
Speaker 1:Well, there's also the cost of ingress and connectivity and all this other stuff that you know just doesn't make sense. Plus there's, you know, deployment issues too around. People want to simplify the cost of deployment, and you know they don't want to have to be sending throwing data back and forth to make things work.
Speaker 2:A hundred percent. A hundred percent Totally.
Speaker 1:Which is kind of one of the. I mean again, I'm biased, coming from the Edge AI Foundation, but the pendulum swinging toward the edge, I'm getting these workloads done. We say connecting AI to the real world, and this is like running AI in the real world in the parking garage itself where cars are driving around. So where do you guys so you sell into? I saw a little bit on your website you did. They have a story there about the US Navy and some cool stuff that you did there and some smart city stuff, and are you primarily focused on the US market.
Speaker 2:Are you doing like? How do you build this business out? I mean, we go where the folks are. We are, you know, a global project. The project is open source, but the binaries are commercial. The binaries come from us and they have our whole security building materials and build chain and all that kind of stuff, and they have our whole security bill of materials and build chain and all that kind of stuff.
Speaker 2:The summary of it is, though, that you know we're talking to satellite manufacturers, we're talking to auto manufacturers, we're talking to transportation aircraft, so on. You know, and I don't mean to like make it all about these like enormous deployments, but it's true Like those people are feeling this pain, right?
Speaker 1:now.
Speaker 2:Now, that said, I like to say look, if you've, you know, got an Ethernet cable or a Wi-Fi card, congratulations, you have a distributed system, right. It's just the way things work. Things will get out of sync, things will, you know, have changes on either side, they will have disconnections, they will have all sorts of things that you end up needing to resolve. What happened while?
Speaker 2:you were disconnected. That is just the reality, and a lot of times human beings do that. A lot of times you have automated systems. You're like, hey, if I get you know. If you look at any of those clustered systems that I talked about earlier, they take a super aggressive position around it. They basically say look, if you disconnect for even Spark and Databricks, it's like five seconds, kubernetes, I think it's 30 seconds. If you disconnect for that amount of time, we're just going to assume you're dead and we're going to move on. Right, we're going to reschedule the job and so on. And that's kind of broken for scenarios where it's a look, it's perfectly fine that this building was offline for three minutes.
Speaker 2:Sure, it's not the end of the world, like just you know when it resolves, bring it back together, but like in the interim, you know, you, you, you figure it out and that's again, that's really what we do.
Speaker 1:So you provide kind of a kind of a resiliency to the organization.
Speaker 2:Exactly Right. So, yeah, what I what I was saying is is that the you know to the your question, it's not that we don't solve problems for a lot, you know for lack of a better term, smaller folks, because I will tell you, you know, as someone who has deployed and I won't badmouth any cloud in particular, but if deployed to two zones in a single cloud, let me tell you it's not fun, right, Like if I could have a single network that spanned both of those. That would be really convenient for me. And you know, again, that's something that we just do right out of the box. All you have to do is put a VM in each zone and congratulations, you know, you have a scheduled job.
Speaker 2:You can use different Kubernetes, clusters, use different storage buckets, use different compute whatever is available in that zone, and it'll feel native. Use different compute whatever is available in that zone and it'll feel native, um. So even if you're really small, like I said, two, two machines and two different zones, we can add value. But where you're really starting to feel the pain is some of the stuff you talked about earlier. You're in, you're geographically distributed or you're um, you have, you know, a thousand retail outlets or you have ingress costs or things. If your telemetry bill is starting to creep up, if your telemetry bill is more than 20 grand a year, I guarantee with no like we will deploy and save you 25% right off the bot, right out of the bat, or we work for free. I tell my customers all the time that's a good one, because we can make that big a difference.
Speaker 1:Yeah, and there's a lot of these systems that are I don't know if the right word is they have intermittent connectivity or somewhat harsh environments. You mentioned satellites. I mean my understanding is that commercial satellites are going to be going down to 10 centimeters soon and then down to 3 centimeters, absolutely are going to be going down to 10 centimeters soon and then down to three centimeters, absolutely. So we're going to see, you know, satellite usage and in ai vision and satellites to become big, a big deal for, like agriculture and a lot of places where you know you're not you're not going to stick a camera out in the middle of every farm field or whatever.
Speaker 1:So, absolutely so. I think those kinds of systems where the connectivity is you, you know, not really inherent is going to be a big deal. And you mentioned also, like with the Navy and like we had a live stream this morning with SoftBank talking about aquaculture.
Speaker 2:Very cool.
Speaker 1:Yeah, they're using like AI vision to count fish and fish sizes and they did this whole thing, the deployment out there and you know underwater cameras and you know connectivity is pretty funky and underwater cameras and connectivity is pretty funky.
Speaker 2:I mean, obviously this isn't a business. What do you call it discussion? But we'd love to hear that.
Speaker 1:Yeah, yeah, but there's lots of those deployments and yeah, and it turns out that the real world isn't great with connectivity and electricity and all these other things that we like to have when we're in a data center. So it's pretty cool that you guys are tackling that and so you've been around for about a year and a half, yep, and you're are you guys like pretty distributed in terms of your workforce and stuff? You have some. You're here, you're in the, I should say, pacific Northwest, but is everyone sort of around? I mean, how do you guys organize?
Speaker 2:Yeah, we're super distributed, which is, you know, good and bad. I miss being in the same room as everyone. Right, I really like it. But you know, you go where the people are. We have a tremendous team from you know folks all over the place. Do you have someone in Portugal? Do you have someone in Portugal from you know folks all over the place.
Speaker 1:Do you have someone in Portugal? Do you have someone in Portugal?
Speaker 2:No, ironically nobody in Portugal. But yeah, so yeah, it's funny. I do wish that we were a little bit more centralized, because I really do love like seeing people, seeing you know, right face to face, and so on yeah, and yeah, it's interesting.
Speaker 1:You mentioned that like we, I think we uh, to a certain extent. Like we have our community events. You know, three times a year we were in austin last week, we're going to be in milan in and then Taipei, and it's a few hundred people and we get together for a few days and for a lot of companies now that are distributed, these are opportunities to get together face to face and it actually has that good community feel, totally agree, and it's like oh, I can actually talk to somebody and have a cup of coffee with them and show them how my stuff is working and we can talk about a deal or argue about a standard or whatever, and so it's. It's kind of cool to kind of see people face to face like that. And I think maybe in in this age cause I also work from home, you know, I look forward to having those kinds of events where you know you actually get to see people in physical space for for some period of time. So that's a good thing.
Speaker 1:Yeah, I was gonna ask you. So you got involved with the aji foundation? Thank you very much for your support of course, what was your?
Speaker 1:uh, what's what? What are you looking to get out of the foundation? Like what's your, because you joined fairly recently, but what's your yes?
Speaker 2:what's well I mean, you know it's. It's funny because it's a lot of what we talked about on this very phone call. Right, like I'm not going to have visibility into all the sort of people who are interested in being part of the Edge AI Foundation and joining you know you talking about all the customers that you mentioned on this call and previously on the thing, on this call and previously on the thing, and for better or worse, as someone who's gone through this several times the past few years, right, both the move to containers and the move to declarative machine learning pipelines.
Speaker 2:I'm well aware that this is an industry move for better or worse. Right, talking about setting aside whether or not you use us though I certainly hope you do, or at least give me feedback about why my product is garbage, our product. Moving the computer for data feels inevitable, right, because of all the things I talked about earlier about cost and security and time and so on. And in order to do that, right, it means us working on standards and frameworks and so on and so forth. And you know, there's kind of no way around that and the only way to get to things like that is through, you know, rallying points like the edge. Uh, you know the the edge AI foundation.
Speaker 1:Yeah, no, it makes sense, it is. It is a team sport at the end of the day, absolutely Great. There's a stack there. I was telling folks last week and we were talking about like the Dagwood sandwich you remember Dagwood and like the, you have the cheese and the all the different layers in there and like, when you look at edge AI, there's, there's a lot of layers in there that need to work together to solve somebody's problems. So so it's good that you guys are you're in there orchestrating moving bits around, which is important well, we're certainly doing our best.
Speaker 1:Appreciate that. That's awesome. So, david, thank you so much for your time. I think this has been a great introduction to expand. So, this whole area of middleware and orchestration and how do we get smarter about moving bits around, especially on the edge, is really a hot space, so I look forward to your future success.
Speaker 2:Well, I really appreciate that and certainly we hope to do it with partners, with the team that you mentioned earlier. The team effort Awesome Sounds good All right, David, I'll see you soon for coffee. See you soon.