Gael and Thabang are joined by Adam Gibson, who shares lessons learned from ten years in open source Java, discusses deep learning infrastructure, and why Konduit adopted a SaaS model.
Follow us on Twitter: @OSS4Biz
To learn more about the business value of open source, visit eclipse.org
Gael and Thabang are joined by Adam Gibson, who shares lessons learned from ten years in open source Java, discusses deep learning infrastructure, and why Konduit adopted a SaaS model.
Follow us on Twitter: @OSS4Biz
To learn more about the business value of open source, visit eclipse.org
Thabang Mashologu:
Hello and welcome to another episode of the Entrepreneurial Open Source podcast, a show where we discuss the power and practical value of open source in business and entrepreneurship with leaders and innovators from around the world. I'm Thabang Mashologu.
Gaël Blondelle:
And I'm Gaël Blondelle.
Thabang Mashologu:
We've got another great episode lined up for you today. Adam Gibson, the CTO of Skymind and Founder of Conduit is with us today. Let's jump right in. Adam, can you give us a sense of how you got your start in open source?
Adam Gibson:
Yeah, so I actually, back in 2011, my university actually got $1 million grant from IBM to develop open source Java software. And so we got started very early on using the J2E, well what's now Eclipse Jakarta EE stack, Spring, Tomcat, MySQL. I learned about the fairly traditional app development stack that we see nowadays in the Java world. I got my start using Eclipse as my first ever development tool in computer science 10 years ago, more than 10 years ago now. And it's been, yeah, what can I say? I mean, I've been doing open source Java this whole time. I moved from full stack web development in open source Java technologies to doing machine learning. Much of that was after I took an AI class in 2011, started building out applications, reading, I did a lot of self-taught work, reading papers and then eventually dropped out and started my own company. And that all happened after deciding I learn a lot more on my own than in school, so let's go that route. And somehow it worked and I did the whole Silicon Valley thing, and here we are.
Thabang Mashologu:
It sounds like the classic Silicon Valley story. What was your first company?
Adam Gibson:
So I was doing just web development consulting back then, and then kind of from there, transitioned to doing, well so basically what happened was that all wound down and then I got a one-way ticket to San Francisco because I got an invite from Y Combinator's startup school. Y Combinator is a startup accelerator and startup school now is a program, like an online program, but it used to be an invite-only program where they would just invite out founders to meet high-end, like we have Mark Zuckerberg, Jack Dorsey, and all these other crazy people presenting and that, and what happened was I was supposed to just, I had $2,000 and a one-way ticket. I just bought my ticket and I just stayed. What I did, I made it up on the fly. I decided, "Okay, well, let me just start by seeing what I can do out here."
Adam Gibson:
And so I rented a bed in downtown San Francisco in a three-story warehouse, and from there, yeah. I mean after that, I started, I accidentally found a consulting job doing data science education, because I had already been doing things at that point. And I started teaching at a now acquired a bootcamp called Zipkin Academy. And I was building Deeplearning4j during that whole time, my open source project we'll discuss later. And from there, the whole journey all the way to Eclipse a couple of years ago.
Thabang Mashologu:
Very, very cool.
Adam Gibson:
And so Skymind started as a San Francisco company that went on to do reinforcement learning. We were originally doing open source deep learning, and then eventually we split off, became Conduit and started doing and continued doing open source, deep learning infrastructure. And they're now doing a SaaS, a SaaS for reinforcement learning. So basically that, from 2013 to 2017, I was essentially kind of based in San Francisco and Japan part-time. I was doing a lot of work in Asia and then eventually just decided to move here full-time and more pursue the open source thing. And that's kind of how the whole, and then the project that donated in 2017, and then we're continuing to work on it now. Conduit is do mainly doing, model serving dev tools, open core, very open core business model focused on model optimization, helping people deploy deep learning workloads.
Adam Gibson:
Because when I started Deeplearning4j, it was always the mission to not do research. Actually it was more about helping people benefit from cutting edge models, not necessarily do all the Python stuff that you see nowadays. So DL4J from day one was never meant to be a tool for researchers, but inevitably they do dictate the bulk of the deep learning stuff. One hard lesson I had to learn is that data scientists are the main consumer of their own models, which most less than Python. So building a tool for Java, we built for a very dedicated space, but we ended up needing to kind of learn how to interact with that world. And we've managed to build a good community around like building models in Java, importing models and making them work and then helping people build applications. So we've been on the same space with different variations for a while now, and that's still the focus. So that's the whole Conduit journey.
Thabang Mashologu:
So how did you connect with your co-founders?
Adam Gibson:
So we met at, we actually met at the place where I was staying, so at the time yes. And then we decided to get together and we both liked doing AI stuff and I was building Deeplearning4j at the time. And I wanted someone who could help me with marketing and all sorts of other things. And we had a good complimentary skillset and we managed, he had the good idea to apply to Y Combinator and do all those things. And despite us going our separate ways, I think things worked out pretty well. And so now we're doing now Skymind itself now has an Asian side as well. So we went in under that team as a product division called Conduit, and that's the focus, the focus is on the product development of Deeplearning4jamie and dev tools for deploying deep learning models.
Thabang Mashologu:
Right. And I think you've touched on this a bit, Adam, but if we can double click, what problems does Deeplearning4j uniquely solve? What is different about the altitude?
Adam Gibson:
So Deeplearning4j was built in a time where there was no Java tools for deep learning. I was kind of scratching my own itch, like many, I think, many startup founders and open source project creators do. And over the years, we had to learn what people wanted from us. In the end, the main reason the project became full-time was we had, I just randomly started getting users. They were filing issues. And I was like, "Oh, what's this? We seem to be getting traction." And what was crazy though, was for the first year and a half, I was by myself, I was doing support and I was the only one writing code.
Adam Gibson:
And so now, and then eventually we grew a team around, a team of core contributors and built the community. And that's kind of where that's kind of how things started. That's kind of where things are now. We have a few people we work on now in this space we work, we still have the same kind of core users I think, people who build models in Java, people who don't like Python and a separate community of users now who use us on Spark. Spark is still a fairly big section of our user base. People who like building, who like easily deploying models on Android, we're still easier to use than some of the tech, some of the stuff that Google builds.
Adam Gibson:
So people prefer that. And from there, where are we now? Well, we focus more on building a complete end-to-end stack for deploying and applying deep learning models now. And in this case, we're working heavily with [inaudible 00:08:55] and the whole ahead of time compiled ecosystem. Now, as far as what's unique, we're still the easiest way to train models in Java, and we're the only one that's kind of neutral. We play with PyTorch, TensorFlow, whatever people want to build their own models in. We even have our own Python distribution. So we actually wrapped, we have this new project called Java CPP that we rely on for a lot of the native inter-op of different C, C++ tools. And so that allows us to build very unique infrastructure. We have our own open CV bindings, our own LLVM bindings, and our own pointer allocation, our own memory management, everything. And so DL4J has a very C++ like internals, but from the Java virtual machine.
Adam Gibson:
So it allows us essentially to pre deploy a deep learning model or accelerated math routines in C++ as Java files. And what we still do to this day that I don't think a lot of people do is we actually run on Windows, and no one still to this day provides a good out of the box experience of just running on Windows and development, and just running in enterprise environments. Some people, they need custom C++ code or all sorts of stuff. So now what we tend to do is to say, we make it easier to deploy Python.
Adam Gibson:
You can take all of our stuff, deploy a whole Python script, and the whole C Python distribution, and everything is just binary now. You can do that or you can deploy an existing model that you already have pre-trained. There's all sorts of things you can do. So we interact with different frameworks. We're neutral. We make it easy to deploy things. I mean in terms of features, it's all around delivering accelerated, deep learning models in any environment you want to run deep learning in. And we're the only framework solely focused on that. Like I said, we started in kind of the age of when Spark was just coming alive, the Hadoop, the age of Hadoop and training models in Java. Nowadays, we still train models in Java, but as Python started taking off, we came up with the first version of model import. This is even before Onyx existed. We actually imported Keras models. Keras was, before it became full-time TensorFlow, was a multi backend framework for training models in Python. And you could just add different back ends. Theano was the popular library at the time, and then TensorFlow support was added later.
Adam Gibson:
And so we've always been for less, since 2000, since we donated to the Eclipse Foundation, we've been about modeling a framework inter op, and making, taking whatever innovation is happening in the Python space and making it easy to consume and deploy models. And so, and we use that in our own in our own software as well.
Gaël Blondelle:
And I understand from your description that one of the traits of Deeplearning4j is definitely to enable companies to use also innovations that come from the AI landscape. So maybe, can you tell us a bit more about this, your role in onboarding those innovations and make them consumable by more maybe traditional companies?
Adam Gibson:
Yeah, so there's a few kinds of segments. One is just people who want an easier way of embedding deep learning in their Android applications. Another is people who want to run things on Spark. And from there, the other thing we do is we actually enable the use of other deep learning frameworks. And so, since Java CPP actually under underlies the TF Java bindings as well, the official ones, we're able to just do inter op and have our ND 4J library, NDArrays Java act like a Numpys, Comic Con Numpy's a common data structure used among all deep learning frameworks. We actually have similar support in our ecosystem. So whether you just want an easy to use Numpy like interface in Java, or you want to train, or you want to build and train deep learning models in Java, we allow any of those.
Adam Gibson:
And the core theme is still just to accelerate deployment of deep learning in any application environment you can imagine. We're running things on Raspberry Pis. We're running things on Android phones. We're running things on big Spark clusters. And so, you can do all that from one place rather than, and again, it's all pre-packaged. One of the things we try to do is we try not to do too much. We outsource the things that we're not going to be as good at. We had a number of experiments over the years. We have a number of projects we actually did sunset because they didn't have users, or they were written in a different time, like Arbiter. We had automatic machine learning and all sorts of other stuff that people may find useful, but ultimately, generally those are going to happen better in other more focused places. And so the main theme is again, just around either complimenting other frameworks or the core focus is accelerating the deployment and delivery of deep learning models to them.
Gaël Blondelle:
One thing you started your story by saying that, okay, we published our project and then we started to get adoption and request. And so that sounds likes a dream bootstrap situation for an open source project and a software startup. But can you tell us more about your business model and we are in this entrepreneurial open source podcast. So where do you make money? How do you make money?
Adam Gibson:
Yeah. So a lot of it, so early on, we were just doing consulting, like I think a lot of open source companies start as. You try to do support, you try to do the Red Hat model. And what we've learned is that people, trying to do that, open source projects are not incentivized to create a good experience for their users. Documentation suffers, you hold something back. And so what we wanted to do is find a way to kind of compliment that. And so what we do now, and what we'll be offering is actually a SaaS model where a lot of open source companies do this now. A lot of people have moved to the cloud now. We were originally focused on on-premise and we still allow that, but that's not really how people, even banks are figuring out how to do the cloud now.
Adam Gibson:
So we're at a point where we're going to offer easier ways to using our framework. People don't even have to know it exists, but it's those stack we use to download optimized, deep learning binaries, you upload some code, you upload your AI machine learning pipeline, and you just download a complete binary that gives you analytics and gives you everything you need, and it's optimized automatically for the hardware you pick. And it will run no matter where you want to. And it's just a binary. This is enabled with all open-source technology, [inaudible 00:16:23], TVM, deep learning compilers, enabling the shrinking of models to make them run on smaller devices and our own C++ stack that it's already pre optimized and pre-compiled for different devices. So that seamless delivery of models to the market, either via just running a Python script, or just uploading a model and downloading a whole application you can run, or in this case, sometimes us hosting it for you so if you want to just do model serving, let us do it for you out of the box. And so what we'll offer essentially is a compliment to our framework and that makes people's lives easier. And that's kind of the core goal now. We started in research consulting and trying to do open source infrastructure or open core infrastructure, like we had a closed source offering at the time that focused on model tracking and all those kinds of things.
Adam Gibson:
But MLOps is fairly common now. It's either open source now or companies do it, but they do it in a Python focused fashion. Neither of that was really our core focus, or I think our strength. Our strength is in performance critical application workloads that need quick delivery. So being able to compile models and seamlessly being able to seamlessly to play Python scripts to just an application without needing to know the details, just downloading a minor, you're good to go, accelerating that deployment or an offering hosting for people is kind of the major value prop. So DL4J is open source. People can use it, they can assemble it themselves, so we get user input. It enables us to understand what models people are building, enables us to understand how people are running in their different environments. And then we can offer the complimentary offering to make people's lives easier. That includes our existing Java user base.
Thabang Mashologu:
Now, Adam, would you say, or can you give us a sense of any usage scenarios or ways that DL4J has been implemented that completely surprised you, that you hadn't envisioned?
Adam Gibson:
So actually, it's kind of a user base we ignored for a long time, NLP. Actually Java has always had, and this is how I got my start as well in building Deeplearning4j was building NLP models. We got our start as a, mainly, it's a model, it's an algorithm called Word2Vec that Google patented, I think a couple of years ago, but it originally came out in 2013 roughly, and it was an automatic way of modeling language, just by brute forcing, showing a neural net, here's a bunch of texts. Here's how they co-occur. Learn these patterns. And then you learn that men and women are similar. Those are both genders. King and queen are similar because those are both classes of monarchy.
Adam Gibson:
It automatically learns concepts. So I built the only Java implementation at the time. And Google actually referenced it as a reference implementation. And that's actually where we got a lot of our traffic, so we built for that audience, but then slowly moved away from them because we were doing model tracking and all these other MLOps things. But then what we discovered was that people, NLP people were still using us, but they were building their own pipelines instead. And what's crazy is models got bigger, the NLP models got huge. You look at GPT-3, you look at BURKE, these are Trillian parameter neural networks now. And so what are people doing with us? They're using us to train these big models and they're still using Java. Why are they using Java? Deployment and the pipelines are faster because it's a real multithreaded language.
Adam Gibson:
It's what people deploy in. And it's still a big use case. So we're taking a pretty hard look at how to serve, I would say two markets. One is embedded computer vision is still very big. People use it for almost everything. Mobile phones, they're using it to do OCR. They're using it to do, obviously they're using it to do safety, public safety use cases. There's all sorts of things. So being able to run the Raspberry Pi is important. To be able to run on phones is important. But then, like I said, these big, the area where they need performance, and so computer vision is about reducing the cost of compute. Whereas I would say NLP is that, but it's just being able to run these models at all because they're so big.
Adam Gibson:
So these two performance area, performance critical areas are where we focus. And so, because that's the bulk of our user base. People are either trying to build small computer vision models to run on smaller devices, or they're trying to just deploy these big models in a large production environment. And so accelerating those workloads is our kind of the primary focus of the framework now. And from there, we'll build tools around that. And that's where people have the most pain. Like if you look at Hugging Face is a great successful example. They built tooling just around NLP. They took one neural network and made it really easy to just customize those models. And it was all open source. And now they have this big SaaS play. They raised a lot of money and they just did one use case.
Adam Gibson:
And so for us, we're trying to do something similar. We're looking at what problems people have because the problem is the bulk of the model serving market is actually very, very small. And no one will tell you that, but it's actually very small. Why is that? Because data scientists largely just need fast iteration. They don't need performance. They just need something fast enough for their own use cases. Because data scientists are actually the main ones that consume their own results. So we focus on accelerated deployment of computer vision and NLP models because that's what people need. People either need us to shrink their computer vision models down because computer vision is easy enough now. We've figured it out. We know how to do it now. Now it's about making it more efficient. NLP is too big to run in a lot of scenarios.
Adam Gibson:
So we either want to make it faster because the models are so big or make it cheaper. And so we're focused on areas where performance is critical and people either need to, generally the alternative is people converting their models to a different format. Like for example, there's TensorFlow and TensorFlow Lite. They're actually completely independent frameworks. They're two separate things. So people actually have to, it's a different file format and everything. So people actually have to convert their models. There's a lot of specialized hardware that have like adapt, they have their own, they'll supply their own SDK. And they'll generally have a converter from the mainstream framework format to another. So to me, a deep learning framework is actually just a file format people use, and it's a way to build a model, but then you can convert it. You can manipulate it, you can run it in any language, you can optimize it.
Adam Gibson:
So to me, taking that model, taking whatever file format people use, and then streamlining it and making it easy to deliver to the market is actually a fairly good start. And so one thing to do there though is also to leverage the existing, what I'll call models zoos. Models zoos are places where people host tons of models, kind of like the Eclipse marketplace actually, or the VS Code marketplace, the Open VSX and everything that we're doing now, but for models. Models have licenses. They're just big zip files basically. And generally, what people do is they download these multi-gig models or sometimes bigger than that. And they want to customize them. That's a process called fine tuning or transfer learning.
Adam Gibson:
And so this is really popular in computer vision and NLP. So what will happen is Google and all these other big companies will use, they'll say, "I spent $2 million on training in this model from scratch based on Google's hardware, my TPU, or my big GPU cluster or whatever." And they'll publish these research papers. That kind of compute power is actually out of the reach of most mainstream research institutions outside of the big industry labs. But they do publish these models as open source, not the data sets though, but just the models. So what anybody can do is they can take them and then fine tune them, because why can they do that? Why does transfer learning exist at all? Why does this whole thing happen?
Adam Gibson:
Because it's trained to recognize lots of different patterns by default. So it's good. It's actually a good entry point for a lot of different use cases, a lot of different languages or whatever else. And then you just have to specialize a model for your particular use case. And so that's actually how most deep learning models are built. No one actually builds models from scratch actually. Most models, think about that for a second, most models are actually customized from Google or Facebook because they already have large compute clusters, training things on all sorts of patterns by default. The problem is, is these models are way too big to run in production most of the time. So actually, the first time that ever happened was Netflix price. What people don't know about the Netflix price is that Netflix didn't use the full solution from the Netflix price. They used a part of it because the full solution was actually too hard to scale and deploy. So they only used a part of it. That's still true in machine learning today, actually. Yeah, go ahead.
Gaël Blondelle:
At some point, you decided to contribute Deeplearning4j to the Eclipse Foundation, and from our point of view, that was a really important moment because you guys tweeted this very nice tweet saying that, "Okay, we selected the Eclipse Foundation because we think that's the best place to do business friendly, open source." So maybe, if you can tell us a bit about how you took this decision and why you took this decision and what you get from it. That would be very helpful for our listeners.
Adam Gibson:
Yeah. So yeah, so I mean, to this day, I still see Eclipse partially as the Java Foundation. It's where Jakarta ended up. It's the defacto IDE for Java developers. It's the place where a lot of, I think a lot of, it's a place where obviously OpenJ9 is as well. It's just a very JVM friendly place. And our main roots were in the Java community. And so that was part of it. But also the governance. Eclipse allows me to do things that I think Apache would not. And with the right amount of, so it's the right amount of freedom with the right amount of governance. Eclipse provides support for a lot of the infrastructure. Eclipse provides industry contacts. Eclipse provides use cases. I would say the working groups, as they've evolved over the years, have become a very compelling reason to be involved.
Adam Gibson:
And so that allows us to apply the open source projects. So it was a mix of things. The governance, I would say the least, I would say it was a good place to put vendor neutral trademarks and everything governance. That give us guarantee of the community. Otherwise, you have, anybody can relicense at any time. There's all these, especially nowadays, there's a lot of paranoia around what is open source now with the SSPL and all these other things. What do you do about that? And then how do you continue to build a business or build on top of something without it going away? And so Eclipse allows us to provide the tools, provides us just the right amount of governance. And it allows us to provide guarantees. And it's a trusted name in the space. So before that, this was even before LFAI as well. Yeah, go ahead.
Gaël Blondelle:
So those good characteristics of vendor neutrality [inaudible 00:28:49]. Tell me, how does it help you to get more users, more contributors maybe or more adopters?
Adam Gibson:
Yeah, so in general, Eclipse helps a lot with promotion of the open-source project. Eclipse helps, I think just with perception alone, Eclipse forces us to think about things we normally wouldn't think about licenses. It forces us to think about the best way to run a project and forces us to think about how do we work with other companies, because we were, and still are largely a single vendor project, but in the last year, we've had more contributors from different companies than we have in a long time, because we started ramping up our engagement with Eclipse and the community. And Eclipse pushed us to build more things in public. So a lot more things are public about the project now, roadmaps, support. I would say one of the biggest assets is just our community forum. Anybody can go on there, they can post questions, they get answers and it's all public and anybody can search it. It's all there.
Adam Gibson:
And so that's actually allowed us to build and engage with the community a lot better. Before that, we used to just use a real-time chat called Getter, which is still being used by a lot of projects. But in general, one thing we found is that people tend to re-ask a lot of the same questions, which is why we mainly focused on our Discourse forum and it works. So Eclipse allowed us to make all those decisions. So it gives just enough constraints to make sure there's good governance, good guidelines, but it gives us enough freedom to also execute and operate how we want. I mean, obviously, Eclipse provides its own GitLab and all these other things, which is important because a lot of source projects may not have the infrastructure. In our case, we had, I'm pretty sure we're still one of Eclipse's bigger projects.
Adam Gibson:
We're 4.1K stars now, almost 5k forks. And we still get 30,000 visitors a month on the website, on our community forum at minimum. So it's not TensorFlow, but it's not small. We have users all over the world. And what I found is that when big companies are using us, they don't email us. I just get an issue and I see a big company and I'm like, "Wait, what are you doing? Holy crap. I want to know more about this. Can I talk to you?" And they're like, "No, we're under NDA. And I'm just here to file an issue." And so it's very hard to engage with big companies sometimes because most of the time, these are just normal developers trying to do their jobs. And they've never engaged with open source before. So they don't really know what to do.
Adam Gibson:
And I think Eclipse gives us a name, a friendly name that says, "Here, you know what? If all this stuff kind of scares you, we're a foundation. Here's some resources and here's how we're governed. Go to the community website. And then you can figure out for yourself if you want to get involved or send this to your manager." And then Eclipse has all these big logos you can show that any manager, any normal manager would understand. And then Eclipse has all these outreach efforts that engage people who are not familiar with open source. So I don't have to deal with that. I can just outsource that problem to the Eclipse Foundation. Anybody who's not familiar, it's a good name to show them because by association, you have all these big names and then I can focus on the governance of the project. And then when that developer and I want to talk, we can just talk code. I can suggest, "Hey, maybe you can contribute. Here's like our community guidelines." And then if they need their manager's approval or whatever else, then we can kind of help them with that. And then Eclipse provides support for all that.
Thabang Mashologu:
Yeah, the good news is that's what we've heard from other folks as well, and glad to see the impact, the positive impact in terms of your business and the growth of your community as well. Switching gears slightly, just in terms of industry mega trends and buzzwords and that kind of thing, are there any misconceptions about deep learning and AI and the general space that you'd want to clear up?
Adam Gibson:
Yeah. I was going to say, not a lot works today and most people don't use deep learning. They use normal, they use models that we were using 15, 20 years ago, decision trees, random forest, all those things. Why is that? Because they're reliable and they're interpretable. Deep learning's interpretability is still a big problem. People are working on it. Deep learning models can be biased. That's still a big problem yet. And so the simpler models are generally more approachable. They're easier to understand. The math is fairly simple. And in general, it's still reasonably fast. Most use cases don't need to deep learning. Only very, very specialized unstructured problems, like vision, speech and text really, that's where deep learning really prevails. Everything else, it's overhyped. And another thing is most problems are just supervised learning. You have a label or a target that you already know that was human labeled. And then the machine just learns to recognize what's similar. And that's it.
Adam Gibson:
Machine learning is categorically stupid. Don't assume that because it's all just data, it's still data curated by humans. Models still will make dumb mistakes. And one thing, even people deploying machine learning today don't understand is that models need to be retrained. Models are a software asset, just like anything else. They have bit rot. The world changes. A great example of this that anyone would understand is fraud. You have machines that predict fraud and the fraudsters understand, "Oh, their models detect this behavior now. Let me change my behavior now so they can't detect this." And so it's a never ending battle between spam people or fraudsters and models and analysts.So everything requires maintenance. The world changes over time.
Adam Gibson:
A great example of this actually is IOT and industrial data. The world will change as sensors change. Actually, sensors become faulty. How do you detect that? How do you know? What if it goes undetected and you start getting bad data from your sensors? What do you do with your machine learning algorithm in that case? How do you make it robust that kind of data? People don't think about these things yet. They train a model on Kaggle. They think that Kaggle is like, they run contests to build models and everybody's like a grand master Kaggler now. That all sounds good, but Kaggle doesn't, Kaggle prepares the data for you. They curate all the results and it's in a controlled environment. But in the real world, you have a lot of people who don't fully understand these things deploying them. What do you do then? Another thing is you hear about Google deploying all these cutting edge things. In reality, a lot of the stuff they put in production, actually, it's a very small amount of their research they actually put in production. And that's true for a lot of the major companies. They may have proof of concepts. They may have one team deploying it, but for a company-wide like touch the whole billion trillion users thing, that's actually research from five years ago.
Adam Gibson:
These things take time to productionize because you have to sort out the bias. You have to sort out how robust it is. You have to make it scale. You have to think about how it will break. And that's the key. Basically, you have to kind of there's this whole process on productionizing. That's why MLOps is a big buzzword now because people are figuring out this is a thing. What do I do about it? But even then, people still are very ad hoc about it. So yeah, to summarize, it's dumb. It's a software asset that needs to be maintained and the real world changes. And it's all based on the data.
Gaël Blondelle:
Yeah, I have one question. So previously you said that what you do at conduit and with Deeplearning4j is to fix people's problem, real problem, and you care a lot about the user experience. So what other advice would you give to companies who want to do open source to succeed in open source and to do business in open source?
Adam Gibson:
So it's find a project that has a neutral governance, a liberal license, so not something where you have to give away your work. That way you can commercially use it while contributing back the parts that don't matter as much, because that's what open source is about. It's about reducing your R&D costs. It's about putting out things for people to use and collaborating on the difficult bits. So find something that's neutral, get involved, and then just understand that open source, it's just like any other code. It can break. It's still an asset, and it produces models, which gives you another problem. So that's also open source. That may have bugs. Understand it, understand the code you're using. Open source software is still software.
Adam Gibson:
So get involved, understand what it is. And if you're going to adopt a dependency, don't just use it and then hope it works. And then all of a sudden later find out things broke, and then raise an issue on GitHub complaining to the guy who gave you the code for free saying, "Oh, I need to deploy tomorrow. And I noticed this isn't working. What do I do?" People don't put enough, that's a big problem. It's not their fault. They're not really, most people aren't malicious. They're just under a deadline and they didn't do their diligence. And they didn't think about their problem. They didn't think their way through a problem. So ope source maintainers will do their best, so find people who are good at managing that and can kind of, and are willing to walk you through it and engage with you when you're stressed. Because what I've found is that a lot of users are distressed. They're under tight deadlines. They're like, "What do I do? I have a bug, give me a workaround or whatever." So if you understand open source, these things won't happen.
Thabang Mashologu:
Yeah, that's a great advice. We run a number of surveys at the Eclipse Foundation, arguably one of the largest technical surveys for IOT and Edge. And for a few years now, AI has been the number one workload at the Edge. So what does the Edge mean to you in terms of your strategy and the evolution of DL4J?
Adam Gibson:
I would say it's critical. I mean, a lot of things run on Android phones. A lot of things run on Raspberry Pis. A lot of things even run on like Jetson Nanos and some of these other things. And so what I would say is that a lot of models will be deployed to the Edge and increasingly, the software for it will need to be more robust. One of the things you find is that the tool chain for embedded is not as good. The software people use to build embedded use cases is not as good and not as robust as what you find in the cloud with all the CICD, the Kubernetes, and all the DevOps stuff. That stuff isn't as established and embedded. And so, with all the security problems you see Edge having, being able to do an on-device compute to comply with security, being able to deploy these models efficiently.
Adam Gibson:
It's very core to what we do. That's actually why we're, one of the things that people need to realize is that most use cases right now where Edge is good is mainly vision. Everything else works sort of. It's sort of being used in speech, if you look at the Alexa smart homes and then all these other things, and in general, one thing to understand about the Edge, especially with AI workloads is that some stuff's online. Some models are just too big to put online or to put on a device. Some people do things on device. So federated learning is a great example of this. We built like a DL4J thing a few years, a community member did three, four years ago for federated learning based on Android devices, because it was just being introduced at the time.
Adam Gibson:
And so we've seen Edge in all sorts of incarnations over the years. Where we're seeing it now as being primarily used in factories, healthcare, I don't know. The use cases are everywhere. The bulk of its vision and people are starting to do other things and it will reduce costs mid to long-term it's pushing us to reduce the cost of running machine learning. One of the problems though, is that a lot of, I would say deep learning workloads, especially anything with GPUs, too power intensive to be applicable to anything outside of R&D. It's hard to run at large scale because of the sheer wattage required. So the Edge is doing the work. It's improving. The tools need to be worked on and it will increase, it will decrease power usage and increase privacy and a lot more scale long term. So there's some work to do, but it will enable a lot of things. And I think we'll play a role in that as well.
Thabang Mashologu:
Yeah, absolutely. That makes a lot of sense. Now, Adam, while we still have you, are there any words of wisdom that you'd like to share with our listeners around your experience, your journey, both as an open source technologist and as a leader and entrepreneur?
Adam Gibson:
What can I say? Actually, I would say understand more. I was lucky. What I've learned is that a lot of people don't know what they want to do. And so for me, I'm really glad I took the path I did. It was rough, but overall I wouldn't, I don't think I'd change a thing, but again, I was lucky in that respect. I learned a hard lesson. I didn't do well in my first programming class, because I was just playing games at the time. And the real life kind of hit me in the face and said, "Oh, you know, you got to work now." It's like, "Oh, okay, let me do that." And so for me, after that, I hunkered down and I really got into what I did, because one of the things that I never wanted to do, I never wanted to work again.
Adam Gibson:
I want to exist, have fun, wake up and not regret it. And that's largely what I've been able to do because I enjoy what I do every day just by getting up. And I would ask anybody to strive for that. So if you don't know what you're doing and you don't know what you want to do, prioritize figuring it out if you can, and getting paid to have fun should be the ultimate goal in life. Wake up, do one fun thing and then do another fun thing. That's how life should be. That's partially what led me to open source in the first place because I enjoyed working with and learning from different people.
Thabang Mashologu:
Yeah. Yeah, for sure. I think those are some inspiring words and a great place for us to wrap up. So thank you so much, Adam, for coming on the show, it's been a really great conversation.
Thabang Mashologu:
Well, that's a wrap on this episode of the Entrepreneurial Open Source podcast. You can find us online at entrepreneurialopensource.com and @OSS4Biz on Twitter. We'll catch you next time on Spotify, Apple Podcasts, or any other place you listen to great podcasts. Thanks for listening.
Thabang Mashologu:
The Entrepreneurial Open Source podcast is brought to you by Eclipse Con 2021. Eclipse Con is the leading conference for developers, architects, and open source business leaders to learn about the Eclipse Foundation's technologies share best practices and to collaborate. Join us for this global virtual event from October 25th to the 28th. Registration is open now. Check out the schedule and reserve your spot at eclipsecon.org/2021. That's eclipsecon.org/2021.