What's Up with Tech?
Tech Transformation with Evan Kirstel: A podcast exploring the latest trends and innovations in the tech industry, and how businesses can leverage them for growth, diving into the world of B2B, discussing strategies, trends, and sharing insights from industry leaders!
With over three decades in telecom and IT, I've mastered the art of transforming social media into a dynamic platform for audience engagement, community building, and establishing thought leadership. My approach isn't about personal brand promotion but about delivering educational and informative content to cultivate a sustainable, long-term business presence. I am the leading content creator in areas like Enterprise AI, UCaaS, CPaaS, CCaaS, Cloud, Telecom, 5G and more!
What's Up with Tech?
Open Source AI Governance: Balancing Innovation with Enterprise Security
Interested in being a guest? Email us at admin@evankirstel.com
The evolution of Python from scientific computing tool to AI powerhouse forms the fascinating backdrop of our conversation with Peter Wang from Anaconda. What began in 2012 as a bet that Python would transform business data analytics has blossomed into something far more profound, with Anaconda now at the forefront of enterprise AI implementation.
Peter shares the origin story of Anaconda, founded when he and co-founder recognized that Python's scientific tools were ready to cross into mainstream business applications. While they correctly predicted Python would become essential for data science and machine learning, they couldn't have foreseen how it would eventually power today's AI revolution through transformers and diffusion models.
The conversation explores Anaconda's new AI platform, which bridges the gap between practitioner freedom and enterprise governance. This balance is increasingly crucial as security threats against open source ecosystems grow more sophisticated and regulators demand greater accountability. For AI systems specifically, proper security isn't optional — it's fundamental given their potential impact and vulnerability to attacks like prompt injection.
We also examine how enterprise AI implementation has matured beyond chasing the latest techniques (from massive context windows to RAG to agentic workflows). Organizations now understand that successful deployment requires meticulous attention to evaluation frameworks and domain-specific considerations. As Peter notes, "There's an easy mode to deceiving yourself that you're doing something interesting. To do actually something correct is not going to be an easy mode."
The discussion concludes with a compelling argument that while closed source AI models may currently dominate headlines, the fundamentals of AI technology point toward an inevitable shift to greater transparency. With AI increasingly embedded in critical systems from healthcare to autonomous vehicles, the need for accountability will drive adoption of more open approaches that can demonstrate safety and establish clear liability chains.
More at https://linktr.ee/EvanKirstel
Hey everybody, fascinating chat today as we dive into the world of enterprise AI through open source, with a true innovator and expert in the field at Anaconda AI. Peter, how are you?
Speaker 2:I'm great, great Glad to be here.
Speaker 1:Well, thanks for being here, Really intrigued by the work you and your team are doing. Before that, maybe introductions about your background and mission, and how did you end up at Anaconda? What drew you to this very exciting space?
Speaker 2:Yeah, it's a very interesting story. I didn't set out this way. I was a consultant using Python and its early stage scientific tools back in the aughts, and we started seeing these tools getting used more and more in industry, beyond just science, and my co-founder, travis Ollivant, and I had the bright idea that maybe these scientific tools are ready to cross over to the mainstream of business data analytics, and so we started Anaconda at the same time that we started a general community movement pushing the use of Python for data science and the open data science movement around Python. We really were seminal in creating, leading that in the 2012 timeframe and, of course, as the years have rolled forward, we've seen that our hunch was correct. Many people do enjoy using Python for machine learning, for data analysis, and now, of course, it's become the language for AI and we're very, very happy to see its massive success.
Speaker 1:Well, let's talk about that. Take a little walk down memory lane. Python has been around forever, it seems, but it's now powering the AI boom. Did you see that coming when Anaconda got started? How did that evolve over time?
Speaker 2:It's funny because the answer to that question is both emphatically yes and like totally no.
Speaker 2:We absolutely had a very, very deep conviction that, coming off the big data sort of boom right in the early 2010s big data and cloud computing a lot more people are going to want to do a lot more things with their data besides just a SQL database query and they needed tools to do a lot more things with their data besides just a SQL database query and they needed tools to do that that were.
Speaker 2:You know, python was a great tool for doing that. R was also around, but Python, I think, had some advantages and so we utterly believed that Python could be the language for data science, data analysis, machine learning at scale. So on that front, we were yeah, I think we were right and we definitely saw that coming. But, on the other hand, to be able to see that diffusion models and transformers and all these things would lead to this like massive portal to God knows what right that we did not see coming. I don't think anyone at that time could have seen it coming, necessarily because AI was still in kind of the AI winter. Deep learning was starting to show some early traction and it wasn't until the late 20 teens, when it was obvious that the AI revolution was here and Python was the language that was preferred by the researchers and by practitioners.
Speaker 1:Got it and fast forward. You just launched the Anaconda AI platform, so congrats on that.
Speaker 2:Thank you.
Speaker 1:So for the viewers, what is it exactly? What makes it different from other AI or data platforms out there in the market?
Speaker 2:Yeah, we've always been out meeting users where they are right, and so I think there's a lot of energy around this space, a lot of people trying to do a lot of various kinds of things. For us, it's all about, okay, what are people actually doing, right? A lot of the data exploration, a lot of data transformation. All that stuff is still a precursor to doing AI at scale, and so you still need the classic, I would say, python, data ML, engineering, data engineering kinds of tools. But in addition to that, you also need a few other things, right, and that includes things like model management. That includes things like, hey, how do I govern all these open source models that people are releasing, quantizations, fine tunes.
Speaker 2:I want to pull all of that into an enterprise ready platform, and so the practitioners have an easy place to collaborate, to share their work, to sort of party on the data and party on the models.
Speaker 2:But enterprises have real concerns about governance, compliance, reproducibility, all of these things, and the Anaconda AI platform is a place to bring both of these things together. So, as essentially an outgrowth and extension of what we've always done with our data science platform, we make it easy for practitioners to continue using the tools they want to use we're not super opinionated about. Do you want to use Jupyter Notebook? Do you want to use VS Code? Whatever front ends and whatever cloud we connect to all the clouds, we work on-prem and all these kinds of hard security and regulated environments, but at the same time, when you start bringing in your AI models and you start building workloads around them, easily deploying them and giving administrators and IT folks a single pane of glass to see in what ways do we have a security vulnerability and what ways might we have some exposure here and there, those are the kinds of things that we've wrapped up into a single pane of glass, so to speak.
Speaker 1:Well done. You mentioned security. Security always comes up in open source discussion. What makes you think it makes some companies or enterprises nervous, and how are you and the rest of the industry tackling that?
Speaker 2:Yeah, the topic of open source security in just traditional straight up software development is becoming more and more a center of focus, right Because of the success of adoption of open source, I would say. But we've also seen some really new kinds of like very audacious attacks deep two year, three year, like sleeper cell kind of attacks on the supply chain, which is incredible. The LibExZ attack that happened last year but that came to light I guess it happened for years and then it came to light last year. That was a very deep state-level actor attacking the very nature and the fabric of what makes open source work. The trust model between open source collaborators was under attack. So we know that enterprise is built on this stuff. We know that there are red team, black hat adversaries looking to weaponize that adoption.
Speaker 2:So as an industry we have to get more serious about this adoption and it's not just because we want to. There are regulatory things coming down from like Europe and also executive orders and guidances from NIST here in the United States. People have got to get serious about the software supply chain for open source software security. And now that is one thing, and when you look at AI, that's a whole additional set of complications on top of that, because all this AI stuff is built on top of and to some extent uses this open source software stuff, but it introduces its own new set of complexities and security vulnerabilities. Right, a lot of it. Right now.
Speaker 2:When we look at real world, like no kidding, who's really doing AI stuff for real? For all the talk of agents and all the hype around the VC space and this stuff, what are people really doing? And when you look at the actual data people are really having, they're struggling to get these things working in production for real, not only because of just the challenges of the technology itself hallucinations and just efficacy and general reproducibility. So figuring out that is hard. But then, additionally, we're seeing people already again Black Hat Red Team they're attacking those kinds of open models.
Speaker 2:People using tools. They're trying to jailbreak the system prompts. These things are vulnerable, sort of like from the get-go. And so if you're an enterprise that is excited about the opportunity as you should be, because I think it's massive you also have to understand that security goes part and parcel with the development of this. This is not like an open source development. Oh, we'd like to secure our Python or Nodejs packages. Wouldn't that be nice if we were checking all the boxes With the AI stuff. It is not optional. The guardrails are just not optional given the scale and scope and complexity of what is happening there and what that tooling actually represents.
Speaker 1:Well said, Things are certainly moving fast and month to month, week to week. Are you seeing any major shifts in how companies are building AI solutions today, their approaches? I mean what's working, what's still broken.
Speaker 2:Yeah, you're right, it changes week to week sometimes, but mostly it's still a month to month cadence. Right, we had for a while a lot of people just trying to get bigger and bigger contacts so they could one-shot everything. Then they realized that kind of falls apart. It's kind of like, you know, hit or miss. And so then coupling these systems up with RAG, rag became all the rage, right. And then now it's agents and agentic workflows, and now it's all these things combined together. Right, can they use tools? Can we have, can we do chain of thought? So we're doing inference, time scaling. All of these things are now all coming together. So that's my view of like whatever the last 18 months in like 30 seconds.
Speaker 2:But I think what's nice is that it feels to me the vibe seems to be that people are starting to ask the hard questions, like if we want to use this as a reproducible engineering discipline we build things that work, we know they work, we turn them on tomorrow, they work again tomorrow Like, if we want to build that on top of these stochastic and probabilistic sort of components, these squishy, soft components, how do we actually do that? And so more and more of the conversations I feel like I'm having with people. They are looking at that problem and not having wishful thinking, not just being like, okay, well, some new paper, at least next week, is going to solve it all for us. I think people have kind of given up on that a little bit. People are starting to understand to actually deploy LLM AI technology at an enterprise grade level, you have to be extremely thoughtful about each piece of it.
Speaker 2:You have to do the evaluations. All the magic is actually in the evals and whether you build your own framework, whether you use one of the existing ones that are out there, there is no silver bullet. I think the sort of the evaporation of the silver bullet might be the biggest vibe shift over the last year. That it's not just RAG, it's not just agents, it's not just chain of thought. It's a lot of these things being put together thoughtfully and then a thoughtful enterprise-specific domain and problem-specific set of evaluations. There's not a shortcut to it. So I think that's kind of where I'm glad to see the industry conversation maturing around that, because that, I think, is the high integrity thing to do.
Speaker 1:Oh, such a great insight and as you look for opportunities for improvement in the AI developer workflow right now, obviously you're focused on tooling. Where do we need improvement? If you were to do a SWOT analysis, Is it tooling, of course, but process mindset, training, other things.
Speaker 2:Yes to all of them. Sorry, that's sort of a cop-out answer, but that is it to all of them. Sorry, that's sort of a cop-out answer, but that is. It's all of these.
Speaker 2:I guess the thing is what I would, maybe as a metaphor we use metaphor, right, if we're moving from like the 1920s and 30s and you've got cars and we know how to drive cars and there's some safety that you need to put around cars, that's maybe traditional software development and then we move to jet airplanes that go like Mach 0.9, 0.95, you have to do all the things, but better, right, your manufacturing tolerances have to get lower, your pilot training has to get better.
Speaker 2:The infrastructure you build, the runways have to be much, much smoother than just some crappy little country road, right, the tires have to be high quality rubber or they explode when you land, like all these things. It's, unfortunately, yes, but if you get all that right, wow, you can hurl 300 people through the air at the speed of sound, right, so there is a benefit to it. But it's not just going to be like just turn the crank and now it's easy. I don't think there's an easy mode. There's an easy mode to deceiving yourself that you're doing something interesting, to do actually something correct, is not going to be an easy mode. It requires upgrading all of these things and actually the hardest thing isn't the technology, the tooling, it is, I think, the mindsets. I think it's setting executive and other stakeholder expectation and that's something on us as an industry to sort of level set right. And that's where conversations like this, I think, hopefully can have some kind of impact where people can say okay, everyone else is also struggling.
Speaker 1:It's not just me. We don't really just suck uniquely. Everyone's struggling with this right Fantastic thought there. So there's a lot of discussion, controversy occasionally, around AI models built behind closed doors.
Speaker 2:Do you?
Speaker 1:think the closed source, if you call it that AI approach is a threat to innovation, or is this just the way business gets done?
Speaker 2:Do I have permission to speak freely here?
Speaker 1:It's up to you and your team. I'm happy.
Speaker 2:Yes, no, I think. Look it's. I think it's going to follow the same cycle of technology innovation and adoption as anything else. Right that? Certainly, for people to garner the investment and to sustain investment enthusiasm, they have to tell a story about some proprietary advantage. And when you look at AI, well, it's either the data or it's the model, or it's your people that know how to train the data into a model. But the algorithms are all being published in papers that are widely available. The hardware you're doing it on it's the same gpus that jensen is selling to everybody else, so it's not like you got a lock on the architecture of the hardware. Now google does have their own hardware. There's a lot of people making custom hardware, but for the most part those those are cost benefits. Those are not like a quantum leap in capability of what an LLM does or what an AML does.
Speaker 2:So the question is, if you didn't say that I have something closed source that's opaque and that's of special value, you'd have some hard conversations from the investor side of the world. So I think there is some motivated reasoning there. But if you think about this from the point of view of just a machine learning expert or someone who's thinking about these things at a technical level. There's really no magic or secret sauce in this. There are techniques, of course, to get the most efficacy. When you're training to get it to converge loss curves, I get it. There's definitely real skill there. But is it a hundred billion dollars of market cap worth of skill? I don't think so, right Cause we've seen many teams frontier teams in the world are always trading off the pole position and the leading position for their models, which means that the smart people that open AI has. Well, anthropics got smart people too, as does Gemini, as does Baidu Everyone's got smart people. So the thing is okay in that case, even if we say the closed source stuff has some special sauce, it looks like lots of people have the special sauce.
Speaker 2:And when we think about okay, where does the data come from? Well, a lot of it is scraped off the open internet. A lot of it is public data. A lot of it is books and other things. Some of them are in the public domain. So I think, in long term, that a huge part of the value in an LLM is based on data that is in the commons or publicly available to everyone. So the baseline that should be publicly and generally freely available is going to be quite high.
Speaker 2:I don't think it's like you either get chat, gpt or Lama 5ives or nothing. I think that the what's publicly available in the open isn't pretty high, which means the commoditization pressure is gonna be quite steep. It's gonna be quite, quite high. And I think in the long run, I think the industry will have to trend towards transparency. Otherwise, I mean you see the latest controversy with what's happening with grok right over at XAI, and you know that's not tenable. We cannot have a customer support chatbot for medical care all of a sudden spewing anti-Semitic stuff because a patient's name is like Feinberg. You cannot have that. That's not the world we want to live in, right? So we have to have transparency, we have to demand accountability in how this technology gets built.
Speaker 1:Amazing. Well, that's quite a mic drop moment for this discussion, but I do have one more question. I mean looking ahead. You're such an innovator. Where do you see the open source AI movement headed over the next couple of years and what's your role in it?
Speaker 2:Yeah. So I think that it will become more apparent that the open source and transparent, public and commons AI can be done and can be done competitively. I'm personally leading some efforts around that and we'll see, hopefully, more of the announcements around some of that stuff here in the fall. But there's a large number of people nonprofits, governments, the UN, a lot of people who want to see this as a technology that belongs to everyone because it's based on the works of everyone. It's like it's. Why would it belong to everyone?
Speaker 2:So that's something we're going to see that the conversation will have to start really focusing on the supply chain, the data supply chain for these models, and we're going to have to have much more focus on evaluations, on how to engineer safety around these kinds of probabilistic systems, how we're going to ensure when we put these sensors into drones, into autonomous vehicles or household robots and humanoid robots. Those things cannot be opaque. There has to be a liability chain. There has to be a place where societies and governments and regulators come in and say these are acceptable, these are not acceptable, and we'll drive real accountability around that. So I think the role that open source has to play in this is that we can show you can do these things in an open and transparent and accountable way and in that way really, you know, just set the conversation so that there's not by default expectation that these have to be black boxes.
Speaker 1:Such a great insight. So you're in Austin, your team is everywhere. That's right, a little bit of a quiet period here in the summer, but where can people meet you virtually in person? Any travel or events or meetups or otherwise in the next few weeks or months?
Speaker 2:Yeah, we've gone through the summer spate of conferences. I'll be at the AI4 conference in Vegas at the beginning or the middle of August. The Anaconda folks are around at a number of different kinds of conferences and coming up in the fall. You know we'll be. All the major industry AI conferences will plan to be there, so just look for the Anaconda booth. Come by, talk to us. Love to talk to people about our AI platform, how we help enterprises do AI in a responsible, governed way, and I'm also happy to chat with anyone who wants to talk more about open source and open source AI.
Speaker 1:A wonderful mission. Congratulations on the success onwards and upwards. Thanks Peter, thank you, evan, and thanks everyone for listening and watching. And be sure to check out our new TV show, techimpacttv, now on Fox, business and Bloomberg. Take care.