AI Makes Security Everyone's Problem—with Tim Olshansky (Fencer)

If AI agents are writing your code, how are you making sure it's secure?

In this episode of Futureproof, Prakash Chandran sits down with Tim Olshansky, CTO and co-founder of Fencer, to explore what application security really looks like in a world where AI writes most of the code and open source software underpins everything. Tim shares his journey from engineer—navigating bureaucratic security processes at larger organizations—to building a platform that makes security accessible for companies under 200 employees. Together, they unpack why compliance certifications often create a false sense of security, how the open source supply chain has become a prime target for attackers, and what "trust but verify" means when Claude is opening your pull requests. They also discuss practical steps any builder can take today—from package manager hygiene to cooldown periods—and why hiring for engineering talent has never been harder to figure out.

Topics covered include:

• Security as hygiene, not a project: Why treating security like brushing your teeth—small, consistent habits—prevents catastrophic outcomes, and why most small companies still skip it.
• The open source supply chain is under attack: How threat actors exploit volunteer-maintained libraries like Axios to gain access to thousands of commercial products at once—and why it's only getting worse.
• AI-generated code and the false sense of security: Why LLMs trained on publicly available code don't encode the highest corporate security standards, and why the code itself may not be what gets you hacked.
• Trust but verify in an AI-first workflow: How Tim's team moved to nearly 100% AI-driven development while still requiring human review.

Episode ID: 19061977-ai-makes-security-everyone-s-problem-with-tim-olshansky-fencer

Subscribe to Futureproof wherever you get your podcasts.

From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

Transcript

SPEAKER_01 0:00

If an AI agent is going and writing code for you, do you have any idea whether it's secure or not? Or are you just relying on the same model that went and created the code to tell you that it's secure?

SPEAKER_00 0:23

Welcome to Future Proof. I'm Prakash Chandran, CEO of Xano. Today I'm joined by Tim Olshansky, co-founder of Fencer, a security platform that replaces the patchwork of scanners, monitors, and spreadsheets most startups cobble together. I know from personal experience, and actually fixes what it finds. Tim is a serial founder and engineering leader who's been CTO of a publicly traded company, scaled engineering teams to over 200, and built products for brands like Chipotle and Burger King, as well as the DoD and three letter agencies that will go unspecified. He's seen firsthand what happens when compliance becomes a prerequisite for growth and what breaks when teams skip the security foundation to ship faster. There's a question Tim's been asking that I think every AI builder needs to hear, and that is who approved this change? As agents start writing code, opening PRs, and sometimes merging changes on their own, the chain of human accountability that every compliance framework was built around is quietly breaking. This is a conversation about what that means and what to do about it. Tim, thank you so much for being here. It's really great to have you. Hi, Prakash, and hi everybody. It's uh it's nice to be here. It's been a it's been a while. It has been a while. And fun fact, if I may mention, you are an investor uh in Xano. Just kind of thank you so much for not only your belief in us, but it also just kind of goes to show like the technologists that you are not only contributing to the space, but investing in platforms like ours. Yeah, I love what you're building. Sorry. It was it wasn't a hard choice. Awesome. So I wanted to start with a little bit of background. So, you know, you've co-founded, as I mentioned in the intro, you know, multiple companies, been the CTO of a public company, scaled engineering teams, and then you decided to build something around security, a security platform for startups. So I'm curious like what you saw that made you feel like the security problem was worth solving.

SPEAKER_01 2:22

Yeah, it's uh poignant, I think, because everybody is now thinking about building their own software and building their own products. And so um, hopefully, security is something that people are thinking about a lot more. But over the course of my career, it was one of those things that I became responsible for as the CTO, technical co-founder, head of engineering, uh, software engineer on a team that I often felt was not being done well at the earlier stages. And then when you know I was a part of larger organizations, the level of bureaucracy and difficulty it was to be able to use these tools and use them well was just crazy. So we we we set out on this mission um to build something effectively to solve our own problem. Uh, myself and my co-founder had felt the pain uh and we wanted to make this easier. I had, you know, obviously had to go and implement a lot of these things myself uh in the the pre fencer days. And so I think we had a great idea for what we wanted to do. And then at the same time, uh knowing the pain ourselves, we'd spoken to lots of different companies. I actually spoke to Sean at Zano and and and others uh across the industry about some of their challenges in the early days, and same message. Like this is this is difficult. Uh so we we saw that problem combined with the fact that AI was writing a lot of code, knowing that more code is going to be written in the future, more products are gonna get built. This felt like a great space to build in.

SPEAKER_00 3:49

Yeah, that makes a lot of sense. In in the conversations that you've been having, I feel like you started or you kind of saw early on that this security gap was something even before AI and AI building has had taken off, like it really, really has today. Um, it only feels actually now like that that has been exacerbated. You know, like there's so much that people rely on now uh in terms of AI building. And I know that one of the in doing research for this, one of the things that you wrote is, you know, AI tools are great for building features, but they're just terrible at security. I wonder if you could just speak to that.

SPEAKER_01 4:23

Yeah. Um, you know, it's funny, just as like a little bit of history, you know, 10 years ago, if not more, uh, it was pretty rare to see a company less than 50 employees go through the SOC2 certification process. Uh and a lot of new platforms got built, you know, Vanta, Drata, others uh that I'm sure folks that are listening, watching have probably heard of, um, have made that easier. So everyone is going and getting these compliance certifications that come with security requirements, who are perhaps selling SaaS products uh or just want to have this checkbox against their business. Uh they're able to get them at a much earlier stage, but they're not actually doing half the things that they're saying or promising that they're going to be doing. And so that's that's like the kind of industry has pushed this uh compliance standard to people, compliance certification to people at earlier and earlier stages, but they aren't themselves equipped with the processes and understanding of what to go do, they just delegate to some tool to go figure that out for them. And so that's that's a big part of what we're trying to help with is actually make security accessible and something that you can actually accomplish. And so when you say that you have uh you know code review processes and approval processes and all these other things, do you really? Do you know what's actually going on? Uh are you just checking the box if uh if an AI agent is going and writing code for you uh and it produces a pull request or a merge request or you know, some sort of change set for you to look at. Do you have any idea whether it's secure or not? Or are you just relying on the same model that went and created the code to tell you that it's secure? It's like asking a junior engineer to tell you whether they think the code they wrote is secure. And so that's that's kind of the premise, right? The models are great, but and they're only getting better. But even still, like you still want a second set of eyes on many changes as much as possible.

SPEAKER_00 6:22

Yeah, I think also no matter how good the models get, I think that security, compliance, and how it relates to your business and the rules around your business will always be something that needs to that needs oversight. Um, so I would love for you to talk a little bit about Fencer in the context of actually security and compliance and how people should be thinking about it. Who are the types of people or what types of applications need to start thinking about security very early on, and then introduce a little bit about what fencer does.

SPEAKER_01 6:53

Yeah. You know, it's timing's interesting because if uh if mythos and all these new LLMs and models and everything else get released to the public in the next week and all of a sudden everything's hacked all the time, and then you know, we all need to be dealing with security. I don't know what our banks are gonna do, you know, uh doom and gloom, etc. Although I think it's a good thing because all of these bugs that have been there forever will finally get fixed. But um, you know, really what we set out to do is to make this easier for companies or businesses that don't have a security team, right? And and probably don't even have a head of security. It's some individual that spends maybe 20% of their time per month maybe uh dealing with security issues. And that usually means companies with less than 200 employees, right? Um often smaller than that, less than 20 employees. And that's the sweet spot, I think, to go and set up a good security foundation. Uh, and that's uh you know, a number of different things that I could go on for hours, but it ranges from just having decent uh awareness of where you may have security holes with scanning tools and and and what have you, uh doing penetration testing if you're building software, uh, and if you're not building software for external consumption, then at least being aware of where you may have holes uh for your own internal consumption is pr is pretty useful. Uh because while you may not sell software, the business that you're in may be a great target for an opportunistic hacker in Eastern Europe or somewhere else, um, not just a nation state. And uh and the things that you do at that stage are pretty straightforward uh and not that expensive. Kind of like brushing your teeth, right? It's good hygiene. You do it a little bit if you do it every day for two minutes. You don't have to go to the dentist. It's kind of a similar premise, uh, albeit the consequences are probably far worse than a filling, um, if you get it wrong.

SPEAKER_00 8:50

Right. And so I'm curious, like in the context of fencer, like I mean, that totally makes sense in terms of like how you need to be thinking about it, especially in the in the world that we're living in now with all these new releases. Um, when you started thinking about fencer, when we talk about security and compliance, that's a really broad surface area. Yeah. So, how do you think about compartmentalizing the different parts of security and how people can and how people can, for example, leverage a tool like Fencer to start becoming secure from day from day one, really?

SPEAKER_01 9:28

So there's there's a lot of different areas that need to be secure uh across a business, right? And while we can't personally cover all of them uh as a business, there's lots of other great players in the space. The way that we generally think about it is for a company in that kind of category that I described, 200 or fewer, uh, you have software that you write, which means code. You have infrastructure that you run on, right? Whether your own, I mean, I don't see many companies going and buying racks and putting them in a data center, uh, they're more likely to be building on top of a platform like Xano, right? And so they're they're building their tools and applications on on top of others. Um they they have networks that they're responsible for, and they may not realize it. You know, just because you're building on top of a service like Xano, you know, implicitly there is a network that your systems communicate with that you need to be aware of. You have your own user accounts. You know, if you're part of a company, you're there's gonna be lots of different identities that are interacting, and not just humans, you've got agents, you've got applications, you've got bots, uh, et cetera. Um and and beyond just being aware of those uh perhaps security holes or misconfigurations, uh, you also need the observability or the monitoring of those things, right? So ingesting or collecting events from lots of different data sources to help you flag and identify when you know somebody has gone and used some automated tool against your app or against your system to see if any of those uh if those holes exist. And and I think these are sort of like baseline things. And I haven't even touched on your own laptop as a potential issue or your phone or something like that, right? Because that as well is uh is another common source of uh security issue. You know, the the most common place where companies get hacked from is just somebody messed up with their uh username and password. They reused a password on a some SaaS site that wasn't doing a good job, and now their password has been exposed, or they're using a common password that someone was able to brute force. So I think it's nearly a quarter of all hacks start with uh somebody's bad password or bad username and password. And so identity is a big piece of it, uh, and we help monitor that. Um and and so there's you know those big areas that I mentioned, the five or six that I mentioned, that we sort of have pulled together into one platform and give you much greater control and visibility. Uh and then if you're the type of business that needs to prove to others that you're secure, we then automate all of the evidence collection and all of the bits and pieces to go and show people, hey, we actually take security seriously. Here are all the things we're doing, here are the things we found, and here are the things that we fixed. Not to mention helping you fix it.

SPEAKER_00 12:19

Yeah, it's really it's really interesting because I feel like um a lot of the other tools uh in the land or compliance tools in the landscape that you mentioned are very much like checklist driven. Like you need to do all of these things to get this certification. Where I think um what's what I've always liked about Fencer, and I think really the way that people need to be approaching building applications today is a very like um behavioral or security first. Like you're doing the the tool is actually helping you remain secure and giving you auditability and observability over thing, and actually the compliance certification is an after-effect of that. That's right. Um so that that that makes a lot of sense to me. You know, you said something interesting around like how there's kind of like a compliance supply chain or they're like a system. And you know, relevant, obviously, we had the uh light LLM supply chain attack. Um, there are other examples of that kind of surface that you're not alone. You don't, when you install one thing, you're not just getting that thing. I was wondering if you could help explain to the audience um, maybe at a higher level, about supply chain risks, uh compliance chain risks, and what they need to be aware of as they're depending on uh some of these foundational, sometimes open source products and and and thereabout.

SPEAKER_01 13:40

Yeah. It's you know, I think um, and I I don't want I don't want to uh make this sound too simple, but just like in the physical world that there's a supply chain of goods, right? Like there's no there's no car today where every individual component gets made by Ford, right, or GM or a single company. Uh there's many different businesses that go and produce these pieces that get brought together to make a car the same way software is made today. It's just the main difference in the digital world is the delivery mechanism is over the internet, so it's basically lightning fast. Uh and that there are uh selfless people who will happily give up their own time to create software for everybody's benefit and expect no financial reward. And so that's open source software. And so a large, huge portion of uh the internet today and a lot of what we have built all of our software on relies on these open source software projects, which are the un effectively the fundamental components of a lot of the digital supply chain uh that we rely on today. And they're they're interesting because these tools, uh, these libraries and these pieces of software are like our light LLM is an example of Trivi, another recent one, uh that was actually the one that caused Lite LLM to be hacked. And then there was Axios, which is a popular uh JavaScript or TypeScript library uh they're used for building a lot of web applications today and used in a lot of different products. Um they're all open source projects, and so if a hacker gains access to them, they inadvertently gain access to lots of commercial software and lots of commercial products. And so what these threat actors, uh, these hackers have figured out is that they don't need to try and catch a break into these private organizations because they're actually doing something around security. They're now finding that actually the open source, you know, uh effectively volunteer community is a great place to go and uh exploit their limited resources and security gaps to be able to then gain access to many, many more um organizations and take advantage of them. So that's only going to get worse and faster over time. And I think the kind of relating to your previous point about fixing things for people, you know, you might not know about these problems, and you know about them when somebody kind of exploits them and they make a public statement, which is what the uh organization, you know, the hacking group, uh did. They tweeted about it and showed everyone how great they are. Um what's important now is that as soon as you find it, you fix it. Because not everyone can be exploited instantly. And so if you are the lucky many who aren't being regularly exploited, uh then as soon as you know, being able to respond and react and fix those things is really, really important. I think that's sort of the future. As we get better at finding these things, we will actually lag in our ability to fix these things, and that's always been the biggest problem. Uh still is the biggest problem, I think will be the biggest problem for a while. Like, how can you quickly go and upgrade your supply chain components uh to be on the safe versions so you yourself aren't at risk?

SPEAKER_00 16:54

Yeah, and that flows through everywhere and proving it, right? So to the compliance piece as well. Totally. And like in I just want to highlight something that you said that I think is really important because as AI tooling and AI building adoption um accelerates and it's been accelerating exponentially, um, so does the dependency surface area. And that dependency surface area, in many cases, people don't even realize the dependencies that they're leveraging, right? Their AI is making the decisions for them. Um there's a lot of dependencies that are installed. And so, you know, it's it's not only just like the security first mentality, is just understanding what is it that you have and also the way that it's being pulled in, right? Is it like the latest? Is it always fetching the latest? What is the secure way to proceed? And I think that's a question that people, number one, they I don't know that they know how to begin. Like real builders today, or if I'm an application development leader uh managing my team, like where do I begin? What is the safe way to do this? Because to say not to leverage some of these open source projects seemed crazy. So, what is the middle ground? Like, how what would you recommend to people thinking about this and they want to do things securely?

SPEAKER_01 18:09

Yeah, it's it's you know, getting started is actually a lot easier now than it was probably a month ago, uh, thanks to some of these uh very public disclosures of uh of these hacks. Yeah, it's it's actually I think um straightforward. You know, we at Fencer have published blogs on the topic, um, just to be a shameless shill uh for a moment. But the I think I'll I'll allow it. You'll allow it. Um, it's use use a package manager, right? I think basically everyone does these days, but you know, use a package manager and don't just use a package manager blindly, but use a package manager with a lock file or some other manifest that you can be explicit in specifying which version of your dependencies should be installed at any moment and which version of your dependencies when you run your application will be present and available. Uh and ensure that you're using a package manager that will effectively create a uh a hash or you know, basically an integrity value, a value with a lot of integrity that you can trust that the underlying software uh that you're importing has not been modified in any way. Um there's in every ecosystem that exists in the Java world, in the JavaScript world, in the uh Python world, Ruby, Go, you name it, there's there's great package managers. Um the that's number one. So use a good package manager with a lock file. Uh use their cooldown feature. Nearly all of them at this point have a way for you to say, hey, don't use the latest thing. Wait. If something is new today when I try to install a new version, uh don't use it unless it's at least seven days old or three days old or whatever. Specify a cooldown. Uh and I generally recommend to people at least two days, uh, but probably beyond five or six days, six days, it's diminishing returns. You know, most uh most open source software, I'm I'd say 95% will have a fix out within two days. Um and so you know if if you're risk-averse like we are, we're a security vendor, we've got it set to a week. Um and the most common time for hacks to occur is a Friday night uh because people are not gonna check some automated process until Monday. And so give yourself some time. Um and then I can obviously keep going on, but that's some of the simplest things that you can go do.

SPEAKER_00 20:39

No, I think that's I think that's really that's really helpful just as a foundation for how people should be thinking about it. So, first and foremost, just understand some of the dependencies that your application, your AI application is generating. And then uh secondly, yeah, use some of those best practices and uh especially that cooldown period. That makes a lot of sense. This world that we're moving into, where agents are, for example, doing more of the work. And one of the things that you have been talking about that teams aren't necessarily confronting is that question that I mentioned up at the top. Who approved this change? Um, I'd love for you to walk us through what breaks in the compliance chain when agents start writing and merging code. Like how do you think about this?

SPEAKER_01 21:21

Yeah, I I can't remember who it was published a blog post recently, you know, big tech company about how something like 30% of their changes are now being reviewed, approved, and merged by AI. And so as as a lot of engineering practices have kind of come from the bigger companies to the smaller companies, I imagine that that's coming for everyone if they're not already doing it in certain cases. Certainly solo builders are probably already doing that, but they're on their own, so that makes sense. Um it's it's a it's an interesting one because general security best practices said that the people who the person who wrote the code should not be the one to approve the code. Right. So you in the pre-AI world, you'd have a teammate take a look at it. And tools like GitHub and GitLab even make that really easy for you. You'd have a pull request, you take a look at the change, looks good. Click a button, it's approved, and you know, merged, and whatever automation you have in place to uh update your systems and servers uh would play out. In AI world, you know, again, I I believe in sort of trust but verify. Uh and if you're if you're confident that the tool is doing the right thing, right, you've let your AI systems run wild and approve and merge things for you. I'm hoping that you have CI in place, continuous integration, uh, where you're constantly running all different kinds of tests and checks to make sure that the changes are safe. Uh and that you're keeping, effectively, you may not realize it, but you're keeping good logs of what happened. So you can see, okay, this change was made, Claude made this change, opened a pull request. All of your checks, the 50 of them hopefully have all ran. Codex went and reviewed the pull request and gave it an A and said that, hey, this is actually a good change. There's no fixes needed, let's approve it, and then it automatically gets merged and deployed. Every one of those gets kept. And these tools already keep this information for you, right? Like every pull request that gets merged is logged in GitHub. Every CI check is logged in GitHub. All of these things are effectively logged for you. So the thing that I think is missing that companies uh should be doing if they're not already doing, is just randomly checking those merged PRs on a regular basis to see, you know, did something actually get missed doing their own effectively post hoc uh code review to verify that that their systems are working correctly. And I think that's the big thing is that as we move into this you know very fast-moving AI AI first world, uh, and we're sort of more as an overseer orchestrator sitting above and watching what they're doing, we've got to we can trust that uh things are working, we've got to verify regularly. And I think that's what's important. It's checking, taking a look, uh, and flagging when things have gone wrong. And when they've gone wrong, that you go and take the time and energy to go fix it uh at a system at systematic level or systemic level so it doesn't happen again. Change the problem.

SPEAKER_00 24:18

Add more CI. I think there's two things that two questions this brings up for me. And I I guess the first, one thing that I'm realizing about the way you're you're talking about this is it's not just a technology problem, actually. It's a it's a it's a people and process problem, right? Like you have to change the way that you operate, uh the way you view security, the way you you know randomly check um the history of uh the PRs, that needs to change in terms of the way that you build. So, you know, at at Fencer, and like when you work with customers, how do you recommend from a process and procedure standpoint that they change out outside of what you just recommended, these random checks, what else needs to change in terms of the way people build software today with AI?

SPEAKER_01 25:05

Yeah. I th I think you know there's a lot of talk about things like harness engineering and all these other sort of very AI-specific mechanisms. But you know, I I think in a broad sense, it's define all of the rules or preconditions a change needs to meet in order to be considered valid or secure or both. And that might just be initially, and that's hopefully a checklist. And then you know, for anyone who's gone through a SOC2 process uh or some sort of compliance certification, they've probably already done that. And then they've gone and created this um this list of pre-approved changes. So under which conditions can something be effectively released to all of our customers and what does it need to show? So go through that exercise and use normal words and language, like as a human, uh, nothing particularly technical. It's like, hey, all of these tests that we wrote need to pass. Uh I need to log in and check every screen in my app to make sure that nothing broke and so forth and so on. Right. So whatever you would do as a as a person, define those things and work on adding more automation around those things so that you can get to that point where things are automatically merged. The second thing is uh making sure you define what the set of things are that do and don't need human oversight. If nothing else, that exercise will help identify for you where you may have gaps or holes, right, in terms of your processes and workflows. And then once you've done that, you know, make sure that you have an actual like monthly review process, weekly review process, quarterly review process, whatever makes sense for your business or your team, uh, and look through all the things that fell into the, hey, this is pure fully automated. Um we will uh go and check them out and make sure that they're uh appropriate, right? That our processes and workflows haven't broken. And that's I think exclusively a people and process problem, right? There's no technology for that.

SPEAKER_00 27:10

Yeah, and I think implicit in all of that is a solid understanding of your business logic and the things that need to be true uh to derive and surface value for your customers. And the reason why I emphasize that point is because uh this kind of gets me to my next question. When you are bytecoding or just kind of letting AI take the wheel, a lot of those decisions are made for you. And it's making a lot of assumptions around, well, I think that this is how it should work. And when you lose oversight around what needs to be true, what needs to be secure, even if the AI is writing technically correct code, I think you are opening your uh yourself up without even knowing it to another attack surface.

SPEAKER_01 27:53

Yeah, 100%. The other thing to consider is that a lot of what the LLMs have been trained on is publicly available code and publicly available processes, typically open source processes, which are not in and of themselves bad, but they're not you know called the highest corporate standards of security. And so what effectively gets encoded in the model that is then used to write code for you is sort of those processes, not what the best companies, you know, with the most secure systems and workflows are doing. And and that's that I think nuance maybe gets lost with folks when they're like, oh yeah, Claude, just go review what I what you changed and tell me if it's secure. You know, it's missing a huge data set that it could have been trained on, that it isn't trained on, that actually would find all those things.

SPEAKER_00 28:43

That's such an important point. Um yeah, so that's the the point I made around vibe coding. I'm I'm kind of curious as to your thoughts around, again, this acceleration of AI software creation and the single player modality of people using vibe coding tools and pushing out applications. Um, because you know, a lot of the things that we've been talking about is like, you know, an application and a process that has CI. They're thinking about those things. But there's a lot of applications being pushed by an individual who doesn't have a lot of experience just leveraging a vibe coding tool to do that. What are some of the dangers that they might encounter or they should be thinking about as they're pushing a vibe coding app to production or having users start to subscribe and use the product?

SPEAKER_01 29:34

Yeah. Um, you know, we again to be shameless, um we have we have tools that help you find a lot of really common things uh that can help you ensure that not just the code, but the other systems that you're built on top of are actually configured correctly. And so what I would say is the code itself may not necessarily be the thing uh that you get wrong. And you know, AI makes it really easy to generate a lot of code, and that code may actually be secure a lot of the time, and that gives a false sense of security. The um recently, it was actually not that recent now, in it's more like a decade in AI years. Uh Maltbook, you you remember Maltbook, uh, that is now gonna be part of Facebook, I guess. It's gonna, we're just gonna be talking to to open claw agents, malt bots, uh, all the time. Um you know what the the the code itself that the team wrote, and it was two guys behind that, by the way, that built um maltbook. The code itself wasn't insecure, it was the way that they were using Superbase was the thing that was insecure. And so then, you know, they got hacked. And you know, the other thing to consider is use platforms that are secure by default, like Xano. That will uh ensure that you don't make that kind of mistake. Uh but I think that's one area that gets overlooked. And I think the second thing is maybe more philosophical. You know, I've I've shifted away from thinking of AI and these model tools uh as being perfect. They're kind of uh maybe now an above-average employee. Right. And so if you were working with other humans, would you assume that everything they say and everything they do is always right and always perfect? No. But we seem to have ascribed that same level of you know precision and accuracy to a tool that we're using. Uh and I think that kind of like change and mindset can go a long way to you know solo vibe coders. You're you're commanding a fleet of digital employees, they're not right all the time. You should check their work and make sure that they're not telling you trash or nonsense.

SPEAKER_00 31:44

Yeah, you know, um it's so funny. I just I sat down with uh Doug Merritt from um from Splunk uh a couple weeks ago, and he said something uh that really hits home and is relevant to what you're saying. He said, as agents become more human in their behavior, humans have become more binary. So we're we're in this moment where the tools are getting smarter, but our thinking is getting shallower, and we're just like instead of that deep synthesis and thinking that humans are great at, we've just been relegated to clicking accept and okay. Yeah. Yeah, yeah, yeah. Yeah. It's just so interesting. And like we have to like infuse humanity back into the build process, right?

SPEAKER_01 32:24

That's right.

SPEAKER_00 32:24

That's it. It's uh uh it's interesting.

SPEAKER_01 32:26

I know if if the way that you interacted with an LLM was through like a video call, uh, where there was some sort of like, you know, call it uh a human-like avatar that you're interacting with, you probably wouldn't trust it. Like you wouldn't take it at face value. But then because you do it through text and Claude has this very authoritative style, or open, you know, open AI Codex GPT has this authoritative style of writing, you're just like, oh yeah, clearly it's right.

SPEAKER_00 32:52

So yes, accept, move on. You know, this is, I mean, maybe this is more philosophical, but one of the things that I've been thinking about is as we have this again of this acceleration of AI writing software, people vibe coding things, we're that means that us as users and as consumers are gonna have just a lot more solutions available at our fingertips. And, you know, it it really feels like, at least for me, I would be looking for some sort of security badge or something that says, like, I've built this in this certain way, because most of the time, the the software is looking for credentials. It asks you to put in your personal information, your address, your your password that then can be exposed. And if it's hacked and if it's not built in the right way, there is a broader ecosystem that is now at risk. How do you think about that? Like as this acceleration of software uh, you know, I guess proliferates throughout the world that's generally AI generated and trained, like you said, on an open source repository. How do you think secure usage looks like in the future as a consumer? Yeah. Uh, you know, there's that's a big one.

SPEAKER_01 34:04

I think there's a lot of different areas that uh get impacted by that change. I think one of them is the underlying supply chain, as we talked about. You know, I think what's possible, maybe probable, is more walled gardens. So instead of this kind of very federated public way of using open source software, what people end up actually doing is buying open source software from vendors that have done the work to make it secure in the first place. There's already a few companies in the world uh starting to do that, you know, where they publish uh hardened Docker images uh so that you can use a piece of software with the belief that this other company has taken the time and effort to make it secure. And so I think that plays out in different layers of the supply chain and how we deliver software to people. I think identity becomes much, much more important. And so there's a there's a draft right now uh in the IETF on how to actually track when agents have been delegated to by a people and how to effectively expose who the person is that the agent has been delegated to. So then you can kind of identify: okay, is this somebody acting on behalf, you know, as an agent, little a agent, the way the word agent is meant to be uh understood. Um is this some sort of rogue piece of software going wild and maybe doing things that it shouldn't, right? So the hacking and whatever else. Trust essentially becomes the core component. And I think what we start looking for more and more, in the same way that deep fakes make you not trust videos anymore or the news, you know, you're gonna trust software less and less. You will, as you said, look for these credentials. I think it would be dangerous for any one vendor to say, hey, we, you know, you can trust us to say that we made this secure. I think you have encryption protocols, crypto actually probably brings some utility here because it distributes trust in this very uh very secure uh manner. And some of these technological things that maybe have not had as much adoption become a uh much more common now.

SPEAKER_00 36:19

I'm curious how you balance that speed with control uh in a world where like it's yes, of course, security is important for a lot of platforms and services, but especially for a security company and platform like Fencer, how do you work responsibly leveraging AI in your day-to-day and with your teams?

SPEAKER_01 36:41

Yeah, uh actually, this has changed literally in the last 30 days. If you if we had been chatting a month ago or longer, you know, what I'm about to say would be quite different. But we're now probably close to 100% AI driven in terms of code and what have you. Uh a month ago, it may have been 40%. So, you know, in the span of a month, we've basically gone to the point where nobody really writes any code by hand. Um still things, you know, we we don't have an AI agent go and drive a browser for us and click buttons on a screen for us yet, but I'm sure that'll come in time. Um But we you know we use tools like Claude and what have you to go and write code for us. Uh I think everyone on the team, even before they open a pull request or something like that, will go review the work themselves. So we haven't yet gone to that point that I think other teams have where they've abdicated responsibility for the reviewing the code themselves. They're just like, okay, it works. Uh I wrote the tests, I wrote a spec. Great, I'm gonna go, you know, throw it over the fence. We don't do that yet. We're still um a bit hand more hands-on there. Uh we've we have probably 25,000 unit tests and I don't know, several thousand end-to-end tests. So we've we've gone pretty wild on writing tests. Uh, and so we have a very extensive test suite, a lot of CI processes and and workflows to kind of encode and enforce our standards as much as we can. And we were doing that just to stop religious debates between engineers. Uh, it just happens that that's paid off for us with with AI agents as well, right? No, no one debates whether we need tabs or spaces, and if Claude goes and adds tabs, like that's just caught and we don't have to worry about it. Um, but we we've done we did a lot of that work pre-AI, uh, and that has now paid off for us. And so when we get to a code review, um we look at humans look at the architecture. We spend less time on the stylistic parts of the code, but we still require all code to go through human review at the moment.

SPEAKER_00 38:43

30 days ago, I would have talked to a Tim that had adopted it 40 or where AI is writing 40% of the code. Now it's 100. And now you're uh at a place where you're still making sure to review everything. You haven't fully abdicated responsibilities. I'm curious for other engineering leaders listening to this who might be at 0% and they're like, look, I've been holding back. What were the gates or goalposts or milestones that made you feel comfortable in increasing, I guess, AI's uh responsibility in writing more and more of your code? And then the final piece to that is when will you feel comfortable abdicating your responsibility because it is writing all of the code for you? Yeah.

SPEAKER_01 39:35

Uh I think firstly, the just the quality of the models improved drastically. So the actual results, it very quickly became clear to me that I didn't need to, you know, closely review every change. Uh and so maybe maybe it's a little bit more than a month ago now. Certainly since November, there was already a change, but even you know, in the last month or two, it's gotten better. Um, the the actual results were very consistent with our quality standards, met you know, the style with which we write code, the the types of patterns that we follow, and things like that, just did a much better job of identifying those things. And you know, pre our extreme adoption, I know that what I was skeptical of was whether an LLM would write code in a way that was consistent with how we wrote it as a team, that would follow the standards or would follow the uh architectural patterns that we had established. You know, the way that you interact with these sorts of things, or the way that you build a feature X over this way is you know, a certain components get wound together versus just write any old sloppy result. Um and I think now that I've seen that it's really good at it, I I think that's what would have held me back a while ago. I think the the larger organizations that have restrained people from using LLMs, I think fear two things. One, you know, is anthropic or open AI just training on their on their code, right? And so they're giving away their secrets, effectively, their core IP, to some third party, which I, you know, you've got to trust the agreements that you sign with companies at some point. Um to put that to one side. I think the one that I acknowledge when people are appropriately skeptical is the security piece. Specifically, will the AI go and find some sort of secret that you've got in an ENV file or a text file or some sort of file on your computer, in your file system, that gives it access to way more than it should, and you don't necessarily see that. Uh and I think the only way to overcome that is to sandbox it better, right? Run it inside of a container. Um because the productivity gains are just undeniable. And I think uh the previous skepticism around hallucination, whatever else, like I don't think is justified at all anymore. Uh and when you tell it not to hallucinate, it doesn't hallucinate, right? So it's uh um uh I I think that that you must, it is a tidal wave, you have to adopt it, and the things that would have made you skeptical one, two, three months ago are no longer there, and now it's just putting the right constraints in place. So spend some time, create a little, you know, isolated Docker box, you know, EC2 instance, whatever you use to allow your team to use it, and just go go for it.

SPEAKER_00 42:36

I love it. Um the final question I had just before we close is how you think about hiring um and building the right team in today's world. Um is that who are the people that you look for? Like what are the qualities, what are the traits? Um, what type of AI capability do they need to have before you consider them, for example, working at a place like Fencer? I don't know if people heard that deep sigh.

SPEAKER_01 43:08

That was a deep sigh. I think this is the single hardest question, at least for me right now, the single hardest question uh in in this new world, more so than these like technical uh things that we've been discussing. I really, I really don't know what to look for right now in hiring. Um, because I think the things that I used to use as good signals of a person's ability and competence and you know success uh are very different in the current world. And I think it's you know, I haven't had many opportunities to test it out. It's not like we've gone and hired a hundred people in the last three months to really refine what we would actually look for uh to find somebody that's successful. A lot of our hiring happened over a year ago um or about a year ago now, and and things have changed drastically in the last 12 months. So I don't know. I I think certain fundamentals are still just as important as they were before. I think certain personality traits are just as important now as they were before. And what motivates people just as important now as it was before. Are people purely intrinsically motivated? Are they only motivated effectively through you know having high compensation or some sort of explicit reward? You know, the the things that uh I would look for is obviously a balance. Um still makes sense, but in terms of assessing a person's ability to get work done, it's very different. And I don't know, I don't have a good answer. It's something I'm still trying to figure out. Um and I, you know, right now, the way I work a lot of the time is here is a feature description that I wrote down. I give it to Claude. Claude creates some code, I look at the code, it makes sense. Kind of how I would have written it. That's good. Like if I tested somebody on their ability to do that, like I don't know that many people would fail. So there's obviously much more to that. Um, and and more importantly, I think now it's about assessing how well someone understands what an LLM is doing, at least in a technology company.

SPEAKER_00 45:13

You're such a uh seasoned engineering leader. You've been at engineering and you've led engineering teams, big and small. And even you are in a place that we are where we're figuring this out. And I actually think that's going to bring a lot of comfort to people that the uncertainty that they feel around who is going to be the highest leverage individual to bring onto your team right now is changing and transforming. And I think like right now it's uh, you know, for us, it's kind of like someone with high agency, intrinsically motivated, and just a voracious appetite to learn. Um, if you have that as a foundation, I mean, I think that at least, you know, gives you like that initial prerequisite to like, I think that you're going to be successful here. But still, I think what we've what we find, especially in engineering, is like the more just fundamental computer science, computer like software engineering principles you foundationally understand, the higher the leverage you are to, for example, leverage something like conductor, use multiple agents uh at the same time, and be productive in a way that someone that doesn't have that muscle memory can't do. That's right. Um but it's still, I mean, I'm I'm with you. And I and I and I'm so glad that you shared that you didn't know the real answer to the question because I think it's something that we're all asking ourselves and trying to figure out. Yeah. I mean, it's if anybody is out there saying that they definitely know, you know, I I don't know that I believe them. People are uh interested in learning more about you, about Fencer, where should they go?

SPEAKER_01 46:49

Yeah, well I'm active on Twitter uh and LinkedIn, uh Timolsch on Twitter, uh Tim Olshansky on LinkedIn, and uh Fencer Fencer Security. Uh you can find us online and uh major social channels.