Full Tech Ahead
On this podcast, I sit down with business leaders, researchers and executives to explore innovative technology solutions and products, whether they’re transforming industries today or still in development. But we go far beyond the tech itself. From real-world use cases and business implementation journeys to cybersecurity challenges and future trends, we uncover what’s shaping the digital landscape.
We also dive into topics that matter to every tech professional: Work/life balance, business communication, education and training. Think of it as your one-stop shop for meaningful technology discussions that inspire and inform.
Full Tech Ahead
Stop Using AI for Everything
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode of "Full Tech Ahead," host Amanda Razani interviews Federico Ramallo, Founder and CEO of Density Labs. They examine the realities of AI engineering, focusing on business outcomes and managing total cost of ownership (TCO) as token consumption surges.
Ramallo addresses why roughly 95% of enterprise AI pilots fail to reach production, citing over-reliance on idealized demo scenarios, uncontrolled complexity, and treating AI purely as a software purchase rather than an organizational transformation.
To deploy non-deterministic AI agents reliably, Density Labs utilizes Eval LLMs, employing an independent secondary model to evaluate the primary model's execution decisions as a "second opinion."
Paradoxically, Ramallo advocates using as little AI as possible: writing deterministic, traditional code for rule-based workflows and reserving costly LLM calls only as a fallback for subjective, intuitive tasks.
Finally, he outlines new cybersecurity threats, including prompt-based phishing designed specifically to deceive AI agents.
Key Quotes
- "Density Labs... what we're doing is AI engineering... We take all the hype away and we focus on business impact outcome... We track the total cost of ownership."
- "Roughly ninety-five percent of enterprise AI pilots never reach production... The main reason is that they test the demo with the best case scenario, and then they don't consider all the use cases."
- "Even though I am advocating for the use of AI, I believe that using as little AI as possible is the best approach."
- "Think of AI as a new hire that doesn't have accountability... no agent, no model can be accountable for anything. Only humans can."
Takeaways
- Code the Deterministic, Model the Intuitive: Do not use AI agents to automate entire business workflows blindly. Build standard, deterministic software code for rule-based mathematical steps (which are cheaper, faster, and easier to test), and restrict non-deterministic AI models to subjective tasks like sentiment analysis as a fallback to optimize token costs.
- Implement "Eval LLMs" for Second Opinions: AI models executing operational actions produce non-deterministic outputs. To prevent autonomous errors from propagating, companies should deploy a separate evaluation model to audit decisions before execution, avoiding the echo-chamber risk of a single model evaluating its own work.
- Code Craftsmanship vs. Disposable Software: AI models struggle with complex architectural abstractions, often generating lower-quality code than senior human engineers. However, software development is shifting: long-term code craftsmanship is becoming less critical as agentic tooling makes regenerating and replacing code faster and cheaper, provided specifications are hyper-detailed.
- Defend Against "Agent Phishing": Threat actors now use autonomous agents capable of bypassing traditional CAPTCHAs and behavioral heuristics. Furthermore, attack vectors have evolved from human-targeted phishing to malicious payloads, links, and emails engineered specifically to trick autonomous AI agents into executing unauthorized actions.
Find Amanda Razani on LinkedIn. https://www.linkedin.com/in/amanda-razani-990a7233/
Follow the FTA LinkedIn Page: https://www.linkedin.com/company/full-tech-ahead/
Visit the FTA website: https://fulltechahead.com/
Check out the Substack Channel: https://fulltechahead.substack.com/
Hello and welcome to Full Tech Ahead. I'm your host, Amanda Razzani, and with me today, I'm excited to have Federico Romalo, founder and CEO of Density Labs. How are you doing today?
SPEAKER_00Hello, Amanda. It's a pleasure to be here today.
SPEAKER_01Happy to have you on the show. Can you share a little bit about your background and what do you do over there at Density Labs?
SPEAKER_00Yeah, sure. So at Density Labs, what we're doing is AI engineering. We partner with companies that are trying to figure out how to leverage AI within the organization. We take all the hype away and we focused on business impact outcome. We track the return of investment of those efforts. The other thing we we do is we track the total cost of ownership. We've seen a lot of projects where the token consumption goes to the roof and then the cost for running these projects goes super high. So we we we work on that.
SPEAKER_01Okay, great. So we're gonna talk today about your experience in working with these companies and what are some of the challenges and the ROI for software development companies when they're utilizing AI tools. I know everybody's trying to harness AI in the best way possible. So let's start with the fact, though, that roughly 95% of enterprise AI pilots never reach production. Why is that, do you think?
SPEAKER_00It's a very interesting situation that is happening with that. The main reason is that they test the demo with the best case scenario, and then they don't consider all the use cases. That's the first issue. The other issue that is happening is that even though AI is a technology tool, it is actually an organizational change. And uh a lot of companies don't see it that way, and that's the the uh underlying reason or the root cause of why uh this type of projects fail. And the the last issue that I've seen is that because you have so many different possibilities of outcomes or use cases, what we start doing is setting up a model that evaluates another model. Uh, that way we reduce those uh that that complexity towards expected outcomes, expected behaviors. And then we set guardrails to to avoid that, right? But basically that that that issue that that that is that is the the core issue is that it becomes unwindling, unwindly complex when you have so many use cases and you don't have a way to track that the outcome is what you actually expect.
SPEAKER_01Can you share a little bit more about you said create a model to review the the model? Can you share more about that?
SPEAKER_00Yes, absolutely. So the first thing we need to understand is that the when we have agents executing and doing actions for us, right? What we are actually doing is we're providing tools to a model, to a large language model that is making those decisions and then executing for us, right? The issue is that these are non-deterministic outcomes, is what we're going to get, right? So, how can we test that the outcome is what we expect? What we do is we build uh it's called Eval LLM, and basically uh what we have is another model, and by model what I mean is uh thinking it of as an engine, right? Uh, it's the core of every um AI automation tool that we use that can that we can have a conversation or we can it can read images, there are different models, but basically what we do is we set up one model to analyze and make a decision to execute, and then another model to evaluate what the first model is going to do, and then make the decision. Okay, does that make sense or it doesn't make sense, right? The reason, technical reason we need to use two models is because what we want is to have uh let's call it second opinions, right? If we if we have the same models evaluating themselves, then we have the higher probability of them to be to agree to a wrong decision.
SPEAKER_01Okay, so when putting AI agents into an engineering org, what are some of the things that break or some of the challenges that you see?
SPEAKER_00Even though I am advocating for uh the use of AI, I believe that using at l as little AI as possible is the best approach. So when uh what are the best scenarios to leverage AI within an organization? So when we have uh business processes, right, um, and we want to automate them, we can use uh an AI agent to just automate the whole workflow, right? Or we can build software, we can build code that is actually going to automate that workflow, right? So what I am advocating is to build code to solve the automation uh for all the deterministic workflows. By deterministic, I mean uh think of math. Math is deterministic, one plus one equals two, right? It's always the same result. So when we have those types of rules, or we can uh set up those types of rules, then we can build code, we can build software that is going to run those automations. But when you have to analyze an email, understand the sentiment within it, or um think of it as intuition, you know, you have a problem and you don't know exactly what the outcome should be. So you have to uh intuitively find the outcome. So for for those problems, AI is great. So what I suggest is to build a software for the deterministic workflows and then use models as fallback only when we have a situation that is non-deterministic, right? That means we can save money on running tokens or on running models, and uh we can have a faster uh automation process and it's uh safer because when you have a model running um within your organization, somebody has to supervise what it's doing, right? Um no agent, no model can be accountable for anything, only humans can. So if we have code, it is easier to manage, it is easier to test and make sure that it works properly, right? And then the models, they can be um supervised, they can be uh tested, but the level of effort is much higher, and the cost of operating them is much higher, right?
SPEAKER_01So you argue that AI agents are not necessarily good junior engineers. Explain that a little bit further. I see a lot of people using AI agents to develop all sorts of things. Where do they encounter problems doing that?
SPEAKER_00Well, what I was explaining before was mostly about automating business processes. Um, what you're talking about, uh asking me now, is about why uh models are not good software engineers. And I think that one of the reasons is that the software engineers, what we do is we abstract, we build abstractions, layers of abstractions of a of a of a problem in real life, right? Um, and then we set uh expectations of behavior. What do we want the application to do? The models are not really good at those levels of abstraction, and they um they lose um they lose that that ability, right, uh of abstracting and understanding what is the core problem. They are becoming better though, but they're not there yet, right? So the quality of the code becomes uh lower than what a senior engineer could do, right? What I called the craftsmanship. Now, what is going on is that once you start building code with agents, now the the code is easier to be replaced. You can build new code easier because you have agents to help you with, right? So even though I I come from that craftsmanship of building high-quality code, now I'm learning that it becomes less relevant when you can just replace it faster, right? Um having said that, the key um uh key tip for successfully build software with agents is uh you need to be very, very specific on the specifications, right? The more specific, the better, because that's when you can do the analysis on how the application should work, and then the agent can focus on executing and building the application, right?
SPEAKER_01So we are seeing the use of AI more and more. What are some of the cybersecurity risks, and how do leaders address that?
SPEAKER_00The security agents can mimic human behavior, right? And that that is becoming an issue. It used to be that you could have captchas to to you know separate bots from humans. Now agents can overcome CAPTCHAS, they can overcome a lot of the behavior of patterns that we used to use to track bots and feed out cyber attacks. Uh, and that complexity is going to increase because the attackers, hackers are also using agents, so they're having they can elaborate, uh, they can build more elaborate uh attacks, right? Um on the other hand, on the on the safety part, what we're seeing is uh more increasingly complex applications that can detect attacks. Because the issue with with attacks is what happens if you have a lot of false positives, right? Then eventually somebody is going to turn off the alarm, right? Because who's going to listen for a tool that is not providing real threats, right? They're building tools to filter that and being being able to double check if it's actually a threat and can make decisions to autonomously uh fix the the attack, right? Or defend for the attack, right? Uh so that's going to be very interesting happening. And the other thing that I've seen happening is we have we used to have phishing for humans, now we have phishing for agents. So we have emails or or links or things like that that uh are target to agents to act on, right? And that that is becoming an issue, right? And the last thing we're going to see, or where we're seeing already, is um a higher volume of both attacks and uh defense actions because both can be automated now.
SPEAKER_01So what tips do you have for business leaders when it comes to building and leading high trust engineering teams across countries and different time zones?
SPEAKER_00The first thing is that building a culture around how to build software, understanding uh or being able to transmit to the software engineers that what they're building has an impact to the users, what is the value for the user becomes the most important question that the developer should ask, right? Once they understand that, then the developers can leverage agents to build the software, right? To build the tests, to test specifically the value for the user, right? To spend additional effort on testing that, right? Because if you do that, then the application you're going to build is going to be uh more useful for the user, right? Um, so that's the first part. Uh on the business process automations, I think that the core suggestion that I can give is think of it as an organizational transformation. It's not only a technology implementation, right? And getting the team to adapt to these new tools, right? There are organizations that uh are capable of those of adapt, uh adapting, and that makes them uh leverage AI faster, and they can have a competitive advantage over other organizations that are still trying to figure that out.
SPEAKER_01Fantastic. Well, if there was one key takeaway you could leave our audience with today, what would that be?
SPEAKER_00Think of AI as a new hire that doesn't have accountability. Use uh or or lean on the context of your team, right? If you have a team with long tenure, lean on that because they know the business, they understand how everything works. So what they they need to think is that the new responsibility of the humans is going to be to manage those agents and to guide them when those agents are making a mistake within the business process automation. It also applies for software, right? But uh, I think that's that's the core takeaway that I can give is lean on that on that team, lean on the humans and help them upskill to use AI as augmentation instead of replacement.
SPEAKER_01Absolutely. Well, thank you so much for coming on the show and sharing your insights.
SPEAKER_00Absolutely. A pleasure to be here, Amanda.
SPEAKER_01And thank you to our audience. If you have any questions or comments, leave them below, and I'll make sure to respond as soon as possible. And have a wonderful day.