Don't Automate the Mess Just Because AI Made It Cheaper

This week on Quanta Bits, I talk through why AI makes an old operations mistake easier to repeat: automating a messy process before asking whether it is consistent, simple, and owned. The episode uses deal review as the example, then connects it to OpenAI and Bain's "micro-productivity trap," Google's legal warning on AI summaries, Anthropic's capability-jump planning, Pew's AI trust data, Stanford's AI practice system for counselors, Addy Osmani's Orchestration Tax, Miro's SQL context-layer lesson, and a short After Hours note on The Hunt.

Read the full issue: https://quanta-bits-newsletter.beehiiv.com/p/quanta-bits-june-20-2026
Subscribe to Quanta Bits: https://quanta-bits-newsletter.beehiiv.com/subscribe

Transcript

SPEAKER_00 0:01

I had this thought this week while watching two different things happen at the same time. On one hand, the World Cup has been everywhere, and I love that part of it. Different countries, different styles, one event sitting in the middle of the day. Nice rhythm to have on. On the other hand, the Frontier model world got oddly quiet. Since Fable was shelved because of government restrictions, OpenAI and Anthropic seem to have moved into a more cautious release mode. Not much new from Cloud Code or Codex lately. And I think that pause is useful. Because when the model news slows down, you can look at the work, not the demo, not the bench bar, the work. Hey, I'm Reza Marakabadi. Welcome back to QuantaBits. This week I wrote about a very old operations mistake that AI has made much easier to repeat. Automating the processes, automating the process that annoys you before asking whether that process deserves to exist in its current form. AI can make a bad process move faster. It can't make it worth scaling. Let me use a very normal example, deal review. A sales team wants an agent to help with deals that need finance review. Maybe the discount is outside policy, maybe the margin is thin. Maybe the payment terms are odd. Maybe legal added a term that finance should see. On paper, that sounds like a great AI use case. Read the opportunity, check the code, flag the risk, draft the finance packet, route the deal. You can almost feel the demo building itself as you say it. Then the actual process shows up. The code says one ARR number, the order form says another. Finance trusts its own spreadsheet more than the CRM field. The customer wants quarterly billing. Legal slipped in determination for convenience clause. Then someone asks whether the approval threshold is based on ARR, ACV, TCV, Net New ARR, or total contract value. This is where the shortcut starts to look expensive. The agent can summarize all of that and route it faster, but the mess does not disappear because the agent can summarize it. And none of this is a new problem. Michael Hammer was warning companies about this in Harvard Business Review back in 1990. His line was basically stop paving the cow paths. That mode should land with anyone who has tried to drive through Boston. The difference now is the entry cost. In 1990, a bad automation project required a budget, a project team, and months of effort. Now someone can connect an agent to Salesforce, Slack, Gmail, and a document store in an afternoon. That is useful, but it also means the old mistake can happen casually. One framework I learned from my operation days at EMC still sits in my head. Make the process consistent, make it simple, then decide how to scale it. I still think EMC was one of the best operations organizations I've seen. This was one of those boring ideas that stays useful for years. Consistent means the trigger, the handoff, the decision rule, and the result mean the same thing every time. Simple means the process has as few steps, inputs, outputs, and exceptions as the work can safely carry. AI does not remove that test, it raises the stakes on it. A vague process does not get clearer when you point an agent at it. A political handoff does not become less political. An untrusted source field does not suddenly become trusted because the summary sounds fluent. Back to the deal desk example, the tempting pat is to point agent at code data and ask it to infer the review process from what people already do. It will give you something, it will probably sound confident. That is the dangerous part. Because what it may learn is not the process, it may learn the workarounds, it may learn the rubber stamps, it may learn the exceptions people stop questioning because everyone was too busy to fix them. Then you give that agent right access to Salesforce, Slack, CPQ, or email, and now the workaround has automation behind it. The better path is slower at the front. Finance, SalesOps, Legal, RevOps, and business applications need to sit with the review path together. Binance owns the ARR definitions, RevOps owns the CRM and code data, Legal owns the terms exceptions, someone needs to own the handoffs between them. That is not glamorous work. It is exactly the work that decides whether automation is useful. OpenAI and Bain had a Harvard Business Review piece that put a useful name on this, the microproductivity trap. The idea is simple enough. You make individual tasks faster, but the value stalls because the surrounding workflow still depends on manual handoffs, tacit knowledge, and old systems that don't agree with each other. The task gets faster, but the business result does not move. Their example was also a code process. The company did not just sprinkle AI over the old steps. It changed who did the early bid work first, then used AI to support the new workflow. The improvement came because the process changed first. That is the part I think more teams need to sit with. The license fee is the easy cost to see. The real bill is underneath. Process redesign, data cleanup training, exception handling, and a clear answer for who owns the decision when the agent gets stuck. And yes, AI can help even there. We can feed it current process notes, workshop transcripts, sample deals, escalation examples, and policy documents, then ask for a first-pass feature state design. That does not replace the operating judgment, but it can get the workshop started. You should not need a giant consulting project just to ask whether the cowpad is worth payment. That to me is the right use of AI at this stage. Use it to understand the work before you automate the work. If the process is consistent, simple and owned, automate with confidence. If it is not, slow down long enough to understand what you're actually touching. That is what I mean by errand your complexity. Let me pull a few quick threads from the rest of the issue because they connect to the same pattern. First, Google got a legal warning in Germany over AI summaries. The issue was that AI overviews made false statements about publishers, and the court treated the summary as Google's own statement, not just a link to someone else's content. That matters for companies using AI internally too. If a tool summarizes a customer, a vendor, an employee, or a candidate, that summary is not part of your product or process. A disclaimer is not much of a control if the system is confidently restating bad information. Second, Anthropics new research arm is talking about fire drill for sudden AI capability jumps. I know that sounds a little science fiction at first, but the operating point is practical. What happens if the model your business depends on changes materially between quarters? Better, worse, more capable, more restricted, more expensive. Any of those can affect a live workflow. For CIOs and operational resilience teams, that belongs in vendor due diligence. Third, Pew's June resurvey had a trust pattern worthwhile. About half of US adults now use chatbots, up from about a third in 2024. So adoption is moving, but 71% expect AI to make their personal information less secure. That is the room AI walks into. More people are using it, and many of them trust it less. And then there was one hopeful story I like from Stan. Di Young's lab built an AI practice system for no vice counselors. The simulated patients are intentionally not agreeable. They push back, withhold information, and resist easy advice, which is closer to real counseling than a chatbot that accepts every suggestion politely. In a randomized trial, practice alone built confidence. Practice plus feedback actually improved skill. That is the kind of AI training story I like, not replacing the human, giving people a safer place to practice before the stakes are real. Two things I put in in what I'm consuming this week are worth calling out. Adi Osmani wrote a piece called The Orchestration Text. It names something I feel all the time with agent work. Strategy work is cheap now, but closing the loop is still human and serial. You can spin up three agents, five agents, ten if you want to make your afternoon miserable, but every useful output still needs judgment. Someone has to decide whether the work is good and whether it fits the actual goal. That is very much the same pattern as this week's essay. The action got cheaper, the review did not. The other piece was from VentureBeat about MIRO using SQL query logs to help AI agents stop hallucinating bad joints. Their agents were wrong more than 65% of the time and pointed at thousands of raw snowflake tables. The fix was not a better prompt, it was a context layer built from validated query history and business meaning. That is the data version of Don't Automate The Mess. The model can only do so much if the operating context is missing. Quick after hours before I wrap, I've watched The Hunt, the 2012 Danish film with Mads Mikkelsen. Tough watch, a small town kindergarten teacher gets caught in an accusation that takes on a life of its own. And what the film captures so well is how quickly a community turns uncertainty into certainty. It reminded me of Middlemarch, oddly enough. Different century, very different story, but the same social mechanism is there. Gossip hardens into truth before anyone stops to ask what they actually know. Matt Mickelson is fantastic in it, and Anika Wetherkopp, the child at the center of the story, is remarkable for someone so young. Not light weekend viewing exactly, but it is the kind of film that stays with you. That's it for this week. The full issue with all the links and sources is in your inbox or at quantabits newsletter.behive.com. I'm Reza Markabadi. Thanks for listening. See you next week.