Is Thinking Expensive? The Economics of Large Reasoning Models : Smriti Kirubanandan, AI Executive

The economics of large reasoning models come down to one question: is thinking expensive? It turns out yes — thinking costs compute, time, and dollars, and like any input a firm uses, it can be allocated well or badly. An LRM isn't a fundamentally different kind of system from a standard LLM; it's the same underlying model put on a different labor contract. Instead of being paid piece rate to answer fast and move on, it's told to take its time, show its work, and get paid for the extra hours. That trade-off follows classic diminishing marginal returns: on easy questions, extra reasoning adds little or even negative value, but on genuinely hard, multi-step problems — math, planning, debugging, contingency-heavy decisions — reasoning keeps paying off well past where it would for a simple task. The real value sits in a "sweet spot" of mid-level difficulty, not at either extreme.

The other side of the ledger is cost, and cost here means more than the token bill — it also means latency and the opportunity cost of a user's attention, which is often the steeper price. That's why production systems route the easy majority of queries to cheap, fast models and reserve expensive reasoning models for the hard tail — straightforward cost minimization, not a technical hack. It also helps to think of reasoning tokens as capital investment rather than a simple expense: an upfront cost that buys a reduced chance of downstream error, except this "capital" never depreciates or compounds — every query starts from zero, so it's rented, not owned. The practical takeaway is a three-question test before reaching for an LRM: is the task genuinely multi-step, does the cost of being wrong outweigh the cost of extra latency, and is there a cheaper substitute that gets you most of the way there? Clear all three, and it's worth paying for the extra thinking. Otherwise, don't.

Transcript

SPEAKER_00 0:01

Welcome to AI Forward, the podcast where we break down the world of artificial intelligence, one conversation at a time. I'm your host, Smirti Kirbanandan, and in each episode, we'll explore the ideas, technologies, and people shaping the future of AI. Artificial intelligence isn't just one thing, it's a collection of technologies working together to transform how we live, work, and connect. From machine learning that helps systems improve with data to natural language processing that enables computers to understand us to computer vision, robotics, and generative AI. Each piece is building towards something bigger: intelligence that augments human potential. Think of AI as a spectrum. On one end, it powers everyday conveniences like recommendation engines, voice assistance, and smart devices. On the other, it drives breakthroughs in medicine, climate science, creativity, and even space exploration. AI is already here, woven into the background of our lives, but its true impact is only just beginning. In this show, we'll dive into how AI works, what it means for industries, and the ethical questions we must face as we move forward. Whether you're an innovator, a curious learner, or someone who just wants to understand what's next, you're in the right place. Let's move beyond the buzzwords, cut through the hype, and take a thoughtful forward-looking journey into the world of artificial intelligence. This is AI Forward, and I'm your host, Smirti Kirbanat. Here's a question an economist would ask before an engineer would. Is thinking expensive? Turns out yes. Thinking costs time, it costs compute, and just like labor, capital, or oil, it can be allocated well or allocated badly. That's a lens I want to use today. Not what is the architecture, not how many parameters, but when is it worth paying for a machine to think longer? And when is that just money burned for no extra output? That question is exactly what separates a regular large language model from what's now called a large reasoning model, an LRM. And once you see it through an economic lens, the whole category stops feeling like a hype and starts feeling like a pretty familiar production decision. Let's get into it. So, first, the basic definition, and I'll get into the boring way and the economic way. The boring way. A large reasoning model is a large language model that has been trained and is allowed at inference time to generate an extended internal chain of reasoning before producing its final answer. Instead of predicting the next word in a single forward pass towards an answer, it produces intermediate steps, hypothesis, checks, backtracking, verification, and only then commits to an output. The economic way. Think of regular LLM as a worker paid piece rate who is incentivized to answer fast in one pass and move to the next task. And LRM, as a worker, you've switched onto a different contract. One way you say, take your time, show your work, and I'll pay for the extra hours. It improves the outcome. That's really the whole shift. It's not a new kind of factory, it's a new labor contract for the same factory. And here is the important part of today's episode. That contract is not free. Every extra reasoning step is additional compute, additional tokens generated, additional latency, additional dollars. So the question a rational ask asks, should I always use an LRM? It's the same question a firm asks about on any input. What's the marginal cost of one more unit of reasoning and what's the marginal benefit? Let's stay with the marginal framing because it's going to carry the whole episode. In economics, one of the first laws you learn is diminishing marginal returns and more of one input, say labor, while holding others fix it and each additional unit contributes less than the last. The first worker on a farm is enormously productive. The 50th worker on the same fixed average is mostly getting in the way. Reasoning tokens behave the same way. For an easy question, what's the capital of a well-known country? Convert 10 miles to kilometers. The marginal product of additional reasoning is basically zero. Maybe even negative because you've introduced more surface area for the model to talk itself into an arrow. That's the equivalent of hiring a consultant to deliberate for three hours over a decision that any competent junior staffer gets right in 10 seconds. You paid for thinking you didn't need. But for a generally hard problem, multi-step math, a subtle logic puzzle, planning across contingencies, debugging a gnarly piece of code, the marginal product of reasoning starts high and stays meaningfully positive for a much longer stretch. Each additional reasoning steps catches an error, considers an alternative branch, checks a constraint, you're not paying for wasted deliberation, you're paying for a real increase in the probability of a correct answer. So here's a big economic insight about LRMs. The value of extended reasoning is not constant, it's a function of the task difficulty. Researchers actually see this in a very little shape when they plot accuracy against problem difficulty for reasoning models versus standard models. Uneasy problems. That middle zone, not too easy, not impossibly hard, is the sweet spot where reasoning has real marginal product. And that's your first practical rule for when you need an LR. Reach for one when the task sits in that band of genuine difficulty, not an either tail. Opportunity cost and the price of latency. Now let's talk about the other side of the ledger, cost. In economics, cost isn't just money out the door, it's opportunity costs. What you give up by choosing one option over another. With LRMs, there are two costs stacked on top of each other. First, the direct cost. Reasoning tokens are still tokens, and providers generally charge for them visible or not. A question that might cost a few hundred tokens for a standard model can balloon into several thousand tokens of internal reasoning for an LRM before it even writes the final answer. That's a real line item. Second, and this one gets underrated, it's latency and opportunity cost. If a customer support system takes four seconds instead of half a second to answer a simple parcel reset question, you haven't just spent compute, you spend the customer's patience and possibly the customer. Time is a scarce resource for the user just as much as compute is a scarce resource for you. An economist would say you're trading off two different currencies, dollars and attention, and attention often has the steeper price. This is why in production systems you rarely see engineers running every single query through a reasoning model. Instead, you see something that looks exactly like a firm doing input substitution based on relative prices. Route the easy 60% of queries to a fast, cheap, non-reasoning model, and route the hard 20% to the ones with real marginal product from thinking to the expensive reasoning model. That's not a technical hack. That's a textbook cost minimalization strategy. Use the cheap input wherever is sufficient substitute and reserve the expensive input for where it's generally necessary. So, rule two for when you need an LRM, ask whether the cost of being wrong exceeds the cost of thinking longer. If a wrong answer costs your customer, a compliance violation, a broken deployment, or a bad medical or financial decision, the extra tokens are cheap insurance. If a wrong answer costs you nothing more than a mildly annoyed user who asks again, you're overpaying for reasoning you didn't need. Reasoning is a capital investment, not just an expense. Here's a reframe I like, and it comes from thinking about reasoning tokens less like a consumable expense and more like a capital investment. When a firm invests in capital, a machine, a piece of infrastructure, it's paying an upfront cost in exchange for a stream of future benefits. Efficiency, reliability, fewer errors down the line. Reasoning at an inference time is a little like that, compressed into a single response. The model pays an upfront cost, extra tokens, extra time to build an internal scaffold, breaking the problem into sub-problems, testing intermediate conclusions, catching contradictions, and that scaffold reduces the probability of downstream failure in the final answer. The analogy is perfect because unlike physical capital, the investment doesn't persist. You pay it again every single time. There's no depreciation schedule, there's no amortization. Every query starts from zero. That's actually a meaningful economic weakness of current LRMs compared to, say, a human expert whose reasoning skill generally compounds over a career and gets cheaper to deploy each time. The reasoning model doesn't get faster at solving the same class of problem the tenth time it sees it. Inference time reasoning is a rented input, not an owned asset. Whether that changes as training techniques evolve is very much an open question. But that's the honest state of things today, not a promise about tomorrow. So the framing I'll leave you with reasoning tokens are a short-lived capital investment with no depreciation curve, valuable in the moment for the task at hand, but not compounding across tasks the way, say RD investment compounds for a firm. That matters for how you budget it. You're not building an asset, you're renting reliability one query at a time. When you actually need an LRM, a decision framework. Let's land this in something usable. If I were running this as a firm's resource allocation problem, here's the checklist at hand, a decision maker. Three questions in the spirit of a cost-benefit test. Question one, is this task multi-step or does it require weighing competing constraints? Single lookup facts, single rewrites, casual conversation, low marginal product from reasoning, multi-step math, planning under constraints, debugging, legal or contractual analysis, anything where an intermediate mistake compounds, high marginal product. If the task doesn't have steps in any real sense, an LRM is a solution in search of a problem. Question 2. What's the cost of an error relative to the cost of extra latency and tokens? This is the direct cost-benefit comparison. High stakes, hard to reverse decisions justify the extra spent. Low stakes, easily corrected ones usually don't. Question 3. Is there a cheaper substitute that gets you 90% of the way there? Sometimes a well-structured prompt, a bit of extra context, or a smaller model with a verification pass gets you nearly the same accuracy at a fraction of the cost. Economists call this checking for substitute goods before assuming you need the premium one. Don't buy the expensive input out of habit. Check whether cheaper one clears the bar first. If you run a task through those three questions and it clears all three, genuinely multi-step, genuinely cost to get wrong, and no cheap substitute clears the bar, that's your signal. That's when you reach for the reasoning model. Everything else, you're likely better served by the faster, cheaper option, putting the savings towards volume or speed instead. So let's zoom back out. What is an LRM? Mechanically, it's a model trained and permitted to spend extra inference time, computation, generating intermediate reasoning before answering. Economically, it's a different labor contract with the same underlying system, one where you are explicitly buying time in exchange for a shot at fewer errors. And like any input decision a firm makes, the right call isn't just always use the premium option or never use it. It's matching the input to the job. Reasoning has real measurable marginal product on genuinely hard, multi-step high-stakes problems, and close to zero, even negative marginal product on easy ones. The winners in the space, whether that's an AI company designing routing systems or a business team deciding how to use these tools, are the ones who treat reasoning as a scarce price resource to be allocated efficiently. Not a feature to switch on for everything because it sounds more impressive. That's the whole episode. Really compress into one line. Don't pay for thinking you don't need, and don't skip on thinking you can afford to skip. Thanks for listening. I'll see you next time.