Model Routers: How Enterprise AI Chooses the Right Model

Most enterprises will not rely on a single AI model forever. Instead, they will use multiple models for different tasks—and model routers will decide where each request should go. 

In this episode of the Macro AI Podcast, Gary and Scott explain how model routers work, where they sit in the enterprise AI architecture, and why the technology is becoming an important control layer for cost, performance, security, and resilience. 

They break down the differences between infrastructure routing, policy-based routing, and intelligent prompt routing, then examine how platforms from Microsoft, Google, Amazon, Cloudflare, Kong, LiteLLM, and Palo Alto Networks approach the problem. 

The episode also takes a closer look at Cloudflare’s broader enterprise AI strategy, including AI Gateway, Workers, Workers AI, Vectorize, AI Search, security, and Zero Trust services. 

Finally, Gary and Scott discuss where model routing is headed as enterprises begin routing not only prompts, but entire AI workflows across models, providers, regions, tools, and security policies. 

For business and technology leaders, the key question is no longer simply which AI model to choose. It is how the enterprise will continuously decide which model should handle each piece of work—and how it will know that decision was correct. 

About your AI Guides

Gary Sloper

https://www.linkedin.com/in/gsloper/

Scott Bryan

https://www.linkedin.com/in/scottjbryan/

Macro AI Website: 

https://www.macroaipodcast.com/

Macro AI LinkedIn Page:  

https://www.linkedin.com/company/macro-ai-podcast/

Gary's Free AI Readiness Assessment:

https://macronetservices.com/events/the-comprehensive-guide-to-ai-readiness

Scott's Content & Blog

https://www.macronomics.ai/blog

Transcript

00:00
Welcome to the Macro AI Podcast,  where your expert guides Gary Sloper and Scott Bryan navigate the ever-evolving world of artificial intelligence.  Step into the future with us as we uncover how AI is revolutionizing the global business landscape from nimble startups to Fortune 500 giants.  Whether you're a seasoned executive,  an ambitious entrepreneur,

00:27
or simply eager to harness AI's potential,  we've got you covered.  Expect actionable insights,  conversations with industry trailblazers and service providers,  and proven strategies to keep you ahead in a world being shaped rapidly by innovation.  Gary and Scott are here to decode the complexities of AI and to bring forward ideas that can transform cutting-edge technology into real-world business success.

00:57
So join us, let's explore, learn and lead together.  Welcome to the Macro AI podcast where we help business and technology leaders understand the development, shaping enterprise, artificial intelligence. I'm Gary Sloper joined as always by my cohost, Scott Brime. So Scott, today we have, you know, we've said this several times recently that, you know, businesses should not automatically send every AI request.

01:25
the largest and most expensive model available.  But I think today we're not going to spend a lot of time making that argument again. think what you and I want to talk about today is examine the technology that actually makes the decision, and it's called Model Router. Yeah. Yeah, it's similar to an internet router that you and I started working on before the dot com days.  instead of an internet router, it's an AI model router.

01:53
just simply sits between an application and then a group of models. So it'll examine incoming requests,  decide which model or which model endpoint should then handle it. And that sounds pretty simple, but once you look under the hood,  model routing can really involve application policies, security rules,  semantic classification,  model performance predictions,  capacity management, lots of things, including payload. Yeah.

02:23
Some really good stuff.  So let's dive in. We're going to explain how model routers work, where they fit in the enterprise AI architecture, because you're probably wondering that. And really who the major players are in companies,  including Microsoft, Google,  Amazon,  Kong, CloudFlare, Palo Alto.  The list is growing and this group of open source platforms are really approaching the problem differently. So we'll kind of explain all that. Yeah.

02:52
Yeah. So just kind of dig into what exactly is being routed. I think the first thing businesses need to understand is that the term model routing is being used to describe a couple of different technologies. But at the most basic level is just simple infrastructure routing. And that might mean sending a request to the same model running in a different cloud region or moving traffic to another deployment because one endpoint is overloaded or not available.

03:21
Or maybe it has a lot of latency.  And so that's really similar to conventional application load balancing that a lot of listeners might be familiar with.  And the system isn't necessarily deciding which model is the smartest for the prompt. It's deciding which available endpoint can process the request most effectively. that's, that's infrastructure routing.  Yeah. And it's a really good distinction.  And I think if you couple that really, that, you know, that next level is policy routing.

03:51
And that's where the route is determined by explicit business rules.  So if you were to think about it this way, a company might route requests differently based on the employee's department, the application being used, the customer services serviced here, the geographic location, or even the sensitivity of data or the remaining budget. So for example, a free customer might receive one model while a paid customer receives another.

04:20
And that obviously helps in that regard with that model organization. But a request containing things like regulated data might be restricted to a model deployed inside a specific environment. So it's not going out to a less costly one or more public.  A development team might be allowed to experiment with several external models while the finance department, again,

04:45
could be more of a controlled governance environment and it's limited to a smaller approved set.  Yeah. Yeah. And the finance department cares about policy routing because of the remaining budget. that's a good.  Yeah. Protecting the sensitive financials and also where it's being spent. Yep. Exactly.  Yes. The next type of model routing is intelligent prompt routing. And this is the version version that really gets the most attention.  So an intelligent router

05:15
analyzes the prompt itself, and then it predicts which model is most likely to produce an acceptable answer at the desired combination of quality, cost, speed, and reliability.  And then some of the routers that use intelligent prompt routing use a trained classification model. So some use semantic similarity, some use manually defined keywords and heuristics, but then there's more sophisticated

05:43
uh, intelligent prompt routing systems that learn from previous results and then adjust their routing behavior over time. Um, and then all three types of routing that we just, um, talked about can, can interoperate. can, they can operate together. So a business might first use security policy to eliminate models that aren't eligible. It might then use, uh, an intelligent router to select from the approved models. And then finally,

06:10
you know, the more basic one, a conventional infrastructure routing might choose the healthiest region or deployment for the selected model. Yeah, it's pretty interesting how that works. And I already know, you know, before anyone sends us a question, probably asking how a model router works, a little bit of a word jumble there.  And maybe we just kind of walk through the technology.  if you were to kind of envision

06:38
you know, an application. So an application generally sends its request to a stable gateway or model router endpoint. And that's rather than calling a particular model directly. So behind that endpoint is usually a model registry containing information about the available models in that moment. That registry may include each model's cost,  context window,  supported languages,  tool calling capabilities,  multimodal support,

07:07
the data location, kind of what you were just talking about, Scott, and the requirements for that.  Expected latency, so that's obviously something important, especially if it's very mission critical, and other things like capacity limits and historical performance. So the router's first job is not to just choose the best model. Its first job is to remove the models that cannot or should not handle the request based on what we just listed out. Yeah, exactly. And that eligibility step right there is critical. Right.

07:36
Suppose a prompt contains a large image and requires a half a million token context window and has to remain inside a particular data zone.  There might be only one or two eligible models before the intelligent decision even begins.  So the router can also take another look at it and apply organizational policy at that stage.  So, you a few examples, like is this user authorized to access an expensive reasoning model?

08:05
Uh, is this application allowed to send data to an external provider or another one would be, you know, does the selected model support the tools required by the agent? If you've got agentic workflows or has that department, mentioned finance as the finance department exceeded its monthly AI budget.  uh, and then once the, the ineligible choices have been pulled out of the decision tree, then the router analyzes the request and it works on.

08:35
Yeah, and to that point, there's several ways to do that.  You know, a rules-based router might look at things like known fields,  headers,  metadata,  keywords. You know, it's fast and predictable, but it always requires the environment to maintain those rules in that model router.  people that go into it. Right. A semantic router can convert the prompt into an embedding and compare it with descriptions or examples of tasks that

09:04
each model handles well.  Then if you look at a classifier, a classifier can assign the request to categories such as coding,  document extraction,  creative writing,  mathematical reasoning, customer support or translation. And then you have a learned router attempts to predict the expected performance of each model on that particular prompt. So it's, you know, it's much different than probably what you would expect. Again, it's not just looking for the cheapest option. Yeah.

09:33
Yeah. And then, then the router goes and applies an optimization objective. So it might be trying to maximize quality,  regardless of the price, or it may be looking for the least expensive model expected to the seed of quality threshold, or it might be balancing several factors at once.  So, but anyway, conceptually the router is assigning a utility score per se to each candidate.  So expected quality minus cost, minus latency.

10:02
adjusted for reliability policy or whatever else the company values. Um, and the model with the highest acceptable score then receives the request. Um, but then the job isn't over when the request is sent, uh, the execution layer then kicks in and it might need to retry the call, um, choose a, uh, fallback provider or enforce a timeout or maybe move the request to a stronger model. If the first attempt just doesn't work.  Yeah. It's almost like a,

10:31
a model VRRP failover.  You know?  So, and I think the last point introduces another routing technique known as Cascade.  Instead of predicting the model in advance, the system begins with one model and evaluates the result. A second model or a separate evaluator can ultimately determine whether the answer satisfies the required standard in place. So if it doesn't, the request is escalated to a more capable model.

11:02
And this can work well because routing becomes particularly evidence-based. The system is no longer judging only on the prompt. It's also judging on the answer. The thing to keep in mind, however, is, is Cascades create additional model calls so they can add latency and potentially eliminate the savings if they are poorly designed. So you want to really think about that again, getting back to the finance team being concerned about the costs.  Yeah. Another thing to watch. Um, yeah. And then the.

11:31
What kicks in then is that the conversation state creates another complication. So switching models in the middle of a conversation can cause some problems. can change tone,  behavior,  pool use,  interpretation of previous messages. And in general, it can reduce the effectiveness of provider-specific prompt caching if you're switching between models.  So that's why some of the routers that are available

12:00
actually support session affinity,  also sometimes called stickiness.  And so once a conversation has been assigned to a model, the router can keep it there unless a failure or escalation requires a change.  So a production router really kind of therefore really needs to understand more than individual prompts. It needs to understand sessions,  users, application, and really entire workflows.

12:30
Yeah,  that's good points.  And I'm sitting here thinking too, you know, we probably need to clarify another term. You know, a model router is not necessarily the same thing as an AI gateway. So just keep that in mind.  And that's probably something, especially for a little bit of confusion out there. Yeah. Yeah. Just, you know, people that are familiar with, you know, layer three routing, it's not a gateway in that traditional sense. The router decides where request goes. The gateway is

12:59
really the broader control point, which, you know, the request travels on.  So an AI gateway may include routing, but it can also provide other things that are important, such as authentication,  logging, cost tracking,  caching,  rate limits,  content filtering,  DLP retries,  provider translation, and just overall security controls, similar to DLP.  So.

13:26
Those are the things to just keep in mind that it's not the same thing as, an AI gateway.  Yeah. So I think really you could think of routing as one decision inside of that gateway. So I think that, that distinction there really matters when you start to actually compare routing,  model routing products.  So for example, Microsoft and Amazon, they've developed managed systems that attempt to predict which model should answer a prompt.

13:53
Then you've got other ones like Cloudflare and Kong, and they provide broader gateway platforms with powerful routing controls. But their emphasis is really more placed on controlling,  securing, observing, and managing the traffic moving amongst the models. So they're related products, but they're not identical. Yeah, that's a point.  to that,  if we were to look at the hyperscalers, like you had just mentioned, and a couple of the others like Microsoft and

14:23
Google and Amazon. We covered Microsoft's Foundry recently in one of the prior episodes, so we won't repeat the broader platform, but Microsoft's model router itself is a trained language model. It analyzes attributes such as complexity,  task type,  and reasoning requirements, and then selects from the models available to the deployment. So Microsoft offers quality, cost,  balanced routing modes.

14:52
It also allows customers to limit the router to selected subset of models, which is really important as well. Yeah, I know we touched on it in the last one, but there are definitely some useful architectural details in the way that Microsoft implemented this.  So the router honors Azure model deployment policies and data zone restrictions. And it now includes automatic failover among eligible models. So that's kind of neat.  If you're in the Microsoft.

15:22
ecosystem there. But one important limitation is that the effective context window can be constrained by the smallest model in the router's overall candidate set. So that means companies still need to configure the model pool really carefully and adding a low cost model with a smaller context window can affect what the overall router can reliably accept.  So I think, like I said, Microsoft's approach is attractive for companies that are already committed to Azure and want a managed decision layer.

15:51
without going out and sourcing and building their own model router. Yeah. Good point. And if you were to look at Google's equivalent,  their intelligent capability is model optimizer. And it's now available within Google's ever-changing Gemini enterprise agent platform. So some of you have probably seen that.  Google describes it as selecting the model and tools based on the desired balance of quality, speed, and cost.

16:20
But one thing to note, Model Optimizer remains a paid experimental offering. Yeah. It's not broadly available or generally. Yeah. Yeah. And I think that's an important qualification for enterprises really evaluating production architecture. So just keep that in the back of your mind. doesn't mean it won't be paid in an experimental fashion in the future, but that's where it's at as of this episode.  Google also has other forms of routing.

16:48
think of these things that they've embedded,  including capacity aware routing across regions and API level routing amongst models. You know, those solve availability and management problems.  But I would say they should not all be confused with intelligent prompt selection. So just keep that in mind. Yeah, exactly.  Yeah, we'll just switch over to Amazon.  So Amazon has bedrock and inside of bedrock, they have intelligent prompt routing.

17:18
So their router, Amazon's router really predicts response quality of the available models that it's reaching out to, and then tries to balance that quality against cost.  its current config shows really how early this market still is.  So a configured Fedrock router requires exactly two models from the same model family.  And Amazon says that the system is optimized for English prompts and doesn't...

17:45
that as of now doesn't adjust its routing decisions using application specific performance data. But that can still be useful, you particularly for companies that are already standardized on bedrock. But it's more constrained than, you know, the long-term vision of a router learning across many models and providers from the company's own results.  then there's another one.  Gary, I think you're pretty familiar with CloudFlare from some of the work that you do with multi-cloud connectivity.

18:14
Yeah, yeah, work with them quite a bit and known them for many years, even back to my Dyn days.  They kind of approach the model problem from a different direction. So Cloudflare is not just starting as a model provider, it's starting from its position in network traffic and other things such as application delivery,  cybersecurity and edge computing.  It's an AI gateway that really sits between an enterprise application

18:41
and model providers such as OpenAI or Anthropic, Google and others. The application calls the Cloudflare endpoint, right? So it starts there and Cloudflare applies the enterprise policies before forwarding the request.  Right. Yeah, they definitely got a interesting solution.  So they have a routing, a dynamic routing feature. And what that does is it lets a company build a routing flow visually or through a JSON configuration.

19:09
And so really a route is essentially a versioned decision graph. So it can contain conditional branches that inspect request data, headers or metadata. And then it can split traffic by percentage for testing.  can enforce rate limits, budget limits, and then it can select a model and define a fallback path if the preferred route fails or reaches a quota like we've been talking about.  So company can create a route that says

19:40
you know, send paid users to one model, free users to another model, and then it could restrict an entire business unit to a daily budget. Those poor finance guys again. And then, you know, or send say 5 % of traffic to a newly released model for evaluation, or, you know, fall back to the second provider if the first provider times out. So you can get pretty complex with it. And then the company can change that flow without rewriting, you know, every application that uses it.

20:08
Yeah, and I think this is good example of why we shouldn't treat every model router as the same technology.  So, know, Cloudflare dynamic routing is currently the strongest as a programmable policy and traffic routing system. It doesn't automatically become a learned prompt router simply because they can... That's a good point. Yeah, choose among several models.  A company could add a classifier or classification result to the route and use that result as metadata.

20:38
Um, but that routing intelligence may need to be supplied by the application, a worker or another service.  So just, you know, that's how cloud flare is. And then if you think about it from Microsoft's perspective, Microsoft's model routers, really attempting to predict model suitability directly. Cloud flares, given the enterprise, a highly flexible control plan for expressing and enforcing its own routing decisions. Yeah. You can, you can do a lot with it if you.

21:07
if you invest in that.  And that control plan includes a lot more than just the routing.  So Cloudflare AI Gateway provides logging and analytics across the model providers.  It can cache repeated responses,  enforce rate and spending limits,  store provider credentials.  It can apply authentication and you can put in guardrails to inspect prompts and responses.

21:35
And they've also been adding DLP, data loss prevention and identity aware controls so that companies can apply different model access and spending policies to the different users, teams and applications. And that's, that's really important for agents.  So an enterprise, um, might not need to know only which employee initiated an action, but which agent, uh, or, or the tool or our workflow made each downstream model request.  Right. And you know,

22:03
Cloudflare also has a broader set of services that can be assembled around the gateway.  So Cloudflare workers provide globally distributed serverless compute.  A company can use workers to run things like processing, classification,  routing logic, transformation, really close to the application.  And workers AI provides serverless inference for models running on Cloudflare's own GPU infrastructure.

22:30
Vectorize provides vector storage for retrieval augmented generation.  So things like AI search can index websites,  documents, and R2 object storage for natural language retrieval. The other thing about Cloudflare,  their application security portfolio includes WAF,  Web Application Firewall. They've been doing that for many years, and API security, bot management, and DDoS protection.

22:58
And then, know, Cloudflare One extends the platform into a zero trust access. So now you have secure web gateways, network connectivity, and SASE. So it's a very broad platform that Cloudflare has gone to market with. Yeah, it's definitely pretty broad. I think that kind of that larger, you know, under the category of model routing, that larger portfolio is why Cloudflare is definitely worth looking into or worth watching.

23:26
So business could use Cloudflare to receive the application request and authenticate, inspect the prompts and all those things that we talked about.  And then it could use an external provider or workers AI to inspect the response, log the transaction,  all those things that we just covered. And that creates a of a potential convergence point between AI infrastructure.

23:49
application delivery, network security, and enterprise access policy. So all these things that IT is kind of juggling,  it does kind of nicely pull it together in one package. And the advantage there is consistency.  But the risk there is that a growing amount of the company's AI control plane becomes concentrated in one platform. So it's that CIO level problem that they need to juggle. Yeah, agreed. mean, really, Cloudflare is not

24:18
simply competing to be the smartest prompt classifier. It's positioning itself to become really part of the infrastructure through which enterprise AI activity moves at, you know, varying speeds.  So, um, other platforms to watch. So we mentioned there was, there was some others.  So if you look at Kong, Kong is another important company in this market, particularly for enterprises already using API gateways.  So, so Kong's API gateway can

24:48
Normalize requests across providers,  perform retries and failovers,  monitor token usage and route traffic accordingly to latency usage,  consistent hashing,  round robin policies or semantic similarity.  It's semantic routing can really compare a prompt with descriptions of the available model targets. And it's using embeddings in a vector database to do so.

25:16
And that gives platform teams a way to bring AI traffic under the same API management discipline. They've already applied to the conventional applications in their business today.  Yeah. Yeah. So that's Kong's AI gateway.  Another one that I looked at was Light LLM.  And I think it's really most prominent amongst teams that really want an open source or totally self-managed option. So it has a common interface.

25:45
across many model providers, along with load balancing,  retries, fallbacks, health checks, of course, has budget controls and then routing across deployments. They do have some newer auto routing capabilities that can classify requests using heuristics, keywords,  semantics,  or smaller language model classifiers.  So, LLM is, they're also working on things like adaptive routing,

26:15
So it'll learn which models perform well for different categories of work. Pretty interesting.  So I think their product really gives you more control, but with that control comes the responsibility for, you know, operating, securing, know, testing and just maintaining that router routing layer.  Yeah. Yeah. Those are good points. Definitely a solid product. And, you know, when you start thinking about layer three, you can't forget.

26:43
Palo Alto Networks, because that's been a lot of their heritage.  And they've entered this category through the acquisition of Portkey. So Portkey was one of the leading independent AI gateway companies.  Palo Alto has incorporated that tech into Prisma,  AIRs and AI Gateway, which became generally available just last month, July 2026.  So its differentiation is likely to...

27:11
is center on security and governance.  so think of identifying applications and agents,  authenticating interactions,  enforcing policy,  monitoring AI activity across the enterprise, which is very important. So I think a lot of people think of Palo Alto from a security perspective.  Absolutely. Absolutely. So for companies that really view AI routing primarily as a cybersecurity and governance problem, um, you know, Prisma errors will be the important platform to evaluate as part of that.

27:41
bolt on for your organization, especially in the CSO's office.  Yeah. Yeah. So I think we covered, covered a bunch of different types of platforms. And so I guess the question would come up, you know, which architecture should a business choose?  think the strategic decision is really whether routing should remain inside the company's primary cloud ecosystem. You know, we talked about Microsoft, for example,  or sitting in an independent layer above several ecosystems. And so, you know, common IT question.

28:11
So a Microsoft-centric company might find that the Foundry model router gives it enough intelligence, governance, and really overall operational simplicity and get the job done for them. A Google-centric company might prefer Model Optimizer as Model Optimizer matures, particularly if most of their workloads use Gemini.  And then an AWS-centric company might choose Bedrock. Yeah.

28:39
Yeah, but then you may have a company that's multi-cloud, so they might be using Azure and Google Cloud, and then have direct Anthropic APIs,  privately hosted open models, and maybe even several specialized providers that may want an independent gateway into the infrastructure. And that could be CloudFlare or Kong or Lite, LLM, Prisma or Ayres, or an internally developed platform.  So a hybrid design may...

29:05
become the most common approach. It really depends on your use case, what you have for tech and what you're trying to achieve.  The enterprise gateways apply identity, security, compliance,  budgets, and really a provider level policy. A cloud native router then makes a more detailed model selection within an approved ecosystem.  really depends on, again, on your use case.  Yeah. think businesses should really kind of resist the temptation to put

29:34
every available model into one giant pool.  That's obvious kind of,  but you know, more choices don't automatically produce better routing is kind of an important thing to think about.  So a practical architecture might establish a small testing group of models for each workload. Perhaps one efficient model, one stronger general purpose model, one specialized model, and then some kind of fallback.  And then allow the routing system to then

30:03
you know, make the decisions in a controlled environment that the company can actually go in and evaluate and see how the platform is working for them. Yeah. Good point. And I think for business leaders, they're probably asking, okay, what is my biggest blind spot, biggest risk here? And I think it's really optimizing the wrong thing. And that risk is not that the router stops working.

30:29
You know, it's really if that router is working exactly as designed, but optimizing the wrong measurement. So a generic benchmark may say that a model produces an acceptable answer, but the company may care about whether the answer resolves a customer issue,  generates valid code,  extracts the correct contract terms, or helps a complete financial process without rework, getting back to the financial team.  So, so I think that's.

30:57
That's really the biggest risk there is optimizing the wrong thing. Yeah. it produce an acceptable answer?  Yep. So I think that's why routing and evaluation, you know, kind of have to be connected. So the enterprise needs representative test cases.  They need, you know, telemetry from the production environment, user feedback, and then, like we just said, measurable business outcomes.  And then they have to kind of record

31:26
you know, which model was selected, why it was selected, what the request costs, how long it took, whether a fallback occurred in the process or whether the result was really ultimately successful for them.  And without that information, the router can become kind of a opaque cost cutting mechanism that slowly reduces quality. So you gotta keep tuning these things.  And then, course, know, model providers,

31:54
change their systems frequently.  So, you the pricing changes,  models are constantly updated. seeing more and more iterations of models. Sometimes they're actually retired and pulled out. And then, you know, context windows and tool capabilities change and a routing policy that performs well, you know, today might perform totally differently next quarter when based on the pace of change right now.

32:20
So I think the model router has to be treated as an operating system that requires continuous evaluation,  not just a configuration that's finished after deployment and everybody focuses on other things.  Yeah. And you might be wondering where model routing is going.  And the next phase of this technology will extend beyond selecting one model or one prompt.  An agent may use one model to create a plan, a second model to write code, a smaller model to

32:49
classified documents and stronger reasoning model to verify the final result of everything that I mentioned in the workflow. The model router will make decisions at every step of the workflow.  Right. Yeah, it's a, there's a lot, lot to it. I think hopefully it doesn't sound confusing, but it'll also route more than models.  It might choose the model provider cloud region, you know, inference, hardware,  reasoning, budget.

33:19
retrieval score,  pool security path, or whether the work should run in the cloud, at the edge, or inside the enterprise. And I think the more advanced systems will learn from business outcomes rather than relying on just generic model benchmarks.  So they might even use adaptive algorithms to test new models on a limited portion of traffic,  compare the results, and then they will gradually adjust their routing policy. So that's kind of where it's going.

33:48
Um, you know, and then just one other note, they, think they'll increasingly evaluate answers after they're generated. So that'll, you know, it'll, they'll then be able to do things like escalating, uh, difficult or uncertain work if they, if they need to, to somewhere else. Yeah. That may make the routing and evaluation layer really strategically important part of your enterprise architecture. Um,

34:15
Individual models will continue to improve and companies will continue to replace them. But the system that understands the company's users, policies, workloads, costs, and really the quality requirements can remain in place even as the underlining models change. The model provides intelligence. The router determines where and how that intelligence is used.  Yeah, that's a good summary.  So I think.

34:40
I'm just trying to wrap it up. think that the main takeaway is that model routing is not just one product category. It includes a bunch of things, including like we talked about, infrastructure routing, business policy routing, intelligent prompt selection,  failover, really the orchestration of entire AI workflows.  so the leader, know, the big guys, Microsoft, Google, Amazon, they're developing managed routers within their ecosystems.

35:09
Then you've got, like we talked about Cloudflare and Con are building broader gateways and control planes. And you've got some of the startups like Light LLM.  They give companies really kind of that open and flexible foundation that you need to be capable of managing.  And then we talked about, you know, the security leader Palo Alto Networks. They're really quickly driving to make the AI gateway part of enterprise cybersecurity in general. And that's a pretty, obviously a no brainer play for them.

35:38
Yeah. And for business leaders, the question is no longer simply which AI model should we buy? I think that was a question that everybody will still be asking, but that's really an older question. The more durable question may be how will our company decide which model provider and environment should handle each piece of work and how will we know that the decision was correct? Yep. Right. That's it for the episode of the Macro AI Podcast. Appreciate you listening.

36:08
please share with your network.  Any questions sent to Scott and myself,  can find us on LinkedIn.  And thank you for listening.  And until next time, we'll see you soon.