What If Efficiency Means Headroom Not Speed

The fastest benchmark number is comforting right up until your video system meets the real world.

Staging looks stable. Charts look clean. Then production introduces variability at scale: messy networks, mixed content, thermal constraints, I/O bottlenecks, memory pressure, and timing issues that never show up in controlled tests. That’s where “efficient” can suddenly mean fragile. 

In this episode of Voices of Video, we sit down with Juan Casal, Partner and Chief R&D Officer at Cires21, alongside Leonardo Nieto from NETINT, to unpack what actually changes when systems move from lab results to real production environments.

We explore why evaluating isolated components such as encoder throughput, codec efficiency, or cost per stream can be misleading, and why the full video pipeline must be measured as a balanced system. Local optimization often shifts pressure downstream, turning encoding gains into decoder frame drops or network bursts that degrade quality of experience.

The core theme is operational confidence.

Predictability beats peak performance. Stable latency, stable resource usage, and clear worst-case behavior are what make capacity planning possible at scale. We also dive into observability and metrics that matter in production: variance vs averages, jitter, timestamp alignment, and how to design systems so failures can be predicted instead of simply reacted to.

Efficiency, in practice, is headroom. It is the margin that allows your infrastructure to absorb variability, whether you are running on CPUs, GPUs, or purpose-built accelerators.

If you are building or upgrading video encoding, transcoding, or streaming infrastructure ahead of NAB, this conversation will challenge how you evaluate performance.

Join the conversation:
https://voicesofvideo.netint.com/join-the-conversation

Learn more about:
Cires21 → https://cires21.com
NETINT → https://netint.com

Key topics covered:

• Why production introduces variability through scale, system interaction, and constraints like thermal limits and I/O bottlenecks
 • Why end-to-end pipeline balance matters more than single-component optimization
 • How local optimizations shift load downstream to decoders and networks
 • Predictability as the real operational goal: stable latency and resource usage
 • Why worst-case behavior and variance matter more than peak throughput
 • Observability gaps: jitter, timestamp alignment, and network bottlenecks
 • Real-world failure modes such as clock accuracy and jitter causing frame drops
 • Efficiency as headroom: stability, burst tolerance, and higher saturation thresholds
 • Designing for failure prediction and testing under sustained load

Stay tuned for more in-depth insights on video technology, trends, and practical applications. Subscribe to Voices of Video: Inside the Tech for exclusive, hands-on knowledge from the experts. For more resources, visit Voices of Video.

Transcript

Why Benchmarks Mislead Teams

Voices of Video 0:07

Voices of video. Voices of video. The voices of video.

Anita Flejter, NETINT 0:12

Voices of video. Welcome to Voices of Video. As we head toward NAB, we are speaking with partners across the VPU ecosystem about the questions that quietly determine whether video systems succeed or fail. Today's episode is called When Benchmarks Lie, How Video Teams Misread Efficiency. Because if you think about it, in modern video infrastructure, disagreement rarely starts with architecture diagrams. It starts with numbers. We are joined by Juan Cajal, partner and chief RD officer at CRES21. Juan's career spans more than two decades of building video technology at every layer, from developing encoding algorithms and working directly with hardware acceleration APIs to designing high-density transcoding systems and contributing to large-scale European RD initiatives. He has work across CPUs, GPUs, purpose-built accelerators, and real-time delivery systems. That bread matters because measurement only makes sense if you understand how systems actually behave under law. Juan, great to have you with us.

Juan Casal, Cires21 1:31

Thank you, Anita.

Anita Flejter, NETINT 1:32

Moderating today's discussion is Leonardo Nito, director of market development at EMEA NETINT. Leo approaches this conversation from the market side. This way he's able to surface the recurring evaluation mistakes, architectural tensions, and scaling blind spots he hears from teams making real deployment decisions. Leo, over to you.

Staging vs Production Variability

Leo Nieto, NETINT 1:57

Thank you, Anita. And uh great having you, Juan. It's always great to make this uh this session so that you know our audience uh can chip in and you know uh ask more questions or you know challenge what we are saying here. But it's always good to keep these conversations open, uh, you know, just to try to solve very complex problems in a in a growing, ever growing industry.

Juan Casal, Cires21 2:20

Thank you, Leonardo. Yes.

Leo Nieto, NETINT 2:22

So we start first, you know, when when you know, as as the title uh of this session says, one part that I consistently see across teams is that everything appears stable uh in staging. You know, the numbers say one thing, but when things actually uh uh they reach production, there's always unexpected behaviors that that begin to surface. And and gathering sometimes data might not be uh as accurate as one thing it is. What changes once a video system moves from a controlled environment into real production? Like start from the ground up. What can you tell me about that?

Juan Casal, Cires21 3:01

I would say that what uh the biggest change is is variability. Uh uh I mean, and when you go into production, uh variability is the real stress test. I mean, because you you usually test your technology in the staging, let's say, and this this means a controlled environment, and a stable network, stable hardware, and and and and this and and well formatted content. There you try, we try, we always try to simulate the conditions of uh production uh uh production uh uh environment. But this is very difficult because what it really uh takes variability in a production uh environment is scale. Okay, when you deploy at scale, equipment uh are interactive. There's interaction in the network, in the different at the different stages, and this is what brings variability into the so at the scales, at the end new new constraints apply. So there are thermal problems that you don't usually have in at in staging. There are uh input-output bottlenecks, there are uh memory pressure, and everything is unexpected. Well at this point, you you go into production and complexity becomes a liability, okay? Because every new uh stage in the pipeline multiplies a potential potential failure paths.

Evaluate The Whole Video Pipeline

Leo Nieto, NETINT 4:44

Oh no, understood. Because I understand the the issue is not the benchmarks are wrong. It's a lot of um a lot of variables to take into consideration. And you know, they are I mean, systems are you know many pieces running together, and they operate, like you say, under very random variable constraints, things that one might think one has control on. But ultimately, once you are live on the field, these unexpected things uh happen. Let's dive a little deeper into that because that's that's the context of what we're talking about. You know, many a lot of evaluations, they they they isolate components. Well, because logically, I mean, it's the first approach, the natural approach to try to enumerate a panoply of different uh variables, let's call them. Uh you know, encoder throughput, uh, codic efficiency, uh, I don't know, quality of experience, uh instance costs, uh, you name it. And as you say, these systems operate as a whole, almost like a like an orchestra, right? You know, first, how how important is it to evaluate the full pipeline rather than optimizing individual components as a as a as an initial approach?

Juan Casal, Cires21 6:03

Okay, I would say that this is fundamental. I mean, this is uh a key um task that it should be should should be always done. I mean video systems are composed as as we know the of a pipeline of different components. Usually there's an ingest, there's a preprocessor, encoder, packager, there's transmission, uh the coding and the rendering. Okay. When uh when video system uh works properly and works smooth, and so that user's experience is smooth and is uh is very good, it usually means that there is a balance uh in the way all of those components are are working. So it doesn't necessarily mean that there uh every of those components is super highly optimized. But there's a there is a balance uh what makes the system work properly and and take a very good user experience to the customer.

Leo Nieto, NETINT 7:14

Interesting. Because but in your experience, how often you know does improving, let's say, a single stage or let's say a single component increases the risk of affecting that you know that entire entire workflow? Because oftentimes people might point out, oh, you know, maybe is the problem with the FPG encoder. Oh no, maybe it's the way how the codec itself is not responding to certain frames, depending on the utility as well, of course. But uh do you feel or how do you feel that let's say fixing one only one and focusing on that affects the entire vision of what one is doing?

Juan Casal, Cires21 7:56

I think that that's quite a quite a common uh issue in video systems, uh because uh whenever you optimize one one of the stages of the of the system, this can uh take uh shift uh let's say low load pressure to to later uh stages in the pipeline. An example of this could be that if you use a higher optimized uh tools for encoding, then uh you may need uh uh a better, a more powerful decoder to decode that that that that video stream so that you have increased your encoding quality, maybe it even decreased the the the size of the stream. But when you take it to your decoder, the decoder needs more processing power that it may have, uh which may lead to frame dropping and and uh worse uh uh user experience. Local optimization uh you often shift uh load uh rather than removing it in our way of speaking. This can also happen with for example what's related to encoding and decoding, but optimizing encoding can also lead to higher network uh burst, uh which also can take to a worse uh uh quality of experience for the for the customer, and these kind of things happen. So at the end, whenever you optimize one of your components, you have to understand first if this is uh an optimization for your system, for your use case, and how this may affect the other components of the system, or the way you may uh degrade the quality of experience.

Predictability Beats Peak Performance

Leo Nieto, NETINT 9:58

Correct. I understand. Obviously, you know, um as you say, you know, there's always a context. You know, these things have to happen within a certain set of realities and objectives that that we're doing. This brings us, I suppose, to I guess the necessity for at least uh be confident on the predictability of set orchestrations. I hear many teams say that that peak performance is impressive, you know, that we can do so many great things at each individual level. But of course, if if all those things are working perfectly, but together they create, I guess you could say, a cacophony of processes and it that does not achieve what you're doing. You know, um in which context or how uh can we build that predictability? Because in the end, let's say to get all these things organized, processes that are very complex, in the end, you know, fundamentally they will never be perfect, but at least they have to yield a level of confidence that can that can that can make the operators feel, look, all right, we might not know, but we can definitely bet that it's going to work this way. So what enables you know that operational confidence? What what why is predictability uh more valuable than that just you know uh peak benchmark my uh benchmark numbers and what what it means to what we're doing?

Juan Casal, Cires21 11:22

Sure, this happens sure but predictability is is a main fact that we we need on uh when operating in video systems, uh that's for sure. Because when in operations, uh what you want guarantees. You so so you you don't relate to let's say marketing numbers. Okay, this is very good in some feature, but at the end, as we said, we want stability. We want something uh that we can guess how it will work in the in the under different conditions, uh something that we want when whenever we go to production and and variability happens, or uh we we we go for worse conditions, let's say, uh it's able to absorb some of those worse conditions. Okay, so what do things value, really really value? They want to have a limited latency, that's sure. They they want to know what the latency is going to be and how it will behave uh uh over time. They want a stable resource usage so uh that they can guess if if I have three streams and I'm working at uh 60% of my capacity, if I uh put a new fourth stream, I will be around uh 80% of my capacity. This is predictability, and this is what uh really makes you able to plan capacity of your system when you go in in production. Okay, so at the end, I may say that in production, the worst case behavior matters more than the best case behavior behavior.

Leo Nieto, NETINT 13:05

If maybe some maybe a question just to you know create a uh you know a benchmark. But the question is uh if two systems show you know similar average throughput, but at the end the result shows a very different variance, like one performs better than the other, but you know, all all all parameters are relatively similar, you know, how should teams interpret that? What can they do and that what what what does that mean for real uh deployment decisions?

Juan Casal, Cires21 13:39

And the those conditions, uh I would guess uh I think that there's a big uh unless you have uh good observation tools and and and you have your system uh properly designed against failure, it whenever you go to production uh and and you go up at scale, I guess that it it would be very difficult to find why one of them is underperforming against the other. When you if you have quite similar, uh let's say benchmarks in the components. Uh but but but I I would guess that there's some uh I think that this question is related as well to the stability of the of the system. So that I would guess that there is some uh component creating an unstability, or you can have some issues that can as maybe jitter, maybe uh buffers underflows because of of of network burst or this kind of things. But but what it's for sure is that if you don't have your system uh designed for failure prediction, let's say, uh it will be very difficult to understand what's really happening in your system.

Observability Gaps And Hidden Jitter

Leo Nieto, NETINT 15:17

Understood. From from your perspective, you know, in that case, what are some small evaluation choices that teams underestimate and that later become significant problems? As you mentioned, you have to have eyes, you know, across across the board. And as you like as we mentioned before, you know, average can look the averages can look similar.

Juan Casal, Cires21 15:38

And you know, it's rather difficult to to interpret, but from your perspective, I think that the most uh dangerous issues that we underestimate are usually metric and observability gaps. I mean, it's about the same, but when you deploy a system, you you need to know what's the latency you have to measure, you have to measure peaks instead of power ages, you you have to take jitter into account, you have to uh uh and uh take um and align the timestamps into account uh and um and and to be able to characterize bottlenecks in the network, let's let's say these kind of things. Because when you are testing uh in staging, so in a small scale, these kind of errors uh usually don't lead uh to a very uh bad experience, the user experience, let's say, and they are quite easy to find, or let's say not that difficult to find. But when when you go to at scale, these uh kind of issues amplify, are amplified, and and so everything gets a mess. And and and it is very difficult to know the source of the problem, or which which state of the system is the source of the problem, which estate of the system is amplifying the problem, uh, and which is not affecting, and which one is not affecting at all. So the the only way to do that is to have good metrics, to be able to characterize the main video metrics, as we have said, uh jitter timestamps, uh uh proper format, a proper content format, and then without errors and these kind of things, so that whenever thing when there's an issue, you can relate the issue with uh some metric.

Leo Nieto, NETINT 17:41

And to that, you know, uh this was all in the in the hypothetical sense. Do you have any examples of you know, or or an anecdote to share with us where you know everything looked well on the on the metrics, on the lab reports, but then later, you know, caused some issues. Something that you are comfortable to share as an anecdote from you know from a practical sense.

Juan Casal, Cires21 18:04

I think there are plenty, depends on the technology, sure, but uh I mean when you relate to if if you relate to broadcast uh where uh clock accuracy is important or IPTV, where with the low delay, IPTV, where clock accuracy is very important. And and and uh it's very difficult to measure the jitter and at uh at that scale at that scale because uh uh it is nanoseconds of jitter and may affect your the the quality of your decoding. And and this is very uh whenever there's a let's say the network condition that uh changes a bit how the traffic behaves, this takes you in the decoder to higher jitter, what at the end makes the decoder lose some frames and and so given uh a user experience as as if the the the video is failing?

Efficiency As Headroom And Stability

Leo Nieto, NETINT 19:13

It sounds like you know uh a lot of these uh failures or they might not necessarily be attributed to hardware limitations. Uh it could be interpretation errors. Efficiency is often discussed as cost per stream or streams per server. But in real deployments, efficiency shows up in different ways. How does efficiency, in your opinion, appear operationally beyond simple cost savings?

Juan Casal, Cires21 19:39

Well, in my opinion, efficiency creates headroom. I mean, so it gives you uh confidence uh for your for your video service, for your video system. Lower power consumption reduces thermal stress, or lower computational uh needs reduces contention at the end. So everything is related when when we uh think about uh efficiency. Operationally, efficiency means more stability, more better behavior against network burst, and uh less saturation and higher saturation thresholds. Okay. So, in my opinion, it is not everything about cost saving, but efficiency buys time and margin. It allows you to have more confidence in your system and it allows your system to absorb viability and reduce it instability.

Leo Nieto, NETINT 20:42

Yeah, I agree. I agree. It's not just about a compression ratio, it's not about just uh power savings. Everything has to, you know, evolve towards that same goal.

Juan Casal, Cires21 20:53

And um Yes, so it's not about cost being that it's more about how close you operate from your fellow boundary.

Design For Failure From Day One

Leo Nieto, NETINT 21:04

Correct. And of this this brings us like you know, teams introduce uh you know purpose-build accelerators. Um, I mean, we're not gonna talk too much about that because I don't the point of this discussion is not to mention us too much, but you know, alongside GPUs, CPUs, does that change how efficiency uh should be measured just when you have accelerators? Because I mean it's it's it's a it's a very crucial point in any video distribution system. But right now this that big question, uh, you know, CPUs or GPUs, even if PGAs are um are not necessarily able to keep up to this growing video demand, this scalable scalability challenges. And obviously, there's uh there's new technologies, new solutions that come into the question. Obviously, you know, there's a lot of tests and everything, a lot of, like you mentioned earlier, you know, a lot of things to consider. Let's just leave it at that. That. But let's say, since it's not just about compression or throughput and also about margin, you know, about control of costs, etc. You know, if a team is preparing today to deploy a production video system for the first time, just to keep it, let's bring it back to the ground, something very hypothetical, but people are doing this today. What is one thing you will encourage them not to overlook? As just to leave the key takeaway for everybody and leave the door open to come talk to us at NAB. Of course, talk to the teams at Cirus, because uh you guys have obviously a very, very sensible approach to fixing this. So, what do you think about this one?

Juan Casal, Cires21 22:42

Uh more than encouraging them not to overlook, uh what I will encourage them is to design for observability and control failure from day one. I mean, I think that's the main focus uh you should have when designing a video system. You should know where your bottlenecks form, how your laden evolves over time, how the system degrades, and maybe what your capacity limits are, yeah, and what the your capacity limits are. Okay. So a good recommendation on this uh would be, let's say, to test under sustained load, not just under peaks, because sometimes when you uh are testing a uh uh system, you uh you try to uh push the uh put the press the the uh put the system under pressure and uh you try to make test uh a lot of bursters and these kind of things, but sometimes components are are able to absorb those kinds of bursts, so real problems may occur more when you're working on a sustainable law. Okay? So what I would say is that operating a system is more about uh predicting failures than to reacting to them. So that's why observability and is that important, okay.

Leo Nieto, NETINT 24:17

Right. Well, thank you for for for uh for all your savoir-faire, you know, what we take from this obviously is that you know benchmarks rarely fail because you know they're inaccurate, you know, there's always a context in which we have to also consider the data that that that we're reading from, that we're applying, that are applying it on. And you know, when systems scale and incorporate specialized compute, measurement becomes a design discipline. And um it certainly shapes architectural decisions rather than reporting results. And obviously, it's um a much more comprehensive uh approach is uh is needed. You know, since we're doing this before NAB, I hope that people listening to this, you guys are invited to come see us at NAB on our booth. It doesn't have to be obviously a commercial discussion, but we we always like to keep these um uh these musings alive because uh there's always uh challenges to resolve. And I think we have experts here that have uh not just a sensible approach, but I think uh a sensible one. Thank you uh for a technically grounded and very practical uh discussion, and uh we hope to see you guys uh soon at NAB.

Juan Casal, Cires21 25:31

Okay, thank you very much. I hope to see you guys as well. Indeed.

NAB Invite And Sponsor Message

Anita Flejter, NETINT 25:36

Thank you both, Juan. I repeat after Leo, what stands out in this discussion is that benchmarks don't fail because they are inaccurate, they fail because they are incomplete. Thank you for consistently pushing the conversation beyond which number is higher towards what those numbers actually mean in production. For teams preparing for NAB, this is the real shift happening in the industry. Efficiency is no longer just about throughput, it's about stability, comparability, and confidence under sustained load. Because in the end, infrastructure decisions are rarely reversed in spreadsheets, they are reversed in production. Thanks to everyone listening, and we'll see you at the next Voices of Video. This episode of Voices of Video is brought to you by NetInt Technologies. If you are looking for cutting edge video encoding solutions, check out net's products at netint.com.