Data Science x Public Health
This podcast discusses the concepts of data science and public health, and then delves into their intersection, exploring the connection between the two fields in greater detail.
Data Science x Public Health
Sample Size and Power Analysis: Why Every Study Starts With a Number
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Before any study begins, one question determines everything:
How many participants do you need?
Get it wrong, and your study can miss real effects… or waste time, money, and lives.
In this episode, we break down sample size and power analysis — the foundation of every clinical trial and research study. You’ll learn how effect size, power, variability, and significance level all come together to determine the one number that decides whether your study succeeds or fails.
👉 Enjoyed the episode? Follow the show to get new episodes automatically.
If you found the content helpful, consider leaving a rating or review — it helps support the podcast.
For business and sponsorship inquiries, email us at:
📧 contact@bjanalytics.com
Youtube: https://www.youtube.com/@BJANALYTICS
Instagram: https://www.instagram.com/bjanalyticsconsulting/
Twitter/X: https://x.com/BJANALYTICS
Welcome to today's deep dive. Our mission today is to decode the invisible math behind every medical headline you read.
SPEAKER_00Yeah, and we're pulling from our source text the vital balance, sample size and power analysis, which is great because it reveals that long before a single blood draw, researchers have this massive Goldilocks problem to solve.
SPEAKER_01Like exactly how many participants do they actually need.
SPEAKER_00Exactly. Too high, and they're wasting millions of dollars in exposing way too many patients to unnecessary experimental risks.
SPEAKER_01But then if they go too low.
SPEAKER_00You get a type two error, a false negative. I mean, the the drug might actually be the cure we're looking for, but because the study was too small, the math just misses it entirely.
SPEAKER_01Oh, wow. Which is why the FDA requires researchers to prove this math before a study even begins. You have to justify the numbers to protect the human lives involved.
SPEAKER_00Right. What's fascinating here is that you simply cannot experiment on people without mathematically proving your trial actually has a chance to succeed. It's really the strict ethical foundation of study design.
SPEAKER_01Okay, so let's unpack this. Because you can't just guess. Researchers have to calculate the exact number beforehand, what the text calls an a priori analysis. So to understand this, imagine tuning an old radio to hear a really faint broadcast.
SPEAKER_00Okay, I'm with you. What are the dials on this radio?
SPEAKER_01Well, there are four main ones. First is effect size, which is like the actual minimum signal you're trying to hear. Then variability, so the static or background noise in the population.
SPEAKER_00The static.
SPEAKER_01Yeah. And third is your significance level, usually set at 0.05, which is basically your tolerance for false alarms. And finally, statistical power.
SPEAKER_00Which is conventionally 80%.
SPEAKER_01Exactly. It's the probability your radio will actually catch a real signal if it's out there.
SPEAKER_00So the key here is how those dials interact. It's a strict seesaw relationship. If you're trying to hear a really loud signal over very little static, you just don't need a massive antenna.
SPEAKER_01That makes sense.
SPEAKER_00Like the text gives a great example of a blood pressure drug. If you want to prove a five milliliter drop against 15 millimeters of normal background variability, the formula spits out exactly 284 patients.
SPEAKER_01Okay. 284? That seems doable. But wait, what if the expected effect is smaller? Like just a three millimeter drop.
SPEAKER_00Well, because the signal is weaker, your antenna has to get much bigger to catch it. To confidently find that tiny three millimeter drop, your required sample size skyrockets to nearly 800 patients.
SPEAKER_01800? Wow.
SPEAKER_00Yeah. If the expected signal goes down, the participant count absolutely must go up to compensate.
SPEAKER_01Well hold on, I'm a bit confused. We have software doing this heavy lifting now, right? Programs like G Power or R. So if the formulas are so well established for everything from cross-sectional surveys to cohort studies, why do some estimates say over half of published trials still end up being underpowered? Can't they just trust the software?
SPEAKER_00Well, because the software only crunches the numbers. Humans still have to guess the inputs. And running a trial with 800 human beings is incredibly expensive.
SPEAKER_01Ah, I see. So if I'm a researcher trying to get funding, I might be, well, tempted to input a massive booming effect size into the software.
SPEAKER_00Yes, exactly. If we connect this to the bigger picture, by claiming the drug will have a huge, obvious effect, the software tells you that you only need a small, cheap trial.
SPEAKER_01But when the actual effect turns out to be just a whisper.
SPEAKER_00Your small trial doesn't have the statistical power to hear it. It practically guarantees an inconclusive result.
SPEAKER_01Wow. So what does this all mean for post hoc analysis? Because the text explicitly warns about researchers calculating power after a failed trial.
SPEAKER_00Right. That's when they essentially say, hey, the drug totally works, our sample size was just too small to prove it. But doing that is mathematically misleading.
SPEAKER_01Wait, really? Because you can't use the failed results of your study to calculate the statistical power of that exact same study.
SPEAKER_00Exactly. It adds absolutely no new information. Going back to your radio analogy, you can't fail to record a faint broadcast and then use that silent tape to prove your antenna was too small. The broadcast is over.
SPEAKER_01Oh, that is such a good point. So getting the sample size right from the jump is really the ultimate intersection of statistics and ethics.
SPEAKER_00It really is. It dictates whether the experiment even has a fighting chance.
SPEAKER_01Which leaves you with an interesting thought to mull over. The next time you see a news headline declaring an experimental treatment doesn't work, ask yourself did the treatment actually fail, or did the researchers simply fail to include enough people to hear the signal through the noise?