Runtime Arguments

38: The Claude Shannon Episode: Surprise, Entropy, and Bits

• Jim McQuillan & Wolf • Episode 38

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 1:03:37

What is Information Theory?

  • Information is not meaning
  • Likely things need very few bits to communicate, maybe less than 1! Unlikely things need more bits.
  • Entropy is in the eye of the beholder. It’s what you already know that informs your ability to extract meaning from a message. To one person, it’s just noise; to one who holds the encryption key, there may be way less information’s, but m much more valuable meaning.
  • This is the plumbing behind everything you rely on: communication, storage, encryption, compression, error correction.

Come join us for lunch!  Every Saturday, we and several of our friends meet for lunch at 11am at Biwako Sushi in Saline Twp, MI. It's just south of Ann Arbor. If you are in the area, we'd love to have you join us.

Hosts:
Jim McQuillan can be reached at jam@RuntimeArguments.fm
Wolf can be reached at wolf@RuntimeArguments.fm

Follow us on Mastodon: @RuntimeArguments@hachyderm.io

If you have feedback for us, please send it to feedback@RuntimeArguments.fm

Check out our webpage at http://RuntimeArguments.fm

Theme music:
Dawn by nuer self, from the album Digital Sky

People on this episode

Podcasts we love

Check out these other fine podcasts recommended by us, not an algorithm.

CoRecursive: Coding Stories Artwork

CoRecursive: Coding Stories

Adam Gordon Bell - Software Developer
Two's Complement Artwork

Two's Complement

Ben Rady and Matt Godbolt
Accidental Tech Podcast Artwork

Accidental Tech Podcast

Marco Arment, Casey Liss, John Siracusa
Python Bytes Artwork

Python Bytes

Michael Kennedy and Calvin Hendryx-Parker
Talk Python To Me Artwork

Talk Python To Me

Michael Kennedy