{"version":"1.0.0","segments":[{"startTime":0.42,"endTime":1.303,"body":"- Hello."},{"startTime":2.82,"endTime":5.01,"body":"When I was writing this lecture,"},{"startTime":5.01,"endTime":8.33,"body":"I thought about the data rates"},{"startTime":8.33,"endTime":11.493,"body":"that were whizzing around my computer."},{"startTime":12.43,"endTime":16.5,"body":"And I realized that a modern"},{"startTime":12.43,"endTime":16.5,"body":"computer sort of juggles data"},{"startTime":16.5,"endTime":21.5,"body":"from memory to disk at"},{"startTime":16.5,"endTime":21.5,"body":"around 500 megabits a second."},{"startTime":22.77,"endTime":27.183,"body":"And so if we start sort of"},{"startTime":22.77,"endTime":27.183,"body":"multiplying that up, you know,"},{"startTime":29.878,"endTime":32.716,"body":"what's that, 500 megabits"},{"startTime":29.878,"endTime":32.716,"body":"per second times 60,"},{"startTime":32.716,"endTime":35.76,"body":"that's 30 gigabits a minute, times 60,"},{"startTime":35.76,"endTime":39.05,"body":"that's 1.8 terabytes an hour."},{"startTime":39.05,"endTime":40.73,"body":"And then, well, I'm a computer scientist,"},{"startTime":40.73,"endTime":44.16,"body":"so I work 10 hours a day of"},{"startTime":40.73,"endTime":44.16,"body":"course, unlike ordinary mortals,"},{"startTime":44.16,"endTime":48.55,"body":"so that's 18 terabytes of information"},{"startTime":48.55,"endTime":52.94,"body":"being shuffled around in a day."},{"startTime":52.94,"endTime":57.94,"body":"And if one of those bits is"},{"startTime":52.94,"endTime":57.94,"body":"wrong, your computer will crash."},{"startTime":59.34,"endTime":61.91,"body":"And when was the last time"},{"startTime":59.34,"endTime":61.91,"body":"your computer crashed?"},{"startTime":61.91,"endTime":65.98,"body":"I don't just mean did something"},{"startTime":61.91,"endTime":65.98,"body":"that was a bit irritating."},{"startTime":65.98,"endTime":69.24,"body":"I mean, blue screen of death type crash."},{"startTime":69.24,"endTime":72.13,"body":"Well, I mean, this used"},{"startTime":69.24,"endTime":72.13,"body":"to happen all the time,"},{"startTime":72.13,"endTime":74.15,"body":"but now, I mean, I can't"},{"startTime":72.13,"endTime":74.15,"body":"remember the last time"},{"startTime":74.15,"endTime":78.01,"body":"it happened with me,"},{"startTime":74.15,"endTime":78.01,"body":"maybe last year some time,"},{"startTime":78.01,"endTime":81.37,"body":"after I'd done something"},{"startTime":78.01,"endTime":81.37,"body":"particularly stupid, no doubt."},{"startTime":81.37,"endTime":82.8,"body":"So if you think about this,"},{"startTime":82.8,"endTime":86.53,"body":"it's a sort of wildly"},{"startTime":82.8,"endTime":86.53,"body":"improbable situation."},{"startTime":86.53,"endTime":88.73,"body":"We humans have designed a system"},{"startTime":89.67,"endTime":92.48,"body":"with billions of electrical parts,"},{"startTime":92.48,"endTime":94.13,"body":"that's operating at an error rate"},{"startTime":94.13,"endTime":96.543,"body":"of less than one in 10 to the 13."},{"startTime":98.22,"endTime":102.67,"body":"About 20 terabytes of"},{"startTime":98.22,"endTime":102.67,"body":"information, our daily figure."},{"startTime":102.67,"endTime":104.32,"body":"It's hard to visualize, isn't it?"},{"startTime":105.86,"endTime":107.82,"body":"There are roughly eight"},{"startTime":105.86,"endTime":107.82,"body":"billion people on the planet."},{"startTime":107.82,"endTime":109.83,"body":"So imagine them, the"},{"startTime":107.82,"endTime":109.83,"body":"way I thought of it is,"},{"startTime":109.83,"endTime":112.6,"body":"well imagine we asked them to toss a coin"},{"startTime":112.6,"endTime":114.27,"body":"2000 times in a day,"},{"startTime":114.27,"endTime":117.7,"body":"and then they had to log in"},{"startTime":114.27,"endTime":117.7,"body":"to a website, some website,"},{"startTime":117.7,"endTime":120.213,"body":"and log all of those coin tosses."},{"startTime":121.63,"endTime":123.39,"body":"Could we be confident that they could log"},{"startTime":123.39,"endTime":128.39,"body":"all of those tosses in a website"},{"startTime":123.39,"endTime":128.39,"body":"without any errors at all?"},{"startTime":129.07,"endTime":132.09,"body":"I mean if I said it like"},{"startTime":129.07,"endTime":132.09,"body":"that, it seems absurd,"},{"startTime":132.09,"endTime":136.723,"body":"yet this is the daily"},{"startTime":132.09,"endTime":136.723,"body":"business of computing."},{"startTime":137.63,"endTime":140.68,"body":"What makes it even more surprising is that"},{"startTime":140.68,"endTime":142.89,"body":"electronic devices and"},{"startTime":140.68,"endTime":142.89,"body":"communication channels"},{"startTime":142.89,"endTime":145.49,"body":"make errors all the time."},{"startTime":145.49,"endTime":149.15,"body":"Underneath this apparent"},{"startTime":145.49,"endTime":149.15,"body":"error-free behavior"},{"startTime":149.15,"endTime":151.973,"body":"is a massively errorful system."},{"startTime":153.09,"endTime":155.51,"body":"So we don't notice those errors"},{"startTime":155.51,"endTime":158.9,"body":"because of a brilliant"},{"startTime":155.51,"endTime":158.9,"body":"bit of information theory"},{"startTime":158.9,"endTime":161.49,"body":"known as error control coding."},{"startTime":161.49,"endTime":164.013,"body":"And that's what this lecture is about."},{"startTime":165.3,"endTime":167.06,"body":"Now, as far as I can tell,"},{"startTime":167.06,"endTime":170.42,"body":"the early history of error control coding"},{"startTime":170.42,"endTime":174.123,"body":"isn't really very well described or known."},{"startTime":175.61,"endTime":176.713,"body":"In a nutshell,"},{"startTime":177.67,"endTime":181.57,"body":"nothing really happened in"},{"startTime":177.67,"endTime":181.57,"body":"error control coding until 1948,"},{"startTime":181.57,"endTime":185.73,"body":"when Claude Shannon published"},{"startTime":181.57,"endTime":185.73,"body":"his brilliant paper,"},{"startTime":185.73,"endTime":188.76,"body":"and that was the start"},{"startTime":185.73,"endTime":188.76,"body":"of information theory"},{"startTime":188.76,"endTime":190.51,"body":"and that was the start of a subject."},{"startTime":190.51,"endTime":191.61,"body":"I'll talk a little bit more about"},{"startTime":191.61,"endTime":194.623,"body":"Shannon's contributions later."},{"startTime":197.32,"endTime":202.01,"body":"There was some error detection,"},{"startTime":197.32,"endTime":202.01,"body":"and perhaps some error,"},{"startTime":202.01,"endTime":204.163,"body":"yeah, some error detection earlier."},{"startTime":205.44,"endTime":210.44,"body":"And I suppose the earliest"},{"startTime":205.44,"endTime":210.44,"body":"form of communication systems,"},{"startTime":210.5,"endTime":212.2,"body":"the ones that we're familiar with,"},{"startTime":213.67,"endTime":216.2,"body":"proved a model for"},{"startTime":213.67,"endTime":216.2,"body":"electronic communication."},{"startTime":216.2,"endTime":219.76,"body":"So, \"I'm sorry, could you say that again,\""},{"startTime":219.76,"endTime":223.21,"body":"that's a form of error control called ARQ,"},{"startTime":223.21,"endTime":228.21,"body":"automatic repeat, request, and"},{"startTime":223.21,"endTime":228.21,"body":"computers do still use that."},{"startTime":229.33,"endTime":231.51,"body":"The internet uses ARQ."},{"startTime":231.51,"endTime":236.51,"body":"So if your TCP/IP packet doesn't"},{"startTime":231.51,"endTime":236.51,"body":"appear in the right order,"},{"startTime":237.06,"endTime":239.11,"body":"it gets lost on the way"},{"startTime":237.06,"endTime":239.11,"body":"or has got errors in it,"},{"startTime":239.11,"endTime":242.09,"body":"and I'll talk about how we"},{"startTime":239.11,"endTime":242.09,"body":"find those errors in a moment,"},{"startTime":242.09,"endTime":246.01,"body":"your receiver says, \"Could"},{"startTime":242.09,"endTime":246.01,"body":"you send that again, please?\""},{"startTime":246.01,"endTime":247.72,"body":"That's ARQ, and there's some evidence"},{"startTime":247.72,"endTime":251.17,"body":"that morse transmissions used to do that."},{"startTime":251.17,"endTime":254.32,"body":"I asked the Royal Signals"},{"startTime":251.17,"endTime":254.32,"body":"and Radar Museum about this,"},{"startTime":254.32,"endTime":256.46,"body":"and they said it was quite"},{"startTime":254.32,"endTime":256.46,"body":"frequent for operators"},{"startTime":256.46,"endTime":258.563,"body":"to ask people to say again."},{"startTime":260.42,"endTime":263.91,"body":"But I'm really interested in talking about"},{"startTime":263.91,"endTime":266.23,"body":"forward error correction,"},{"startTime":266.23,"endTime":269.11,"body":"and forward error correction"},{"startTime":266.23,"endTime":269.11,"body":"is the business of"},{"startTime":269.11,"endTime":270.94,"body":"sending some side information"},{"startTime":270.94,"endTime":273.71,"body":"alongside your original message,"},{"startTime":273.71,"endTime":278.423,"body":"that allows the receiver to"},{"startTime":273.71,"endTime":278.423,"body":"know that an error has occurred."},{"startTime":279.64,"endTime":282.65,"body":"And here's an example of that."},{"startTime":282.65,"endTime":284.153,"body":"This is an early telegram."},{"startTime":286.1,"endTime":288.02,"body":"For those of you who don't"},{"startTime":286.1,"endTime":288.02,"body":"know, it's a bit odd,"},{"startTime":288.02,"endTime":291.78,"body":"but what used to happen was,"},{"startTime":288.02,"endTime":291.78,"body":"I expect everyone does know,"},{"startTime":291.78,"endTime":295.27,"body":"but the idea was if you wanted"},{"startTime":291.78,"endTime":295.27,"body":"to send a message rapidly,"},{"startTime":295.27,"endTime":297.48,"body":"you would write it down"},{"startTime":295.27,"endTime":297.48,"body":"on a telegram form."},{"startTime":297.48,"endTime":299.73,"body":"You would then hand it"},{"startTime":297.48,"endTime":299.73,"body":"to a telegraph clerk"},{"startTime":299.73,"endTime":303.15,"body":"who would then send it via"},{"startTime":299.73,"endTime":303.15,"body":"morse code electrically,"},{"startTime":303.15,"endTime":304.51,"body":"and then the other end"},{"startTime":303.15,"endTime":304.51,"body":"it would be written down"},{"startTime":304.51,"endTime":307.203,"body":"by a telegraph clerk and"},{"startTime":304.51,"endTime":307.203,"body":"then they would charge you."},{"startTime":308.76,"endTime":310.38,"body":"And it was quite common apparently"},{"startTime":310.38,"endTime":314.2,"body":"to write the number of words"},{"startTime":310.38,"endTime":314.2,"body":"down, a sort of word count."},{"startTime":314.2,"endTime":317.0,"body":"So if the receiver's"},{"startTime":314.2,"endTime":317.0,"body":"word count didn't match"},{"startTime":317.0,"endTime":318.77,"body":"the word count on the telegram,"},{"startTime":318.77,"endTime":321.29,"body":"you knew that there'd been an error."},{"startTime":321.29,"endTime":323.32,"body":"And I think it's here,"},{"startTime":323.32,"endTime":325.29,"body":"there's an indication at the top of this,"},{"startTime":325.29,"endTime":326.67,"body":"how many words were paid for."},{"startTime":326.67,"endTime":327.98,"body":"Generally you paid by the word."},{"startTime":327.98,"endTime":330.83,"body":"And I think it says 129 characters there,"},{"startTime":330.83,"endTime":333.81,"body":"and I did do a quick count and"},{"startTime":330.83,"endTime":333.81,"body":"it seemed to be about right."},{"startTime":333.81,"endTime":338.33,"body":"Now, that's a form of"},{"startTime":333.81,"endTime":338.33,"body":"forward error correction,"},{"startTime":338.33,"endTime":340.603,"body":"sorry, error control, known as a checksum."},{"startTime":341.51,"endTime":343.293,"body":"So you just add up something."},{"startTime":344.69,"endTime":348.88,"body":"The TCP/IP packets that"},{"startTime":344.69,"endTime":348.88,"body":"are used in the internet"},{"startTime":348.88,"endTime":351.65,"body":"are quite, they use a checksum."},{"startTime":351.65,"endTime":353.95,"body":"So in the header of those packets"},{"startTime":353.95,"endTime":357.77,"body":"they add up how many ones"},{"startTime":353.95,"endTime":357.77,"body":"are in the lump of data"},{"startTime":357.77,"endTime":360.36,"body":"that you're sending, they write that down,"},{"startTime":360.36,"endTime":363.027,"body":"and then on reception, you"},{"startTime":360.36,"endTime":363.027,"body":"add up the number of ones,"},{"startTime":363.027,"endTime":367.253,"body":"and if it doesn't match,"},{"startTime":363.027,"endTime":367.253,"body":"you say send again."},{"startTime":369.21,"endTime":371.39,"body":"Now I'd really like to focus I think on"},{"startTime":371.39,"endTime":374.22,"body":"forward error correction,"},{"startTime":371.39,"endTime":374.22,"body":"and that doesn't really have"},{"startTime":374.22,"endTime":377.513,"body":"a precise analog in human communication."},{"startTime":379.06,"endTime":381.44,"body":"There's a little bit of an analog,"},{"startTime":381.44,"endTime":385.24,"body":"which is humans use quite"},{"startTime":381.44,"endTime":385.24,"body":"a bit of side knowledge,"},{"startTime":385.24,"endTime":389.57,"body":"common knowledge, in order"},{"startTime":385.24,"endTime":389.57,"body":"to work out what was said,"},{"startTime":389.57,"endTime":392.03,"body":"for example, in spoken communication."},{"startTime":392.03,"endTime":395.06,"body":"But here we're talking about something"},{"startTime":395.06,"endTime":398.373,"body":"a bit more precise than that."},{"startTime":399.29,"endTime":400.47,"body":"And let's start with"},{"startTime":400.47,"endTime":404.443,"body":"the earliest form that I"},{"startTime":400.47,"endTime":404.443,"body":"could find, called parity."},{"startTime":406.99,"endTime":409.25,"body":"And if there's anyone listening,"},{"startTime":409.25,"endTime":411.37,"body":"or anyone in the audience"},{"startTime":409.25,"endTime":411.37,"body":"who is a scholar,"},{"startTime":411.37,"endTime":413.03,"body":"I think it would be quite fun to know"},{"startTime":413.03,"endTime":415.6,"body":"when parity really dates from."},{"startTime":415.6,"endTime":416.623,"body":"I don't really know."},{"startTime":417.695,"endTime":420.03,"body":"I wasted, wasted is the wrong word,"},{"startTime":420.03,"endTime":422.93,"body":"I spent many a happy"},{"startTime":420.03,"endTime":422.93,"body":"hour trying to work out"},{"startTime":422.93,"endTime":426.35,"body":"who'd thought of parity first,"},{"startTime":422.93,"endTime":426.35,"body":"and still not entirely sure."},{"startTime":426.35,"endTime":428.53,"body":"But let's leave the history aside,"},{"startTime":428.53,"endTime":430.79,"body":"I'll just explain how it works."},{"startTime":430.79,"endTime":434.19,"body":"So the idea behind parity is really nice."},{"startTime":434.19,"endTime":435.98,"body":"Let's say we've got a message."},{"startTime":435.98,"endTime":439.76,"body":"So this is my binary"},{"startTime":435.98,"endTime":439.76,"body":"message and it says 1011101."},{"startTime":441.46,"endTime":442.67,"body":"Probably means something to somebody,"},{"startTime":442.67,"endTime":445.28,"body":"but we don't have to worry"},{"startTime":442.67,"endTime":445.28,"body":"about the meaning here."},{"startTime":445.28,"endTime":449.02,"body":"And the idea is we're going to add up"},{"startTime":449.02,"endTime":451.91,"body":"how many ones are in a particular block."},{"startTime":451.91,"endTime":455.87,"body":"So the parity works with a block of data."},{"startTime":455.87,"endTime":459.86,"body":"And if the number of ones is not even,"},{"startTime":459.86,"endTime":463.79,"body":"then I'm going to make it"},{"startTime":459.86,"endTime":463.79,"body":"even by adding another one."},{"startTime":463.79,"endTime":467.44,"body":"So the analysis says,"},{"startTime":463.79,"endTime":467.44,"body":"well, there's five ones,"},{"startTime":467.44,"endTime":469.66,"body":"an odd number, that's"},{"startTime":467.44,"endTime":469.66,"body":"not acceptable to me,"},{"startTime":469.66,"endTime":472.743,"body":"'cause I'm going to work in"},{"startTime":469.66,"endTime":472.743,"body":"this here with even parity."},{"startTime":474.38,"endTime":479.18,"body":"Let's make it even, we'll add on a one,"},{"startTime":479.18,"endTime":481.203,"body":"and that's the message"},{"startTime":479.18,"endTime":481.203,"body":"we're going to send."},{"startTime":482.33,"endTime":486.71,"body":"So zoom, off it goes, down"},{"startTime":482.33,"endTime":486.71,"body":"some channel or onto a disk,"},{"startTime":486.71,"endTime":488.06,"body":"or something like that."},{"startTime":488.06,"endTime":490.99,"body":"Then the channel has"},{"startTime":488.06,"endTime":490.99,"body":"noise, there's some errors,"},{"startTime":490.99,"endTime":494.04,"body":"some corruptions, and that's"},{"startTime":490.99,"endTime":494.04,"body":"what I'm showing here in pink."},{"startTime":494.04,"endTime":499.04,"body":"Okay, so you can probably"},{"startTime":494.04,"endTime":499.04,"body":"see that this digit one here"},{"startTime":500.06,"endTime":501.96,"body":"has been converted into a zero."},{"startTime":501.96,"endTime":503.08,"body":"So that's an error."},{"startTime":503.08,"endTime":508.08,"body":"And on reception I say"},{"startTime":503.08,"endTime":508.08,"body":"five ones, that's not even,"},{"startTime":508.75,"endTime":511.39,"body":"something bad must've"},{"startTime":508.75,"endTime":511.39,"body":"happened, send it again."},{"startTime":511.39,"endTime":513.16,"body":"So that's the basis of parity."},{"startTime":513.16,"endTime":515.58,"body":"And you can see that"},{"startTime":513.16,"endTime":515.58,"body":"you might think, well,"},{"startTime":515.58,"endTime":516.8,"body":"how does parity work,"},{"startTime":516.8,"endTime":518.8,"body":"what if there's an error"},{"startTime":516.8,"endTime":518.8,"body":"in the parity digit,"},{"startTime":518.8,"endTime":519.853,"body":"it still works."},{"startTime":521.39,"endTime":524.08,"body":"Now what it doesn't do of"},{"startTime":521.39,"endTime":524.08,"body":"course is it doesn't allow you"},{"startTime":524.08,"endTime":528.42,"body":"to correct the errors, it"},{"startTime":524.08,"endTime":528.42,"body":"just signals the errors."},{"startTime":528.42,"endTime":532.45,"body":"And you might say, if you"},{"startTime":528.42,"endTime":532.45,"body":"know something about hardware,"},{"startTime":532.45,"endTime":535.4,"body":"you might be sort of cringing"},{"startTime":532.45,"endTime":535.4,"body":"at this moment thinking, well,"},{"startTime":535.4,"endTime":537.07,"body":"I've got to build some low level hardware"},{"startTime":537.07,"endTime":540.34,"body":"to add up a large number of things."},{"startTime":540.34,"endTime":542.67,"body":"That's a bit complicated, isn't it?"},{"startTime":542.67,"endTime":545.03,"body":"And well, nowadays you"},{"startTime":542.67,"endTime":545.03,"body":"can do anything can't you,"},{"startTime":545.03,"endTime":547.6,"body":"but it is a bit complicated,"},{"startTime":545.03,"endTime":547.6,"body":"and if put like that,"},{"startTime":547.6,"endTime":549.81,"body":"it seems a bit unpleasant."},{"startTime":549.81,"endTime":551.92,"body":"But it turns out that you can do it"},{"startTime":551.92,"endTime":556.92,"body":"a little bit more cheaply,"},{"startTime":551.92,"endTime":556.92,"body":"using some very common hardware."},{"startTime":556.97,"endTime":561.0,"body":"And that common hardware"},{"startTime":556.97,"endTime":561.0,"body":"is called the exclusive or."},{"startTime":561.0,"endTime":562.22,"body":"The exclusive or,"},{"startTime":562.22,"endTime":567.22,"body":"so digital logic has"},{"startTime":562.22,"endTime":567.22,"body":"quite a few operations,"},{"startTime":567.33,"endTime":569.13,"body":"there's an and operation,"},{"startTime":567.33,"endTime":569.13,"body":"there's an or operation,"},{"startTime":569.13,"endTime":572.17,"body":"and here we're going to talk"},{"startTime":569.13,"endTime":572.17,"body":"about the exclusive or operation."},{"startTime":572.17,"endTime":575.98,"body":"The exclusive or operation"},{"startTime":572.17,"endTime":575.98,"body":"is a bit like the one we use"},{"startTime":575.98,"endTime":576.82,"body":"in everyday speech."},{"startTime":576.82,"endTime":581.25,"body":"So you might say Richard is"},{"startTime":576.82,"endTime":581.25,"body":"either a computer scientist"},{"startTime":581.25,"endTime":583.86,"body":"or a poet, and the"},{"startTime":581.25,"endTime":583.86,"body":"implication behind that is"},{"startTime":583.86,"endTime":586.35,"body":"he's either a computer"},{"startTime":583.86,"endTime":586.35,"body":"scientist or he's a poet,"},{"startTime":586.35,"endTime":587.76,"body":"but he can't be both."},{"startTime":587.76,"endTime":590.31,"body":"That's called the exclusive"},{"startTime":587.76,"endTime":590.31,"body":"or, and it works like this."},{"startTime":590.31,"endTime":593.8,"body":"So if we have two zeros, which you might,"},{"startTime":593.8,"endTime":596.98,"body":"logicians would refer to as two falses,"},{"startTime":596.98,"endTime":598.89,"body":"the XOR gives a zero."},{"startTime":598.89,"endTime":603.44,"body":"It only gives a one if"},{"startTime":598.89,"endTime":603.44,"body":"either B is one or A is one."},{"startTime":603.44,"endTime":606.01,"body":"If they're both one, it gives a zero."},{"startTime":606.01,"endTime":608.55,"body":"And if I write here in"},{"startTime":606.01,"endTime":608.55,"body":"this column even parity,"},{"startTime":608.55,"endTime":609.72,"body":"I think you can see immediately"},{"startTime":609.72,"endTime":613.5,"body":"that even parity is the same"},{"startTime":609.72,"endTime":613.5,"body":"as the binary exclusive or,"},{"startTime":613.5,"endTime":615.84,"body":"and the good news about this,"},{"startTime":613.5,"endTime":615.84,"body":"the reason I labor this point,"},{"startTime":615.84,"endTime":619.95,"body":"is that you can go out and buy"},{"startTime":615.84,"endTime":619.95,"body":"a binary exclusive or gate,"},{"startTime":619.95,"endTime":620.87,"body":"and they're very cheap."},{"startTime":620.87,"endTime":623.923,"body":"So they fit onto silicon very easily."},{"startTime":624.98,"endTime":627.14,"body":"If you are really into this"},{"startTime":627.14,"endTime":629.16,"body":"and you go and follow up"},{"startTime":627.14,"endTime":629.16,"body":"some of the literature,"},{"startTime":629.16,"endTime":632.023,"body":"what you'll see is that"},{"startTime":629.16,"endTime":632.023,"body":"binary is a bit of a faff,"},{"startTime":633.76,"endTime":635.28,"body":"using the XOR symbol."},{"startTime":635.28,"endTime":637.01,"body":"So we'll often use this"},{"startTime":635.28,"endTime":637.01,"body":"symbol in the middle,"},{"startTime":637.01,"endTime":639.41,"body":"just a good old fashioned plus."},{"startTime":639.41,"endTime":642.1,"body":"And when I was learning digital logic,"},{"startTime":642.1,"endTime":644.01,"body":"this caused me great confusion because,"},{"startTime":644.01,"endTime":645.61,"body":"perhaps I shouldn't confess this,"},{"startTime":645.61,"endTime":649.73,"body":"I missed the lecture where"},{"startTime":645.61,"endTime":649.73,"body":"the lecturer defined plus"},{"startTime":649.73,"endTime":654.31,"body":"to mean exclusive or,"},{"startTime":649.73,"endTime":654.31,"body":"and for hours afterwards"},{"startTime":654.31,"endTime":656.17,"body":"I couldn't work out what"},{"startTime":654.31,"endTime":656.17,"body":"the hell he was doing."},{"startTime":656.17,"endTime":658.7,"body":"He didn't seem to be"},{"startTime":656.17,"endTime":658.7,"body":"adding numbers at all."},{"startTime":658.7,"endTime":660.193,"body":"It is a bit like an addition."},{"startTime":661.58,"endTime":663.14,"body":"That's why we use the plus symbol."},{"startTime":663.14,"endTime":665.5,"body":"But it's an addition where"},{"startTime":663.14,"endTime":665.5,"body":"you don't carry anything."},{"startTime":665.5,"endTime":668.41,"body":"So if you think of doing,"},{"startTime":665.5,"endTime":668.41,"body":"adding these numbers in binary,"},{"startTime":668.41,"endTime":670.52,"body":"zero plus zero equals zero, yes."},{"startTime":670.52,"endTime":672.46,"body":"Zero plus one equals one, yes."},{"startTime":672.46,"endTime":673.95,"body":"One plus zero equals one."},{"startTime":673.95,"endTime":678.063,"body":"One plus one, well that"},{"startTime":673.95,"endTime":678.063,"body":"equals two, which is one zero."},{"startTime":679.26,"endTime":680.3,"body":"Ignore the carry."},{"startTime":680.3,"endTime":681.8,"body":"So it's just zero."},{"startTime":681.8,"endTime":686.603,"body":"Okay, so we also think of"},{"startTime":681.8,"endTime":686.603,"body":"exclusive or as a addition,"},{"startTime":687.62,"endTime":690.02,"body":"and it's an addition"},{"startTime":687.62,"endTime":690.02,"body":"in a particular type of"},{"startTime":690.02,"endTime":692.47,"body":"mathematical object, actually"},{"startTime":690.02,"endTime":692.47,"body":"called a Galois field,"},{"startTime":692.47,"endTime":693.81,"body":"if you're interested."},{"startTime":693.81,"endTime":697.64,"body":"There's a wonderful set of"},{"startTime":693.81,"endTime":697.64,"body":"theoretical underpinnings"},{"startTime":697.64,"endTime":699.79,"body":"to coding, based upon Galois field theory."},{"startTime":699.79,"endTime":702.71,"body":"Galois field is a set of"},{"startTime":699.79,"endTime":702.71,"body":"numbers and some operations,"},{"startTime":702.71,"endTime":705.03,"body":"and you can define on GF2,"},{"startTime":705.03,"endTime":707.63,"body":"which is the numbers"},{"startTime":705.03,"endTime":707.63,"body":"we're dealing with here,"},{"startTime":707.63,"endTime":710.463,"body":"plus, minus, times and division."},{"startTime":711.3,"endTime":716.3,"body":"Minus, in our wonderful"},{"startTime":711.3,"endTime":716.3,"body":"world here, subtraction,"},{"startTime":717.0,"endTime":719.18,"body":"is exactly the same as plus."},{"startTime":719.18,"endTime":722.28,"body":"So that's brilliant too,"},{"startTime":719.18,"endTime":722.28,"body":"'cause it means even I,"},{"startTime":722.28,"endTime":724.29,"body":"a thicko when it comes to doing maths,"},{"startTime":724.29,"endTime":728.683,"body":"can do addition and subtraction in GF2."},{"startTime":729.7,"endTime":733.61,"body":"Well, even this simple system"},{"startTime":729.7,"endTime":733.61,"body":"finds some application,"},{"startTime":733.61,"endTime":735.803,"body":"and the simplest one I could think of was,"},{"startTime":736.74,"endTime":739.65,"body":"something called the rapid"},{"startTime":736.74,"endTime":739.65,"body":"array of independent,"},{"startTime":739.65,"endTime":741.94,"body":"redundant array of independent drives,"},{"startTime":741.94,"endTime":743.53,"body":"otherwise known as RAID."},{"startTime":743.53,"endTime":745.46,"body":"So when somebody says"},{"startTime":743.53,"endTime":745.46,"body":"they have a RAID drive,"},{"startTime":745.46,"endTime":748.31,"body":"what it means is they don't"},{"startTime":745.46,"endTime":748.31,"body":"just have one disk drive."},{"startTime":748.31,"endTime":750.303,"body":"They've got a whole army of,"},{"startTime":750.303,"endTime":752.34,"body":"a whole battalion of"},{"startTime":750.303,"endTime":752.34,"body":"these things stacked up,"},{"startTime":752.34,"endTime":753.893,"body":"in this case we have three."},{"startTime":754.78,"endTime":756.97,"body":"And if we've got some data"},{"startTime":756.97,"endTime":759.08,"body":"that we want to put onto those drives,"},{"startTime":759.08,"endTime":762.24,"body":"then this is a relatively"},{"startTime":759.08,"endTime":762.24,"body":"early form of RAID,"},{"startTime":762.24,"endTime":765.34,"body":"designed to provide redundancy."},{"startTime":765.34,"endTime":768.3,"body":"I should say, RAID is sometimes"},{"startTime":765.34,"endTime":768.3,"body":"used to provide redundancy,"},{"startTime":768.3,"endTime":771.7,"body":"but it's often also sometimes"},{"startTime":768.3,"endTime":771.7,"body":"used to provide speed."},{"startTime":771.7,"endTime":773.34,"body":"I'm not talking about the speed bit here,"},{"startTime":773.34,"endTime":775.15,"body":"it's just redundancy."},{"startTime":775.15,"endTime":776.28,"body":"So what we might do is we say"},{"startTime":776.28,"endTime":778.25,"body":"we're going to put that block on there,"},{"startTime":778.25,"endTime":780.223,"body":"and that block on there."},{"startTime":781.21,"endTime":786.163,"body":"And then we're going"},{"startTime":781.21,"endTime":786.163,"body":"to XOR those together."},{"startTime":789.41,"endTime":790.243,"body":"Which is that."},{"startTime":790.243,"endTime":792.83,"body":"So just to quickly check that,"},{"startTime":790.243,"endTime":792.83,"body":"I hope I've got that right."},{"startTime":792.83,"endTime":796.75,"body":"Wherever you see these the"},{"startTime":792.83,"endTime":796.75,"body":"same, you should get a zero."},{"startTime":796.75,"endTime":800.64,"body":"Wherever they're different,"},{"startTime":796.75,"endTime":800.64,"body":"we should get a one."},{"startTime":800.64,"endTime":803.16,"body":"And we'll put that on the third drive."},{"startTime":803.16,"endTime":806.05,"body":"So if one of those"},{"startTime":803.16,"endTime":806.05,"body":"drives went up in smoke,"},{"startTime":806.05,"endTime":811.05,"body":"then we could extract"},{"startTime":806.05,"endTime":811.05,"body":"the two redundant drives."},{"startTime":811.89,"endTime":814.35,"body":"And instead of adding them"},{"startTime":811.89,"endTime":814.35,"body":"together, we subtract them,"},{"startTime":814.35,"endTime":816.254,"body":"which is the same thing,"},{"startTime":816.254,"endTime":818.88,"body":"in our wonderful world"},{"startTime":816.254,"endTime":818.88,"body":"of binary arithmetic."},{"startTime":818.88,"endTime":821.01,"body":"And we can recreate the original."},{"startTime":821.01,"endTime":825.34,"body":"So there's a good example"},{"startTime":821.01,"endTime":825.34,"body":"of how parity saves the day."},{"startTime":825.34,"endTime":827.67,"body":"Now it's come at quite"},{"startTime":825.34,"endTime":827.67,"body":"a cost, I have to admit,"},{"startTime":827.67,"endTime":831.65,"body":"we're having to use more drives"},{"startTime":831.65,"endTime":835.24,"body":"than we would really like here,"},{"startTime":831.65,"endTime":835.24,"body":"in order to get redundancy."},{"startTime":838.08,"endTime":840.37,"body":"Now, what might not be"},{"startTime":838.08,"endTime":840.37,"body":"evident at this point,"},{"startTime":840.37,"endTime":842.43,"body":"but now is a good time to introduce it,"},{"startTime":842.43,"endTime":845.63,"body":"is that the communication"},{"startTime":842.43,"endTime":845.63,"body":"system I'm describing"},{"startTime":845.63,"endTime":848.12,"body":"is sort of generically"},{"startTime":845.63,"endTime":848.12,"body":"described like this."},{"startTime":848.12,"endTime":851.37,"body":"And this is the, it's called"},{"startTime":848.12,"endTime":851.37,"body":"the Shannon-Weaver model"},{"startTime":851.37,"endTime":854.593,"body":"after two scientists, Claude"},{"startTime":851.37,"endTime":854.593,"body":"Shannon, who wrote this,"},{"startTime":857.087,"endTime":860.01,"body":"\"A Mathematical Theory of"},{"startTime":857.087,"endTime":860.01,"body":"Communication\" in 1948."},{"startTime":861.69,"endTime":866.02,"body":"And Shannon said, I"},{"startTime":861.69,"endTime":866.02,"body":"can't remember if he said"},{"startTime":866.02,"endTime":867.3,"body":"every communication system"},{"startTime":867.3,"endTime":869.24,"body":"or every electronic communication system."},{"startTime":869.24,"endTime":870.78,"body":"Let's say, let's be kind and say he said"},{"startTime":870.78,"endTime":872.64,"body":"every electronic communication system,"},{"startTime":872.64,"endTime":875.13,"body":"since being much more widely applied,"},{"startTime":875.13,"endTime":877.76,"body":"much to the aggravation of"},{"startTime":875.13,"endTime":877.76,"body":"social scientists, I should say,"},{"startTime":877.76,"endTime":881.083,"body":"who frequently complain this"},{"startTime":877.76,"endTime":881.083,"body":"is far too simple a model."},{"startTime":884.2,"endTime":885.08,"body":"It looks a bit like this."},{"startTime":885.08,"endTime":886.81,"body":"We've got something over"},{"startTime":885.08,"endTime":886.81,"body":"here sending information"},{"startTime":886.81,"endTime":887.89,"body":"on the left hand side."},{"startTime":887.89,"endTime":889.64,"body":"That goes through some"},{"startTime":887.89,"endTime":889.64,"body":"sort of transmission."},{"startTime":889.64,"endTime":891.2,"body":"You might call that modulation."},{"startTime":891.2,"endTime":893.61,"body":"Then noise isn't necessarily added,"},{"startTime":893.61,"endTime":897.4,"body":"but noise does something"},{"startTime":893.61,"endTime":897.4,"body":"bad to the signal."},{"startTime":897.4,"endTime":900.2,"body":"Then it hits the receiver and"},{"startTime":897.4,"endTime":900.2,"body":"then it hits the destination."},{"startTime":900.2,"endTime":903.323,"body":"That's the Shannon-Weaver model."},{"startTime":904.34,"endTime":906.3,"body":"It's a bit cruel really,"},{"startTime":904.34,"endTime":906.3,"body":"Shannon thought of all of this,"},{"startTime":906.3,"endTime":909.15,"body":"and then Warren Weaver wrote"},{"startTime":906.3,"endTime":909.15,"body":"it up in a nice little book,"},{"startTime":912.316,"endTime":913.149,"body":"a little booklet."},{"startTime":913.149,"endTime":914.2,"body":"And if I remember rightly,"},{"startTime":914.2,"endTime":916.25,"body":"the \"A Mathematical"},{"startTime":914.2,"endTime":916.25,"body":"Theory of Communication\""},{"startTime":916.25,"endTime":918.01,"body":"became \"The Mathematical"},{"startTime":916.25,"endTime":918.01,"body":"Theory of Communication\","},{"startTime":918.01,"endTime":919.76,"body":"that was Weaver's contribution"},{"startTime":919.76,"endTime":923.274,"body":"when he got named on"},{"startTime":919.76,"endTime":923.274,"body":"the name of the thing."},{"startTime":923.274,"endTime":926.343,"body":"So the Shannon model is"},{"startTime":923.274,"endTime":926.343,"body":"usually what engineers call it."},{"startTime":929.22,"endTime":932.55,"body":"So if we were sending some"},{"startTime":929.22,"endTime":932.55,"body":"data down this channel,"},{"startTime":932.55,"endTime":934.36,"body":"there might be a whole load of gubbins"},{"startTime":934.36,"endTime":936.11,"body":"to do with the transmission"},{"startTime":934.36,"endTime":936.11,"body":"and the receiver"},{"startTime":936.11,"endTime":936.943,"body":"and all of that."},{"startTime":936.943,"endTime":938.95,"body":"I'm going to ignore all the"},{"startTime":936.943,"endTime":938.95,"body":"gubbins for the time being,"},{"startTime":938.95,"endTime":942.61,"body":"and just assume that we're"},{"startTime":938.95,"endTime":942.61,"body":"going to send binary data"},{"startTime":942.61,"endTime":945.07,"body":"down the channel, 'cause"},{"startTime":942.61,"endTime":945.07,"body":"that helps us understand"},{"startTime":945.07,"endTime":947.56,"body":"this error control mechanism."},{"startTime":947.56,"endTime":950.8,"body":"So if we had say a black and white image,"},{"startTime":950.8,"endTime":953.34,"body":"which can be represented"},{"startTime":950.8,"endTime":953.34,"body":"by zeros and ones,"},{"startTime":953.34,"endTime":955.75,"body":"that's a nice little sort of visual analog"},{"startTime":955.75,"endTime":957.27,"body":"of what we might send down the channel."},{"startTime":957.27,"endTime":959.45,"body":"So here is my black and white image,"},{"startTime":959.45,"endTime":962.47,"body":"in this case it's the"},{"startTime":959.45,"endTime":962.47,"body":"Gresham College logo."},{"startTime":962.47,"endTime":967.47,"body":"And here is a part of that"},{"startTime":962.47,"endTime":967.47,"body":"signal, all these zeros and ones."},{"startTime":967.67,"endTime":969.53,"body":"It goes into my thing which I've labeled"},{"startTime":969.53,"endTime":971.07,"body":"binary symmetric channel."},{"startTime":971.07,"endTime":972.76,"body":"And I'll say what that is in a moment."},{"startTime":972.76,"endTime":976.37,"body":"Some noise does something,"},{"startTime":972.76,"endTime":976.37,"body":"and then bits get flipped."},{"startTime":976.37,"endTime":978.42,"body":"And the model here then is bit flipping."},{"startTime":978.42,"endTime":982.15,"body":"We are just going to randomly flip a one,"},{"startTime":982.15,"endTime":985.8,"body":"and it'll become a zero, or"},{"startTime":982.15,"endTime":985.8,"body":"flip a zero and become a one."},{"startTime":985.8,"endTime":988.61,"body":"And you can imagine what happens which is"},{"startTime":988.61,"endTime":989.67,"body":"it looks a bit like that,"},{"startTime":989.67,"endTime":992.05,"body":"we're seeing some black"},{"startTime":989.67,"endTime":992.05,"body":"pixels turn to white"},{"startTime":992.05,"endTime":994.013,"body":"and some white pixels turn to black."},{"startTime":994.96,"endTime":998.44,"body":"Now what about this binary"},{"startTime":994.96,"endTime":998.44,"body":"symmetric channel model?"},{"startTime":998.44,"endTime":1001.21,"body":"Not all channels are binary,"},{"startTime":1001.21,"endTime":1003.18,"body":"but this one is a binary channel."},{"startTime":1003.18,"endTime":1006.55,"body":"It has two symbols that"},{"startTime":1003.18,"endTime":1006.55,"body":"could be transmitted,"},{"startTime":1006.55,"endTime":1010.33,"body":"and two symbols that could"},{"startTime":1006.55,"endTime":1010.33,"body":"be received, a zero and one."},{"startTime":1010.33,"endTime":1013.17,"body":"And we characterize it using an error,"},{"startTime":1013.17,"endTime":1015.02,"body":"which I'm going to call"},{"startTime":1013.17,"endTime":1015.02,"body":"a probability of error,"},{"startTime":1015.02,"endTime":1017.19,"body":"or error probability, PE."},{"startTime":1017.19,"endTime":1019.61,"body":"So there's a probability"},{"startTime":1017.19,"endTime":1019.61,"body":"that a zero might get flipped"},{"startTime":1019.61,"endTime":1020.65,"body":"to a one."},{"startTime":1020.65,"endTime":1023.12,"body":"That's what that arrow represents."},{"startTime":1023.12,"endTime":1027.14,"body":"And there must be a probability"},{"startTime":1023.12,"endTime":1027.14,"body":"that a zero stays the same,"},{"startTime":1027.14,"endTime":1030.19,"body":"and that's one minus PE, and"},{"startTime":1027.14,"endTime":1030.19,"body":"I'm going to make it symmetric."},{"startTime":1030.19,"endTime":1033.08,"body":"So the probability that a"},{"startTime":1030.19,"endTime":1033.08,"body":"one gets flipped to a zero"},{"startTime":1034.08,"endTime":1035.38,"body":"is also PE."},{"startTime":1035.38,"endTime":1037.22,"body":"And so the probably"},{"startTime":1035.38,"endTime":1037.22,"body":"that a one remains a one"},{"startTime":1037.22,"endTime":1038.66,"body":"is one minus PE."},{"startTime":1038.66,"endTime":1040.523,"body":"That's a binary symmetric channel."},{"startTime":1041.58,"endTime":1044.17,"body":"To be fair, I've missed"},{"startTime":1041.58,"endTime":1044.17,"body":"off another caveat,"},{"startTime":1044.17,"endTime":1047.703,"body":"which is this is a binary"},{"startTime":1044.17,"endTime":1047.703,"body":"symmetric memory-less channel."},{"startTime":1048.58,"endTime":1050.78,"body":"The channel doesn't remember"},{"startTime":1048.58,"endTime":1050.78,"body":"what's been down it before,"},{"startTime":1050.78,"endTime":1054.243,"body":"it just spews stuff down it."},{"startTime":1055.59,"endTime":1059.73,"body":"If you fancy doing a PhD"},{"startTime":1055.59,"endTime":1059.73,"body":"in error control coding,"},{"startTime":1059.73,"endTime":1063.63,"body":"then you might want to"},{"startTime":1059.73,"endTime":1063.63,"body":"consider an Emory channel,"},{"startTime":1063.63,"endTime":1066.46,"body":"that's where instead of binary"},{"startTime":1063.63,"endTime":1066.46,"body":"you're sending M symbols,"},{"startTime":1066.46,"endTime":1070.43,"body":"so lots of fancy dancey coding methods"},{"startTime":1070.43,"endTime":1074.903,"body":"that use Emory coding, multi-phase"},{"startTime":1070.43,"endTime":1074.903,"body":"shift keying, and so on."},{"startTime":1076.68,"endTime":1081.68,"body":"And the symmetry isn't"},{"startTime":1076.68,"endTime":1081.68,"body":"given either in reality,"},{"startTime":1082.08,"endTime":1084.33,"body":"but let's just stick with this model."},{"startTime":1084.33,"endTime":1085.32,"body":"Okay."},{"startTime":1085.32,"endTime":1086.57,"body":"So here's the model."},{"startTime":1086.57,"endTime":1088.003,"body":"As I increase the error,"},{"startTime":1088.9,"endTime":1091.53,"body":"I hope you can see that"},{"startTime":1088.9,"endTime":1091.53,"body":"the image on the right"},{"startTime":1091.53,"endTime":1094.223,"body":"is getting more and more incomprehensible,"},{"startTime":1096.24,"endTime":1100.86,"body":"until finally we get to what"},{"startTime":1096.24,"endTime":1100.86,"body":"looks like random noise."},{"startTime":1100.86,"endTime":1104.73,"body":"So I've set up now a way of characterizing"},{"startTime":1104.73,"endTime":1106.62,"body":"all communication systems."},{"startTime":1106.62,"endTime":1110.04,"body":"Well, all ones based upon binary memory"},{"startTime":1110.04,"endTime":1113.063,"body":"of symmetric channels, which"},{"startTime":1110.04,"endTime":1113.063,"body":"will do for the time being."},{"startTime":1113.93,"endTime":1118.93,"body":"So the way we might do that"},{"startTime":1113.93,"endTime":1118.93,"body":"is by plotting the error,"},{"startTime":1122.11,"endTime":1127.11,"body":"and how that error gets"},{"startTime":1122.11,"endTime":1127.11,"body":"affected by my choice of code."},{"startTime":1127.26,"endTime":1130.17,"body":"So let's pick a really simple code."},{"startTime":1130.17,"endTime":1132.17,"body":"So the receiver and transmitter agree"},{"startTime":1132.17,"endTime":1134.3,"body":"that instead of sending something once,"},{"startTime":1134.3,"endTime":1136.38,"body":"they will send it three times."},{"startTime":1136.38,"endTime":1139.56,"body":"So the transmitter says well"},{"startTime":1136.38,"endTime":1139.56,"body":"I'm not just going to send"},{"startTime":1139.56,"endTime":1142.3,"body":"one of those things, I'm"},{"startTime":1139.56,"endTime":1142.3,"body":"going to make three copies"},{"startTime":1142.3,"endTime":1144.48,"body":"and I'm going to send it"},{"startTime":1142.3,"endTime":1144.48,"body":"three times down the channel."},{"startTime":1144.48,"endTime":1147.39,"body":"So the theory is that those"},{"startTime":1144.48,"endTime":1147.39,"body":"errors are independent."},{"startTime":1147.39,"endTime":1149.14,"body":"They don't happen at the same time."},{"startTime":1150.01,"endTime":1152.31,"body":"And then we'll get three"},{"startTime":1150.01,"endTime":1152.31,"body":"copies at the other end,"},{"startTime":1152.31,"endTime":1155.05,"body":"and then we'll stack them"},{"startTime":1152.31,"endTime":1155.05,"body":"up and take a majority vote,"},{"startTime":1155.05,"endTime":1157.83,"body":"and that's what we'll get at the output."},{"startTime":1157.83,"endTime":1161.2,"body":"And hope you can see that the noisiness"},{"startTime":1161.2,"endTime":1165.82,"body":"of these things up here has"},{"startTime":1161.2,"endTime":1165.82,"body":"been considerably reduced."},{"startTime":1165.82,"endTime":1169.71,"body":"So, this is an example"},{"startTime":1165.82,"endTime":1169.71,"body":"of a repetition code."},{"startTime":1169.71,"endTime":1171.4,"body":"So the question you might ask,"},{"startTime":1171.4,"endTime":1173.78,"body":"and there's a reason for"},{"startTime":1171.4,"endTime":1173.78,"body":"asking this, is well,"},{"startTime":1173.78,"endTime":1176.24,"body":"how could we sort of characterize that?"},{"startTime":1176.24,"endTime":1179.89,"body":"And the way engineers would, might do it."},{"startTime":1179.89,"endTime":1181.74,"body":"They'd use something"},{"startTime":1179.89,"endTime":1181.74,"body":"similar to this actually,"},{"startTime":1181.74,"endTime":1184.47,"body":"but I'll talk a bit about"},{"startTime":1181.74,"endTime":1184.47,"body":"how it varies a little bit,"},{"startTime":1184.47,"endTime":1185.75,"body":"is something like this."},{"startTime":1185.75,"endTime":1187.7,"body":"So what I'm plotting on the bottom here"},{"startTime":1188.58,"endTime":1191.33,"body":"is the rate that we're"},{"startTime":1188.58,"endTime":1191.33,"body":"achieving over the channel."},{"startTime":1191.33,"endTime":1195.813,"body":"So this dot here, which"},{"startTime":1191.33,"endTime":1195.813,"body":"has rate one, an error .1,"},{"startTime":1196.8,"endTime":1199.71,"body":"that's the raw channel where"},{"startTime":1196.8,"endTime":1199.71,"body":"I'm just spitting data out"},{"startTime":1199.71,"endTime":1202.25,"body":"without any control coding at all."},{"startTime":1202.25,"endTime":1203.53,"body":"And then the next dot,"},{"startTime":1203.53,"endTime":1206.87,"body":"well obviously I've reduced"},{"startTime":1203.53,"endTime":1206.87,"body":"the rate by a third,"},{"startTime":1206.87,"endTime":1209.02,"body":"but I've reduced the error."},{"startTime":1209.02,"endTime":1211.173,"body":"So as I reduce the rate,"},{"startTime":1212.38,"endTime":1214.69,"body":"I increase the number of repetitions,"},{"startTime":1214.69,"endTime":1217.193,"body":"which reduces the rate,"},{"startTime":1214.69,"endTime":1217.193,"body":"but the error goes down."},{"startTime":1218.76,"endTime":1220.33,"body":"So I'll just label those, okay,"},{"startTime":1220.33,"endTime":1223.46,"body":"these are repetition"},{"startTime":1220.33,"endTime":1223.46,"body":"codes going down there."},{"startTime":1223.46,"endTime":1225.05,"body":"And that sort of looks,"},{"startTime":1225.05,"endTime":1227.52,"body":"well perhaps I'll just plot"},{"startTime":1225.05,"endTime":1227.52,"body":"it for you in another way."},{"startTime":1227.52,"endTime":1231.81,"body":"Very often engineers like to"},{"startTime":1227.52,"endTime":1231.81,"body":"work at very small error rates,"},{"startTime":1231.81,"endTime":1234.77,"body":"and you just can't see"},{"startTime":1231.81,"endTime":1234.77,"body":"the error rates down here."},{"startTime":1234.77,"endTime":1238.03,"body":"So we'll often plot this on a log scale."},{"startTime":1238.03,"endTime":1239.393,"body":"It's exactly the same data,"},{"startTime":1239.393,"endTime":1241.303,"body":"it's just plotted a different way."},{"startTime":1243.1,"endTime":1245.13,"body":"Right, so you can imagine"},{"startTime":1243.1,"endTime":1245.13,"body":"how this might work."},{"startTime":1245.13,"endTime":1248.29,"body":"Your customer might say well,"},{"startTime":1245.13,"endTime":1248.29,"body":"I want an error rate of,"},{"startTime":1248.29,"endTime":1249.61,"body":"you know, 10 to the 15."},{"startTime":1249.61,"endTime":1253.643,"body":"And then you say, well, that"},{"startTime":1249.61,"endTime":1253.643,"body":"will mean 30 repeats then."},{"startTime":1254.63,"endTime":1259.63,"body":"So it looks like we're"},{"startTime":1254.63,"endTime":1259.63,"body":"already getting to something"},{"startTime":1262.24,"endTime":1264.54,"body":"that engineers or optimizers might call"},{"startTime":1264.54,"endTime":1266.38,"body":"a no-free lunch theorem,"},{"startTime":1266.38,"endTime":1269.46,"body":"where of course I can"},{"startTime":1266.38,"endTime":1269.46,"body":"give you what you want,"},{"startTime":1269.46,"endTime":1271.16,"body":"but it will have a compromise."},{"startTime":1271.16,"endTime":1272.58,"body":"It's a sort of theorem where"},{"startTime":1272.58,"endTime":1274.14,"body":"if I push in the balloon over here,"},{"startTime":1274.14,"endTime":1276.23,"body":"it bulges out over there."},{"startTime":1276.23,"endTime":1278.5,"body":"If you want lower error"},{"startTime":1276.23,"endTime":1278.5,"body":"rate, then you can have that,"},{"startTime":1278.5,"endTime":1281.103,"body":"but the rate of your"},{"startTime":1278.5,"endTime":1281.103,"body":"channel will decrease."},{"startTime":1282.26,"endTime":1285.8,"body":"Right, that was pretty"},{"startTime":1282.26,"endTime":1285.8,"body":"much what people thought"},{"startTime":1285.8,"endTime":1287.86,"body":"in the 1940s."},{"startTime":1287.86,"endTime":1290.61,"body":"It was pretty much the state of the art."},{"startTime":1290.61,"endTime":1295.03,"body":"And then along came a guy,"},{"startTime":1290.61,"endTime":1295.03,"body":"well along came Shannon,"},{"startTime":1295.03,"endTime":1296.7,"body":"but along came a guy"},{"startTime":1295.03,"endTime":1296.7,"body":"who was working with him"},{"startTime":1296.7,"endTime":1297.933,"body":"called Richard Hamming."},{"startTime":1298.9,"endTime":1301.21,"body":"And Hamming, by all accounts,"},{"startTime":1301.21,"endTime":1303.69,"body":"would be doing quite a bit of computing,"},{"startTime":1303.69,"endTime":1308.69,"body":"and had become intensely irked"},{"startTime":1303.69,"endTime":1308.69,"body":"by errors that he was seeing"},{"startTime":1311.9,"endTime":1312.75,"body":"in his machine,"},{"startTime":1312.75,"endTime":1314.32,"body":"'cause every time he got"},{"startTime":1312.75,"endTime":1314.32,"body":"an error it was stopping."},{"startTime":1314.32,"endTime":1316.68,"body":"So he started scratching his head thinking"},{"startTime":1316.68,"endTime":1319.29,"body":"what can I do to change this?"},{"startTime":1319.29,"endTime":1322.04,"body":"Now it's not that, the"},{"startTime":1319.29,"endTime":1322.04,"body":"idea of parity was around,"},{"startTime":1322.04,"endTime":1324.279,"body":"and as far as I can see,"},{"startTime":1324.279,"endTime":1325.53,"body":"I mean this is actually referenced in,"},{"startTime":1325.53,"endTime":1328.673,"body":"the first reference in Richard"},{"startTime":1325.53,"endTime":1328.673,"body":"Hamming's paper is this one,"},{"startTime":1330.1,"endTime":1332.337,"body":"which is a system built by RCA."},{"startTime":1332.337,"endTime":1335.56,"body":"And it's a high-speed telegraphy idea,"},{"startTime":1335.56,"endTime":1338.253,"body":"and the idea is that"},{"startTime":1335.56,"endTime":1338.253,"body":"you have a message here."},{"startTime":1338.253,"endTime":1340.17,"body":"It's called a seven unit printer code."},{"startTime":1340.17,"endTime":1342.32,"body":"These are binary digits."},{"startTime":1342.32,"endTime":1346.17,"body":"And this would be whizzed"},{"startTime":1342.32,"endTime":1346.17,"body":"at high speed down one end."},{"startTime":1346.17,"endTime":1347.44,"body":"And because it's high speed now,"},{"startTime":1347.44,"endTime":1349.85,"body":"we don't have two human"},{"startTime":1347.44,"endTime":1349.85,"body":"operators who can say stop, stop,"},{"startTime":1349.85,"endTime":1351.52,"body":"I didn't hear, didn't catch that one."},{"startTime":1351.52,"endTime":1355.01,"body":"We have to have some way of"},{"startTime":1351.52,"endTime":1355.01,"body":"working out there was an error."},{"startTime":1355.01,"endTime":1358.85,"body":"And I don't actually know"},{"startTime":1355.01,"endTime":1358.85,"body":"quite how this worked,"},{"startTime":1358.85,"endTime":1361.26,"body":"but Hamming's innovation was work out"},{"startTime":1361.26,"endTime":1365.61,"body":"a beautifully systematic way of doing"},{"startTime":1365.61,"endTime":1370.61,"body":"not only error detection,"},{"startTime":1365.61,"endTime":1370.61,"body":"but error correction."},{"startTime":1373.34,"endTime":1376.66,"body":"So let's have a look at"},{"startTime":1373.34,"endTime":1376.66,"body":"the Hamming code model."},{"startTime":1376.66,"endTime":1380.2,"body":"And this is the first code"},{"startTime":1376.66,"endTime":1380.2,"body":"that he works out in his paper."},{"startTime":1380.2,"endTime":1381.82,"body":"His paper is a great read."},{"startTime":1381.82,"endTime":1384.83,"body":"Everyone, you know, it's"},{"startTime":1381.82,"endTime":1384.83,"body":"perfectly understandable,"},{"startTime":1384.83,"endTime":1389.28,"body":"even by human beings who don't need a PhD"},{"startTime":1389.28,"endTime":1391.84,"body":"in Galois field algebra"},{"startTime":1389.28,"endTime":1391.84,"body":"to understand his paper,"},{"startTime":1391.84,"endTime":1393.25,"body":"far from it."},{"startTime":1393.25,"endTime":1394.77,"body":"Great read."},{"startTime":1394.77,"endTime":1396.42,"body":"Hamming said right, let's,"},{"startTime":1396.42,"endTime":1398.81,"body":"I'm going to allow myself seven digits."},{"startTime":1398.81,"endTime":1401.62,"body":"That's what we would call a block size."},{"startTime":1401.62,"endTime":1404.87,"body":"And I'm going to make four of those data,"},{"startTime":1404.87,"endTime":1407.3,"body":"and three of them are"},{"startTime":1404.87,"endTime":1407.3,"body":"going to be check digits."},{"startTime":1407.3,"endTime":1408.66,"body":"I'll call those parity,"},{"startTime":1408.66,"endTime":1412.58,"body":"just to provide continuity with the past."},{"startTime":1412.58,"endTime":1414.72,"body":"And if you are interested in the parts,"},{"startTime":1414.72,"endTime":1417.1,"body":"that's what we would call a 7,4 code,"},{"startTime":1417.1,"endTime":1419.21,"body":"seven because the block size is seven,"},{"startTime":1419.21,"endTime":1420.99,"body":"four because the amount"},{"startTime":1419.21,"endTime":1420.99,"body":"of data is in there."},{"startTime":1420.99,"endTime":1422.59,"body":"So the brackets, if you like,"},{"startTime":1422.59,"endTime":1425.0,"body":"tell you the redundancy in the code."},{"startTime":1425.0,"endTime":1427.683,"body":"And then he reasoned"},{"startTime":1425.0,"endTime":1427.683,"body":"like this, he said well,"},{"startTime":1430.7,"endTime":1433.3,"body":"maybe this is obvious, but"},{"startTime":1430.7,"endTime":1433.3,"body":"he said an important thing."},{"startTime":1433.3,"endTime":1435.9,"body":"He said, well, the thing with parity,"},{"startTime":1435.9,"endTime":1438.32,"body":"parity tells me if there's been an error."},{"startTime":1438.32,"endTime":1441.69,"body":"What if I knew where the error was?"},{"startTime":1441.69,"endTime":1443.26,"body":"I didn't just know there was an error,"},{"startTime":1443.26,"endTime":1446.453,"body":"I knew where it had occurred"},{"startTime":1443.26,"endTime":1446.453,"body":"in the word in the block."},{"startTime":1447.67,"endTime":1449.51,"body":"Ah, well then I could"},{"startTime":1447.67,"endTime":1449.51,"body":"correct it, couldn't I."},{"startTime":1449.51,"endTime":1452.01,"body":"So knowing where an error is"},{"startTime":1452.01,"endTime":1454.43,"body":"is equivalent to correcting an error."},{"startTime":1454.43,"endTime":1457.76,"body":"So he said ah right,"},{"startTime":1454.43,"endTime":1457.76,"body":"well, so what I need is,"},{"startTime":1457.76,"endTime":1459.15,"body":"if not my parity digits,"},{"startTime":1459.15,"endTime":1462.64,"body":"something related to my parity"},{"startTime":1459.15,"endTime":1462.64,"body":"digits, which I've got three,"},{"startTime":1462.64,"endTime":1464.98,"body":"they need to point to the error."},{"startTime":1464.98,"endTime":1467.21,"body":"Now I've got 0, 0, 0,"},{"startTime":1467.21,"endTime":1470.92,"body":"I'm going to insist"},{"startTime":1467.21,"endTime":1470.92,"body":"that's used for no error."},{"startTime":1470.92,"endTime":1472.69,"body":"So I'm going to reserve that."},{"startTime":1472.69,"endTime":1475.64,"body":"So this is my natural binary"},{"startTime":1472.69,"endTime":1475.64,"body":"count of three digits."},{"startTime":1475.64,"endTime":1480.09,"body":"So this is 1, 2, 3, 4, 5, so on, up to 7."},{"startTime":1480.09,"endTime":1485.09,"body":"So clearly three parity"},{"startTime":1480.09,"endTime":1485.09,"body":"digits could, in principle,"},{"startTime":1485.25,"endTime":1487.33,"body":"address seven positions."},{"startTime":1487.33,"endTime":1489.01,"body":"They could point to seven positions."},{"startTime":1489.01,"endTime":1491.41,"body":"That's good, 'cause I've got"},{"startTime":1489.01,"endTime":1491.41,"body":"seven digits in the code."},{"startTime":1491.41,"endTime":1494.34,"body":"So the overall sort of"},{"startTime":1491.41,"endTime":1494.34,"body":"architecture of this"},{"startTime":1494.34,"endTime":1496.98,"body":"is laid out in Hamming's"},{"startTime":1494.34,"endTime":1496.98,"body":"paper quite nicely,"},{"startTime":1496.98,"endTime":1498.62,"body":"but this is the start of his reasoning."},{"startTime":1498.62,"endTime":1501.03,"body":"He says well, I need"},{"startTime":1498.62,"endTime":1501.03,"body":"to be able to find out"},{"startTime":1501.03,"endTime":1503.46,"body":"these positions of these errors."},{"startTime":1503.46,"endTime":1505.07,"body":"And once I found the positions,"},{"startTime":1505.07,"endTime":1508.453,"body":"then everything is oojah-cum-spiff."},{"startTime":1509.34,"endTime":1510.99,"body":"Right then, how does it work?"},{"startTime":1510.99,"endTime":1512.95,"body":"So what he did is he said well,"},{"startTime":1512.95,"endTime":1516.73,"body":"I'm going to label these"},{"startTime":1512.95,"endTime":1516.73,"body":"bits from one to seven."},{"startTime":1516.73,"endTime":1520.49,"body":"That's why they're called b1"},{"startTime":1516.73,"endTime":1520.49,"body":"to b7 in this diagram here."},{"startTime":1520.49,"endTime":1525.49,"body":"And then I'm going to insist"},{"startTime":1520.49,"endTime":1525.49,"body":"that I have even parity"},{"startTime":1525.95,"endTime":1527.93,"body":"among these sets of digits."},{"startTime":1527.93,"endTime":1531.93,"body":"So this is just that mathematical"},{"startTime":1527.93,"endTime":1531.93,"body":"way of saying even parity."},{"startTime":1531.93,"endTime":1535.08,"body":"So that says I want b"},{"startTime":1531.93,"endTime":1535.08,"body":"position one, position three,"},{"startTime":1535.08,"endTime":1538.523,"body":"position five, and"},{"startTime":1535.08,"endTime":1538.523,"body":"position seven, to be even."},{"startTime":1539.67,"endTime":1542.55,"body":"And then he did a clever"},{"startTime":1539.67,"endTime":1542.55,"body":"thing, which he said well,"},{"startTime":1542.55,"endTime":1546.03,"body":"I'm going to assign my parity digits"},{"startTime":1546.03,"endTime":1549.857,"body":"in these positions, which"},{"startTime":1546.03,"endTime":1549.857,"body":"happen to be at powers of two."},{"startTime":1551.628,"endTime":1554.21,"body":"There's a whole load"},{"startTime":1551.628,"endTime":1554.21,"body":"of YouTube explanations"},{"startTime":1554.21,"endTime":1555.663,"body":"as to why they have to"},{"startTime":1554.21,"endTime":1555.663,"body":"be at powers of two."},{"startTime":1555.663,"endTime":1559.95,"body":"I don't think we need to bother"},{"startTime":1555.663,"endTime":1559.95,"body":"ourselves with it right now."},{"startTime":1559.95,"endTime":1562.68,"body":"So let's just sort of check how it works."},{"startTime":1562.68,"endTime":1566.92,"body":"So if we had a data digit, data byte four,"},{"startTime":1566.92,"endTime":1569.82,"body":"we've got four bits,"},{"startTime":1566.92,"endTime":1569.82,"body":"'cause it's a 7,4 code."},{"startTime":1569.82,"endTime":1570.98,"body":"So the first thing we're going to do"},{"startTime":1570.98,"endTime":1573.403,"body":"is stuff those into the code,"},{"startTime":1574.24,"endTime":1579.24,"body":"So 11001100, it's got"},{"startTime":1574.24,"endTime":1579.24,"body":"stuffed into the code there."},{"startTime":1579.88,"endTime":1581.28,"body":"Word of caution."},{"startTime":1581.28,"endTime":1584.95,"body":"If you go away and look this"},{"startTime":1581.28,"endTime":1584.95,"body":"up there are some differences,"},{"startTime":1584.95,"endTime":1588.21,"body":"because some people put the bits in"},{"startTime":1588.21,"endTime":1589.623,"body":"the other way round, 0011."},{"startTime":1590.89,"endTime":1593.55,"body":"Some people don't have the parity digits"},{"startTime":1593.55,"endTime":1596.616,"body":"where Richard Hamming"},{"startTime":1593.55,"endTime":1596.616,"body":"decided to have them,"},{"startTime":1596.616,"endTime":1598.08,"body":"and I'll mention why that is later."},{"startTime":1598.08,"endTime":1601.18,"body":"But I'm following the"},{"startTime":1598.08,"endTime":1601.18,"body":"Hamming methodology here."},{"startTime":1601.18,"endTime":1602.6,"body":"So, and then he says right,"},{"startTime":1602.6,"endTime":1607.6,"body":"so the first parity has to"},{"startTime":1602.6,"endTime":1607.6,"body":"cover b1, b3, b5 and b7."},{"startTime":1609.99,"endTime":1612.03,"body":"Well, p1 is the one I'm going to set,"},{"startTime":1612.03,"endTime":1613.71,"body":"so that's at the first position,"},{"startTime":1613.71,"endTime":1618.71,"body":"b3 is one, b5 is one, and b7 is zero."},{"startTime":1619.55,"endTime":1621.75,"body":"So that's already even."},{"startTime":1621.75,"endTime":1623.27,"body":"So I need to make that a zero"},{"startTime":1623.27,"endTime":1625.373,"body":"in order to maintain its evenness."},{"startTime":1627.57,"endTime":1629.93,"body":"So that's how we set the parity digits."},{"startTime":1629.93,"endTime":1632.42,"body":"We go through and we"},{"startTime":1629.93,"endTime":1632.42,"body":"check all of these things."},{"startTime":1632.42,"endTime":1637.42,"body":"And we've now created a"},{"startTime":1632.42,"endTime":1637.42,"body":"seven bit block of data"},{"startTime":1638.75,"endTime":1643.75,"body":"that checks subsets of the incoming data."},{"startTime":1644.86,"endTime":1648.53,"body":"So if you're feeling a bit"},{"startTime":1644.86,"endTime":1648.53,"body":"sort of, you know, it's,"},{"startTime":1648.53,"endTime":1650.5,"body":"I'm a bit scratching"},{"startTime":1648.53,"endTime":1650.5,"body":"my head at this point,"},{"startTime":1650.5,"endTime":1653.2,"body":"don't worry too much about it, the detail."},{"startTime":1653.2,"endTime":1655.87,"body":"But the important innovation is"},{"startTime":1655.87,"endTime":1660.33,"body":"each one of these parities"},{"startTime":1655.87,"endTime":1660.33,"body":"isn't checking all of the data"},{"startTime":1660.33,"endTime":1662.81,"body":"as it used to, it's checking"},{"startTime":1660.33,"endTime":1662.81,"body":"subsets of the data."},{"startTime":1662.81,"endTime":1666.35,"body":"And you can imagine there's"},{"startTime":1662.81,"endTime":1666.35,"body":"quite a lot of human ingenuity"},{"startTime":1666.35,"endTime":1668.43,"body":"in choosing those subsets,"},{"startTime":1668.43,"endTime":1672.84,"body":"but let's just stick with"},{"startTime":1668.43,"endTime":1672.84,"body":"Hamming for the time being."},{"startTime":1672.84,"endTime":1676.353,"body":"So now we transmit it, and"},{"startTime":1672.84,"endTime":1676.353,"body":"let's imagine there's an error."},{"startTime":1677.393,"endTime":1678.74,"body":"There's an error in position five,"},{"startTime":1678.74,"endTime":1681.61,"body":"which is precisely where"},{"startTime":1678.74,"endTime":1681.61,"body":"Richard Hamming talks about it"},{"startTime":1681.61,"endTime":1684.35,"body":"in his example, if you want"},{"startTime":1681.61,"endTime":1684.35,"body":"to follow along at home"},{"startTime":1684.35,"endTime":1685.49,"body":"as it were."},{"startTime":1685.49,"endTime":1688.57,"body":"So on the decode, obviously the"},{"startTime":1685.49,"endTime":1688.57,"body":"thing we're going to have to do"},{"startTime":1688.57,"endTime":1691.18,"body":"is we're going to recompute"},{"startTime":1688.57,"endTime":1691.18,"body":"those parity digits"},{"startTime":1691.18,"endTime":1693.14,"body":"with the data I've received now,"},{"startTime":1693.14,"endTime":1698.14,"body":"and see if I can interpret"},{"startTime":1693.14,"endTime":1698.14,"body":"that in some way."},{"startTime":1698.17,"endTime":1700.24,"body":"So the way I've chosen to do it here is"},{"startTime":1700.24,"endTime":1703.71,"body":"I'm going to recompute those parities,"},{"startTime":1703.71,"endTime":1707.57,"body":"and then I'm going to use a"},{"startTime":1703.71,"endTime":1707.57,"body":"zero, if the parities match,"},{"startTime":1707.57,"endTime":1709.7,"body":"that's the recomputed parity matches"},{"startTime":1709.7,"endTime":1711.65,"body":"with the transmitted parity,"},{"startTime":1711.65,"endTime":1714.397,"body":"and I'm going to use a one if it doesn't."},{"startTime":1714.397,"endTime":1717.093,"body":"That's all quite easy to"},{"startTime":1714.397,"endTime":1717.093,"body":"do in low level hardware."},{"startTime":1719.14,"endTime":1720.76,"body":"So here are my checks."},{"startTime":1720.76,"endTime":1724.11,"body":"So if we look at the"},{"startTime":1720.76,"endTime":1724.11,"body":"first one, for example,"},{"startTime":1724.11,"endTime":1728.47,"body":"position b1, b3, b5 and b7,"},{"startTime":1724.11,"endTime":1728.47,"body":"are all meant to be even,"},{"startTime":1735.91,"endTime":1737.71,"body":"which they are."},{"startTime":1737.71,"endTime":1742.71,"body":"So did I, sorry, what did I,"},{"startTime":1737.71,"endTime":1742.71,"body":"b1, b3, no they're not, sorry,"},{"startTime":1744.826,"endTime":1749.826,"body":"b1, zero, b3 is one, b5 and"},{"startTime":1744.826,"endTime":1749.826,"body":"b7, that is an odd parity,"},{"startTime":1753.64,"endTime":1756.143,"body":"that's a mistake, so"},{"startTime":1753.64,"endTime":1756.143,"body":"I'll put a one in there."},{"startTime":1757.63,"endTime":1761.77,"body":"The next set, which is here, is correct."},{"startTime":1761.77,"endTime":1763.63,"body":"Next set is wrong."},{"startTime":1763.63,"endTime":1765.24,"body":"So I've got a one."},{"startTime":1765.24,"endTime":1769.57,"body":"1, 0, 1 is five in decimal."},{"startTime":1772.13,"endTime":1776.94,"body":"And the error is in the fifth position."},{"startTime":1776.94,"endTime":1779.52,"body":"So the neatness of the Hamming scheme"},{"startTime":1779.52,"endTime":1783.89,"body":"was to make all of this"},{"startTime":1779.52,"endTime":1783.89,"body":"work in a nice, easy way,"},{"startTime":1783.89,"endTime":1787.83,"body":"which was the way he"},{"startTime":1783.89,"endTime":1787.83,"body":"placed those parity digits."},{"startTime":1787.83,"endTime":1790.57,"body":"You don't have to place the parity digits"},{"startTime":1790.57,"endTime":1792.92,"body":"in the middle of the words like this."},{"startTime":1792.92,"endTime":1795.78,"body":"Hamming did it because in those days"},{"startTime":1795.78,"endTime":1797.02,"body":"hardware was quite expensive,"},{"startTime":1797.02,"endTime":1799.97,"body":"but a lot of people now just"},{"startTime":1797.02,"endTime":1799.97,"body":"stuff 'em on at the end."},{"startTime":1799.97,"endTime":1802.58,"body":"The important thing is that we're checking"},{"startTime":1802.58,"endTime":1804.5,"body":"subsets of the data,"},{"startTime":1804.5,"endTime":1807.073,"body":"and then inferring where the mistakes are."},{"startTime":1808.03,"endTime":1811.31,"body":"Right, at this point you're"},{"startTime":1808.03,"endTime":1811.31,"body":"sort of longing to know"},{"startTime":1811.31,"endTime":1813.53,"body":"whether the Hamming code is any better"},{"startTime":1813.53,"endTime":1816.033,"body":"than any other sort of code, I suspect."},{"startTime":1817.03,"endTime":1820.02,"body":"So here we are back"},{"startTime":1817.03,"endTime":1820.02,"body":"with our diagram again."},{"startTime":1820.02,"endTime":1822.5,"body":"So these are the codes that we discussed"},{"startTime":1822.5,"endTime":1826.33,"body":"in the first section, which"},{"startTime":1822.5,"endTime":1826.33,"body":"were the repetition codes,"},{"startTime":1826.33,"endTime":1829.36,"body":"and I've just sketched in"},{"startTime":1826.33,"endTime":1829.36,"body":"roughly where Hamming is,"},{"startTime":1829.36,"endTime":1833.0,"body":"that Hamming is in this"},{"startTime":1829.36,"endTime":1833.0,"body":"square position here."},{"startTime":1833.0,"endTime":1838.0,"body":"Well, it's sort of, now"},{"startTime":1833.0,"endTime":1838.0,"body":"it's sort of all pointing"},{"startTime":1838.97,"endTime":1841.853,"body":"in one direction really,"},{"startTime":1838.97,"endTime":1841.853,"body":"isn't it, which is,"},{"startTime":1843.67,"endTime":1845.85,"body":"you can have lower error rates,"},{"startTime":1845.85,"endTime":1849.26,"body":"but in order to have lower"},{"startTime":1845.85,"endTime":1849.26,"body":"probabilities of error"},{"startTime":1849.26,"endTime":1850.47,"body":"at the output,"},{"startTime":1850.47,"endTime":1853.26,"body":"then you're going to have"},{"startTime":1850.47,"endTime":1853.26,"body":"to affect the amount of data"},{"startTime":1853.26,"endTime":1854.56,"body":"you send down the channel."},{"startTime":1856.16,"endTime":1859.99,"body":"And that is pretty much"},{"startTime":1856.16,"endTime":1859.99,"body":"the conventional wisdom."},{"startTime":1859.99,"endTime":1863.83,"body":"Now I've plotted here the"},{"startTime":1859.99,"endTime":1863.83,"body":"sort of idea of better codes,"},{"startTime":1863.83,"endTime":1867.56,"body":"which I hope you can sort of"},{"startTime":1863.83,"endTime":1867.56,"body":"see immediately, which is,"},{"startTime":1867.56,"endTime":1869.26,"body":"as we move in this direction,"},{"startTime":1869.26,"endTime":1871.5,"body":"we get lower error and higher rates."},{"startTime":1871.5,"endTime":1873.3,"body":"So if we had a code out here, for example,"},{"startTime":1873.3,"endTime":1876.253,"body":"that would be a better"},{"startTime":1873.3,"endTime":1876.253,"body":"code than that code."},{"startTime":1877.17,"endTime":1879.943,"body":"This diagram is sort of, you know,"},{"startTime":1881.34,"endTime":1883.44,"body":"in a way it was sort of rather comforting."},{"startTime":1885.56,"endTime":1887.093,"body":"The only problem is,"},{"startTime":1888.23,"endTime":1891.54,"body":"it's the inference that"},{"startTime":1888.23,"endTime":1891.54,"body":"I'm making for you,"},{"startTime":1891.54,"endTime":1895.32,"body":"and I've made it twice now, is"},{"startTime":1891.54,"endTime":1895.32,"body":"completely and utterly wrong."},{"startTime":1895.32,"endTime":1897.63,"body":"And surprisingly so."},{"startTime":1897.63,"endTime":1901.84,"body":"And the way, the person"},{"startTime":1897.63,"endTime":1901.84,"body":"who showed this to be wrong"},{"startTime":1901.84,"endTime":1903.763,"body":"was our man Shannon, again."},{"startTime":1905.28,"endTime":1907.39,"body":"So this is called"},{"startTime":1905.28,"endTime":1907.39,"body":"Shannon's coding theorem,"},{"startTime":1907.39,"endTime":1909.84,"body":"or sometimes Shannon's"},{"startTime":1907.39,"endTime":1909.84,"body":"noisy coding theorem."},{"startTime":1909.84,"endTime":1913.12,"body":"Sometimes it's called the"},{"startTime":1909.84,"endTime":1913.12,"body":"Shannon-Hartley theorem,"},{"startTime":1913.12,"endTime":1915.09,"body":"after Ralph Hartley,"},{"startTime":1915.09,"endTime":1917.363,"body":"who did some very early work in the '20s."},{"startTime":1918.53,"endTime":1921.98,"body":"Again, not sure, to be"},{"startTime":1918.53,"endTime":1921.98,"body":"perfectly honest, you know,"},{"startTime":1921.98,"endTime":1923.99,"body":"it really is Shannon's theorem,"},{"startTime":1923.99,"endTime":1926.08,"body":"and Hartley's been given"},{"startTime":1923.99,"endTime":1926.08,"body":"a bit of a free ride"},{"startTime":1926.08,"endTime":1927.7,"body":"on this theorem, but there we go."},{"startTime":1927.7,"endTime":1930.793,"body":"And what I've done here"},{"startTime":1927.7,"endTime":1930.793,"body":"is I've plotted this line."},{"startTime":1932.25,"endTime":1934.14,"body":"What Shannon's theorem says,"},{"startTime":1934.14,"endTime":1935.6,"body":"and it's easy to say this in a way"},{"startTime":1935.6,"endTime":1936.92,"body":"that makes it sound trivial,"},{"startTime":1936.92,"endTime":1939.36,"body":"is this side of the line is forbidden,"},{"startTime":1939.36,"endTime":1941.11,"body":"you cannot go this side of a line."},{"startTime":1941.11,"endTime":1941.943,"body":"It's impossible."},{"startTime":1942.88,"endTime":1946.56,"body":"What is perhaps not so"},{"startTime":1942.88,"endTime":1946.56,"body":"obvious from this diagram,"},{"startTime":1946.56,"endTime":1948.521,"body":"the diagram on the left,"},{"startTime":1948.521,"endTime":1950.69,"body":"but might be more obvious"},{"startTime":1948.521,"endTime":1950.69,"body":"from the diagram on the right,"},{"startTime":1950.69,"endTime":1955.083,"body":"is that this line here goes to zero."},{"startTime":1956.04,"endTime":1957.283,"body":"It hits zero."},{"startTime":1959.26,"endTime":1960.75,"body":"Oh."},{"startTime":1960.75,"endTime":1965.75,"body":"That means we can take any noisy channel"},{"startTime":1966.55,"endTime":1969.233,"body":"and operate it with zero error."},{"startTime":1970.98,"endTime":1974.706,"body":"Ooh, that is quite, well you"},{"startTime":1970.98,"endTime":1974.706,"body":"can imagine how surprising"},{"startTime":1974.706,"endTime":1976.4,"body":"that was at the time."},{"startTime":1976.4,"endTime":1980.26,"body":"It seems really somewhat"},{"startTime":1976.4,"endTime":1980.26,"body":"quite astonishing."},{"startTime":1980.26,"endTime":1984.47,"body":"However noisy your channel"},{"startTime":1980.26,"endTime":1984.47,"body":"is, however errorful it is,"},{"startTime":1984.47,"endTime":1987.66,"body":"we can operate that"},{"startTime":1984.47,"endTime":1987.66,"body":"channel with zero error,"},{"startTime":1987.66,"endTime":1991.24,"body":"and I can tell you the rate"},{"startTime":1987.66,"endTime":1991.24,"body":"at which we could operate it."},{"startTime":1991.24,"endTime":1994.08,"body":"What a wonderful bit of"},{"startTime":1991.24,"endTime":1994.08,"body":"reasoning and mathematics."},{"startTime":1994.08,"endTime":1998.88,"body":"And Shannon managed to do this"},{"startTime":1994.08,"endTime":1998.88,"body":"without knowledge of codes,"},{"startTime":1998.88,"endTime":2000.31,"body":"which is a stunning thing."},{"startTime":2000.31,"endTime":2002.78,"body":"So technically, it looks a bit like this,"},{"startTime":2002.78,"endTime":2006.92,"body":"which is the channel"},{"startTime":2002.78,"endTime":2006.92,"body":"capacity is always less than"},{"startTime":2006.92,"endTime":2008.52,"body":"or equal to some number,"},{"startTime":2008.52,"endTime":2011.297,"body":"which is based upon a bit of entropy."},{"startTime":2011.297,"endTime":2012.95,"body":"And I don't want to talk about entropy,"},{"startTime":2012.95,"endTime":2016.06,"body":"I have mentioned it in"},{"startTime":2012.95,"endTime":2016.06,"body":"other lectures and it's,"},{"startTime":2016.06,"endTime":2017.21,"body":"you're welcome to look it up."},{"startTime":2017.21,"endTime":2019.383,"body":"It's not for today."},{"startTime":2021.44,"endTime":2025.67,"body":"But it's really, it's really"},{"startTime":2021.44,"endTime":2025.67,"body":"quite an astonishing result"},{"startTime":2025.67,"endTime":2027.94,"body":"and it has some consequences,"},{"startTime":2027.94,"endTime":2030.41,"body":"and I've sort of paraphrased"},{"startTime":2027.94,"endTime":2030.41,"body":"them a bit really"},{"startTime":2030.41,"endTime":2032.12,"body":"with Richard Harvey sort of"},{"startTime":2032.12,"endTime":2035.244,"body":"Claude Eustace Shannon"},{"startTime":2032.12,"endTime":2035.244,"body":"Agony Aunt dialogue,"},{"startTime":2035.244,"endTime":2037.244,"body":"a sort of Aristotelian dialogue between,"},{"startTime":2038.681,"endTime":2039.514,"body":"you know, an engineer."},{"startTime":2039.514,"endTime":2040.75,"body":"So here's an example."},{"startTime":2040.75,"endTime":2043.01,"body":"So he's an engineer and she's saying"},{"startTime":2043.01,"endTime":2044.76,"body":"my channel has an error rate of 0.1"},{"startTime":2045.604,"endTime":2047.5,"body":"and a data rate of six megabits a second."},{"startTime":2047.5,"endTime":2049.08,"body":"How many repeats would I require"},{"startTime":2049.08,"endTime":2051.576,"body":"to get the error rate down"},{"startTime":2049.08,"endTime":2051.576,"body":"to 10 to the minus 15,"},{"startTime":2051.576,"endTime":2054.52,"body":"10 to the minus 15 is a"},{"startTime":2051.576,"endTime":2054.52,"body":"number that is often used for,"},{"startTime":2054.52,"endTime":2057.22,"body":"you know, reliable communication,"},{"startTime":2054.52,"endTime":2057.22,"body":"and what's the cost."},{"startTime":2058.07,"endTime":2059.47,"body":"Okay, well we know the answer to that,"},{"startTime":2059.47,"endTime":2061.18,"body":"we could go back to my first diagram"},{"startTime":2061.18,"endTime":2062.94,"body":"when we were talking about repeats."},{"startTime":2062.94,"endTime":2065.44,"body":"So well, consult your diagram"},{"startTime":2065.44,"endTime":2067.545,"body":"and you just count the number of repeats,"},{"startTime":2067.545,"endTime":2068.58,"body":"and it's about 60 repeats."},{"startTime":2068.58,"endTime":2070.16,"body":"But there's a cost, right?"},{"startTime":2070.16,"endTime":2072.06,"body":"The cost is the bandwidth will reduce"},{"startTime":2072.06,"endTime":2075.2,"body":"to a measly six divided by 60 megabits,"},{"startTime":2075.2,"endTime":2077.763,"body":"a hundred kilobits a"},{"startTime":2075.2,"endTime":2077.763,"body":"second, which is lousy."},{"startTime":2079.16,"endTime":2082.393,"body":"Okay says quizzical engineer."},{"startTime":2083.69,"endTime":2085.55,"body":"Can't I do better than that?"},{"startTime":2085.55,"endTime":2088.19,"body":"Well, the agony aunt says yes."},{"startTime":2088.19,"endTime":2092.1,"body":"My band is 0.54, so you"},{"startTime":2088.19,"endTime":2092.1,"body":"multiply 0.54 by six megabits."},{"startTime":2092.1,"endTime":2096.25,"body":"You could communicate at zero error"},{"startTime":2096.25,"endTime":2100.01,"body":"with a bandwidth of"},{"startTime":2096.25,"endTime":2100.01,"body":"3.24 megabits a second."},{"startTime":2100.01,"endTime":2101.46,"body":"Wow, fantastic!"},{"startTime":2101.46,"endTime":2105.77,"body":"So you can imagine what"},{"startTime":2101.46,"endTime":2105.77,"body":"the next question is"},{"startTime":2105.77,"endTime":2107.76,"body":"from our engineer."},{"startTime":2107.76,"endTime":2110.77,"body":"Well, they're a young engineer,"},{"startTime":2107.76,"endTime":2110.77,"body":"so they say amaze balls!"},{"startTime":2110.77,"endTime":2112.633,"body":"How do I design such a code?"},{"startTime":2114.09,"endTime":2114.923,"body":"Ah."},{"startTime":2115.93,"endTime":2118.67,"body":"That is the issue, ladies and gentlemen."},{"startTime":2118.67,"endTime":2122.12,"body":"That is the whole of coding"},{"startTime":2118.67,"endTime":2122.12,"body":"theory in a nutshell."},{"startTime":2122.12,"endTime":2124.553,"body":"I could almost stop the"},{"startTime":2122.12,"endTime":2124.553,"body":"lecture at this point."},{"startTime":2125.6,"endTime":2129.24,"body":"So let me draw for you the"},{"startTime":2125.6,"endTime":2129.24,"body":"map of human ingenuity,"},{"startTime":2129.24,"endTime":2131.14,"body":"in trying to design codes, okay."},{"startTime":2131.14,"endTime":2136.14,"body":"So this is the 1940s,"},{"startTime":2131.14,"endTime":2136.14,"body":"when Shannon did things."},{"startTime":2137.95,"endTime":2142.04,"body":"And we got this early"},{"startTime":2137.95,"endTime":2142.04,"body":"guy, Richard Hamming."},{"startTime":2142.04,"endTime":2143.56,"body":"Actually, to be perfectly honest,"},{"startTime":2143.56,"endTime":2145.8,"body":"Hamming was slightly predated, I think,"},{"startTime":2145.8,"endTime":2150.8,"body":"by Golay, and I'll show you"},{"startTime":2145.8,"endTime":2150.8,"body":"Golay stuff in a minute."},{"startTime":2152.8,"endTime":2154.21,"body":"There was parity knocking around here,"},{"startTime":2154.21,"endTime":2156.68,"body":"but it wasn't in a sort"},{"startTime":2154.21,"endTime":2156.68,"body":"of significant framework."},{"startTime":2156.68,"endTime":2158.29,"body":"And then we've got all"},{"startTime":2156.68,"endTime":2158.29,"body":"of these codes here,"},{"startTime":2158.29,"endTime":2160.73,"body":"some of which you will have come across,"},{"startTime":2160.73,"endTime":2161.99,"body":"because they are mentioned"},{"startTime":2161.99,"endTime":2164.44,"body":"and people often mention"},{"startTime":2161.99,"endTime":2164.44,"body":"Reed-Solomon coding,"},{"startTime":2164.44,"endTime":2167.44,"body":"'cause it's used in this compact"},{"startTime":2164.44,"endTime":2167.44,"body":"disc standard, for example."},{"startTime":2168.39,"endTime":2171.76,"body":"A lot of these sort of"},{"startTime":2168.39,"endTime":2171.76,"body":"rather rarefied codes"},{"startTime":2171.76,"endTime":2174.003,"body":"were used first in satellite systems."},{"startTime":2174.98,"endTime":2177.88,"body":"And then there was another"},{"startTime":2174.98,"endTime":2177.88,"body":"burst of activity around here."},{"startTime":2177.88,"endTime":2182.49,"body":"Now I started lecturing"},{"startTime":2177.88,"endTime":2182.49,"body":"somewhere around here, I think,"},{"startTime":2182.49,"endTime":2186.18,"body":"and I remember cringing when"},{"startTime":2182.49,"endTime":2186.18,"body":"having to lecture this stuff"},{"startTime":2186.18,"endTime":2189.47,"body":"because you'd tell people"},{"startTime":2186.18,"endTime":2189.47,"body":"about Hamming codes,"},{"startTime":2189.47,"endTime":2192.09,"body":"because they're nice and"},{"startTime":2189.47,"endTime":2192.09,"body":"beautiful and easy to explain,"},{"startTime":2192.09,"endTime":2194.92,"body":"and then people would ask you well, yeah,"},{"startTime":2194.92,"endTime":2196.37,"body":"but what does everyone use."},{"startTime":2196.37,"endTime":2199.06,"body":"And you'd have to go through"},{"startTime":2196.37,"endTime":2199.06,"body":"all these things here,"},{"startTime":2199.06,"endTime":2203.24,"body":"which were complicated"},{"startTime":2199.06,"endTime":2203.24,"body":"and difficult to explain,"},{"startTime":2203.24,"endTime":2206.11,"body":"and didn't bear much"},{"startTime":2203.24,"endTime":2206.11,"body":"relation to each other,"},{"startTime":2206.11,"endTime":2207.98,"body":"and weren't mathematically beautiful."},{"startTime":2207.98,"endTime":2210.04,"body":"So it was all a bit disappointing."},{"startTime":2210.04,"endTime":2215.04,"body":"Fortunately, these two codes"},{"startTime":2210.04,"endTime":2215.04,"body":"out here, LDPC and polar codes,"},{"startTime":2215.84,"endTime":2219.1,"body":"I'll talk about LDPC,"},{"startTime":2215.84,"endTime":2219.1,"body":"low-density parity-check codes,"},{"startTime":2219.1,"endTime":2224.01,"body":"pretty much hold the same"},{"startTime":2219.1,"endTime":2224.01,"body":"sort of handle-turning aspect"},{"startTime":2224.01,"endTime":2226.11,"body":"as Hamming codes, which"},{"startTime":2224.01,"endTime":2226.11,"body":"I've already explained."},{"startTime":2226.11,"endTime":2229.79,"body":"So now, treating error"},{"startTime":2226.11,"endTime":2229.79,"body":"control coding is trivial,"},{"startTime":2229.79,"endTime":2231.18,"body":"you can just ignore all of this stuff."},{"startTime":2231.18,"endTime":2233.25,"body":"Which is what I'm going"},{"startTime":2231.18,"endTime":2233.25,"body":"to do in this lecture."},{"startTime":2233.25,"endTime":2236.43,"body":"I'm going to rather sweepingly ignore"},{"startTime":2236.43,"endTime":2239.203,"body":"whole ranges of interesting"},{"startTime":2236.43,"endTime":2239.203,"body":"and useful code,"},{"startTime":2240.73,"endTime":2242.75,"body":"because everyone at the moment"},{"startTime":2242.75,"endTime":2247.47,"body":"is rushing to get their"},{"startTime":2242.75,"endTime":2247.47,"body":"hardware onto polar codes"},{"startTime":2247.47,"endTime":2250.19,"body":"and low-density parity-check codes."},{"startTime":2250.19,"endTime":2254.57,"body":"So if you have a 5G"},{"startTime":2250.19,"endTime":2254.57,"body":"telephone, for example,"},{"startTime":2254.57,"endTime":2257.92,"body":"then one of the principle"},{"startTime":2254.57,"endTime":2257.92,"body":"differences between 5G and 4G"},{"startTime":2257.92,"endTime":2262.92,"body":"is its, the use of these new"},{"startTime":2257.92,"endTime":2262.92,"body":"error control coding standards."},{"startTime":2264.52,"endTime":2266.28,"body":"Right, brief diversion though."},{"startTime":2266.28,"endTime":2270.863,"body":"Golay, Golay codes, first"},{"startTime":2266.28,"endTime":2270.863,"body":"codes, systematic codes."},{"startTime":2272.44,"endTime":2274.49,"body":"This is really a sort"},{"startTime":2272.44,"endTime":2274.49,"body":"of mathematician's joke,"},{"startTime":2274.49,"endTime":2275.89,"body":"but I thought I would,"},{"startTime":2275.89,"endTime":2280.89,"body":"this is the complete"},{"startTime":2275.89,"endTime":2280.89,"body":"paper from Golay in 1949."},{"startTime":2282.6,"endTime":2285.98,"body":"Elwyn Berlekamp, who was one"},{"startTime":2282.6,"endTime":2285.98,"body":"of the sort of great gurus"},{"startTime":2285.98,"endTime":2287.88,"body":"of coding theory, and a"},{"startTime":2285.98,"endTime":2287.88,"body":"friend of Martin Gardner"},{"startTime":2287.88,"endTime":2289.8,"body":"who died a few years ago,"},{"startTime":2289.8,"endTime":2292.45,"body":"called this the single"},{"startTime":2289.8,"endTime":2292.45,"body":"best published paper"},{"startTime":2292.45,"endTime":2294.27,"body":"in coding theory."},{"startTime":2294.27,"endTime":2296.55,"body":"Now, the reason this"},{"startTime":2294.27,"endTime":2296.55,"body":"is mildly entertaining"},{"startTime":2296.55,"endTime":2298.67,"body":"to people like me is because nowadays"},{"startTime":2298.67,"endTime":2299.97,"body":"you don't seem to be able to get away with"},{"startTime":2299.97,"endTime":2303.72,"body":"less than 30 pages saying"},{"startTime":2299.97,"endTime":2303.72,"body":"much less than this guy said."},{"startTime":2303.72,"endTime":2308.48,"body":"This is the ultimate compact bit of paper."},{"startTime":2308.48,"endTime":2313.13,"body":"It's so compact that many,"},{"startTime":2308.48,"endTime":2313.13,"body":"many, many, many, many papers"},{"startTime":2313.13,"endTime":2315.73,"body":"have been written on"},{"startTime":2313.13,"endTime":2315.73,"body":"Golay code subsequently,"},{"startTime":2315.73,"endTime":2319.72,"body":"basically unpacking and trying"},{"startTime":2315.73,"endTime":2319.72,"body":"to explain how they work."},{"startTime":2319.72,"endTime":2320.66,"body":"And I'm not going to be,"},{"startTime":2320.66,"endTime":2322.62,"body":"I'm not going to contribute"},{"startTime":2320.66,"endTime":2322.62,"body":"to that literature,"},{"startTime":2322.62,"endTime":2324.32,"body":"but I thought it might amuse you to see"},{"startTime":2324.32,"endTime":2327.2,"body":"Marcel Golay's original paper."},{"startTime":2327.2,"endTime":2330.603,"body":"Now then, back to Hamming"},{"startTime":2327.2,"endTime":2330.603,"body":"and his innovations."},{"startTime":2331.73,"endTime":2334.047,"body":"I'd like to talk a little"},{"startTime":2331.73,"endTime":2334.047,"body":"bit about modern codes,"},{"startTime":2334.047,"endTime":2335.42,"body":"and in order to do that"},{"startTime":2335.42,"endTime":2339.053,"body":"I want to explain something"},{"startTime":2335.42,"endTime":2339.053,"body":"called Hamming distance."},{"startTime":2341.39,"endTime":2345.76,"body":"So this is, just take your mind back"},{"startTime":2345.76,"endTime":2349.18,"body":"to our 7,4 Hamming code, so has four bits."},{"startTime":2349.18,"endTime":2350.93,"body":"So what I've done here is I've enumerated"},{"startTime":2350.93,"endTime":2353.6,"body":"all the possible"},{"startTime":2350.93,"endTime":2353.6,"body":"combinations of four bits."},{"startTime":2353.6,"endTime":2356.74,"body":"So there are 16 combinations there."},{"startTime":2356.74,"endTime":2359.53,"body":"And as I said, we sort of pack those into"},{"startTime":2359.53,"endTime":2362.193,"body":"a seven bit block or word."},{"startTime":2363.17,"endTime":2364.57,"body":"So let's do that."},{"startTime":2364.57,"endTime":2368.13,"body":"Now this time though,"},{"startTime":2364.57,"endTime":2368.13,"body":"what I'm going to do is"},{"startTime":2368.13,"endTime":2369.61,"body":"pack them all on one end."},{"startTime":2369.61,"endTime":2371.39,"body":"I said, I warned you I"},{"startTime":2369.61,"endTime":2371.39,"body":"was going to do this."},{"startTime":2371.39,"endTime":2375.14,"body":"Instead of Hamming interspersed"},{"startTime":2371.39,"endTime":2375.14,"body":"the data with the parity,"},{"startTime":2375.14,"endTime":2376.32,"body":"nowadays it's quite common"},{"startTime":2376.32,"endTime":2378.76,"body":"to put all the parities on one end."},{"startTime":2378.76,"endTime":2380.5,"body":"Doesn't affect the mathematical properties"},{"startTime":2380.5,"endTime":2382.5,"body":"of the code whatsoever."},{"startTime":2382.5,"endTime":2384.65,"body":"It just makes it easier to"},{"startTime":2382.5,"endTime":2384.65,"body":"explain to undergraduates."},{"startTime":2384.65,"endTime":2386.853,"body":"So that's why people tend to do it."},{"startTime":2388.75,"endTime":2391.78,"body":"Now, what you might ask"},{"startTime":2388.75,"endTime":2391.78,"body":"is you might say, well,"},{"startTime":2391.78,"endTime":2395.0,"body":"if I make an error on these codes,"},{"startTime":2395.0,"endTime":2396.95,"body":"how many errors could I make"},{"startTime":2396.95,"endTime":2401.03,"body":"before one of these transmitted words,"},{"startTime":2401.03,"endTime":2403.42,"body":"on the right hand side,"},{"startTime":2401.03,"endTime":2403.42,"body":"becomes another one."},{"startTime":2403.42,"endTime":2406.13,"body":"That would be an interesting"},{"startTime":2403.42,"endTime":2406.13,"body":"thing to know, right?"},{"startTime":2406.13,"endTime":2410.53,"body":"So if I was interested in say"},{"startTime":2406.13,"endTime":2410.53,"body":"the second word on the list,"},{"startTime":2410.53,"endTime":2413.99,"body":"and I wanted to compare its"},{"startTime":2410.53,"endTime":2413.99,"body":"distance to say, I dunno,"},{"startTime":2413.99,"endTime":2415.3,"body":"the ninth word,"},{"startTime":2415.3,"endTime":2418.55,"body":"what I would do is I would"},{"startTime":2415.3,"endTime":2418.55,"body":"pick out the second word"},{"startTime":2418.55,"endTime":2419.87,"body":"and the ninth word,"},{"startTime":2419.87,"endTime":2422.24,"body":"and then I would look how"},{"startTime":2419.87,"endTime":2422.24,"body":"different they were, okay."},{"startTime":2422.24,"endTime":2426.08,"body":"So we can do that now, the"},{"startTime":2422.24,"endTime":2426.08,"body":"first two bits are identical"},{"startTime":2426.08,"endTime":2427.53,"body":"and then there's a difference,"},{"startTime":2427.53,"endTime":2429.35,"body":"and then there's a"},{"startTime":2427.53,"endTime":2429.35,"body":"difference, and a difference,"},{"startTime":2429.35,"endTime":2431.49,"body":"an identical, an identical,"},{"startTime":2429.35,"endTime":2431.49,"body":"and a difference."},{"startTime":2431.49,"endTime":2434.8,"body":"So I could just sort of scan across there,"},{"startTime":2434.8,"endTime":2438.02,"body":"and my crosses aren't"},{"startTime":2434.8,"endTime":2438.02,"body":"quite in the right place,"},{"startTime":2438.02,"endTime":2441.17,"body":"but you get the idea, a cross"},{"startTime":2438.02,"endTime":2441.17,"body":"means there's a difference."},{"startTime":2441.17,"endTime":2442.91,"body":"And so what we would say is that"},{"startTime":2442.91,"endTime":2445.83,"body":"the Hamming distance"},{"startTime":2442.91,"endTime":2445.83,"body":"between two code words,"},{"startTime":2445.83,"endTime":2449.253,"body":"in this case two and nine, is four."},{"startTime":2450.13,"endTime":2452.52,"body":"So it's a sort of measure of how separate"},{"startTime":2452.52,"endTime":2453.74,"body":"the code words are."},{"startTime":2453.74,"endTime":2455.27,"body":"You can see where I'm"},{"startTime":2453.74,"endTime":2455.27,"body":"going with this, can't you,"},{"startTime":2455.27,"endTime":2457.12,"body":"which is you want those code words to be"},{"startTime":2457.12,"endTime":2458.55,"body":"as separate as possible,"},{"startTime":2458.55,"endTime":2461.98,"body":"'cause you want them to be"},{"startTime":2458.55,"endTime":2461.98,"body":"very resilient to errors."},{"startTime":2461.98,"endTime":2464.33,"body":"Okay, well let's do"},{"startTime":2461.98,"endTime":2464.33,"body":"that with all the codes."},{"startTime":2464.33,"endTime":2466.5,"body":"So that's what I've done here"},{"startTime":2466.5,"endTime":2468.85,"body":"in this great big matrix of differences,"},{"startTime":2468.85,"endTime":2472.53,"body":"and that's the intersection"},{"startTime":2468.85,"endTime":2472.53,"body":"of two and nine."},{"startTime":2472.53,"endTime":2475.52,"body":"So it's symmetrical, so"},{"startTime":2472.53,"endTime":2475.52,"body":"let's just remove the top"},{"startTime":2475.52,"endTime":2477.65,"body":"and let's just measure down."},{"startTime":2477.65,"endTime":2480.65,"body":"And you can see, I think fairly quickly,"},{"startTime":2480.65,"endTime":2482.77,"body":"just by scanning your"},{"startTime":2480.65,"endTime":2482.77,"body":"eye down these columns,"},{"startTime":2482.77,"endTime":2485.4,"body":"that the smallest number"},{"startTime":2482.77,"endTime":2485.4,"body":"for this code is three,"},{"startTime":2485.4,"endTime":2487.07,"body":"and the largest is seven."},{"startTime":2487.07,"endTime":2488.533,"body":"So the Hamming code,"},{"startTime":2489.38,"endTime":2492.64,"body":"if we want to take a sort"},{"startTime":2489.38,"endTime":2492.64,"body":"of worst case analysis,"},{"startTime":2492.64,"endTime":2493.93,"body":"and there might be very good reasons"},{"startTime":2493.93,"endTime":2494.95,"body":"why we don't want to do that,"},{"startTime":2494.95,"endTime":2496.34,"body":"but let's just take a worst case."},{"startTime":2496.34,"endTime":2499.76,"body":"The worst case is that there"},{"startTime":2496.34,"endTime":2499.76,"body":"are two words that we send"},{"startTime":2499.76,"endTime":2502.263,"body":"that are different only by three bits."},{"startTime":2503.62,"endTime":2508.62,"body":"So can you see why a 7,4 Hamming code"},{"startTime":2508.92,"endTime":2513.55,"body":"can guarantee to correct any single error?"},{"startTime":2513.55,"endTime":2517.02,"body":"Because if you think of"},{"startTime":2513.55,"endTime":2517.02,"body":"this in a sort of space,"},{"startTime":2517.02,"endTime":2519.59,"body":"here's one code word,"},{"startTime":2517.02,"endTime":2519.59,"body":"here's the other one,"},{"startTime":2519.59,"endTime":2521.96,"body":"I'm going to have to do"},{"startTime":2519.59,"endTime":2521.96,"body":"three hops to get to this."},{"startTime":2521.96,"endTime":2524.13,"body":"If I've, the noise makes me one hop,"},{"startTime":2524.13,"endTime":2525.77,"body":"I'm still close enough to this one"},{"startTime":2525.77,"endTime":2528.77,"body":"to always guarantee that I can"},{"startTime":2525.77,"endTime":2528.77,"body":"find my way back to that one."},{"startTime":2530.15,"endTime":2533.27,"body":"If I make more than three hops,"},{"startTime":2533.27,"endTime":2536.42,"body":"I might be able to detect"},{"startTime":2533.27,"endTime":2536.42,"body":"that there's an error,"},{"startTime":2536.42,"endTime":2539.313,"body":"but I might not be able"},{"startTime":2536.42,"endTime":2539.313,"body":"to guarantee correction."},{"startTime":2540.8,"endTime":2545.8,"body":"So this is, one of the"},{"startTime":2540.8,"endTime":2545.8,"body":"ways of speaking about this"},{"startTime":2546.61,"endTime":2550.56,"body":"is we say this code has a minimum"},{"startTime":2546.61,"endTime":2550.56,"body":"Hamming distance of three."},{"startTime":2550.56,"endTime":2554.55,"body":"And, well, let me just sort"},{"startTime":2550.56,"endTime":2554.55,"body":"of sketch that for you,"},{"startTime":2554.55,"endTime":2557.56,"body":"and then I'll talk about"},{"startTime":2554.55,"endTime":2557.56,"body":"some of the consequences."},{"startTime":2557.56,"endTime":2560.4,"body":"So how to visualize this."},{"startTime":2560.4,"endTime":2563.31,"body":"Well, you'll see diagrams in, you know,"},{"startTime":2563.31,"endTime":2566.31,"body":"of people trying to plot"},{"startTime":2563.31,"endTime":2566.31,"body":"seven dimensional space"},{"startTime":2566.31,"endTime":2568.55,"body":"and the distance between"},{"startTime":2566.31,"endTime":2568.55,"body":"seven dimensional spaces,"},{"startTime":2568.55,"endTime":2570.81,"body":"and it's a bit, you know, a bit alarming,"},{"startTime":2570.81,"endTime":2572.68,"body":"'cause we can't visualize"},{"startTime":2570.81,"endTime":2572.68,"body":"seven dimensional space."},{"startTime":2572.68,"endTime":2574.43,"body":"And most people project it onto 2D."},{"startTime":2576.8,"endTime":2579.36,"body":"And here I've used a technique"},{"startTime":2579.36,"endTime":2581.85,"body":"that I haven't seen used before"},{"startTime":2579.36,"endTime":2581.85,"body":"actually, for this purpose,"},{"startTime":2581.85,"endTime":2586.33,"body":"called t-SNE, which is an"},{"startTime":2581.85,"endTime":2586.33,"body":"adaptive visualization technique."},{"startTime":2586.33,"endTime":2588.75,"body":"But this is essentially me"},{"startTime":2586.33,"endTime":2588.75,"body":"taking a seven dimensional space"},{"startTime":2588.75,"endTime":2591.64,"body":"and mapping it down to 2D."},{"startTime":2591.64,"endTime":2594.41,"body":"So if I took two words,"},{"startTime":2594.41,"endTime":2597.713,"body":"and this is sort of somewhat conceptual."},{"startTime":2599.69,"endTime":2601.76,"body":"Let's just take two words."},{"startTime":2601.76,"endTime":2603.773,"body":"This is the line that separates them."},{"startTime":2604.86,"endTime":2609.02,"body":"And there's a hyper-sphere"},{"startTime":2604.86,"endTime":2609.02,"body":"that sits around"},{"startTime":2609.02,"endTime":2610.46,"body":"each one of these,"},{"startTime":2610.46,"endTime":2614.95,"body":"and that represents the"},{"startTime":2610.46,"endTime":2614.95,"body":"distance around a code word,"},{"startTime":2614.95,"endTime":2616.113,"body":"which we can correct."},{"startTime":2616.98,"endTime":2621.98,"body":"And what you'd want to do is"},{"startTime":2616.98,"endTime":2621.98,"body":"perfectly fill out the space"},{"startTime":2622.37,"endTime":2624.09,"body":"with as many code words as you can,"},{"startTime":2624.09,"endTime":2626.69,"body":"where there's a maximum"},{"startTime":2624.09,"endTime":2626.69,"body":"amount of space between them."},{"startTime":2626.69,"endTime":2631.69,"body":"And codes that do that, in"},{"startTime":2626.69,"endTime":2631.69,"body":"analogy with chemical physics,"},{"startTime":2631.9,"endTime":2633.99,"body":"where we talk about perfect packing,"},{"startTime":2633.99,"endTime":2635.453,"body":"are called perfect codes."},{"startTime":2636.56,"endTime":2638.67,"body":"There's a lot written about perfect codes,"},{"startTime":2638.67,"endTime":2641.88,"body":"but most codes are not perfect codes,"},{"startTime":2641.88,"endTime":2643.46,"body":"and they don't need to be."},{"startTime":2643.46,"endTime":2646.83,"body":"There's also a little bit"},{"startTime":2643.46,"endTime":2646.83,"body":"of rubbish written about"},{"startTime":2646.83,"endTime":2648.68,"body":"minimum distance codes."},{"startTime":2648.68,"endTime":2649.9,"body":"People tend to say"},{"startTime":2652.17,"endTime":2655.78,"body":"if the minimum distance"},{"startTime":2652.17,"endTime":2655.78,"body":"in this code is say three,"},{"startTime":2655.78,"endTime":2660.743,"body":"I won't bother to detect"},{"startTime":2655.78,"endTime":2660.743,"body":"more than three over two,"},{"startTime":2662.67,"endTime":2667.67,"body":"one error, because I can't guarantee it."},{"startTime":2667.75,"endTime":2670.41,"body":"That's not a good"},{"startTime":2667.75,"endTime":2670.41,"body":"strategy, as it turns out."},{"startTime":2670.41,"endTime":2673.96,"body":"It's best to, even though"},{"startTime":2670.41,"endTime":2673.96,"body":"you, in a real code,"},{"startTime":2673.96,"endTime":2676.24,"body":"the distance between"},{"startTime":2673.96,"endTime":2676.24,"body":"various words will differ,"},{"startTime":2676.24,"endTime":2678.423,"body":"maybe quite substantially,"},{"startTime":2679.49,"endTime":2682.38,"body":"you are better off working"},{"startTime":2679.49,"endTime":2682.38,"body":"with what you've got"},{"startTime":2682.38,"endTime":2684.593,"body":"rather than working to"},{"startTime":2682.38,"endTime":2684.593,"body":"some sort of minimum."},{"startTime":2685.5,"endTime":2687.71,"body":"There's an entertaining"},{"startTime":2685.5,"endTime":2687.71,"body":"guy called David Mackay"},{"startTime":2687.71,"endTime":2691.4,"body":"who called this a terrible"},{"startTime":2687.71,"endTime":2691.4,"body":"example of worst case-ism."},{"startTime":2691.4,"endTime":2693.29,"body":"Okay, well that's what a sort of toy code,"},{"startTime":2693.29,"endTime":2694.87,"body":"I don't want to call this a toy code."},{"startTime":2694.87,"endTime":2699.29,"body":"There are devices out there"},{"startTime":2694.87,"endTime":2699.29,"body":"that still use Hamming 7,4 code,"},{"startTime":2699.29,"endTime":2701.04,"body":"and without that 7,4 code,"},{"startTime":2701.04,"endTime":2702.52,"body":"we'd be getting errors all over the place."},{"startTime":2702.52,"endTime":2705.843,"body":"So it's not a toy code, but"},{"startTime":2702.52,"endTime":2705.843,"body":"let's look at a big code."},{"startTime":2706.76,"endTime":2709.01,"body":"This is what you do, this is"},{"startTime":2706.76,"endTime":2709.01,"body":"what happens when you have"},{"startTime":2709.01,"endTime":2711.31,"body":"a bigger code."},{"startTime":2711.31,"endTime":2713.17,"body":"This is the code used in,"},{"startTime":2713.17,"endTime":2715.81,"body":"if you've got an up to date wifi system"},{"startTime":2715.81,"endTime":2718.493,"body":"and it supports what's called 802.11n,"},{"startTime":2720.1,"endTime":2722.84,"body":"you can go and look at table F-1,"},{"startTime":2722.84,"endTime":2727.84,"body":"rates 3/4 code, page"},{"startTime":2722.84,"endTime":2727.84,"body":"2304 is where I found it."},{"startTime":2728.95,"endTime":2730.26,"body":"Just polish my halo,"},{"startTime":2730.26,"endTime":2731.78,"body":"I'm pointing out that I scanned through"},{"startTime":2731.78,"endTime":2734.97,"body":"3,500 pages of standards looking for this,"},{"startTime":2734.97,"endTime":2736.02,"body":"ladies and gentlemen."},{"startTime":2736.86,"endTime":2741.86,"body":"This is a block of 648"},{"startTime":2736.86,"endTime":2741.86,"body":"bits, of which 486 are data."},{"startTime":2745.91,"endTime":2749.58,"body":"And this is my little visualization"},{"startTime":2745.91,"endTime":2749.58,"body":"of it on a single plane."},{"startTime":2749.58,"endTime":2754.58,"body":"And maybe you can't look at"},{"startTime":2749.58,"endTime":2754.58,"body":"this and intuit what's going on,"},{"startTime":2756.26,"endTime":2758.49,"body":"but you can certainly get the impression"},{"startTime":2758.49,"endTime":2762.52,"body":"that not all code words"},{"startTime":2758.49,"endTime":2762.52,"body":"are equally spaced,"},{"startTime":2762.52,"endTime":2765.06,"body":"and there's big holes in the code."},{"startTime":2765.06,"endTime":2769.13,"body":"And a lot of human ingenuity"},{"startTime":2765.06,"endTime":2769.13,"body":"goes into finding these codes."},{"startTime":2769.13,"endTime":2773.95,"body":"This is an example of a"},{"startTime":2769.13,"endTime":2773.95,"body":"low-density parity-check code."},{"startTime":2773.95,"endTime":2775.53,"body":"Very interesting history."},{"startTime":2775.53,"endTime":2777.25,"body":"The idea of these was first thought up"},{"startTime":2777.25,"endTime":2780.96,"body":"by a guy called Gallager in the 1970s,"},{"startTime":2780.96,"endTime":2783.71,"body":"but he didn't have enough"},{"startTime":2780.96,"endTime":2783.71,"body":"computing power to find them."},{"startTime":2784.96,"endTime":2787.38,"body":"And later authors later"},{"startTime":2784.96,"endTime":2787.38,"body":"sort of figured out"},{"startTime":2787.38,"endTime":2790.38,"body":"how best to find these."},{"startTime":2790.38,"endTime":2794.78,"body":"One of the most irritating"},{"startTime":2790.38,"endTime":2794.78,"body":"things about Gallager codes,"},{"startTime":2794.78,"endTime":2797.843,"body":"or LDPC codes, is describing them."},{"startTime":2799.04,"endTime":2800.98,"body":"Imagine you've got"},{"startTime":2800.98,"endTime":2803.51,"body":"several hundred parity digits to describe."},{"startTime":2803.51,"endTime":2806.35,"body":"So you've got to write them in a document."},{"startTime":2806.35,"endTime":2809.06,"body":"Unless you can find a system"},{"startTime":2806.35,"endTime":2809.06,"body":"for writing them down,"},{"startTime":2809.06,"endTime":2811.27,"body":"you've got a license for"},{"startTime":2809.06,"endTime":2811.27,"body":"trouble, haven't you,"},{"startTime":2811.27,"endTime":2813.27,"body":"'cause you're going to"},{"startTime":2811.27,"endTime":2813.27,"body":"make errors in them."},{"startTime":2813.27,"endTime":2815.13,"body":"So there's quite a lot of ingenuity"},{"startTime":2815.13,"endTime":2818.78,"body":"in finding ways to decompose"},{"startTime":2818.78,"endTime":2823.78,"body":"the apparently random"},{"startTime":2818.78,"endTime":2823.78,"body":"collections of parity digits"},{"startTime":2823.87,"endTime":2826.21,"body":"into ways that people"},{"startTime":2823.87,"endTime":2826.21,"body":"can humanly understand."},{"startTime":2826.21,"endTime":2829.54,"body":"So a lot of work in LDPC codes"},{"startTime":2826.21,"endTime":2829.54,"body":"has been in trying to find"},{"startTime":2829.54,"endTime":2831.67,"body":"butterflies or bits of algebra"},{"startTime":2831.67,"endTime":2834.15,"body":"that allow you to compute these matrices."},{"startTime":2834.15,"endTime":2836.15,"body":"And in fact, if you look"},{"startTime":2834.15,"endTime":2836.15,"body":"at the wifi standard,"},{"startTime":2836.15,"endTime":2836.983,"body":"that's what it does."},{"startTime":2836.983,"endTime":2839.72,"body":"It has a set of sub-matrices"},{"startTime":2836.983,"endTime":2839.72,"body":"that you sort of,"},{"startTime":2839.72,"endTime":2843.01,"body":"you move across a bigger"},{"startTime":2839.72,"endTime":2843.01,"body":"matrix to create it."},{"startTime":2843.01,"endTime":2845.66,"body":"Polar codes are even"},{"startTime":2843.01,"endTime":2845.66,"body":"better because they have"},{"startTime":2845.66,"endTime":2848.71,"body":"a more principled way of doing so."},{"startTime":2848.71,"endTime":2851.9,"body":"Now I can see you're longing to look at"},{"startTime":2851.9,"endTime":2853.78,"body":"the performance of these."},{"startTime":2853.78,"endTime":2857.66,"body":"It's a bit complicated to"},{"startTime":2853.78,"endTime":2857.66,"body":"compute it myself, so I didn't."},{"startTime":2857.66,"endTime":2861.99,"body":"I picked an example from a very nice book"},{"startTime":2861.99,"endTime":2866.55,"body":"by a guy called David Mackay,"},{"startTime":2861.99,"endTime":2866.55,"body":"who sadly died recently."},{"startTime":2866.55,"endTime":2869.16,"body":"Very enterprising and ingenious man"},{"startTime":2869.16,"endTime":2871.7,"body":"who worked on LDPC codes,"},{"startTime":2869.16,"endTime":2871.7,"body":"amongst other things."},{"startTime":2871.7,"endTime":2874.25,"body":"He was also one of the"},{"startTime":2871.7,"endTime":2874.25,"body":"government chief scientists."},{"startTime":2876.169,"endTime":2880.57,"body":"And this is sort of,"},{"startTime":2876.169,"endTime":2880.57,"body":"on the left hand side,"},{"startTime":2880.57,"endTime":2883.99,"body":"we've got repeat codes, Hamming codes,"},{"startTime":2883.99,"endTime":2886.71,"body":"and all the sort of conventional"},{"startTime":2883.99,"endTime":2886.71,"body":"stuff that didn't exist,"},{"startTime":2886.71,"endTime":2888.44,"body":"and slam right in here,"},{"startTime":2888.44,"endTime":2890.93,"body":"it's got error bars 'cause"},{"startTime":2888.44,"endTime":2890.93,"body":"he's doing a simulation,"},{"startTime":2890.93,"endTime":2893.35,"body":"is low-density parity-check code."},{"startTime":2893.35,"endTime":2894.89,"body":"So you can see what's happened,"},{"startTime":2894.89,"endTime":2897.4,"body":"polar codes will be similar down here."},{"startTime":2897.4,"endTime":2900.27,"body":"So what's happened recently"},{"startTime":2897.4,"endTime":2900.27,"body":"over the last five to 10 years"},{"startTime":2900.27,"endTime":2902.14,"body":"is these new codes have appeared,"},{"startTime":2902.14,"endTime":2904.61,"body":"and suddenly everyone"},{"startTime":2902.14,"endTime":2904.61,"body":"is ditching everything"},{"startTime":2904.61,"endTime":2907.463,"body":"in a desperate attempt to get to these,"},{"startTime":2909.06,"endTime":2914.06,"body":"which is, or that's what's"},{"startTime":2909.06,"endTime":2914.06,"body":"caused the excitement."},{"startTime":2914.47,"endTime":2917.63,"body":"What's beautiful about"},{"startTime":2914.47,"endTime":2917.63,"body":"these codes, however,"},{"startTime":2917.63,"endTime":2920.4,"body":"is the mathematics and the understanding"},{"startTime":2920.4,"endTime":2922.26,"body":"that you got from the"},{"startTime":2920.4,"endTime":2922.26,"body":"early part of this lecture,"},{"startTime":2922.26,"endTime":2925.59,"body":"the Hamming understanding,"},{"startTime":2922.26,"endTime":2925.59,"body":"if you like, is the same."},{"startTime":2925.59,"endTime":2929.41,"body":"It's the same sort of"},{"startTime":2925.59,"endTime":2929.41,"body":"handle-turning methodology."},{"startTime":2929.41,"endTime":2932.08,"body":"It's just bigger, on a bigger scale."},{"startTime":2932.08,"endTime":2933.04,"body":"And that's what's so nice."},{"startTime":2933.04,"endTime":2935.75,"body":"A lot of the competing"},{"startTime":2933.04,"endTime":2935.75,"body":"codes don't do that."},{"startTime":2935.75,"endTime":2939.11,"body":"Now those of you who are"},{"startTime":2935.75,"endTime":2939.11,"body":"of a academic disposition"},{"startTime":2939.11,"endTime":2942.603,"body":"may be puzzling what's this"},{"startTime":2939.11,"endTime":2942.603,"body":"thing called GV down here."},{"startTime":2943.54,"endTime":2947.023,"body":"That is the Gilbert-Varshamov conjecture."},{"startTime":2948.27,"endTime":2950.78,"body":"It goes back to the my"},{"startTime":2948.27,"endTime":2950.78,"body":"worst case-ism example"},{"startTime":2950.78,"endTime":2952.58,"body":"I was talking about later."},{"startTime":2952.58,"endTime":2957.58,"body":"So if we assume that because"},{"startTime":2952.58,"endTime":2957.58,"body":"a code cannot guarantee"},{"startTime":2960.89,"endTime":2963.48,"body":"to correct more than T errors,"},{"startTime":2963.48,"endTime":2965.97,"body":"then we shouldn't correct more than T."},{"startTime":2965.97,"endTime":2967.95,"body":"So if you operate under that wisdom,"},{"startTime":2967.95,"endTime":2970.78,"body":"you say this code has a"},{"startTime":2967.95,"endTime":2970.78,"body":"minimum distance of T,"},{"startTime":2970.78,"endTime":2973.29,"body":"I am not going to bother"},{"startTime":2970.78,"endTime":2973.29,"body":"to correct more than T,"},{"startTime":2973.29,"endTime":2978.29,"body":"then this is where GV would"},{"startTime":2973.29,"endTime":2978.29,"body":"sit, for this example."},{"startTime":2978.29,"endTime":2979.47,"body":"So there's a limit,"},{"startTime":2979.47,"endTime":2983.58,"body":"and it doesn't get you"},{"startTime":2979.47,"endTime":2983.58,"body":"to the Shannon limit."},{"startTime":2983.58,"endTime":2985.33,"body":"It's got a conjecture,"},{"startTime":2983.58,"endTime":2985.33,"body":"'cause no one can prove it."},{"startTime":2985.33,"endTime":2987.16,"body":"So I mention it because I can see"},{"startTime":2987.16,"endTime":2990.61,"body":"very intelligent people in the audience."},{"startTime":2990.61,"endTime":2993.62,"body":"So if you have nothing to do this evening,"},{"startTime":2993.62,"endTime":2995.24,"body":"could you just go and prove for me"},{"startTime":2995.24,"endTime":2998.463,"body":"that the Gilbert-Varshamov"},{"startTime":2995.24,"endTime":2998.463,"body":"conjecture works?"},{"startTime":3000.12,"endTime":3003.7,"body":"If you can do that, just put"},{"startTime":3000.12,"endTime":3003.7,"body":"my name on the paper please"},{"startTime":3003.7,"endTime":3006.12,"body":"and submit it to IEE"},{"startTime":3003.7,"endTime":3006.12,"body":"Trans Information Theory,"},{"startTime":3006.12,"endTime":3007.75,"body":"'cause I've always wanted a paper in"},{"startTime":3007.75,"endTime":3010.04,"body":"IEE Trans Information Theory."},{"startTime":3010.04,"endTime":3014.233,"body":"So there's been this sort"},{"startTime":3010.04,"endTime":3014.233,"body":"of leap forward in coding,"},{"startTime":3015.12,"endTime":3018.64,"body":"and a number of rather innovative codes,"},{"startTime":3018.64,"endTime":3021.975,"body":"but not linear block codes of"},{"startTime":3018.64,"endTime":3021.975,"body":"the type I was talking about,"},{"startTime":3021.975,"endTime":3024.9,"body":"have just been sort of"},{"startTime":3021.975,"endTime":3024.9,"body":"left for dust, as it were."},{"startTime":3024.9,"endTime":3027.333,"body":"So all those convolutional"},{"startTime":3024.9,"endTime":3027.333,"body":"codes and all those other codes"},{"startTime":3027.333,"endTime":3029.28,"body":"have been left for dust."},{"startTime":3029.28,"endTime":3032.9,"body":"But it's the same ideas"},{"startTime":3029.28,"endTime":3032.9,"body":"that were dreamt up"},{"startTime":3032.9,"endTime":3037.9,"body":"in basically 1950 by Richard"},{"startTime":3032.9,"endTime":3037.9,"body":"Hamming and Claude Shannon."},{"startTime":3039.78,"endTime":3041.35,"body":"Hamming is a very interesting man,"},{"startTime":3041.35,"endTime":3043.92,"body":"not just because he invented these codes,"},{"startTime":3043.92,"endTime":3045.67,"body":"but he also wrote a book about"},{"startTime":3045.67,"endTime":3047.96,"body":"what it was like to be creative."},{"startTime":3047.96,"endTime":3050.469,"body":"And if you've got a moment do read it,"},{"startTime":3050.469,"endTime":3051.46,"body":"it's an interesting read."},{"startTime":3051.46,"endTime":3055.78,"body":"What it's like to invent something."},{"startTime":3055.78,"endTime":3057.58,"body":"And I'm sure you've"},{"startTime":3055.78,"endTime":3057.58,"body":"noticed in these lectures,"},{"startTime":3057.58,"endTime":3059.62,"body":"I've been talking about inventions,"},{"startTime":3059.62,"endTime":3063.15,"body":"and some of the inventions"},{"startTime":3059.62,"endTime":3063.15,"body":"I've been talking about"},{"startTime":3063.15,"endTime":3066.9,"body":"are sort of old fashioned"},{"startTime":3063.15,"endTime":3066.9,"body":"professor brainstorm inventions,"},{"startTime":3066.9,"endTime":3069.33,"body":"they've been artifacts."},{"startTime":3069.33,"endTime":3073.72,"body":"But very often they're"},{"startTime":3069.33,"endTime":3073.72,"body":"inventions of algorithms,"},{"startTime":3073.72,"endTime":3075.14,"body":"or ways of doing things."},{"startTime":3075.14,"endTime":3080.14,"body":"And this is lecture has been"},{"startTime":3075.14,"endTime":3080.14,"body":"about inventions of algorithms,"},{"startTime":3080.49,"endTime":3084.3,"body":"and I suppose also theories of algorithms,"},{"startTime":3084.3,"endTime":3088.423,"body":"which is a very attractive sort of idea."},{"startTime":3089.47,"endTime":3092.32,"body":"I would just briefly point you"},{"startTime":3092.32,"endTime":3096.1,"body":"to future lectures in this series,"},{"startTime":3096.1,"endTime":3098.6,"body":"and the next one is"},{"startTime":3096.1,"endTime":3098.6,"body":"also about an invention,"},{"startTime":3098.6,"endTime":3102.35,"body":"but it's a different"},{"startTime":3098.6,"endTime":3102.35,"body":"sort of invention, right?"},{"startTime":3102.35,"endTime":3106.5,"body":"It's an invention of a system,"},{"startTime":3106.5,"endTime":3109.34,"body":"and these aren't really"},{"startTime":3106.5,"endTime":3109.34,"body":"understood very well"},{"startTime":3109.34,"endTime":3110.79,"body":"in the modern world."},{"startTime":3110.79,"endTime":3113.26,"body":"People sort of tell their"},{"startTime":3110.79,"endTime":3113.26,"body":"kids about inventions"},{"startTime":3113.26,"endTime":3115.63,"body":"as vacuum cleaners and devices,"},{"startTime":3115.63,"endTime":3120.63,"body":"but a lot of our modern world"},{"startTime":3115.63,"endTime":3120.63,"body":"is inventions of agreements"},{"startTime":3121.14,"endTime":3123.89,"body":"on protocols for the MPEG standard,"},{"startTime":3123.89,"endTime":3125.55,"body":"which we talked about in previous lectures"},{"startTime":3125.55,"endTime":3127.08,"body":"as an example of that,"},{"startTime":3127.08,"endTime":3129.66,"body":"and cellular telephony is one of the most"},{"startTime":3129.66,"endTime":3131.04,"body":"interesting examples of that"},{"startTime":3131.04,"endTime":3132.67,"body":"because it's extremely complicated."},{"startTime":3132.67,"endTime":3135.89,"body":"So I hope you will be"},{"startTime":3132.67,"endTime":3135.89,"body":"able to join me then,"},{"startTime":3135.89,"endTime":3140.85,"body":"and needless to say cellular"},{"startTime":3135.89,"endTime":3140.85,"body":"telephony would not happen"},{"startTime":3140.85,"endTime":3144.67,"body":"without the beautiful"},{"startTime":3140.85,"endTime":3144.67,"body":"and powerful mathematics"},{"startTime":3144.67,"endTime":3146.649,"body":"that is error control coding."},{"startTime":3146.649,"endTime":3147.635,"body":"Thank you."},{"startTime":3147.635,"endTime":3150.802,"body":"(audience applauding)"},{"startTime":3151.858,"endTime":3155.36,"body":"- [Man] Thank you very"},{"startTime":3151.858,"endTime":3155.36,"body":"much indeed professor."},{"startTime":3155.36,"endTime":3157.52,"body":"I'm an 83-year-old aging git"},{"startTime":3157.52,"endTime":3158.98,"body":"and I'm making more and more errors"},{"startTime":3158.98,"endTime":3160.853,"body":"as I get older, the frequency."},{"startTime":3164.25,"endTime":3166.14,"body":"I did do a course,"},{"startTime":3166.14,"endTime":3168.01,"body":"I gave it up actually, on cybernetics,"},{"startTime":3168.01,"endTime":3171.7,"body":"because the mathematics"},{"startTime":3168.01,"endTime":3171.7,"body":"in it wasn't explained."},{"startTime":3171.7,"endTime":3174.61,"body":"Ross Ashby's book,"},{"startTime":3171.7,"endTime":3174.61,"body":"\"Introduction to Cybernetics\"."},{"startTime":3174.61,"endTime":3175.692,"body":"- Yeah."},{"startTime":3175.692,"endTime":3177.21,"body":"- [Man] And also, I think"},{"startTime":3175.692,"endTime":3177.21,"body":"I still have a copy of"},{"startTime":3177.21,"endTime":3178.957,"body":"Shannon's Theory of Communication."},{"startTime":3178.957,"endTime":3181.24,"body":"- That's a good read, although it's quite,"},{"startTime":3181.24,"endTime":3183.64,"body":"there are bits of it that"},{"startTime":3181.24,"endTime":3183.64,"body":"are a bit difficult, I mean."},{"startTime":3183.64,"endTime":3186.57,"body":"- [Man] But one of the"},{"startTime":3183.64,"endTime":3186.57,"body":"things that occurs to me."},{"startTime":3186.57,"endTime":3187.74,"body":"I should explain by the way"},{"startTime":3187.74,"endTime":3190.17,"body":"that I'm the world's most ignorant person"},{"startTime":3190.17,"endTime":3194.55,"body":"as regards computers in"},{"startTime":3190.17,"endTime":3194.55,"body":"information technology."},{"startTime":3194.55,"endTime":3197.69,"body":"But at two of the"},{"startTime":3194.55,"endTime":3197.69,"body":"universities I've worked,"},{"startTime":3197.69,"endTime":3200.36,"body":"the very friendly computer assistance boys"},{"startTime":3200.36,"endTime":3202.02,"body":"have said they've had more trouble with me"},{"startTime":3202.02,"endTime":3206.38,"body":"than anybody else on my computer,"},{"startTime":3202.02,"endTime":3206.38,"body":"because I've done things,"},{"startTime":3206.38,"endTime":3207.77,"body":"I don't know what."},{"startTime":3207.77,"endTime":3211.22,"body":"And they said they'd never"},{"startTime":3207.77,"endTime":3211.22,"body":"seen anything like it before."},{"startTime":3211.22,"endTime":3212.97,"body":"And another thing that I had"},{"startTime":3212.97,"endTime":3215.93,"body":"with a previous word processor I had,"},{"startTime":3215.93,"endTime":3218.75,"body":"is when I was beginning to type words,"},{"startTime":3218.75,"endTime":3221.36,"body":"the computer would complete the word,"},{"startTime":3221.36,"endTime":3222.71,"body":"I'd put the first two letters"},{"startTime":3222.71,"endTime":3224.63,"body":"but very often the completed word,"},{"startTime":3224.63,"endTime":3227.36,"body":"when I say often I mean 90% of the time,"},{"startTime":3227.36,"endTime":3229.317,"body":"was not the word I was looking for."},{"startTime":3229.317,"endTime":3233.76,"body":"Does that have any relevance"},{"startTime":3233.76,"endTime":3235.4,"body":"in fact to what you've been talking about?"},{"startTime":3235.4,"endTime":3236.65,"body":"I do apologize for my ignorance,"},{"startTime":3236.65,"endTime":3238.31,"body":"- It does-"},{"startTime":3236.65,"endTime":3238.31,"body":"- which is unlimited."},{"startTime":3238.31,"endTime":3239.51,"body":"- It does."},{"startTime":3239.51,"endTime":3241.02,"body":"I have two things to say to you sir."},{"startTime":3241.02,"endTime":3245.1,"body":"The first one is one of"},{"startTime":3241.02,"endTime":3245.1,"body":"the major systems we use"},{"startTime":3245.1,"endTime":3247.78,"body":"for controlling source code, deciding,"},{"startTime":3247.78,"endTime":3250.26,"body":"getting things that are up"},{"startTime":3247.78,"endTime":3250.26,"body":"to date, is called GitHub."},{"startTime":3250.26,"endTime":3254.057,"body":"And people have often asked"},{"startTime":3250.26,"endTime":3254.057,"body":"why it was called GitHub,"},{"startTime":3254.057,"endTime":3256.21,"body":"and the inventor Linus Torvalds said"},{"startTime":3256.21,"endTime":3258.613,"body":"I called it GitHub because I am a git."},{"startTime":3258.613,"endTime":3261.59,"body":"So don't be embarrassed about that."},{"startTime":3261.59,"endTime":3264.29,"body":"Now, what you're doing"},{"startTime":3261.59,"endTime":3264.29,"body":"when you get auto correct,"},{"startTime":3264.29,"endTime":3266.396,"body":"or when we're having this conversation now"},{"startTime":3266.396,"endTime":3269.42,"body":"and I'm listening to"},{"startTime":3266.396,"endTime":3269.42,"body":"you and working it out,"},{"startTime":3269.42,"endTime":3273.54,"body":"is I'm using this model of what"},{"startTime":3269.42,"endTime":3273.54,"body":"you're likely to be saying."},{"startTime":3273.54,"endTime":3276.71,"body":"And if you go outside that"},{"startTime":3273.54,"endTime":3276.71,"body":"model, as you often do,"},{"startTime":3276.71,"endTime":3279.904,"body":"because you're writing highfalutin prose,"},{"startTime":3279.904,"endTime":3282.79,"body":"obviously you get"},{"startTime":3279.904,"endTime":3282.79,"body":"corrected back to a model."},{"startTime":3282.79,"endTime":3285.15,"body":"And the spell checkers"},{"startTime":3282.79,"endTime":3285.15,"body":"have got better and better"},{"startTime":3285.15,"endTime":3286.61,"body":"at doing this."},{"startTime":3286.61,"endTime":3289.42,"body":"It's the same idea in coding."},{"startTime":3289.42,"endTime":3292.73,"body":"But for error control coding,"},{"startTime":3289.42,"endTime":3292.73,"body":"which is a much lower level,"},{"startTime":3292.73,"endTime":3295.08,"body":"our model is very strict indeed."},{"startTime":3295.08,"endTime":3298.4,"body":"We know what messages could be sent,"},{"startTime":3298.4,"endTime":3302.55,"body":"so we can correct back to"},{"startTime":3298.4,"endTime":3302.55,"body":"the most probable model,"},{"startTime":3302.55,"endTime":3306.92,"body":"and fancy receivers"},{"startTime":3302.55,"endTime":3306.92,"body":"model the whole channel,"},{"startTime":3306.92,"endTime":3309.73,"body":"they model the noise,"},{"startTime":3309.73,"endTime":3312.543,"body":"and they do something"},{"startTime":3309.73,"endTime":3312.543,"body":"called soft decoding."},{"startTime":3313.47,"endTime":3316.58,"body":"You do soft decoding, but"},{"startTime":3313.47,"endTime":3316.58,"body":"it's novel for low level,"},{"startTime":3316.58,"endTime":3318.92,"body":"computer hard software decoding says,"},{"startTime":3318.92,"endTime":3323.92,"body":"I'm not just going to"},{"startTime":3318.92,"endTime":3323.92,"body":"pick the nearest word."},{"startTime":3324.08,"endTime":3325.79,"body":"I'm going to model a probability"},{"startTime":3325.79,"endTime":3327.48,"body":"of various things happening,"},{"startTime":3327.48,"endTime":3329.51,"body":"maybe thinking about the"},{"startTime":3327.48,"endTime":3329.51,"body":"probability of the data"},{"startTime":3329.51,"endTime":3332.55,"body":"in the first place and pick"},{"startTime":3329.51,"endTime":3332.55,"body":"the most probable word."},{"startTime":3332.55,"endTime":3335.93,"body":"So it does that by"},{"startTime":3332.55,"endTime":3335.93,"body":"measuring voltages really."},{"startTime":3335.93,"endTime":3340.93,"body":"So there is an analogy"},{"startTime":3335.93,"endTime":3340.93,"body":"with what you're saying."},{"startTime":3341.377,"endTime":3346.377,"body":"The high level sort of error"},{"startTime":3341.377,"endTime":3346.377,"body":"control that you're used to,"},{"startTime":3347.03,"endTime":3351.663,"body":"however, is too errorful for"},{"startTime":3347.03,"endTime":3351.663,"body":"us to use at a low level."},{"startTime":3352.71,"endTime":3356.6,"body":"If Microsoft spell check"},{"startTime":3352.71,"endTime":3356.6,"body":"was in charge of correcting"},{"startTime":3356.6,"endTime":3359.5,"body":"the bits on your computer system,"},{"startTime":3359.5,"endTime":3362.283,"body":"we would be in a shed load of trouble."},{"startTime":3363.45,"endTime":3365.32,"body":"- [Man] You've spoken mainly, I think,"},{"startTime":3365.32,"endTime":3366.903,"body":"about data transmission."},{"startTime":3367.79,"endTime":3370.81,"body":"What about programs"},{"startTime":3367.79,"endTime":3370.81,"body":"actually inside a computer?"},{"startTime":3370.81,"endTime":3374.55,"body":"Is there any error correction in those"},{"startTime":3374.55,"endTime":3377.75,"body":"when they're transmitted over"},{"startTime":3374.55,"endTime":3377.75,"body":"relatively short distances"},{"startTime":3377.75,"endTime":3379.74,"body":"within a computer?"},{"startTime":3379.74,"endTime":3382.09,"body":"- Yes, that's a very good point."},{"startTime":3382.09,"endTime":3387.09,"body":"Almost every bit of your iPhone"},{"startTime":3382.09,"endTime":3387.09,"body":"or hardware or everything"},{"startTime":3388.14,"endTime":3390.31,"body":"will have error control coding in it."},{"startTime":3390.31,"endTime":3392.6,"body":"'Cause errors are objectionable,"},{"startTime":3392.6,"endTime":3396.523,"body":"and the lower the level"},{"startTime":3392.6,"endTime":3396.523,"body":"you are at the hardware,"},{"startTime":3397.41,"endTime":3400.23,"body":"on the one hand, the less likely it is"},{"startTime":3400.23,"endTime":3402.01,"body":"that you're going to"},{"startTime":3400.23,"endTime":3402.01,"body":"make errors, you know,"},{"startTime":3402.01,"endTime":3404.48,"body":"because it's in a contained system,"},{"startTime":3404.48,"endTime":3406.14,"body":"but the more damaging they are."},{"startTime":3406.14,"endTime":3409.2,"body":"So parity is everywhere."},{"startTime":3409.2,"endTime":3410.84,"body":"You know, parity is everywhere."},{"startTime":3410.84,"endTime":3412.023,"body":"Error control."},{"startTime":3413.79,"endTime":3417.82,"body":"Even your memory stick,"},{"startTime":3413.79,"endTime":3417.82,"body":"the RAM in your computer,"},{"startTime":3417.82,"endTime":3420.64,"body":"will have some parity control on it."},{"startTime":3420.64,"endTime":3424.14,"body":"Your disk most certainly has"},{"startTime":3420.64,"endTime":3424.14,"body":"error control coding on it."},{"startTime":3424.14,"endTime":3425.67,"body":"Anything that goes over the airways"},{"startTime":3425.67,"endTime":3427.18,"body":"has error control coding."},{"startTime":3427.18,"endTime":3430.82,"body":"Everything has ECC, it's"},{"startTime":3427.18,"endTime":3430.82,"body":"just you don't notice."},{"startTime":3430.82,"endTime":3431.99,"body":"And the reason you don't notice"},{"startTime":3431.99,"endTime":3436.04,"body":"is because of this beautiful"},{"startTime":3431.99,"endTime":3436.04,"body":"idea which is, I think,"},{"startTime":3436.04,"endTime":3439.82,"body":"quite counterintuitive,"},{"startTime":3436.04,"endTime":3439.82,"body":"which is whatever the error,"},{"startTime":3439.82,"endTime":3441.443,"body":"we can correct it back to zero."},{"startTime":3443.621,"endTime":3445.36,"body":"I think that's a good time to stop."},{"startTime":3445.36,"endTime":3447.32,"body":"Thank you very much, have a good evening."},{"startTime":3447.32,"endTime":3448.275,"body":"(audience applauding)"}]}