{"version":"1.0.0","segments":[{"speaker":"Alex Hanna","startTime":0.25,"endTime":5.67,"body":"Welcome everyone to Mystery"},{"speaker":"Alex Hanna","startTime":0.25,"endTime":5.67,"body":"AI Hype Theater 3000, where we seek"},{"speaker":"Alex Hanna","startTime":5.67,"endTime":7.72,"body":"catharsis in this age of AI hype."},{"speaker":"Alex Hanna","startTime":8.26,"endTime":11.8,"body":"We find the worst of it and pop it"},{"speaker":"Alex Hanna","startTime":8.26,"endTime":11.8,"body":"with the sharpest needles we can find."},{"speaker":"Emily M. Bender","startTime":12.059,"endTime":14.53,"body":"Along the way, we learn"},{"speaker":"Emily M. Bender","startTime":12.059,"endTime":14.53,"body":"to always read the footnotes, and each"},{"speaker":"Emily M. Bender","startTime":14.53,"endTime":17.83,"body":"time we think we've reached peak AI"},{"speaker":"Emily M. Bender","startTime":14.53,"endTime":17.83,"body":"hype, the summit of Bullshit Mountain,"},{"speaker":"Emily M. Bender","startTime":18.039,"endTime":19.44,"body":"we discover there's worse to come."},{"speaker":"Emily M. Bender","startTime":19.76,"endTime":20.43,"body":"I'm Emily M."},{"speaker":"Emily M. Bender","startTime":20.45,"endTime":22.88,"body":"Bender, Professor of Linguistics"},{"speaker":"Emily M. Bender","startTime":20.45,"endTime":22.88,"body":"at the University of Washington."},{"speaker":"Alex Hanna","startTime":23.41,"endTime":26.42,"body":"And I'm Alex Hanna,"},{"speaker":"Alex Hanna","startTime":23.41,"endTime":26.42,"body":"Director of Research for the"},{"speaker":"Alex Hanna","startTime":26.42,"endTime":28.3,"body":"Distributed AI Research Institute."},{"speaker":"Alex Hanna","startTime":28.79,"endTime":34.23,"body":"This is episode 48, which we're"},{"speaker":"Alex Hanna","startTime":28.79,"endTime":34.23,"body":"recording on January 13th of 2025."},{"speaker":"Alex Hanna","startTime":34.74,"endTime":38.19,"body":"And we're starting the new year"},{"speaker":"Alex Hanna","startTime":34.74,"endTime":38.19,"body":"right, with a focus on OpenAI."},{"speaker":"Alex Hanna","startTime":38.775,"endTime":44.325,"body":"Not only is their new o3 model allegedly"},{"speaker":"Alex Hanna","startTime":38.775,"endTime":44.325,"body":"breaking records for how close an LLM"},{"speaker":"Alex Hanna","startTime":44.575,"endTime":49.795,"body":"can get to the mythical human like"},{"speaker":"Alex Hanna","startTime":44.575,"endTime":49.795,"body":"thinking of AGI, Sam Altman also has some"},{"speaker":"Alex Hanna","startTime":50.045,"endTime":55.285,"body":"reflections for us, as he marks two years"},{"speaker":"Alex Hanna","startTime":50.045,"endTime":55.285,"body":"since the official launch of ChatGPT."},{"speaker":"Emily M. Bender","startTime":55.915,"endTime":58.835,"body":"Yes, in a new blog"},{"speaker":"Emily M. Bender","startTime":55.915,"endTime":58.835,"body":"post, Altman opines about his abrupt"},{"speaker":"Emily M. Bender","startTime":58.835,"endTime":62.775,"body":"firing and return to the company last"},{"speaker":"Emily M. Bender","startTime":58.835,"endTime":62.775,"body":"year, and predicts a, quote, \"glorious"},{"speaker":"Emily M. Bender","startTime":62.775,"endTime":67.225,"body":"future\" of AI agents, quote, \"joining"},{"speaker":"Emily M. Bender","startTime":62.775,"endTime":67.225,"body":"the workforce,\" as soon as this year."},{"speaker":"Emily M. Bender","startTime":67.555,"endTime":71.615,"body":"He's sure, so sure, that the company"},{"speaker":"Emily M. Bender","startTime":67.555,"endTime":71.615,"body":"knows how to build AGI, and spins a"},{"speaker":"Emily M. Bender","startTime":71.615,"endTime":75.915,"body":"yarn about future in, quote, \"super"},{"speaker":"Emily M. Bender","startTime":71.615,"endTime":75.915,"body":"intelligent tools\" that could accelerate"},{"speaker":"Emily M. Bender","startTime":75.915,"endTime":79.665,"body":"scientific discovery, innovation, and"},{"speaker":"Emily M. Bender","startTime":75.915,"endTime":79.665,"body":"at last, abundance and prosperity."},{"speaker":"Emily M. Bender","startTime":79.675,"endTime":81.235,"body":"For whom, though, he doesn't specify."},{"speaker":"Emily M. Bender","startTime":81.6,"endTime":85.83,"body":"So, let's get into it and start"},{"speaker":"Emily M. Bender","startTime":81.6,"endTime":85.83,"body":"unraveling this fantasy story, shall we?"},{"speaker":"Alex Hanna","startTime":86.56,"endTime":87.67,"body":"Let's shall."},{"speaker":"Alex Hanna","startTime":87.7,"endTime":88.69,"body":"We shall."},{"speaker":"Alex Hanna","startTime":89.13,"endTime":90.01,"body":"What is the phrase?"},{"speaker":"Alex Hanna","startTime":90.01,"endTime":91.2,"body":"Anyways, let's do it."},{"speaker":"Emily M. Bender","startTime":91.439,"endTime":92.37,"body":"Yes, let's do it."},{"speaker":"Emily M. Bender","startTime":92.62,"endTime":94.939,"body":"All right let me just make sure that"},{"speaker":"Emily M. Bender","startTime":92.62,"endTime":94.939,"body":"I've got my things queued up the right"},{"speaker":"Emily M. Bender","startTime":94.939,"endTime":98.999,"body":"way and I'm going to take us first"},{"speaker":"Emily M. Bender","startTime":94.939,"endTime":98.999,"body":"to the ARC Prize presentation here."},{"speaker":"Emily M. Bender","startTime":101.1,"endTime":106.97,"body":"Where this is by Francois Chollet"},{"speaker":"Emily M. Bender","startTime":101.1,"endTime":106.97,"body":"from December 20th of 2024."},{"speaker":"Emily M. Bender","startTime":107.29,"endTime":109.66,"body":"And it is published on ARCPrize.Org."},{"speaker":"Emily M. Bender","startTime":109.68,"endTime":115.64,"body":"ARC is A R C and the, um, headline"},{"speaker":"Emily M. Bender","startTime":109.68,"endTime":115.64,"body":"is \"OpenAI o3 Breakthrough"},{"speaker":"Emily M. Bender","startTime":115.649,"endTime":118.41,"body":"High Score on the ARC-AGI-PUB."},{"speaker":"Alex Hanna","startTime":119.29,"endTime":119.79,"body":"Yes."},{"speaker":"Alex Hanna","startTime":119.83,"endTime":124.145,"body":"So just a little bit of background"},{"speaker":"Alex Hanna","startTime":119.83,"endTime":124.145,"body":"of like what the ARC prize is."},{"speaker":"Alex Hanna","startTime":124.175,"endTime":127.815,"body":"Just on their, from their main page,"},{"speaker":"Alex Hanna","startTime":124.175,"endTime":127.815,"body":"\"The ARC prize--\" And I'm pretty sure"},{"speaker":"Alex Hanna","startTime":127.835,"endTime":134.085,"body":"ARC stands for Alignment Research Center,"},{"speaker":"Alex Hanna","startTime":127.835,"endTime":134.085,"body":"um, \"The ARC prize is a $1 million"},{"speaker":"Alex Hanna","startTime":134.545,"endTime":137.135,"body":"plus--\" I don't know why it's plus, um."},{"speaker":"Alex Hanna","startTime":137.975,"endTime":140.95,"body":"\"--public competition to beat"},{"speaker":"Alex Hanna","startTime":137.975,"endTime":140.95,"body":"and open source a solution to"},{"speaker":"Alex Hanna","startTime":140.95,"endTime":143.2,"body":"the ARC hyphen AGI benchmark."},{"speaker":"Alex Hanna","startTime":143.58,"endTime":149.159,"body":"Hosted by Mike Knoop, co founder"},{"speaker":"Alex Hanna","startTime":143.58,"endTime":149.159,"body":"of Zapier and Francois Chollet,"},{"speaker":"Alex Hanna","startTime":149.179,"endTime":151.39,"body":"creator of the ARC AGI and Keras.\""},{"speaker":"Alex Hanna","startTime":151.7,"endTime":157.895,"body":"Um, and they're the, uh, folks"},{"speaker":"Alex Hanna","startTime":151.7,"endTime":157.895,"body":"at the helm of this situation."},{"speaker":"Emily M. Bender","startTime":157.995,"endTime":158.205,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":158.205,"endTime":159.065,"body":"So it's not alignment."},{"speaker":"Emily M. Bender","startTime":159.065,"endTime":162.175,"body":"Cause we see here over in this"},{"speaker":"Emily M. Bender","startTime":159.065,"endTime":162.175,"body":"next little bit, ARC-AGI, they say"},{"speaker":"Emily M. Bender","startTime":162.185,"endTime":165.745,"body":"\"Most AI benchmarks measure skill,"},{"speaker":"Emily M. Bender","startTime":162.185,"endTime":165.745,"body":"but skill is not intelligence."},{"speaker":"Emily M. Bender","startTime":166.125,"endTime":169.365,"body":"General intelligence is the ability"},{"speaker":"Emily M. Bender","startTime":166.125,"endTime":169.365,"body":"to efficiently acquire new skills."},{"speaker":"Emily M. Bender","startTime":169.725,"endTime":174.265,"body":"Chollet's unbeaten 2019 Abstraction and"},{"speaker":"Emily M. Bender","startTime":169.725,"endTime":174.265,"body":"Reasoning Corpus for Artificial General"},{"speaker":"Emily M. Bender","startTime":174.265,"endTime":175.214,"body":"Intelligence--\" So that's what it is."},{"speaker":"Emily M. Bender","startTime":175.215,"endTime":176.475,"body":"Abstraction and Reasoning Corpus."},{"speaker":"Alex Hanna","startTime":176.475,"endTime":176.845,"body":"Oh, got it."},{"speaker":"Alex Hanna","startTime":176.925,"endTime":177.175,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":177.82,"endTime":179.6,"body":"They're overloading ARC, yeah, sorry."},{"speaker":"Emily M. Bender","startTime":180.05,"endTime":183.18,"body":"\"--is the only"},{"speaker":"Emily M. Bender","startTime":180.05,"endTime":183.18,"body":"formal benchmark for, of AGI progress."},{"speaker":"Emily M. Bender","startTime":183.44,"endTime":185.72,"body":"It's easy for humans, but hard for AI.\""},{"speaker":"Emily M. Bender","startTime":185.72,"endTime":186.97,"body":"That's their tagline."},{"speaker":"Emily M. Bender","startTime":187.409,"endTime":188.829,"body":"And then they have some examples here."},{"speaker":"Emily M. Bender","startTime":188.829,"endTime":190.929,"body":"So it's these visual puzzles."},{"speaker":"Emily M. Bender","startTime":192.12,"endTime":196.11,"body":"And, you know, they've got these,"},{"speaker":"Emily M. Bender","startTime":192.12,"endTime":196.11,"body":"uh, input output pairs and you get"},{"speaker":"Emily M. Bender","startTime":196.11,"endTime":199.19,"body":"a few examples to try out or just"},{"speaker":"Emily M. Bender","startTime":196.11,"endTime":199.19,"body":"to sort of figure out the pattern"},{"speaker":"Emily M. Bender","startTime":199.2,"endTime":203.65,"body":"is and then the test is taking"},{"speaker":"Emily M. Bender","startTime":199.2,"endTime":203.65,"body":"new input and give the output."},{"speaker":"Emily M. Bender","startTime":204.11,"endTime":207.999,"body":"And I guess the thought here is that"},{"speaker":"Emily M. Bender","startTime":204.11,"endTime":207.999,"body":"each one of these represents a different"},{"speaker":"Emily M. Bender","startTime":207.999,"endTime":212.55,"body":"skill that the system is supposedly"},{"speaker":"Emily M. Bender","startTime":207.999,"endTime":212.55,"body":"acquiring by studying the examples, but"},{"speaker":"Emily M. Bender","startTime":212.55,"endTime":218.25,"body":"it is, all of the examples I've seen are"},{"speaker":"Emily M. Bender","startTime":212.55,"endTime":218.25,"body":"sort of this general genre of, of task."},{"speaker":"Alex Hanna","startTime":218.71,"endTime":219.09,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":219.09,"endTime":222.74,"body":"So it's, it's getting at the sense"},{"speaker":"Alex Hanna","startTime":219.09,"endTime":222.74,"body":"of generability and, uh, we can try"},{"speaker":"Alex Hanna","startTime":222.74,"endTime":224.429,"body":"to describe what this looks like."},{"speaker":"Alex Hanna","startTime":224.869,"endTime":229.649,"body":"So this, there's this one puzzle"},{"speaker":"Alex Hanna","startTime":224.869,"endTime":229.649,"body":"that, um, shows up on the first"},{"speaker":"Alex Hanna","startTime":229.65,"endTime":232.29,"body":"one and they're like these grids."},{"speaker":"Alex Hanna","startTime":232.62,"endTime":232.92,"body":"And--"},{"speaker":"Emily M. Bender","startTime":232.92,"endTime":234.2,"body":"Oh, you want"},{"speaker":"Emily M. Bender","startTime":232.92,"endTime":234.2,"body":"the ones down at the bottom here?"},{"speaker":"Alex Hanna","startTime":234.54,"endTime":236.32,"body":"Uh, yeah, no, that"},{"speaker":"Alex Hanna","startTime":234.54,"endTime":236.32,"body":"that's the one I'm looking at."},{"speaker":"Alex Hanna","startTime":236.32,"endTime":236.65,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":236.69,"endTime":236.86,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":237.195,"endTime":237.895,"body":"Oh, that one, that one."},{"speaker":"Alex Hanna","startTime":237.895,"endTime":238.035,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":238.035,"endTime":238.235,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":238.235,"endTime":240.085,"body":"The, the one that was looking,"},{"speaker":"Alex Hanna","startTime":238.235,"endTime":240.085,"body":"no, no, the one that we were"},{"speaker":"Alex Hanna","startTime":240.085,"endTime":241.475,"body":"looking at on the image page."},{"speaker":"Alex Hanna","startTime":241.475,"endTime":241.745,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":242.055,"endTime":247.655,"body":"So it, so there's, there's this kind of"},{"speaker":"Alex Hanna","startTime":242.055,"endTime":247.655,"body":"like a partial Tetris piece and there's"},{"speaker":"Alex Hanna","startTime":247.655,"endTime":253.335,"body":"like, um, that are at a right angle and"},{"speaker":"Alex Hanna","startTime":247.655,"endTime":253.335,"body":"effectively you're trying to, the output"},{"speaker":"Alex Hanna","startTime":253.355,"endTime":256.925,"body":"is like dropping the missing piece here."},{"speaker":"Alex Hanna","startTime":257.205,"endTime":259.615,"body":"Uh, and they're all in"},{"speaker":"Alex Hanna","startTime":257.205,"endTime":259.615,"body":"different configurations."},{"speaker":"Alex Hanna","startTime":259.735,"endTime":264.125,"body":"So they make that two by two square,"},{"speaker":"Alex Hanna","startTime":259.735,"endTime":264.125,"body":"uh, and then they have kinds of"},{"speaker":"Alex Hanna","startTime":264.125,"endTime":265.675,"body":"different kinds of grid things."},{"speaker":"Alex Hanna","startTime":266.045,"endTime":270.394,"body":"And the graph, uh, a little lower is"},{"speaker":"Alex Hanna","startTime":266.045,"endTime":270.394,"body":"kind of interesting because it's, you"},{"speaker":"Alex Hanna","startTime":270.394,"endTime":275.664,"body":"know, it's, it's this fallacy of, of"},{"speaker":"Alex Hanna","startTime":270.394,"endTime":275.664,"body":"basically saying what is human capability."},{"speaker":"Alex Hanna","startTime":275.924,"endTime":278.444,"body":"So they've got, uh, common"},{"speaker":"Alex Hanna","startTime":275.924,"endTime":278.444,"body":"benchmarks we've talked about"},{"speaker":"Alex Hanna","startTime":278.444,"endTime":283.275,"body":"before, including a SQuAD and GLUE"},{"speaker":"Alex Hanna","startTime":278.444,"endTime":283.275,"body":"and SuperGLUE, um, and ImageNet."},{"speaker":"Alex Hanna","startTime":283.605,"endTime":288.96,"body":"Uh, and then there's this line that"},{"speaker":"Alex Hanna","startTime":283.605,"endTime":288.96,"body":"is \"human capability\" in quotes."},{"speaker":"Alex Hanna","startTime":289.34,"endTime":296.299,"body":"And then the thing that's really silly"},{"speaker":"Alex Hanna","startTime":289.34,"endTime":296.299,"body":"here is that there's a ARC-AGI, uh,"},{"speaker":"Alex Hanna","startTime":296.309,"endTime":302.389,"body":"benchmark, and then there's a, like,"},{"speaker":"Alex Hanna","startTime":296.309,"endTime":302.389,"body":"kind of an, a, uh, something that kind"},{"speaker":"Alex Hanna","startTime":302.389,"endTime":305.874,"body":"of approximate, like, approximates"},{"speaker":"Alex Hanna","startTime":302.389,"endTime":305.874,"body":"a logarithmic curve that would say"},{"speaker":"Alex Hanna","startTime":305.874,"endTime":310.864,"body":"like, before we had this prize, it"},{"speaker":"Alex Hanna","startTime":305.874,"endTime":310.864,"body":"would be negative 0.5 percent or"},{"speaker":"Alex Hanna","startTime":310.914,"endTime":314.304,"body":"negative 0.5, but then we launched"},{"speaker":"Alex Hanna","startTime":310.914,"endTime":314.304,"body":"the prize and like, look how much,"},{"speaker":"Alex Hanna","startTime":314.324,"endTime":315.904,"body":"and it's just like, goes straight up."},{"speaker":"Alex Hanna","startTime":316.834,"endTime":319.584,"body":"So exponential growth, because"},{"speaker":"Alex Hanna","startTime":316.834,"endTime":319.584,"body":"we had a benchmark on it."},{"speaker":"Emily M. Bender","startTime":319.585,"endTime":323.004,"body":"I mean, talk about"},{"speaker":"Emily M. Bender","startTime":319.585,"endTime":323.004,"body":"when a metric becomes a metric"},{"speaker":"Emily M. Bender","startTime":323.004,"endTime":324.594,"body":"it ceases to be a metric, right?"},{"speaker":"Emily M. Bender","startTime":324.754,"endTime":325.964,"body":"Like there's a better way of saying that."},{"speaker":"Alex Hanna","startTime":326.114,"endTime":326.594,"body":"Yeah, yeah."},{"speaker":"Emily M. Bender","startTime":326.754,"endTime":330.764,"body":"But yeah, this"},{"speaker":"Emily M. Bender","startTime":326.754,"endTime":330.764,"body":"is, this is so frustrating."},{"speaker":"Emily M. Bender","startTime":330.764,"endTime":332.974,"body":"So they, yeah, they had this"},{"speaker":"Emily M. Bender","startTime":330.764,"endTime":332.974,"body":"dash line, which is the \"status"},{"speaker":"Emily M. Bender","startTime":332.974,"endTime":334.754,"body":"quo forecast pre ARC Prize\"."},{"speaker":"Emily M. Bender","startTime":334.764,"endTime":337.484,"body":"So, you know, completely"},{"speaker":"Emily M. Bender","startTime":334.764,"endTime":337.484,"body":"fake line on the graph."},{"speaker":"Emily M. Bender","startTime":338.294,"endTime":340.654,"body":"Um, and then we put the prize"},{"speaker":"Emily M. Bender","startTime":338.294,"endTime":340.654,"body":"out and instead of following that"},{"speaker":"Emily M. Bender","startTime":340.654,"endTime":343.314,"body":"fake line, look, the systems were"},{"speaker":"Emily M. Bender","startTime":340.654,"endTime":343.314,"body":"being trained to do the thing."},{"speaker":"Emily M. Bender","startTime":343.324,"endTime":344.374,"body":"And the score went up."},{"speaker":"Alex Hanna","startTime":345.269,"endTime":345.539,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":345.759,"endTime":347.349,"body":"No, we made, we made the score get up."},{"speaker":"Alex Hanna","startTime":347.349,"endTime":349.339,"body":"We did, we did Godwin's law."},{"speaker":"Alex Hanna","startTime":349.359,"endTime":349.869,"body":"Check it out."},{"speaker":"Emily M. Bender","startTime":350.449,"endTime":350.769,"body":"Thank you."},{"speaker":"Emily M. Bender","startTime":350.769,"endTime":351.009,"body":"Yes."},{"speaker":"Emily M. Bender","startTime":351.009,"endTime":352.039,"body":"That's what I'm looking for."},{"speaker":"Emily M. Bender","startTime":352.039,"endTime":356.819,"body":"So they also have a couple of"},{"speaker":"Emily M. Bender","startTime":352.039,"endTime":356.819,"body":"paragraphs here on AGI and defining AGI."},{"speaker":"Emily M. Bender","startTime":356.819,"endTime":361.559,"body":"So AGI says, \"LLMs are trained on"},{"speaker":"Emily M. Bender","startTime":356.819,"endTime":361.559,"body":"unimaginably vast amounts of data yet"},{"speaker":"Emily M. Bender","startTime":361.559,"endTime":365.489,"body":"remain unable to adapt to simple problems"},{"speaker":"Emily M. Bender","startTime":361.559,"endTime":365.489,"body":"they haven't been trained on or make"},{"speaker":"Emily M. Bender","startTime":365.489,"endTime":367.439,"body":"novel inventions, no matter how basic."},{"speaker":"Emily M. Bender","startTime":367.719,"endTime":371.599,"body":"Strong market incentives have pushed"},{"speaker":"Emily M. Bender","startTime":367.719,"endTime":371.599,"body":"frontier AI research to go closed source."},{"speaker":"Emily M. Bender","startTime":372.224,"endTime":375.104,"body":"Research attention and resources"},{"speaker":"Emily M. Bender","startTime":372.224,"endTime":375.104,"body":"are being pulled towards a dead end."},{"speaker":"Emily M. Bender","startTime":375.574,"endTime":378.384,"body":"ARC Prize is designed to inspire"},{"speaker":"Emily M. Bender","startTime":375.574,"endTime":378.384,"body":"researchers to discover new"},{"speaker":"Emily M. Bender","startTime":378.424,"endTime":381.714,"body":"technical approaches that push"},{"speaker":"Emily M. Bender","startTime":378.424,"endTime":381.714,"body":"open AGI progress forward.\""},{"speaker":"Alex Hanna","startTime":382.914,"endTime":386.474,"body":"Yeah, it's, it's, it's"},{"speaker":"Alex Hanna","startTime":382.914,"endTime":386.474,"body":"a, it's a bad, it's some bad vibes."},{"speaker":"Alex Hanna","startTime":386.504,"endTime":391.134,"body":"And this, their definition"},{"speaker":"Alex Hanna","startTime":386.504,"endTime":391.134,"body":"of AGI is, uh, great."},{"speaker":"Alex Hanna","startTime":391.374,"endTime":395.684,"body":"I mean that in a parentheses derogatory."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":396.044,"endTime":400.474,"body":"AGI is a system that can automate the"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":400.474,"endTime":403.054,"body":"majority of economically valuable work.\""},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":403.414,"endTime":407.844,"body":"Which is, uh, I'm pretty sure, pretty"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":403.414,"endTime":407.844,"body":"close to what's in the OpenAI charter."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":408.174,"endTime":413.384,"body":"It's also pretty close to what I"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":408.174,"endTime":413.384,"body":"think Mustafa Solomon says, um, an,"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":413.384,"endTime":417.944,"body":"an, uh, former, um exec at DeepMind."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":418.364,"endTime":421.324,"body":"Now they define \"correct,\""},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":418.364,"endTime":421.324,"body":"which is very funny."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":421.574,"endTime":426.674,"body":"No citations or, you know, like any kind"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":421.574,"endTime":426.674,"body":"of backing of this, but, \"Correct: AGI is"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":426.674,"endTime":431.674,"body":"a system that can efficiently acquire new"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":426.674,"endTime":431.674,"body":"skills and solve open ended problems.\""},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":432.724,"endTime":433.244,"body":"Okay."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":433.334,"endTime":437.284,"body":"Like, why is this now, why"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":433.334,"endTime":437.284,"body":"is this now the definition?"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":437.554,"endTime":442.179,"body":"And secondly, why does that"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":437.554,"endTime":442.179,"body":"you know, why have we moved?"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":442.669,"endTime":443.509,"body":"It's so interesting."},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":443.509,"endTime":448.719,"body":"Like why has this moved from like a"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":443.509,"endTime":448.719,"body":"political economy, um, angle, which to,"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":448.759,"endTime":455.699,"body":"to, to be fair, should be ridiculed, but,"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":448.759,"endTime":455.699,"body":"uh, also has sort of like some kind of"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":455.739,"endTime":460.539,"body":"externality to something that can quote,"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":455.739,"endTime":460.539,"body":"\"acquire new skills,\" not defining what"},{"speaker":"Uh, so the \"Consensus, but wrong","startTime":460.539,"endTime":462.839,"body":"a skill is or an open ended problem."},{"speaker":"Emily M. Bender","startTime":464.059,"endTime":466.179,"body":"But they say,"},{"speaker":"Emily M. Bender","startTime":464.059,"endTime":466.179,"body":"Alex, definitions are important."},{"speaker":"Alex Hanna","startTime":466.499,"endTime":466.859,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":466.859,"endTime":469.549,"body":"They say definitions"},{"speaker":"Alex Hanna","startTime":466.859,"endTime":469.549,"body":"are important and yet."},{"speaker":"Emily M. Bender","startTime":470.019,"endTime":470.259,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":470.419,"endTime":470.629,"body":"So--"},{"speaker":"Alex Hanna","startTime":470.629,"endTime":471.749,"body":"I have questions."},{"speaker":"Alex Hanna","startTime":471.824,"endTime":471.894,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":472.334,"endTime":473.884,"body":"So the next bit"},{"speaker":"Emily M. Bender","startTime":472.334,"endTime":473.884,"body":"is, \"Definitions are important."},{"speaker":"Emily M. Bender","startTime":473.894,"endTime":477.194,"body":"We turn them into benchmarks to"},{"speaker":"Emily M. Bender","startTime":473.894,"endTime":477.194,"body":"measure progress towards AGI.\""},{"speaker":"Emily M. Bender","startTime":477.474,"endTime":478.914,"body":"Which comes right back to this thing."},{"speaker":"Emily M. Bender","startTime":478.914,"endTime":480.764,"body":"And so, um, SJayLett is helping us out."},{"speaker":"Emily M. Bender","startTime":480.964,"endTime":484.404,"body":"\"It's when a metric becomes a target,"},{"speaker":"Emily M. Bender","startTime":480.964,"endTime":484.404,"body":"it ceases to be an effective metric.\""},{"speaker":"Emily M. Bender","startTime":484.944,"endTime":487.664,"body":"And Kh0rish in the chat also"},{"speaker":"Emily M. Bender","startTime":484.944,"endTime":487.664,"body":"saying, \"Benchmarks: definitely"},{"speaker":"Emily M. Bender","startTime":487.665,"endTime":490.174,"body":"for sure, always perfectly"},{"speaker":"Emily M. Bender","startTime":487.665,"endTime":490.174,"body":"measuring what they say they are."},{"speaker":"Emily M. Bender","startTime":490.345,"endTime":494.174,"body":"\""},{"speaker":"Emily M. Bender","startTime":490.345,"endTime":494.174,"body":"And then the last thing like gives"},{"speaker":"Emily M. Bender","startTime":494.174,"endTime":494.304,"body":"Right."},{"speaker":"Emily M. Bender","startTime":494.304,"endTime":498.234,"body":"\"Without AGI, we will never have"},{"speaker":"Emily M. Bender","startTime":494.304,"endTime":498.234,"body":"symptoms, sorry, systems that can"},{"speaker":"Emily M. Bender","startTime":498.234,"endTime":500.614,"body":"invent and discover alongside humans.\""},{"speaker":"Emily M. Bender","startTime":501.164,"endTime":504.454,"body":"And it's like, maybe we"},{"speaker":"Emily M. Bender","startTime":501.164,"endTime":504.454,"body":"don't actually need that."},{"speaker":"Alex Hanna","startTime":504.744,"endTime":505.214,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":505.964,"endTime":508.104,"body":"At the end of the page, it's also"},{"speaker":"Alex Hanna","startTime":505.964,"endTime":508.104,"body":"interesting because it looks like"},{"speaker":"Alex Hanna","startTime":508.224,"endTime":511.904,"body":"Knoop and Chollet, like are, it says"},{"speaker":"Alex Hanna","startTime":508.224,"endTime":511.904,"body":"that they're actually sponsoring"},{"speaker":"Alex Hanna","startTime":511.904,"endTime":513.914,"body":"this 'cause I'm like, well, where"},{"speaker":"Alex Hanna","startTime":511.904,"endTime":513.914,"body":"are they getting this money?"},{"speaker":"Emily M. Bender","startTime":514.034,"endTime":514.184,"body":"Mm-hmm"},{"speaker":"Emily M. Bender","startTime":514.424,"endTime":517.734,"body":"."},{"speaker":"Emily M. Bender","startTime":514.424,"endTime":517.734,"body":"Alex Hanna: And are are Knoop"},{"speaker":"Emily M. Bender","startTime":517.754,"endTime":518.234,"body":"I don't know."},{"speaker":"Emily M. Bender","startTime":518.544,"endTime":521.354,"body":"Knoop and Chollet are, are they"},{"speaker":"Emily M. Bender","startTime":518.544,"endTime":521.354,"body":"putting up their own money?"},{"speaker":"Emily M. Bender","startTime":521.354,"endTime":525.134,"body":"Their own one, one mi--they just think"},{"speaker":"Emily M. Bender","startTime":521.354,"endTime":525.134,"body":"it's important enough that they--"},{"speaker":"Emily M. Bender","startTime":525.554,"endTime":526.634,"body":"Uh, they're"},{"speaker":"Emily M. Bender","startTime":525.554,"endTime":526.634,"body":"making it look like that."},{"speaker":"Alex Hanna","startTime":526.854,"endTime":527.164,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":527.344,"endTime":528.744,"body":"Well, good for them."},{"speaker":"Emily M. Bender","startTime":528.994,"endTime":531.124,"body":"Okay, so, so this"},{"speaker":"Emily M. Bender","startTime":528.994,"endTime":531.124,"body":"is the background on the prize."},{"speaker":"Emily M. Bender","startTime":531.124,"endTime":534.624,"body":"Let's go back to their"},{"speaker":"Emily M. Bender","startTime":531.124,"endTime":534.624,"body":"thing about OpenAI's o3."},{"speaker":"Emily M. Bender","startTime":534.839,"endTime":539.579,"body":"So, uh, \"OpenAI's new o3 system"},{"speaker":"Emily M. Bender","startTime":534.839,"endTime":539.579,"body":"trained on the ARC AGI 1 public"},{"speaker":"Emily M. Bender","startTime":539.589,"endTime":544.289,"body":"training set--\" So this is not like"},{"speaker":"Emily M. Bender","startTime":539.589,"endTime":544.289,"body":"completely held out kinds of tasks."},{"speaker":"Emily M. Bender","startTime":544.289,"endTime":546.619,"body":"We're going to show"},{"speaker":"Emily M. Bender","startTime":544.289,"endTime":546.619,"body":"them this thing, right?"},{"speaker":"Emily M. Bender","startTime":546.939,"endTime":550.769,"body":"\"--has scored a breakthrough"},{"speaker":"Emily M. Bender","startTime":546.939,"endTime":550.769,"body":"75.7 percent on the semi private"},{"speaker":"Emily M. Bender","startTime":550.779,"endTime":555.679,"body":"evaluation set at our stated public"},{"speaker":"Emily M. Bender","startTime":550.779,"endTime":555.679,"body":"leaderboard, 10,000K compute limit."},{"speaker":"Emily M. Bender","startTime":556.229,"endTime":561.979,"body":"A high-compute, 172X o3"},{"speaker":"Emily M. Bender","startTime":556.229,"endTime":561.979,"body":"configuration scored 87.5 percent.\""},{"speaker":"Emily M. Bender","startTime":563.434,"endTime":566.744,"body":"So what they've done is they have"},{"speaker":"Emily M. Bender","startTime":563.434,"endTime":566.744,"body":"taken the public training set, which"},{"speaker":"Emily M. Bender","startTime":566.744,"endTime":568.114,"body":"is a whole bunch of these things."},{"speaker":"Emily M. Bender","startTime":568.534,"endTime":571.024,"body":"And so the idea with generalization"},{"speaker":"Emily M. Bender","startTime":568.534,"endTime":571.024,"body":"here is that each one of the"},{"speaker":"Emily M. Bender","startTime":571.024,"endTime":574.164,"body":"little picture puzzles has a"},{"speaker":"Emily M. Bender","startTime":571.024,"endTime":574.164,"body":"different generalization in it."},{"speaker":"Emily M. Bender","startTime":574.414,"endTime":578.344,"body":"But the general shape of it is the same"},{"speaker":"Emily M. Bender","startTime":574.414,"endTime":578.344,"body":"across all of them, as far as I can tell."},{"speaker":"Emily M. Bender","startTime":578.704,"endTime":584.804,"body":"And so OpenAI had access to, um, I think"},{"speaker":"Emily M. Bender","startTime":578.704,"endTime":584.804,"body":"about 300 of those and train that and also"},{"speaker":"Emily M. Bender","startTime":584.804,"endTime":589.194,"body":"probably, who knows, did some architecture"},{"speaker":"Emily M. Bender","startTime":584.804,"endTime":589.194,"body":"changes in their system to like optimize"},{"speaker":"Emily M. Bender","startTime":589.194,"endTime":591.574,"body":"it for this and then handed it over."},{"speaker":"Emily M. Bender","startTime":592.224,"endTime":596.824,"body":"Um, uh, I think we're going to"},{"speaker":"Emily M. Bender","startTime":592.224,"endTime":596.824,"body":"skip this little bit of hype here."},{"speaker":"Emily M. Bender","startTime":597.094,"endTime":602.214,"body":"Um, somewhere it says, yes, \"At OpenAI's"},{"speaker":"Emily M. Bender","startTime":597.094,"endTime":602.214,"body":"direction, we tested at two levels of"},{"speaker":"Emily M. Bender","startTime":602.214,"endTime":603.674,"body":"compute with variable sample sizes.\""},{"speaker":"Emily M. Bender","startTime":603.684,"endTime":608.384,"body":"So ARC now is like collaborating"},{"speaker":"Emily M. Bender","startTime":603.684,"endTime":608.384,"body":"with OpenAI on this."},{"speaker":"Alex Hanna","startTime":609.549,"endTime":611.679,"body":"Yeah, I mean, they're,"},{"speaker":"Alex Hanna","startTime":609.549,"endTime":611.679,"body":"they're, I mean, they're effectively"},{"speaker":"Alex Hanna","startTime":611.679,"endTime":614.859,"body":"saying, like, you need to run it"},{"speaker":"Alex Hanna","startTime":611.679,"endTime":614.859,"body":"like this, which, you know, like,"},{"speaker":"Alex Hanna","startTime":614.909,"endTime":620.139,"body":"to some degree is like, okay, like,"},{"speaker":"Alex Hanna","startTime":614.909,"endTime":620.139,"body":"there's certain configurations"},{"speaker":"Alex Hanna","startTime":620.139,"endTime":623.789,"body":"that you need to get right, and I"},{"speaker":"Alex Hanna","startTime":620.139,"endTime":623.789,"body":"will grant you that, but it is then"},{"speaker":"Alex Hanna","startTime":623.799,"endTime":630.435,"body":"saying, um, is then saying, like, you"},{"speaker":"Alex Hanna","startTime":623.799,"endTime":630.435,"body":"know, this is not a true benchmark."},{"speaker":"Alex Hanna","startTime":630.435,"endTime":632.734,"body":"This is not kind of like"},{"speaker":"Alex Hanna","startTime":630.435,"endTime":632.734,"body":"a withheld sort of thing."},{"speaker":"Alex Hanna","startTime":632.984,"endTime":637.654,"body":"There is like some collusion in"},{"speaker":"Alex Hanna","startTime":632.984,"endTime":637.654,"body":"this that makes this like not a"},{"speaker":"Alex Hanna","startTime":637.654,"endTime":639.574,"body":"very useful research instrument."},{"speaker":"Alex Hanna","startTime":640.044,"endTime":645.985,"body":"Um, and there's a lot of things that"},{"speaker":"Alex Hanna","startTime":640.044,"endTime":645.985,"body":"are not reported in this, which is, um,"},{"speaker":"Alex Hanna","startTime":645.994,"endTime":650.709,"body":"which is a hallmark of hype, you know,"},{"speaker":"Alex Hanna","startTime":645.994,"endTime":650.709,"body":"um, effectively saying that this is a"},{"speaker":"Alex Hanna","startTime":650.719,"endTime":658.16,"body":"semi private sort, um, this is kind of a,"},{"speaker":"Alex Hanna","startTime":650.719,"endTime":658.16,"body":"this is a semi private eval in addition"},{"speaker":"Alex Hanna","startTime":658.16,"endTime":661.66,"body":"to sort of saying, and they kind of"},{"speaker":"Alex Hanna","startTime":658.16,"endTime":661.66,"body":"give something at the end where they're"},{"speaker":"Alex Hanna","startTime":661.66,"endTime":665.799,"body":"saying, we're going to open source this"},{"speaker":"Alex Hanna","startTime":661.66,"endTime":665.799,"body":"kind of at the end, but the kind of thing"},{"speaker":"Alex Hanna","startTime":665.88,"endTime":670.45,"body":"that's being said here is like, well,"},{"speaker":"Alex Hanna","startTime":665.88,"endTime":670.45,"body":"OpenAI, we're working with them closely."},{"speaker":"Alex Hanna","startTime":670.8,"endTime":674.3,"body":"And not disclosing all of this"},{"speaker":"Alex Hanna","startTime":670.8,"endTime":674.3,"body":"because there's strong incentives"},{"speaker":"Alex Hanna","startTime":674.3,"endTime":675.89,"body":"to be like a first mover here."},{"speaker":"Alex Hanna","startTime":676.31,"endTime":680.2,"body":"Um, so we're going to hype them up a bit,"},{"speaker":"Alex Hanna","startTime":676.31,"endTime":680.2,"body":"uh, and then we'll just open source it"},{"speaker":"Alex Hanna","startTime":680.339,"endTime":684.31,"body":"eventually, which is, um, pretty annoying."},{"speaker":"Emily M. Bender","startTime":684.839,"endTime":685.009,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":686.05,"endTime":688.48,"body":"So they, and they're also sort"},{"speaker":"Emily M. Bender","startTime":686.05,"endTime":688.48,"body":"of like hemming and hawing"},{"speaker":"Emily M. Bender","startTime":688.489,"endTime":690.28,"body":"about the cost per task."},{"speaker":"Emily M. Bender","startTime":690.52,"endTime":694.02,"body":"Um, and they say, \"Despite the"},{"speaker":"Emily M. Bender","startTime":690.52,"endTime":694.02,"body":"significant cost per task, these numbers"},{"speaker":"Emily M. Bender","startTime":694.03,"endTime":697.35,"body":"aren't just the result of applying"},{"speaker":"Emily M. Bender","startTime":694.03,"endTime":697.35,"body":"brute force compute to the benchmark."},{"speaker":"Emily M. Bender","startTime":697.46,"endTime":701.22,"body":"OpenAI's new o3 model represents"},{"speaker":"Emily M. Bender","startTime":697.46,"endTime":701.22,"body":"a significant leap forward in AI's"},{"speaker":"Emily M. Bender","startTime":701.22,"endTime":703.09,"body":"ability to adapt to novel tasks.\""},{"speaker":"Emily M. Bender","startTime":704.46,"endTime":709.76,"body":"This is so frustrating because"},{"speaker":"Emily M. Bender","startTime":704.46,"endTime":709.76,"body":"they are talking about AI as"},{"speaker":"Emily M. Bender","startTime":709.76,"endTime":711.51,"body":"if it were one thing, right?"},{"speaker":"Emily M. Bender","startTime":711.86,"endTime":715.295,"body":"OpenAI tuned their o3 model"},{"speaker":"Emily M. Bender","startTime":711.86,"endTime":715.295,"body":"to be able to do this task."},{"speaker":"Emily M. Bender","startTime":715.535,"endTime":721.155,"body":"And now, um, Chollet and Knoop, Knoop,"},{"speaker":"Emily M. Bender","startTime":715.535,"endTime":721.155,"body":"whatever his name is, are saying, look,"},{"speaker":"Emily M. Bender","startTime":721.205,"endTime":724.425,"body":"AI as a whole has made a leap forward."},{"speaker":"Emily M. Bender","startTime":724.434,"endTime":725.705,"body":"There's a new ability here."},{"speaker":"Alex Hanna","startTime":727.175,"endTime":731.535,"body":"Well, it's interesting too,"},{"speaker":"Alex Hanna","startTime":727.175,"endTime":731.535,"body":"because when you kind of go down on"},{"speaker":"Alex Hanna","startTime":731.535,"endTime":734.805,"body":"this and there's, they talk about cost"},{"speaker":"Alex Hanna","startTime":731.535,"endTime":734.805,"body":"and performance and tasks, and we'll"},{"speaker":"Alex Hanna","startTime":734.805,"endTime":740.374,"body":"get to the, the kind of estimation of"},{"speaker":"Alex Hanna","startTime":734.805,"endTime":740.374,"body":"sustainability and how much carbon this"},{"speaker":"Alex Hanna","startTime":740.374,"endTime":744.505,"body":"takes, which I mean, when you look at"},{"speaker":"Alex Hanna","startTime":740.374,"endTime":744.505,"body":"compute costs, you also need to always"},{"speaker":"Alex Hanna","startTime":744.505,"endTime":746.285,"body":"look at energy and sustainability."},{"speaker":"Alex Hanna","startTime":746.734,"endTime":751.555,"body":"Um, but the, the discussion of"},{"speaker":"Alex Hanna","startTime":746.734,"endTime":751.555,"body":"the architecture here, so there's"},{"speaker":"Alex Hanna","startTime":751.555,"endTime":754.965,"body":"a heading that's what's different"},{"speaker":"Alex Hanna","startTime":751.555,"endTime":754.965,"body":"about o3 compared to older models."},{"speaker":"Alex Hanna","startTime":755.555,"endTime":756.125,"body":"So."},{"speaker":"Alex Hanna","startTime":756.66,"endTime":761.04,"body":"So there's this thing here, which,"},{"speaker":"Alex Hanna","startTime":756.66,"endTime":761.04,"body":"which Chollet says, \"My mental"},{"speaker":"Alex Hanna","startTime":761.06,"endTime":765.82,"body":"model for LLMs is that they work as"},{"speaker":"Alex Hanna","startTime":761.06,"endTime":765.82,"body":"a repository of vector programs.\""},{"speaker":"Alex Hanna","startTime":766.389,"endTime":768.77,"body":"So first off, that's like,"},{"speaker":"Alex Hanna","startTime":766.389,"endTime":768.77,"body":"let's break that down."},{"speaker":"Alex Hanna","startTime":768.78,"endTime":772.15,"body":"So like a repository of vector programs."},{"speaker":"Alex Hanna","startTime":772.69,"endTime":779.09,"body":"So that's sort of saying, this is such"},{"speaker":"Alex Hanna","startTime":772.69,"endTime":779.09,"body":"a weird epistemology or like, this is"},{"speaker":"Alex Hanna","startTime":779.09,"endTime":784.11,"body":"a weird metaphysics of like what an LLM"},{"speaker":"Alex Hanna","startTime":779.09,"endTime":784.11,"body":"is, because it's sort of saying that"},{"speaker":"Alex Hanna","startTime":784.42,"endTime":794.0,"body":"tokens themselves are programs or like"},{"speaker":"Alex Hanna","startTime":784.42,"endTime":794.0,"body":"in, in vectors of tokens are programs and"},{"speaker":"Alex Hanna","startTime":794.0,"endTime":800.015,"body":"you just need to sort of execute that,"},{"speaker":"Alex Hanna","startTime":794.0,"endTime":800.015,"body":"which is like such a bizarre internal"},{"speaker":"Alex Hanna","startTime":800.095,"endTime":802.655,"body":"mental model of what these things are."},{"speaker":"Emily M. Bender","startTime":802.705,"endTime":803.005,"body":"Right."},{"speaker":"Emily M. Bender","startTime":803.035,"endTime":806.755,"body":"And I forget whose point this was,"},{"speaker":"Emily M. Bender","startTime":803.035,"endTime":806.755,"body":"but someone was pointing out that if"},{"speaker":"Emily M. Bender","startTime":806.755,"endTime":814.0,"body":"you asked ChatGPT to do arithmetic,"},{"speaker":"Emily M. Bender","startTime":806.755,"endTime":814.0,"body":"its response took exactly the same"},{"speaker":"Emily M. Bender","startTime":814.0,"endTime":817.06,"body":"amount of time, no matter how much"},{"speaker":"Emily M. Bender","startTime":814.0,"endTime":817.06,"body":"time it would take to actually compute"},{"speaker":"Emily M. Bender","startTime":817.06,"endTime":820.77,"body":"the problem that you're asking it to"},{"speaker":"Emily M. Bender","startTime":817.06,"endTime":820.77,"body":"do, which is a very clear indication,"},{"speaker":"Emily M. Bender","startTime":820.77,"endTime":823.8,"body":"as if we didn't know already that it"},{"speaker":"Emily M. Bender","startTime":820.77,"endTime":823.8,"body":"is not actually executing a program."},{"speaker":"Alex Hanna","startTime":823.8,"endTime":827.08,"body":"Yeah, yeah, that's right."},{"speaker":"Alex Hanna","startTime":827.13,"endTime":827.519,"body":"That's right."},{"speaker":"Alex Hanna","startTime":827.539,"endTime":829.18,"body":"That's, that's an excellent point."},{"speaker":"Alex Hanna","startTime":829.689,"endTime":834.78,"body":"And so to pose it, I mean, and just"},{"speaker":"Alex Hanna","startTime":829.689,"endTime":834.78,"body":"to back up, like, as a repository of"},{"speaker":"Alex Hanna","startTime":834.78,"endTime":840.83,"body":"vector programs, like a, you're kind"},{"speaker":"Alex Hanna","startTime":834.78,"endTime":840.83,"body":"of, is that like an analog of saying"},{"speaker":"Alex Hanna","startTime":840.83,"endTime":845.72,"body":"that--then what's, an LLM then is"},{"speaker":"Alex Hanna","startTime":840.83,"endTime":845.72,"body":"like importing a class library and a"},{"speaker":"Alex Hanna","startTime":845.72,"endTime":847.27,"body":"programming language, is that an LLM?"},{"speaker":"Alex Hanna","startTime":847.97,"endTime":854.629,"body":"Like, is it, is, is, is, is a, is"},{"speaker":"Alex Hanna","startTime":847.97,"endTime":854.629,"body":"a set of bytecode encoded like the"},{"speaker":"Alex Hanna","startTime":854.63,"endTime":856.659,"body":"string library in Java, is that an LLM?"},{"speaker":"Alex Hanna","startTime":856.67,"endTime":857.79,"body":"Like what is now?"},{"speaker":"Alex Hanna","startTime":857.83,"endTime":862.1,"body":"And I'm so, I'm just like, I'm so, I'm"},{"speaker":"Alex Hanna","startTime":857.83,"endTime":862.1,"body":"very befuddled at that, at that claim."},{"speaker":"Alex Hanna","startTime":863.39,"endTime":867.35,"body":"That said, if you believe Cholet and you"},{"speaker":"Alex Hanna","startTime":863.39,"endTime":867.35,"body":"start there, you say, \"When prompted,"},{"speaker":"Alex Hanna","startTime":867.35,"endTime":871.5,"body":"that they will fetch the program that"},{"speaker":"Alex Hanna","startTime":867.35,"endTime":871.5,"body":"your prompt maps to and quote, 'executes'"},{"speaker":"Alex Hanna","startTime":871.63,"endTime":873.53,"body":"it--\" Again, doesn't execute it at all."},{"speaker":"Alex Hanna","startTime":873.9,"endTime":878.04,"body":"Uh, it just predicts, uh, most"},{"speaker":"Alex Hanna","startTime":873.9,"endTime":878.04,"body":"probable next token to some degree"},{"speaker":"Alex Hanna","startTime":878.04,"endTime":882.73,"body":"with, with some, with some, um, some"},{"speaker":"Alex Hanna","startTime":878.04,"endTime":882.73,"body":"like reinforcement learning on the"},{"speaker":"Alex Hanna","startTime":882.73,"endTime":884.72,"body":"tail end and, \"--on the input at hand."},{"speaker":"Alex Hanna","startTime":884.73,"endTime":887.839,"body":"LLMs are a way to store and"},{"speaker":"Alex Hanna","startTime":884.73,"endTime":887.839,"body":"operationalize millions of useful"},{"speaker":"Alex Hanna","startTime":887.85,"endTime":892.44,"body":"mini programs via passive exposure"},{"speaker":"Alex Hanna","startTime":887.85,"endTime":892.44,"body":"to human generated content.\""},{"speaker":"Alex Hanna","startTime":892.79,"endTime":895.33,"body":"And I'm like, wait, wait,"},{"speaker":"Alex Hanna","startTime":892.79,"endTime":895.33,"body":"that's not what it's doing."},{"speaker":"Emily M. Bender","startTime":896.4,"endTime":898.099,"body":"And on top of"},{"speaker":"Emily M. Bender","startTime":896.4,"endTime":898.099,"body":"that, that requires thinking"},{"speaker":"Emily M. Bender","startTime":898.1,"endTime":900.635,"body":"about people as computers."},{"speaker":"Emily M. Bender","startTime":901.29,"endTime":904.92,"body":"Right, so this human generated content"},{"speaker":"Emily M. Bender","startTime":901.29,"endTime":904.92,"body":"is, you know, either just the text"},{"speaker":"Emily M. Bender","startTime":904.92,"endTime":907.51,"body":"that's been scraped by the web or"},{"speaker":"Emily M. Bender","startTime":904.92,"endTime":907.51,"body":"OpenAI commissioned a whole bunch of"},{"speaker":"Emily M. Bender","startTime":907.51,"endTime":909.399,"body":"chain of thought sequences probably."},{"speaker":"Emily M. Bender","startTime":909.4,"endTime":913.519,"body":"And so Chollet is saying those are"},{"speaker":"Emily M. Bender","startTime":909.4,"endTime":913.519,"body":"programs, they're programs that are"},{"speaker":"Emily M. Bender","startTime":913.519,"endTime":917.51,"body":"produced naturally by people and"},{"speaker":"Emily M. Bender","startTime":913.519,"endTime":917.51,"body":"LLMs can execute them now because"},{"speaker":"Emily M. Bender","startTime":917.769,"endTime":919.359,"body":"they have an index of all of them."},{"speaker":"Alex Hanna","startTime":919.61,"endTime":920.1,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":920.75,"endTime":924.31,"body":"Well, it's, I don't even, I would say"},{"speaker":"Alex Hanna","startTime":920.75,"endTime":924.31,"body":"like, you don't have to go as far as"},{"speaker":"Alex Hanna","startTime":924.32,"endTime":928.689,"body":"seeing it's humans as programs is that"},{"speaker":"Alex Hanna","startTime":924.32,"endTime":928.689,"body":"you think that human generated content"},{"speaker":"Alex Hanna","startTime":929.209,"endTime":938.259,"body":"is, has effectively like, you know, humans"},{"speaker":"Alex Hanna","startTime":929.209,"endTime":938.259,"body":"generate outputs and there's ostensibly"},{"speaker":"Alex Hanna","startTime":938.31,"endTime":943.57,"body":"maps to some kind of internal input at"},{"speaker":"Alex Hanna","startTime":938.31,"endTime":943.57,"body":"that you can operationalize those as"},{"speaker":"Alex Hanna","startTime":943.78,"endTime":950.34,"body":"programs as if like humans--I mean, like,"},{"speaker":"Alex Hanna","startTime":943.78,"endTime":950.34,"body":"yeah, I mean, I could see it, but I'm"},{"speaker":"Alex Hanna","startTime":950.34,"endTime":956.03,"body":"just like, it's such a, it takes you down"},{"speaker":"Alex Hanna","startTime":950.34,"endTime":956.03,"body":"like a model of thinking that is bizarre."},{"speaker":"Alex Hanna","startTime":956.03,"endTime":959.659,"body":"And one of these days we have to"},{"speaker":"Alex Hanna","startTime":956.03,"endTime":959.659,"body":"dig into, Chollet has this essay on"},{"speaker":"Alex Hanna","startTime":959.659,"endTime":963.849,"body":"intelligence that I've been meaning to"},{"speaker":"Alex Hanna","startTime":959.659,"endTime":963.849,"body":"read because it's, it's sort of like an"},{"speaker":"Alex Hanna","startTime":963.86,"endTime":967.57,"body":"insight into like, what he thinks humans"},{"speaker":"Alex Hanna","startTime":963.86,"endTime":967.57,"body":"are doing and what intelligence is."},{"speaker":"Emily M. Bender","startTime":967.65,"endTime":970.33,"body":"So there's a wonderful"},{"speaker":"Emily M. Bender","startTime":967.65,"endTime":970.33,"body":"paper by Baria and Cross that I"},{"speaker":"Emily M. Bender","startTime":970.34,"endTime":973.74,"body":"don't, I think it only exists as a"},{"speaker":"Emily M. Bender","startTime":970.34,"endTime":973.74,"body":"manuscript called something like,"},{"speaker":"Emily M. Bender","startTime":974.09,"endTime":977.56,"body":"uh, \"The brain is a computer is a"},{"speaker":"Emily M. Bender","startTime":974.09,"endTime":977.56,"body":"brain,\" and it's talking about the"},{"speaker":"Emily M. Bender","startTime":977.56,"endTime":979.44,"body":"computational metaphor in neuroscience."},{"speaker":"Emily M. Bender","startTime":979.74,"endTime":982.7,"body":"Um, and it's, it's a really valuable"},{"speaker":"Emily M. Bender","startTime":979.74,"endTime":982.7,"body":"piece where they sort of call out this"},{"speaker":"Emily M. Bender","startTime":982.71,"endTime":987.35,"body":"thinking that basically says our mental"},{"speaker":"Emily M. Bender","startTime":982.71,"endTime":987.35,"body":"models of what happens in machines are"},{"speaker":"Emily M. Bender","startTime":987.35,"endTime":988.87,"body":"a good model of what happens in humans."},{"speaker":"Emily M. Bender","startTime":988.87,"endTime":992.16,"body":"And I think that Chollet is, or whoever"},{"speaker":"Emily M. Bender","startTime":988.87,"endTime":992.16,"body":"wrote this is deep in that here."},{"speaker":"Alex Hanna","startTime":992.429,"endTime":993.579,"body":"Yeah, completely."},{"speaker":"Emily M. Bender","startTime":993.659,"endTime":993.929,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":994.12,"endTime":995.72,"body":"So there's some stuff"},{"speaker":"Emily M. Bender","startTime":994.12,"endTime":995.72,"body":"in the chat to bring up."},{"speaker":"Alex Hanna","startTime":996.18,"endTime":996.45,"body":"I know."},{"speaker":"Alex Hanna","startTime":996.47,"endTime":1000.28,"body":"I just wanted to say this thing from"},{"speaker":"Alex Hanna","startTime":996.47,"endTime":1000.28,"body":"Scupper Club, first time chatter."},{"speaker":"Alex Hanna","startTime":1000.28,"endTime":1000.68,"body":"Hey."},{"speaker":"Alex Hanna","startTime":1001.015,"endTime":1003.385,"body":"\"JVM--\" As in the Java Virtual Machine."},{"speaker":"Alex Hanna","startTime":1003.395,"endTime":1007.725,"body":"\"equals, equals LLM,\""},{"speaker":"Alex Hanna","startTime":1003.395,"endTime":1007.725,"body":"which is very, very funny."},{"speaker":"Alex Hanna","startTime":1007.765,"endTime":1008.185,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1008.335,"endTime":1010.325,"body":"And it's also"},{"speaker":"Emily M. Bender","startTime":1008.335,"endTime":1010.325,"body":"a bunch of commentary in here"},{"speaker":"Emily M. Bender","startTime":1010.355,"endTime":1014.595,"body":"about, um, how benchmarks aren't,"},{"speaker":"Emily M. Bender","startTime":1010.355,"endTime":1014.595,"body":"there's no construct validity."},{"speaker":"Emily M. Bender","startTime":1014.765,"endTime":1017.514,"body":"So I think it was Abstract"},{"speaker":"Emily M. Bender","startTime":1014.765,"endTime":1017.514,"body":"Tesseract probably?"},{"speaker":"Emily M. Bender","startTime":1017.745,"endTime":1017.975,"body":"Yes."},{"speaker":"Emily M. Bender","startTime":1018.005,"endTime":1020.175,"body":"\"Apparently you don't have to"},{"speaker":"Emily M. Bender","startTime":1018.005,"endTime":1020.175,"body":"worry about a benchmark's construct"},{"speaker":"Emily M. Bender","startTime":1020.194,"endTime":1021.575,"body":"validity if it's the only one.\""},{"speaker":"Alex Hanna","startTime":1022.31,"endTime":1023.77,"body":"That's,"},{"speaker":"Alex Hanna","startTime":1022.31,"endTime":1023.77,"body":"that's, that's very funny."},{"speaker":"Emily M. Bender","startTime":1024.56,"endTime":1027.28,"body":"And Faster And"},{"speaker":"Emily M. Bender","startTime":1024.56,"endTime":1027.28,"body":"Worse points out like, \"No concrete"},{"speaker":"Emily M. Bender","startTime":1027.28,"endTime":1029.88,"body":"purpose means no criteria to"},{"speaker":"Emily M. Bender","startTime":1027.28,"endTime":1029.88,"body":"determine success or failure.\""},{"speaker":"Emily M. Bender","startTime":1029.88,"endTime":1030.83,"body":"And that's exactly it."},{"speaker":"Emily M. Bender","startTime":1030.83,"endTime":1033.46,"body":"Like, what is this benchmark"},{"speaker":"Emily M. Bender","startTime":1030.83,"endTime":1033.46,"body":"actually measuring?"},{"speaker":"Emily M. Bender","startTime":1033.68,"endTime":1037.229,"body":"And if we were doing serious science"},{"speaker":"Emily M. Bender","startTime":1033.68,"endTime":1037.229,"body":"or engineering in the world, we would"},{"speaker":"Emily M. Bender","startTime":1037.24,"endTime":1041.41,"body":"be either saying, this is a way that"},{"speaker":"Emily M. Bender","startTime":1037.24,"endTime":1041.41,"body":"we know that we can use this safely and"},{"speaker":"Emily M. Bender","startTime":1041.42,"endTime":1043.53,"body":"effectively for this particular task."},{"speaker":"Emily M. Bender","startTime":1043.835,"endTime":1046.695,"body":"Or we'd be establishing construct"},{"speaker":"Emily M. Bender","startTime":1043.835,"endTime":1046.695,"body":"validity because we're doing science."},{"speaker":"Emily M. Bender","startTime":1046.715,"endTime":1047.695,"body":"And neither of those things are happening."},{"speaker":"Alex Hanna","startTime":1048.335,"endTime":1050.485,"body":"Well, I was, I was"},{"speaker":"Alex Hanna","startTime":1048.335,"endTime":1050.485,"body":"having this conversation this morning"},{"speaker":"Alex Hanna","startTime":1050.495,"endTime":1055.094,"body":"with one of, um, one of the PhD"},{"speaker":"Alex Hanna","startTime":1050.495,"endTime":1055.094,"body":"students that works at DAIR, Raesetje"},{"speaker":"Alex Hanna","startTime":1055.094,"endTime":1059.965,"body":"Sefala, uh, who's doing some really"},{"speaker":"Alex Hanna","startTime":1055.094,"endTime":1059.965,"body":"interesting work on, on error analysis."},{"speaker":"Alex Hanna","startTime":1060.004,"endTime":1063.504,"body":"And we were just having this"},{"speaker":"Alex Hanna","startTime":1060.004,"endTime":1063.504,"body":"conversation about construct validity"},{"speaker":"Alex Hanna","startTime":1063.505,"endTime":1069.335,"body":"and how the field of AI slash computer"},{"speaker":"Alex Hanna","startTime":1063.505,"endTime":1069.335,"body":"science really doesn't do anything"},{"speaker":"Alex Hanna","startTime":1069.335,"endTime":1072.885,"body":"with it, doesn't really think about"},{"speaker":"Alex Hanna","startTime":1069.335,"endTime":1072.885,"body":"measurement, hasn't engaged with"},{"speaker":"Alex Hanna","startTime":1072.885,"endTime":1078.154,"body":"the giant literature on measurement"},{"speaker":"Alex Hanna","startTime":1072.885,"endTime":1078.154,"body":"from, from scientometrics, from"},{"speaker":"Alex Hanna","startTime":1078.154,"endTime":1080.384,"body":"political methodology, from education."},{"speaker":"Alex Hanna","startTime":1080.694,"endTime":1080.944,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1080.944,"endTime":1087.545,"body":"Just like giant kinds of portions, uh,"},{"speaker":"Alex Hanna","startTime":1080.944,"endTime":1087.545,"body":"like, um, um, you know, social sciences"},{"speaker":"Alex Hanna","startTime":1087.545,"endTime":1091.375,"body":"and even, I mean, like, especially the"},{"speaker":"Alex Hanna","startTime":1087.545,"endTime":1091.375,"body":"quantitative social sciences, yeah, what"},{"speaker":"Alex Hanna","startTime":1091.375,"endTime":1093.245,"body":"the hell is a Lickert scale actually."},{"speaker":"Alex Hanna","startTime":1093.994,"endTime":1096.84,"body":"Um, but no, there's no, there's"},{"speaker":"Alex Hanna","startTime":1093.994,"endTime":1096.84,"body":"no engagement in that at all."},{"speaker":"Emily M. Bender","startTime":1096.9,"endTime":1097.3,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1097.59,"endTime":1097.84,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1098.54,"endTime":1098.85,"body":"All right."},{"speaker":"Emily M. Bender","startTime":1098.92,"endTime":1100.65,"body":"What else do we want to"},{"speaker":"Emily M. Bender","startTime":1098.92,"endTime":1100.65,"body":"say about this in here?"},{"speaker":"Emily M. Bender","startTime":1100.77,"endTime":1104.059,"body":"Um, I guess also this, yeah, so this,"},{"speaker":"Emily M. Bender","startTime":1100.77,"endTime":1104.059,"body":"the whole idea of like, it's retrieving"},{"speaker":"Emily M. Bender","startTime":1104.06,"endTime":1105.35,"body":"these things and executing them."},{"speaker":"Emily M. Bender","startTime":1105.619,"endTime":1106.26,"body":"No, it's not."},{"speaker":"Emily M. Bender","startTime":1106.27,"endTime":1108.04,"body":"As you say, Alex, it's just"},{"speaker":"Emily M. Bender","startTime":1106.27,"endTime":1108.04,"body":"outputting the next word."},{"speaker":"Emily M. Bender","startTime":1108.04,"endTime":1111.339,"body":"And that was then conditioned on"},{"speaker":"Emily M. Bender","startTime":1108.04,"endTime":1111.339,"body":"the so called program that it wrote."},{"speaker":"Emily M. Bender","startTime":1111.76,"endTime":1117.52,"body":"Um, and it's just, this is--the other"},{"speaker":"Emily M. Bender","startTime":1111.76,"endTime":1117.52,"body":"thing that's super frustrating here is"},{"speaker":"Emily M. Bender","startTime":1117.52,"endTime":1122.79,"body":"that even though OpenAI has collaborated"},{"speaker":"Emily M. Bender","startTime":1117.52,"endTime":1122.79,"body":"with the ARC Prize people on this."},{"speaker":"Emily M. Bender","startTime":1123.2,"endTime":1125.35,"body":"They have not been transparent"},{"speaker":"Emily M. Bender","startTime":1123.2,"endTime":1125.35,"body":"about what's inside their"},{"speaker":"Emily M. Bender","startTime":1125.35,"endTime":1126.47,"body":"system, how they built it."},{"speaker":"Emily M. Bender","startTime":1126.71,"endTime":1131.29,"body":"And so here we have Chollet like,"},{"speaker":"Emily M. Bender","startTime":1126.71,"endTime":1131.29,"body":"making stuff up, sort of, you know--"},{"speaker":"Alex Hanna","startTime":1132.14,"endTime":1132.45,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1132.83,"endTime":1135.72,"body":"--guessing what might"},{"speaker":"Emily M. Bender","startTime":1132.83,"endTime":1135.72,"body":"be going on in the inside when there"},{"speaker":"Emily M. Bender","startTime":1135.72,"endTime":1137.67,"body":"are people who design that system"},{"speaker":"Emily M. Bender","startTime":1135.72,"endTime":1137.67,"body":"who could answer these questions."},{"speaker":"Alex Hanna","startTime":1138.21,"endTime":1142.785,"body":"Yeah, that's that's very true"},{"speaker":"Alex Hanna","startTime":1138.21,"endTime":1142.785,"body":"and very frustrating that they do this."},{"speaker":"Alex Hanna","startTime":1142.785,"endTime":1144.615,"body":"And, and Chollet kind of speculates."},{"speaker":"Alex Hanna","startTime":1144.615,"endTime":1148.765,"body":"He said, basically there's certain"},{"speaker":"Alex Hanna","startTime":1144.615,"endTime":1148.765,"body":"kinds of spaces of possible chains"},{"speaker":"Alex Hanna","startTime":1148.765,"endTime":1153.624,"body":"of thought, which are, uh, may or may"},{"speaker":"Alex Hanna","startTime":1148.765,"endTime":1153.624,"body":"not be human annotated, describing"},{"speaker":"Alex Hanna","startTime":1153.624,"endTime":1157.925,"body":"the steps required to, to, to solve"},{"speaker":"Alex Hanna","startTime":1153.624,"endTime":1157.925,"body":"a task in a fashion, and this is what"},{"speaker":"Alex Hanna","startTime":1157.925,"endTime":1160.335,"body":"gets me that I think is really funny."},{"speaker":"Alex Hanna","startTime":1160.795,"endTime":1165.264,"body":"Um, in a task, \"in a fashion, perhaps"},{"speaker":"Alex Hanna","startTime":1160.795,"endTime":1165.264,"body":"not too dissimilar to Alpha Zero"},{"speaker":"Alex Hanna","startTime":1165.264,"endTime":1167.174,"body":"style Monte Carlo Tree Search.\""},{"speaker":"Alex Hanna","startTime":1167.784,"endTime":1171.504,"body":"And so I'm just like, okay, so if you're,"},{"speaker":"Alex Hanna","startTime":1167.784,"endTime":1171.504,"body":"if you're developing something like Alpha"},{"speaker":"Alex Hanna","startTime":1171.504,"endTime":1179.319,"body":"Zero which is aimed, which is with the"},{"speaker":"Alex Hanna","startTime":1171.504,"endTime":1179.319,"body":"goal of either solving, uh, Go or chess,"},{"speaker":"Alex Hanna","startTime":1179.409,"endTime":1181.259,"body":"which I think, was AlphaZero chess?"},{"speaker":"Alex Hanna","startTime":1181.699,"endTime":1186.349,"body":"Um, I think it was originally chess"},{"speaker":"Alex Hanna","startTime":1181.699,"endTime":1186.349,"body":"and then AlphaZero Go was chess."},{"speaker":"Alex Hanna","startTime":1186.619,"endTime":1191.154,"body":"And so if you actually are like doing"},{"speaker":"Alex Hanna","startTime":1186.619,"endTime":1191.154,"body":"some kind of a tree search of possible"},{"speaker":"Alex Hanna","startTime":1191.164,"endTime":1197.534,"body":"outcomes and you're just sort of,"},{"speaker":"Alex Hanna","startTime":1191.164,"endTime":1197.534,"body":"you know, optimizing perhaps like the"},{"speaker":"Alex Hanna","startTime":1197.534,"endTime":1199.784,"body":"most optimal way to do a tree search."},{"speaker":"Alex Hanna","startTime":1200.224,"endTime":1204.424,"body":"Then again, like, why do you think"},{"speaker":"Alex Hanna","startTime":1200.224,"endTime":1204.424,"body":"these tasks have any kind of external"},{"speaker":"Alex Hanna","startTime":1204.424,"endTime":1209.049,"body":"validity to not just filling in"},{"speaker":"Alex Hanna","startTime":1204.424,"endTime":1209.049,"body":"patterns on a little grid format?"},{"speaker":"Alex Hanna","startTime":1209.049,"endTime":1209.644,"body":"Uh, like."},{"speaker":"Alex Hanna","startTime":1210.059,"endTime":1214.329,"body":"Like, why does this have any"},{"speaker":"Alex Hanna","startTime":1210.059,"endTime":1214.329,"body":"construct validity outside"},{"speaker":"Alex Hanna","startTime":1214.349,"endTime":1217.289,"body":"of this little game scenario?"},{"speaker":"Alex Hanna","startTime":1217.319,"endTime":1221.569,"body":"And I thought that was just"},{"speaker":"Alex Hanna","startTime":1217.319,"endTime":1221.569,"body":"so bizarre as like in really"},{"speaker":"Alex Hanna","startTime":1221.609,"endTime":1224.049,"body":"unreflective to even assert that."},{"speaker":"Emily M. Bender","startTime":1224.279,"endTime":1224.529,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1224.699,"endTime":1227.759,"body":"And I think basically because when"},{"speaker":"Emily M. Bender","startTime":1224.699,"endTime":1227.759,"body":"Chollet came up with this thing, none"},{"speaker":"Emily M. Bender","startTime":1227.759,"endTime":1229.239,"body":"of the existing systems could do it."},{"speaker":"Emily M. Bender","startTime":1229.579,"endTime":1232.649,"body":"He then gets to claim, this is"},{"speaker":"Emily M. Bender","startTime":1229.579,"endTime":1232.649,"body":"the thing that's going to drive"},{"speaker":"Emily M. Bender","startTime":1232.649,"endTime":1235.009,"body":"people towards AGI because we"},{"speaker":"Emily M. Bender","startTime":1232.649,"endTime":1235.009,"body":"don't have a solution for it yet."},{"speaker":"Emily M. Bender","startTime":1235.029,"endTime":1238.679,"body":"And then OpenAI goes and takes a"},{"speaker":"Emily M. Bender","startTime":1235.029,"endTime":1238.679,"body":"bunch of the examples as training"},{"speaker":"Emily M. Bender","startTime":1238.679,"endTime":1240.029,"body":"data, like, hey, look, we can do it."},{"speaker":"Emily M. Bender","startTime":1240.179,"endTime":1240.609,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1240.859,"endTime":1241.089,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1241.549,"endTime":1245.279,"body":"So I love this comment in"},{"speaker":"Emily M. Bender","startTime":1241.549,"endTime":1245.279,"body":"the chat from, um, Method And Structure,"},{"speaker":"Emily M. Bender","startTime":1245.619,"endTime":1251.159,"body":"\"Kremlinology around LLM contents is kind"},{"speaker":"Emily M. Bender","startTime":1245.619,"endTime":1251.159,"body":"of like SEO snake oilers trying to divine"},{"speaker":"Emily M. Bender","startTime":1251.159,"endTime":1252.559,"body":"what the Google algorithm is doing.\""},{"speaker":"Alex Hanna","startTime":1253.014,"endTime":1256.114,"body":"Yeah, no, that's,"},{"speaker":"Alex Hanna","startTime":1253.014,"endTime":1256.114,"body":"that's a, that's a great, um,"},{"speaker":"Alex Hanna","startTime":1256.304,"endTime":1257.634,"body":"that's a great comparison there."},{"speaker":"Emily M. Bender","startTime":1258.114,"endTime":1258.454,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1259.394,"endTime":1259.824,"body":"Um."},{"speaker":"Alex Hanna","startTime":1259.894,"endTime":1260.494,"body":"Okay."},{"speaker":"Alex Hanna","startTime":1260.554,"endTime":1264.454,"body":"What else is in this, uh, artifact?"},{"speaker":"Alex Hanna","startTime":1264.484,"endTime":1270.294,"body":"Um, there's, um, some stuff, um,"},{"speaker":"Alex Hanna","startTime":1264.484,"endTime":1270.294,"body":"there's what comes, so this is"},{"speaker":"Alex Hanna","startTime":1270.294,"endTime":1273.164,"body":"the, what comes next, which I"},{"speaker":"Alex Hanna","startTime":1270.294,"endTime":1273.164,"body":"thought was pretty interesting."},{"speaker":"Alex Hanna","startTime":1273.174,"endTime":1274.624,"body":"\"So what comes next?"},{"speaker":"Alex Hanna","startTime":1275.114,"endTime":1279.224,"body":"First of all, open source replication"},{"speaker":"Alex Hanna","startTime":1275.114,"endTime":1279.224,"body":"of o3--\" Uh, I'm like, okay."},{"speaker":"Alex Hanna","startTime":1280.004,"endTime":1281.504,"body":"\"--facilitated by ARC Prize.\""},{"speaker":"Emily M. Bender","startTime":1281.524,"endTime":1282.294,"body":"Based on what?"},{"speaker":"Emily M. Bender","startTime":1282.504,"endTime":1283.264,"body":"Based on what?"},{"speaker":"Alex Hanna","startTime":1283.374,"endTime":1284.864,"body":"Yeah, I, I don't know."},{"speaker":"Alex Hanna","startTime":1284.984,"endTime":1286.734,"body":"It says, it just asserts that."},{"speaker":"Alex Hanna","startTime":1287.264,"endTime":1292.264,"body":"Uh, \"--facilitated by ARC Prize"},{"speaker":"Alex Hanna","startTime":1287.264,"endTime":1292.264,"body":"competition 2025 will be crucial to"},{"speaker":"Alex Hanna","startTime":1292.264,"endTime":1293.604,"body":"move the research community forward."},{"speaker":"Alex Hanna","startTime":1294.084,"endTime":1297.994,"body":"A thorough analysis of o3's strengths"},{"speaker":"Alex Hanna","startTime":1294.084,"endTime":1297.994,"body":"and limitations is necessary to"},{"speaker":"Alex Hanna","startTime":1297.994,"endTime":1301.444,"body":"understand its scaling behavior, the"},{"speaker":"Alex Hanna","startTime":1297.994,"endTime":1301.444,"body":"nature of its potential bottlenecks,"},{"speaker":"Alex Hanna","startTime":1302.064,"endTime":1305.789,"body":"and anticipate what abilities further,"},{"speaker":"Alex Hanna","startTime":1302.064,"endTime":1305.789,"body":"these further developments might unlock.\""},{"speaker":"Alex Hanna","startTime":1306.569,"endTime":1307.509,"body":"Um."},{"speaker":"Emily M. Bender","startTime":1308.049,"endTime":1310.289,"body":"Can we just appreciate"},{"speaker":"Emily M. Bender","startTime":1308.049,"endTime":1310.289,"body":"the video game metaphor here?"},{"speaker":"Alex Hanna","startTime":1310.689,"endTime":1311.149,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1311.419,"endTime":1311.859,"body":"Right?"},{"speaker":"Emily M. Bender","startTime":1311.929,"endTime":1313.869,"body":"These developments are"},{"speaker":"Emily M. Bender","startTime":1311.929,"endTime":1313.869,"body":"going to unlock abilities."},{"speaker":"Emily M. Bender","startTime":1313.87,"endTime":1315.569,"body":"In what world does that happen?"},{"speaker":"Emily M. Bender","startTime":1315.789,"endTime":1316.179,"body":"Oh, right."},{"speaker":"Emily M. Bender","startTime":1316.179,"endTime":1316.969,"body":"Video games."},{"speaker":"Alex Hanna","startTime":1317.449,"endTime":1322.519,"body":"Well, I'm also, like,"},{"speaker":"Alex Hanna","startTime":1317.449,"endTime":1322.519,"body":"very, it's just, I'm so tickled about,"},{"speaker":"Alex Hanna","startTime":1322.519,"endTime":1326.654,"body":"like, what is the, well the next,"},{"speaker":"Alex Hanna","startTime":1322.519,"endTime":1326.654,"body":"the next sentence kind of throws me"},{"speaker":"Alex Hanna","startTime":1326.654,"endTime":1331.104,"body":"because I don't understand what this"},{"speaker":"Alex Hanna","startTime":1326.654,"endTime":1331.104,"body":"means and how you quantify this."},{"speaker":"Alex Hanna","startTime":1331.114,"endTime":1335.394,"body":"And I've never seen this language in AI."},{"speaker":"Alex Hanna","startTime":1335.404,"endTime":1337.384,"body":"And I'm like, how do you"},{"speaker":"Alex Hanna","startTime":1335.404,"endTime":1337.384,"body":"even describe these problems?"},{"speaker":"Alex Hanna","startTime":1337.384,"endTime":1342.124,"body":"Anyways, they say, \"Moreover,"},{"speaker":"Alex Hanna","startTime":1337.384,"endTime":1342.124,"body":"ARC AGI One is now saturating.\""},{"speaker":"Alex Hanna","startTime":1342.605,"endTime":1345.914,"body":"Um, which I don't, I've never"},{"speaker":"Alex Hanna","startTime":1342.605,"endTime":1345.914,"body":"heard as like, well, I've heard of"},{"speaker":"Alex Hanna","startTime":1345.914,"endTime":1349.759,"body":"this term saturation in terms of"},{"speaker":"Alex Hanna","startTime":1345.914,"endTime":1349.759,"body":"benchmarks, but like, again, what"},{"speaker":"Alex Hanna","startTime":1349.759,"endTime":1352.809,"body":"does that mean with regards to"},{"speaker":"Alex Hanna","startTime":1349.759,"endTime":1352.809,"body":"real world, real world problems?"},{"speaker":"Alex Hanna","startTime":1353.219,"endTime":1357.459,"body":"\"But setting that aside, beside"},{"speaker":"Alex Hanna","startTime":1353.219,"endTime":1357.459,"body":"o1's, o3's new score, the fact is"},{"speaker":"Alex Hanna","startTime":1357.459,"endTime":1362.059,"body":"that a large ensemble of low compute"},{"speaker":"Alex Hanna","startTime":1357.459,"endTime":1362.059,"body":"Kaggle solutions can now score"},{"speaker":"Alex Hanna","startTime":1362.089,"endTime":1364.299,"body":"83 percent on the private eval.\""},{"speaker":"Alex Hanna","startTime":1364.739,"endTime":1368.794,"body":"So that's effectively say the pub, so"},{"speaker":"Alex Hanna","startTime":1364.739,"endTime":1368.794,"body":"the public, the public eval are these"},{"speaker":"Alex Hanna","startTime":1368.804,"endTime":1371.764,"body":"quote quote low compute Kaggle solutions."},{"speaker":"Alex Hanna","startTime":1372.154,"endTime":1376.934,"body":"Uh, and for those of you, those of you"},{"speaker":"Alex Hanna","startTime":1372.154,"endTime":1376.934,"body":"listening, Kaggles like this, um, you"},{"speaker":"Alex Hanna","startTime":1376.934,"endTime":1381.414,"body":"know, it's a, it's kind of a competition"},{"speaker":"Alex Hanna","startTime":1376.934,"endTime":1381.414,"body":"platform that was once independent"},{"speaker":"Alex Hanna","startTime":1381.434,"endTime":1382.634,"body":"and then Google bought them up."},{"speaker":"Alex Hanna","startTime":1382.974,"endTime":1386.259,"body":"And so it's sort of like, and it's had"},{"speaker":"Alex Hanna","startTime":1382.974,"endTime":1386.259,"body":"small little data science problems,"},{"speaker":"Alex Hanna","startTime":1386.259,"endTime":1390.754,"body":"like, you know, predict--what was the"},{"speaker":"Alex Hanna","startTime":1386.259,"endTime":1390.754,"body":"Titanic one, like predict who like got"},{"speaker":"Alex Hanna","startTime":1390.774,"endTime":1395.194,"body":"off the Titanic or whatever, um, and"},{"speaker":"Alex Hanna","startTime":1390.774,"endTime":1395.194,"body":"then--which is macabre in its own right."},{"speaker":"Alex Hanna","startTime":1395.194,"endTime":1398.434,"body":"But then they've also got other"},{"speaker":"Alex Hanna","startTime":1395.194,"endTime":1398.434,"body":"things, like predict, you know,"},{"speaker":"Alex Hanna","startTime":1398.464,"endTime":1402.224,"body":"income from this old census data set"},{"speaker":"Alex Hanna","startTime":1398.464,"endTime":1402.224,"body":"and all these other toy problems."},{"speaker":"Emily M. Bender","startTime":1402.224,"endTime":1403.834,"body":"Yeah, basically"},{"speaker":"Emily M. Bender","startTime":1402.224,"endTime":1403.834,"body":"like a machine learning leaderboard."},{"speaker":"Emily M. Bender","startTime":1403.885,"endTime":1406.564,"body":"It was an interesting moment because"},{"speaker":"Emily M. Bender","startTime":1403.885,"endTime":1406.564,"body":"prior to that, you would have a bunch of"},{"speaker":"Emily M. Bender","startTime":1406.564,"endTime":1410.034,"body":"shared tasks, but they were always like"},{"speaker":"Emily M. Bender","startTime":1406.564,"endTime":1410.034,"body":"connected to academic events where people"},{"speaker":"Emily M. Bender","startTime":1410.224,"endTime":1411.474,"body":"would come and talk about what they did."},{"speaker":"Emily M. Bender","startTime":1411.474,"endTime":1412.264,"body":"And then Kaggle's like, oh, yeah."},{"speaker":"Emily M. Bender","startTime":1412.434,"endTime":1415.644,"body":"Here, let's just have the competitions"},{"speaker":"Emily M. Bender","startTime":1412.434,"endTime":1415.644,"body":"without the actual discussion and then"},{"speaker":"Emily M. Bender","startTime":1415.644,"endTime":1418.134,"body":"people would put it on their resumes"},{"speaker":"Emily M. Bender","startTime":1415.644,"endTime":1418.134,"body":"like I won, you know, I was the top of"},{"speaker":"Emily M. Bender","startTime":1418.134,"endTime":1419.404,"body":"the leaderboard on this Kaggle thing."},{"speaker":"Emily M. Bender","startTime":1419.405,"endTime":1423.094,"body":"And yeah, so this thing about saturating."},{"speaker":"Alex Hanna","startTime":1423.224,"endTime":1423.474,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1423.494,"endTime":1425.904,"body":"Um, like that is a"},{"speaker":"Emily M. Bender","startTime":1423.494,"endTime":1425.904,"body":"metaphor that we talk about, right?"},{"speaker":"Emily M. Bender","startTime":1426.264,"endTime":1429.344,"body":"You see it when they're talking about the"},{"speaker":"Emily M. Bender","startTime":1426.264,"endTime":1429.344,"body":"benchmarks and what's sort of shocking"},{"speaker":"Emily M. Bender","startTime":1429.364,"endTime":1434.504,"body":"here is that to anyone who's not wedded"},{"speaker":"Emily M. Bender","startTime":1429.364,"endTime":1434.504,"body":"to this idea that these benchmarks"},{"speaker":"Emily M. Bender","startTime":1434.504,"endTime":1438.624,"body":"are actually driving progress, it"},{"speaker":"Emily M. Bender","startTime":1434.504,"endTime":1438.624,"body":"just clearly shows that benchmarks get"},{"speaker":"Emily M. Bender","startTime":1438.624,"endTime":1442.094,"body":"saturated or worn out because they're"},{"speaker":"Emily M. Bender","startTime":1438.624,"endTime":1442.094,"body":"not effectively measuring anything."},{"speaker":"Emily M. Bender","startTime":1442.704,"endTime":1445.494,"body":"But the people who are down inside this"},{"speaker":"Emily M. Bender","startTime":1442.704,"endTime":1445.494,"body":"rabbit hole are like okay, well this"},{"speaker":"Emily M. Bender","startTime":1445.494,"endTime":1447.044,"body":"one saturates we got to build a new one."},{"speaker":"Alex Hanna","startTime":1447.524,"endTime":1448.054,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1448.544,"endTime":1448.884,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1449.134,"endTime":1452.604,"body":"And it's really it's pretty"},{"speaker":"Alex Hanna","startTime":1449.134,"endTime":1452.604,"body":"upsetting, like people get"},{"speaker":"Alex Hanna","startTime":1452.604,"endTime":1453.934,"body":"really taken in about this."},{"speaker":"Alex Hanna","startTime":1454.234,"endTime":1459.674,"body":"I saw, um, like, you know, Terence Tao,"},{"speaker":"Alex Hanna","startTime":1454.234,"endTime":1459.674,"body":"the famous mathematician, he's become"},{"speaker":"Alex Hanna","startTime":1459.674,"endTime":1463.864,"body":"like very fascinated with like LLMs and"},{"speaker":"Alex Hanna","startTime":1459.674,"endTime":1463.864,"body":"like solving a mathematical problem."},{"speaker":"Emily M. Bender","startTime":1463.934,"endTime":1464.324,"body":"I know."},{"speaker":"Emily M. Bender","startTime":1464.354,"endTime":1465.134,"body":"Super disappointing."},{"speaker":"Alex Hanna","startTime":1465.144,"endTime":1465.404,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1465.404,"endTime":1468.729,"body":"And I'm like, ah, he was actually"},{"speaker":"Alex Hanna","startTime":1465.404,"endTime":1468.729,"body":"a, A friend of mine's advisor"},{"speaker":"Alex Hanna","startTime":1468.729,"endTime":1471.549,"body":"and I'm like, uh, I'm like, ah,"},{"speaker":"Alex Hanna","startTime":1468.729,"endTime":1471.549,"body":"your man is, come get your man."},{"speaker":"Alex Hanna","startTime":1471.549,"endTime":1476.409,"body":"He is just like completely like, like,"},{"speaker":"Alex Hanna","startTime":1471.549,"endTime":1476.409,"body":"uh, engrossed in this LLM uh, nonsense."},{"speaker":"Emily M. Bender","startTime":1477.839,"endTime":1478.169,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1478.169,"endTime":1478.249,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1478.609,"endTime":1481.229,"body":"So before we leave this, I just"},{"speaker":"Emily M. Bender","startTime":1478.609,"endTime":1481.229,"body":"wanna point out that there's a"},{"speaker":"Emily M. Bender","startTime":1481.229,"endTime":1482.879,"body":"really obnoxious visual effect."},{"speaker":"Emily M. Bender","startTime":1482.909,"endTime":1483.809,"body":"The font is terrible."},{"speaker":"Emily M. Bender","startTime":1483.809,"endTime":1484.529,"body":"It's hard to read."},{"speaker":"Emily M. Bender","startTime":1484.829,"endTime":1486.929,"body":"And then the background, they've"},{"speaker":"Emily M. Bender","startTime":1484.829,"endTime":1486.929,"body":"got this moving thing that looks"},{"speaker":"Emily M. Bender","startTime":1486.949,"endTime":1488.369,"body":"like you're in the Tron universe."},{"speaker":"Emily M. Bender","startTime":1488.419,"endTime":1490.709,"body":"I was like, I didn't need that"},{"speaker":"Emily M. Bender","startTime":1488.419,"endTime":1490.709,"body":"while I was trying to read this."},{"speaker":"Emily M. Bender","startTime":1492.659,"endTime":1495.259,"body":"Okay, so here's the open source"},{"speaker":"Emily M. Bender","startTime":1492.659,"endTime":1495.259,"body":"analysis that I think we can skip in"},{"speaker":"Emily M. Bender","startTime":1495.259,"endTime":1498.899,"body":"some examples, but we should go to"},{"speaker":"Emily M. Bender","startTime":1495.259,"endTime":1498.899,"body":"this analysis that went up on LinkedIn."},{"speaker":"Alex Hanna","startTime":1498.989,"endTime":1499.249,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1499.369,"endTime":1506.179,"body":"Um, someone named Boris,"},{"speaker":"Emily M. Bender","startTime":1499.369,"endTime":1506.179,"body":"um, Gamazaychikov, whose tag is \"head"},{"speaker":"Emily M. Bender","startTime":1506.179,"endTime":1512.264,"body":"of AI sustainability at Salesforce\","},{"speaker":"Emily M. Bender","startTime":1506.179,"endTime":1512.264,"body":"um, sat down and tried to estimate the"},{"speaker":"Emily M. Bender","startTime":1512.294,"endTime":1515.664,"body":"carbon emissions for each of those tasks."},{"speaker":"Alex Hanna","startTime":1516.304,"endTime":1519.304,"body":"And I want to, I want to"},{"speaker":"Alex Hanna","startTime":1516.304,"endTime":1519.304,"body":"note just his first sentence because"},{"speaker":"Alex Hanna","startTime":1519.354,"endTime":1521.104,"body":"I think it's, it's a little deceptive."},{"speaker":"Alex Hanna","startTime":1521.104,"endTime":1524.964,"body":"So, \"OpenAI has announced o3,"},{"speaker":"Alex Hanna","startTime":1521.104,"endTime":1524.964,"body":"which appears to be the most"},{"speaker":"Alex Hanna","startTime":1524.964,"endTime":1526.294,"body":"powerful AI model to date.\""},{"speaker":"Alex Hanna","startTime":1526.294,"endTime":1531.124,"body":"And it's first off, no, because it"},{"speaker":"Alex Hanna","startTime":1526.294,"endTime":1531.124,"body":"seems like o3 has just been engineered"},{"speaker":"Alex Hanna","startTime":1531.709,"endTime":1537.769,"body":"particularly for this ARC AGI"},{"speaker":"Alex Hanna","startTime":1531.709,"endTime":1537.769,"body":"challenge, if I'm not wrong, um, where"},{"speaker":"Alex Hanna","startTime":1537.939,"endTime":1543.039,"body":"it is, it is meant to do a certain"},{"speaker":"Alex Hanna","startTime":1537.939,"endTime":1543.039,"body":"kind of search, like deep search"},{"speaker":"Alex Hanna","startTime":1543.039,"endTime":1545.079,"body":"according to like Chollet's analysis."},{"speaker":"Alex Hanna","startTime":1545.449,"endTime":1550.889,"body":"So like, and again, like there's, there's,"},{"speaker":"Alex Hanna","startTime":1545.449,"endTime":1550.889,"body":"it's one of those situations where you're"},{"speaker":"Alex Hanna","startTime":1550.889,"endTime":1552.849,"body":"like, what, what do you mean by powerful?"},{"speaker":"Alex Hanna","startTime":1552.879,"endTime":1555.319,"body":"And what do you mean by AI?"},{"speaker":"Alex Hanna","startTime":1555.319,"endTime":1559.329,"body":"But that said, um, you know, it's"},{"speaker":"Alex Hanna","startTime":1555.319,"endTime":1559.329,"body":"the, it's the environmental cost."},{"speaker":"Alex Hanna","startTime":1559.629,"endTime":1561.949,"body":"And of course, they didn't"},{"speaker":"Alex Hanna","startTime":1559.629,"endTime":1561.949,"body":"really release this."},{"speaker":"Alex Hanna","startTime":1561.949,"endTime":1568.099,"body":"They had sort of a, a, um, a measure"},{"speaker":"Alex Hanna","startTime":1561.949,"endTime":1568.099,"body":"of compute, which he is then, um, doing"},{"speaker":"Alex Hanna","startTime":1568.099,"endTime":1572.009,"body":"some, um, uh, um, some calculation."},{"speaker":"Alex Hanna","startTime":1572.359,"endTime":1579.839,"body":"So, \"Each task consumed approximately"},{"speaker":"Alex Hanna","startTime":1572.359,"endTime":1579.839,"body":"1,785 kilowatt hours of energy, about"},{"speaker":"Alex Hanna","startTime":1579.839,"endTime":1583.959,"body":"the same amount of electricity an"},{"speaker":"Alex Hanna","startTime":1579.839,"endTime":1583.959,"body":"average US household uses in two months."},{"speaker":"Alex Hanna","startTime":1584.739,"endTime":1591.489,"body":"This translates to 684 kilograms of, um,"},{"speaker":"Alex Hanna","startTime":1584.739,"endTime":1591.489,"body":"of carbon dioxide emissions, based on last"},{"speaker":"Alex Hanna","startTime":1591.489,"endTime":1593.959,"body":"month's US, uh, uh, grid emission factor."},{"speaker":"Alex Hanna","startTime":1594.269,"endTime":1596.249,"body":"To put that in perspective,"},{"speaker":"Alex Hanna","startTime":1594.269,"endTime":1596.249,"body":"that's equivalent to the carbon"},{"speaker":"Alex Hanna","startTime":1596.249,"endTime":1600.359,"body":"emissions from more than five"},{"speaker":"Alex Hanna","startTime":1596.249,"endTime":1600.359,"body":"tanks of gas, full tanks of gas.\""},{"speaker":"Emily M. Bender","startTime":1601.189,"endTime":1601.539,"body":"Wonderful."},{"speaker":"Emily M. Bender","startTime":1601.539,"endTime":1604.209,"body":"And there's details of assumptions"},{"speaker":"Emily M. Bender","startTime":1601.539,"endTime":1604.209,"body":"down below, which is, which is"},{"speaker":"Emily M. Bender","startTime":1604.209,"endTime":1607.749,"body":"nice for sort of substantiating"},{"speaker":"Emily M. Bender","startTime":1604.209,"endTime":1607.749,"body":"that this was a serious analysis."},{"speaker":"Emily M. Bender","startTime":1607.899,"endTime":1611.624,"body":"Um, and yeah, it's like, and for what?"},{"speaker":"Emily M. Bender","startTime":1611.994,"endTime":1612.184,"body":"Right?"},{"speaker":"Alex Hanna","startTime":1613.624,"endTime":1614.104,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1614.664,"endTime":1619.244,"body":"To solve a little, a little,"},{"speaker":"Alex Hanna","startTime":1614.664,"endTime":1619.244,"body":"a little graph, um, problem"},{"speaker":"Alex Hanna","startTime":1619.244,"endTime":1621.254,"body":"here, not even a graph problem."},{"speaker":"Alex Hanna","startTime":1621.304,"endTime":1625.704,"body":"I mean, you could formulate as a"},{"speaker":"Alex Hanna","startTime":1621.304,"endTime":1625.704,"body":"graph problem, but OpenAI is, is"},{"speaker":"Alex Hanna","startTime":1626.054,"endTime":1627.504,"body":"formulating as a token problem."},{"speaker":"Emily M. Bender","startTime":1627.684,"endTime":1630.224,"body":"Oh, and I guess I wanted,"},{"speaker":"Emily M. Bender","startTime":1627.684,"endTime":1630.224,"body":"I wanted to, um, click through to that"},{"speaker":"Emily M. Bender","startTime":1630.224,"endTime":1634.724,"body":"too, because they do give the link here"},{"speaker":"Emily M. Bender","startTime":1630.224,"endTime":1634.724,"body":"to the prompt that they gave to o3."},{"speaker":"Emily M. Bender","startTime":1635.364,"endTime":1636.124,"body":"Where did that go?"},{"speaker":"Emily M. Bender","startTime":1636.124,"endTime":1638.964,"body":"Oh, Um, but just my little"},{"speaker":"Emily M. Bender","startTime":1636.124,"endTime":1638.964,"body":"search function here."},{"speaker":"Emily M. Bender","startTime":1639.504,"endTime":1642.484,"body":"Um, uh, here's the prompt."},{"speaker":"Emily M. Bender","startTime":1642.644,"endTime":1644.934,"body":"So we keep seeing these"},{"speaker":"Emily M. Bender","startTime":1642.644,"endTime":1644.934,"body":"pictures where they're colorful."},{"speaker":"Emily M. Bender","startTime":1645.254,"endTime":1649.724,"body":"But that's not how the"},{"speaker":"Emily M. Bender","startTime":1645.254,"endTime":1649.724,"body":"GPTs from OpenAI are built."},{"speaker":"Emily M. Bender","startTime":1649.724,"endTime":1652.184,"body":"And so what they've done instead"},{"speaker":"Emily M. Bender","startTime":1649.724,"endTime":1652.184,"body":"is they've, they've basically, I"},{"speaker":"Emily M. Bender","startTime":1652.184,"endTime":1655.404,"body":"guess, numbered the colors and said,"},{"speaker":"Emily M. Bender","startTime":1652.184,"endTime":1655.404,"body":"here's the input, here's the output."},{"speaker":"Emily M. Bender","startTime":1655.794,"endTime":1660.199,"body":"Um, and I don't understand why these"},{"speaker":"Emily M. Bender","startTime":1655.794,"endTime":1660.199,"body":"grids aren't the same size, um, but"},{"speaker":"Emily M. Bender","startTime":1660.199,"endTime":1663.609,"body":"basically the prompt is, \"Find the"},{"speaker":"Emily M. Bender","startTime":1660.199,"endTime":1663.609,"body":"common rule that maps an input grid to an"},{"speaker":"Emily M. Bender","startTime":1663.619,"endTime":1666.959,"body":"output grid,\" example one, input output,"},{"speaker":"Emily M. Bender","startTime":1663.619,"endTime":1666.959,"body":"example two, input output, example three."},{"speaker":"Emily M. Bender","startTime":1667.269,"endTime":1670.479,"body":"And , \"Below is a test input grid,"},{"speaker":"Emily M. Bender","startTime":1667.269,"endTime":1670.479,"body":"predict the corresponding output"},{"speaker":"Emily M. Bender","startTime":1670.979,"endTime":1672.889,"body":"grid by applying the rule you found."},{"speaker":"Emily M. Bender","startTime":1673.139,"endTime":1675.939,"body":"Your final answer should be just"},{"speaker":"Emily M. Bender","startTime":1673.139,"endTime":1675.939,"body":"the text of the output grid itself.\""},{"speaker":"Emily M. Bender","startTime":1676.079,"endTime":1677.519,"body":"So it's all translated to these numbers."},{"speaker":"Alex Hanna","startTime":1677.579,"endTime":1679.999,"body":"I think that the"},{"speaker":"Alex Hanna","startTime":1677.579,"endTime":1679.999,"body":"grids aren't the same because the"},{"speaker":"Alex Hanna","startTime":1679.999,"endTime":1682.249,"body":"task is like an expansion task."},{"speaker":"Alex Hanna","startTime":1682.819,"endTime":1686.789,"body":"Effectively, it's like, um, so"},{"speaker":"Alex Hanna","startTime":1682.819,"endTime":1686.789,"body":"you have this five by five grid."},{"speaker":"Emily M. Bender","startTime":1686.949,"endTime":1687.189,"body":"Oh yeah."},{"speaker":"Emily M. Bender","startTime":1687.269,"endTime":1688.209,"body":"It's just doubling everything, isn't it."},{"speaker":"Alex Hanna","startTime":1688.209,"endTime":1692.029,"body":"And then it, and then"},{"speaker":"Alex Hanna","startTime":1688.209,"endTime":1692.029,"body":"what it does is that each, each, uh,"},{"speaker":"Alex Hanna","startTime":1692.089,"endTime":1698.129,"body":"five in it, each color, let's say, uh,"},{"speaker":"Alex Hanna","startTime":1692.089,"endTime":1698.129,"body":"becomes four squares and then it draws"},{"speaker":"Alex Hanna","startTime":1698.169,"endTime":1699.859,"body":"like a line through it, it looks like."},{"speaker":"Alex Hanna","startTime":1701.019,"endTime":1702.339,"body":"So there's like--anyways."},{"speaker":"Emily M. Bender","startTime":1703.169,"endTime":1705.169,"body":"Yeah, it's got"},{"speaker":"Emily M. Bender","startTime":1703.169,"endTime":1705.169,"body":"it's got something is doing there."},{"speaker":"Emily M. Bender","startTime":1705.509,"endTime":1706.739,"body":"And that's why these"},{"speaker":"Emily M. Bender","startTime":1705.509,"endTime":1706.739,"body":"ones are getting bigger."},{"speaker":"Emily M. Bender","startTime":1706.749,"endTime":1707.819,"body":"Okay, that makes some sense."},{"speaker":"Emily M. Bender","startTime":1708.199,"endTime":1710.839,"body":"I'm kind of curious how often"},{"speaker":"Emily M. Bender","startTime":1708.199,"endTime":1710.839,"body":"o3 gave back something that"},{"speaker":"Emily M. Bender","startTime":1710.839,"endTime":1711.819,"body":"wasn't even the right format."},{"speaker":"Emily M. Bender","startTime":1712.334,"endTime":1714.534,"body":"But that's a separate question."},{"speaker":"Emily M. Bender","startTime":1715.994,"endTime":1719.484,"body":"So should we shift on over"},{"speaker":"Emily M. Bender","startTime":1715.994,"endTime":1719.484,"body":"to Sam Altman's reflections?"},{"speaker":"Alex Hanna","startTime":1719.514,"endTime":1720.814,"body":"Ugh, if we must."},{"speaker":"Emily M. Bender","startTime":1722.814,"endTime":1727.754,"body":"So this is a blog,"},{"speaker":"Emily M. Bender","startTime":1722.814,"endTime":1727.754,"body":"um, on, so it's Blog.SamAltman.Com."},{"speaker":"Emily M. Bender","startTime":1727.974,"endTime":1730.634,"body":"And this is dated January 5th of 2025."},{"speaker":"Emily M. Bender","startTime":1730.974,"endTime":1734.034,"body":"And the title is just, \"Reflections,\""},{"speaker":"Emily M. Bender","startTime":1730.974,"endTime":1734.034,"body":"which kind of made me wonder,"},{"speaker":"Emily M. Bender","startTime":1734.034,"endTime":1737.674,"body":"like, does he reuse titles or does"},{"speaker":"Emily M. Bender","startTime":1734.034,"endTime":1737.674,"body":"he just not reflect that often?"},{"speaker":"Alex Hanna","startTime":1738.904,"endTime":1739.934,"body":"It'd be very ironic."},{"speaker":"Emily M. Bender","startTime":1741.384,"endTime":1741.714,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1741.764,"endTime":1743.144,"body":"So you want to take us"},{"speaker":"Emily M. Bender","startTime":1741.764,"endTime":1743.144,"body":"into this one, Alex?"},{"speaker":"Alex Hanna","startTime":1743.544,"endTime":1745.164,"body":"Yeah, let's see."},{"speaker":"Alex Hanna","startTime":1745.164,"endTime":1748.014,"body":"What's the ch--what's the,"},{"speaker":"Alex Hanna","startTime":1745.164,"endTime":1748.014,"body":"what's the chonk of this?"},{"speaker":"Alex Hanna","startTime":1748.074,"endTime":1751.384,"body":"So the first paragraph, he says,"},{"speaker":"Alex Hanna","startTime":1748.074,"endTime":1751.384,"body":"\"The second birthday of ChatGPT"},{"speaker":"Alex Hanna","startTime":1751.784,"endTime":1754.104,"body":"was only a little over a month ago."},{"speaker":"Alex Hanna","startTime":1754.144,"endTime":1756.654,"body":"And now we have transitioned"},{"speaker":"Alex Hanna","startTime":1754.144,"endTime":1756.654,"body":"into the next paradigm of models"},{"speaker":"Alex Hanna","startTime":1756.654,"endTime":1758.054,"body":"that can do complex reasoning.\""},{"speaker":"Alex Hanna","startTime":1758.874,"endTime":1768.954,"body":"Um, and this I'm just assuming that he's,"},{"speaker":"Alex Hanna","startTime":1758.874,"endTime":1768.954,"body":"uh, referencing, um, o3 and also GPT-4o."},{"speaker":"Alex Hanna","startTime":1769.134,"endTime":1772.704,"body":"\"New years get people"},{"speaker":"Alex Hanna","startTime":1769.134,"endTime":1772.704,"body":"in a reflective mood."},{"speaker":"Alex Hanna","startTime":1772.724,"endTime":1777.234,"body":"And I want to share some personal thoughts"},{"speaker":"Alex Hanna","startTime":1772.724,"endTime":1777.234,"body":"about how it has gone so far and some of"},{"speaker":"Alex Hanna","startTime":1777.304,"endTime":1779.644,"body":"the things I've learned along the way.\""},{"speaker":"Alex Hanna","startTime":1780.674,"endTime":1781.604,"body":"Ah, okay."},{"speaker":"Emily M. Bender","startTime":1782.604,"endTime":1784.344,"body":"\"As"},{"speaker":"Emily M. Bender","startTime":1782.604,"endTime":1784.344,"body":"we get closer to AGI--\""},{"speaker":"Emily M. Bender","startTime":1785.075,"endTime":1788.225,"body":"There's some serious presuppositions"},{"speaker":"Emily M. Bender","startTime":1785.075,"endTime":1788.225,"body":"going on in that phrase, that AGI is"},{"speaker":"Emily M. Bender","startTime":1788.225,"endTime":1791.395,"body":"a thing, that we're getting closer"},{"speaker":"Emily M. Bender","startTime":1788.225,"endTime":1791.395,"body":"to it, but he's just sort of sticking"},{"speaker":"Emily M. Bender","startTime":1791.395,"endTime":1793.535,"body":"in there as a presupposition, so"},{"speaker":"Emily M. Bender","startTime":1791.395,"endTime":1793.535,"body":"you got to go along with it, right?"},{"speaker":"Emily M. Bender","startTime":1794.255,"endTime":1797.565,"body":"\"As we get closer to AGI, it feels"},{"speaker":"Emily M. Bender","startTime":1794.255,"endTime":1797.565,"body":"like an important time to look"},{"speaker":"Emily M. Bender","startTime":1797.575,"endTime":1799.344,"body":"at the progress of our company."},{"speaker":"Emily M. Bender","startTime":1799.595,"endTime":1802.474,"body":"There is still so much to understand,"},{"speaker":"Emily M. Bender","startTime":1799.595,"endTime":1802.474,"body":"still so much we don't know, and it's"},{"speaker":"Emily M. Bender","startTime":1802.475,"endTime":1806.215,"body":"still so early, but we know a lot"},{"speaker":"Emily M. Bender","startTime":1802.475,"endTime":1806.215,"body":"more than we did when we started.\""},{"speaker":"Alex Hanna","startTime":1807.105,"endTime":1809.905,"body":"Yeah, I want to, and"},{"speaker":"Alex Hanna","startTime":1807.105,"endTime":1809.905,"body":"let's discuss, but I really want"},{"speaker":"Alex Hanna","startTime":1809.905,"endTime":1811.125,"body":"to read this next paragraph."},{"speaker":"Alex Hanna","startTime":1811.625,"endTime":1816.025,"body":"So, \"We started OpenAI almost nine years"},{"speaker":"Alex Hanna","startTime":1811.625,"endTime":1816.025,"body":"ago because we believe that AGI was"},{"speaker":"Alex Hanna","startTime":1816.035,"endTime":1820.545,"body":"possible and that it could be the most"},{"speaker":"Alex Hanna","startTime":1816.035,"endTime":1820.545,"body":"impactful technology in human history."},{"speaker":"Alex Hanna","startTime":1820.975,"endTime":1823.514,"body":"We wanted to figure out how to"},{"speaker":"Alex Hanna","startTime":1820.975,"endTime":1823.514,"body":"build it and make it broadly"},{"speaker":"Alex Hanna","startTime":1823.515,"endTime":1828.024,"body":"beneficial; we were excited to"},{"speaker":"Alex Hanna","startTime":1823.515,"endTime":1828.024,"body":"try to make our mark on history."},{"speaker":"Alex Hanna","startTime":1828.514,"endTime":1830.354,"body":"Our ambitions were extraordinarily high."},{"speaker":"Alex Hanna","startTime":1830.73,"endTime":1833.64,"body":"And so is our belief that the"},{"speaker":"Alex Hanna","startTime":1830.73,"endTime":1833.64,"body":"work might benefit society in"},{"speaker":"Alex Hanna","startTime":1833.64,"endTime":1835.17,"body":"an equally extraordinary way.\""},{"speaker":"Alex Hanna","startTime":1835.64,"endTime":1839.89,"body":"And just like, man, talk about"},{"speaker":"Alex Hanna","startTime":1835.64,"endTime":1839.89,"body":"someone that is just completely"},{"speaker":"Alex Hanna","startTime":1840.3,"endTime":1843.13,"body":"disbelieves they're, they're the"},{"speaker":"Alex Hanna","startTime":1840.3,"endTime":1843.13,"body":"best thing since sliced bread."},{"speaker":"Alex Hanna","startTime":1843.159,"endTime":1845.719,"body":"Just, just absolutely nonsense."},{"speaker":"Emily M. Bender","startTime":1846.199,"endTime":1849.0,"body":"But meanwhile, we also"},{"speaker":"Emily M. Bender","startTime":1846.199,"endTime":1849.0,"body":"have the like, \"still so early,\" which"},{"speaker":"Emily M. Bender","startTime":1849.049,"endTime":1853.0,"body":"puts me in mind of Anna Lauren Hoffman's"},{"speaker":"Emily M. Bender","startTime":1849.049,"endTime":1853.0,"body":"analysis of this like infancy metaphor."},{"speaker":"Emily M. Bender","startTime":1854.945,"endTime":1855.785,"body":"Um, okay."},{"speaker":"Emily M. Bender","startTime":1856.385,"endTime":1859.425,"body":"So, uh, 'people didn't care cause they"},{"speaker":"Emily M. Bender","startTime":1856.385,"endTime":1859.425,"body":"thought we had no chance of success.'"},{"speaker":"Emily M. Bender","startTime":1859.705,"endTime":1865.135,"body":"And then, um, \"In 2022, OpenAI was a"},{"speaker":"Emily M. Bender","startTime":1859.705,"endTime":1865.135,"body":"quiet research lab working on something"},{"speaker":"Emily M. Bender","startTime":1865.135,"endTime":1867.815,"body":"temporarily called, 'Chat with GPT-3.5'.\""},{"speaker":"Emily M. Bender","startTime":1868.225,"endTime":1871.684,"body":"And then parenthetically, oh, so"},{"speaker":"Emily M. Bender","startTime":1868.225,"endTime":1871.684,"body":"humbly, \"We are much better at"},{"speaker":"Emily M. Bender","startTime":1871.684,"endTime":1873.175,"body":"research than we are at naming things.\""},{"speaker":"Emily M. Bender","startTime":1874.965,"endTime":1879.935,"body":"And I'm like, okay, maybe OpenAI wasn't"},{"speaker":"Emily M. Bender","startTime":1874.965,"endTime":1879.935,"body":"like a household topic of conversation"},{"speaker":"Emily M. Bender","startTime":1879.935,"endTime":1884.015,"body":"in 2022, but they were sure making"},{"speaker":"Emily M. Bender","startTime":1879.935,"endTime":1884.015,"body":"enough noise to annoy, like all of the"},{"speaker":"Emily M. Bender","startTime":1884.015,"endTime":1885.435,"body":"field of natural language processing."},{"speaker":"Emily M. Bender","startTime":1885.84,"endTime":1891.4,"body":"Like, so, you know, um, and yeah, so, um--"},{"speaker":"Alex Hanna","startTime":1892.35,"endTime":1895.32,"body":"they're also saying"},{"speaker":"Alex Hanna","startTime":1892.35,"endTime":1895.32,"body":"that we are much better at research"},{"speaker":"Alex Hanna","startTime":1895.32,"endTime":1896.39,"body":"than we are at naming things."},{"speaker":"Alex Hanna","startTime":1896.71,"endTime":1899.709,"body":"I mean, I would say you're not very"},{"speaker":"Alex Hanna","startTime":1896.71,"endTime":1899.709,"body":"good at research, but you know,"},{"speaker":"Emily M. Bender","startTime":1900.42,"endTime":1902.63,"body":"But you know,"},{"speaker":"Emily M. Bender","startTime":1900.42,"endTime":1902.63,"body":"also they did not run their name"},{"speaker":"Emily M. Bender","startTime":1902.639,"endTime":1904.049,"body":"past speakers of other languages."},{"speaker":"Emily M. Bender","startTime":1904.049,"endTime":1906.09,"body":"You know, the whole thing"},{"speaker":"Emily M. Bender","startTime":1904.049,"endTime":1906.09,"body":"about ChatGPT in French, right?"},{"speaker":"Alex Hanna","startTime":1906.35,"endTime":1907.04,"body":"No, I don't."},{"speaker":"Emily M. Bender","startTime":1907.5,"endTime":1908.8,"body":"You pronounce"},{"speaker":"Emily M. Bender","startTime":1907.5,"endTime":1908.8,"body":"that, in France it's \"Chat, j'ai"},{"speaker":"Emily M. Bender","startTime":1908.8,"endTime":1910.65,"body":"pété,\" which is \"Cat, I farted.\""},{"speaker":"Alex Hanna","startTime":1912.59,"endTime":1914.89,"body":"I say this as"},{"speaker":"Alex Hanna","startTime":1912.59,"endTime":1914.89,"body":"I'm holding one of my cats."},{"speaker":"Alex Hanna","startTime":1914.89,"endTime":1917.025,"body":"And hopefully she does not fart on me."},{"speaker":"Emily M. Bender","startTime":1917.755,"endTime":1921.765,"body":"Okay, then, uh, \"We"},{"speaker":"Emily M. Bender","startTime":1917.755,"endTime":1921.765,"body":"always knew abstractly that at some point"},{"speaker":"Emily M. Bender","startTime":1921.765,"endTime":1925.525,"body":"we would hit a tipping point and the AI"},{"speaker":"Emily M. Bender","startTime":1921.765,"endTime":1925.525,"body":"revolution would get kicked off, but we"},{"speaker":"Emily M. Bender","startTime":1925.525,"endTime":1927.035,"body":"didn't know what that moment would be."},{"speaker":"Emily M. Bender","startTime":1927.364,"endTime":1929.784,"body":"To our surprise, it turned out to"},{"speaker":"Emily M. Bender","startTime":1927.364,"endTime":1929.784,"body":"be this--\" And this is the launch"},{"speaker":"Emily M. Bender","startTime":1929.784,"endTime":1932.995,"body":"of ChatGPT, and it's like, no?"},{"speaker":"Emily M. Bender","startTime":1933.345,"endTime":1937.93,"body":"I mean, certainly, what changed in the"},{"speaker":"Emily M. Bender","startTime":1933.345,"endTime":1937.93,"body":"world with the release of ChatGPT was a"},{"speaker":"Emily M. Bender","startTime":1937.93,"endTime":1941.11,"body":"tsunami of AI hype and everyone trying"},{"speaker":"Emily M. Bender","startTime":1937.93,"endTime":1941.11,"body":"to get on board and all that kind of"},{"speaker":"Emily M. Bender","startTime":1941.11,"endTime":1946.17,"body":"stuff, but it's not the technological"},{"speaker":"Emily M. Bender","startTime":1941.11,"endTime":1946.17,"body":"revolution that they're imagining."},{"speaker":"Alex Hanna","startTime":1946.38,"endTime":1947.94,"body":"Yeah, that's,"},{"speaker":"Alex Hanna","startTime":1946.38,"endTime":1947.94,"body":"that's, that's right."},{"speaker":"Alex Hanna","startTime":1947.94,"endTime":1953.369,"body":"I mean, the kind of what you did is"},{"speaker":"Alex Hanna","startTime":1947.94,"endTime":1953.369,"body":"that you had a public interface and then"},{"speaker":"Alex Hanna","startTime":1953.92,"endTime":1962.68,"body":"hyped it up incredibly lots and then"},{"speaker":"Alex Hanna","startTime":1953.92,"endTime":1962.68,"body":"made teaching a lot harder and were the"},{"speaker":"Alex Hanna","startTime":1962.68,"endTime":1964.59,"body":"bane of every instructor's existence."},{"speaker":"Emily M. Bender","startTime":1965.5,"endTime":1965.81,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":1965.87,"endTime":1968.9,"body":"And lots and lots of other things"},{"speaker":"Emily M. Bender","startTime":1965.87,"endTime":1968.9,"body":"too, as we document in our book."},{"speaker":"Alex Hanna","startTime":1969.15,"endTime":1969.49,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":1970.23,"endTime":1970.71,"body":"Yes."},{"speaker":"Emily M. Bender","startTime":1970.71,"endTime":1973.59,"body":"Medusa Skirt says,"},{"speaker":"Emily M. Bender","startTime":1970.71,"endTime":1973.59,"body":"\"Truly the writing of a man who"},{"speaker":"Emily M. Bender","startTime":1973.619,"endTime":1976.419,"body":"really wants you to think he's the"},{"speaker":"Emily M. Bender","startTime":1973.619,"endTime":1976.419,"body":"Oppenheimer of machine learning.\""},{"speaker":"Alex Hanna","startTime":1978.015,"endTime":1981.255,"body":"Oh, it also reminds"},{"speaker":"Alex Hanna","startTime":1978.015,"endTime":1981.255,"body":"me of this, I don't know if"},{"speaker":"Alex Hanna","startTime":1981.255,"endTime":1982.475,"body":"we covered it on the podcast."},{"speaker":"Alex Hanna","startTime":1982.475,"endTime":1988.495,"body":"It was, but it was the, one of the people"},{"speaker":"Alex Hanna","startTime":1982.475,"endTime":1988.495,"body":"who was, um, developing like weapons"},{"speaker":"Alex Hanna","startTime":1988.495,"endTime":1992.444,"body":"and using AI for weapons, and they"},{"speaker":"Alex Hanna","startTime":1988.495,"endTime":1992.444,"body":"were saying, oh, I'm like Oppenheimer."},{"speaker":"Alex Hanna","startTime":1992.444,"endTime":1993.215,"body":"This is great."},{"speaker":"Alex Hanna","startTime":1994.635,"endTime":1995.92,"body":"I'm like, that's, what?"},{"speaker":"Alex Hanna","startTime":1996.77,"endTime":1998.23,"body":"Are we watching the same movie?"},{"speaker":"Alex Hanna","startTime":1998.26,"endTime":1998.65,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":1999.01,"endTime":2000.29,"body":"I need to"},{"speaker":"Emily M. Bender","startTime":1999.01,"endTime":2000.29,"body":"know better than to take sips"},{"speaker":"Emily M. Bender","startTime":2000.29,"endTime":2001.44,"body":"of tea during this podcast."},{"speaker":"Alex Hanna","startTime":2001.62,"endTime":2001.86,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2001.86,"endTime":2002.33,"body":"Sorry."},{"speaker":"Alex Hanna","startTime":2002.37,"endTime":2003.42,"body":"I apologize."},{"speaker":"Emily M. Bender","startTime":2004.49,"endTime":2006.35,"body":"I avoided the"},{"speaker":"Emily M. Bender","startTime":2004.49,"endTime":2006.35,"body":"spit take, but just barely."},{"speaker":"Alex Hanna","startTime":2007.45,"endTime":2012.02,"body":"So there's a bunch in here"},{"speaker":"Alex Hanna","startTime":2007.45,"endTime":2012.02,"body":"basically about, you know, the, the palace"},{"speaker":"Alex Hanna","startTime":2012.27,"endTime":2015.17,"body":"intrigue, um, you know, about OpenAI."},{"speaker":"Alex Hanna","startTime":2015.52,"endTime":2020.83,"body":"Which, you know, is, is, is"},{"speaker":"Alex Hanna","startTime":2015.52,"endTime":2020.83,"body":"funny and sad and annoying."},{"speaker":"Alex Hanna","startTime":2021.21,"endTime":2024.9,"body":"Um, also give it, giving a"},{"speaker":"Alex Hanna","startTime":2021.21,"endTime":2024.9,"body":"shout to Karen Hao's book."},{"speaker":"Alex Hanna","startTime":2024.919,"endTime":2029.949,"body":"She's, she's coming out with"},{"speaker":"Alex Hanna","startTime":2024.919,"endTime":2029.949,"body":"a book called Empire of AI."},{"speaker":"Alex Hanna","startTime":2029.95,"endTime":2031.12,"body":"I gotta search this."},{"speaker":"Alex Hanna","startTime":2031.43,"endTime":2036.58,"body":"And I'm sorry to, um, sorry to Christie,"},{"speaker":"Alex Hanna","startTime":2031.43,"endTime":2036.58,"body":"clacky I just typed on my very clacking"},{"speaker":"Alex Hanna","startTime":2036.58,"endTime":2037.71,"body":"keyboard trying to search for this."},{"speaker":"Alex Hanna","startTime":2038.05,"endTime":2038.48,"body":"Uh, yeah."},{"speaker":"Alex Hanna","startTime":2038.48,"endTime":2039.75,"body":"\"Empire of AI.\""},{"speaker":"Alex Hanna","startTime":2041.255,"endTime":2046.345,"body":"Which is coming, gonna come out in"},{"speaker":"Alex Hanna","startTime":2041.255,"endTime":2046.345,"body":"July, but it's all about the, um,"},{"speaker":"Alex Hanna","startTime":2047.015,"endTime":2048.905,"body":"the kind of drama around OpenAI."},{"speaker":"Alex Hanna","startTime":2048.905,"endTime":2054.74,"body":"So, um, but he goes into this,"},{"speaker":"Alex Hanna","startTime":2048.905,"endTime":2054.74,"body":"talking about like how he got fired."},{"speaker":"Alex Hanna","startTime":2055.32,"endTime":2060.48,"body":"Um, he learned the importance of a board,"},{"speaker":"Alex Hanna","startTime":2055.32,"endTime":2060.48,"body":"a board with complex, \"with diverse"},{"speaker":"Alex Hanna","startTime":2060.49,"endTime":2064.27,"body":"viewpoints and broad experience in"},{"speaker":"Alex Hanna","startTime":2060.49,"endTime":2064.27,"body":"managing a complex set of challenges.\""},{"speaker":"Alex Hanna","startTime":2064.67,"endTime":2067.49,"body":"And \"good governance requires a"},{"speaker":"Alex Hanna","startTime":2064.67,"endTime":2067.49,"body":"lot of trust and credibility.\""},{"speaker":"Alex Hanna","startTime":2067.869,"endTime":2073.27,"body":"Um, meanwhile, you know, half"},{"speaker":"Alex Hanna","startTime":2067.869,"endTime":2073.27,"body":"of his board resigned, um, in"},{"speaker":"Alex Hanna","startTime":2073.28,"endTime":2074.93,"body":"the, in the intervening period."},{"speaker":"Emily M. Bender","startTime":2075.495,"endTime":2077.245,"body":"And this, this"},{"speaker":"Emily M. Bender","startTime":2075.495,"endTime":2077.245,"body":"next paragraph at the end of that"},{"speaker":"Emily M. Bender","startTime":2077.245,"endTime":2080.475,"body":"little section, which he's got set"},{"speaker":"Emily M. Bender","startTime":2077.245,"endTime":2080.475,"body":"off by horizontal rules, uh, \"We"},{"speaker":"Emily M. Bender","startTime":2080.475,"endTime":2083.715,"body":"all got back to the work in a more"},{"speaker":"Emily M. Bender","startTime":2080.475,"endTime":2083.715,"body":"cohesive and positive way, and I'm"},{"speaker":"Emily M. Bender","startTime":2083.725,"endTime":2085.335,"body":"very proud of our focus since then."},{"speaker":"Emily M. Bender","startTime":2085.584,"endTime":2088.325,"body":"We have done what is easily"},{"speaker":"Emily M. Bender","startTime":2085.584,"endTime":2088.325,"body":"some of our best research ever.\""},{"speaker":"Emily M. Bender","startTime":2088.685,"endTime":2091.595,"body":"And then the way he substantiates"},{"speaker":"Emily M. Bender","startTime":2088.685,"endTime":2091.595,"body":"that, next sentence, \"We grew from"},{"speaker":"Emily M. Bender","startTime":2091.595,"endTime":2094.665,"body":"about 100 million weekly active"},{"speaker":"Emily M. Bender","startTime":2091.595,"endTime":2094.665,"body":"users to more than 300 million.\""},{"speaker":"Emily M. Bender","startTime":2095.355,"endTime":2098.695,"body":"That's, that's not a sign"},{"speaker":"Emily M. Bender","startTime":2095.355,"endTime":2098.695,"body":"of doing good research."},{"speaker":"Emily M. Bender","startTime":2098.695,"endTime":2100.015,"body":"That's a sign of marketing."},{"speaker":"Alex Hanna","startTime":2100.345,"endTime":2100.735,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2100.775,"endTime":2104.764,"body":"Like the sign of just selling"},{"speaker":"Alex Hanna","startTime":2100.775,"endTime":2104.764,"body":"this and selling big enterprise"},{"speaker":"Alex Hanna","startTime":2104.765,"endTime":2106.055,"body":"contracts is what you're doing."},{"speaker":"Emily M. Bender","startTime":2106.335,"endTime":2106.715,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2106.885,"endTime":2107.145,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2107.804,"endTime":2109.045,"body":"Um, okay."},{"speaker":"Alex Hanna","startTime":2109.465,"endTime":2114.045,"body":"The last, the last part of this"},{"speaker":"Alex Hanna","startTime":2109.465,"endTime":2114.045,"body":"is really, really, really rough."},{"speaker":"Alex Hanna","startTime":2114.045,"endTime":2114.825,"body":"Do you want to kick it off?"},{"speaker":"Emily M. Bender","startTime":2114.975,"endTime":2115.355,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2116.285,"endTime":2120.675,"body":"\"Nine years ago, we really had no idea"},{"speaker":"Emily M. Bender","startTime":2116.285,"endTime":2120.675,"body":"what we were eventually going to become."},{"speaker":"Emily M. Bender","startTime":2120.695,"endTime":2122.395,"body":"Even now we only sort of know."},{"speaker":"Emily M. Bender","startTime":2122.585,"endTime":2126.585,"body":"AI development has taken many twists and"},{"speaker":"Emily M. Bender","startTime":2122.585,"endTime":2126.585,"body":"turns and we expect more in the future."},{"speaker":"Emily M. Bender","startTime":2127.115,"endTime":2129.465,"body":"Some of the twists have been"},{"speaker":"Emily M. Bender","startTime":2127.115,"endTime":2129.465,"body":"joyful, some have been hard."},{"speaker":"Emily M. Bender","startTime":2129.735,"endTime":2133.095,"body":"It's been fun watching a steady"},{"speaker":"Emily M. Bender","startTime":2129.735,"endTime":2133.095,"body":"stream of research miracles occur."},{"speaker":"Emily M. Bender","startTime":2133.475,"endTime":2136.384,"body":"A lot of naysayers have"},{"speaker":"Emily M. Bender","startTime":2133.475,"endTime":2136.384,"body":"become true believers.\""},{"speaker":"Alex Hanna","startTime":2140.475,"endTime":2141.265,"body":"That's what you want."},{"speaker":"Alex Hanna","startTime":2141.315,"endTime":2142.475,"body":"You want true believers."},{"speaker":"Emily M. Bender","startTime":2142.885,"endTime":2143.195,"body":"Right."},{"speaker":"Emily M. Bender","startTime":2143.225,"endTime":2144.995,"body":"Like that, that's a real sign of science."},{"speaker":"Emily M. Bender","startTime":2144.995,"endTime":2145.345,"body":"Isn't it?"},{"speaker":"Emily M. Bender","startTime":2147.225,"endTime":2150.035,"body":"Um, \"We've also seen some colleagues"},{"speaker":"Emily M. Bender","startTime":2147.225,"endTime":2150.035,"body":"split off and become competitors."},{"speaker":"Emily M. Bender","startTime":2150.095,"endTime":2153.554,"body":"Teams tend to turn over as they"},{"speaker":"Emily M. Bender","startTime":2150.095,"endTime":2153.554,"body":"scale and open AI scales really fast."},{"speaker":"Emily M. Bender","startTime":2153.755,"endTime":2155.224,"body":"I think some of this is unavoidable."},{"speaker":"Emily M. Bender","startTime":2155.235,"endTime":2158.065,"body":"Startups usually see a lot of"},{"speaker":"Emily M. Bender","startTime":2155.235,"endTime":2158.065,"body":"turnover at each new major level of"},{"speaker":"Emily M. Bender","startTime":2158.075,"endTime":2161.905,"body":"scale and open AI numbers go up by"},{"speaker":"Emily M. Bender","startTime":2158.075,"endTime":2161.905,"body":"orders of magnitude every few months."},{"speaker":"Emily M. Bender","startTime":2162.22,"endTime":2165.02,"body":"The last two years have been like"},{"speaker":"Emily M. Bender","startTime":2162.22,"endTime":2165.02,"body":"a decade at a normal company.\""},{"speaker":"Alex Hanna","startTime":2165.59,"endTime":2167.7,"body":"It's all that"},{"speaker":"Alex Hanna","startTime":2165.59,"endTime":2167.7,"body":"AI they've got in the sauce."},{"speaker":"Emily M. Bender","startTime":2170.6,"endTime":2172.399,"body":"\"When any"},{"speaker":"Emily M. Bender","startTime":2170.6,"endTime":2172.399,"body":"company grows and evolves so"},{"speaker":"Emily M. Bender","startTime":2172.399,"endTime":2174.13,"body":"fast, interests naturally diverge."},{"speaker":"Emily M. Bender","startTime":2174.16,"endTime":2177.14,"body":"And when any company in an important"},{"speaker":"Emily M. Bender","startTime":2174.16,"endTime":2177.14,"body":"industry is in the lead, lots of"},{"speaker":"Emily M. Bender","startTime":2177.149,"endTime":2179.809,"body":"people attack it for all sorts"},{"speaker":"Emily M. Bender","startTime":2177.149,"endTime":2179.809,"body":"of reasons, especially when they"},{"speaker":"Emily M. Bender","startTime":2179.809,"endTime":2180.929,"body":"are trying to compete with it.\""},{"speaker":"Emily M. Bender","startTime":2181.26,"endTime":2182.91,"body":"Oh, poor, poor OpenAI."},{"speaker":"Emily M. Bender","startTime":2183.23,"endTime":2184.97,"body":"They're just trying to be in the lead."},{"speaker":"Emily M. Bender","startTime":2185.14,"endTime":2186.39,"body":"They don't deserve all of that."},{"speaker":"Alex Hanna","startTime":2187.209,"endTime":2188.799,"body":"I know they"},{"speaker":"Alex Hanna","startTime":2187.209,"endTime":2188.799,"body":"don't deserve all this smoke."},{"speaker":"Alex Hanna","startTime":2189.2,"endTime":2191.45,"body":"And hey, the people that are attacking it."},{"speaker":"Alex Hanna","startTime":2191.87,"endTime":2194.95,"body":"It's just, it's just that, it's"},{"speaker":"Alex Hanna","startTime":2191.87,"endTime":2194.95,"body":"just that they just want to be us."},{"speaker":"Emily M. Bender","startTime":2195.229,"endTime":2195.58,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2195.759,"endTime":2196.279,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2196.54,"endTime":2196.85,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2196.859,"endTime":2197.675,"body":"Really rough."},{"speaker":"Alex Hanna","startTime":2198.039,"endTime":2201.489,"body":"Um, so the next part,"},{"speaker":"Alex Hanna","startTime":2198.039,"endTime":2201.489,"body":"\"Our vision won't change."},{"speaker":"Alex Hanna","startTime":2201.499,"endTime":2203.439,"body":"Our tactics will continue to evolve."},{"speaker":"Alex Hanna","startTime":2203.679,"endTime":2207.219,"body":"For example, when we started, we had no"},{"speaker":"Alex Hanna","startTime":2203.679,"endTime":2207.219,"body":"idea we would build a product company."},{"speaker":"Alex Hanna","startTime":2207.579,"endTime":2209.949,"body":"We thought we were just"},{"speaker":"Alex Hanna","startTime":2207.579,"endTime":2209.949,"body":"going to do great research."},{"speaker":"Alex Hanna","startTime":2210.239,"endTime":2213.329,"body":"We also had no idea we would need"},{"speaker":"Alex Hanna","startTime":2210.239,"endTime":2213.329,"body":"such a crazy amount of capital.\""},{"speaker":"Alex Hanna","startTime":2214.279,"endTime":2214.669,"body":"Sorry."},{"speaker":"Alex Hanna","startTime":2215.029,"endTime":2216.669,"body":"I didn't expect to cackle at this."},{"speaker":"Alex Hanna","startTime":2216.669,"endTime":2219.789,"body":"I swear I read this and I just"},{"speaker":"Alex Hanna","startTime":2216.669,"endTime":2219.789,"body":"thought, you know, and I just"},{"speaker":"Alex Hanna","startTime":2219.789,"endTime":2221.394,"body":"go back to the, what is it?"},{"speaker":"Alex Hanna","startTime":2221.394,"endTime":2225.644,"body":"The 7 trillion amounts or"},{"speaker":"Alex Hanna","startTime":2221.394,"endTime":2225.644,"body":"whatever he says they have needed."},{"speaker":"Emily M. Bender","startTime":2225.645,"endTime":2228.884,"body":"And also"},{"speaker":"Emily M. Bender","startTime":2225.645,"endTime":2228.884,"body":"this, \"we had no idea\"."},{"speaker":"Emily M. Bender","startTime":2228.924,"endTime":2232.874,"body":"Like we had Brian Merchant on here"},{"speaker":"Emily M. Bender","startTime":2228.924,"endTime":2232.874,"body":"two episodes ago calling BS on that."},{"speaker":"Alex Hanna","startTime":2232.934,"endTime":2233.364,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2233.434,"endTime":2233.794,"body":"Right."},{"speaker":"Alex Hanna","startTime":2233.844,"endTime":2234.084,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2234.084,"endTime":2238.309,"body":"That's not a, if you have someone"},{"speaker":"Alex Hanna","startTime":2234.084,"endTime":2238.309,"body":"like like Musk as an early investor."},{"speaker":"Alex Hanna","startTime":2238.329,"endTime":2238.579,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2238.579,"endTime":2242.559,"body":"You're going to make products and you're"},{"speaker":"Alex Hanna","startTime":2238.579,"endTime":2242.559,"body":"going to aim to make boatloads of money."},{"speaker":"Alex Hanna","startTime":2243.289,"endTime":2244.509,"body":"Um, that's the goal."},{"speaker":"Alex Hanna","startTime":2245.009,"endTime":2248.329,"body":"Uh, \"There are new things we have"},{"speaker":"Alex Hanna","startTime":2245.009,"endTime":2248.329,"body":"to go build now that we didn't"},{"speaker":"Alex Hanna","startTime":2248.369,"endTime":2249.639,"body":"understand a few years ago."},{"speaker":"Alex Hanna","startTime":2249.869,"endTime":2252.989,"body":"And there are new things in the"},{"speaker":"Alex Hanna","startTime":2249.869,"endTime":2252.989,"body":"future we can barely imagine now.\""},{"speaker":"Alex Hanna","startTime":2254.165,"endTime":2259.114,"body":"\"We are proud of our track record"},{"speaker":"Alex Hanna","startTime":2254.165,"endTime":2259.114,"body":"on research and deployment so far--\""},{"speaker":"Alex Hanna","startTime":2259.514,"endTime":2261.504,"body":"Um, well, that's, that's interesting."},{"speaker":"Alex Hanna","startTime":2261.754,"endTime":2264.544,"body":"\"--and are committed to continuing"},{"speaker":"Alex Hanna","startTime":2261.754,"endTime":2264.544,"body":"to advance our thinking on"},{"speaker":"Alex Hanna","startTime":2264.544,"endTime":2266.784,"body":"safety and benefits sharing.\""},{"speaker":"Alex Hanna","startTime":2268.334,"endTime":2271.974,"body":"\"We continue to believe that the best"},{"speaker":"Alex Hanna","startTime":2268.334,"endTime":2271.974,"body":"way to make an AI system safe is by"},{"speaker":"Alex Hanna","startTime":2272.064,"endTime":2276.794,"body":"iteratively and gradually releasing"},{"speaker":"Alex Hanna","startTime":2272.064,"endTime":2276.794,"body":"it into the world, Giving society"},{"speaker":"Alex Hanna","startTime":2276.794,"endTime":2281.724,"body":"time to adapt and co evolve with the"},{"speaker":"Alex Hanna","startTime":2276.794,"endTime":2281.724,"body":"technology, learning from experience, and"},{"speaker":"Alex Hanna","startTime":2281.754,"endTime":2283.634,"body":"continuing to make the technology safer.\""},{"speaker":"Alex Hanna","startTime":2284.174,"endTime":2286.954,"body":"Um, I'm going to finish the"},{"speaker":"Alex Hanna","startTime":2284.174,"endTime":2286.954,"body":"paragraph and then I want to"},{"speaker":"Alex Hanna","startTime":2286.954,"endTime":2288.294,"body":"come back to open the sentence."},{"speaker":"Emily M. Bender","startTime":2288.534,"endTime":2288.784,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2288.784,"endTime":2291.504,"body":"\"We believe the"},{"speaker":"Alex Hanna","startTime":2288.784,"endTime":2291.504,"body":"importance of being world leaders"},{"speaker":"Alex Hanna","startTime":2291.534,"endTime":2295.099,"body":"on safety and alignment research and"},{"speaker":"Alex Hanna","startTime":2291.534,"endTime":2295.099,"body":"in guiding research with feedback"},{"speaker":"Alex Hanna","startTime":2295.109,"endTime":2296.849,"body":"for real world applications.\""},{"speaker":"Emily M. Bender","startTime":2297.209,"endTime":2299.219,"body":"Excuse me, but"},{"speaker":"Emily M. Bender","startTime":2297.209,"endTime":2299.219,"body":"what real world applications have we"},{"speaker":"Emily M. Bender","startTime":2299.219,"endTime":2301.089,"body":"actually seen that are worth anything?"},{"speaker":"Alex Hanna","startTime":2301.119,"endTime":2301.549,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2301.609,"endTime":2305.199,"body":"I mean, you're not, you're not"},{"speaker":"Alex Hanna","startTime":2301.609,"endTime":2305.199,"body":"getting to real world applications."},{"speaker":"Alex Hanna","startTime":2305.199,"endTime":2310.879,"body":"You're infesting different products"},{"speaker":"Alex Hanna","startTime":2305.199,"endTime":2310.879,"body":"with, with more synthetic texts."},{"speaker":"Alex Hanna","startTime":2311.369,"endTime":2315.824,"body":"And then the thing that really"},{"speaker":"Alex Hanna","startTime":2311.369,"endTime":2315.824,"body":"like grinds my gears on this is the"},{"speaker":"Alex Hanna","startTime":2315.824,"endTime":2319.574,"body":"\"giving society time to adapt and"},{"speaker":"Alex Hanna","startTime":2315.824,"endTime":2319.574,"body":"cool evolve with the technology.\""},{"speaker":"Alex Hanna","startTime":2320.024,"endTime":2325.794,"body":"Like what kind of Asimovian, you"},{"speaker":"Alex Hanna","startTime":2320.024,"endTime":2325.794,"body":"know, Foundation bullshit is this?"},{"speaker":"Emily M. Bender","startTime":2325.854,"endTime":2327.739,"body":"And who"},{"speaker":"Emily M. Bender","startTime":2325.854,"endTime":2327.739,"body":"do you think you are?"},{"speaker":"Alex Hanna","startTime":2328.009,"endTime":2328.299,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2328.374,"endTime":2332.934,"body":"Who do, who the fuck do you think you"},{"speaker":"Alex Hanna","startTime":2328.374,"endTime":2332.934,"body":"are that you're like some kind of people"},{"speaker":"Alex Hanna","startTime":2332.934,"endTime":2338.564,"body":"who are doing this kind of really forward"},{"speaker":"Alex Hanna","startTime":2332.934,"endTime":2338.564,"body":"thinking or predictive sort of, you"},{"speaker":"Alex Hanna","startTime":2338.564,"endTime":2342.284,"body":"know, predictive sort of technological"},{"speaker":"Alex Hanna","startTime":2338.564,"endTime":2342.284,"body":"breakthroughs and that we're doing it"},{"speaker":"Alex Hanna","startTime":2342.284,"endTime":2344.859,"body":"iteratively to let society catch up."},{"speaker":"Alex Hanna","startTime":2345.179,"endTime":2346.419,"body":"And I'm like, no, you're not."},{"speaker":"Alex Hanna","startTime":2346.599,"endTime":2348.909,"body":"You're careening from here to there."},{"speaker":"Alex Hanna","startTime":2349.189,"endTime":2353.499,"body":"You're basically trying to skirt every"},{"speaker":"Alex Hanna","startTime":2349.189,"endTime":2353.499,"body":"regulatory constraint there is, even"},{"speaker":"Alex Hanna","startTime":2353.499,"endTime":2358.489,"body":"though there aren't that many, you're"},{"speaker":"Alex Hanna","startTime":2353.499,"endTime":2358.489,"body":"trying to buy up as much data as you can,"},{"speaker":"Alex Hanna","startTime":2358.519,"endTime":2360.509,"body":"and you're being as reckless as possible."},{"speaker":"Emily M. Bender","startTime":2360.519,"endTime":2362.749,"body":"And taking no"},{"speaker":"Emily M. Bender","startTime":2360.519,"endTime":2362.749,"body":"accountability for the damage that's"},{"speaker":"Emily M. Bender","startTime":2362.759,"endTime":2365.769,"body":"being done to the actual environment,"},{"speaker":"Emily M. Bender","startTime":2362.759,"endTime":2365.769,"body":"to the information ecosystem,"},{"speaker":"Emily M. Bender","startTime":2365.769,"endTime":2369.634,"body":"to all of these other industries"},{"speaker":"Emily M. Bender","startTime":2365.769,"endTime":2369.634,"body":"that are falling for the AI hype."},{"speaker":"Emily M. Bender","startTime":2369.944,"endTime":2373.134,"body":"But if it's framed as, well, society"},{"speaker":"Emily M. Bender","startTime":2369.944,"endTime":2373.134,"body":"gets to evolve because we're doing this"},{"speaker":"Emily M. Bender","startTime":2373.134,"endTime":2377.179,"body":"little by little, that's basically saying,"},{"speaker":"Emily M. Bender","startTime":2373.134,"endTime":2377.179,"body":"none of the hardships are our fault."},{"speaker":"Emily M. Bender","startTime":2377.609,"endTime":2379.319,"body":"None of the harms, it's not us."},{"speaker":"Emily M. Bender","startTime":2379.349,"endTime":2382.889,"body":"We're just taking the AGI, which was"},{"speaker":"Emily M. Bender","startTime":2379.349,"endTime":2382.889,"body":"going to happen anyway, and sort of"},{"speaker":"Emily M. Bender","startTime":2383.089,"endTime":2384.769,"body":"slowly putting it out into society."},{"speaker":"Alex Hanna","startTime":2385.249,"endTime":2385.569,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2385.649,"endTime":2387.309,"body":"It's, it's ridiculous."},{"speaker":"Alex Hanna","startTime":2387.379,"endTime":2391.829,"body":"Um, and I'm just very, it's"},{"speaker":"Alex Hanna","startTime":2387.379,"endTime":2391.829,"body":"just absolutely infuriating."},{"speaker":"Alex Hanna","startTime":2391.859,"endTime":2398.569,"body":"And there's just such a delusion from"},{"speaker":"Alex Hanna","startTime":2391.859,"endTime":2398.569,"body":"the head of the stinky fish, um, of what,"},{"speaker":"Alex Hanna","startTime":2398.889,"endTime":2400.469,"body":"you know, they think they're doing here."},{"speaker":"Emily M. Bender","startTime":2400.649,"endTime":2400.969,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2401.169,"endTime":2403.889,"body":"I, there's, there's a wonderful"},{"speaker":"Emily M. Bender","startTime":2401.169,"endTime":2403.889,"body":"discourse in the chat here about,"},{"speaker":"Emily M. Bender","startTime":2403.939,"endTime":2407.569,"body":"um, how we're talking about how"},{"speaker":"Emily M. Bender","startTime":2403.939,"endTime":2407.569,"body":"they don't want to be beat up."},{"speaker":"Emily M. Bender","startTime":2407.579,"endTime":2408.599,"body":"They're just trying to leave."},{"speaker":"Emily M. Bender","startTime":2408.629,"endTime":2411.829,"body":"So people in the chat, starting"},{"speaker":"Emily M. Bender","startTime":2408.629,"endTime":2411.829,"body":"with Steal Case says, \" Wow, they're"},{"speaker":"Emily M. Bender","startTime":2411.829,"endTime":2413.549,"body":"so widdle, just widdle boys.\""},{"speaker":"Emily M. Bender","startTime":2413.789,"endTime":2418.349,"body":"And then Abstract Tesseract comes in"},{"speaker":"Emily M. Bender","startTime":2413.789,"endTime":2418.349,"body":"with, \"$7 trillion smol bean,\" and Six"},{"speaker":"Emily M. Bender","startTime":2418.349,"endTime":2422.22,"body":"Pairs Of Feet, \"Still just a little"},{"speaker":"Emily M. Bender","startTime":2418.349,"endTime":2422.22,"body":"stand on the sidewalk selling LLM-onade.\""},{"speaker":"Alex Hanna","startTime":2423.31,"endTime":2423.74,"body":"Very good."},{"speaker":"Alex Hanna","startTime":2423.959,"endTime":2425.209,"body":"The chat's very good today."},{"speaker":"Alex Hanna","startTime":2425.229,"endTime":2425.914,"body":"Keep going."},{"speaker":"Alex Hanna","startTime":2425.915,"endTime":2427.505,"body":"Love it."},{"speaker":"Alex Hanna","startTime":2427.525,"endTime":2431.404,"body":"And, uh, C Graziul says, \"I was"},{"speaker":"Alex Hanna","startTime":2427.525,"endTime":2431.404,"body":"going to say Borg, resistance is"},{"speaker":"Alex Hanna","startTime":2431.404,"endTime":2433.655,"body":"futile, but very Foundation-y.\""},{"speaker":"Alex Hanna","startTime":2433.675,"endTime":2434.095,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2434.125,"endTime":2436.265,"body":"So, I mean, it's, yeah, it's, it's."},{"speaker":"Alex Hanna","startTime":2436.45,"endTime":2441.169,"body":"And then SJayLett says, \"Oh, he absolutely"},{"speaker":"Alex Hanna","startTime":2436.45,"endTime":2441.169,"body":"would love to imagine he's Selden.\""},{"speaker":"Emily M. Bender","startTime":2441.259,"endTime":2442.169,"body":"Oh, of course."},{"speaker":"Emily M. Bender","startTime":2442.459,"endTime":2442.85,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2443.529,"endTime":2443.92,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2444.2,"endTime":2446.96,"body":"Actually, not Selden in the"},{"speaker":"Emily M. Bender","startTime":2444.2,"endTime":2446.96,"body":"original Foundation, but Selden in"},{"speaker":"Emily M. Bender","startTime":2446.96,"endTime":2449.17,"body":"the pretty terrible TV adaptation"},{"speaker":"Emily M. Bender","startTime":2446.96,"endTime":2449.17,"body":"that we've got right now."},{"speaker":"Alex Hanna","startTime":2449.85,"endTime":2451.54,"body":"I haven't seen that."},{"speaker":"Alex Hanna","startTime":2451.54,"endTime":2455.0,"body":"People said, it's it's worth"},{"speaker":"Alex Hanna","startTime":2451.54,"endTime":2455.0,"body":"watching, but you don't like it?"},{"speaker":"Emily M. Bender","startTime":2455.16,"endTime":2457.49,"body":"Eh, parts"},{"speaker":"Emily M. Bender","startTime":2455.16,"endTime":2457.49,"body":"of it are, it's super slow."},{"speaker":"Alex Hanna","startTime":2458.08,"endTime":2458.35,"body":"Okay."},{"speaker":"Alex Hanna","startTime":2458.48,"endTime":2459.52,"body":"I also heard that."},{"speaker":"Alex Hanna","startTime":2459.62,"endTime":2459.87,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2459.92,"endTime":2460.2,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2461.0,"endTime":2463.4,"body":"No, I I think there's things that people"},{"speaker":"Emily M. Bender","startTime":2461.0,"endTime":2463.4,"body":"object to about it, which are not the"},{"speaker":"Emily M. Bender","startTime":2463.41,"endTime":2469.239,"body":"things that I object to about it but"},{"speaker":"Emily M. Bender","startTime":2463.41,"endTime":2469.239,"body":"anyway, um, okay, so uh we've got, I"},{"speaker":"Emily M. Bender","startTime":2469.24,"endTime":2471.894,"body":"think just a couple paragraphs left."},{"speaker":"Emily M. Bender","startTime":2471.924,"endTime":2475.865,"body":"So, \"We are now confident we"},{"speaker":"Emily M. Bender","startTime":2471.924,"endTime":2475.865,"body":"know how to build AGI as we have"},{"speaker":"Emily M. Bender","startTime":2475.874,"endTime":2476.974,"body":"traditionally understood it."},{"speaker":"Emily M. Bender","startTime":2477.054,"endTime":2480.784,"body":"We believe that in 2025, we may"},{"speaker":"Emily M. Bender","startTime":2477.054,"endTime":2480.784,"body":"see the first AI agents, quote,"},{"speaker":"Emily M. Bender","startTime":2480.914,"endTime":2484.865,"body":"'join the workforce' and materially"},{"speaker":"Emily M. Bender","startTime":2480.914,"endTime":2484.865,"body":"change the output of companies."},{"speaker":"Emily M. Bender","startTime":2485.074,"endTime":2488.744,"body":"We continue to believe that"},{"speaker":"Emily M. Bender","startTime":2485.074,"endTime":2488.744,"body":"iteratively putting great tools in"},{"speaker":"Emily M. Bender","startTime":2488.744,"endTime":2491.774,"body":"the hands of people leads to great"},{"speaker":"Emily M. Bender","startTime":2488.744,"endTime":2491.774,"body":"broadly distributed outcomes.\""},{"speaker":"Alex Hanna","startTime":2492.424,"endTime":2493.034,"body":"Oh Lord."},{"speaker":"Alex Hanna","startTime":2493.034,"endTime":2494.504,"body":"There's so much here."},{"speaker":"Alex Hanna","startTime":2494.854,"endTime":2495.734,"body":"Um, yeah."},{"speaker":"Alex Hanna","startTime":2495.734,"endTime":2499.174,"body":"Abstract Tesseract says, \"We are"},{"speaker":"Alex Hanna","startTime":2495.734,"endTime":2499.174,"body":"here for the glorious future."},{"speaker":"Alex Hanna","startTime":2499.174,"endTime":2501.764,"body":"Did anyone hear the ominous"},{"speaker":"Alex Hanna","startTime":2499.174,"endTime":2501.764,"body":"change in background music?\""},{"speaker":"Alex Hanna","startTime":2503.444,"endTime":2509.74,"body":"And, um, yeah, so it's, it's, it's"},{"speaker":"Alex Hanna","startTime":2503.444,"endTime":2509.74,"body":"just really um, absolutely bizarre."},{"speaker":"Alex Hanna","startTime":2510.04,"endTime":2511.88,"body":"And then AI agents."},{"speaker":"Alex Hanna","startTime":2511.88,"endTime":2514.759,"body":"We've already, we talked about"},{"speaker":"Alex Hanna","startTime":2511.88,"endTime":2514.759,"body":"this, I think, on the last podcast."},{"speaker":"Alex Hanna","startTime":2514.799,"endTime":2518.509,"body":"And when we were going through"},{"speaker":"Alex Hanna","startTime":2514.799,"endTime":2518.509,"body":"the, uh, the hell backlog of that"},{"speaker":"Alex Hanna","startTime":2518.99,"endTime":2524.449,"body":"weird San Francisco company that"},{"speaker":"Alex Hanna","startTime":2518.99,"endTime":2524.449,"body":"has like hire an AI employee."},{"speaker":"Alex Hanna","startTime":2524.869,"endTime":2526.924,"body":"So it's really giving, giving that."},{"speaker":"Alex Hanna","startTime":2527.564,"endTime":2532.024,"body":"And then 'materially change the output"},{"speaker":"Alex Hanna","startTime":2527.564,"endTime":2532.024,"body":"of companies.' I don't know what that"},{"speaker":"Alex Hanna","startTime":2532.954,"endTime":2538.554,"body":"signals or what--is itit likehe amounts"},{"speaker":"Alex Hanna","startTime":2532.954,"endTime":2538.554,"body":"of output, like just putting, you know,"},{"speaker":"Alex Hanna","startTime":2538.554,"endTime":2542.444,"body":"are they going to become more productive"},{"speaker":"Alex Hanna","startTime":2538.554,"endTime":2542.444,"body":"and kind of a later economic sense?"},{"speaker":"Alex Hanna","startTime":2542.454,"endTime":2547.064,"body":"Are they just, are they literally,"},{"speaker":"Alex Hanna","startTime":2542.454,"endTime":2547.064,"body":"is, is Microsoft now going to become a"},{"speaker":"Alex Hanna","startTime":2547.064,"endTime":2549.939,"body":"company that you know, generates ponchos."},{"speaker":"Alex Hanna","startTime":2549.969,"endTime":2550.409,"body":"I don't know."},{"speaker":"Alex Hanna","startTime":2550.409,"endTime":2553.459,"body":"Like what is, what is materially"},{"speaker":"Alex Hanna","startTime":2550.409,"endTime":2553.459,"body":"change the output of companies mean?"},{"speaker":"Emily M. Bender","startTime":2554.099,"endTime":2554.819,"body":"Yeah, yeah, exactly."},{"speaker":"Emily M. Bender","startTime":2554.969,"endTime":2559.009,"body":"I think it's looking at companies as"},{"speaker":"Emily M. Bender","startTime":2554.969,"endTime":2559.009,"body":"functions from, uh, yesterday's stock"},{"speaker":"Emily M. Bender","startTime":2559.009,"endTime":2560.649,"body":"prices to tomorrow's stock prices."},{"speaker":"Alex Hanna","startTime":2561.689,"endTime":2561.899,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2562.7,"endTime":2563.91,"body":"So, okay."},{"speaker":"Emily M. Bender","startTime":2563.91,"endTime":2564.7,"body":"But it gets worse."},{"speaker":"Alex Hanna","startTime":2564.96,"endTime":2565.61,"body":"Yeah, it does get worse."},{"speaker":"Emily M. Bender","startTime":2565.61,"endTime":2567.995,"body":"Because this was"},{"speaker":"Emily M. Bender","startTime":2565.61,"endTime":2567.995,"body":"\"confident we know how to build AGI"},{"speaker":"Emily M. Bender","startTime":2568.015,"endTime":2571.085,"body":"as we have traditionally understood"},{"speaker":"Emily M. Bender","startTime":2568.015,"endTime":2571.085,"body":"it,\" was previous paragraph."},{"speaker":"Emily M. Bender","startTime":2571.115,"endTime":2572.945,"body":"And now you want to do the"},{"speaker":"Emily M. Bender","startTime":2571.115,"endTime":2572.945,"body":"honors on this one, Alex?"},{"speaker":"Alex Hanna","startTime":2573.135,"endTime":2574.044,"body":"Oh, geez."},{"speaker":"Alex Hanna","startTime":2574.325,"endTime":2577.904,"body":"\"We are beginning to turn our aim"},{"speaker":"Alex Hanna","startTime":2574.325,"endTime":2577.904,"body":"beyond that to superintelligence"},{"speaker":"Alex Hanna","startTime":2577.965,"endTime":2579.594,"body":"in the true sense of the word.\""},{"speaker":"Alex Hanna","startTime":2579.835,"endTime":2581.735,"body":"What the fuck is the"},{"speaker":"Alex Hanna","startTime":2579.835,"endTime":2581.735,"body":"true sense of the word?"},{"speaker":"Alex Hanna","startTime":2581.744,"endTime":2583.304,"body":"We don't have, that doesn't mean anything."},{"speaker":"Emily M. Bender","startTime":2583.545,"endTime":2584.545,"body":"But"},{"speaker":"Emily M. Bender","startTime":2583.545,"endTime":2584.545,"body":"definitions are important."},{"speaker":"Emily M. Bender","startTime":2584.995,"endTime":2586.335,"body":"The ARC Prize people taught us that."},{"speaker":"Alex Hanna","startTime":2586.415,"endTime":2588.555,"body":"I know, but,"},{"speaker":"Alex Hanna","startTime":2586.415,"endTime":2588.555,"body":"but Sam's not listening."},{"speaker":"Alex Hanna","startTime":2588.855,"endTime":2592.894,"body":"\"We love our current products, but"},{"speaker":"Alex Hanna","startTime":2588.855,"endTime":2592.894,"body":"we are here for the glorious future.\""},{"speaker":"Alex Hanna","startTime":2593.13,"endTime":2596.41,"body":"And here's the, oh, I thought glorious"},{"speaker":"Alex Hanna","startTime":2593.13,"endTime":2596.41,"body":"future was in the prior, uh, graf."},{"speaker":"Alex Hanna","startTime":2596.43,"endTime":2597.64,"body":"But no, here, here it is."},{"speaker":"Alex Hanna","startTime":2597.84,"endTime":2600.23,"body":"The glorious future, Dear Leader says."},{"speaker":"Alex Hanna","startTime":2600.63,"endTime":2603.79,"body":"\"With superintelligence,"},{"speaker":"Alex Hanna","startTime":2600.63,"endTime":2603.79,"body":"we can do anything else."},{"speaker":"Alex Hanna","startTime":2604.47,"endTime":2608.559,"body":"Superintelligent tools could massively"},{"speaker":"Alex Hanna","startTime":2604.47,"endTime":2608.559,"body":"accelerate scientific discovery and"},{"speaker":"Alex Hanna","startTime":2608.559,"endTime":2613.83,"body":"innovation well beyond what we are capable"},{"speaker":"Alex Hanna","startTime":2608.559,"endTime":2613.83,"body":"of doing on our own and in turn, massively"},{"speaker":"Alex Hanna","startTime":2613.84,"endTime":2615.57,"body":"increase abundance and prosperity.\""},{"speaker":"Alex Hanna","startTime":2615.94,"endTime":2620.799,"body":"So we're back here to this idea of"},{"speaker":"Alex Hanna","startTime":2615.94,"endTime":2620.799,"body":"science as being this black box, this"},{"speaker":"Alex Hanna","startTime":2620.799,"endTime":2624.84,"body":"kind of notion, this thing that we've"},{"speaker":"Alex Hanna","startTime":2620.799,"endTime":2624.84,"body":"gotten from, uh, Sakura, Sakura?"},{"speaker":"Alex Hanna","startTime":2624.87,"endTime":2625.15,"body":"No."},{"speaker":"Emily M. Bender","startTime":2625.43,"endTime":2626.18,"body":"Sakana."},{"speaker":"Alex Hanna","startTime":2626.259,"endTime":2626.749,"body":"Sakana."},{"speaker":"Alex Hanna","startTime":2626.75,"endTime":2627.24,"body":"Thank you."},{"speaker":"Alex Hanna","startTime":2627.35,"endTime":2630.415,"body":"Of course, uh, confusing it"},{"speaker":"Alex Hanna","startTime":2627.35,"endTime":2630.415,"body":"with another Japanese word."},{"speaker":"Alex Hanna","startTime":2630.745,"endTime":2635.605,"body":"And, um, and then, uh, the, uh,"},{"speaker":"Alex Hanna","startTime":2630.745,"endTime":2635.605,"body":"the Nobel, um, Turing Prize."},{"speaker":"Alex Hanna","startTime":2636.135,"endTime":2640.864,"body":"Um, and these kinds, I feel that,"},{"speaker":"Alex Hanna","startTime":2636.135,"endTime":2640.864,"body":"like, science is this thing that you"},{"speaker":"Alex Hanna","startTime":2640.875,"endTime":2645.734,"body":"just do more thinking, or you execute"},{"speaker":"Alex Hanna","startTime":2640.875,"endTime":2645.734,"body":"more tasks, and then, brrr, you know,"},{"speaker":"Alex Hanna","startTime":2645.745,"endTime":2648.065,"body":"more economic productivity happens."},{"speaker":"Alex Hanna","startTime":2648.114,"endTime":2652.545,"body":"And it's just, ugh, just absolute,"},{"speaker":"Alex Hanna","startTime":2648.114,"endTime":2652.545,"body":"this is, this is the, the"},{"speaker":"Alex Hanna","startTime":2652.565,"endTime":2654.195,"body":"bullshit of the highest degree."},{"speaker":"Emily M. Bender","startTime":2654.865,"endTime":2655.325,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2655.445,"endTime":2658.785,"body":"Uh, so last paragraph, right."},{"speaker":"Emily M. Bender","startTime":2659.115,"endTime":2660.795,"body":"Oh, and I want to say, you know"},{"speaker":"Emily M. Bender","startTime":2659.115,"endTime":2660.795,"body":"what would actually increase"},{"speaker":"Emily M. Bender","startTime":2660.795,"endTime":2661.935,"body":"abundance and prosperity?"},{"speaker":"Emily M. Bender","startTime":2662.935,"endTime":2665.045,"body":"Doing something about the climate crisis."},{"speaker":"Alex Hanna","startTime":2665.325,"endTime":2665.735,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2666.855,"endTime":2667.274,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2667.274,"endTime":2672.644,"body":"Well, also the idea that like you,"},{"speaker":"Alex Hanna","startTime":2667.274,"endTime":2672.644,"body":"as if this is an aspect of, you"},{"speaker":"Alex Hanna","startTime":2672.645,"endTime":2676.905,"body":"know, we're, we're recording this the"},{"speaker":"Alex Hanna","startTime":2672.645,"endTime":2676.905,"body":"second week, I already said the second"},{"speaker":"Alex Hanna","startTime":2676.905,"endTime":2679.475,"body":"week of January and at this moment."},{"speaker":"Alex Hanna","startTime":2679.485,"endTime":2679.505,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":2680.065,"endTime":2684.095,"body":"You know, we're seeing the massive"},{"speaker":"Alex Hanna","startTime":2680.065,"endTime":2684.095,"body":"devastation in LA County and the"},{"speaker":"Alex Hanna","startTime":2684.095,"endTime":2686.155,"body":"Palisades and the Eaton fire."},{"speaker":"Alex Hanna","startTime":2686.605,"endTime":2691.335,"body":"Just mountains, just mounds"},{"speaker":"Alex Hanna","startTime":2686.605,"endTime":2691.335,"body":"and mounds of people displaced."},{"speaker":"Alex Hanna","startTime":2692.535,"endTime":2699.465,"body":"And it's not like this stuff and"},{"speaker":"Alex Hanna","startTime":2692.535,"endTime":2699.465,"body":"climate refugees weren't being made."},{"speaker":"Alex Hanna","startTime":2700.035,"endTime":2707.435,"body":"Uh, but it's now we're seeing this"},{"speaker":"Alex Hanna","startTime":2700.035,"endTime":2707.435,"body":"massive thing in the US and you know,"},{"speaker":"Alex Hanna","startTime":2707.445,"endTime":2712.534,"body":"what we're also seeing is landlords now"},{"speaker":"Alex Hanna","startTime":2707.445,"endTime":2712.534,"body":"using it as an opportunity to extract"},{"speaker":"Alex Hanna","startTime":2712.605,"endTime":2718.044,"body":"even higher rents from people fleeing"},{"speaker":"Alex Hanna","startTime":2712.605,"endTime":2718.044,"body":"LA County, um, in the surrounding area."},{"speaker":"Alex Hanna","startTime":2718.065,"endTime":2723.485,"body":"We're seeing, uh, insurers drop people"},{"speaker":"Alex Hanna","startTime":2718.065,"endTime":2723.485,"body":"right before their homes had burned."},{"speaker":"Alex Hanna","startTime":2723.695,"endTime":2730.15,"body":"We're seeing this, um, like massive"},{"speaker":"Alex Hanna","startTime":2723.695,"endTime":2730.15,"body":"destruction of people's personal"},{"speaker":"Alex Hanna","startTime":2730.15,"endTime":2732.82,"body":"property and huge state failure."},{"speaker":"Alex Hanna","startTime":2733.28,"endTime":2737.27,"body":"And I'm just like, in what"},{"speaker":"Alex Hanna","startTime":2733.28,"endTime":2737.27,"body":"world would burning through all"},{"speaker":"Alex Hanna","startTime":2737.27,"endTime":2739.49,"body":"this carbon aid any of that?"},{"speaker":"Alex Hanna","startTime":2739.64,"endTime":2740.759,"body":"Increase abundance?"},{"speaker":"Alex Hanna","startTime":2740.759,"endTime":2745.939,"body":"We have the material necessary and it's"},{"speaker":"Alex Hanna","startTime":2740.759,"endTime":2745.939,"body":"a political, it's a distribution problem."},{"speaker":"Emily M. Bender","startTime":2746.169,"endTime":2747.149,"body":"Exactly, exactly."},{"speaker":"Emily M. Bender","startTime":2747.2,"endTime":2750.34,"body":"And it, abundance gets harder and"},{"speaker":"Emily M. Bender","startTime":2747.2,"endTime":2750.34,"body":"harder and harder, the more we"},{"speaker":"Emily M. Bender","startTime":2750.34,"endTime":2751.82,"body":"make the planet uninhabitable."},{"speaker":"Alex Hanna","startTime":2751.96,"endTime":2752.34,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2752.565,"endTime":2752.775,"body":"Right?"},{"speaker":"Emily M. Bender","startTime":2753.215,"endTime":2755.395,"body":"Okay, so last paragraph."},{"speaker":"Emily M. Bender","startTime":2755.455,"endTime":2757.715,"body":"\"This sounds like science"},{"speaker":"Emily M. Bender","startTime":2755.455,"endTime":2757.715,"body":"fiction right now and somewhat"},{"speaker":"Emily M. Bender","startTime":2757.715,"endTime":2758.975,"body":"crazy to even talk about it."},{"speaker":"Emily M. Bender","startTime":2759.345,"endTime":2759.945,"body":"That's all right."},{"speaker":"Emily M. Bender","startTime":2760.145,"endTime":2762.814,"body":"We've been there before and we're"},{"speaker":"Emily M. Bender","startTime":2760.145,"endTime":2762.814,"body":"okay with being there again."},{"speaker":"Emily M. Bender","startTime":2763.425,"endTime":2766.584,"body":"We're pretty confident that in the"},{"speaker":"Emily M. Bender","startTime":2763.425,"endTime":2766.584,"body":"next few years everyone will see what"},{"speaker":"Emily M. Bender","startTime":2766.584,"endTime":2771.124,"body":"we see and that the need to act with"},{"speaker":"Emily M. Bender","startTime":2766.584,"endTime":2771.124,"body":"great care while still maximizing broad"},{"speaker":"Emily M. Bender","startTime":2771.124,"endTime":2773.06,"body":"benefit and empowerment is so important."},{"speaker":"Emily M. Bender","startTime":2773.86,"endTime":2777.77,"body":"Given the possibilities of our work,"},{"speaker":"Emily M. Bender","startTime":2773.86,"endTime":2777.77,"body":"OpenAI cannot be a normal company.\""},{"speaker":"Emily M. Bender","startTime":2778.8,"endTime":2782.259,"body":"So Magidin the chat says, \"We've"},{"speaker":"Emily M. Bender","startTime":2778.8,"endTime":2782.259,"body":"been there before and we're"},{"speaker":"Emily M. Bender","startTime":2782.26,"endTime":2783.629,"body":"okay with being there again."},{"speaker":"Emily M. Bender","startTime":2783.649,"endTime":2785.77,"body":"Sounds like you aren't making"},{"speaker":"Emily M. Bender","startTime":2783.649,"endTime":2785.77,"body":"progress and you're okay with that."},{"speaker":"Emily M. Bender","startTime":2786.029,"endTime":2786.74,"body":"Nice to know.\""},{"speaker":"Emily M. Bender","startTime":2789.63,"endTime":2792.079,"body":"And then he ends with how lucky"},{"speaker":"Emily M. Bender","startTime":2789.63,"endTime":2792.079,"body":"and humbling it is to be able"},{"speaker":"Emily M. Bender","startTime":2792.079,"endTime":2793.62,"body":"to play a role in this work."},{"speaker":"Emily M. Bender","startTime":2793.99,"endTime":2797.265,"body":"I don't think Altman knows the"},{"speaker":"Emily M. Bender","startTime":2793.99,"endTime":2797.265,"body":"meaning of the word humble."},{"speaker":"Alex Hanna","startTime":2798.065,"endTime":2799.935,"body":"No, no, certainly not."},{"speaker":"Alex Hanna","startTime":2800.185,"endTime":2805.535,"body":"Every time, every time I think"},{"speaker":"Alex Hanna","startTime":2800.185,"endTime":2805.535,"body":"about the, you know, the kinds of"},{"speaker":"Alex Hanna","startTime":2805.715,"endTime":2811.204,"body":"illusions that he says, the, the"},{"speaker":"Alex Hanna","startTime":2805.715,"endTime":2811.204,"body":"magnanimousness of his company, I think"},{"speaker":"Alex Hanna","startTime":2811.314,"endTime":2818.265,"body":"of him driving that 2 million car um,"},{"speaker":"Alex Hanna","startTime":2811.314,"endTime":2818.265,"body":"around the streets of San Francisco."},{"speaker":"Alex Hanna","startTime":2818.335,"endTime":2821.795,"body":"Like oh, yes, you're very fe--oh"},{"speaker":"Alex Hanna","startTime":2818.335,"endTime":2821.795,"body":"he also says something here."},{"speaker":"Alex Hanna","startTime":2821.795,"endTime":2825.904,"body":"I think we we left it, but he was"},{"speaker":"Alex Hanna","startTime":2821.795,"endTime":2825.904,"body":"like, it's in the middle part, but he's"},{"speaker":"Alex Hanna","startTime":2825.904,"endTime":2832.074,"body":"like, you know sometimes I think--yeah"},{"speaker":"Alex Hanna","startTime":2825.904,"endTime":2832.074,"body":"this I love this paragraph and then"},{"speaker":"Alex Hanna","startTime":2832.074,"endTime":2833.635,"body":"after this we'll get into Hell."},{"speaker":"Alex Hanna","startTime":2833.895,"endTime":2837.055,"body":"He says, \"These years have been"},{"speaker":"Alex Hanna","startTime":2833.895,"endTime":2837.055,"body":"the most rewarding, fun, best,"},{"speaker":"Alex Hanna","startTime":2837.075,"endTime":2840.785,"body":"interesting, exhausting, stressful,"},{"speaker":"Alex Hanna","startTime":2837.075,"endTime":2840.785,"body":"and particularly for the last few,"},{"speaker":"Alex Hanna","startTime":2840.835,"endTime":2842.935,"body":"unpleasant years of my life so far."},{"speaker":"Alex Hanna","startTime":2844.01,"endTime":2846.34,"body":"The overwhelming feeling is gratitude."},{"speaker":"Alex Hanna","startTime":2846.69,"endTime":2850.93,"body":"I know that someday I'll be retired"},{"speaker":"Alex Hanna","startTime":2846.69,"endTime":2850.93,"body":"at our ranch watching the plants"},{"speaker":"Alex Hanna","startTime":2850.94,"endTime":2853.82,"body":"grow, a little bored, and we'll"},{"speaker":"Alex Hanna","startTime":2850.94,"endTime":2853.82,"body":"think back at how cool it was."},{"speaker":"Alex Hanna","startTime":2854.119,"endTime":2857.54,"body":"I got to do the work I've dreamed"},{"speaker":"Alex Hanna","startTime":2854.119,"endTime":2857.54,"body":"of since I was a little kid.\""},{"speaker":"Alex Hanna","startTime":2858.129,"endTime":2865.169,"body":"First off, only, only folks like you"},{"speaker":"Alex Hanna","startTime":2858.129,"endTime":2865.169,"body":"have a ranch, you know, like most"},{"speaker":"Alex Hanna","startTime":2865.169,"endTime":2867.459,"body":"of us don't have a ranch by far."},{"speaker":"Alex Hanna","startTime":2867.909,"endTime":2873.87,"body":"Uh, second off, you know, like my vision"},{"speaker":"Alex Hanna","startTime":2867.909,"endTime":2873.87,"body":"of what you're going to be doing 30"},{"speaker":"Alex Hanna","startTime":2873.87,"endTime":2880.19,"body":"years from now is probably hunkering"},{"speaker":"Alex Hanna","startTime":2873.87,"endTime":2880.19,"body":"down in your ranch because you've, uh,"},{"speaker":"Alex Hanna","startTime":2880.2,"endTime":2884.81,"body":"really sown and aided and abetted in"},{"speaker":"Alex Hanna","startTime":2880.2,"endTime":2884.81,"body":"the destruction of the natural world."},{"speaker":"Alex Hanna","startTime":2884.819,"endTime":2889.789,"body":"And you're just hiding away thinking"},{"speaker":"Alex Hanna","startTime":2884.819,"endTime":2889.789,"body":"that you could, you could avoid any"},{"speaker":"Alex Hanna","startTime":2889.789,"endTime":2893.805,"body":"kind of, you know, accountability"},{"speaker":"Alex Hanna","startTime":2889.789,"endTime":2893.805,"body":"that you should be encountering now."},{"speaker":"Emily M. Bender","startTime":2893.955,"endTime":2896.035,"body":"That ranch is"},{"speaker":"Emily M. Bender","startTime":2893.955,"endTime":2896.035,"body":"probably in New Zealand where all the"},{"speaker":"Emily M. Bender","startTime":2896.035,"endTime":2897.515,"body":"billionaires have bought up their escape."},{"speaker":"Emily M. Bender","startTime":2898.245,"endTime":2898.555,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":2899.034,"endTime":2899.384,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":2899.855,"endTime":2903.134,"body":"So are we doing musical or non"},{"speaker":"Emily M. Bender","startTime":2899.855,"endTime":2903.134,"body":"musical for the transition this time?"},{"speaker":"Alex Hanna","startTime":2903.194,"endTime":2905.515,"body":"No, I'm feeling, I'm"},{"speaker":"Alex Hanna","startTime":2903.194,"endTime":2905.515,"body":"feeling pretty musical today."},{"speaker":"Emily M. Bender","startTime":2905.824,"endTime":2906.355,"body":"All right."},{"speaker":"Emily M. Bender","startTime":2906.455,"endTime":2906.845,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":2906.864,"endTime":2909.125,"body":"So what's the, what's your,"},{"speaker":"Emily M. Bender","startTime":2906.864,"endTime":2909.125,"body":"what's your musical genre?"},{"speaker":"Alex Hanna","startTime":2909.845,"endTime":2912.055,"body":"Uh, well, we're"},{"speaker":"Alex Hanna","startTime":2909.845,"endTime":2912.055,"body":"talking about a ranch, so."},{"speaker":"Emily M. Bender","startTime":2914.075,"endTime":2914.605,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":2914.855,"endTime":2915.955,"body":"Um, so here's the deal."},{"speaker":"Emily M. Bender","startTime":2916.595,"endTime":2920.295,"body":"Uh, OpenAI and company have figured"},{"speaker":"Emily M. Bender","startTime":2916.595,"endTime":2920.295,"body":"out, you know how all of this stuff is"},{"speaker":"Emily M. Bender","startTime":2920.315,"endTime":2922.035,"body":"usually people in the background, right?"},{"speaker":"Emily M. Bender","startTime":2922.435,"endTime":2923.595,"body":"And they're shipping the tasks off."},{"speaker":"Emily M. Bender","startTime":2923.605,"endTime":2927.174,"body":"Well, they figured out how to ship those"},{"speaker":"Emily M. Bender","startTime":2923.605,"endTime":2927.174,"body":"tasks off to the demons of AI hell."},{"speaker":"Emily M. Bender","startTime":2927.795,"endTime":2933.204,"body":"And so you're there in the, you know,"},{"speaker":"Emily M. Bender","startTime":2927.795,"endTime":2933.204,"body":"the mines doing the, the, the, what"},{"speaker":"Emily M. Bender","startTime":2933.204,"endTime":2934.764,"body":"are they called on Mechanical Turk?"},{"speaker":"Emily M. Bender","startTime":2934.904,"endTime":2935.284,"body":"The human--"},{"speaker":"Alex Hanna","startTime":2935.375,"endTime":2936.605,"body":"Yeah, like that click work."},{"speaker":"Emily M. Bender","startTime":2936.775,"endTime":2937.415,"body":"Yeah, the click work."},{"speaker":"Emily M. Bender","startTime":2937.415,"endTime":2941.285,"body":"But, um, and then these ARC Prize"},{"speaker":"Emily M. Bender","startTime":2937.415,"endTime":2941.285,"body":"graphs start falling on you."},{"speaker":"Emily M. Bender","startTime":2941.875,"endTime":2943.615,"body":"And so you're going to give me a what?"},{"speaker":"Emily M. Bender","startTime":2944.085,"endTime":2947.675,"body":"Country music appropriate for"},{"speaker":"Emily M. Bender","startTime":2944.085,"endTime":2947.675,"body":"the ranch hand's song about that."},{"speaker":"Alex Hanna","startTime":2947.985,"endTime":2949.075,"body":"This is so deep."},{"speaker":"Alex Hanna","startTime":2949.765,"endTime":2950.405,"body":"All right."},{"speaker":"Alex Hanna","startTime":2950.415,"endTime":2951.385,"body":"So you're a demon."},{"speaker":"Alex Hanna","startTime":2951.385,"endTime":2954.035,"body":"You're like, all right,"},{"speaker":"Alex Hanna","startTime":2951.385,"endTime":2954.035,"body":"so let me think of it."},{"speaker":"Alex Hanna","startTime":2954.425,"endTime":2960.725,"body":"Down here in the content minds,"},{"speaker":"Alex Hanna","startTime":2954.425,"endTime":2960.725,"body":"storing matrices, putting them in"},{"speaker":"Alex Hanna","startTime":2960.725,"endTime":2964.854,"body":"line, doing tasks that only we can do."},{"speaker":"Alex Hanna","startTime":2965.855,"endTime":2967.922,"body":"Just so Sam can say he's smart too."},{"speaker":"Alex Hanna","startTime":2967.922,"endTime":2970.137,"body":"Winning the ARC Prize down in AI Hell."},{"speaker":"Alex Hanna","startTime":2971.027,"endTime":2976.197,"body":"Winning the ARC Prize, AGI's going swell."},{"speaker":"Alex Hanna","startTime":2976.357,"endTime":2977.797,"body":"Wining the"},{"speaker":"Alex Hanna","startTime":2980.177,"endTime":2981.377,"body":"ARC Prize."},{"speaker":"Alex Hanna","startTime":2981.647,"endTime":2986.31,"body":"We're doing it for humanity,"},{"speaker":"Alex Hanna","startTime":2981.647,"endTime":2986.31,"body":"but don't look too closely or"},{"speaker":"Alex Hanna","startTime":2986.31,"endTime":2989.099,"body":"you might not like what you see."},{"speaker":"Emily M. Bender","startTime":2990.16,"endTime":2992.04,"body":"You might not like"},{"speaker":"Emily M. Bender","startTime":2990.16,"endTime":2992.04,"body":"what you see, but I love the song."},{"speaker":"Alex Hanna","startTime":2993.53,"endTime":2993.9,"body":"(speaking) Thank you."},{"speaker":"Alex Hanna","startTime":2993.9,"endTime":2997.7,"body":"I think this is the first one in a while"},{"speaker":"Alex Hanna","startTime":2993.9,"endTime":2997.7,"body":"I just haven't completely bailed on."},{"speaker":"Emily M. Bender","startTime":2998.44,"endTime":2999.3,"body":"It went really well."},{"speaker":"Emily M. Bender","startTime":2999.97,"endTime":3000.39,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3000.39,"endTime":3002.02,"body":"So here we are in Fresh AI Hell."},{"speaker":"Emily M. Bender","startTime":3002.04,"endTime":3004.48,"body":"I've got an ambitious number, but most"},{"speaker":"Emily M. Bender","startTime":3002.04,"endTime":3004.48,"body":"of these, we agree, we're basically"},{"speaker":"Emily M. Bender","startTime":3004.48,"endTime":3005.65,"body":"just going to hit the headlines."},{"speaker":"Emily M. Bender","startTime":3006.22,"endTime":3009.48,"body":"So this is from Popular"},{"speaker":"Emily M. Bender","startTime":3006.22,"endTime":3009.48,"body":"Mechanics by Darren Orff"},{"speaker":"Emily M. Bender","startTime":3009.51,"endTime":3011.63,"body":"published November 30th of 2024."},{"speaker":"Emily M. Bender","startTime":3011.98,"endTime":3014.679,"body":"Um, headline, \"Humanity may"},{"speaker":"Emily M. Bender","startTime":3011.98,"endTime":3014.679,"body":"reach singularity within"},{"speaker":"Emily M. Bender","startTime":3014.68,"endTime":3017.08,"body":"just six years, trend shows.\""},{"speaker":"Alex Hanna","startTime":3017.309,"endTime":3020.58,"body":"'Trend shows' is doing"},{"speaker":"Alex Hanna","startTime":3017.309,"endTime":3020.58,"body":"a lot of work on this headline."},{"speaker":"Emily M. Bender","startTime":3020.92,"endTime":3021.5,"body":"Exactly."},{"speaker":"Emily M. Bender","startTime":3021.66,"endTime":3024.97,"body":"And so here, the, the main point I"},{"speaker":"Emily M. Bender","startTime":3021.66,"endTime":3024.97,"body":"wanted to bring up is in their summary,"},{"speaker":"Emily M. Bender","startTime":3025.37,"endTime":3030.35,"body":"\"A translation company developed a"},{"speaker":"Emily M. Bender","startTime":3025.37,"endTime":3030.35,"body":"metric, time to edit, TTE, to calculate"},{"speaker":"Emily M. Bender","startTime":3030.35,"endTime":3033.65,"body":"the time it takes for professional"},{"speaker":"Emily M. Bender","startTime":3030.35,"endTime":3033.65,"body":"human editors to fix AI generated"},{"speaker":"Emily M. Bender","startTime":3033.65,"endTime":3035.269,"body":"translations compared to human ones."},{"speaker":"Emily M. Bender","startTime":3035.51,"endTime":3038.54,"body":"This may help quantify the"},{"speaker":"Emily M. Bender","startTime":3035.51,"endTime":3038.54,"body":"speed towards singularity.\""},{"speaker":"Alex Hanna","startTime":3039.339,"endTime":3041.459,"body":"This is,"},{"speaker":"Alex Hanna","startTime":3039.339,"endTime":3041.459,"body":"this is absolutely wild."},{"speaker":"Alex Hanna","startTime":3041.59,"endTime":3042.529,"body":"Do they have a link?"},{"speaker":"Alex Hanna","startTime":3042.529,"endTime":3044.4,"body":"Like who's, what metric is this?"},{"speaker":"Emily M. Bender","startTime":3044.5,"endTime":3046.14,"body":"Uh, so."},{"speaker":"Alex Hanna","startTime":3046.25,"endTime":3049.3,"body":"Oh, it's a translation"},{"speaker":"Alex Hanna","startTime":3046.25,"endTime":3049.3,"body":"company, so they don't even have a paper,"},{"speaker":"Alex Hanna","startTime":3049.3,"endTime":3050.88,"body":"like an arXiv paper or anything, do they?"},{"speaker":"Emily M. Bender","startTime":3050.89,"endTime":3052.6,"body":"Yeah, it just, yeah."},{"speaker":"Emily M. Bender","startTime":3052.66,"endTime":3057.035,"body":"So basically the idea is, we can"},{"speaker":"Emily M. Bender","startTime":3052.66,"endTime":3057.035,"body":"see some kind of progress because"},{"speaker":"Emily M. Bender","startTime":3057.055,"endTime":3059.785,"body":"this translation system is getting"},{"speaker":"Emily M. Bender","startTime":3057.055,"endTime":3059.785,"body":"better and better on first go."},{"speaker":"Emily M. Bender","startTime":3060.015,"endTime":3064.045,"body":"So that change is going to tell us"},{"speaker":"Emily M. Bender","startTime":3060.015,"endTime":3064.045,"body":"how fast the singularity is coming."},{"speaker":"Emily M. Bender","startTime":3064.455,"endTime":3065.725,"body":"Because those are the same thing."},{"speaker":"Emily M. Bender","startTime":3066.105,"endTime":3071.735,"body":"And this, this comes back to the"},{"speaker":"Emily M. Bender","startTime":3066.105,"endTime":3071.735,"body":"problem of treating all of this as AI,"},{"speaker":"Emily M. Bender","startTime":3071.735,"endTime":3076.654,"body":"as one thing and like, including like"},{"speaker":"Emily M. Bender","startTime":3071.735,"endTime":3076.654,"body":"various technologies that actually"},{"speaker":"Emily M. Bender","startTime":3076.654,"endTime":3080.434,"body":"exist that work well or not on there"},{"speaker":"Emily M. Bender","startTime":3076.654,"endTime":3080.434,"body":"for specific tasks, and then like,"},{"speaker":"Emily M. Bender","startTime":3080.444,"endTime":3081.974,"body":"imagine things like the singularity."},{"speaker":"Alex Hanna","startTime":3082.304,"endTime":3082.754,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":3083.124,"endTime":3083.354,"body":"It's."},{"speaker":"Emily M. Bender","startTime":3083.729,"endTime":3083.949,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3083.949,"endTime":3084.299,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3084.39,"endTime":3086.56,"body":"Just incredible category error."},{"speaker":"Emily M. Bender","startTime":3086.899,"endTime":3087.209,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3087.359,"endTime":3088.139,"body":"Next."},{"speaker":"Emily M. Bender","startTime":3088.539,"endTime":3090.029,"body":"Um, you want to read that one headline?"},{"speaker":"Alex Hanna","startTime":3090.029,"endTime":3090.169,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3090.169,"endTime":3091.759,"body":"So this is from the BBC."},{"speaker":"Alex Hanna","startTime":3091.789,"endTime":3096.069,"body":"The, uh, the author is David"},{"speaker":"Alex Hanna","startTime":3091.789,"endTime":3096.069,"body":"Robson, December 23rd, 2024."},{"speaker":"Alex Hanna","startTime":3096.439,"endTime":3101.409,"body":"\"An AI started, quote, 'tasting"},{"speaker":"Alex Hanna","startTime":3096.439,"endTime":3101.409,"body":"colors and shapes.' That is more"},{"speaker":"Alex Hanna","startTime":3101.409,"endTime":3102.749,"body":"human than you might think.\""},{"speaker":"Alex Hanna","startTime":3103.449,"endTime":3106.239,"body":"And what, what is this"},{"speaker":"Alex Hanna","startTime":3103.449,"endTime":3106.239,"body":"actually about, the brain?"},{"speaker":"Alex Hanna","startTime":3106.249,"endTime":3110.479,"body":"So, \"The brain often blurs the senses,"},{"speaker":"Alex Hanna","startTime":3106.249,"endTime":3110.479,"body":"a fact that marketers often use in the"},{"speaker":"Alex Hanna","startTime":3110.479,"endTime":3115.069,"body":"design of food packaging and AIs--\""},{"speaker":"Alex Hanna","startTime":3110.479,"endTime":3115.069,"body":"Infuriating linguistic thing, uh,"},{"speaker":"Alex Hanna","startTime":3115.099,"endTime":3118.409,"body":"um, of using AIs as a, like a, um--"},{"speaker":"Emily M. Bender","startTime":3118.829,"endTime":3119.349,"body":"Common noun?"},{"speaker":"Emily M. Bender","startTime":3119.35,"endTime":3123.739,"body":"Not even a common noun,"},{"speaker":"Emily M. Bender","startTime":3119.35,"endTime":3123.739,"body":"but a, um, uh, count noun."},{"speaker":"Alex Hanna","startTime":3123.869,"endTime":3124.309,"body":"Count noun."},{"speaker":"Alex Hanna","startTime":3124.31,"endTime":3124.529,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3124.529,"endTime":3124.709,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3125.219,"endTime":3126.409,"body":"\"--AIs appear to do the same."},{"speaker":"Alex Hanna","startTime":3126.689,"endTime":3128.329,"body":"What is the flavor of a pink sphere?"},{"speaker":"Alex Hanna","startTime":3128.329,"endTime":3130.409,"body":"And what is the sound"},{"speaker":"Alex Hanna","startTime":3128.329,"endTime":3130.409,"body":"of a Sauvignon Blanc?\""},{"speaker":"Alex Hanna","startTime":3131.029,"endTime":3134.109,"body":"Um, and then, yeah."},{"speaker":"Alex Hanna","startTime":3134.159,"endTime":3135.889,"body":"So anyways, I don't know."},{"speaker":"Emily M. Bender","startTime":3136.069,"endTime":3136.289,"body":"Right."},{"speaker":"Emily M. Bender","startTime":3136.289,"endTime":3138.789,"body":"So, so basically I think what's going"},{"speaker":"Emily M. Bender","startTime":3136.289,"endTime":3138.789,"body":"on here is that the large language"},{"speaker":"Emily M. Bender","startTime":3138.789,"endTime":3141.629,"body":"model will spit out answers to those"},{"speaker":"Emily M. Bender","startTime":3138.789,"endTime":3141.629,"body":"questions because it's doing next token"},{"speaker":"Emily M. Bender","startTime":3141.629,"endTime":3144.044,"body":"prediction and someone got excited."},{"speaker":"Alex Hanna","startTime":3144.284,"endTime":3144.664,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3144.665,"endTime":3145.314,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3145.314,"endTime":3145.514,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3145.514,"endTime":3145.794,"body":"All right."},{"speaker":"Emily M. Bender","startTime":3146.644,"endTime":3150.774,"body":"So speaking of who asked for this,"},{"speaker":"Emily M. Bender","startTime":3146.644,"endTime":3150.774,"body":"this is something DHGate.Com."},{"speaker":"Emily M. Bender","startTime":3150.974,"endTime":3153.344,"body":"This was contributed by, by"},{"speaker":"Emily M. Bender","startTime":3150.974,"endTime":3153.344,"body":"the way, through our form where"},{"speaker":"Emily M. Bender","startTime":3153.344,"endTime":3154.854,"body":"people can send us Fresh AI Hell."},{"speaker":"Emily M. Bender","startTime":3155.144,"endTime":3159.164,"body":"And only one of these things that I pulled"},{"speaker":"Emily M. Bender","startTime":3155.144,"endTime":3159.164,"body":"up had a person's name associated with"},{"speaker":"Emily M. Bender","startTime":3159.164,"endTime":3160.384,"body":"it, like the person who contributed it."},{"speaker":"Emily M. Bender","startTime":3160.384,"endTime":3162.714,"body":"And that name was Warai Otoko."},{"speaker":"Emily M. Bender","startTime":3163.134,"endTime":3166.794,"body":"Which is, looks like someone's trying"},{"speaker":"Emily M. Bender","startTime":3163.134,"endTime":3166.794,"body":"to write 'funny guy' in Japanese."},{"speaker":"Emily M. Bender","startTime":3167.154,"endTime":3168.604,"body":"So I don't think it's really a name."},{"speaker":"Emily M. Bender","startTime":3168.824,"endTime":3171.694,"body":"If it is, I'm sorry, but that doesn't look"},{"speaker":"Emily M. Bender","startTime":3168.824,"endTime":3171.694,"body":"like a Japanese last name in the least."},{"speaker":"Emily M. Bender","startTime":3172.014,"endTime":3176.844,"body":"Um, anyway, so, um, this came in"},{"speaker":"Emily M. Bender","startTime":3172.014,"endTime":3176.844,"body":"through that form and we will post a"},{"speaker":"Emily M. Bender","startTime":3176.864,"endTime":3178.034,"body":"link to the form in the show notes."},{"speaker":"Emily M. Bender","startTime":3178.294,"endTime":3183.624,"body":"Um, but DHGate.Com, \"AI smart display"},{"speaker":"Emily M. Bender","startTime":3178.294,"endTime":3183.624,"body":"stainless steel kitchen faucet"},{"speaker":"Emily M. Bender","startTime":3183.624,"endTime":3185.174,"body":"with LED temperature display.\""},{"speaker":"Emily M. Bender","startTime":3186.914,"endTime":3187.374,"body":"What?"},{"speaker":"Emily M. Bender","startTime":3187.924,"endTime":3189.434,"body":"You just need to turn the water on."},{"speaker":"Alex Hanna","startTime":3189.444,"endTime":3191.894,"body":"Just turn, turn the"},{"speaker":"Alex Hanna","startTime":3189.444,"endTime":3191.894,"body":"water on and use the switch."},{"speaker":"Emily M. Bender","startTime":3192.124,"endTime":3192.324,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3192.574,"endTime":3194.824,"body":"And wait, wait, like,"},{"speaker":"Alex Hanna","startTime":3192.574,"endTime":3194.824,"body":"and it's doing your prediction."},{"speaker":"Alex Hanna","startTime":3194.824,"endTime":3199.244,"body":"Like, what if I'm doing the dishes with"},{"speaker":"Alex Hanna","startTime":3194.824,"endTime":3199.244,"body":"like really hot and then, and then I"},{"speaker":"Alex Hanna","startTime":3199.304,"endTime":3200.914,"body":"want to get a glass of water, anyways."},{"speaker":"Emily M. Bender","startTime":3200.984,"endTime":3202.544,"body":"I can't tell"},{"speaker":"Emily M. Bender","startTime":3200.984,"endTime":3202.544,"body":"what the AI is supposed to be."},{"speaker":"Emily M. Bender","startTime":3202.844,"endTime":3204.734,"body":"Anyway, I'm going to take us to this"},{"speaker":"Emily M. Bender","startTime":3202.844,"endTime":3204.734,"body":"one next, because I want it to be your"},{"speaker":"Emily M. Bender","startTime":3204.734,"endTime":3205.974,"body":"turn and you get to talk about this one."},{"speaker":"Alex Hanna","startTime":3205.984,"endTime":3206.364,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3206.364,"endTime":3206.584,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3206.584,"endTime":3207.504,"body":"So this one is."},{"speaker":"Alex Hanna","startTime":3207.979,"endTime":3212.879,"body":"I posted this on, on BlueSky, but"},{"speaker":"Alex Hanna","startTime":3207.979,"endTime":3212.879,"body":"this is a, this is a pull request"},{"speaker":"Alex Hanna","startTime":3212.959,"endTime":3218.559,"body":"for something called WL, uh,"},{"speaker":"Alex Hanna","startTime":3212.959,"endTime":3218.559,"body":"WSL, and this is a config file."},{"speaker":"Alex Hanna","startTime":3218.999,"endTime":3223.159,"body":"And so basically someone said they"},{"speaker":"Alex Hanna","startTime":3218.999,"endTime":3223.159,"body":"were rewriting the lead for clarity."},{"speaker":"Alex Hanna","startTime":3223.509,"endTime":3226.344,"body":"Um, this, it's a, it's"},{"speaker":"Alex Hanna","startTime":3223.509,"endTime":3226.344,"body":"a GitHub pull request."},{"speaker":"Alex Hanna","startTime":3226.764,"endTime":3230.104,"body":"Uh, those of you don't know what GitHub"},{"speaker":"Alex Hanna","startTime":3226.764,"endTime":3230.104,"body":"is, uh, for open source projects, if"},{"speaker":"Alex Hanna","startTime":3230.104,"endTime":3233.274,"body":"you want to contribute something, a"},{"speaker":"Alex Hanna","startTime":3230.104,"endTime":3233.274,"body":"change, you do a pull request, and then"},{"speaker":"Alex Hanna","startTime":3233.274,"endTime":3237.114,"body":"a maintainer decides that they want to"},{"speaker":"Alex Hanna","startTime":3233.274,"endTime":3237.114,"body":"merge it or not into the main code base."},{"speaker":"Alex Hanna","startTime":3237.494,"endTime":3242.154,"body":"So this person re rewrites it, blah, blah,"},{"speaker":"Alex Hanna","startTime":3237.494,"endTime":3242.154,"body":"gives this, okay, cool, doesn't look very"},{"speaker":"Alex Hanna","startTime":3242.154,"endTime":3243.824,"body":"complicated, scroll down a little bit."},{"speaker":"Alex Hanna","startTime":3244.584,"endTime":3249.559,"body":"They go down and then, the maintainer,"},{"speaker":"Alex Hanna","startTime":3244.584,"endTime":3249.559,"body":"Matt, uh, Wojo, says, \"Thanks"},{"speaker":"Alex Hanna","startTime":3249.559,"endTime":3252.359,"body":"for the contribution here and"},{"speaker":"Alex Hanna","startTime":3249.559,"endTime":3252.359,"body":"appreciate your attention to detail."},{"speaker":"Alex Hanna","startTime":3252.739,"endTime":3254.659,"body":"We have decided to keep it as is."},{"speaker":"Alex Hanna","startTime":3254.699,"endTime":3258.449,"body":"Part of that decision is that more"},{"speaker":"Alex Hanna","startTime":3254.699,"endTime":3258.449,"body":"and more folks are using AI chat to"},{"speaker":"Alex Hanna","startTime":3258.489,"endTime":3262.719,"body":"access guidance, and tables don't"},{"speaker":"Alex Hanna","startTime":3258.489,"endTime":3262.719,"body":"always translate well in that context.\""},{"speaker":"Alex Hanna","startTime":3263.149,"endTime":3272.379,"body":"And this has 906 thumbs downs, 58"},{"speaker":"Alex Hanna","startTime":3263.149,"endTime":3272.379,"body":"laughing, and then 44 just like sad faces."},{"speaker":"Alex Hanna","startTime":3272.829,"endTime":3276.869,"body":"Um, and so people are, developers,"},{"speaker":"Alex Hanna","startTime":3272.829,"endTime":3276.869,"body":"these are software developers, they're"},{"speaker":"Alex Hanna","startTime":3276.869,"endTime":3278.419,"body":"like, what the hell is wrong with you?"},{"speaker":"Alex Hanna","startTime":3278.449,"endTime":3279.399,"body":"These are terrible."},{"speaker":"Alex Hanna","startTime":3279.609,"endTime":3285.689,"body":"Like, um, the next user in the chain,"},{"speaker":"Alex Hanna","startTime":3279.609,"endTime":3285.689,"body":"uh, Peter K Murphy says, \"Would you"},{"speaker":"Alex Hanna","startTime":3285.729,"endTime":3288.519,"body":"and your team be kind enough to revisit"},{"speaker":"Alex Hanna","startTime":3285.729,"endTime":3288.519,"body":"your decision to accept the commit?"},{"speaker":"Alex Hanna","startTime":3288.689,"endTime":3292.899,"body":"As a WSL user, I found the documentation"},{"speaker":"Alex Hanna","startTime":3288.689,"endTime":3292.899,"body":"easy to understand and very helpful."},{"speaker":"Alex Hanna","startTime":3293.2,"endTime":3296.519,"body":"As a developer, I would never trust"},{"speaker":"Alex Hanna","startTime":3293.2,"endTime":3296.519,"body":"an LLM to understand text for me."},{"speaker":"Alex Hanna","startTime":3296.769,"endTime":3299.619,"body":"The professional thing to do would"},{"speaker":"Alex Hanna","startTime":3296.769,"endTime":3299.619,"body":"be to always read it myself.\""},{"speaker":"Alex Hanna","startTime":3300.029,"endTime":3301.429,"body":"Uh, 400 hearts to that."},{"speaker":"Alex Hanna","startTime":3301.809,"endTime":3305.329,"body":"Uh, next person, Scott.Web, \"This is"},{"speaker":"Alex Hanna","startTime":3301.809,"endTime":3305.329,"body":"hands down the worst response to a"},{"speaker":"Alex Hanna","startTime":3305.329,"endTime":3307.139,"body":"documentation patch I've ever seen.\""},{"speaker":"Alex Hanna","startTime":3307.629,"endTime":3310.719,"body":"Um, and then, uh, someone else, \"Yep."},{"speaker":"Alex Hanna","startTime":3310.729,"endTime":3312.409,"body":"Microsoft are losing"},{"speaker":"Alex Hanna","startTime":3310.729,"endTime":3312.409,"body":"their fucking minds again."},{"speaker":"Alex Hanna","startTime":3312.419,"endTime":3313.819,"body":"A normal day in 2024.\""},{"speaker":"Alex Hanna","startTime":3314.239,"endTime":3317.989,"body":"Um, next person, \"More and more"},{"speaker":"Alex Hanna","startTime":3314.239,"endTime":3317.989,"body":"folks are using AI chat because"},{"speaker":"Alex Hanna","startTime":3317.999,"endTime":3320.379,"body":"it's being forced down their throats"},{"speaker":"Alex Hanna","startTime":3317.999,"endTime":3320.379,"body":"without expressing a desire for it."},{"speaker":"Alex Hanna","startTime":3320.599,"endTime":3322.649,"body":"Less accessible documentation"},{"speaker":"Alex Hanna","startTime":3320.599,"endTime":3322.649,"body":"is not the way.\""},{"speaker":"Alex Hanna","startTime":3323.069,"endTime":3324.859,"body":"And then, um, scroll down."},{"speaker":"Alex Hanna","startTime":3324.879,"endTime":3329.954,"body":"There's a great comment, which"},{"speaker":"Alex Hanna","startTime":3324.879,"endTime":3329.954,"body":"is basically like, um, let's"},{"speaker":"Alex Hanna","startTime":3329.954,"endTime":3331.984,"body":"see, this, this one's great."},{"speaker":"Alex Hanna","startTime":3331.984,"endTime":3334.604,"body":"Someone wrote, \"PR bad"},{"speaker":"Alex Hanna","startTime":3331.984,"endTime":3334.604,"body":"cause AI can't read."},{"speaker":"Alex Hanna","startTime":3334.604,"endTime":3335.204,"body":"Sad face.\""},{"speaker":"Alex Hanna","startTime":3335.634,"endTime":3338.724,"body":"Uh, and then there's, there's"},{"speaker":"Alex Hanna","startTime":3335.634,"endTime":3338.724,"body":"been some good things here."},{"speaker":"Alex Hanna","startTime":3338.724,"endTime":3341.724,"body":"Like basically, like I've"},{"speaker":"Alex Hanna","startTime":3338.724,"endTime":3341.724,"body":"always taught that, like."},{"speaker":"Alex Hanna","startTime":3342.134,"endTime":3346.714,"body":"Oh yeah, here, I like this reply"},{"speaker":"Alex Hanna","startTime":3342.134,"endTime":3346.714,"body":"from Mike Taylor who says, um,"},{"speaker":"Alex Hanna","startTime":3347.224,"endTime":3350.294,"body":"\"After decades of being told, quite"},{"speaker":"Alex Hanna","startTime":3347.224,"endTime":3350.294,"body":"rightly, you should write your code"},{"speaker":"Alex Hanna","startTime":3350.294,"endTime":3354.204,"body":"first to be comprehensible to humans,"},{"speaker":"Alex Hanna","startTime":3350.294,"endTime":3354.204,"body":"and only secondarily for computers."},{"speaker":"Alex Hanna","startTime":3354.534,"endTime":3358.534,"body":"Now we have Microsoft telling us our"},{"speaker":"Alex Hanna","startTime":3354.534,"endTime":3358.534,"body":"freaking documentation is supposed to be"},{"speaker":"Alex Hanna","startTime":3358.554,"endTime":3360.744,"body":"written for computers instead of humans."},{"speaker":"Alex Hanna","startTime":3361.164,"endTime":3362.284,"body":"Oh dear, poor dear.\""},{"speaker":"Alex Hanna","startTime":3362.495,"endTime":3366.874,"body":"Um, and then a quote from somebody"},{"speaker":"Alex Hanna","startTime":3362.495,"endTime":3366.874,"body":"named Martin Fowler, who says,"},{"speaker":"Alex Hanna","startTime":3366.884,"endTime":3368.994,"body":"\"Any fool can write code that"},{"speaker":"Alex Hanna","startTime":3366.884,"endTime":3368.994,"body":"the computer can understand."},{"speaker":"Alex Hanna","startTime":3369.284,"endTime":3371.364,"body":"Good programmers write code"},{"speaker":"Alex Hanna","startTime":3369.284,"endTime":3371.364,"body":"that humans can understand."},{"speaker":"Alex Hanna","startTime":3371.764,"endTime":3375.474,"body":"Anyways, the long and short of it"},{"speaker":"Alex Hanna","startTime":3371.764,"endTime":3375.474,"body":"is that they, uh, griped enough"},{"speaker":"Alex Hanna","startTime":3375.724,"endTime":3377.344,"body":"and they reversed the change."},{"speaker":"Emily M. Bender","startTime":3377.374,"endTime":3377.544,"body":"Right."},{"speaker":"Emily M. Bender","startTime":3378.114,"endTime":3379.304,"body":"After, after whining about it."},{"speaker":"Alex Hanna","startTime":3379.614,"endTime":3379.904,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3379.934,"endTime":3384.434,"body":"After it was, it was like a day"},{"speaker":"Alex Hanna","startTime":3379.934,"endTime":3384.434,"body":"and like hundreds of comments."},{"speaker":"Emily M. Bender","startTime":3384.744,"endTime":3384.954,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":3384.954,"endTime":3387.884,"body":"And then this, this, this"},{"speaker":"Emily M. Bender","startTime":3384.954,"endTime":3387.884,"body":"maintainer says, \"Okay, fine."},{"speaker":"Emily M. Bender","startTime":3388.074,"endTime":3390.244,"body":"Everyone seems upset about the AI comment."},{"speaker":"Emily M. Bender","startTime":3390.564,"endTime":3393.734,"body":"Rest assured humans are my number"},{"speaker":"Emily M. Bender","startTime":3390.564,"endTime":3393.734,"body":"one priority for these docs, y'all.\""},{"speaker":"Alex Hanna","startTime":3393.914,"endTime":3394.354,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3394.784,"endTime":3395.194,"body":"My God."},{"speaker":"Alex Hanna","startTime":3395.564,"endTime":3395.654,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3395.854,"endTime":3399.604,"body":"Someone in the chat, uh, John Near"},{"speaker":"Alex Hanna","startTime":3395.854,"endTime":3399.604,"body":"Seattle goes, \"Ratioed on GitHub.\""},{"speaker":"Alex Hanna","startTime":3400.379,"endTime":3404.949,"body":"Yeah, uh, and, and Steal Case uh,"},{"speaker":"Alex Hanna","startTime":3400.379,"endTime":3404.949,"body":"helpfully says, \"WSL is Windows"},{"speaker":"Alex Hanna","startTime":3404.949,"endTime":3406.619,"body":"subs, uh, subsystem for Linux."},{"speaker":"Alex Hanna","startTime":3406.849,"endTime":3409.659,"body":"It's basically how you can run a"},{"speaker":"Alex Hanna","startTime":3406.849,"endTime":3409.659,"body":"Linux terminal locally on Windows,"},{"speaker":"Alex Hanna","startTime":3409.699,"endTime":3410.959,"body":"uh, and many developers use it.\""},{"speaker":"Alex Hanna","startTime":3410.959,"endTime":3411.309,"body":"Thank you."},{"speaker":"Alex Hanna","startTime":3411.329,"endTime":3414.249,"body":"I've, I've used Cygwin to do that"},{"speaker":"Alex Hanna","startTime":3411.329,"endTime":3414.249,"body":"in the past, but I didn't know"},{"speaker":"Alex Hanna","startTime":3414.269,"endTime":3415.709,"body":"Microsoft maintained a tool like that."},{"speaker":"Alex Hanna","startTime":3416.424,"endTime":3416.784,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3417.344,"endTime":3417.764,"body":"All right."},{"speaker":"Emily M. Bender","startTime":3417.774,"endTime":3419.774,"body":"So taking us over to"},{"speaker":"Emily M. Bender","startTime":3417.774,"endTime":3419.774,"body":"Seattle Public Schools."},{"speaker":"Emily M. Bender","startTime":3419.864,"endTime":3421.834,"body":"Um, and this is near and dear to my heart."},{"speaker":"Emily M. Bender","startTime":3421.974,"endTime":3423.284,"body":"I went to Seattle Public Schools."},{"speaker":"Emily M. Bender","startTime":3423.284,"endTime":3424.764,"body":"My kids went to Seattle Public Schools."},{"speaker":"Emily M. Bender","startTime":3424.784,"endTime":3426.494,"body":"I want them to be doing better than this."},{"speaker":"Emily M. Bender","startTime":3426.774,"endTime":3429.514,"body":"Um, and here is their"},{"speaker":"Emily M. Bender","startTime":3426.774,"endTime":3429.514,"body":"documentation on AI tools."},{"speaker":"Emily M. Bender","startTime":3429.754,"endTime":3431.664,"body":"And this is linked from another"},{"speaker":"Emily M. Bender","startTime":3429.754,"endTime":3431.664,"body":"page where they're talking about"},{"speaker":"Emily M. Bender","startTime":3431.664,"endTime":3433.164,"body":"like their principles for AI use."},{"speaker":"Emily M. Bender","startTime":3433.204,"endTime":3436.454,"body":"And like reading that, it's like, okay,"},{"speaker":"Emily M. Bender","startTime":3433.204,"endTime":3436.454,"body":"they're a school district, they have to"},{"speaker":"Emily M. Bender","startTime":3436.454,"endTime":3439.564,"body":"deal with this somehow, they have to come"},{"speaker":"Emily M. Bender","startTime":3436.454,"endTime":3439.564,"body":"up with a policy, and the way they were"},{"speaker":"Emily M. Bender","startTime":3439.564,"endTime":3443.854,"body":"putting it together, it was, you know,"},{"speaker":"Emily M. Bender","startTime":3439.564,"endTime":3443.854,"body":"grounded in their principles, and like,"},{"speaker":"Emily M. Bender","startTime":3444.055,"endTime":3447.084,"body":"that was all well and good, but it's done"},{"speaker":"Emily M. Bender","startTime":3444.055,"endTime":3447.084,"body":"without an actual understanding of how"},{"speaker":"Emily M. Bender","startTime":3447.084,"endTime":3448.904,"body":"the systems work, and so it falls apart."},{"speaker":"Emily M. Bender","startTime":3449.224,"endTime":3454.264,"body":"And in particular, um, I wanted"},{"speaker":"Emily M. Bender","startTime":3449.224,"endTime":3454.264,"body":"to look at these FAQs here."},{"speaker":"Emily M. Bender","startTime":3454.439,"endTime":3459.879,"body":"Right, um, so, um, \"What is"},{"speaker":"Emily M. Bender","startTime":3454.439,"endTime":3459.879,"body":"generative AI and how can it be used?"},{"speaker":"Emily M. Bender","startTime":3459.88,"endTime":3462.189,"body":"Generative AI is a type of"},{"speaker":"Emily M. Bender","startTime":3459.88,"endTime":3462.189,"body":"artificial intelligence that can"},{"speaker":"Emily M. Bender","startTime":3462.189,"endTime":3465.309,"body":"produce various types of content,"},{"speaker":"Emily M. Bender","startTime":3462.189,"endTime":3465.309,"body":"including text, images, and audio."},{"speaker":"Emily M. Bender","startTime":3465.829,"endTime":3469.069,"body":"ChatGPT is an example of generative"},{"speaker":"Emily M. Bender","startTime":3465.829,"endTime":3469.069,"body":"AI that creates content based"},{"speaker":"Emily M. Bender","startTime":3469.069,"endTime":3471.769,"body":"on receiving prompt, a prompt"},{"speaker":"Emily M. Bender","startTime":3469.069,"endTime":3471.769,"body":"from someone using the program.\""},{"speaker":"Emily M. Bender","startTime":3472.239,"endTime":3475.839,"body":"And some, somewhere they talk"},{"speaker":"Emily M. Bender","startTime":3472.239,"endTime":3475.839,"body":"about it being transformative."},{"speaker":"Emily M. Bender","startTime":3476.539,"endTime":3476.949,"body":"Yes."},{"speaker":"Emily M. Bender","startTime":3477.229,"endTime":3481.299,"body":"\"When and why did SPS unblock ChatGPT"},{"speaker":"Emily M. Bender","startTime":3477.229,"endTime":3481.299,"body":"access for staff and students?\""},{"speaker":"Emily M. Bender","startTime":3481.674,"endTime":3485.314,"body":"First answer, \"SPS has never"},{"speaker":"Emily M. Bender","startTime":3481.674,"endTime":3485.314,"body":"blocked ChatGPT on staff devices."},{"speaker":"Emily M. Bender","startTime":3485.774,"endTime":3490.044,"body":"Fall 2023, SPS opened ChatGPT"},{"speaker":"Emily M. Bender","startTime":3485.774,"endTime":3490.044,"body":"on SPS student devices."},{"speaker":"Emily M. Bender","startTime":3490.414,"endTime":3494.214,"body":"This decision ensures our students have"},{"speaker":"Emily M. Bender","startTime":3490.414,"endTime":3494.214,"body":"access to this transformative technology.\""},{"speaker":"Emily M. Bender","startTime":3495.394,"endTime":3499.444,"body":"And it's like, no, like you could"},{"speaker":"Emily M. Bender","startTime":3495.394,"endTime":3499.444,"body":"have resisted and I'm sad and"},{"speaker":"Emily M. Bender","startTime":3499.444,"endTime":3501.124,"body":"I'm, you know, I'm right here."},{"speaker":"Emily M. Bender","startTime":3501.174,"endTime":3503.954,"body":"I would have volunteered my"},{"speaker":"Emily M. Bender","startTime":3501.174,"endTime":3503.954,"body":"time to help you understand it."},{"speaker":"Emily M. Bender","startTime":3504.014,"endTime":3508.234,"body":"If anyone from SPS is listening,"},{"speaker":"Emily M. Bender","startTime":3504.014,"endTime":3508.234,"body":"um, I'm here and I care."},{"speaker":"Emily M. Bender","startTime":3509.644,"endTime":3510.194,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3511.124,"endTime":3513.254,"body":"Uh, next we can do these last"},{"speaker":"Emily M. Bender","startTime":3511.124,"endTime":3513.254,"body":"couple ones really quickly."},{"speaker":"Alex Hanna","startTime":3513.384,"endTime":3513.524,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3513.524,"endTime":3518.009,"body":"So this is TechCrunch and this"},{"speaker":"Alex Hanna","startTime":3513.524,"endTime":3518.009,"body":"is, um, written by, Charles"},{"speaker":"Alex Hanna","startTime":3518.009,"endTime":3520.879,"body":"Rollet, from December 16th."},{"speaker":"Alex Hanna","startTime":3520.899,"endTime":3525.029,"body":"Uh, \"Cohere is quietly working with"},{"speaker":"Alex Hanna","startTime":3520.899,"endTime":3525.029,"body":"Palantir to deploy its AI models.\""},{"speaker":"Alex Hanna","startTime":3525.499,"endTime":3527.929,"body":"Um, it's what the tin says."},{"speaker":"Alex Hanna","startTime":3528.839,"endTime":3534.719,"body":"Big companies are working with, uh,"},{"speaker":"Alex Hanna","startTime":3528.839,"endTime":3534.719,"body":"Palantir and Anduril and every Lord"},{"speaker":"Alex Hanna","startTime":3534.719,"endTime":3538.759,"body":"of the Rings, um, name that has"},{"speaker":"Alex Hanna","startTime":3534.719,"endTime":3538.759,"body":"been twisted to be a stupid product."},{"speaker":"Emily M. Bender","startTime":3538.799,"endTime":3540.589,"body":"And if you"},{"speaker":"Emily M. Bender","startTime":3538.799,"endTime":3540.589,"body":"thought Cohere was somehow better,"},{"speaker":"Emily M. Bender","startTime":3540.589,"endTime":3541.789,"body":"cause it was a research lab."},{"speaker":"Emily M. Bender","startTime":3541.879,"endTime":3543.259,"body":"Um, no, of course not."},{"speaker":"Alex Hanna","startTime":3543.259,"endTime":3548.049,"body":"Well, Cohere is not,"},{"speaker":"Alex Hanna","startTime":3543.259,"endTime":3548.049,"body":"Cohere is like Hugging Face where"},{"speaker":"Alex Hanna","startTime":3548.059,"endTime":3551.349,"body":"they have a business side of the house"},{"speaker":"Alex Hanna","startTime":3548.059,"endTime":3551.349,"body":"and then a product side of the house."},{"speaker":"Alex Hanna","startTime":3551.349,"endTime":3554.949,"body":"So Cohere for AI, that Sara"},{"speaker":"Alex Hanna","startTime":3551.349,"endTime":3554.949,"body":"Hooker directs, is not."},{"speaker":"Alex Hanna","startTime":3555.759,"endTime":3557.559,"body":"That said, it doesn't make it any better."},{"speaker":"Emily M. Bender","startTime":3558.759,"endTime":3559.029,"body":"No."},{"speaker":"Emily M. Bender","startTime":3559.099,"endTime":3559.399,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3560.009,"endTime":3563.54,"body":"Uh, next, \"Elsevier rewrites"},{"speaker":"Emily M. Bender","startTime":3560.009,"endTime":3563.54,"body":"academic papers with AI, without"},{"speaker":"Emily M. Bender","startTime":3563.669,"endTime":3565.399,"body":"telling editors or authors.\""},{"speaker":"Emily M. Bender","startTime":3565.429,"endTime":3566.829,"body":"And this is by Amy N."},{"speaker":"Emily M. Bender","startTime":3566.849,"endTime":3572.919,"body":"David on January 5th of 2025, um,"},{"speaker":"Emily M. Bender","startTime":3566.849,"endTime":3572.919,"body":"under, on a site called PivotToAI.com."},{"speaker":"Emily M. Bender","startTime":3573.474,"endTime":3579.035,"body":"Um, and so apparently, um, a journal"},{"speaker":"Emily M. Bender","startTime":3573.474,"endTime":3579.035,"body":"published by Elsevier, um, so Journal"},{"speaker":"Emily M. Bender","startTime":3579.035,"endTime":3585.455,"body":"of Human Evolution, and Elsevier went"},{"speaker":"Emily M. Bender","startTime":3579.035,"endTime":3585.455,"body":"in and edited things with some AI editor"},{"speaker":"Emily M. Bender","startTime":3585.845,"endTime":3590.065,"body":"and, um, didn't tell the authors or"},{"speaker":"Emily M. Bender","startTime":3585.845,"endTime":3590.065,"body":"the editors that this was happening."},{"speaker":"Alex Hanna","startTime":3590.425,"endTime":3594.034,"body":"Yeah, well, then this"},{"speaker":"Alex Hanna","startTime":3590.425,"endTime":3594.034,"body":"also resulted in the editors, the"},{"speaker":"Alex Hanna","startTime":3594.034,"endTime":3597.395,"body":"editor, a bunch of editors assigned"},{"speaker":"Alex Hanna","startTime":3594.034,"endTime":3597.395,"body":"from the board, the editorial board"},{"speaker":"Alex Hanna","startTime":3597.395,"endTime":3599.765,"body":"of this, just absolutely nightmare."},{"speaker":"Emily M. Bender","startTime":3600.625,"endTime":3601.005,"body":"Okay."},{"speaker":"Emily M. Bender","startTime":3601.015,"endTime":3604.36,"body":"And then, as sort of a look ahead, you"},{"speaker":"Emily M. Bender","startTime":3601.015,"endTime":3604.36,"body":"want to do the honors on this one, Alex?"},{"speaker":"Alex Hanna","startTime":3604.36,"endTime":3608.32,"body":"Yeah, so we're previewing"},{"speaker":"Alex Hanna","startTime":3604.36,"endTime":3608.32,"body":"an episode for next week, uh, where we"},{"speaker":"Alex Hanna","startTime":3608.32,"endTime":3611.76,"body":"look at the UK and their turn to AI."},{"speaker":"Alex Hanna","startTime":3611.809,"endTime":3615.15,"body":"Um, so if you thought the Labor"},{"speaker":"Alex Hanna","startTime":3611.809,"endTime":3615.15,"body":"government was going to be any"},{"speaker":"Alex Hanna","startTime":3615.15,"endTime":3616.93,"body":"better, you would be wrong."},{"speaker":"Alex Hanna","startTime":3617.02,"endTime":3622.99,"body":"Uh, the, the, uh, headline, \"'Mainlined"},{"speaker":"Alex Hanna","startTime":3617.02,"endTime":3622.99,"body":"into the UK's veins,'\" which is an actual"},{"speaker":"Alex Hanna","startTime":3623.02,"endTime":3625.42,"body":"quote from, um, I believe a minister."},{"speaker":"Alex Hanna","startTime":3625.74,"endTime":3630.565,"body":"\"Labor announces huge public rollout"},{"speaker":"Alex Hanna","startTime":3625.74,"endTime":3630.565,"body":"of AI, plans to make the UK the world"},{"speaker":"Alex Hanna","startTime":3630.605,"endTime":3635.065,"body":"leader in AI sector, including opening"},{"speaker":"Alex Hanna","startTime":3630.605,"endTime":3635.065,"body":"access to NHS and other public data.\""},{"speaker":"Emily M. Bender","startTime":3636.665,"endTime":3639.094,"body":"And I just, I want"},{"speaker":"Emily M. Bender","startTime":3636.665,"endTime":3639.094,"body":"to sit for a moment with that metaphor"},{"speaker":"Emily M. Bender","startTime":3639.095,"endTime":3644.304,"body":"there, like mainline, so they're"},{"speaker":"Emily M. Bender","startTime":3639.095,"endTime":3644.304,"body":"basically saying this is a drug and it's"},{"speaker":"Emily M. Bender","startTime":3644.305,"endTime":3649.215,"body":"like, what, what is the entity that is"},{"speaker":"Emily M. Bender","startTime":3644.305,"endTime":3649.215,"body":"going to get the high from this drug?"},{"speaker":"Emily M. Bender","startTime":3649.545,"endTime":3651.835,"body":"It's their stock index, right?"},{"speaker":"Alex Hanna","startTime":3652.295,"endTime":3652.595,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3653.295,"endTime":3654.575,"body":"Well, yeah."},{"speaker":"Alex Hanna","startTime":3654.575,"endTime":3656.835,"body":"Well, no, it's, it's, it's the UK public."},{"speaker":"Alex Hanna","startTime":3656.855,"endTime":3657.595,"body":"Don't you know?"},{"speaker":"Emily M. Bender","startTime":3658.215,"endTime":3658.675,"body":"Yeah."},{"speaker":"Emily M. Bender","startTime":3658.765,"endTime":3659.045,"body":"Right."},{"speaker":"Emily M. Bender","startTime":3659.065,"endTime":3659.664,"body":"Of course."},{"speaker":"Alex Hanna","startTime":3659.945,"endTime":3660.365,"body":"Yeah."},{"speaker":"Alex Hanna","startTime":3661.205,"endTime":3662.055,"body":"Uh, well that's it."},{"speaker":"Alex Hanna","startTime":3662.535,"endTime":3663.424,"body":"Join us next week."},{"speaker":"Alex Hanna","startTime":3663.424,"endTime":3667.375,"body":"We'll have a very special guest,"},{"speaker":"Alex Hanna","startTime":3663.424,"endTime":3667.375,"body":"uh, joining us to talk about that."},{"speaker":"Alex Hanna","startTime":3667.685,"endTime":3669.765,"body":"And we'll, we won't tell"},{"speaker":"Alex Hanna","startTime":3667.685,"endTime":3669.765,"body":"you who until later."},{"speaker":"Alex Hanna","startTime":3670.275,"endTime":3671.335,"body":"That's it for this week."},{"speaker":"Alex Hanna","startTime":3671.375,"endTime":3672.915,"body":"Our theme song is by Toby Menon."},{"speaker":"Alex Hanna","startTime":3672.945,"endTime":3674.845,"body":"Graphic design by Naomi Pleasure-Park."},{"speaker":"Alex Hanna","startTime":3674.865,"endTime":3676.375,"body":"Production by Christie Taylor."},{"speaker":"Alex Hanna","startTime":3676.825,"endTime":3680.084,"body":"And thanks, as always, to the"},{"speaker":"Alex Hanna","startTime":3676.825,"endTime":3680.084,"body":"Distributed AI Research Institute."},{"speaker":"Alex Hanna","startTime":3680.285,"endTime":3683.845,"body":"If you like this show, you can"},{"speaker":"Alex Hanna","startTime":3680.285,"endTime":3683.845,"body":"support us in so many ways."},{"speaker":"Alex Hanna","startTime":3684.125,"endTime":3687.024,"body":"Rate and review us on"},{"speaker":"Alex Hanna","startTime":3684.125,"endTime":3687.024,"body":"Apple Podcasts and Spotify."},{"speaker":"Alex Hanna","startTime":3687.025,"endTime":3693.605,"body":"Pre order The AI Con at TheCon.AI"},{"speaker":"Alex Hanna","startTime":3687.025,"endTime":3693.605,"body":"or wherever you get your books."},{"speaker":"Alex Hanna","startTime":3694.07,"endTime":3699.8,"body":"Subscribe to the Mystery AI Hype"},{"speaker":"Alex Hanna","startTime":3694.07,"endTime":3699.8,"body":"Theater newsletter at ButtonDown or"},{"speaker":"Alex Hanna","startTime":3699.8,"endTime":3702.24,"body":"donate to DAIR at DAIR-Institute.Org."},{"speaker":"Alex Hanna","startTime":3702.24,"endTime":3705.72,"body":"That's D A I R hyphen Institute dot O R G."},{"speaker":"Emily M. Bender","startTime":3706.2,"endTime":3708.57,"body":"Find all our"},{"speaker":"Emily M. Bender","startTime":3706.2,"endTime":3708.57,"body":"past episodes on PeerTube and"},{"speaker":"Emily M. Bender","startTime":3708.57,"endTime":3709.7,"body":"wherever you get your podcasts."},{"speaker":"Emily M. Bender","startTime":3709.719,"endTime":3711.64,"body":"You can watch and comment on"},{"speaker":"Emily M. Bender","startTime":3709.719,"endTime":3711.64,"body":"the show while it's happening"},{"speaker":"Emily M. Bender","startTime":3711.65,"endTime":3712.859,"body":"live on our Twitch stream."},{"speaker":"Emily M. Bender","startTime":3713.08,"endTime":3715.99,"body":"That's Twitch.TV/DAIR_Institute."},{"speaker":"Emily M. Bender","startTime":3716.09,"endTime":3718.51,"body":"Again, that's D A I R"},{"speaker":"Emily M. Bender","startTime":3716.09,"endTime":3718.51,"body":"underscore Institute."},{"speaker":"Emily M. Bender","startTime":3718.66,"endTime":3719.24,"body":"I'm Emily M."},{"speaker":"Emily M. Bender","startTime":3719.25,"endTime":3719.75,"body":"Bender."},{"speaker":"Alex Hanna","startTime":3720.139,"endTime":3721.27,"body":"And I'm Alex Hanna."},{"speaker":"Alex Hanna","startTime":3721.3,"endTime":3722.924,"body":"Stay out of AI Hell, y'all."}]}