{"version":"1.1.0","chapters":[{"startTime":34.0,"title":"Introduction"},{"startTime":155.0,"title":"Summary"},{"startTime":292.0,"title":"Main Article Text Begins"},{"startTime":375.0,"title":"1. Many AI experts think there’s a non-negligible chance AI will lead to outcomes as bad as extinction"},{"startTime":656.0,"title":"2. We’re making advances in AI extremely quickly"},{"startTime":909.0,"title":"Current trends show rapid progress in the capabilities of ML systems"},{"startTime":1076.0,"title":"When can we expect transformative AI?"},{"startTime":1123.0,"title":"Footnote 21"},{"startTime":1146.0,"title":"(Text resumes)"},{"startTime":1353.0,"title":"3. Power-seeking AI could pose an existential threat to humanity"},{"startTime":1461.0,"title":"It’s likely we’ll build advanced planning systems"},{"startTime":1610.0,"title":"These systems seem technically possible and we’ll have strong incentives to build them"},{"startTime":1660.0,"title":"Footnote 26"},{"startTime":1697.0,"title":"(Main text resumes)"},{"startTime":1770.0,"title":"Footnote 27"},{"startTime":1797.0,"title":"(Text resumes)"},{"startTime":1833.0,"title":"Footnote 28"},{"startTime":1901.0,"title":"(Text Resumes)"},{"startTime":1908.0,"title":"Advanced planning systems could easily be dangerously ‘misaligned’"},{"startTime":1977.0,"title":"Three examples of “misalignment” in a variety of systems"},{"startTime":2143.0,"title":"Footnote 32"},{"startTime":2235.0,"title":"Why these systems could (by default) be dangerously misaligned"},{"startTime":2496.0,"title":"It might be hard to find ways to prevent this sort of misalignment"},{"startTime":2921.0,"title":"At this point, you may have questions like:"},{"startTime":2968.0,"title":"Disempowerment by AI systems would be an existential catastrophe"},{"startTime":3060.0,"title":"People might deploy misaligned AI systems despite the risk"},{"startTime":3291.0,"title":"This all sounds very abstract. What could an existential catastrophe caused by AI actually look like?"},{"startTime":3493.0,"title":"How could a power-seeking AI actually take power?"},{"startTime":3532.0,"title":"1. Hacking"},{"startTime":3665.0,"title":"2. Gaining financial resources"},{"startTime":3708.0,"title":"3. Persuading or coercing humans"},{"startTime":3804.0,"title":"4. Gaining broader social influence"},{"startTime":3848.0,"title":"5. Developing new technology"},{"startTime":3920.0,"title":"6. Scaling up its own capabilities"},{"startTime":3990.0,"title":"7. Developing destructive capacity"},{"startTime":4059.0,"title":"How could the full story play out?"},{"startTime":4136.0,"title":"Existential catastrophe through getting what you measure"},{"startTime":4391.0,"title":"Existential catastrophe through a single extremely advanced artificial intelligence"},{"startTime":4704.0,"title":"4. Even if we find a way to avoid power-seeking, there are still risks"},{"startTime":4727.0,"title":"AI could worsen war"},{"startTime":4774.0,"title":"Footnote 41"},{"startTime":4898.0,"title":"(Text resumes)"},{"startTime":4928.0,"title":"AI could be used to develop dangerous new technology"},{"startTime":4938.0,"title":"Footnote 42"},{"startTime":4989.0,"title":"AI could empower totalitarian governments"},{"startTime":5011.0,"title":"Footnote 45"},{"startTime":5054.0,"title":"(Text resumes)"},{"startTime":5069.0,"title":"Other risks from AI"},{"startTime":5140.0,"title":"So, how likely is an AI-related catastrophe?"},{"startTime":5303.0,"title":"Footnote 46"},{"startTime":5343.0,"title":"(Text resumes)"},{"startTime":5470.0,"title":"5. We can tackle these risks"},{"startTime":5519.0,"title":"Technical AI safety research"},{"startTime":5576.0,"title":"AI governance research and implementation"},{"startTime":5635.0,"title":"Here are some more questions you might have:"},{"startTime":5664.0,"title":"6. This work is extremely neglected"},{"startTime":5814.0,"title":"What do we think are the best arguments we’re wrong?"},{"startTime":5868.0,"title":"We might have a lot of time to work on this problem"},{"startTime":6012.0,"title":"AI might improve gradually over time"},{"startTime":6109.0,"title":"We might need to solve alignment anyway to make AI useful"},{"startTime":6267.0,"title":"The problem could be extremely difficult to solve"},{"startTime":6360.0,"title":"We could be wrong that strategic AI systems are likely to seek power"},{"startTime":6656.0,"title":"Arguments against working on AI risk to which we think there are strong responses"},{"startTime":6757.0,"title":"Is it even possible to produce artificial general intelligence?"},{"startTime":6899.0,"title":"Why can't we just unplug a dangerous AI?"},{"startTime":6996.0,"title":"Couldn't we just 'sandbox' any potentially dangerous AI system until we know it's safe?"},{"startTime":7082.0,"title":"Surely a truly intelligent AI system would know not to disempower everyone?"},{"startTime":7171.0,"title":"Can't you just not give an AI system bad goals?"},{"startTime":7327.0,"title":"Isn't the real danger from actual current AI — not some sort of futuristic superintelligence?"},{"startTime":7459.0,"title":"But can't AI also do a lot of good?"},{"startTime":7523.0,"title":"You'd have to be really stupid to build or use a system that could genuinely kill everyone, right?"},{"startTime":7647.0,"title":"Footnote 50"},{"startTime":7697.0,"title":"Why shouldn't I dismiss this as motivated reasoning by a group of people who just like playing with computers and want to think that's important?"},{"startTime":7780.0,"title":"This all reads, and feels, like science fiction"},{"startTime":7934.0,"title":"Can it make sense to dedicate my career to solving an issue based on a speculative story about a technology that may or may not ever exist?"},{"startTime":8031.0,"title":"Is this a form of Pascal's mugging — taking a big bet on tiny probabilities?"},{"startTime":8278.0,"title":"What you can do concretely to help"},{"startTime":8342.0,"title":"Technical AI safety"},{"startTime":8345.0,"title":"Approaches"},{"startTime":8489.0,"title":"Key organisations"},{"startTime":8625.0,"title":"Conceptual AI safety labs:"},{"startTime":8699.0,"title":"AI Safety In Academia"},{"startTime":8815.0,"title":"AI governance and strategy"},{"startTime":8819.0,"title":"Approaches"},{"startTime":8940.0,"title":"Key organisations"},{"startTime":9132.0,"title":"Complementary (yet crucial) roles"},{"startTime":9189.0,"title":"Other ways to help"},{"startTime":9289.0,"title":"Want one-on-one advice on pursuing this path?"},{"startTime":9326.0,"title":"Find vacancies on our job board"},{"startTime":9337.0,"title":"Top resources to learn more"},{"startTime":9357.0,"title":"Note from the author:"},{"startTime":9381.0,"title":"Footnote 5"},{"startTime":9482.0,"title":"(Text Resumes)"}]}