‹ All episodes

Joe Carlsmith Audio

Arguments for/against scheming that focus on the path SGD takes (Section 3 of "Scheming AIs")

November 16, 2023 Joe Carlsmith
Joe Carlsmith Audio
Arguments for/against scheming that focus on the path SGD takes (Section 3 of "Scheming AIs")
Chapters
Chapters
0:35
3. Arguments for/against scheming that focus on the path that SGD takes
2:38
3.1 The training-game-independent proxy-goals story
7:14
3.2 The “nearest max-reward goal” story
12:21
3.2.1 Barriers to schemer-like modifications from SGD’s incrementalism
13:53
3.2.2 Which model is “nearest”?
14:28
3.2.2.1 The common-ness of schemer-like goals in goal space
17:43
3.2.2.2 The nearness of non-schemer goals
22:53
3.2.2.3 The relevance of messy goal-directedness to nearness
24:30
3.2.3 Overall take on the “nearest max-reward goal” argument
25:22
3.3 The possible relevance of properties like simplicity and speed to the path SGD takes
27:33
3.4 Overall assessment of arguments that focus on the path SGD takes
Info
Joe Carlsmith Audio
Arguments for/against scheming that focus on the path SGD takes (Section 3 of "Scheming AIs")
Nov 16, 2023
Joe Carlsmith

This is section 3 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” 

Text of the report here: https://arxiv.org/abs/2311.08379
 
Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
 
Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power

Share
Apple Podcasts Spotify Amazon Music Podcast Index Overcast Podcast Addict +
Show Notes Chapter Markers

This is section 3 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” 

Text of the report here: https://arxiv.org/abs/2311.08379
 
Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
 
Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power

0:35
3. Arguments for/against scheming that focus on the path that SGD takes
2:38
3.1 The training-game-independent proxy-goals story
7:14
3.2 The “nearest max-reward goal” story
12:21
3.2.1 Barriers to schemer-like modifications from SGD’s incrementalism
13:53
3.2.2 Which model is “nearest”?
14:28
3.2.2.1 The common-ness of schemer-like goals in goal space
17:43
3.2.2.2 The nearness of non-schemer goals
22:53
3.2.2.3 The relevance of messy goal-directedness to nearness
24:30
3.2.3 Overall take on the “nearest max-reward goal” argument
25:22
3.3 The possible relevance of properties like simplicity and speed to the path SGD takes
27:33
3.4 Overall assessment of arguments that focus on the path SGD takes