Fellowship Syllabus
Strategy and Forecasting
As frontier models grow more capable and the cost of frontier-level performance continues to fall, transformative AI is approaching quickly. Navigating it well requires a coherent picture of where AGI is headed, one that integrates technical progress with the political dynamics of an AGI arms race. This reading and discussion group is dedicated to building shared understanding around the crucial questions of transformative AI. Fellows engage in forecasting informed by scaling laws, threat modeling, and a tabletop wargame exercise, culminating in each fellow writing their own AGI takeoff scenario.
Week 01
Scaling Laws and Interactive Forecasting
Explore various epoch AI papers and engage in interactive forecasting exercises.
- From GPT-4 to AGI: Counting the OOMs
Situational awareness and scaling analysis
- AI Safety Forecasting Workshop
Interactive workshop on AI forecasting
Week 02
Threat Modelling I: Deceptive Alignment
Deep dive into deceptive alignment and its implications.
- How likely is deceptive alignment?
Critical analysis of deceptive alignment risks
- Threat Models
Different ways that AI risk could manifest
- AI Risk Concrete Stories
Explore concrete scenarios and encourage constructive debate
Week 03
Threat Modelling II: Model Organisms
Examine model organisms and their role in alignment research.
- Model Organisms of Misalignment
The case for a new pillar of alignment research
- Alignment faking in large language models
Analysis of how AI models may fake alignment
- Model Organisms Workshop
Come up with your own ideas for model organisms & critiques
Week 04
AGI Tabletop Wargame
Participate in an immersive AGI tabletop wargame simulation.
Content to be announced.
Week 05
AGI Tabletop Wargame (Continued)
Continue and conclude the AGI tabletop wargame simulation.
Content to be announced.
Week 06
International Game Theory
Analyze unipolar vs. bipolar takeoff scenarios and AGI realism.
- National Security is not International Security
A critique from an Oxford international relations scholar
- Superintelligence Strategy Expert
Analysis of strategic implications
Week 07
Writing Your Own Takeoff Scenario
Synthesize your learning by writing your own AGI takeoff scenario.
- Dave Kasten's AGI-by-2027 vignette
Example scenario for inspiration