Our flagship fellowship introduces fellows from any background to the core ideas in AI safety, with a particular focus on existential risk from advanced AI. By the program's end, fellows will have a working map of the field and the foundation to dive deeper into the subareas that interest them.
Explore the implications of increasingly intelligent systems.
A narrative-form scenario describing the geopolitical dynamics and risks of the development of AGI.
A benchmark measuring the trajectory of the length of tasks AIs can complete.
Joe Carlsmith lays out the case for why advanced AIs might develop power-seeking tendencies and how this could lead to catastrophe.
Dario Amodei's essay on the benefits powerful AI systems could bring.
A measurement of the central trends driving continued AI progress.
Nick Bostrom's paper on the potential goals advanced AI systems are likely or unlikely to develop by default.
Examine the challenges in correctly specifying training goals for AI systems.
A fun video from 2023 that discusses the problem of specification gaming.
A comprehensive overview of outer alignment issues from DeepMind researchers.
Explore how alignment researchers have attempted to address issues in goal specification using human preferences.
Investigate the concept of mesa-optimizers and the potential for deceptive behavior in AI systems.
Anthropic's research on alignment faking, where LLMs strategically attempt to preserve their values during training.
An informal note on some intuitions related to Mechanistic Interpretability by Chris Olah.
An in-depth exploration of deceptive alignment and pseudo-alignment, providing insights into inner alignment issues.
Explore various AI security issues including jailbreaks, adversarial examples, and potential vulnerabilities.
A comprehensive playbook for protecting AI models from theft and misuse.
Aguement that AI cybersecurity must learn from past security lessons, not reinvent them.
How Researchers extract embedding layers from language models through inexpensive API attacks.
China's new regulations force companies to report software vulnerabilities to government agencies.
A paper review post about adversarial examples, their implications, and potential solutions.
Explore a famous case of LLM jailbreaking and its implications for AI security.
Examine the challenges and approaches to governing AI development and deployment.
An overview of technical AI governance and its methods for evaluating and enforcing AI control mechanisms.
A proposal for FDA-style approval regulation for frontier AI systems.
Examine critiques of AI safety concerns and alternative perspectives on AI development.
Listen to 00:00 - 10:30 for a discussion on criticisms of AI safety concerns.
A useful reading for understanding infohazards in AI development.
A critical examination of interpretability approaches in AI alignment.
Explore various AI alignment approaches and dive deeper into specific areas of interest. Fellows will choose one of the optional readings to focus on for the week.
An overview of various AI alignment approaches, providing a foundation for further exploration.
A deeper dive into the concept of agent foundations in AI alignment.
An in-depth exploration of inner alignment issues and goal misgeneralization.
A technical exploration of interpretability in neural networks.
An examination of techniques for controlling large language models.
