The Economic Turing Test
by Benjamin Mann • Co-founder at Anthropic
Former architect of GPT-3 at OpenAI, now leads product engineering and safety alignment at Anthropic.
🎙️ Episode Context
Benjamin Mann discusses the trajectory of AI development towards superintelligence by 2027-2028, the critical importance of AI safety, and Anthropic's unique approach to alignment. He details the implementation of Constitutional AI, the Responsible Scaling Policy (ASL levels), and the 'Resting in Motion' mindset for navigating high-stakes work.
Problem It Solves
Provides a concrete, measurable definition of AGI rooted in economic impact rather than abstract intelligence.
Framework Overview
A metric to define Transformative AI or AGI based on an agent's ability to autonomously perform economically valuable work indistinguishable from a human.
📊 Evaluation Scorecard
When to Use
Forecasting the arrival of AGI or evaluating the practical capability of an AI agent.
Common Mistakes
Focusing on abstract benchmarks (IQ tests) instead of practical economic utility.
Real World Example
Assessing if an AI can fully replace a remote software engineer or customer support agent over a 3-month contract.
If you contract an agent for a month... and it turns out to be a machine rather than a person, then it's passed the Economic Turing Test for that role.
— Benjamin Mann