The Economic Turing Test
by Benjamin Mann • Co-founder at Anthropic
Former architect of GPT-3 at OpenAI; currently serves as tech lead for product engineering at Anthropic, focusing on aligning AI to be helpful, harmless, and honest.
🎙️ Episode Context
Benjamin Mann discusses the trajectory of AI development, predicting a 50% chance of superintelligence by 2028. He details Anthropic's departure from OpenAI due to safety concerns, the mechanics of Constitutional AI (RLAIF), and how to build products for exponential technologies. He also introduces the 'Economic Turing Test' as a metric for AGI and offers advice on future-proofing careers.
Problem It Solves
Provides a concrete, measurable benchmark for 'AGI' that moves beyond vague definitions of general intelligence to economic impact.
Framework Overview
A pragmatic definition for identifying Transformative AI (or AGI). Instead of philosophical sentience, it tests whether an AI agent can competitively replace a human in a specific economic role over a sustained period.
✔️ Verification Checklist
When to Use
When evaluating the maturity of AI agents for workforce deployment or forecasting macroeconomic shifts.
Common Mistakes
Judging AI based on short, isolated tasks rather than sustained job performance over time.
Real World Example
Hiring a remote contractor for data entry or coding, finding their work excellent, and discovering it was an autonomous agent.
If you contract an agent for a month... and it turns out to be a machine rather than a person, then it's passed the Economic Turing Test for that role.
— Benjamin Mann