by Brendan Foody
A cyclical framework for building AI products where the evaluation metric (Eval) serves as the Product Requirement Document (PRD). Instead of traditional feature shipping, the product loop focuses on defining success criteria and using reinforcement learning to climb that metric.
Core Principles
- 1.The model is the product; the eval is the PRD.
- 2.Success is measured by automated verifiers, not just human feel.
- 3.Reinforcement Learning (RL) climbs the eval metric.
- +1 more...
"If the model is the product, then the eval is the product requirement document."