The 'Wizard of Oz' Validation Protocol
by Crystal Widjaja • Chief Product Officer at Kumu / Former SVP of Growth at Gojek at Kumu / Gojek (GoTo)
Crystal Widjaja is a product and growth executive who built the data, fraud, and growth teams from scratch at Gojek, helping scale it into Southeast Asia's largest super app. She is currently the CPO at Kumu and a Section 4 instructor on data strategies.
🎙️ Episode Context
Crystal Widjaja shares her playbook for scaling Southeast Asia's biggest startups, focusing on the transition from intuition to rigorous data science. She details how to execute 'Wizard of Oz' experiments to validate features without code, establishes concrete benchmarks for retention, and explains why most analytics implementations fail to generate actual insights.
Problem It Solves
Prevents over-engineering features that users don't actually want or understand.
Framework Overview
A methodology for validating demand and user experience by manually simulating the product backend. It prioritizes the 'front-stage' user experience over 'back-stage' scalability to get immediate data.
🧠 Framework Structure
Define the Ideal Experience. What doe...
Hack the Interface. Use existing tool...
Manual Execution. Use humans (interns...
Measure Intent, Not Just Clicks. Did ...
Scale Only After Validation. Once the...
When to Use
Early-stage startups or when testing high-risk features (like subscriptions or new verticals) in mature companies.
Common Mistakes
Thinking you can't test because you 'need more data' or 'need to build the backend first.'
Real World Example
To test a subscription model, Gojek added 100 drivers to a WhatsApp group. Drivers sold the 'subscription' to riders manually. Interns sat by a phone, manually deducted driver balances, and credited user accounts. It proved the model before a line of code was written.
If you don't have a tested hypothesis... honestly that idea is pretty useless. Testing the actual experience and seeing how people respond to it, that's the best possible data.
— Crystal Widjaja