The Low-Volume Conviction Framework
by Brian Tolkin • Head of Product and Design at Opendoor
Brian is the Head of Product and Design at Opendoor. Previously, he was Employee #100 at Uber, where he led the global launch of uberPOOL and established the original Product Operations function.
🎙️ Episode Context
Brian Tolkin shares deep insights on building products that bridge the physical and digital worlds, drawing from his experiences scaling Uber and Opendoor. He discusses the evolution of operational processes into scalable software, how to run non-threatening product reviews, applying Jobs-to-be-Done for low-frequency use cases, and how to rigorously experiment when you lack high transaction volume.
Problem It Solves
How to make data-driven decisions when you don't have enough traffic (sample size) for traditional, high-speed A/B testing.
Framework Overview
A hierarchy of validation techniques for businesses with high-value, low-frequency transactions (like real estate). It prioritizes honest statistical analysis over false precision and offers alternatives to standard A/B tests.
🧠 Framework Structure
Honest Power Analysis. Calculate the ...
Adjust Confidence Intervals. Accept a...
use Macro-Comparison Methods. If user...
Long-Term Holdouts. Set aside a contr...
Proxy Feedback Loops. If you must rel...
When to Use
In B2B contexts, high-ticket consumer products (real estate, cars), or early-stage startups with low traffic.
Common Mistakes
Running an A/B test for a month, getting insignificant results, and then pretending the data provides an answer (false precision).
Real World Example
Opendoor running 6-month long experiments or using 'Sister City' comparisons because people only sell homes once every 7 years, making standard daily A/B testing difficult.
The only mistake here is thinking you'll get an answer in a month when you won't, and then pretending you do.
— Brian Tolkin