by Various Product Leaders (Hosted by Lenny Rachitzky)
Instead of isolating variables (e.g., changing just the email subject line), this methodology argues for changing every possible variable at once to maximize the chance of moving the metric. If the metric moves, you can cost-rationalize and isolate variables later. If it doesn't move after maximum effort, you can conclusively abandon the hypothesis.
Core Principles
- 1.Maximize the Treatment Effect: In the absence of large 'M' (sample size), increase the intensity of the intervention.
- 2.The 'Kitchen Sink' Approach: Combine email, design, phone calls, and personalization into a single test group to force a result.
- 3.Fail Conclusively: If the most expensive, intensive version of the idea doesn't work, the idea is wrong.
- +1 more...
"If you have a hypothesis... just throw all of the possible tactics and resources that you think would move that needle because you can always cost rationalize later if it works."