💡

InsightHunt

Hunt the Insights

R

Ramesh Johari

Episode #244

Professor at Stanford University / Data Science Advisor

Stanford University (Advisor to Airbnb, Uber, Stitch Fix, etc.)

🎯Product Strategy📈Growth & Metrics🔍User Research

📝Full Transcript

16,482 words
Ramesh Johari (00:00:00): Marketplaces are a little bit like a game of whac-a-mole. One example that I came across with one of the companies I worked with that I love is our new supply side was having a pretty bad experience. (00:00:12): So what we decided to do is build some custom bespoke features that were really going to direct them to more experienced folks on the other side of the market. Good. And then, yeah, lo and behold, pretty soon those metrics start to look better. But then we're looking at it, we're like, "Wait a second. Now the existing folks on the other side are having a worse experience," so you kind of whiplash around. You're like, "Oh, wait a second. We better do something about that." So we take them, we try to match them up with the more experienced folks, and now suddenly a month after that, you're like, "Wait a second," and your metrics just keep moving around. And that's because the whac-a-mole game here is ultimately, a lot of marketplace management is moving attention and inventory around. Many of the changes that are most consequential create winners and losers. And rolling with those changes is about recognizing whether the winners you've created are more important to your business than the losers you've created in the process. Lenny (00:01:00): Today my guest is Ramesh Johari. Ramesh is a professor at Stanford University, where he does research on and teaches data science methods and practices with a specific focus on the design and operation of online marketplaces. He's advised and worked with some of the biggest marketplaces in the world, including Airbnb, Uber, Stripe, Bumble, Stitch Fix, Upwork, and many others. And in our conversation, we get super nerdy on how to build a thriving marketplace, including where to focus your resources to fuel the marketplace flywheel growth, why data and data science is so central to building a successful marketplace, how to design a better review system. Why as a founder, you shouldn't think of yo...

💡 Key Takeaways

  • 1Marketplaces create value by removing transaction friction (finding, matching, trusting), not just aggregating supply.
  • 2Don't define yourself as a 'marketplace founder' until you have scaled liquidity on both sides; start by solving a specific problem.
  • 3Machine Learning models excel at prediction (correlation), but business strategy requires causal inference (decision making).
  • 4Experimentation isn't just about 'winning' or 'losing'; you must pay for learning, even from failed experiments.
  • 5Simple averaging of ratings hurts new entrants; use priors and Bayesian approaches to ensure distributional fairness.
  • 6Marketplace management is like 'Whac-a-mole': optimizing one metric often negatively impacts another group, creating winners and losers.

📚Methodologies (3)

🎯 Product Strategy

Instead of starting as a marketplace, focus on solving specific transaction frictions (search, trust, payment) to build liquidity. Only adopt marketplace dynamics once you have scaled liquidity on at least one side.

Core Principles

  • 1.Identify the Transaction Cost: Determine what friction (trust, payment, discovery) prevents the transaction.
  • 2.Solve for One Side: Build a solution that attracts scaled liquidity on one side (e.g., drivers or hosts) before worrying about matching.
  • 3.Pivot to Platform: Only introduce marketplace complexity (matching algos, fees) once liquidity allows for organic connections.

"A marketplace business never starts as a marketplace business... simply as a founder."

#"friction-first"#liquidity#strategy
View Deep Dive →
📈 Growth & Metrics

Shift focus from 'Prediction' (Machine Learning/Correlation) to 'Decision Making' (Causal Inference). The goal is not to predict the future, but to understand how a specific intervention changes the future.

Core Principles

  • 1.Prediction ≠ Decision: Recognizing that high LTV users might buy anyway; true value comes from identifying lift (incremental impact).
  • 2.Experiment for Learning: Move beyond 'winners vs. losers' to testing hypotheses about user behavior and elasticity.
  • 3.Bayesian Updates: Incorporate prior beliefs and past data into current experiment analysis rather than treating every test as a blank slate.

"Prediction is inherently about correlation. But when we ask people to make decisions, we're asking them to think about causation."

#causal#decision-making#growth
View Deep Dive →
🔍 User Research

Design rating systems that account for the 'cold start' of reputation. Use priors to smooth out early variance and recognize that missing reviews ('sound of silence') contain predictive data.

Core Principles

  • 1.Avoid Raw Averages: Don't let a single data point define a new user; use a 'prior' belief to blend their score until sufficient data exists.
  • 2.Leverage the Sound of Silence: Treat non-reviews as data points (often indicating 'satisfactory but not amazing' or 'avoidance of conflict').
  • 3.Renorming Expectations: Ask specific comparative questions (e.g., 'Did this exceed expectations?') to combat grade inflation.

"The biggest change... is that we get to see what happened with our matches."

#distributionally#rating#research
View Deep Dive →
← Browse All Episodes