💡

InsightHunt

Hunt the Insights

J

Judd Antin

Episode #157

Former Head of Research at Airbnb & Meta

Independent Consultant

🔍User Research👥Team & Culture🚀Career & Leadership

📝Full Transcript

14,140 words
Judd Antin (00:00:00): User-centered performance refers to customer obsession or user-centered practice that is symbolic rather than focused on learning. It's hugely common, I would argue. It's work we do to signal to each other how customer obsessed we are, not because we want to make a different decision. If your listeners are like, "I don't do that." I'm like, "Think about it for a second. This is extremely common." Every time a PM comes to a researcher at the end of a product process and says, "Can you just run a quick user study just to validate our assumptions," that's user-centered performance. It's too late to matter. We got to ship it. What they want is to check the box. One of my big mantras was, "We don't validate, we falsify. We are looking to be wrong." Many PMs, many designers are not in that place. They do not want to be wrong. They're looking to validate, and that's user-centered performance. Lenny (00:00:54): Today my guest is Judd Anton. Judd helped build the user research practice at Facebook. He was a longtime head of research at Airbnb, and his direct reports have gone on to lead research teams at Figma, Notion, Slack, Robinhood, Duolingo, Fair, and other amazing companies. These days, Judd spends his time consulting, helping companies with organizational challenges, product strategy, design, research, hiring, onboarding, and crisis management. In our conversation, we unpack a conclusion that Jud has come to recently about how the user research field is going through a reckoning and what needs to change both within the user research field and how companies leverage user research going forward. (00:01:37): Judd shares what the user research field has gotten wrong over the last decade, how PMs and designers rely on user research too often, and to answer the wrong questions, where user research will continue to provide significant value, and how to best leverage your researchers, why it's important for researchers to think about the business goa...

💡 Key Takeaways

  • 1User research is currently dying because it operates as a service function rather than a strategic partner.
  • 2Stop doing 'Middle Range' research; focus on Macro (Strategic) and Micro (Tactical) research instead.
  • 3User-Centered Performance is a trap where teams signal customer obsession without actually wanting to learn or change decisions.
  • 4Researchers must understand the P&L, conversion funnels, and OKRs to be effective.
  • 5Stop asking for validation; start asking researchers to falsify your assumptions.
  • 6NPS is a flawed metric; use Customer Satisfaction (CSAT) instead for better data integrity.
  • 7The best researchers are 'Swiss Army Knives' with skills in qual, quant, stats, surveys, and SQL/AI.

📚Methodologies (3)

🔍 User Research

A classification system for research impact. Teams should maximize Macro and Micro research while minimizing the nebulous 'Middle Range' research that often confirms biases or states the obvious.

Core Principles

  • 1.Focus on Macro: Strategic, forward-looking work like concept cars, TAM studies, and competitive landscape analysis.
  • 2.Focus on Micro: Tactical, fast evaluative work (usability testing, A/B test deep dives) that optimizes specific funnel steps.
  • 3.Eliminate Middle Range: Avoid vague questions like 'How do users feel about X?' which yield blobular insights that are hard to operationalize.

"Product managers love to ask for middle range research that they can use to justify decisions they're reluctant to make on their own."

#macro-middle-micro#research#users
View Deep Dive →
👥 Team & Culture

User-Centered Performance is doing work to signal customer obsession rather than to learn. To fix this, teams must shift from seeking validation to seeking falsification.

Core Principles

  • 1.Don't Validate, Falsify: Approach research wanting to be proven wrong. Look for evidence that destroys your assumptions.
  • 2.Avoid 'Check the Box' Studies: If the decision is already made or it's too late to change the product, do not run the study.
  • 3.Beware of Executive Listening: Founders/Execs sitting in focus groups often reinforces bias rather than providing rigorous insight.

"It's work we do to signal to each other how customer obsessed we are, not because we want to make a different decision."

#escaping#user-centered#performance
View Deep Dive →
🚀 Career & Leadership

Modern researchers cannot just be empathetic listeners. They must be 'Swiss Army Knives' capable of mixed methods to answer business questions quickly and rigorously.

Core Principles

  • 1.Formative/Generative: Innovation-focused, ethnographic fieldwork to find new opportunities.
  • 2.Evaluative: Usability testing to ensure execution excellence.
  • 3.Survey Design: Rigorous quantitative data collection (avoiding bad survey science).
  • +2 more...

"If all you have is a hammer, then everything looks like a nail. You want a researcher who's got a Swiss army knife."

#5-tool#researcher#career
View Deep Dive →
← Browse All Episodes