💡

InsightHunt

Hunt the Insights

K

Keith Coleman & Jay Baxter

Episode #167

Product Lead (Keith) & Founding ML Engineer (Jay) for Community Notes

X (formerly Twitter)

🎯Product StrategyExecution

📝Full Transcript

19,600 words
Lenny Rachitsky (00:00:00): The work that you guys do has had such a tremendous impact on the way the world works. I want to start with just giving people a brief understanding of what is Community Notes. Keith Coleman (00:00:09): Someone on X can see a post. If they think it's misleading, they can propose a note that they think other people might find informative. Other people can then rate that note. Jay Baxter (00:00:18): We actually look for agreement from people who have disagreed in the past. And what we see is when people actually have that sort of surprising agreement, that's what makes the notes so neutral and accurate and well- written, really, overall. Lenny Rachitsky (00:00:31): There's many people that are very polarized. How do you deal with people that are super anti-vax, super Jan 6? Keith Coleman (00:00:36): One philosophical thing that's important is that we want all of humanity to participate and sometimes people are surprised by that. We have all of humanity. We then have the data to understand what notes will be helpful to actual humanity. Every post is eligible for notes. We shouldn't exempt Elon. We shouldn't exempt government figures. We should be like everyone... Even advertisers can get notes. Jay Baxter (00:00:58): There have been external studies run by people totally independent of us who have found that if you take a post with or without a Community Note, that actually people's agreement with the core claims in the post does change if they see it with a note versus without. Lenny Rachitsky (00:01:13): Is there anything else along the lines of just working for Elon within an org Elon runs that might surprise people? Keith Coleman (00:01:18): If I were to start a company in that company, it would be even leaner than I would've made it before. I've been amazed with just how much the team is able to accomplish with a small group and I think because of a small group- Lenny Rachitsky (00:01:33): Today, my guests are Keith Coleman, Pro...

💡 Key Takeaways

  • 1Community Notes uses a 'Bridging-based ranking' algorithm, not majority rule, requiring agreement from users who typically disagree.
  • 2The 'Thermal' team model operates like a startup within a big corp: small autonomous teams, no middle management, and direct lines to executive decision-makers.
  • 3Paradoxically, anonymous contributors are more likely to cross partisan lines and agree on facts than those using real names.
  • 4Open sourcing the algorithm and data was critical for trust; users can audit and replicate the system's decisions.
  • 5Community Notes reduces the virality of misinformation by 50-60% without actually deleting the content.
  • 6Launch strategy: Prove viability at every step (Mockups -> MTurk -> Pilot -> Scale) to overcome internal skepticism.

📚Methodologies (3)

Bridging-Based Ranking Framework

by Keith Coleman & Jay Baxter

🎯 Product Strategy

Instead of asking 'is this note popular?', the algorithm asks 'is this note helpful to people who usually disagree?'. It utilizes matrix factorization to identify user clusters and only surfaces content that bridges the divide between polarized groups.

Core Principles

  • 1.Identifies 'polarized' clusters based on past rating behavior.
  • 2.Requires a 'Bridging Signal': Positive ratings must come from diverse viewpoints, not just one side.
  • 3.Prioritizes context over censorship: Add context to misleading posts rather than deleting them.
  • +1 more...

"We actually look for agreement from people who have disagreed in the past... that's what makes the notes so neutral and accurate."

#bridging-based#ranking#strategy
View Deep Dive →
The 'Thermal' Team Operating Model

by Keith Coleman & Jay Baxter

Execution

A governance structure that insulates a small, mission-driven team from corporate inertia. It mimics early-stage startups by removing middle management and focusing 100% on shipping code.

Core Principles

  • 1.Single-Threaded Leadership: One 'Founder' lead with a direct line to the ultimate decision maker (e.g., Elon/CEO).
  • 2.Radical Focus: Team members are 100% dedicated to the project, no split allocation.
  • 3.Lean Staffing: Start with exactly one of each necessary role (1 ML, 1 Backend, 1 Frontend, 1 Design).
  • +1 more...

"If I were to start a company... it would be even leaner than I would've made it before. I've been amazed with just how much the team is able to accomplish with a small group."

#'thermal'#operating#execution
View Deep Dive →
Progressive De-risking Launch Strategy

by Keith Coleman & Jay Baxter

🎯 Product Strategy

A methodical approach to validation where the product must 'earn' its next expansion by proving quality metrics at the current stage. It moves from static concepts to restricted pilots before public rollout.

Core Principles

  • 1.Concept Validation: Test static mockups with diverse user groups to prove the *demand* for truth exists.
  • 2.Capability Validation: Use MTurk/Contractors to prove average humans *can* write high-quality notes.
  • 3.Controlled Pilot: Launch to a small group (e.g., 1k users) to refine the algorithm.
  • +1 more...

"We were very disciplined... about having the product prove itself at every given point."

#progressive#de-risking#launch
View Deep Dive →