Core thesis: Sean Goedecke argues against the popular notion that “everyone talking to the same model gets the same results.” Instead, he claims domain expertise is the most important variable in LLM output quality — and that this gap will persist as models improve.
The Terence Tao example: Goedecke points to Tao’s public conversation with ChatGPT about a counterexample to the Jacobian Conjecture. Tao’s prompting style — short messages, pushback framed as “this looks more complex than expected” rather than direct correction, and rarely taking the model’s suggested next steps — produces dramatically better output than an amateur asking the same model about the same topic. The model shifts into “talking-to-mathematicians” mode simply by detecting Tao’s fluency.
Why expertise matters more than prompting technique: You can’t mimic Tao’s style without his math knowledge underneath it. The skill isn’t the phrasing — it’s knowing what “looks wrong,” which idea to extract from a wall of model output, and which alternate formulation to suggest. He draws a parallel to his own work as an engineer: familiarity with a specific codebase lets him say “don’t we already do X?” or “I think it could be simpler here,” pushing the LLM much harder than generic system-design knowledge would.
The conclusion: As models get more capable, the human becomes the bottleneck, not the model — the knowledge is “in the model” already, but extracting it well requires a human who knows what good looks like. He’s careful to note this isn’t a case against novices — “clinging to the LLM” with no domain knowledge still gets you something — but the ceiling is much higher with expertise.
Pushback worth noting (from his own edit): Some HN commenters flagged this as reassuring-but-self-serving for anyone whose value proposition rests on expertise. Goedecke concedes the point but expects the landscape to shift before it’s really testable. Others noted OpenAI’s own math-verification pipeline still required expert mathematicians to check model outputs — supporting rather than undermining his argument.
Recommendations for Action
- For individuals: Don’t treat “learning to prompt” as a substitute for building domain depth. If you’re using LLMs in a field you already know, invest in deepening that knowledge — it compounds directly into better AI-assisted output. If you’re using LLMs outside your expertise, treat the output as a rough first draft, not a finished product.
- For teams/orgs: Resist the urge to think LLMs flatten skill differences enough to cut expert headcount. The bottleneck shifts to review/steering capacity — you likely still need experts in the loop to catch what “looks wrong,” even if raw output volume goes up.
- For hiring/skills strategy: Prioritize hiring or training people who can articulate precise, specific pushback (“don’t we already do X?”) over people who are merely good at writing prompts in the abstract — the latter is a shallower and more easily commoditized skill.
- For your own workflow: Worth testing directly — try steering Claude on a topic you know deeply versus one you don’t, using Tao’s pattern (short prompts, targeted skepticism, proposing your own next steps rather than taking the model’s) and see if you notice the same quality gap.