Categories
AI

LLMs Reward Expertise — Key Takeaways

Core thesis: Sean Goedecke argues against the popular notion that “everyone talking to the same model gets the same results.” Instead, he claims domain expertise is the most important variable in LLM output quality — and that this gap will persist as models improve.

The Terence Tao example: Goedecke points to Tao’s public conversation with ChatGPT about a counterexample to the Jacobian Conjecture. Tao’s prompting style — short messages, pushback framed as “this looks more complex than expected” rather than direct correction, and rarely taking the model’s suggested next steps — produces dramatically better output than an amateur asking the same model about the same topic. The model shifts into “talking-to-mathematicians” mode simply by detecting Tao’s fluency.

Why expertise matters more than prompting technique: You can’t mimic Tao’s style without his math knowledge underneath it. The skill isn’t the phrasing — it’s knowing what “looks wrong,” which idea to extract from a wall of model output, and which alternate formulation to suggest. He draws a parallel to his own work as an engineer: familiarity with a specific codebase lets him say “don’t we already do X?” or “I think it could be simpler here,” pushing the LLM much harder than generic system-design knowledge would.