Tag: domain expertise

Categories
AI Business Software

Ice Rinks

Nobody in the Valley thinks about ice rinks.

That line sat with me longer than the rest of a conversation between Patrick Collison and Amjad Masad. Collison runs Stripe. Masad runs Replit. Someone asked, more or less, where Collison would look if he were starting over. He did not name an AI lab. He named the software nobody fashionable wants to touch.

I have heard versions of this advice for years. Vertical SaaS. Boring industries. Schlep blindness. What felt new was the timing. The distance from an idea to the first dollar has roughly halved. On Stripe’s numbers, the time to reach a million, ten million, even a hundred million in recurring revenue is about half what it was in the last SaaS boom. Twenty percent of new startups now charge a customer inside thirty days, up from eight percent in 2020. New business creation on the platform nearly doubled in a year, a bigger jump than the pandemic spike.

When building the thing was the scarce skill, the programmer won. When a working product can be stood up quickly, the scarce skill is knowing why the old thing is still tolerated. The domain expert gets the edge. A teacher who understood schools built MagicSchool on Replit and rode it toward a company later described as worth half a billion dollars. The insight was the teacher’s. The tools closed the gap.

Collison’s map has three drawers.

Continue reading “Ice Rinks”
Categories
AI

LLMs Reward Expertise — Key Takeaways

Core thesis: Sean Goedecke argues against the popular notion that “everyone talking to the same model gets the same results.” Instead, he claims domain expertise is the most important variable in LLM output quality — and that this gap will persist as models improve.

The Terence Tao example: Goedecke points to Tao’s public conversation with ChatGPT about a counterexample to the Jacobian Conjecture. Tao’s prompting style — short messages, pushback framed as “this looks more complex than expected” rather than direct correction, and rarely taking the model’s suggested next steps — produces dramatically better output than an amateur asking the same model about the same topic. The model shifts into “talking-to-mathematicians” mode simply by detecting Tao’s fluency.

Why expertise matters more than prompting technique: You can’t mimic Tao’s style without his math knowledge underneath it. The skill isn’t the phrasing — it’s knowing what “looks wrong,” which idea to extract from a wall of model output, and which alternate formulation to suggest. He draws a parallel to his own work as an engineer: familiarity with a specific codebase lets him say “don’t we already do X?” or “I think it could be simpler here,” pushing the LLM much harder than generic system-design knowledge would.

Continue reading “LLMs Reward Expertise — Key Takeaways”
Categories
AI

The Taste Beneath the Summary

The real work of staying informed has never been volume. It has been the quiet, repeated acts of judgment: does this matter, to whom, why now, what is the signal beneath the noise.

A recent piece from Bridgewater’s AIA Labs and Thinking Machines Lab, “Learning to Replicate Expert Judgment in Financial Tasks,” describes training models to do the triage investors actually do—filtering news, research, central bank documents, internal notes, for relevance. Frontier models struggled with judgments that looked simple and weren’t. The fix wasn’t a bigger model. It was Qwen, fine-tuned on labeled examples from practitioners, and it beat the frontier leaders while costing a fraction to run.

The bottleneck was never model size. It was taste. And taste, it turns out, can be taught to something small and cheap, if you’re precise enough about what you’re teaching it—a market’s worth of Mercors is already proving the same thing at scale.

The researchers were clear that expert judgment doesn’t reduce to rules or prompts. It took high-quality, domain-specific labels from people doing the actual work. The most powerful systems will be built in partnership with practitioners who can say, and keep saying, what “good” looks like in their own context.

Which raises the question I haven’t answered yet: what would I actually put in the labels, if someone asked me to teach my own taste to a cheap model.