Tag: epistemology

Categories
Science

The Whiteness of the Lather

“Consider a bar of soap. Has it ever struck you that soap lather is always white no matter what color the soap is? That isn’t because the soap somehow changes color when it is moistened and rubbed. Molecularly, it’s exactly as it was before. It’s just that the foam reflects light in a different way. You get the same effect with crashing waves on a beach—greeny-blue water, white foam—and lots of other phenomena. That is because color isn’t a fixed reality but a perception.” (Bill Bryson, The Body)

I have been thinking about soap. Not the smell of it, not the brand, not the dish it sits in by the sink, but its color, and what becomes of that color the moment the soap is wet and worked between the hands. However dark the bar the foam that comes off it is white. It is always white.

The soap has not changed. This is the part that unsettles me a little, still, though I understand the physics of it well enough to explain it to a child. Nothing has been dissolved out of the soap, no pigment washed down the drain with the first rinse. The bar, examined afterward, is exactly what it was — same weight, same tint, same faint milled scent. What has changed is only the shape of the surface presented to the light. A solid surface reflects color; a foam of small bubbles scatters light in every direction at once, and the scattering reads, to the eye, as white. The water does this too, at the shoreline, where the sea that looked green or the color of a bruise all the way to the horizon turns to white foam the instant it breaks against the sand. Nothing in the ocean has been added or subtracted. Only its surface has been shattered into a million small mirrors, and a million small mirrors, taken together, show us nothing but the light itself.

There is a machine built on the same principle, aimed at the opposite end. A stealth aircraft is not invisible in the way a child means invisible — it has not been made transparent, or absent, or small. It has been shaped. Every facet of the old F-117, every curved plate of the B-2, is angled to take an incoming radar signal and send it away, off at some oblique angle, into empty sky, anywhere but back to the dish that sent it out looking. Where the geometry alone cannot manage it, a coating finishes the job, converting the beam into a small, undetectable warmth rather than an echo. The aircraft has not changed what it is. It has changed only where its reflection goes. The operator at the radar screen sees nothing, and concludes, wrongly, that there is nothing there.

The foam and the aircraft are the same trick, pointed in opposite directions. Both break a surface’s geometry until the return signal — light, in one case, radar, in the other — stops describing the object honestly. The foam scatters light everywhere at once and reads, to the eye, as brilliant and total. The aircraft scatters radar everywhere except back to the observer and reads, to the screen, as nothing at all. One hides by flooding the return. The other hides by starving it. Neither the soap nor the aircraft has changed what it is. What has changed is the transaction between the object and whatever signal goes out to find it — and it is that transaction, not the object, that a color or a presence turns out to describe.

I do not think this makes color, or presence, less real. I think it makes real a smaller and stranger category than we had assumed. We want our perceptions to be reports from the world, filed accurately, awaiting only our attention. It is more honest, and harder to sit with, to admit that a great many of our perceptions are not reports at all but events — things that happen at the meeting point of a self and a surface, and that would not happen in quite the same way to anyone standing an inch to the left.

I rinse the soap and set it back in the dish. In a moment the foam is gone, down through the drain, and the bar is only itself again, the color it always was, waiting for the next hand to come along and change it into something briefly, purely white.

Categories
AI

The Shape of the Question

Marc Andreessen made two claims recently that don’t quite fit together, and I haven’t been able to stop pulling at the seam.

The first: for almost any topic, the top AI systems now give him better answers than the world-class experts he could call on the phone. And he can call basically anyone. This isn’t a casual observation from someone without access — it’s a meaningful data point about what AI is actually doing to the value of expertise.

The second: the only real skill left in using AI is knowing what to ask. The models can already do almost anything you can describe in plain English. The bottleneck lives in your own head.

Hold those two claims next to each other. If the AI beats the experts, then the quality of your question only has to clear a low bar — good enough to unlock what the system already knows. You don’t need to ask like a cardiologist to get a cardiologist-quality answer. You just need to ask.

Except that’s not how it works in practice. And the gap between the two claims is where something important lives.

The better the question, the better the answer — even from a system that already knows more than any human alive. Expert-level interrogation of a superhuman system produces something qualitatively different from naive interrogation of the same system. The gap between a good question and a bad one doesn’t shrink because the underlying capability grows. It may widen. A sharper instrument in an unskilled hand doesn’t close the distance — it just makes the skilled hand more lethal.

What the AI has done is commoditize answers. What it has not done — cannot do — is commoditize the ability to know which question to ask.

There is a concept from epistemology that keeps surfacing here: the unknown unknown. Donald Rumsfeld made the phrase famous and then spent years living down the mockery, which was unfair, because the underlying idea is genuinely important. There are things you know you don’t know — the gaps you can name, the questions you can form. And there are things you don’t know you don’t know — the territory you can’t even see the edge of. The naive user of AI operates almost entirely in the second category. They ask what they already suspect. They get answers that confirm the shape of what they already believe. The system is brilliant and they are using it as a mirror.

The sophisticated user has learned to ask the AI to challenge their assumptions. To find the holes. To steelman the opposing view. To identify what’s missing from the framing. That second posture requires a kind of intellectual self-awareness — an ability to stand outside your own thinking and interrogate it — that is neither common nor easily taught.

Here is the uncomfortable implication: that self-awareness is not randomly distributed. It correlates with education, with reading, with having thought carefully about hard things for a long time. The people best positioned to ask good questions are, largely, the people who already had access to good answers through the old system. The gate moved. It didn’t disappear.

There’s a democratic story told about AI and I believe parts of it. The kid in rural South Dakota with a good question now gets an answer that rivals what the partner at McKinsey gets.

But access to information was never really the binding constraint. The binding constraint was always the ability to know what information you need — to feel the shape of your own ignorance precisely enough to ask for what fills it. That skill wasn’t distributed by the old system and it won’t be distributed by the new one. It has to be built, slowly, through years of reading and thinking and being wrong and trying again.

What AI may actually be doing is widening the gap between people who ask well and people who don’t — making the former dramatically more capable while leaving the latter approximately where they were, just with a faster way to get answers to questions they already knew to ask.

Somewhere right now, someone is sitting with the most capable thinking tool in human history, asking it to write a cover letter. The tool will do it beautifully. And the gap will quietly widen.

Categories
AI Living

The Threshold

There is a specific feeling. You are trying to understand something — a medical term in a lab report, a clause in a contract, how a particular piece of software actually works under the hood — and you hit the edge of what you know. The territory beyond is unfamiliar and the path is unclear, and something in you decides, quietly and almost without announcement: I don’t know how to figure this out.

And then you move on.

Marc Andreessen, talking to Joe Rogan recently, buried something important inside a longer riff about AI prompting tricks. Most of his list was the kind of thing you’d read in a productivity newsletter — ask it to steelman both sides, pretend it’s a panel of experts. Useful, not revelatory. But one observation was different: pay attention to the exact moment you think “I don’t know how to figure this out.” That’s the moment you should open the AI.

He said it almost offhandedly. I haven’t been able to stop thinking about it.

What he’s really describing isn’t a technique. It’s a behavioral pattern that most of us developed so gradually we don’t recognize it as a choice. The feeling of epistemic overreach — of arriving at the edge of one’s competence — became, over decades, a stopping condition. We learned to treat not-knowing as a wall rather than a door because, most of the time, it functionally was one. The library was closed. The expert was unavailable. The research was paywalled. You moved on.

The habit calcified. Now it persists even when the conditions that produced it no longer apply.

I notice it in myself, and I’m someone who is genuinely curious — who likes knowing how things work, who will follow a thread further than most people bother to. That’s not modesty; it’s relevant context. Because even with that disposition, I still hit the wall. I’ll be reading something and encounter a concept I only vaguely follow — some nuance in immunology, some historical episode I’ve only half absorbed — and I feel the familiar slight contraction, the small withdrawal. I read past it. The curiosity was there. The friction was higher.

Curiosity alone was never enough. What determined whether I pushed through wasn’t how much I wanted to understand — it was whether understanding felt retrievable at all. Most of the time, it didn’t. So I moved on, and the curiosity found something else to chase.

There’s a darker version of this worth sitting with. The people who never developed the quit reflex — who hit not-knowing and felt compelled rather than defeated — are, disproportionately, the ones who built things. The intellectual persistence wasn’t incidental to their contributions; it was probably constitutive of them. Curiosity as stubbornness. The refusal to accept the wall as final.

Elon Musk is the limit case. When he decided he wanted to go to Mars and found the rockets prohibitively expensive, he didn’t defer to the aerospace industry’s consensus about what was possible. He started reading propulsion manuals and cold-calling engineers. The quit signal either never fired or got overridden so fast it made no practical difference. The result was reusable orbital rockets, which the industry had largely decided weren’t worth pursuing. The dig reflex, taken to its extreme, rewrote what was considered feasible.

But the trait is undifferentiated. It doesn’t come with a calibration mechanism. The same refusal to accept expert consensus that produced SpaceX also produces a certain amount of confident wrongness — the Twitter decisions, the Covid takes, the occasional foray into geopolitics with the certainty of someone who has read a lot of Wikipedia. The dig reflex, unregulated, has no obvious stopping condition.

AI doesn’t change that underlying trait. What it changes is the access cost for everyone else.

For most of human history, the friction wasn’t random. It selected for people whose drive was strong enough to overcome it regardless of cost — the right connections, the right institution, the time to burn. Now that friction is lower for everyone, nearly to zero, for an enormous range of questions.

What I’m trying to build is the opposite of the quit reflex. Not the Musk version — boundless, uncalibrated, occasionally catastrophic. Something more modest: the habit of checking before giving up. Noticing the moment of not-knowing and treating it as a question rather than a verdict.

It requires noticing the moment. Which is harder than it sounds, because the reflex is fast and the moment is brief.

The contraction happens. You’ve already moved on. Somewhere behind you, the question is still there.

Categories
AI Technology

The Bathwater Problem

Gary Kamiya was writing about the Tenderloin when he said it, but the line has been following me around: “The problem is that by saving the baby, you also save the bathwater.”

The pattern is remarkably consistent across every major information technology. Each one arrives promising to liberate the deserving — the faithful, the learned, the civic-minded — and each one immediately, inevitably, arms everyone else too. Gutenberg’s press was understood by its champions as a device for spreading the true Word; within decades it was the primary infrastructure for Protestant schism, Catholic counter-propaganda, astrological almanacs, and pornography. The reformers got their Bible. They also got their pamphlet wars.

The telegraph was greeted as a force for peace — shared information would make war irrational, commerce would bind nations. It also became the nervous system of commodity speculation, financial manipulation, and the first truly industrial-scale news hoaxes. The telephone: connection and the crank call, the crisis line and the threatening voice in the dark. Radio: FDR’s fireside chats and Father Coughlin. Television: Murrow taking down McCarthy, and also fifty years of manufactured consent. The internet: the largest library ever assembled and the largest sewer.

The pattern isn’t coincidental. It’s structural. Each technology expands what’s possible for human expression and coordination — and human expression and coordination contain both the noblest and the worst of us in roughly fixed proportion. The tool doesn’t change the ratio. It scales both sides of it.

What’s interesting historically is how each generation believes their technology will be different — that this time the architecture can be designed to select for the good. The internet era produced the most elaborate version of this belief: algorithmic curation would surface truth, network effects would reward quality, the wisdom of crowds would outcompete misinformation. Instead it turned out that engagement was the attractor, and outrage was the highest-engagement content. The bath got hotter.

The AI moment is the same belief system, restated with more technical sophistication. But the Kamiya line stands. You are saving a baby, and you are saving bathwater, and no one has yet designed a tub that can tell the difference.

The question isn’t whether the bathwater comes with the baby. It always does. The question is whether you turn on the tap.

Categories
AI AI: Transformers Books

The Updating Machine

Tom Chivers puts Bayes’ theorem in plain English and it sounds almost obvious: “the probability of event A, given event B, equals the probability of B given A, times the probability of A on its own, divided by the probability of B on its own.” A formula for revising what you believe when new evidence arrives. You started somewhere. Something changed. Now you believe something slightly different. Repeat.

The obvious part is the mechanics. The hard part is the loop.

Most reasoning errors I catch in myself aren’t failures of logic — they’re failures to update. I hold a view, evidence accumulates against it, and I find reasons the evidence is flawed rather than reasons the view might be.

Psychologists have a name for this: confirmation bias. But I’ve always found that label a bit too clean, like it describes a bug rather than a feature.

The prior isn’t wrong to be sticky. It represents everything you’ve learned up to this point. The problem is when it becomes load-bearing — when the prior stops being a starting position and starts being a conclusion.

“Strong opinions, loosely held” is supposed to solve this. It’s a useful phrase — it captures something true about the right posture toward your own beliefs. But in practice the second half is harder to honor than it sounds. The strong opinion gets stated, new evidence arrives, and changing your mind in public feels like losing. The “loosely held” part quietly becomes decorative.

What Bayes actually demands is something closer to epistemic humility with arithmetic attached. You don’t get to say I don’t know. You have to say I estimate 0.4, and here is what would move me to 0.6. That’s harder. It requires you to specify not just what you believe but how you’d know if you were wrong.

This is why Bayesian thinking keeps surfacing in AI conversations. Modern language models do something structurally adjacent to this — not consciously, but mechanically. Every token generated is a probability distribution revised forward by context. The model doesn’t know the next word; it updates a prior over all possible words, given everything that came before. It’s not reasoning the way humans reason, but it’s updating the way Bayes updates: continuously, contextually, without the luxury of certainty.

Whether that’s comforting or unsettling probably depends on your own prior.

The deeper thing Chivers is pointing at, I think, is that Bayesian reasoning is essentially a description of intellectual honesty as a process rather than a trait. You can’t just decide to be open-minded. You have to build the loop: form a belief, assign it a probability, watch for evidence that should move it, and then actually move it. Most of us do the first three. The fourth step is where it gets expensive.

I’ve been wrong about enough things by now that I’ve started to treat my own confident views with mild suspicion. Not paralysis — you have to act on something — but a background awareness that the prior I’m acting on was formed by a person who had less information than I do now, and less than I’ll have next year.

Strong opinions, loosely held, sounds right. The trick is meaning it.