Categories
AI Aviation

Buffer Overflow

There is a moment in a stall, before the airplane actually stalls, when the controls go soft. The yoke stops talking back. You can still pull it toward you, and the nose will still come up, but the airplane is no longer answering in the language it used thirty seconds earlier, and if you do not recognize the change in dialect you will keep asking questions in a tongue the airplane has stopped speaking. Pilots have a phrase for the general condition this belongs to, which is broader than stalls and covers weather, traffic, radio calls, checklists, an unfamiliar airport with three runways and no tower: getting behind the airplane. The airplane is still flying. It is you who have stopped keeping pace with what it is doing.

I flew a Cherokee 235 for years, a airplane with enough useful load to make it forgiving and enough control weight to make it honest, and I only got behind it twice that I can remember with any precision, both times on approach, both times because I let a secondary task — a frequency change, a passenger question, a glance at a chart — eat the attention that the airplane needed at exactly the moment it needed it most. What is strange, looking back, is that the airplane never sped up. The airplane was doing what it always does on a three-degree glide path. I was the one who fell behind a constant.

I have started to notice the same falling-behind, unrelated to constants, in conversations with a language model.

It happens on the good days, which is the part that took me a while to understand. It is not the model being slow or confused. It is the model being unusually generative — pulling a thread from something I said four exchanges ago, connecting it to a domain I had not mentioned, offering three candidate framings where I had expected one — and somewhere in the second or third of these, I notice that I have stopped actually absorbing and started merely receiving. The words are still arriving. I have quietly stopped being the kind of reader who can do anything with them.

The name I have for this, mostly because I spent some years around fraud systems and payments infrastructure and the vocabulary never entirely leaves you, is buffer overflow. In a computer, a buffer is a fixed patch of memory set aside to hold data until a program is ready to process it — a loading dock, essentially, sized for a delivery truck of a known dimension. A buffer overflow is what happens when the truck backs in and keeps unloading past the edge of the dock. The classic and dangerous version of this is not that the extra data spills onto the floor and is lost. It is that the extra data lands on the memory sitting just past the dock, and overwrites whatever was stored there — a return address, a variable, something the program needed intact to know where to go next. The failure is not loss. It is corruption. The fifth insight does not politely fall away; it lands on top of the second insight and changes what the second insight was.

This is, I think, the more accurate complaint than “overwhelm,” which is the word I would have reached for a few years ago and which suggests simple excess, more water than the glass can hold. What I am describing is not excess. It is a rate mismatch between generation and integration, and the damage happens specifically at the boundary — not in the ideas that never arrived, but in the ones that arrived and were still being turned over when the next one came in and knocked them loose.

Aviation, as it turns out, has more than one name for this family of failure, and the names are not redundant, because they describe different mechanisms. Task saturation is the CRM term — Crew Resource Management, the discipline built in the seventies and eighties largely in response to accidents where a competent, rested, well-trained crew flew a functioning airplane into terrain because attention had been consumed by something lower priority than staying alive. Task saturation is measured, in training, less by how much is happening and more by whether the pilot can still prioritize — whether they know which thing to drop. Channelized attention is the adjacent and opposite failure: not too many things competing for a narrow channel, but one thing filling it entirely, a fixation on the landing gear light while the airplane, unflown, descends into the Everglades. And John Boyd’s OODA loop, developed for fighter pilots and stolen since by nearly every field that has ever needed a name for out-thinking someone under time pressure, describes what it feels like structurally to fall behind: you are not reacting to what the situation is, you are reacting to what the situation was, one iteration back, and every loop after that the gap does not close on its own.

I suspect what I am calling buffer overflow is closest to task saturation, with the wrinkle that in a cockpit the incoming data is at least all real-time and load-bearing — the runway is where the runway is — whereas a model in full flow is producing a mix of load-bearing insight and elaboration that only sounds load-bearing, and no light comes on to tell you which is which. Sweller’s cognitive load theory gives this a cleaner anatomy than aviation does: intrinsic load, which is the actual difficulty of the idea; extraneous load, which is how badly or well the idea is presented; and germane load, which is the effort of building the new idea into the structure of what you already know. My buffer does not overflow on intrinsic load — the ideas themselves are usually not hard. It overflows on germane load. The model can generate connections faster than I can lay the track that would let each new connection actually attach to something.

None of the aviation solutions to task saturation involve asking the airplane to slow down, and this is the part I keep returning to, because slowing down is the intervention that occurs to me first and is also, I think, the least aviation-like response available. A pilot who is task-saturated on approach does not usually ask the tower to widen the pattern. He drops something. He un-couples the autopilot from one axis and flies it by hand so the workload becomes tactile instead of cognitive, or he tells the passenger the question will have to wait, or he reads back only the clearance and lets the weather advisory go unacknowledged for ninety seconds because the weather advisory is not what is going to kill him in the next ninety seconds. The skill is not deceleration. It is triage performed at full speed, which looks, from outside the cockpit, indistinguishable from calm.

I do not yet know what the triage move is for a conversation with a model that is generating faster than I can integrate. I have a guess, which is that it looks less like asking the model to slow down and more like periodically stepping outside the exchange entirely — not to catch up on what was said, but to write down, in my own words, the one thing from the last five minutes I actually want to keep, before asking it to continue. That would make the move not deceleration but discard: choosing, the way the saturated pilot chooses, which incoming data does not get processed at all, on the theory that a buffer with something deliberately thrown out of it still holds its shape, and a buffer that tries to keep everything is the one that overflows.

Or maybe the real answer is the one the checkride examiner gave me in Springfield, on a September morning in 1978, when I came in too fast and too high and asked, afterward, what I should have done differently. He said the airplane had told me everything I needed to know about forty seconds before I noticed, and that the only skill that mattered was noticing forty seconds earlier next time. Not slower. Earlier.

Categories
AI

The Shape of the Question

Marc Andreessen made two claims recently that don’t quite fit together, and I haven’t been able to stop pulling at the seam.

The first: for almost any topic, the top AI systems now give him better answers than the world-class experts he could call on the phone. And he can call basically anyone. This isn’t a casual observation from someone without access — it’s a meaningful data point about what AI is actually doing to the value of expertise.

The second: the only real skill left in using AI is knowing what to ask. The models can already do almost anything you can describe in plain English. The bottleneck lives in your own head.

Hold those two claims next to each other. If the AI beats the experts, then the quality of your question only has to clear a low bar — good enough to unlock what the system already knows. You don’t need to ask like a cardiologist to get a cardiologist-quality answer. You just need to ask.

Except that’s not how it works in practice. And the gap between the two claims is where something important lives.

The better the question, the better the answer — even from a system that already knows more than any human alive. Expert-level interrogation of a superhuman system produces something qualitatively different from naive interrogation of the same system. The gap between a good question and a bad one doesn’t shrink because the underlying capability grows. It may widen. A sharper instrument in an unskilled hand doesn’t close the distance — it just makes the skilled hand more lethal.

What the AI has done is commoditize answers. What it has not done — cannot do — is commoditize the ability to know which question to ask.

There is a concept from epistemology that keeps surfacing here: the unknown unknown. Donald Rumsfeld made the phrase famous and then spent years living down the mockery, which was unfair, because the underlying idea is genuinely important. There are things you know you don’t know — the gaps you can name, the questions you can form. And there are things you don’t know you don’t know — the territory you can’t even see the edge of. The naive user of AI operates almost entirely in the second category. They ask what they already suspect. They get answers that confirm the shape of what they already believe. The system is brilliant and they are using it as a mirror.

The sophisticated user has learned to ask the AI to challenge their assumptions. To find the holes. To steelman the opposing view. To identify what’s missing from the framing. That second posture requires a kind of intellectual self-awareness — an ability to stand outside your own thinking and interrogate it — that is neither common nor easily taught.

Here is the uncomfortable implication: that self-awareness is not randomly distributed. It correlates with education, with reading, with having thought carefully about hard things for a long time. The people best positioned to ask good questions are, largely, the people who already had access to good answers through the old system. The gate moved. It didn’t disappear.

There’s a democratic story told about AI and I believe parts of it. The kid in rural South Dakota with a good question now gets an answer that rivals what the partner at McKinsey gets.

But access to information was never really the binding constraint. The binding constraint was always the ability to know what information you need — to feel the shape of your own ignorance precisely enough to ask for what fills it. That skill wasn’t distributed by the old system and it won’t be distributed by the new one. It has to be built, slowly, through years of reading and thinking and being wrong and trying again.

What AI may actually be doing is widening the gap between people who ask well and people who don’t — making the former dramatically more capable while leaving the latter approximately where they were, just with a faster way to get answers to questions they already knew to ask.

Somewhere right now, someone is sitting with the most capable thinking tool in human history, asking it to write a cover letter. The tool will do it beautifully. And the gap will quietly widen.

Categories
AI Living

The Threshold

There is a specific feeling. You are trying to understand something — a medical term in a lab report, a clause in a contract, how a particular piece of software actually works under the hood — and you hit the edge of what you know. The territory beyond is unfamiliar and the path is unclear, and something in you decides, quietly and almost without announcement: I don’t know how to figure this out.

And then you move on.

Marc Andreessen, talking to Joe Rogan recently, buried something important inside a longer riff about AI prompting tricks. Most of his list was the kind of thing you’d read in a productivity newsletter — ask it to steelman both sides, pretend it’s a panel of experts. Useful, not revelatory. But one observation was different: pay attention to the exact moment you think “I don’t know how to figure this out.” That’s the moment you should open the AI.

He said it almost offhandedly. I haven’t been able to stop thinking about it.

What he’s really describing isn’t a technique. It’s a behavioral pattern that most of us developed so gradually we don’t recognize it as a choice. The feeling of epistemic overreach — of arriving at the edge of one’s competence — became, over decades, a stopping condition. We learned to treat not-knowing as a wall rather than a door because, most of the time, it functionally was one. The library was closed. The expert was unavailable. The research was paywalled. You moved on.

The habit calcified. Now it persists even when the conditions that produced it no longer apply.

I notice it in myself, and I’m someone who is genuinely curious — who likes knowing how things work, who will follow a thread further than most people bother to. That’s not modesty; it’s relevant context. Because even with that disposition, I still hit the wall. I’ll be reading something and encounter a concept I only vaguely follow — some nuance in immunology, some historical episode I’ve only half absorbed — and I feel the familiar slight contraction, the small withdrawal. I read past it. The curiosity was there. The friction was higher.

Curiosity alone was never enough. What determined whether I pushed through wasn’t how much I wanted to understand — it was whether understanding felt retrievable at all. Most of the time, it didn’t. So I moved on, and the curiosity found something else to chase.

There’s a darker version of this worth sitting with. The people who never developed the quit reflex — who hit not-knowing and felt compelled rather than defeated — are, disproportionately, the ones who built things. The intellectual persistence wasn’t incidental to their contributions; it was probably constitutive of them. Curiosity as stubbornness. The refusal to accept the wall as final.

Elon Musk is the limit case. When he decided he wanted to go to Mars and found the rockets prohibitively expensive, he didn’t defer to the aerospace industry’s consensus about what was possible. He started reading propulsion manuals and cold-calling engineers. The quit signal either never fired or got overridden so fast it made no practical difference. The result was reusable orbital rockets, which the industry had largely decided weren’t worth pursuing. The dig reflex, taken to its extreme, rewrote what was considered feasible.

But the trait is undifferentiated. It doesn’t come with a calibration mechanism. The same refusal to accept expert consensus that produced SpaceX also produces a certain amount of confident wrongness — the Twitter decisions, the Covid takes, the occasional foray into geopolitics with the certainty of someone who has read a lot of Wikipedia. The dig reflex, unregulated, has no obvious stopping condition.

AI doesn’t change that underlying trait. What it changes is the access cost for everyone else.

For most of human history, the friction wasn’t random. It selected for people whose drive was strong enough to overcome it regardless of cost — the right connections, the right institution, the time to burn. Now that friction is lower for everyone, nearly to zero, for an enormous range of questions.

What I’m trying to build is the opposite of the quit reflex. Not the Musk version — boundless, uncalibrated, occasionally catastrophic. Something more modest: the habit of checking before giving up. Noticing the moment of not-knowing and treating it as a question rather than a verdict.

It requires noticing the moment. Which is harder than it sounds, because the reflex is fast and the moment is brief.

The contraction happens. You’ve already moved on. Somewhere behind you, the question is still there.

Categories
AI Thinking Tools

Outsourcing Thinking but not Understanding

There’s a line mentioned in a recent discussion by Andrej Karpathy that I keep turning over: You can outsource your thinking but you can’t outsource your understanding.

It sounds like a warning. Maybe it is. But the more I sit with it, the more it feels like something older — a distinction philosophers have been trying to draw for centuries, suddenly made urgent by the fact that we now have a tool that makes outsourcing thinking almost frictionless.

Here’s what I notice when I use AI well: I get the answer, and I feel satisfied. There’s a small dopamine tick. Task closed. But if someone asks me an hour later to explain the reasoning, I often can’t. The thinking happened — somewhere — but not in me. I was a conduit. A confident one, too, which is the dangerous part.

This is different from looking something up. When I Google a fact and paste it into a document, I know I’m borrowing. The seam is visible. But when I ask an AI to reason through a problem with me, the output arrives in first person, in fluent prose that matches my own register, and something in my brain says I worked this out. The seam disappears. That’s new. That’s the thing we don’t yet have good instincts for.

Karpathy’s deeper point is about construction. He’s a builder by temperament — his mantra, which he traces to Feynman, is that if you can’t build it, you don’t understand it. What you can’t yet construct, you merely think you understand. There are always micro-gaps in your knowledge, invisible until you try to arrange the pieces yourself and find they don’t quite fit. The AI doesn’t change that equation. It just makes it easier to mistake the map for the territory — and to feel strangely proud of a map you didn’t draw.

Hesse understood this, in a different century and a different idiom. In Siddhartha, the young seeker travels to meet the Buddha himself — the most perfectly articulated wisdom in the world, delivered by the man who actually found it. Siddhartha listens, acknowledges that the teaching is flawless, internally consistent, the most complete account of liberation ever assembled. And then walks away. Not from arrogance, but from recognition: even the Illustrious One cannot hand you his liberation. The path was his. He walked it. That walking is not transferable, no matter how perfect the description of the destination. Received knowledge, however exquisite, is not the same as earned knowledge. The gap between them is exactly the size of your own unlived experience.

That’s the same argument, made across two and a half millennia. Feynman says you have to build it. Hesse says you have to live it. Karpathy says the AI can do neither for you.

He’s also made a related observation about educational video — that a lot of content on YouTube gives the appearance of learning but is really just entertainment, convenient for everyone involved. Nobody has to do the hard part. AI-assisted thinking has the same shape, just more intimate. You’re not passively watching — you’re actively typing, prompting, engaging. It feels like cognition. But engagement isn’t understanding. Typing a question is not the same as wrestling with it.

I don’t think the answer is to use AI less. That’s not Karpathy’s argument either — he’s spent the last year building a school premised on AI tutors expanding what people can learn. The lesson is about custody. When I hand a problem to an AI, I need to stay in the loop as a learner, not just as a reviewer. There’s a real difference between asking give me an answer and asking help me build the reasoning. The first outsources thinking. The second — if you insist on it, if you refuse to be a passenger — can still leave the understanding in you, where it belongs.

But insisting is the work. And the work is now easier to skip than it has ever been.

Understanding isn’t a product you receive. It’s a residue — what settles in you after genuine struggle, after the confusion and the dead ends and the small hard-won moments of clarity. Siddhartha couldn’t get it from the Buddha. You can’t get it from the AI. Karpathy’s line is a custody argument: the thinking can travel, but the understanding has to stay home.

What unsettles me is that we’re building tools that make the borrowing invisible — that dress outsourced reasoning in the first person, that let us feel like we’ve understood something we’ve only processed. Siddhartha at least knew he was walking away from the teaching. He felt the gap. We might not even notice ours.