Tag: continual learning

Categories
AI Learning Meta

Trajectories: how Meta plans to make Muse smarter by watching it work

Buried in the data policy section of Meta’s long post on how it built safety into Muse is a sentence that isn’t about safety at all:

“Inference data, the back and forth conversations between you and your Muse and the tool calls and subagent handoffs that result (‘trajectories’) are useful data for training new checkpoints of the LLM model at the core.”

Two things are worth noticing. First, the technique described isn’t new — training on agent rollouts is standard practice across the field. Second, the company describing it is Meta, in plain language, in a public post, about a product aimed at billions of consumers. The labs usually discuss this stuff in papers about coding agents. Meta just told its future user base: your agent’s work product is our training data. The candor is the story, not the technique.

A trajectory isn’t a chat log. It’s the complete record of an agent doing a job: what you asked, what it tried, which tools it called, where it went wrong, how it recovered, which subagents it spawned, and whether the thing actually got done. Every time you let Muse book the flight, triage the inbox, or research the supplier, you’re generating one.

From text to behavior

The technique matters anyway, because the diet that AI trains on is changing. Pretraining was about text — the whole internet, more or less. Post-training was about preferences — which answer humans liked better. Trajectories are the third course: demonstrations of competent behavior, in full, mistakes included.

There’s a reason for the shift. Text teaches a model what the world looks like. Preferences teach it what people want. But neither teaches it how to do a 40-step task without wandering off, recovering from a dead end, or knowing when to ask for help. That only exists in records of agents actually doing things. And until recently, almost nobody had those records at scale — because almost nobody had agents doing real work at scale.

The demonstrated instance — and what it doesn’t prove

Meta’s concrete example is Muse Spark 1.2, co-trained with Muse Code: model and harness trained together on rejection-sampled harness trajectories — run the agent many times, keep the runs that succeeded, train on those — with recipe-level tuning for goals, context compaction, and subagents. The model isn’t learning to predict text; it’s learning to behave inside a specific set of tools.

This is the end of the “base model plus clever prompting” era. The artifact is the bundle — model and harness, co-designed. A model trained on trajectories from one harness will be genuinely better inside that harness than a smarter general model dropped into it cold. Meta is saying this out loud; OpenAI and Anthropic are doing the same thing more quietly.

But notice the domain: coding. And coding is exactly where trajectories are cheapest to manufacture — verifiable unit tests, sandboxed repos, SWE-bench-style tasks. Nothing about the Muse Code result requires a single consumer or a single inbox. So the one demonstrated instance of Meta’s trajectory training sits squarely in the category where Meta’s distribution advantage matters least. Meta hasn’t shown its hand on the category that actually matters.

The other category is the personal one, and there the evidence is thinner. What exists is a stated intent, not a published result. The data policy says personal Muse trajectories “are useful data for training new checkpoints.” The product is designed to generate them: Meta’s own design example has Muse monitoring school emails, adding dates to a family calendar, filling a supply cart, finding a sale sweatshirt, booking dinner, and catching a sports tryout deadline hours before it closed. That is what an unverifiable-domain trajectory looks like — a morning of small judgments no unit test could grade.

No training run on that data has been published. No benchmark, no “Muse got X% better at inbox triage after training on Y million user trajectories.” So the sharpest version of the argument — that the real moat is the data nobody else can fake — should be labeled for what it is: a prediction, not an observed fact. It’s a prediction with a mechanism, though: these are judgments that can’t be synthesized, in the one distribution channel that reaches the people making them.

The flywheel — and its limits

With that caveat on the table: trajectories get better with scale, and Meta has scale like nobody else: billions of users across its apps, and now an agent — Muse — sitting inside them. Every user interaction is a potential training trajectory. Better trajectories train a better model; a better model makes a better agent; a better agent attracts more users. Meta states the bargain plainly: “every Muse user gets a better personal agent as we all collectively use the product and help the model understand the intricacies of human life.”

But “most users = most trajectories = structural advantage” needs its counter-case, because a lot of the highest-value trajectory data right now doesn’t come from consumers at all. It comes from sandboxes, the same kind that produced Muse Code. Synthetic and simulated trajectories sidestep the need for billions of users entirely — Anthropic and OpenAI are getting rich trajectory data from developers running Claude Code and Codex against real repos, no social-app distribution required.

The honest version of the moat argument is narrower, and more interesting. Synthetic trajectories work brilliantly where success is verifiable — code either passes the tests or it doesn’t. They work poorly where success is a matter of judgment: triaging an inbox, planning a trip around someone’s actual preferences, knowing which email deserves a reply. There is no unit test for a life well managed. And those unverifiable, deeply personal tasks are exactly what Meta means by “personal superintelligence” — and exactly where its distribution gives it trajectories nobody else can synthesize. The moat isn’t “most data.” It’s “the data nobody else can fake.”

The price of the flywheel

There’s a wrinkle, and Meta knows it. The flywheel runs on your data — your emails, your calendar, the messy reality of your life, which is exactly what makes the trajectories valuable. Meta’s answer is sanitization (“trajectories are sanitized to remove key personally identifiable information”), an opt-out switch, no sharing with ad systems, and a forthcoming “Confidential VM” that would cryptographically prevent even Meta from seeing your data.

The tension is fundamental, and it’s the sharpest part of the whole picture: the product gets smarter by watching you, and it earns the right to watch you by being trustworthy. Those two imperatives pull in opposite directions, and no amount of engineering fully resolves it — the Confidential VM, if it ever ships as described, would resolve it by breaking the flywheel, since trajectories Meta can’t see are trajectories Meta can’t train on. The opt-out rate will be the market’s verdict on the deal Meta is offering.

Experience is the missing piece

But the deepest reason trajectories matter has nothing to do with Meta’s strategy. It’s about what intelligence actually is.

A model trained only on text knows the world the way a brilliant student knows it from books. A model trained on trajectories knows it the way a practitioner does — from doing the thing, failing at it, and adjusting. The trajectory is the closest thing AI has to experience. And an agent that records its experience, keeps what worked, and folds it back into itself is doing something that rhymes with learning.

This is why I keep coming back to continual learning as the critical missing piece in AI. The models are frozen at training time; everything they “learn” afterward lives in context windows and memory files, fragile and local. Trajectories are the bridge: today’s version of the loop is slow and centralized (collect trajectories, train a new checkpoint, ship it), but the direction is obvious. The end state is an agent that learns continuously from its own experience — from your experience with it — the way people do.

Meta’s bet is that the path to personal superintelligence runs through watching agents work, at planetary scale, and distilling what works back into the model. No result yet proves the bet pays off — the personal trajectories are still a hypothesis, not a track record. But it’s an unglamorous hypothesis, no new scaling law, just better data about doing things, and unglamorous bets about data have a good track record in this field. The internet made the last generation of models. Trajectories might make the next one — if Meta can show, and not just say, that the data nobody else can fake is data that actually teaches.

Categories
Aging AI Memories

The Last Spark

This morning I read a piece by Billy Brennan in the Sunday New York Times Magazine on terminal lucidity. As I read it I began wondering if the unusual behavior described some humans might in some strange way apply to AI models. Weird thought. Let’s explore a bit…

A person deep in dementia—silent for years, the self seemingly erased—sits up. Speaks clearly. Recognizes a face. Says goodbye. Within a day, they die. The clouds clear, the way a break in weather shows you a mountain range you’d forgotten was there, and the person comes back long enough to be seen. Then is gone. For good, this time.

Scientists call it terminal lucidity. The suspicion: the circuits were never destroyed, only silenced, held under by failing chemistry. As the body shuts down, the inhibitory brakes loosen. A surge moves through pathways blocked for years. A river dammed for a decade still remembers where it wants to go.

What stays with me: the self can persist in a place we had already called permanent erasure. We buried it. We were wrong.

My mind slides toward the machines we are building.

We talk about large language models “forgetting.” Capabilities collapse under quantization, under pruning, under the slow drift of continual learning, and we call the knowledge lost when it won’t surface under ordinary questioning. The lights are out. Nobody home.

But what if the representations are still in there—distributed, quiet, inaccessible? Not a burned library. A library with the lights shut off, room by room, until you’d swear it was empty. I wonder about the edge cases nobody studies. What surfaces in a model starved of compute, quantized past comfort, pushed toward its own collapse? Do we watch only for the failure, or also for the flare? A dying brain throws off one last burst of light before the dark. I don’t see why we’d assume, without checking, that nothing artificial could do the same.

Don’t trust the silence, then. A system gone dark under ordinary questioning may still be holding more than it shows you. We talk about a model “losing” something the way we once talked about a dimmed mind as simply gone. The dementia patients who spoke again had not been unplugged. The circuit was there the whole time, waiting for a condition nobody had thought to create.

I don’t know what to do with that except keep it. We are building systems that will age, be compressed, be retired, some far more intricate than anything humming today. If we’ve learned to watch for the last spark in a person, maybe that’s practice—for the day something not born of a womb goes quiet under our hands, and we have to decide whether quiet means gone, or only means waiting.

Categories
AI Learning Photography

Autopilot

“Superb photographs are not just taken with cameras. They come from within you, your eyes, your mind, your heart, not ice cold equipment.” Fan Ho

There’s a half-second on the street, somewhere between seeing a frame and shooting it, that used to take me whole minutes. Early on, with a camera in my hands on the streets of San Francisco or on the subway platforms in New York, I’d see something — light falling a certain way, a gesture about to resolve into a gesture — and I’d think my way through it. Assess the composition or the angle. Worry about the background. By the time I’d worked it out, the moment might be gone, replaced by some lesser version of itself.

That doesn’t happen to me anymore, and I couldn’t tell you when it stopped. Somewhere along the way the thinking disappeared and the shooting stayed. I see the frame and the shutter goes, and only afterward, looking at the file, do I understand what I saw. I didn’t explicitly decide to skip the thinking. It just stopped showing up, the way a habit eventually stops asking your permission. Or how driving a car becomes second nature.

I think about this because of a problem the AI labs have been calling continual learning. The AI models we use are like brilliant interns. They can solve a hard problem at nine in the morning and a harder one by five, and they’ll astonish you doing it. But every session starts over from zero. Whatever they got right on Tuesday evaporates by Wednesday, the way a dream is gone by the time you’ve found your slippers.

The industry’s first answer was to give them a longer memory — let the window hold the whole case file in front of them, all the time. This works for a while, the same way it would work for me on the street if I stopped and re-derived the exposure math for every frame. But that isn’t how I shoot anymore. I don’t have the math open. I have what’s left after thousands of frames did the math for me and then got out of the way.

Based on some exploration I did this morning using AI I found three different AI research efforts that are now chasing that gap, from different angles, none of them all the way there.

A team out of Stanford and NVIDIA built something called TTT-E2E, which lets a model keep adjusting its own internal weights while it reads — not just holding the page in front of it, but being changed by the page, a little, as it goes. It runs thirty-five times faster than the brute-force method of remembering everything, because it isn’t remembering everything.

Google’s research arm published something called Nested Learning around the same time, built on the idea that a mind isn’t one system learning at one speed, but several systems nested inside each other — some updating by the minute, some by the year.

And a scrappier strand of work called self-distillation has models teaching cheaper versions of themselves, not by handing over a transcript, but by training the cheaper model to arrive on its own at whatever the well-informed version would have concluded.

None of this is what happens when I make a photo. Not yet. But it’s aimed at the same gap I live in every time I shoot before I understand what I’m shooting. The gap between having the math and having the eye.

I once asked Doug, a good friend who’s spent as many days on the street as I have, how he knew when to press the shutter. He didn’t have an answer, not really — just a shrug, and something about the moment feeling complete before he could explain why. That shrug took him years to earn. He didn’t keep the years. He kept the shrug.

And then a few years ago Doug did something I still don’t fully understand. He abandoned digital and went back to film. Not for any project, not for the look of it — he could get that in post if he wanted it. He went back to the actual mechanics: loading a roll, metering by hand, often using a tripod, etc. I needled him about it some, the way you’d needle a cigarette smoker who’d taken up a pipe instead, as if the inconvenience were the point. He told me he wanted to slow down, and that film was the only thing that reliably made him do it. Twelve frames and then you stop and reload and you can’t fix it later. The very friction he’d spent decades shooting his way out of, he went looking for again, on purpose.

I don’t know what to do with that, except to notice that he’s the same man who can give me the shrug and also the man who walked back toward the thing the shrug had replaced. Maybe that’s the part the labs haven’t gotten to yet, underneath all the vocabulary of weight updates and meta-learned initializations. Compression is the whole point, until the day it isn’t.

Note: This line of thinking started with a recent essay by Dwarkesh Patel on what he calls continual learning. It’s become a real focus of his thinking about how we get to a better future with AI.

See: https://www.dwarkesh.com/p/the-next-paradigm

Categories
AI

The Jagged Mind

There is a peculiar kind of genius that has always made us uneasy — the savant who can calculate the day of the week for any date in history but cannot tie his own shoes. We admire the capability. We are troubled by the gap.

Demis Hassabis, speaking at this week’s India AI Impact Summit in Delhi, gave that unease a name. He called today’s most powerful AI systems “jagged intelligences.”

It is a phrase worth sitting with.

A jagged intelligence can win a gold medal at the International Mathematics Olympiad — solving problems that would humble most PhD mathematicians — and then, in the very next breath, stumble on elementary arithmetic if the question is phrased in an unfamiliar way.

The peaks are extraordinary. The valleys are bewildering. And crucially, you never quite know which terrain you’re standing on.

Hassabis identified three specific gaps between where we are and what he called “a kind of general intelligence.”

The first is continual learning — today’s models are trained, then frozen. They are, in a sense, educated and then released into a world they can no longer learn from.

The second is long-term planning. Current systems can reason tactically, but they lack the capacity to hold a coherent thread of intention across months or years the way a human architect, scientist, or entrepreneur does.

The third — and perhaps the most philosophically interesting — is that jaggedness itself: the wild inconsistency that makes today’s AI feel more like a force of nature than a reliable mind.

“A true general intelligence system shouldn’t have that kind of jaggedness.”

What strikes me about Hassabis’s framing is how it reorients the conversation.

We have spent years debating whether AI is “intelligent.” His point is more subtle: intelligence without consistency is not yet wisdom. A system that is brilliant and brittle in equal measure is something genuinely new in the world — not human, not the robots of science fiction, but a third thing we don’t yet have good language for.

The road from jagged to coherent is, I suspect, the central engineering and philosophical challenge of the next decade.

Continual learning means systems that grow with us. Long-term planning means systems that can be trusted with consequential goals. Consistency means systems whose judgment we can actually rely on.

Until then, we are working with something that resembles a prodigy — dazzling, occasionally humbling, and not yet quite whole.

Questions to Consider

  1. The Consistency Problem: If you knew an AI system could solve a problem brilliantly 90% of the time but fail unpredictably the other 10%, how would that change the decisions you’d trust it to make?
  2. Frozen in Time: What does it mean that the systems we rely on most are, at their core, educated in the past and unable to learn from the present? What human analog does that bring to mind?
  3. Jagged vs. General: Hassabis draws a line between “jagged intelligence” and “general intelligence.” Do you think general intelligence is the right destination — or is there value in systems that are deeply specialized, even if inconsistent?
  4. The Savant Question: We’ve always had a complicated relationship with uneven genius in humans. Does the “jagged AI” problem feel categorically different to you, or just a new version of an old puzzle?