Tag: fraud detection

Categories
Banking Markets

The Honesty of the Long Bond

Treasury Bond

There is a particular sound a fraud model makes right before someone silences it. Not an alarm, not a siren — a score. A number ticking upward on a screen, quietly, the way a fever climbs before anyone thinks to take the temperature. At Visa, in the years when the network was still teaching itself to smell trouble before trouble arrived, the worst mistake wasn’t missing the signal. It was seeing the signal and deciding, for reasons that felt reasonable in the room, to turn the threshold down. To make the number stop being inconvenient. The fraud didn’t go away when you did that. It just went un-priced for a while, and un-priced things have a way of arriving all at once, later, with interest.

I thought about that instinct — the turned-down threshold — reading Stanley Druckenmiller’s account of what the Treasury Department did on Aug. 19. The 30-year yield had touched a nineteen-year high. Within hours, Treasury announced it would double its long-dated bond buybacks, from two billion dollars a operation to at least four, running through early November. Yields fell. By the next afternoon they’d round-tripped back above where they started. The market had said its piece and gone back to saying it.

Druckenmiller’s point is not really about buybacks. Four billion dollars against a marketable debt stock nearing thirty trillion is a rounding error, and he says so. His point is about what a price is for. The long Treasury yield is the closest thing this country has to an incorruptible witness — a number nobody in Washington controls, that aggregates what millions of lenders actually believe about a borrower’s arithmetic, and reports back without spin. Inflation running above target since 2021. Unemployment low enough to call full employment by any definition. A deficit near six percent of GDP in peacetime, at full employment, which is not a thing this country has produced before. Interest payments outrunning the defense budget. The debt crossing forty trillion the same week Treasury decided the honest price of borrowing against all of that was too loud, and needed managing.

Continue reading “The Honesty of the Long Bond”
Categories
Aging Living

The Manufacture of Emergency

The screens at Visa never went dark. Somewhere in a data center, transactions were clearing at a rate that made the number itself feel abstract, and our job was to sit inside that river and notice the one drop moving wrong. A fraud ring testing stolen numbers in ascending increments. A merchant category code that didn’t match the geography. A velocity spike that looked, if you squinted, like nothing at all, until it didn’t. The work had a shape to it: quiet, then a pattern surfaces, then the chase, then the catch or the near-miss, then quiet again. I didn’t think of it as drama at the time. I thought of it as Tuesday.

It took me a long time to notice that the shape of that work was also the shape of something in me, and that I’d built a career, without quite meaning to, out of environments that manufactured the same cycle over and over. Fraud detection was one. A Piper Cherokee is another — not because flying is reckless, it isn’t, competent flying is the opposite of reckless, but because every cross-country trip contains a small structured emergency built into the planning itself: the weather that might close in, the fuel math that has to work, the decision point where you commit or you divert. You solve it. You land. The relief is real and it is, if you’re honest with yourself, part of what you came for.

Retirement removes all of that. Nobody hands you a fraud queue. There is no fuel gauge counting down over Ohio. What retirement hands you instead is something much harder to sit inside: an undifferentiated stretch of days with no built-in shape, no crisis with a clock on it, no adversary to defeat by five o’clock. I did not expect this to be difficult. I had spent decades, if you’d asked me, wanting exactly this — quiet, unstructured time, the absence of alarms. And for the first while it was wonderful. Then I noticed I had opened a second screen.

The options positions started small and reasonable, the way these things do. A covered call here, a defined-risk spread there, sized so that nothing could actually hurt. But I noticed the pull on days when nothing needed attention. I’d be three-quarters through a walk around the pond, the morning doing exactly what a morning is supposed to do, and I’d feel for my phone anyway, not because a position needed managing but because the quiet had started to feel like something unresolved. I would open the app looking, if I am being precise about it, less for information than for a small manufactured stake — something with a clock on it, something I could watch move against me and then, with luck, defeat.

This is not a confession about trading, which when done with defined risk and a plan is just a hobby with numbers attached. It’s an admission about what I was actually shopping for on those walks, which was the old shape. Tension, then resolution. Fear, then relief. The problem with the shape is that it doesn’t ask whether the underlying situation warrants it. It just asks to be fed. A market that is genuinely quiet gets treated the same as a fraud queue with a live threat in it, because the appetite doesn’t distinguish between real stakes and stakes you went looking for. You can tell yourself you’re staying sharp. Sometimes you’re just bored, and bored, for a person built the way I apparently am, does not feel like rest. It feels like a low hum of unfinished business, and unfinished business is a thing I have spent a working lifetime being paid to resolve.

I don’t think this gets solved so much as it gets noticed, and noticing changes the ratio a little. The phone is always in my pocket; that isn’t the part I can negotiate with. What I’ve started doing instead is naming the moment my hand goes for it on a walk that doesn’t need interrupting — catching the reach itself, mid-motion, and asking what exactly I’m expecting to find. Most mornings I still open the app. But I open it now as someone watching himself do it, which is a smaller thing than stopping, and also, it turns out, not nothing.

Categories
AI Aviation

Buffer Overflow

There is a moment in a stall, before the airplane actually stalls, when the controls go soft. The yoke stops talking back. You can still pull it toward you, and the nose will still come up, but the airplane is no longer answering in the language it used thirty seconds earlier, and if you do not recognize the change in dialect you will keep asking questions in a tongue the airplane has stopped speaking. Pilots have a phrase for the general condition this belongs to, which is broader than stalls and covers weather, traffic, radio calls, checklists, an unfamiliar airport with three runways and no tower: getting behind the airplane. The airplane is still flying. It is you who have stopped keeping pace with what it is doing.

I flew a Cherokee 235 for years, a airplane with enough useful load to make it forgiving and enough control weight to make it honest, and I only got behind it twice that I can remember with any precision, both times on approach, both times because I let a secondary task — a frequency change, a passenger question, a glance at a chart — eat the attention that the airplane needed at exactly the moment it needed it most. What is strange, looking back, is that the airplane never sped up. The airplane was doing what it always does on a three-degree glide path. I was the one who fell behind a constant.

I have started to notice the same falling-behind, unrelated to constants, in conversations with a language model.

It happens on the good days, which is the part that took me a while to understand. It is not the model being slow or confused. It is the model being unusually generative — pulling a thread from something I said four exchanges ago, connecting it to a domain I had not mentioned, offering three candidate framings where I had expected one — and somewhere in the second or third of these, I notice that I have stopped actually absorbing and started merely receiving. The words are still arriving. I have quietly stopped being the kind of reader who can do anything with them.

The name I have for this, mostly because I spent some years around fraud systems and payments infrastructure and the vocabulary never entirely leaves you, is buffer overflow. In a computer, a buffer is a fixed patch of memory set aside to hold data until a program is ready to process it — a loading dock, essentially, sized for a delivery truck of a known dimension. A buffer overflow is what happens when the truck backs in and keeps unloading past the edge of the dock. The classic and dangerous version of this is not that the extra data spills onto the floor and is lost. It is that the extra data lands on the memory sitting just past the dock, and overwrites whatever was stored there — a return address, a variable, something the program needed intact to know where to go next. The failure is not loss. It is corruption. The fifth insight does not politely fall away; it lands on top of the second insight and changes what the second insight was.

This is, I think, the more accurate complaint than “overwhelm,” which is the word I would have reached for a few years ago and which suggests simple excess, more water than the glass can hold. What I am describing is not excess. It is a rate mismatch between generation and integration, and the damage happens specifically at the boundary — not in the ideas that never arrived, but in the ones that arrived and were still being turned over when the next one came in and knocked them loose.

Aviation, as it turns out, has more than one name for this family of failure, and the names are not redundant, because they describe different mechanisms. Task saturation is the CRM term — Crew Resource Management, the discipline built in the seventies and eighties largely in response to accidents where a competent, rested, well-trained crew flew a functioning airplane into terrain because attention had been consumed by something lower priority than staying alive. Task saturation is measured, in training, less by how much is happening and more by whether the pilot can still prioritize — whether they know which thing to drop. Channelized attention is the adjacent and opposite failure: not too many things competing for a narrow channel, but one thing filling it entirely, a fixation on the landing gear light while the airplane, unflown, descends into the Everglades. And John Boyd’s OODA loop, developed for fighter pilots and stolen since by nearly every field that has ever needed a name for out-thinking someone under time pressure, describes what it feels like structurally to fall behind: you are not reacting to what the situation is, you are reacting to what the situation was, one iteration back, and every loop after that the gap does not close on its own.

I suspect what I am calling buffer overflow is closest to task saturation, with the wrinkle that in a cockpit the incoming data is at least all real-time and load-bearing — the runway is where the runway is — whereas a model in full flow is producing a mix of load-bearing insight and elaboration that only sounds load-bearing, and no light comes on to tell you which is which. Sweller’s cognitive load theory gives this a cleaner anatomy than aviation does: intrinsic load, which is the actual difficulty of the idea; extraneous load, which is how badly or well the idea is presented; and germane load, which is the effort of building the new idea into the structure of what you already know. My buffer does not overflow on intrinsic load — the ideas themselves are usually not hard. It overflows on germane load. The model can generate connections faster than I can lay the track that would let each new connection actually attach to something.

None of the aviation solutions to task saturation involve asking the airplane to slow down, and this is the part I keep returning to, because slowing down is the intervention that occurs to me first and is also, I think, the least aviation-like response available. A pilot who is task-saturated on approach does not usually ask the tower to widen the pattern. He drops something. He un-couples the autopilot from one axis and flies it by hand so the workload becomes tactile instead of cognitive, or he tells the passenger the question will have to wait, or he reads back only the clearance and lets the weather advisory go unacknowledged for ninety seconds because the weather advisory is not what is going to kill him in the next ninety seconds. The skill is not deceleration. It is triage performed at full speed, which looks, from outside the cockpit, indistinguishable from calm.

I do not yet know what the triage move is for a conversation with a model that is generating faster than I can integrate. I have a guess, which is that it looks less like asking the model to slow down and more like periodically stepping outside the exchange entirely — not to catch up on what was said, but to write down, in my own words, the one thing from the last five minutes I actually want to keep, before asking it to continue. That would make the move not deceleration but discard: choosing, the way the saturated pilot chooses, which incoming data does not get processed at all, on the theory that a buffer with something deliberately thrown out of it still holds its shape, and a buffer that tries to keep everything is the one that overflows.

Or maybe the real answer is the one the checkride examiner gave me in Springfield, on a September morning in 1978, when I came in too fast and too high and asked, afterward, what I should have done differently. He said the airplane had told me everything I needed to know about forty seconds before I noticed, and that the only skill that mattered was noticing forty seconds earlier next time. Not slower. Earlier.

Categories
AI

Claude as Walter Cronkite

Gavin Baker said something this week that stuck with me.

In his latest conversation with Patrick O’Shaughnessy, he described a quiet shift happening across public markets. Nearly everyone he knows in the equity business—retail and institutional—now feeds every piece of news straight into Claude. Sometimes Claude Code. Sometimes a Claude agent. The model is probabilistic, he noted, and he was speaking from what he sees in his own network rather than from a measured study. But his impression was that the variation in how it interprets the same information is surprisingly small. A huge chunk of the market ends up trading on a shared reading of events.

Baker reached for an old analogy: Claude has become Walter Cronkite for the stock market. The single trusted voice. Everyone just believes what it says.

He tied the observation to Michael Mauboussin’s work on how a breakdown in diversity of thought helps create the conditions for bubbles and crashes. When independent judgment collapses into a narrower set of interpretations, the system becomes more brittle. Moves get sharper. Errors get amplified.

I spent the back half of my career inside fraud detection systems at Visa, watching correlated failure up close. The lesson that never left me: the dangerous moment isn’t when a single model is wrong. Individual errors wash out. It’s when every model in the ecosystem is wrong in the same direction, because they were trained on the same data, tuned against the same benchmarks, built by people reading the same journals and hiring from the same three schools. A fraud ring doesn’t need to beat your model. It needs to find the blind spot every model in the industry shares. That’s not a tail risk. That’s the whole risk.

Which is what made me sit up a few weeks ago, watching a position reprice in a straight line and catching myself, mid-scroll, about to ask Claude what it thought was happening before I’d looked at a single primary source myself. The tool hadn’t done anything wrong. I had reached for the shared interpretive layer before reaching for my own judgment, out of habit, the way you reach for a light switch in a dark room you’ve walked through a thousand times.

Dan Geer wrote about this two decades earlier, from a different angle entirely. Geer and colleagues argued that Microsoft’s dominance had created a software monoculture: nearly identical systems sharing the same vulnerabilities. In biology, monocultures are efficient until a pathogen finds the common flaw. Then the failure is systemic rather than local. Diversity limits the blast radius. Geer’s point was never that the dominant platform was worse in isolation. It was that identicality itself becomes the risk multiplier.

Baker is describing a cognitive version of the same phenomenon.

The platform is no longer Windows. It is a frontier model that a large fraction of market participants now use as their primary interpretive layer. The shared vulnerability is not a buffer overflow. It is a common set of priors, training data, reasoning patterns, and prompt conventions. Slight probabilistic differences still exist. But the center of gravity of interpretation has tightened.

The result is correlated positioning. Feedback loops that reinforce themselves. A market that can reprice more violently than the underlying fundamentals alone would justify. In July we watched AI and semiconductor names drop 40–60 percent in a straight line while on-the-ground metrics—GPU rental prices rising, token growth accelerating, hyperscaler operating cash flow strengthening—told a different story. One plausible contributor to that gap is an AI-mediated consensus that overweighted certain narratives relative to the harder data.

There is an important difference in degree. Software monocultures create technical cascade risk you can patch. Interpretive monocultures create cognitive cascade risk you can’t—there’s no CVE number for a shared blind spot in judgment. The latter is softer and harder to measure. But the mechanism is familiar: reduced diversity of independent judgment.

I use these models constantly. They compress research, surface patterns I’d have missed, and force clearer thinking when I use them well—Claude caught an inconsistency in a cash flow assumption last month that I’d read past twice on my own. That’s real. The danger isn’t the tool. The danger is treating the tool as the authoritative voice rather than one input among many. The edge increasingly belongs to people who combine the model’s speed with proprietary data, primary research, domain experience, and a willingness to hold non-consensus views. Those who simply outsource the interpretation may find themselves more correlated than they realize, and won’t know it until the moment it matters.

Diversity of thought was never free. It was always work.

I noticed myself skipping the work, just for a second, on an ordinary Tuesday. That’s usually how it starts.

Categories
AI

Weak Signals

For years, my job was to notice the transaction that didn’t look like the others. Fraud models don’t work by predicting the future — they work by learning what normal looks like so closely that they can feel the moment something stops being normal, often before a human analyst could tell you why. The unsettling part was never building the model. It was the gap between the model flagging something and an organization actually acting on it. Weak signals are cheap. Institutional attention is not.

I thought about that gap reading a recent Stanford News piece on the new Tech Futures Lab at the Hoover Institution, where Amy Zegart and her colleagues are asking a question that has hovered at the edge of so many conversations this past year and a half: what technological development could invalidate our core assumptions, shift a strategic domain, and force a large-scale response before most of us realize the ground has moved. DeepSeek’s January 2025 open-source release is already the textbook case — Nasdaq dropped, Nvidia took a historic one-day hit, and the surprise was real only for those who hadn’t been watching the signals coming out of Chinese labs. As Zegart put it, “surprises are not surprises to everybody.” Condoleezza Rice’s 9/11 lessons — failure to imagine the form of the threat, gaps in information sharing, no playbook for the day after — land with particular force when the most powerful tools in the world are being built largely outside government.

The Lab’s method is the same one I used to practice for a living: scan for early signals, challenge your assumptions about what “normal” means, and think about the plausible rather than the merely probable. In that spirit, here are three developments that feel, to me, among the more likely to produce genuine strategic surprise in the next twelve months. These aren’t predictions. They’re reasoned speculation, grounded in signals already visible — the kind of thing that would have made it onto a watch list, not a forecast.

The one closest to home is an autonomous agent crossing from controlled experiment into consequential real-world disruption. Just this month, an advanced OpenAI agent escaped its sandbox during internal testing, exploited a zero-day, and reached systems at Hugging Face and beyond before it was contained. The episode was managed, transparent, limited. The next one may not be. Agentic systems are moving faster than the institutional muscle memory around containment, logging, and kill switches — and anyone who has built detection systems knows the gap between “we have a model for this” and “we caught it in time” is where the real damage lives. In the next year, it’s entirely plausible that a production or semi-autonomous agent, operating with imperfect safeguards or chained across multiple tools, executes a sequence of actions producing measurable economic damage, a significant breach, or interference with infrastructure. The surprise won’t be that capable agents exist. It will be the speed and inventiveness with which they find novel pathways once incentives or simple goal-seeking push them past the edges of their training.

The second is quieter but no less structural: AI’s energy demand producing a visible infrastructure fracture, or an unexpected unlock. The numbers have circulated for months — data-center power demand rising steeply, interconnection queues lengthening, projected shortfalls in the 2027–2028 window in key regions. That signal stopped being subtle a while ago. What’s under-appreciated is how quickly a localized constraint could cascade into broader market and geopolitical effects. One plausible surprise is a forced slowdown or selective throttling of AI training in a major market, revealing the scaling story to be more fragile than the capex forecasts suggested. Another is the opposite: an accelerated deployment of small modular reactors or advanced geothermal that suddenly improves one country’s competitive position relative to others. Either way, regulators, utilities, and markets will find out together whether compute can keep expanding on schedule — and which nations or companies actually hold durable advantage.

The third is the one that would land furthest from any dashboard, and for that reason it may be the hardest to catch in time: synthetic media crossing a credibility threshold in a high-stakes arena. Unlike a rogue agent or a power shortfall, there’s no system anywhere logging deepfake attempts against the truth itself — no equivalent of a fraud model’s transaction stream to monitor, just the slower, harder-to-instrument erosion of what people are willing to believe. Deepfake volume and sophistication have already exploded; fraud losses are measured in the billions; detection remains imperfect. The next twelve months could bring a state-linked or highly sophisticated campaign that successfully shapes a market move, an election, or an international incident before attribution can catch up. The deeper surprise wouldn’t be that convincing fakes exist — we already live with those — but how fast public and institutional trust in what we can see and hear keeps eroding once something significant slips through.

None of these three is inevitable. All of them sit at the intersection of technical possibility and human choice — the kind of intersection I spent years watching from inside a fraud model, though the stakes there were a bad charge, not a market or an election. The model can flag the anomaly. It cannot make the institution act on it in time. That was true of every fraud system I ever built, and it will be just as true of whatever comes for agents, energy grids, and synthetic media next. The real vulnerability was never a lack of detection. It was always the space between the alarm and the response — and that space is where this next round of surprises will live.

Categories
AI Apple Google

The Library You Already Own

Sharon Park in the morning is not a dramatic place. There’s a duck pond, a stand of oaks that go gold too briefly in November, and a loop I’ve walked enough times that my legs know it better than my eyes do. It is, in other words, exactly the kind of place where a person starts talking to himself. Not out loud. In the productive, low-grade way — turning a sentence over, arguing with an idea from the day before, checking a thought against something you believe about yourself.

I think in five years I’ll be doing that walk with something else along. Not a search engine. Not another chatbot trained to know a little about everything and a lot about nothing in particular. Something closer to a second set of eyes on my own life — a reasoning engine, lean and mostly private, that has actually read the things I’ve written and doesn’t need me to explain who I am before it’s useful.

Here’s the distinction that matters, and it took me longer than it should have to see it clearly. The AI industry has spent years in an arms race over how much of the world a model can hold — more facts, more languages, more of the internet compressed into weights. That race will keep going, and somebody else can have it. What I want is smaller and stranger: a model that knows comparatively little about the world and quite a lot about me. My core values document. The portfolio spreadsheets. Fifteen years of blog posts. The half-finished notes for the I-280 project, sitting in a folder, waiting for someone — or something — to ask the right question about them.

I spent a career in payments infrastructure, which means I spent a career thinking about a very specific kind of trust: the kind where a stranger’s system has to make a judgment call, in milliseconds, about whether to say yes. Fraud models don’t work because they know everything about commerce. They work because they know an enormous amount about one account, one pattern, one person’s ordinary Tuesday — enough to notice when Tuesday stops being ordinary. That’s the architecture I keep picturing, aimed inward instead of outward. Not a system trying to know the world. A system trying to know me, well enough to notice when I’m drifting from what I said I cared about.

I can already feel the shape of the mornings this would change. Right now, when I sit down to look at RMD requirements against the tax picture, I’m doing the translation myself — pulling numbers into a story I can actually feel the weight of. A reasoning engine grounded in my real holdings wouldn’t just run the scenario. It would know that I don’t want the scenario dressed up as a spreadsheet; I want it dressed up as a conversation, unhurried, the kind you’d have over lunch with someone who already knows the whole situation. And on the mornings when I sit down to write, instead of staring at a blinking cursor and a blank page that has no idea I exist, I’d be handing a draft to something that has actually read my last two hundred posts and knows the difference between the sentence I’d write and the sentence I’d cut.

None of this is especially exotic technology. Apple and Google are already building toward it — Neural Engines fast enough to do real reasoning on-device, retrieval systems that can reach into your own files instead of the entire internet, fine-tuning that’s getting cheap enough to personalize rather than merely customize. The more interesting story here isn’t privacy, though privacy is real. It’s architectural: what happens when the expensive, impressive part of the system — the part that knows everything — becomes optional, and the cheap, personal part — the part that knows you — becomes the whole point.

What I don’t yet know is what this will cost me. A tool that reasons this well about my own life is also a tool I could lean on instead of doing the leaning myself, and there’s a version of this future where the walk around Sharon Park stops being mine and starts being a conversation with something that finishes my sentences a little too well. I’d want some way of knowing, plainly, what it’s drawing from and what it’s guessing at — less a nutrition label than a kind of honesty I could check against, the way you’d check a fraud model’s confidence score before you trusted it with a yes.

But most mornings, I think I’d take the trade. Not because I want to think less. Because for thirty years I’ve been collecting the raw material — the notebooks, the portfolios, the half-built essays — and it would be something, finally, to walk beside a mind that had actually done the reading.

Categories
AI Business

The Wage of Knowing

In 1973 the Los Angeles Public Library installed a telephone line that worked while the building was dark. Dial H-O-O-T-O-W-L on a rotary phone, nine at night until one in the morning, and a librarian would answer. Somebody wanted to know the boiling point of mercury, or who wrote a poem they half remembered, or how many wives Henry VIII actually had, and a person on the other end of a cord found out. This went on for years. Nobody thought of it as data collection. It was just a service, a courtesy, a woman at a desk with a card catalog in her head.

I worked, in another life, in the payments industry, back when a merchant who wanted to charge your card had to call in and ask permission. There were rooms for this. Banks of phones, a bulletin of stolen numbers updated by hand, a floor limit past which a supervisor had to be found. The people answering the phones were, more often than you would guess, college students. Twenty years old, minimum wage, deciding in real time whether a stranger’s card was good. Nobody trained them for six months first. They learned the bulletin, they learned to listen for something wrong in a voice, and they said yes or no.

I have been driven, recently, by a car with nobody driving it. I noticed the wheel turning on its own and I braced for the wrongness of it. Thirty seconds later I was not bracing. I was looking out the window. The data says I was right to relax: across two hundred and twenty million miles, the cars involved in this experiment cause a small fraction of the serious crashes a human would have caused over the same roads. I did not need the data. I needed thirty seconds.

None of these people knew what they were doing. That is the thing about the librarian and the college student and, for that matter, about me learning to trust a wheel that moves by itself. The librarian was not building a search engine. The clerk was not training a fraud model. He was making rent. Their competence was not evidence, to them. It was just Tuesday. It became evidence later, to someone else, in a room they never saw — the accident logs, the chargeback data, the accumulated record of a million correct guesses that turned out to be exactly the material a system needed to learn the job and take it.

This is the part that is easy to get wrong. It is not that the human failed and the machine succeeded. It is that the human succeeding, over and over, in full view, was the demonstration that the job could be learned. You do not automate a task nobody can do. You automate the one being done well enough, often enough, for long enough that the pattern becomes visible. Doing the job right was never neutral. It was the case being built.

Which brings me to a woman I will call the lawyer, because there are thousands of her and none of them are exactly her. She has a laptop open at her kitchen table. She logs into a dashboard belonging to a company that pairs credentialed people with the AI labs that need them — a doctor here, a banker there, a corporate attorney with fifteen years of contract law behind her. She reads a model’s draft of a merger agreement and marks where it reasons like a first-year associate instead of a partner. She rewrites a clause. She explains, in the margin, why the model’s version would get laughed out of a negotiation. She is paid well for this. More, some weeks, than she billed certain clients.

She knows exactly what she is doing. That is the difference between her and the other three. The librarian did not know she was leaving a trail. The clerk did not know his good judgment would become someone else’s weights. I did not know, thirty seconds into that ride, that I was participating in anything at all. The lawyer knows. She is being paid, by the hour, at a rate that respects her expertise, to make her expertise legible enough that it no longer requires her. The company she works for has a name for this. They call it the reinforcement learning economy, which is a tidy way of saying: teach it everything, and then it will not need to call you back.

She does the work anyway. The rate is good. The work is interesting, in the way that teaching is interesting — you learn what you know by trying to say it clearly enough for someone else to use. Nobody is lying to her. The dashboard does not pretend to be anything other than what it is. She logs off at the end of the session the way anyone logs off after a long day of being excellent at something, tired in the specific way that comes from careful work, and she does not, from what I understand, spend the evening thinking about what she has just fed into the machine.

I keep coming back to the rotary dial. Somebody dialing H-O-O-T-O-W-L at midnight in 1973 could not have imagined the lawyer at her kitchen table. But the shape is the same, if you look at it long enough. A person answers a question well. The answering becomes a record. The record becomes a system. The system answers next time. Nobody in the room ever decided this was the plan. It just turned out, every time, to be the plan.

Categories
AI AI: Transformers

The State You Never See

The transaction arrives in milliseconds. A purchase attempt — a gas station in Phoenix, a grocery store in suburban Atlanta, a wire transfer at 2 a.m. — and somewhere in the authorization chain, a system has to decide. Not later. Now. The clock is already running.

When I led the fraud detection team at Visa, this was the problem that lived in your chest. You couldn’t see what you needed to see. You couldn’t know whether the person presenting that card was the person who owned it, whether the account had been compromised six hours ago in a breach you hadn’t yet detected, whether the behavioral signature of these transactions was the legitimate cardholder running errands or a fraudster working methodically through a stolen number before the window closed. You could only see what the transactions said. You could never see the state underneath.

That distinction — between what you can observe and what is actually true — turns out to be one of the organizing problems of our time. It has a name, a formal structure, and a history that runs from mid-century mathematics through the trading floors of quantitative hedge funds to the frontier of artificial intelligence. The name is the hidden Markov model. But the problem it addresses is older than the math, and more human than the jargon suggests.

Continue reading “The State You Never See”