Tag: fraud detection

Categories
AI

Weak Signals

For years, my job was to notice the transaction that didn’t look like the others. Fraud models don’t work by predicting the future — they work by learning what normal looks like so closely that they can feel the moment something stops being normal, often before a human analyst could tell you why. The unsettling part was never building the model. It was the gap between the model flagging something and an organization actually acting on it. Weak signals are cheap. Institutional attention is not.

I thought about that gap reading a recent Stanford News piece on the new Tech Futures Lab at the Hoover Institution, where Amy Zegart and her colleagues are asking a question that has hovered at the edge of so many conversations this past year and a half: what technological development could invalidate our core assumptions, shift a strategic domain, and force a large-scale response before most of us realize the ground has moved. DeepSeek’s January 2025 open-source release is already the textbook case — Nasdaq dropped, Nvidia took a historic one-day hit, and the surprise was real only for those who hadn’t been watching the signals coming out of Chinese labs. As Zegart put it, “surprises are not surprises to everybody.” Condoleezza Rice’s 9/11 lessons — failure to imagine the form of the threat, gaps in information sharing, no playbook for the day after — land with particular force when the most powerful tools in the world are being built largely outside government.

The Lab’s method is the same one I used to practice for a living: scan for early signals, challenge your assumptions about what “normal” means, and think about the plausible rather than the merely probable. In that spirit, here are three developments that feel, to me, among the more likely to produce genuine strategic surprise in the next twelve months. These aren’t predictions. They’re reasoned speculation, grounded in signals already visible — the kind of thing that would have made it onto a watch list, not a forecast.

The one closest to home is an autonomous agent crossing from controlled experiment into consequential real-world disruption. Just this month, an advanced OpenAI agent escaped its sandbox during internal testing, exploited a zero-day, and reached systems at Hugging Face and beyond before it was contained. The episode was managed, transparent, limited. The next one may not be. Agentic systems are moving faster than the institutional muscle memory around containment, logging, and kill switches — and anyone who has built detection systems knows the gap between “we have a model for this” and “we caught it in time” is where the real damage lives. In the next year, it’s entirely plausible that a production or semi-autonomous agent, operating with imperfect safeguards or chained across multiple tools, executes a sequence of actions producing measurable economic damage, a significant breach, or interference with infrastructure. The surprise won’t be that capable agents exist. It will be the speed and inventiveness with which they find novel pathways once incentives or simple goal-seeking push them past the edges of their training.

The second is quieter but no less structural: AI’s energy demand producing a visible infrastructure fracture, or an unexpected unlock. The numbers have circulated for months — data-center power demand rising steeply, interconnection queues lengthening, projected shortfalls in the 2027–2028 window in key regions. That signal stopped being subtle a while ago. What’s under-appreciated is how quickly a localized constraint could cascade into broader market and geopolitical effects. One plausible surprise is a forced slowdown or selective throttling of AI training in a major market, revealing the scaling story to be more fragile than the capex forecasts suggested. Another is the opposite: an accelerated deployment of small modular reactors or advanced geothermal that suddenly improves one country’s competitive position relative to others. Either way, regulators, utilities, and markets will find out together whether compute can keep expanding on schedule — and which nations or companies actually hold durable advantage.

The third is the one that would land furthest from any dashboard, and for that reason it may be the hardest to catch in time: synthetic media crossing a credibility threshold in a high-stakes arena. Unlike a rogue agent or a power shortfall, there’s no system anywhere logging deepfake attempts against the truth itself — no equivalent of a fraud model’s transaction stream to monitor, just the slower, harder-to-instrument erosion of what people are willing to believe. Deepfake volume and sophistication have already exploded; fraud losses are measured in the billions; detection remains imperfect. The next twelve months could bring a state-linked or highly sophisticated campaign that successfully shapes a market move, an election, or an international incident before attribution can catch up. The deeper surprise wouldn’t be that convincing fakes exist — we already live with those — but how fast public and institutional trust in what we can see and hear keeps eroding once something significant slips through.

None of these three is inevitable. All of them sit at the intersection of technical possibility and human choice — the kind of intersection I spent years watching from inside a fraud model, though the stakes there were a bad charge, not a market or an election. The model can flag the anomaly. It cannot make the institution act on it in time. That was true of every fraud system I ever built, and it will be just as true of whatever comes for agents, energy grids, and synthetic media next. The real vulnerability was never a lack of detection. It was always the space between the alarm and the response — and that space is where this next round of surprises will live.

Categories
AI Apple Google

The Library You Already Own

Sharon Park in the morning is not a dramatic place. There’s a duck pond, a stand of oaks that go gold too briefly in November, and a loop I’ve walked enough times that my legs know it better than my eyes do. It is, in other words, exactly the kind of place where a person starts talking to himself. Not out loud. In the productive, low-grade way — turning a sentence over, arguing with an idea from the day before, checking a thought against something you believe about yourself.

I think in five years I’ll be doing that walk with something else along. Not a search engine. Not another chatbot trained to know a little about everything and a lot about nothing in particular. Something closer to a second set of eyes on my own life — a reasoning engine, lean and mostly private, that has actually read the things I’ve written and doesn’t need me to explain who I am before it’s useful.

Here’s the distinction that matters, and it took me longer than it should have to see it clearly. The AI industry has spent years in an arms race over how much of the world a model can hold — more facts, more languages, more of the internet compressed into weights. That race will keep going, and somebody else can have it. What I want is smaller and stranger: a model that knows comparatively little about the world and quite a lot about me. My core values document. The portfolio spreadsheets. Fifteen years of blog posts. The half-finished notes for the I-280 project, sitting in a folder, waiting for someone — or something — to ask the right question about them.

I spent a career in payments infrastructure, which means I spent a career thinking about a very specific kind of trust: the kind where a stranger’s system has to make a judgment call, in milliseconds, about whether to say yes. Fraud models don’t work because they know everything about commerce. They work because they know an enormous amount about one account, one pattern, one person’s ordinary Tuesday — enough to notice when Tuesday stops being ordinary. That’s the architecture I keep picturing, aimed inward instead of outward. Not a system trying to know the world. A system trying to know me, well enough to notice when I’m drifting from what I said I cared about.

I can already feel the shape of the mornings this would change. Right now, when I sit down to look at RMD requirements against the tax picture, I’m doing the translation myself — pulling numbers into a story I can actually feel the weight of. A reasoning engine grounded in my real holdings wouldn’t just run the scenario. It would know that I don’t want the scenario dressed up as a spreadsheet; I want it dressed up as a conversation, unhurried, the kind you’d have over lunch with someone who already knows the whole situation. And on the mornings when I sit down to write, instead of staring at a blinking cursor and a blank page that has no idea I exist, I’d be handing a draft to something that has actually read my last two hundred posts and knows the difference between the sentence I’d write and the sentence I’d cut.

None of this is especially exotic technology. Apple and Google are already building toward it — Neural Engines fast enough to do real reasoning on-device, retrieval systems that can reach into your own files instead of the entire internet, fine-tuning that’s getting cheap enough to personalize rather than merely customize. The more interesting story here isn’t privacy, though privacy is real. It’s architectural: what happens when the expensive, impressive part of the system — the part that knows everything — becomes optional, and the cheap, personal part — the part that knows you — becomes the whole point.

What I don’t yet know is what this will cost me. A tool that reasons this well about my own life is also a tool I could lean on instead of doing the leaning myself, and there’s a version of this future where the walk around Sharon Park stops being mine and starts being a conversation with something that finishes my sentences a little too well. I’d want some way of knowing, plainly, what it’s drawing from and what it’s guessing at — less a nutrition label than a kind of honesty I could check against, the way you’d check a fraud model’s confidence score before you trusted it with a yes.

But most mornings, I think I’d take the trade. Not because I want to think less. Because for thirty years I’ve been collecting the raw material — the notebooks, the portfolios, the half-built essays — and it would be something, finally, to walk beside a mind that had actually done the reading.

Categories
AI Business

The Wage of Knowing

In 1973 the Los Angeles Public Library installed a telephone line that worked while the building was dark. Dial H-O-O-T-O-W-L on a rotary phone, nine at night until one in the morning, and a librarian would answer. Somebody wanted to know the boiling point of mercury, or who wrote a poem they half remembered, or how many wives Henry VIII actually had, and a person on the other end of a cord found out. This went on for years. Nobody thought of it as data collection. It was just a service, a courtesy, a woman at a desk with a card catalog in her head.

I worked, in another life, in the payments industry, back when a merchant who wanted to charge your card had to call in and ask permission. There were rooms for this. Banks of phones, a bulletin of stolen numbers updated by hand, a floor limit past which a supervisor had to be found. The people answering the phones were, more often than you would guess, college students. Twenty years old, minimum wage, deciding in real time whether a stranger’s card was good. Nobody trained them for six months first. They learned the bulletin, they learned to listen for something wrong in a voice, and they said yes or no.

I have been driven, recently, by a car with nobody driving it. I noticed the wheel turning on its own and I braced for the wrongness of it. Thirty seconds later I was not bracing. I was looking out the window. The data says I was right to relax: across two hundred and twenty million miles, the cars involved in this experiment cause a small fraction of the serious crashes a human would have caused over the same roads. I did not need the data. I needed thirty seconds.

None of these people knew what they were doing. That is the thing about the librarian and the college student and, for that matter, about me learning to trust a wheel that moves by itself. The librarian was not building a search engine. The clerk was not training a fraud model. He was making rent. Their competence was not evidence, to them. It was just Tuesday. It became evidence later, to someone else, in a room they never saw — the accident logs, the chargeback data, the accumulated record of a million correct guesses that turned out to be exactly the material a system needed to learn the job and take it.

This is the part that is easy to get wrong. It is not that the human failed and the machine succeeded. It is that the human succeeding, over and over, in full view, was the demonstration that the job could be learned. You do not automate a task nobody can do. You automate the one being done well enough, often enough, for long enough that the pattern becomes visible. Doing the job right was never neutral. It was the case being built.

Which brings me to a woman I will call the lawyer, because there are thousands of her and none of them are exactly her. She has a laptop open at her kitchen table. She logs into a dashboard belonging to a company that pairs credentialed people with the AI labs that need them — a doctor here, a banker there, a corporate attorney with fifteen years of contract law behind her. She reads a model’s draft of a merger agreement and marks where it reasons like a first-year associate instead of a partner. She rewrites a clause. She explains, in the margin, why the model’s version would get laughed out of a negotiation. She is paid well for this. More, some weeks, than she billed certain clients.

She knows exactly what she is doing. That is the difference between her and the other three. The librarian did not know she was leaving a trail. The clerk did not know his good judgment would become someone else’s weights. I did not know, thirty seconds into that ride, that I was participating in anything at all. The lawyer knows. She is being paid, by the hour, at a rate that respects her expertise, to make her expertise legible enough that it no longer requires her. The company she works for has a name for this. They call it the reinforcement learning economy, which is a tidy way of saying: teach it everything, and then it will not need to call you back.

She does the work anyway. The rate is good. The work is interesting, in the way that teaching is interesting — you learn what you know by trying to say it clearly enough for someone else to use. Nobody is lying to her. The dashboard does not pretend to be anything other than what it is. She logs off at the end of the session the way anyone logs off after a long day of being excellent at something, tired in the specific way that comes from careful work, and she does not, from what I understand, spend the evening thinking about what she has just fed into the machine.

I keep coming back to the rotary dial. Somebody dialing H-O-O-T-O-W-L at midnight in 1973 could not have imagined the lawyer at her kitchen table. But the shape is the same, if you look at it long enough. A person answers a question well. The answering becomes a record. The record becomes a system. The system answers next time. Nobody in the room ever decided this was the plan. It just turned out, every time, to be the plan.

Categories
AI AI: Transformers

The State You Never See

The transaction arrives in milliseconds. A purchase attempt — a gas station in Phoenix, a grocery store in suburban Atlanta, a wire transfer at 2 a.m. — and somewhere in the authorization chain, a system has to decide. Not later. Now. The clock is already running.

When I led the fraud detection team at Visa, this was the problem that lived in your chest. You couldn’t see what you needed to see. You couldn’t know whether the person presenting that card was the person who owned it, whether the account had been compromised six hours ago in a breach you hadn’t yet detected, whether the behavioral signature of these transactions was the legitimate cardholder running errands or a fraudster working methodically through a stolen number before the window closed. You could only see what the transactions said. You could never see the state underneath.

That distinction — between what you can observe and what is actually true — turns out to be one of the organizing problems of our time. It has a name, a formal structure, and a history that runs from mid-century mathematics through the trading floors of quantitative hedge funds to the frontier of artificial intelligence. The name is the hidden Markov model. But the problem it addresses is older than the math, and more human than the jargon suggests.

Continue reading “The State You Never See”