Tag: Inference

Categories
AI AI: Inference Semiconductors Uncategorized

5 Critical Management Lessons from the Founders at Etched

How two young founders are building what could become one of the most important companies in the AI era — and what their story teaches about leadership, execution, and building at the edge of the possible.

I recently listened to the latest Invest Like the Best podcast from Patrick O’Shaughnessey which was a remarkable conversation with Gavin and Rob, the founders of Etched, the company building specialized AI inference hardware that’s aiming to be radically better than existing solutions. Their story — starting as very young founders against massive skepticism, raising serious capital, and now shipping full rack-scale systems — is packed with hard-earned wisdom.

One of the comments Patrick makes at the beginning was how during his due diligence on the company he kept being told that semiconductor technology wasn’t a place for young people. You need seasoned, middle age experts to master this domain. Exactly not these founders.

Note: the following is based upon an AI’s analysis of the conversation transcript with me asking “What are the five most important management lessons from this conversation?” These lessons are relevant whether you’re leading a team, building a product, or simply trying to do meaningful work in our fast-moving world.

1. Velocity Compounds — Prioritize Speed Ruthlessly

In hardware, and increasingly in any deep-tech endeavor, speed isn’t just an advantage; it’s often the deciding factor.

Etched didn’t just design a chip — they built the full inference solution (chip, board, power delivery, interconnects, cold plates, and production processes) in parallel. They sent engineers to live in Bangalore for months to unblock vendors. They ran 24/7 shifts and did massive pre-work (including putting full chip designs on FPGA clusters) so that when the silicon finally arrived, they had working inference in racks in just 40 days.

Key takeaway: Look for every opportunity to parallelize. Accept higher short-term costs if they buy meaningful time. As they put it, “You win by shipping.” The best part is often no part — and the best vendor is no vendor, when vertical integration lets you move faster. Velocity, velocity, velocity.

2. Build Teams with Legends + High-Drive Talent

One of the most distinctive parts of their approach is how they recruit. They seek out “Legends” — people who have done the hardest versions of the problem before (like the engineer who built Nvidia’s HGX and DGX systems) — and pair them with exceptionally driven, somewhat naive high-performers who refuse to accept conventional limits.

They use “project-based recruiting,” mapping the hardest technical problems ever solved and persistently pursuing the actual people who did the real work. Their culture self-selects for people willing to move their families to San Jose to bet on two young founders taking on the world.

Key takeaway: For breakthrough work, average talent doesn’t suffice. The combination of deep experience and raw, first-principles energy creates magic. Invest heavily in finding and retaining these people — even if it takes 20 conversations. You can also learn a lot if the best in the world talent turns down the opportunity to work with you!

3. Assume It’s Possible, Then Solve the “Unsolvable” Problems

Repeatedly in their story, experts told them certain things were impossible. Their response? Assume it is possible and figure out how.

The most striking example was a clock domain crossing issue that required aligning signals to within 50 picoseconds — something many engineers said couldn’t be done. People quit. They solved it in about two weeks during a very dark period.

Key takeaway: When you hear “impossible,” treat it as the beginning of the investigation, not the end. Cultivate a “find a way” mindset across the team. The moments when things feel hopeless are often when the most important progress happens. I’m constantly struck by how often persistence results from simply realizing (or assuming) that something is actually possible.

4. Production Is the Real Product

Etched’s mantra is “Production is the product.” They obsess over not just technical performance but manufacturability, supply chain resilience, serviceability, and the ability to scale to gigawatts.

They made deliberate choices around process nodes and memory to avoid zero-sum competition. They built their own factory processes and test infrastructure early. Future designs are being simplified specifically for faster production cycles and higher reliability at massive scale.

Key takeaway: In any business that hopes to reach real scale, think end-to-end from the beginning. Technical excellence without production excellence is just a prototype. Optimize for output (tokens, units, whatever your metric is) at volume. There’s a lot of “zero to one” thinking here.

5. Bet Big and Stay Existentially Focused

Building in semiconductors requires enormous capital. Etched raised roughly $100 million early on when they were still very young and pre-tapeout — after most traditional investors had passed. They knew half-measures wouldn’t work.

This existential focus (this one product determines whether the company lives or dies) creates a different level of intensity that attracts talent, suppliers, and customers who believe.

Key takeaway: Match your ambition with appropriate resources and commitment. Clear existential stakes help filter for the right people and partners. In a world of distractions, singular focus on what truly matters is a superpower.

Final Thoughts

Gavin and Rob’s story is the combination of technical sophistication and deep human resilience. They faced a tough personal battle with cancer (in Rob’s case), widespread doubt, brutal technical challenges, and fundraising pressure — and kept moving forward with curiosity, determination, and humility.

In an age of AI and accelerating technology, the ability to build teams that can solve seemingly impossible problems at speed may be one of the most valuable capabilities a leader can develop. Their example reminds us that the future belongs not just to the smartest, but to those who can execute with urgency while maintaining clear principles. Velocity, velocity, velocity.

Categories
AI AI: Transformers

The State You Never See

The transaction arrives in milliseconds. A purchase attempt — a gas station in Phoenix, a grocery store in suburban Atlanta, a wire transfer at 2 a.m. — and somewhere in the authorization chain, a system has to decide. Not later. Now. The clock is already running.

When I led the fraud detection team at Visa, this was the problem that lived in your chest. You couldn’t see what you needed to see. You couldn’t know whether the person presenting that card was the person who owned it, whether the account had been compromised six hours ago in a breach you hadn’t yet detected, whether the behavioral signature of these transactions was the legitimate cardholder running errands or a fraudster working methodically through a stolen number before the window closed. You could only see what the transactions said. You could never see the state underneath.

That distinction — between what you can observe and what is actually true — turns out to be one of the organizing problems of our time. It has a name, a formal structure, and a history that runs from mid-century mathematics through the trading floors of quantitative hedge funds to the frontier of artificial intelligence. The name is the hidden Markov model. But the problem it addresses is older than the math, and more human than the jargon suggests.

Continue reading “The State You Never See”
Categories
Business History IBM Infrastructure Nvidia Programming Semiconductors

The Half-Life of Moats

Prompted by an article on X by @magicsilicon on the CUDA moat. Research and drafting assistance from my AI intern assistant Clark.

The NVIDIA H100 looks, in retrospect, like an inevitability. It wasn’t.

What Jensen Huang built is more accurately understood as a sixteen-year accumulation of optionality — a platform investment made in 2006 for a market that wouldn’t fully materialize until 2022. NVIDIA intros the G80 architecture in November 2006, laying the groundwork for CUDA’s release a few months later. The stated ambition was to let scientists write C++ that ran on GPU cores without needing to understand 3D graphics pipelines. The unstated bet was that parallel computation would eventually matter for something bigger than rendering shadows in video games.

For sixteen years, it mostly didn’t. Not at scale. Not commercially. CUDA lived in research labs and HPC clusters. It attracted a small, devoted, and economically marginal user base — the kind that papers cite but investors ignore. NVIDIA kept investing in it anyway: cuDNN for deep learning operations, cuBLAS for linear algebra, a layered ecosystem of libraries that made CUDA not just accessible but nearly irreplaceable for anyone doing serious numerical computation. When TensorFlow and PyTorch emerged as the standard frameworks for neural network research, they didn’t adopt CUDA because it was the only option. They adopted it because CUDA was where the optimized kernels already lived.

AlexNet won the ImageNet competition in 2012 and did it on two NVIDIA GPUs. The deep learning community noticed immediately. The financial community largely did not.

Then ChatGPT launched in November 2022, and suddenly everyone needed H100s they couldn’t get.

The parallel to Intel is instructive and also undersells how strange this kind of story looks while you’re living through it. Intel was founded in 1968 as a memory company. DRAM. The founders — Noyce, Moore, Grove — were materials scientists and engineers who believed the future was in silicon memory chips. They were right, briefly: in the early 1970s Intel dominated the DRAM market. By 1984, that share had collapsed to 1.3%, ceded almost entirely to Japanese manufacturers who had commoditized the product.

What saved Intel wasn’t a pivot so much as a realization that a stopgap had become a foundation. The 8086, conceived in 1976 as an internal hedge and launched in 1978 was never supposed to matter. It was a 16-bit processor designed to hold off Zilog while Intel finished its ambitious 32-bit iAPX 432 architecture. The 8086 was assigned to a single engineer. “If management had any inkling that this architecture would live on through many generations,” its designer Stephen Morse later recalled, “they never would have trusted this task to a single person.”

IBM chose the 8088 — a cost-reduced variant — for the original IBM PC in 1981. That decision wasn’t destiny, it was simply a procurement. And yet from that accident of selection, Intel’s x86 line became the backbone of personal computing for four decades. The Pentium in 1993 was Intel’s Wintel moment — the flag bearer the @magicsilicon tweet gestures at — but the flag had been quietly sewn since 1978.

What these histories share is not just a pattern of “slow build, explosive payoff.” The structural similarity is subtler: in both cases, the moat was a software abstraction layer built on top of hardware. Intel’s real lock-in wasn’t transistor count or clock speed. It was backward compatibility — the commitment, formalized with the 80386 in 1985, that every future Intel chip would run software written for older ones. That promise created a flywheel that trapped developers and buyers in a virtuous (for Intel) dependency loop for decades.

CUDA is the same architecture at a different layer. The lock-in isn’t the H100’s 80 gigabytes of HBM3. It’s that switching to an AMD MI300X or Google TPU means potentially rewriting training pipelines that have been optimized against CUDA kernels for years. AMD’s ROCm platform exists. It is, by most accounts, maturing. Engineers who have tried the migration report that it costs months and hundreds of thousands of dollars. The moat isn’t a wall. It’s accumulated friction — the switching cost of a decade of engineering decisions baked into codebases that no one wants to touch.

But to find the actual origin of this pattern, you have to go back further than Intel. To 1964, and to a decision IBM made that Fred Brooks — its project manager — called a bet-the-business move.

The IBM System/360 was announced on April 7, 1964, after five years of turbulent internal development. What it introduced wasn’t just a new computer. It was a new concept: the separation of architecture from implementation. Before the 360, IBM ran five incompatible product lines simultaneously. A customer who outgrew their machine had to scrap all existing software and start over. The 360 replaced all five lines with a single unified architecture — six models covering a fiftyfold performance range, all running the same operating system, all sharing the same instruction set. The name itself encoded the ambition: 360 degrees, all directions, all users.

Gene Amdahl, the 360’s chief architect, had a precise formulation for what this meant: the architecture was “an interface for which software is written, independent of any implementation.” The Principles of Operation manual described what the machine did; separate Functional Characteristics documents described how each model did it. This distinction — separating the contract from the execution — was genuinely new. It’s the conceptual root of everything that came after.

The 360 generated over $100 billion in revenue for IBM and established the first platform business model in computing. Jim Collins would later rank it alongside the Model T and the Boeing 707 as one of the three greatest business achievements of the twentieth century. But its deepest legacy was architectural: the insight that if you make your abstraction layer the standard, the hardware underneath becomes fungible. Customers didn’t buy specific IBM machines. They bought into OS/360. The machines were an implementation detail.

Intel understood this by the 1980s, even if implicitly. The 80386’s backward compatibility commitment in 1985 was IBM’s 360 insight applied to microprocessors — the architecture is the product, the silicon is the vehicle. CUDA is the same insight applied to GPU compute. What NVIDIA sold researchers in 2006 wasn’t the G80 card. It was the abstraction: write parallel code in C++, run it on any NVIDIA hardware, trust that the next generation will be faster and compatible.

The pattern is now sixty years old. It has reproduced in every major platform transition. And it keeps working for the same reason it worked in 1964: when you own the layer that developers write to, your customers’ switching costs compound every year they stay.

There’s something worth sitting with here. Neither Jensen Huang in 2006 nor Gordon Moore in 1968 could have specified exactly what the payoff would look like. What they shared was a willingness to build infrastructure for a demand they could sense but not yet see — and the discipline to keep investing in it through the long years when it looked like a research project rather than a business.

The question that doesn’t resolve cleanly is whether that kind of patience is a strategy or a personality. And whether, in an industry that now moves faster than the cycles it’s lived through, sixteen-year moats are still the kind that get built.

Which raises the uncomfortable corollary: the same AI tools that CUDA enabled may be what ultimately erodes it.

The attack on CUDA’s moat is now structurally different from anything AMD or Intel could mount before. OpenAI’s Triton compiler lets developers write GPU kernels in Python without touching CUDA at all, and generates optimized machine code that often matches hand-tuned CUDA performance. MLIR — Multi-Level Intermediate Representation, originally from Google — provides a compiler infrastructure that can target any hardware backend from a single codebase. AMD’s ROCm has historically been dismissed as immature; ROCm 7, released this year, delivers meaningfully better inference performance than its predecessors. And perhaps most directly: Claude Code reportedly ported a CUDA codebase to AMD’s ROCm in thirty minutes — work that previously took months of engineering time.

The irony is almost too neat. CUDA’s moat was built on accumulated switching costs: the friction of rewriting code, the library dependencies, the tribal knowledge encoded in a decade of kernel optimizations. AI coding tools are specifically good at exactly that kind of mechanical, high-context translation. The weapon is attacking the wall it was built behind.

That said, it’s worth being careful about the speed of this. Abstraction layers that “should” erode moats often take far longer than expected, because the moat isn’t just the code — it’s the ecosystem of tooling, documentation, community knowledge, and hardware-software co-optimization that took eighteen years to compound. Triton and MLIR are real. They’re also early. The question isn’t whether the moat is vulnerable; it’s whether it erodes before NVIDIA’s next generation of chips makes it irrelevant to argue about.

As for what comes next — which company is building the IBM 360 of this decade — the honest answer is that it’s too early to call with confidence. But there’s a candidate worth watching.

Anthropic’s Model Context Protocol, launched in late 2024, has the structural fingerprint of a platform play. MCP is a standard for how AI agents connect to external tools and data sources — a common interface layer, hardware-agnostic (or rather, model-agnostic), that any system can implement. By late 2025 it had been donated to the Linux Foundation, adopted by OpenAI and Google, and was tracking 97 million monthly SDK downloads. There are now over 10,000 MCP servers. It is becoming the way agents talk to the world.

The parallel to OS/360 is imprecise but instructive. What IBM built in 1964 was a standard interface between software and hardware that decoupled what you wrote from what you ran it on. MCP is attempting something similar one abstraction layer higher: decoupling what an agent does from the specific models, tools, and data sources it does it with. If it becomes the standard — the layer that developers write to — then whoever owns or most deeply shapes that standard controls the integration tax of an industry whose applications we can’t fully specify yet.

The counterargument is that open standards, once donated to foundations and broadly adopted, don’t generate the same lock-in as proprietary platforms. OS/360 was IBM’s. CUDA is NVIDIA’s. MCP is now the Linux Foundation’s, with OpenAI and Google as co-stewards. The historical pattern suggests the moat accrues to whoever owns the layer, not whoever invented it.

Which may mean the next great platform play is still being assembled in a room we haven’t seen yet — the way IBM’s System/360 was being architected in a Connecticut motor lodge in 1961, three years before anyone else knew what was coming.

Categories
AI Software

The Thermodynamics of Thought

For the last two decades, we have lived in the era of zero marginal cost. The defining characteristic of the internet age was that once software was written, distributing it to the billionth user cost virtually the same as distributing it to the first. We grew accustomed to the economics of abundance—infinite copies, infinite reach, lightweight infrastructure.

But the recent commentary regarding the true nature of Artificial Intelligence forces a jarring mental correction:

“AI is not software riding on old infrastructure. It is a new industrial system that converts energy into intelligence – requiring a capital stack measured in trillions, not billions.”

This distinction is not merely semantic; it is physical.

When we view AI through the lens of traditional SaaS (Software as a Service), we miss the magnitude of the shift. We are looking for an app; what is being built is a refinery. We are witnessing a return to heavy industry, but the commodity being refined isn’t crude oil—it is information, and the byproduct is reasoning.

This requires us to think less in terms of code and more in terms of thermodynamics. In this new industrial system, intelligence is an energy-intensive output. Every token generated, every inference drawn, requires a specific, measurable conversion of electricity into heat and computation. Unlike the static code of a website, an AI model is a furnace. It must be fueled constantly.

This explains the capital stack. We are seeing numbers that seem irrational in the context of venture capital—trillions, not billions. But if you view a data center not as a server farm, but as a power plant that generates intelligence, the numbers align with historical precedents. We are not funding startups; we are funding the modern equivalent of the electric grid, the transcontinental railroad, or the petrochemical complex.

We are pouring concrete, smelting copper, and manufacturing silicon on a planetary scale. The “cloud” was always a misleading metaphor—it sounded fluffy and ethereal. The reality of the AI transition is heavy, hot, and incredibly expensive.

We are moving from an era where we organized the world’s information (low energy) to an era where we synthesize new reasoning (high energy). We are building a machine that eats electricity and excretes intelligence. That isn’t a software update; that is a new industrial revolution.