Podcast InsightsPodcast InsightsPODCAST INSIGHTS
中/EN

What Justifies an Independent AI-Chip Startup in the Hyperscaler Era?

Long Read·8 min read·11 sections

Every hyperscaler now ships its own accelerator, and most flow through the same ASIC partners. Walter Goodwin maps that landscape and explains Fractile's bet: a full-stack approach to inference silicon whose claim is speed above all, and what justifies an independent chip startup in an era of look-alike silicon.

Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile
No PriorsFrontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile
35:39 · 2026-10-02Listen
Contents 11 sections
  1. Introduction
  2. A Luxury Of Choice, The Same Chips
  3. How A Chip Actually Gets Made
  4. One Hundred Fifty People, One Loop
  5. Inference-First, When Inference Needed Explaining
  6. DRAM With The Speed Of SRAM
  7. Two Weeks Versus Three-To-Five Months
  8. AI Inside The Design Loop
  9. The Bandwidth Wall
  10. Who Buys What In Five Years
  11. The Asymmetric Game

Introduction

On No Priors, Sarah Guo sat down with Walter Goodwin, founder and CEO of Fractile, the full-stack AI chip company he started in the summer of 2022 to build very fast inference chips for the largest models in the world. The conversation works both as a chip-industry walkthrough and as a thesis: beneath today's apparent luxury of choice in AI accelerators, almost everyone ships similar silicon through the same partners — and the scarce, bet-worthy move is a full-stack effort aimed at the one property the industry has let lag: memory bandwidth.

A Luxury Of Choice, The Same Chips

Asked where Fractile sits in the accelerator landscape four years on, Goodwin's answer is a map. The hyperscaler era of in-house silicon has a starting point — — and a hidden common structure: Google's TPU, Meta's MTIA, Microsoft's Maia, and OpenAI's effort now underway development-and-supply firms Strip the brand labels and the platforms share their load-bearing parts: — the DRAM type common to Nvidia GPUs, AMD GPUs, and every other ASIC — alongside the same tensor-core bets for matrix multiplication and the same advanced TSMC packaging. What the industry still lacks, he argues, is in pursuit of fundamentally new capability: not many teams have committed to going from architecture and front-end design — which is mostly like writing code — all the way down to physical design, process engineering, and the foundry relationships that have you flying to Taiwan or Korea every week.

How A Chip Actually Gets Made

The episode's most durable explainer is the handoff. If you are Google building a TPU, the architect's role belongs to someone who understands both chip behavior and the workloads — a decade ago that produced circuits like the tensor core, which does one thing extremely well because in these models. Inside the same organizations, a team turns that architectural idea into a description of how the chip should work at the circuit level — front-end design, which mechanically still looks like writing code. That code is what passes to a player like Broadcom, and the design ultimately becomes what is sent to TSMC: literally a bitmap of where every metal layer and transistor goes. The complexity Broadcom holds is the analog IP for chip-to-chip interconnect and the physical placement bound to a specific TSMC node — 3-nanometer, 5-nanometer. To this day, that part of the stack

One Hundred Fifty People, One Loop

Fractile's counter-move is to keep the whole loop in-house: front-end designers, a physical design team, backend implementation, even advanced packaging — limited resources in every area but a much more flexible closed loop. The team works ahead of the curve — where model architectures could move to align with the chip, what to make of the direction of scaling laws — and argues that view to customers and partners, assembling a set of bets for a chip that will not reach mass production for another year or two. The industry's rhythm makes the structure matter: in AI chips, like in no industry before, you have to bet right — a lot of skill and a little luck — and move very quickly.

Inference-First, When Inference Needed Explaining

The first defining bet was the topic itself. Four and a half years ago Fractile stood at the center of inference, then spent the following two years explaining to the world what inference was — some inference chips on the market today The second bet was that a mass-deployment era was coming; the third was on speed, which shaped everything after as an architectural response. Architecturally, the first two years were SRAM on the same die as the logic gives extreme bandwidth between compute and the model weights or KV cache during deployment — the property behind thousands of tokens per second. What started to worry the team in late 2023, and certainly in 2024, was scalability: parameters were growing, and so was context length, an increasingly important part of how these models deploy.

DRAM With The Speed Of SRAM

The pivot — working more closely with memory manufacturers and logic foundry partners toward higher-capacity memory with extremely high bandwidth — is the bet the company is now built around: that combines the scalability of the high-capacity, low-cost DRAM found in GPUs and TPUs with all the speed advantages of Groq or Cerebras chips. The Henry Ford faster-horse anecdote does the framing work: a faster chatbot is the faster horse; running trillion-parameter models comfortably at thousands of tokens per second is what long-lived agents actually require. The economics underneath are unforgiving — at data-center scale for thousands of users, inference comes down to — which is why the search is for

Two Weeks Versus Three-To-Five Months

The middle of the argument is a collision. On one side, workloads move at software speed: and in the advanced Chinese open-source models as does MoE sparsity. The common denominator — new large language models desperately need more memory bandwidth, and generation remains autoregressive in tiny batches — is what a chip can safely target. On the other side, the physical and financial world: foundries take a chip needs to be a sound financial decision, and a flagship still faces Goodwin holds both beliefs at once: compress the development cycle as much as possible, and keep making deliberate, subtle architectural bets, because the chip you build must stay useful for more than three years. Compression, in his framing, means more attempts at the goal — and the prize is real: Nvidia systems already bundle six to nine custom chips built together, and is, in his words, the chip-industry equivalent of the leading model. His own caveat is equally firm: nobody will release a fundamentally new chip every few weeks, because data-center power has a ceiling, installation takes time, and fundamental silicon has to be financed.

AI Inside The Design Loop

The design loop itself is where Goodwin expects AI to bite. A CEO of a top-three semiconductor company privately told him that the trip Goodwin's heuristic for such estimates is to What is actually slow today is the NP-hard middle — — and the situation mirrors recursive self-improvement in AI: extraordinary intelligence now guides the experiments, but the experiments and compute time are the bottleneck, so accelerating the tests matters as much as accelerating the thinking. Not everything moves: final signoff — the foundry-rule confirmation Cadence and Synopsys have owned for decades with TSMC — The unlock worth working on, he argues, is and lets you iterate faster.

The Bandwidth Wall

The workload bet, stated plainly, is to maximize memory bandwidth. The chip still has to do well across the whole hardware-lottery spectrum — labs train on GPUs and XPUs with HBM — but with Fractile can also pull on the model landscape itself. The clearest example is sparsity: at the same level of intelligence, but on modern HBM parts such models hit bandwidth bottlenecks and very low compute utilization. Behind all of it sits a twenty-year asymmetry: Raising the lagging property, Goodwin argues, multiplies everything else — fewer operations for the same intelligence.

Who Buys What In Five Years

Guo's closing question — what will major AI players and hyperscalers actually buy in five years — gets a structural answer. Anyone deploying at scale tries to leverage as many independent platforms as possible, and the in-house efforts are architecturally similar; the industry joke, which he half-endorses, is that The premium segment is where things get interesting: leading labs need solutions that launch models orders of magnitude faster, because premium intelligence is their reason for existence — Best model weights plus fastest deployment: he argues, is becoming an extremely important part of this world.

The Asymmetric Game

Why won't the labs just build everything themselves? Because they are playing an asymmetric game. If the first lab bets entirely on its own silicon and a second lab lands a fivefold-efficiency breakthrough that only runs on the second lab's chip, the first while catching up. So labs stay on shared platforms and compete at the model level — and for anyone working at the limit of their capabilities, radically different hardware bets are, in Goodwin's words, That asymmetry is his decades-long case for independent chip companies at the frontier — and the gap Fractile is built to occupy.

Interview Highlights

5 exchanges
  • Q

    How does the handoff between architecture-focused players and companies like Broadcom actually work?

    Sarah Guo asked04:56
    A
    Walter Goodwin answered

    If you are, say, Google developing a TPU, you have people on the team with a very good understanding of the workloads they are trying to accelerate. Looking back ten years, this is a world where you build things like the tensor core — a special circuit the architect came up with that does matrix multiplication extremely well, because matrix multiplication is the vast majority of floating-point operations in the models you run. That is the architect's prerogative: understanding how chips behave and, ideally, the target workloads. Inside those same organizations, a group of smart people transforms this into essentially a description of how the chip should work at the circuit level — front-end design, which mechanically still looks like writing code. That same code is what gets passed to a player like Broadcom, and the design ultimately becomes what is sent to TSMC: a GDSII file, literally a bitmap of sorts — information about where to place the metal layers and each individual transistor, the complete layout. A big part of the complexity there is ownership of the analog intellectual property for chip-to-chip connectivity and the physical placement specific to a particular TSMC process node — 3 nanometers, 5 nanometers. That part, to this day, is generally an outsourcing activity for all of these projects.

  • Q

    What are the most important technical bets the company has made?

    Sarah Guo asked09:49
    A
    Walter Goodwin answered

    Two bets mattered most. First, the chosen direction: four and a half years ago Fractile was at the very center of the topic of inference, yet for two years after that it was still explaining to the world what inference was — some inference chips on the market today were training chips eighteen months ago. The bet that an era of mass deployment was coming was crucial, and so was the bet on speed, which largely determined the architectural response. In terms of architecture the first two years were spent on an SRAM chip similar to Groq or Cerebras: SRAM sits on the same silicon die as the logic, so bandwidth between the compute unit and the model weights or KV cache during deployment is extremely high — exactly what allows speeds of thousands of tokens per second in these language models. Toward the end of 2023 and certainly in 2024, the worry became scalability: model parameters are growing, and so is the length of context, an increasingly important part of how these models deploy. That drove the more exciting bet — working more closely with memory manufacturers and logic foundry partners to access higher-capacity memory with extremely high bandwidth, combining the scalability of the higher-capacity, lower-cost DRAM found in GPUs or TPUs with all the speed advantages of Groq or Cerebras chips, on a platform fully operational in the second half of next year. A faster chatbot is the faster horse; comfortably running trillion-parameter models at many thousands of tokens per second is, in his view, a fundamental advancement in AI's ability to create long-lasting agents.

  • Q

    As you plan the architecture ahead of coming workloads, what do you see?

    Sarah Guo asked28:18
    A
    Walter Goodwin answered

    The most important thing is the pursuit of speed: maximizing memory bandwidth to run these models faster. A chip company has to embrace a duality — create something that does equally well across the spectrum of workloads that come up within the idea of a hardware lottery, where labs train on GPUs and XPUs with HBM, while also exploring whether the chip's own properties apply gravitational forces to a new landscape of models. The chip copes exceptionally well with today's ultra-aggressive MoE transformers, running them much faster and more efficiently, and 25 times more bandwidth per chip than an HBM-based chip lets Fractile explore scaling laws for bandwidth. Some are already visible: mixture-of-experts models should be made more and more sparse — for the same level of intelligence, going from 1-to-16 sparsity to 1-to-128 or 1-to-256 saves a bunch of operations — but on modern GPUs and XPUs with HBM memory such models become extremely difficult to serve efficiently, hitting bandwidth bottlenecks and very low compute utilization. Some forms of attention are less demanding on bandwidth and more demanding on computing resources for a given level of intelligence. Over the past twenty years computing power scaled a million times while memory bandwidth increased about forty times — improving that property is a real multiplier of overall performance for such models.

Back to Episode OverviewView Podcast Channel