Google Part III: The AI Company. Google is amazingly well-positioned... will they win in AI? (Audio)
Executive Summary
This episode tells Google's AI history as one continuous causal chain. It opens inside Google Brain-era engineering — Noam Shazeer's early years, n-gram translation, the async infrastructure that made Google-scale computation possible — then runs the AlexNet conversion, the founding of DeepMind in London, and the 2013-14 auction that ended with Google paying $550 million for Demis Hassabis's lab. From there the saga compounds: the secret TPU skunkworks (Jeff Dean's 'we need another Google,' a satellite office in Madison, Wisconsin, a year of silence before Google I/O), the transformer paper whose eight authors all left within a couple of years, Noam's exit, and the Rosewood dinner that seeded OpenAI. ChatGPT's launch is treated as the disruptive-innovation flip: Sundar's December 2022 code red, the rushed Bard launch, Microsoft's escalating OpenAI stakes, and the delicate AI Overviews/AI Mode dance of cannibalizing search without killing it. The second half moves to the present business and the AI-era balance sheet. Waymo's arc from DARPA challenge to five driverless cities is told with Ben's milestones run and the company's own 91% safety-study figure; Google Cloud's Kurian-era turnaround (from $4 billion in 2017 to a $50 billion-plus run rate) is framed as the enterprise go-to-market fix plus a TPU tailwind; and David's four-pillars argument — application, model, chip, cloud, with only Google holding more than one — carries the bull case alongside distribution, dark fiber, the YouTube video internet, and self-sustaining funding. The bear case is the value-capture question: $400 per US user of ad economics against subscription willingness, and 90% search share against AI's maybe-25% steady state. The episode's closing frame is governance: Larry and Sergey control the company and have said they would rather go bankrupt than lose at AI, the antitrust case that might have broken Google up instead ended with immaterial remedies, and the hosts read founder control as the innovator's-dilemma antidote — the reason Google could rebuild around Gemini rather than defend ten blue links. Carve-outs and thank-yous close the episode, crediting the Levy, Olson and Metz books and a long research-interview list.
Chapters & Key Takeaways
9 takeawaysThe DeepMind acquisition is the hinge of the episode's causal chain: the $550 million deal happened because of the kinship between Larry and Demis, and the counterfactual the hosts draw is that without it OpenAI likely never gets founded.
The TPU began as a defensive infrastructure necessity — speech recognition alone would have needed 'another Google' worth of hardware — and became a strategic moat: a complete secret for a year, weak in V1, and now an estimated 2-3 million TPUs against Nvidia's ~4 million annual GPU shipments.
Google created the transformer and lost the value: all eight authors of 'Attention Is All You Need' left within a couple of years, and the company that knew a chatbot interface was possible shipped nothing until ChatGPT forced the issue.
ChatGPT flipped AI from a sustaining innovation Google could absorb into a disruptive one overnight — the December 2022 code red meant 'build and ship real native AI products ASAP,' the textbook response to disruption.
Microsoft's escalation — $1 billion, then $2 billion, then $10 billion for a reported ~49% — converted OpenAI into Google's first true existential threat, with Satya openly savoring making Google dance.
Waymo's decade-long march (DARPA 2004 to Phoenix 2020) now runs five cities with Tokyo next, and the company's released study claims 91% fewer injury crashes — presented as the payoff of Google's willingness to fund a moonshot for sixteen years.
Google Cloud is the Kurian turnaround story: an enterprise go-to-market buildout (10,000 hires on a base of 150) plus the Kubernetes multicloud counterposition plus a TPU tailwind took the business from $4 billion in 2017 to a $50 billion-plus run rate growing 30%.
The four-pillars frame is the episode's strategic core: to play in AI you need an application, a model, a chip, or a cloud — Google is trying to have all four, and no other company has more than one.
Founder control is the innovator's-dilemma antidote: Larry and Sergey control the company, would 'rather go bankrupt than lose at AI,' and the hosts read the antitrust outcome — no Chrome divestiture — as the domino that Ilya's departure tipped toward Google's survival.
Deep Read
How did Google create modern AI - and can it still win the AI era?
The transformer, the TPU, and DeepMind were all Google's - yet ChatGPT was built by people who left with the ideas. This episode's history explains how Google invented modern AI, why it let the value leak, and whether founder control plus a full-stack position - model, chip, cloud, distribution - can win the AI era back.
Read the full articleInterview Highlights
18 exchanges- Q
Why was AlexNet so much better than every other entry in ImageNet 2012, and what did Hinton's team figure out?
Ben Gilbert asked49:02ADavid Rosenthal answeredThey realized the theory had been sitting around since the 1980s, but the hardware finally caught up. Deep neural networks needed massive parallelism, and Moore's Law had only taken CPUs far enough to build a few layers. The aha moment was rearchitecting neural networks for a completely different class of chip: consumer video game graphics cards, which are inherently highly parallel. It was counterintuitive — normally cutting-edge academic research ran on supercomputers, and instead they used gaming cards that retailed for a couple hundred bucks. Hinton, Krizhevsky and Sutskever walked into a local electronics store, bought two Nvidia GeForce GTX 580s — Nvidia's top-of-the-line gaming cards at the time — rewrote their algorithms in CUDA, Nvidia's programming language, and trained on those two off-the-shelf cards. That is how they did roughly 40 percent better than any other entry in the ImageNet competition. Jensen Huang calls this the big bang moment of the modern AI era: if two gaming cards can do this, imagine more of them, or a purpose-built chip. It set Nvidia on the path from struggling PC gaming accessory maker to the most valuable company in the world, and it showed the field that breakthroughs arrive when you match an old idea to new hardware — then spend years optimizing it.
- Q
How did DeepMind end up bidding in Hinton's DNN Research auction while Facebook did not?
Ben Gilbert asked54:54ADavid Rosenthal answeredThe timing is the giveaway: the auction ran concurrently with the NIPS conference — then called NIPS, now NeurIPS. Geoff Hinton ran it himself from his hotel room at the Harrah's Casino in Lake Tahoe, and Cade Metz's book The Genius Makers chronicles it. The bidding climbed with Microsoft dropping out and coming back in, while DeepMind dropped out early as promised: a startup with no cash. At the end it came down to Baidu and Google going at each other. At that point the three researchers looked at each other, decided where they actually wanted to land, and stopped the bidding at $44 million with the line: 'Google, this is more than enough money. We're going with you.' The plan had been to split it evenly, but Alex Krizhevsky and Ilya Sutskever went to Hinton and insisted he take 40 percent, keeping 30 percent each — which is how it ends up breaking down.
- Q
How did the deal to sell DeepMind to Google actually come together — and why did Larry Page buy it?
Ben Gilbert asked01:19:17ADavid Rosenthal answeredEarly in DeepMind's life, Elon Musk was sitting on a private jet with Luke Nosek — PayPal mafia member and DeepMind angel investor — reading Demis's email about DeepMind's AI learning, with no human training, a clever way to win at the Atari game Breakout: bounce the ball around the edge of the bricks and let it ricochet along the top. Larry Page was on that same plane — he and Musk were still close then — and when Larry asked what company this was, that is how Google's founder found out about DeepMind. The deal itself got done on kinship: Larry has always seen Google as an AI company, and Demis felt Larry got the mission. Google already had Brain shipping products, so Larry could say stay in London and keep working on intelligence, and DeepMind could plug into effectively infinite compute. Alan Eustace, carrying Larry's 'Let's really go big' blank check from the strategy offsite, flew to London in December 2013 — chartering a jet with a custom harness rig so that Hinton, who cannot sit, could lie on the floor during takeoff and landing — ran the diligence, and Google bought DeepMind for $550 million, with an independent oversight board for the mission. Had DeepMind stayed independent, it would be worth half a trillion dollars today.
- Q
What were the second-order effects of Google buying DeepMind?
Ben Gilbert asked01:30:45ADavid Rosenthal answeredOne person was really, really, really upset: Elon Musk — and maybe two if you count Mark Zuckerberg, who plays his cards closer to the vest. Elon had invested in DeepMind and wanted it for his own AI team at Tesla, and now Alan Eustace was flying around the world sucking up every AI researcher while Elon was already souring on Google. That pressure produced one of the most fateful dinners in Silicon Valley history, organized in summer 2015 at the Rosewood Hotel on Sand Hill Road by Elon and Sam Altman, then president of Y Combinator. The pitch: Google — and to a certain extent Facebook — have created a duopoly on AI talent, so let's do something about it. The one researcher they wanted that Google wouldn't let go was Ilya Sutskever; a counter-offer from Jeff Dean reportedly doubled whatever number was floated. Still, seven researchers left Google, and the group founded OpenAI as a nonprofit with a mission to build safe AGI benefiting humanity. A billion dollars was pledged by Musk, Altman, Hoffman, Livingston and Thiel — though it was later reported only about $130 million was ever actually collected, which was plenty for the research and compute of those first years.
- Q
After OpenAI raided Google's AI talent, where did Google actually stand — and what happened when Krizhevsky arrived in 2013?
Ben Gilbert asked01:40:22ADavid Rosenthal answeredSo back in 2013, when Alex Krizhevsky arrives at Google with Hinton and Ilya, he is shocked to discover that all machine learning runs on CPUs — infrastructure pushes back on GPUs, saying 'we're a CPU shop here.' So he does what any academic would: buys a GPU machine from an electronics store, sticks it in a closet down the hall, and trains like in academia — except Google pays the electricity bill. DistBelief, Google's first-generation deep learning system, needs rearchitecting for GPUs, and by spring 2014 Jeff Dean and John Giannandrea — who would later lead AI at Google and then Apple — hatch the plan to formally put GPUs into the fleet: 40,000 GPUs from Nvidia for $130 million. The proposal is elevated to Larry Page, who personally approves it over his finance team's objection, reasoning that the future of Google is deep learning. For perspective, Nvidia then is a $4-billion-revenue consumer graphics company worth about $10 billion — almost like Google handed Nvidia a secret. The GPUs pay for themselves many times over in Google Photos, Gmail typing suggestions and AdWords click prediction — and running neural networks at that scale exposes the next giant problem: giant matrix multiplications.
- Q
Facing billions of dollars a year flowing to Nvidia, how did Google decide to build its own AI chip?
Ben Gilbert asked01:44:21ADavid Rosenthal answeredAfter Google ships speech recognition on Nexus phones, there's an amazing moment: it only runs on Nexus — no infrastructure exists to serve all of Android. Jeff Dean does the math — three minutes per day across a billion Android phones would require twice the data centers — and delivers the line: 'we need another Google,' or we build a new type of chip. The idea lands with Jonathan Ross noodling on FPGAs in his 20-percent time; that becomes a formal project, then a custom ASIC: the TPU. The trade-off: far more efficient than a GPU, but it only does one thing — the matrix multiplications behind neural nets. The core insight is reduced precision: 4586.8272 becomes 4586.8 becomes 4586, quantization handled in software, so the same power, memory and transistors yield far more calculations per second. The chip is designed, verified, built and deployed in 15 months — FPGAs as the stopgap test fleet, a hard-drive form factor so it slots into existing racks — built in Madison, Wisconsin, kept secret for over a year before Google I/O, and ready just in time for AlphaGo — one machine, four TPUs. Today Google runs an estimated two to three million TPUs against the roughly four million GPUs Nvidia ships in a year: an internal chip effort at almost Nvidia scale.
- Q
Where did the transformer come from, and what had Google's language models been doing before it?
Ben Gilbert asked01:52:17ADavid Rosenthal answeredAfter Franz Och's Google Translate success and the post-AlexNet switch to neural language models, a Jeff-Dean-led group parachuting into Translate proved the concept and threw away the entire old codebase — the New York Times Magazine's famous 2016 story. That rewrite was state-of-the-art but RNN-based, and recurrent networks have a fundamental limitation: they forget too quickly, like reading a book and retaining only the last few pages. LSTMs — long short-term memory networks — fixed some of that with persistent memory, and in 2016 Google shipped an LSTM-based Translate with error rates down roughly 60 percent. But LSTMs are computationally intensive and don't parallelize — the opposite of where hardware was heading. So the Brain team searched for a better architecture. Jakob Uszkoreit's insight was to broaden 'attention': instead of attending to the previous few words, attend to the entire corpus, like professional human translators who read the whole text before translating. Expensive computationally, but extremely parallelizable. Noam Shazeer joined as co-author, pulled a Jeff Dean by rewriting the codebase from scratch, and the transformer crushed LSTM-based Translate — the bigger they made the model, the better the results. When Steven Levy profiled the team for Wired, his lead quote was Noam's three words: 'It works now.'
- Q
What came of Google letting its eight researchers publish 'Attention Is All You Need'?
Ben Gilbert asked02:02:23ADavid Rosenthal answeredObviously the title nods to the classic Beatles song about love. As of 2025 the paper has been cited over 173,000 times and sits seventh among the most-cited papers of the twenty-first century — every paper above it on the list has been out much longer. Within a couple of years, all eight authors had left Google to start or join AI startups, OpenAI among them. Noam Shazeer went off to found Character.AI and later came back to Google through a strange licensing-and-hiring arrangement reported on the order of a few billion dollars — a very, very expensive mistake. 2017 begins what you can call the five-year period of Google not sufficiently seizing the opportunity that it had created with the transformer.
- Q
After Elon walked away, where did OpenAI's first billion dollars come from?
Ben Gilbert asked02:08:48ADavid Rosenthal answeredSam Altman turns to one of the other board members of OpenAI: Reid Hoffman, who had sold LinkedIn to Microsoft a year or so earlier and now sat on Microsoft's board. Reid's intro: come talk to Satya. And do you know where Satya and Sam actually met? The Allen & Company Sun Valley Conference in Idaho, in July 2018. The deal: one billion dollars in cash, plus Azure cloud credits, and in exchange Microsoft gets an exclusive license to OpenAI's technology for Microsoft products. Structurally, the nonprofit OpenAI Inc. creates a captive for-profit, OpenAI LP, and Microsoft's investment goes into the LP; the board includes Hoffman, Altman, Sutskever, Greg Brockman, Adam D'Angelo and Tasha McCauley. Thus the modern OpenAI for-profit-nonprofit-question-mark is created — a structure people are still trying to figure out in 2025. It is, in a sense, the complete history of AI in one deal. And with Microsoft's billion dollars on the table, Microsoft is back as Google's mortal enemy.
- Q
Google had a ChatGPT-like chatbot before ChatGPT — why could it never launch?
Ben Gilbert asked02:20:26ADavid Rosenthal answeredNoam Shazeer — the engineer who rearchitected the transformer and co-authored the paper — looked at what his team had built and said: we can use this for a lot more than Google Translate. The transformer paper's last paragraph says almost exactly that: the approach extends to images, audio, video. Google did none of it for a while. Noam's advocacy went further: throw away the search index and the ten blue links, and transform all of Google into one giant transformer model. He built a chatbot interface — before LaMDA; it was called Meena — in the late 2010s, and it was arguably close to ChatGPT in capability. What it didn't have was post-training for safety — its 'who should die'-style answers made it unshippable — and no RLHF, which is what ChatGPT had in 3.5. Even at OpenAI's quality bar it couldn't launch, for two business-model reasons. One: direct answers drop revenue versus ten blue links — a non-starter inside Google until 2021. Two: the legal risk of sitting between publishers and users; Google spent decades fighting the disintermediation perception, shipping info boxes only on non-monetizable queries. And no compelling reason to force it: nobody was really asking — the general public didn't know this was possible, and OpenAI didn't really know either. GPT was out there, but quietly.
- Q
How did the ChatGPT launch actually happen?
Ben Gilbert asked02:25:34ADavid Rosenthal answeredWell, I think I do — I have it in my notes here. By late 2022 GPT-3.5 was useful, but how do you productize it? Sam Altman's take: we should make a chatbot — can someone just make a chat interface? Within roughly a week, someone wrapped GPT-3.5 API calls into a chat product. It was a magic product, completely unexpected, and the servers started tipping over; OpenAI found itself cutting deals with Microsoft in real time for more compute, Azure credits and cash advances. Expecting an API-and-B2B business, they threw a Stripe paywall over a weekend to dampen expensive consumer usage — and consumer revenue took off anyway. To say OpenAI had any idea what was coming would be completely false; Ben Thompson's line is that ChatGPT made them an accidental consumer tech company. There's an alternative motivation floating around too — a rumor that Dario Amodei and Anthropic were building a Claude chat interface, so launch before they do. It shipped November 30, 2022, as a low-key research preview; Sam tweeted 'Today we launched ChatGPT. Try talking with it here.' Within a week: one million users. By December 31: 30 million. By the end of January 2023 — two months in — it crossed 100 million registered users, the fastest product in history to that milestone.
- Q
What did Sundar Pichai's December 2022 'code red' at Google actually mean?
Ben Gilbert asked02:34:06ADavid Rosenthal answeredUp until this point, Google and Sundar and Larry and everyone had been thinking about AI as a sustaining innovation, in Clayton Christensen's terms — it makes great products better, entrenches incumbents, and is predictable in its capital requirements. ChatGPT changed the category overnight: this is disruptive innovation, an existential threat, and the strengths of ten to twenty years of AI work suddenly look like liabilities, because Google has a lot of castles to protect — filters and safeguards stand between it and simply out-OpenAI-ing OpenAI. So the code red means: we need to build and ship real native AI products ASAP. It is the textbook incumbent response to disruption: don't bury your head in the sand, build comparable products, and be laser-operational about where the new product cannibalizes versus complements. What follows has been a multi-year ballet: protect search growth while creating the best AI experiences. AI Overviews appear for some queries but not all; AI Mode opens to some users but not all; Gemini ships as the full app, but Google.com does not redirect. The delicate dance is protecting the existing franchise while building a hopefully non-cannibalizing new one.
- Q
How did Google respond in 2023 — and what were Sundar Pichai's two big decisions?
Ben Gilbert asked02:37:27ADavid Rosenthal answeredCode Red goes out December 2022. In January OpenAI hits a hundred million users and Microsoft puts in another ten billion — 49 percent of the for-profit entity — Google's mortal enemy owns the first true existential threat. In February Microsoft ships Bing with GPT, and Google scrambles: LaMDA plus a chat interface rebranded as Bard ships publicly. Bard is a bad product — missing the RLHF magic of tone and correctness tuning — and the launch video contains a factual error: one of history's worst keynotes, the stock down eight percent that day. In May LaMDA gives way to Brain's PaLM — better, still behind GPT-3.5 and March's GPT-4. Sundar makes two big decisions. First: merge Brain and DeepMind into Google DeepMind, in violation of the original deal terms — Demis becomes CEO, a decade of culture change in a stroke; read between the lines: the models came from the DeepMind side. Second: one model for everything — Gemini serves all of Google internally, all AI products externally. The logic is scaling laws — multiple model families repeat enormous costs — and unlike Google Plus, this is Google's technical wheelhouse. And naming the consumer service after the model — Bard becoming Gemini — is the Gmail move: this product is nothing but technology, a rallying point for Google culture.
- Q
What is the origin story of Waymo, from the DARPA Grand Challenge to the Larry 1000?
Ben Gilbert asked02:46:57ADavid Rosenthal answeredRewind to 2004 and the DARPA Grand Challenge — the seed pool of the self-driving revolution twenty years later: 132 miles of Mojave dirt roads, no humans allowed, a one-million-dollar prize — a break from DARPA's grants-only policy that took an act of Congress. In 2004, zero of a hundred-plus teams finished. In 2005 the pot doubled, twenty-two of twenty-three finalists beat the previous year's best, five finished. Two were Carnegie Mellon's; the other was Stanford's, led by Sebastian Thrun — commodity sensors on a nearly unmodified VW, innovating only in software — very googly. Their early sensor fusion: a precise narrow-field roof laser, a coarse wide-angle camera; a real-time overlay extrapolates safe-area colors forward to find a safe path. Stanford's car passed a Carnegie Mellon team autonomously. Larry hired Thrun — don't raise from Benchmark or Sequoia, we'll hire you — into Street View, and Thrun co-founded Google X, whose first project, Chauffeur, became Waymo. Larry: that self-driving car stuff, do it. Thrun resisted — multi-ton killer robots — until Larry asked what technical reason made it impossible — there wasn't one: 'I'm just afraid.' The Larry 1000 — ten difficult California stretches — was done by a dozen people within eighteen months of the 2009 start; the unreal part is the second 99 percent: ten years to productize.
- Q
How did the US v. Google search-monopoly case land, and why did the court decline real remedies?
Ben Gilbert asked03:15:08ADavid Rosenthal answeredBetween our Alphabet episode and this one, the United States government v. Google search-monopoly case played out. The judge ruled Google is a monopoly in internet search — and then declined to impose material remedies. There are some remedies, but immaterial ones: no Chrome spinoff, no stopping the tens of billions Google pays Apple for default placement. The reasoning, essentially: Google's a monopoly, but the cost of doing anything about that would have too many downstream consequences — and the judge cited the AI race: with tens of billions flowing into OpenAI, Anthropic and Perplexity, there's a new war on, leave it to the free market, and don't hamstring Google. David is skeptical: none of those challengers generates net income, fundraises don't last forever, cash burns fast — if the spigots dry up, Google faces no self-sustaining competition in search or AI. But the glib-but-meritorious counter: as flat-footed as Google was, avoiding a Microsoft-level distraction and judgment was worth it. There's even a domino theory where Ilya leaving Google leads to OpenAI leads to Google not being broken up — the antitrust forbearance actually saves Google. It's totally wild.
- Q
Why did Google Cloud struggle for a decade, and how did Thomas Kurian turn it around?
Ben Gilbert asked03:22:55ADavid Rosenthal answeredWe left cloud out of our Alphabet episode — it seemed peripheral then and turned out to be hugely strategic for AI. It started as App Engine in 2008: opinionated platform-as-a-service. AWS proved IaaS right, Microsoft pivoted Azure fast, and Google launched its IaaS only in 2012 — Compute Engine, then GCP. The knock: Google never figured out the enterprise interface — self-serve, advertiser-funded, never forced to delight enterprises. And cloud is a knife fight — lowest price, enterprise relationships, bundling, solutions; you say solution, I hear gross margin. Google was out of its habitat and wouldn't give away the crown jewels — no Bigtable, no Borg. Then came Thomas Kurian, ex-Oracle president, hired late 2018 — and everything changed: cloud took ten years to reach four billion; by 2020 it was thirteen billion, tripled in three years, with ten thousand go-to-market hires on a base of one-fifty. Google was a cloud company all along — best engineers, best infrastructure — but go-to-market and productization were too googly. Twenty twenty-two: twenty-six billion. Twenty twenty-three: a real third cloud, and profitable. Today: a fifty-billion-plus run rate, up thirty percent year over year, the fastest-growing major cloud — riding a religion about serving enterprises, multicloud strategy, and the AI tailwind: Google Cloud sells TPUs while everyone else begs Nvidia for GPU allocations.
- Q
Why is Google Cloud the distribution mechanism for AI — and why is Google the only company with every AI pillar?
Ben Gilbert asked03:28:49ADavid Rosenthal answeredOne is simply that cloud is the distribution mechanism for AI. To play at all in AI you need one of: a great application, a great model, a great chip, or the cloud — and Google is trying to have all four. No other company has more than one. Nvidia has chips — and, kind of, cloud, but not really. Meta has an application only, and at the moment it's out of the frontier-model race — though the hiring spree might change that. Amazon has the cloud, and maybe an application. Microsoft has the cloud and makes some models. Apple has nothing here; AMD has chips; OpenAI and Anthropic have models — neither has its own data centers at scale, and the chip efforts are not at scale either. Google, meanwhile, has data centers at scale, chips at scale, model usage at scale through google.com's AI Overviews, and applications at scale — every pillar of AI. And it has the very most net-income dollars to lose.
- Q
What are the unit economics of AI chips — and why does Google's TPU margin matter so much?
Ben Gilbert asked03:36:45ADavid Rosenthal answeredEveryone pays Nvidia 75 to 80 percent gross margins, implying a four-to-five-x markup on the underlying hardware — the Jensen tax. Pricing power and scarcity: not building your own chips means paying a giant premium. Google pays Broadcom roughly a 50 percent margin on TPU work — the difference between paying a two-x markup versus a five-x one, which at this scale is a giant cost impact. Chips drive AI total cost of ownership: given the pace of next-generation models, depreciation runs five years or less; estimates put over half the cost of an AI data center at chips and their depreciation, people and R&D at a quarter to a third, power at two to six percent. Even Jensen, introducing Blackwell at GTC, said you really don't want Hopper — the H100s everyone chased at our last Nvidia episode. The economics are incredibly sensitive to your chip supplier's margin. One sanity check from Gavin Baker of Atreides Management: historically it hasn't mattered to be the low-cost producer — Google didn't win search by being cheapest, and Apple didn't win on cost. But this era might be different: AI companies run fifty percent gross margins at best, so Google being definitively the low-cost producer of tokens — own infrastructure, low-markup hardware — might decide who wins producing tokens for the world.