Google’s growing AI chip business threatens Nvidia and could alter the marketplace.
Google caused two significant disruptions in the AI chip field last month. The first one is the release of its seventh-generation tensor processing unit (TPU), codenamed Ironwood. The chip offers a significant improvement in inference processing, for which it was custom built. Ironwood also offers massive memory scale and bandwidth, which are needed in AI processing.
The second disruption came a few weeks later and is much more significant. Word spread quickly that Meta is considering making a significant purchase – reportedly 100,000 units – of the Google TPUs for its own hyperscale facilities. There is also speculation that Google will seek out other customers for its TPU.
This sent ripples through the AI silicon market, particularly because of the potential Nvidia impact. Nvidia is so dominant now that people are looking for any excuse to take it down a peg, and a legitimate competitor is a good one.
In addition, Google going into the business of selling AI processors would be a significant departure from the direction hyperscalers have been taking up until now. Virtually every hyperscaler is building its own custom silicon, some for general computing and others specifically for AI processing. But up until now, hyperscalers have kept their homegrown silicon for themselves.
So, could Google kick off a new arms race in silicon, where Nvidia and AMD face competition from their biggest customers: Google, AWS, and Microsoft? Analysts say maybe but not likely.
“Are they likely to sell the TPUs? Yes. Are they going to compete directly with Nvidia? No. Because the TPUs are not meant to compete directly with Nvidia. The TPUs really are meant to be more targeted at doing smaller scale, less intense model processors,” said Jack Gold, president of J.Gold & Associates.
The Nvidia processors, he explains, are for processing massive, large language models (LLM), while the Google TPU is used for inferencing, the next step after processing the LLM. So, the two chips don’t compete with each other, they complement each other, according to Gold.
Selling and supporting processors may not be Google’s core competence, but they have the skills and experience to do this, said Alvin Nguyen, senior analyst with Forrester Research. “They have had their TPUs available, from what I understand, to some outside companies already, mainly startups from ex-Googlers or Google-sponsored startups,” he said.
As to the rumor of the Meta purchase, the question is what Meta wants to do with them, said Gold. “If they’ve already built out a model and they’re running inference workloads, then Nvidia B100s and B200s are overkill,” Gold said. “And so what are the options there are now? There are a number of startups that are trying to do inference-based chips as well, and Intel and AMD are moving in that direction as well. So it really is a function of getting a chip that’s optimized for their environment, and again, Google’s TPUs are optimized for a hyperscaler cloud type of environment.”
Nguyen said it’s one thing to make their own chips for their own use, but it’s another thing to be selling them. That’s an infrastructure and a competency that Google doesn’t have, and Intel, AMD and Nvidia are way ahead of Google in that regard, he said.
“Yes, they know how to do it for themselves, as long as you were talking about as a service or as a cloud service. For on premises, or for people who want to take it for themselves, that’s a muscle memory they have to develop,” Nguyen said.
For that reason, Nguyen doubts that other hyperscalers with their own custom silicon will go into the chip-selling business for themselves. “There’s nothing to stop them, but each of them has their own challenges,” said Nguyen. Microsoft, AWS and OpenAI all have multiple partnerships and would inevitably end up in competition with somebody.
Gold said he can’t see AWS and Microsoft going into the chips business. “I can’t see that happening. I really can’t. It’s just not a business model for those guys that makes a lot of sense in my mind,” Gold said.
