How competition among ASIC service providers can help Google save billions in TPU

Investing.com -- Google could save billions of dollars annually on its custom AI chips, known as Tensor Processing Units (TPUs), thanks to rising competition among application-specific integrated circuit (ASIC) service providers, according to analysts at Bernstein.

In a new report this week, Bernstein estimates that TPUs currently offer about “~4x performance per dollar vs. current-gen GPU,” though this measure only reflects chip acquisition cost. 

When accounting for total system costs—including software, R&D, and packaging—savings may still range from “1.5-4x,” underscoring TPUs’ economic advantage in certain AI workloads, according to Bernstein.

Much of this advantage is said to stem from high margins on rival graphics processing units (GPUs). 

Bernstein writes that while the bill of materials (BOM)—covering memory and foundry costs—differs by only “1.3-2.4x between TPU & GPU,” the remaining cost gap is “from NVIDIA’s gross margins which can run 70% or higher.”

As the size and cost of training large AI models continue to rise faster than improvements in hardware efficiency, Bernstein believes increased use of ASICs like TPUs will be necessary. 

“Moore’s Law isn’t sufficient,” the analysts noted, as it cuts cost per operation by just 25%–40% annually, while model size grows roughly 3.5x per year.

Although some hyperscalers like Amazon (NASDAQ:AMZN) and Google (NASDAQ:GOOGL) are building ASICs in-house, others—including Meta (NASDAQ:META), Microsoft (NASDAQ:MSFT), OpenAI, and xAI—remain dependent on external providers.

Bernstein concludes that despite potential “demand lumpiness” and the addition of new customers, “will keep the overall ASIC market growing in the long run.”

3rd party Ad. Not an offer or recommendation by Investing.com. See disclosure here or remove ads.

 

MSFT: A Bull or Bear Market Play?

Don't miss out on the next big opportunity! Stay ahead of the curve with ProPicks – 6 model portfolios fueled by AI stock picks with a stellar performance this year..In 2024 alone, ProPicks' AI identified 2 stocks that surged over 150%, 4 additional stocks that leaped over 30%, and 3 more that climbed over 25%. That's an impressive track record.With portfolios tailored for Dow stocks, S&P stocks, Tech Stocks, and Mid Cap stocks, you can explore various wealth-building strategies.So if MSFT is on your watchlist, it could be very wise to know whether or not it made the ProPicks lists.

Unlock ProPicks now