For greater than a decade, Nvidia’s GPUs have underpinned nearly every major advance in modern AI. That position is now being challenged.
Frontier models reminiscent of Google’s Gemini 3 and Anthropic’s Claude 4.5 Opus were trained not on Nvidia hardware, but on Google’s latest Tensor Processing Units, the Ironwood-based TPUv7. This signals that a viable alternative to the GPU-centric AI stack has already arrived — one with real implications for the economics and architecture of frontier-scale training.
Nvidia's CUDA (Compute Unified Device Architecture), the platform that gives access to the GPU's massive parallel architecture, and its surrounding tools have created what many have dubbed the "CUDA moat"; once a team has built pipelines on CUDA, switching to a different platform is prohibitively expensive due to dependencies on Nvidia’s software stack. This, combined with Nvidia's first-mover advantage, helped the corporate achieve a staggering 75% gross margin.
Unlike GPUs, TPUs were designed from day one as purpose-built silicon for machine learning. With each generation, Google has pushed further into large-scale AI acceleration, but now, because the hardware behind two of essentially the most capable AI models ever trained, TPUv7 signals a broader technique to challenge Nvidia’s dominance.
GPUs and TPUs each speed up machine learning, but they reflect different design philosophies: GPUs are general-purpose parallel processors, while TPUs are purpose-built systems optimized almost exclusively for large-scale matrix multiplication. With TPUv7, Google has pushed that specialization further by tightly integrating high-speed interconnects directly into the chip, allowing TPU pods to scale like a single supercomputer and reducing the fee and latency penalties that typically include GPU-based clusters.
TPUs are "designed as an entire 'system' fairly than simply a chip," Val Bercovici, Chief AI Officer at WEKA, told VentureBeat.
Google's business pivot from internal to industry-wide
Historically, Google restricted access to TPUs solely through cloud rentals on the Google Cloud Platform. In recent months, Google has began offering the hardware on to external customers, effectively unbundling the chip from the cloud service. Customers can choose from treating compute as an operating expense by renting via cloud, or a capital expenditure (purchasing hardware outright), removing a significant friction point for big AI labs that prefer to own their very own hardware and effectively bypassing the "cloud rent" premium for the bottom hardware.
The centerpiece of Google's shift in strategy is a landmark cope with Anthropic, where the Claude 4.5 Opus creator will receive access to as much as 1 million TPUv7 chips — greater than a gigawatt of compute capability. Through Broadcom, Google's longtime physical design partner, roughly 400,000 chips are being sold on to Anthropic. The remaining 600,000 chips are leased through traditional Google Cloud contracts. Anthropic's commitment adds billions of dollars to Google's bottom line and locks one in all OpenAI's key competitors into Google's ecosystem.
Eroding the "CUDA moat"
For years, Nvidia’s GPUs have been the clear market leader in AI infrastructure. Along with its powerful hardware, Nvidia's CUDA ecosystem incorporates a vast library of optimized kernels and frameworks. Combined with broad developer familiarity and an enormous installed base, enterprises regularly became locked into the "CUDA moat," a structural barrier that made it impractically expensive to desert a GPU-based infrastructure.
Considered one of the important thing blockers stopping wider TPU adoption has been ecosystem friction. Prior to now, TPUs worked best with JAX, Google's own numerical computing library designed for AI/ML research. Nonetheless, mainstream AI development relies totally on PyTorch, an open-source ML framework that may be tuned for CUDA.
Google is now directly addressing the gap. TPUv7 supports native PyTorch integration, including eager execution, full support for distributed APIs, torch.compile, and custom TPU kernel support under PyTorch’s toolchain. The goal is for PyTorch to run as easily on TPUs because it does on Nvidia GPUs.
Google can also be contributing heavily to vLLM and SGLang, two popular open-source inference frameworks. By optimizing these widely-used tools for TPU, Google ensures that developers are in a position to switch hardware without rewriting their entire codebase.
Benefits and drawbacks of TPUs versus GPUs
For enterprises comparing TPUs and GPUs for large-scale ML workloads, the advantages center totally on cost, performance, and scalability. SemiAnalysis recently published a deep dive weighing the benefits and drawbacks of the 2 technologies, measuring cost efficiency, in addition to technical performance.
Due to its specialized architecture and greater energy efficiency, TPUv7 offers significantly higher throughput-per-dollar for large-scale training and high-volume inference. This permits enterprises to scale back operational costs related to power, cooling, and data center resources. SemiAnalysis estimates that, for Google's internal systems, the full cost of ownership (TCO) for an Ironwood-based server is roughly 44% lower than the TCO for an equivalent Nvidia GB200 Blackwell server. Even after factoring within the profit margins for each Google and Broadcom, external customers like Anthropic are seeing a ~30% reduction in costs in comparison with Nvidia. "When cost is paramount, TPUs make sense for AI projects at massive scale. With TPUs, hyperscalers and AI labs can achieve 30-50% TCO reductions, which could translate to billions in savings," Bercovici said.
This economic leverage is already reshaping the market. Just the existence of a viable alternative allowed OpenAI to negotiate a ~30% discount by itself Nvidia hardware. OpenAI is one in all the biggest purchasers for Nvidia GPUs, nevertheless, earlier this 12 months, the corporate added Google TPUs via Google Cloud to support its growing compute requirements. Meta can also be reportedly in advanced discussions to acquire Google TPUs for its data centers.
At this stage, it would seem to be Ironwood is the best solution for enterprise architecture, but there are quite a lot of trade-offs. While TPUs excel at specific deep learning workloads, they’re far less flexible than GPUs, which might run a wide selection of algorithms, including non-AI tasks. If a brand new AI technique is invented tomorrow, a GPU will run it immediately. This makes GPUs more suitable for organizations that run a wide selection of computational workloads beyond standard deep learning.
Migration from a GPU-centric environment will also be expensive and time-consuming, especially for teams with existing CUDA-based pipelines, custom GPU kernels, or that leverage frameworks not yet optimized for TPUs.
Bercovici recommends that corporations "go for GPUs after they must move fast and time to market matters. GPUs leverage standard infrastructure and the biggest developer ecosystem, handle dynamic and sophisticated workloads that TPUs aren't optimized for, and deploy into existing on-premises standards-based data centers without requiring custom power and networking rebuilds."
Moreover, the ubiquity of GPUs implies that there’s more engineering talent available. TPUs demand a rare skillset. "Leveraging the facility of TPUs requires a company to have engineering depth, which implies having the ability to recruit and retain the rare engineering talent that may write custom kernels and optimize compilers," Bercovici said.
In practice, Ironwood’s benefits may be realized mostly for enterprises with large, tensor-heavy workloads. Organizations requiring broader hardware flexibility, hybrid-cloud strategies, or HPC-style versatility may find GPUs the higher fit. In lots of cases, a hybrid approach combining the 2 may offer one of the best balance of specialization and adaptability.
The long run of AI architecture
The competition for AI hardware dominance is heating up, however it's far too early to predict a winner — or if there’ll even be a winner in any respect. With Nvidia and Google innovating at such a rapid pace and corporations like Amazon joining the fray, the highest-performing AI systems of the long run could possibly be hybrid, integrating each TPUs and GPUs.
"Google Cloud is experiencing accelerating demand for each our custom TPUs and Nvidia GPUs,” a Google spokesperson told VentureBeat. “Consequently, we’re significantly expanding our Nvidia GPU offerings to satisfy substantial customer demand. The truth is that the vast majority of our Google Cloud customers use each GPUs and TPUs. With our big range of the newest Nvidia GPUs and 7 generations of custom TPUs, we provide customers the pliability of selection to optimize for his or her specific needs."
