TPU vs GPU explained: what Google's tensor processing units are and how they differ from Nvidia chips

When people ask whether Google's chips can compete with Nvidia's, they are comparing two different designs for the same job. That job is multiplying very large grids of numbers, which is what training and running modern AI models mostly consists of. A GPU, or graphics processing unit, was designed first for graphics and later adopted for AI. A TPU, or tensor processing unit, was designed from the start for machine learning. The difference is real, and it is smaller in practice than the marketing suggests.

What a TPU is

Google's documentation describes TPUs as "Google's custom-developed, application-specific integrated circuits (ASICs)" used to accelerate machine learning workloads. They use hardware built for the large matrix operations common in machine learning, and they include on-chip high bandwidth memory. According to Wikipedia, Google began using TPUs internally in 2015 and announced them in May 2016 at Google I/O, saying they had been running in its data centers for over a year.

The core hardware unit is the matrix multiply unit. Google's documentation describes it as a 128 by 128 systolic array. A systolic array is a grid of simple arithmetic units that pass data to their neighbors in a steady rhythm, so numbers flow through the grid and results come out the other side without the chip constantly fetching data from memory. The first generation, per Wikipedia, used a 256 by 256 array of 8 bit multipliers.

What a GPU is

A GPU is a processor with thousands of small cores that run the same operation on many pieces of data at once. Nvidia describes CUDA, the software layer for its GPUs, as "a parallel computing platform and programming model developed by NVIDIA" that "enables dramatic increases in computing performance by harnessing the power of the GPU." Modern Nvidia data center GPUs add dedicated units for matrix math, which narrows the hardware gap with TPUs. We have not compared the two on spec sheets here, because the numbers depend on precision, generation and how the chips are connected.

The real difference is flexibility versus focus

Google's own guidance on when to use each is useful because it is not marketing. It says CPUs are best for quick prototyping and simple models. GPUs suit models with many custom PyTorch or JAX operations that must partly run on the CPU, models with TensorFlow operations not available on Cloud TPU, and medium to large models. TPUs, it says, suit models dominated by matrix computations, models without custom PyTorch or JAX operations in the training loop, and long running training with large batch sizes.

In other words, a TPU is a specialist. If your workload is mostly large matrix multiplication, it can be very efficient. If your workload has unusual operations, a GPU is more forgiving. That is the trade.

The software gap

The less discussed difference is software. TPU code must be compiled by the XLA compiler, which Google describes as a just in time compiler. GPU programs are usually written against CUDA or frameworks built on it. Teams that built their tools around one stack pay a switching cost to move to the other. Google's documentation says it supports TensorFlow, PyTorch and JAX on TPUs, which lowers that cost, but it does not remove the engineering work of tuning a model for a new chip.

This is why reports of labs moving between chips deserve scrutiny. Our piece on the reported OpenAI plan to buy Google TPUs notes that the same report mentions job postings for engineers to optimize frontier models for TPUs. That is the software work in action.

Generations and who can buy

Google lists TPU7x (Ironwood), v6e, v5p, v5e and v4 among its generations, with older v3 and v2 documented separately. Wikipedia gives Ironwood's peak compute as 4,614 TFLOP/s at fp8 precision and notes it was announced in April 2025, but those are the encyclopedia's figures and we have not checked them against a Google datasheet. Treat comparisons across chips as indicative only.

TPUs began as an internal Google tool. Google made them available to third parties in 2018 through Cloud TPU, and Wikipedia notes Kaggle and Colaboratory also provide access. In 2025, it adds, Google reportedly discussed deploying TPUs with neoclouds such as Crusoe and CoreWeave, and Meta reportedly discussed deploying them in its AI data centers. The word reportedly matters there.

Which is better

No honest answer exists without naming the workload. For a team already on PyTorch with custom operations, a GPU is the lower-friction choice. For a very large training run dominated by matrix math, where the team can adapt its code for XLA, a TPU can be cost effective. Both vendors publish their own benchmarks, and vendor benchmarks favor the vendor. Look for independent measurements on your own workload.

A short glossary

• ASIC: an application-specific integrated circuit, a chip designed for one kind of job.
• XLA: the compiler Google's documentation says TPU code must go through.
• HBM: high bandwidth memory placed close to the processor so data moves quickly.
• TFLOP/s: trillions of floating point operations per second, a peak-speed measure that depends heavily on numeric precision.

Keep the last item in mind whenever a chip announcement quotes a single large number: peak figures at low precision are not comparable with figures at higher precision. ## Why it matters for the AI race

Compute supply is a main constraint on frontier labs, as we discuss in our explainer on AI compute. If more labs can credibly run on TPUs, the dependence on a single chip supplier eases. If switching costs stay high, it does not. Our take: the right reading of any TPU versus GPU headline is a question about software cost and workload, not a verdict on whose silicon is faster.