Hugging Face + TorchTPU: Transformers and Diffusers Now Run on TPUs Natively

Hugging Face now says its two main libraries, Transformers and Diffusers, run on Google TPUs natively through TorchTPU, a PyTorch backend that makes the TPU look like any other device. In the company's post, Hugging Face loves TorchTPU, the pitch is that you no longer need the older torch_xla path or a JAX rewrite: you load a model with device_map="tpu" and go. This article walks through the setup, the numbers Hugging Face reports, and the places where it still breaks.

All performance figures below are the Hugging Face authors' own measurements; we have not reproduced them.

Update (October 20, 2026): Google open-sources TorchTPU

TechCrunch reports that Google announced TorchTPU as an open-source project on Tuesday, a day after Amazon released TorchNeuron for its Trainium chip (see our TorchNeuron claimed-vs-verified post). According to the article, developers can move existing PyTorch apps to Google's TPUs on Google Cloud with a configuration change rather than a rewrite, which is Google's claim and not something we have tested. Google reportedly worked with Meta, Hugging Face, Inferact (vLLM) and RadixArk (SGLang) so popular tools work out of the box, and also launched a TPU agent suite to help migrate apps and fix errors, plus an updated LiteRT for on-device PyTorch apps. TechCrunch frames both moves as chipping at Nvidia's software advantage, while noting developers will not necessarily leave Nvidia hardware en masse.

TL;DR: TorchTPU on Hugging Face

What is TorchTPU and why does it matter?

TPUs are Google's AI accelerators, and until now using them from PyTorch meant going through torch_xla, a bridge that many teams found awkward, or rewriting models in JAX. TorchTPU is Google's attempt to remove that friction. Google's own developer-blog post, TorchTPU: Running PyTorch Natively on TPUs at Google Scale, describes it as an out-of-tree PyTorch backend where eager mode is the default and an existing script mainly needs its device initialization changed to "tpu". For torch.compile, it captures graphs with Dynamo but lowers through XLA rather than Inductor.

The Hugging Face post is the first large-library proof point. It answers the question practitioners actually ask: can I take the Transformers or Diffusers code I already have and run it on a TPU without a fork? Hugging Face's answer is yes, with caveats that we cover below. For context on why this matters beyond Google's cloud, read our coverage of Amazon's TorchNeuron claim for Trainium: every major cloud chip is racing to become a drop-in PyTorch device, because CUDA's moat is largely software.

How do I run Transformers on a TPU?

On a TPU VM, install the package from the CPU PyTorch index so you avoid pulling the larger CUDA build:

pip install torch-tpu --extra-index-url https://download.pytorch.org/whl/cpu

For Transformers, load the model with device_map="tpu" and send inputs with .to("tpu"). Models run in eager mode by default. You can opt into torch.compile for compiled execution. Hugging Face also points to TPU kernels on the Kernels Hub and to Tokamax ragged-dot kernels for mixture-of-experts models.

TPU custom silicon chip, representing Hugging Face TorchTPU running Transformers on Google TPUs

How do I run Diffusers on a TPU?

For image and video pipelines, call pipe.to("tpu") for eager mode. For compiled mode, Hugging Face shows compiling the transformer component:

pipe.transformer.compile(backend="tpu", fullgraph=True, dynamic=False)

Larger models can be sharded across chips with a TensorParallelConfig. Some pipelines need a custom tp_plan, which is the fiddly part: sharding is not automatic for every architecture. If you are used to running Diffusers on GPUs, see how a recent open image model is set up there in our guide to Qwen-Image 2.1 Turbo with Diffusers.

What do the numbers say?

Hugging Face reports the following, all on Google TPU hardware:

• Test coverage: 96.5% of tests that run on TPU pass. The remaining roughly 3.5% fail.
• Upstream feedback loop: 34 issues reported to the TorchTPU repository, 19 already fixed.
• Kernel decode speedup at batch size 64 on a TPU v6e-8: 21.5x on Qwen3-30B-A3B and 12.8x on Gemma 4 26B-A4B.
• Image generation on a v6e-4: Qwen-Image at 1024x1024 and 40 steps takes 3.5 seconds compiled versus 8.0 seconds eager.
• Video example: 1344x768 at 49 steps takes 188 seconds compiled versus 262 seconds eager.

Read the speedups carefully. The 21.5x and 12.8x figures compare TPU-specific kernels against a baseline in the same stack, at a large batch size, so they say how much the kernels help, not how a TPU compares with a GPU. Hugging Face itself notes that the kernel win grows with batch size. The compiled versus eager numbers are more transferable: compile roughly halves image generation time and trims about 28% from the video run.

How does TorchTPU work under the hood?

Google's developer post gives the design in outline. TorchTPU is built on PyTorch's PrivateUse1 interface, which lets hardware vendors plug in an out-of-tree backend instead of maintaining a fork of PyTorch. Eager mode is the default, so ordinary imperative code runs op by op on the TPU. When you call torch.compile, the graph is captured with Dynamo, then lowered to StableHLO and handed to XLA, which emits the TPU executable. Inductor and Triton, the usual GPU compile path, are not involved.

That matters for two reasons. First, any model that is plain PyTorch should work without being rewritten, which is exactly what Hugging Face is testing at library scale. Second, custom kernels follow a different route: Google says custom kernels written in Pallas and JAX are supported, which is why Hugging Face can ship TPU kernels through its Kernels Hub rather than asking users to write them. Distributed training primitives such as DDP, FSDP2 and DTensor are part of the stated scope, though the Hugging Face post focuses on inference and generation.

On availability, the Hugging Face post links a torch_tpu repository, while Google's post lists the launch of a public GitHub repository among its plans for the rest of 2026 without a date. Check both before building on it.

TorchTPU versus torch_xla versus staying on GPUs

The honest reading is that TorchTPU narrows the first row against the others. It does not make TPUs faster than GPUs for your workload; that still needs your own benchmark, with your batch size, sequence length and resolution. The compile-versus-eager gap in the Hugging Face numbers also shows that tuning matters as much as the backend itself.

What are people asking about TorchTPU?

Is this only for Google Cloud customers? TPUs are available through Google's cloud, so in practice yes, unless Google widens hardware access. The software layer is the part that is changing.

Will my fine-tuning scripts work? The Hugging Face post covers inference and generation for Transformers and Diffusers. Training workflows depend on the distributed APIs, which Google lists as in scope; test them on a small run first.

Can I trust the speedup numbers? They are the authors' measurements on specific TPU slices (v6e-8 and v6e-4). Treat them as upper-bound indications of what tuned kernels can do rather than as a cross-vendor comparison.

What are the limitations?

The post is candid about the rough edges:

1. Not fully green. About 3.5% of TPU tests still fail, and nightly TPU CI in Transformers is still being built.
2. Compile cost. The first call in compiled mode pays compilation time, and later calls reuse the graph only at the same image size. Changing resolution triggers a recompile.
3. Sharding is manual. Large models need tensor-parallel configuration, and some pipelines need a custom plan.
4. Kernel gains vary. Small batches will see less benefit than the headline numbers.

In short, this is a strong beta for people already on TPUs, not a reason to migrate a working GPU stack.

What does this mean for builders?

If you already have TPU credits or capacity, the practical effect is that Hugging Face model code you wrote for GPUs should now run with a few device-string changes, which lowers the cost of trying TPU capacity when GPUs are scarce. If you are GPU-only, it is mostly a signal about where competition is heading: more accelerators becoming first-class PyTorch devices puts pressure on GPU pricing and availability over time. It also lines up with Google's wider TPU push, which we covered in Project Suncatcher's TPU satellite plans, and with PyTorch's own hardware work such as PyTorch 2.15 on Apple Silicon.

Hugging Face's role is also worth noting: Transformers is the common layer between the model hub and many accelerators, which is part of why our NVIDIA and Hugging Face coverage matters for anyone choosing hardware.

Practical next steps

1. Spin up a TPU VM and install torch-tpu from the CPU index as shown above.
2. Start with eager mode on a small Transformers model, confirm outputs match your GPU baseline, then try torch.compile.
3. For Diffusers, benchmark eager versus compiled at your fixed resolution to amortize the compile cost.
4. For large or MoE models, plan time for a TensorParallelConfig and test the Kernels Hub TPU kernels.
5. File issues upstream: Hugging Face says 19 of its 34 reports were already fixed, so the loop is working.

Details reflect the Hugging Face and Google posts as of October 20, 2026; installation commands and test numbers may change.

Related reading

• Amazon TorchNeuron and Trainium: claimed vs verified
• PyTorch 2.15 on Apple Silicon
• Google Project Suncatcher TPU satellite plans
• Qwen-Image 2.1 Turbo with Diffusers
• NVIDIA and Hugging Face
• Transformers and GGUF parity with llama.cpp
• NVIDIA CUDA Rust GPU kernels

Official sources: Hugging Face TorchTPU post and Google Developers Blog on TorchTPU.