Trillium – Google’s sixth-generation Tensor Processing Unit (TPU)

Announcement making Trillium Faster and more efficient for training and serving AI models!

Google’s I/O 2024 was a significant event in the field of artificial intelligence (AI) and machine learning, with the unveiling of Trillium, Google’s sixth-generation Tensor Processing Unit (TPU). Trillium represents a major leap forward in AI hardware, delivering a 4.7x increase in peak compute performance per chip compared to its predecessor, the TPU v5e. This impressive boost in performance is achieved through expanded matrix multiply units and increased clock speeds.

In addition to its enhanced compute performance, Trillium also boasts a doubling of the High Bandwidth Memory (HBM) capacity and bandwidth, as well as the Inter-chip Interconnect (ICI) bandwidth, compared to TPU v5e. These improvements significantly enhance the efficiency and speed of training and serving AI models.

One of the standout features of Trillium is its third-generation Sparse Core, a specialized accelerator for processing ultra-large embeddings common in advanced ranking and recommendation workloads. This feature underscores Google’s commitment to addressing the specific needs of advanced A I workloads.

Trillium also shines in terms of energy efficiency, being over 67% more energy-efficient than TPU v5e. Furthermore, it offers impressive scalability, capable of scaling up to 256 TPUs in a single high-bandwidth, low-latency pod. With multi-slice technology and Titanium Intelligence Processing Units (IPUs), Trillium TPUs can scale to hundreds of pods, connecting tens of thousands of chips in a building-scale supercomputer interconnected by a multi-petabit-per-second datacenter network.

The introduction of Trillium marks a significant milestone in the evolution of AI hardware. It not only sets a new benchmark for AI accelerators but also opens up exciting possibilities for the future of AI and machine learning.

HIGHLIGHTS AND LINKS TO ARTICLES

https://cloud.google.com/blog/products/compute/introducing-trillium-6th-gen-tpus