[Updated] Google's new beast: Ironwood TPU System Architecture and Performance

Описание: The slide provides overview on the Ironwood TPU System, a significant breakthrough in hardware for training and serving reasoning models. Key innovations include linking 9216 Ironwood chips using Optical Circuit Switches (OCS) to establish a 1.77 PB directly addressable shared HBM memory, setting a new record for shared-memory multiprocessors. This superpod configuration achieves a massive 42.5 Exaflops of ML Compute using FP8 precision, while emphasizing RAS (reliability, availability, and serviceability) for hyperscale deployment. Furthermore, the system boasts industry-leading compute power efficiency, showing a 2x perf/W increase over the previous generation, facilitated by a 3rd generation liquid cooling infrastructure and the acceleration capabilities of the 4th-generation SparseCore.