Google Cloud announced a new supercomputer virtual-machine series aimed at rapidly training large AI models.\nUnveiled at the Google I\/O conference,\u00a0the new A3\u00a0supercomputer VMs are purpose-built to handle the considerable resource demands of a large language model (LLM).\u00a0\n\u201cA3 GPU VMs were purpose-built to deliver the highest-performance training for today\u2019s ML workloads, complete with modern CPU, improved host memory, next-generation Nvidia GPUs and major network upgrades,\u201d the company said in a statement.\nThe instances are powered by eight Nvidia H100 GPUs, Nvidia\u2019s newest GPU that just begin shipping earlier this month, as well as Intel\u2019s 4th Generation Xeon Scalable processors, 2TB of host memory and 3.6 TBs bisectional bandwidth between the eight GPUs via Nvidia\u2019s NVSwitch and NVLink 4.0 interconnects.\nAll together, Google is claiming these machines can provide up to 26 exaFlops of power. That\u2019s the cumulative performance of the entire supercomputer, not each individual instance. Still, it blows away the old record for the fastest supercomputer, Frontier, which was just a little over one exaFlop.\nAccording to Google, A3 is the first production-level deployment of its GPU-to-GPU data interface, which Google calls the infrastructure processing unit (IPU). It allows for sharing data at 200 Gbps directly between GPUs without having to go through the CPU. This result is a ten-fold increase in available network bandwidth for A3 virtual machines compared to prior-generation A2 VMs.\nA3 workloads will be run on Google\u2019s specialized Jupiter data center networking fabric, which the company says \u201cscales to tens of thousands of highly interconnected GPUs and allows for full-bandwidth reconfigurable optical links that can adjust the topology on demand.\u201d\nGoogle will be offering the A3 in two ways: customers can run it themselves or as a managed service where Google handles most of the work. If you opt to do it yourself, the A3 VMs run on Google Kubernetes Engine (GKE) and Google Compute Engine (GCE). If you go with a managed service, the VMs run on Vertex, the company\u2019s managed machine learning platform.\nThe A3 virtual machines are available for preview, which requires filling out an application to join the Early Access Program. Google makes no promises you will get a spot in the program.