Microsoft Azure has announced its new virtual machines powered by NVIDIA’s top-of-the-line H100 GPUs to accelerate generative AI like ChatGPT.
NVIDIA H100 GPUs are powering the latest Microsoft Azure virtual machines powering generative AI content, including ChatGPT
Press release: Delivering on the promise of advanced AI for our customers requires supercomputing infrastructure, services and expertise to address the exponentially increasing size and complexity of the latest models.
At Microsoft, we’re meeting this challenge by applying a decade of supercomputing experience and supporting the largest AI training workloads to create an AI infrastructure capable of massive performance at scale. The Microsoft Azure cloud, and specifically our Graphics Processing Unit (GPU) Accelerated Virtual Machines (VMs), provide the foundation for many generative AI advances from both Microsoft and our customers.
“Co-designing supercomputers with Azure has been crucial in scaling our demanding AI training needs, enabling our research and alignment work on systems like ChatGPT..”—Greg Brockman, president and co-founder of OpenAI.
Azure’s most powerful and massively scalable AI virtual machine series
Today, Microsoft is introducing the ND H100 v5 VM that enables on-demand sizes ranging from eight to thousands of NVIDIA H100 GPUs interconnected over the NVIDIA Quantum-2 InfiniBand network. Customers will see significantly faster performance for AI models than our next-generation ND A100 v4 virtual machines with innovative technologies such as:
- 8 NVIDIA H100 Tensor Core GPUs interconnected via next-generation NVSwitch and NVLink 4.0
- NVIDIA Quantum-2 CX7 400 Gb/s InfiniBand per GPU with 3.2 Tb/s per VM on a non-blocking fat-tree network
- NVSwitch and NVLink 4.0 with 3.6 TB/s bisectional bandwidth between 8 local GPUs within each VM
- 4th generation Intel Xeon Scalable processors
- PCIE Gen5 host interconnect to GPU with 64 GB/s bandwidth per GPU
- 16 channels of 4800 MHz DDR5 DIMMs
Delivering Exascale AI Supercomputers to the Cloud
Generative AI applications are rapidly evolving and adding unique value in almost every industry. From the reinvention of search with a new AI-powered Microsoft Bing and Edge to AI-powered support in Microsoft Dynamics 365, AI is quickly becoming a ubiquitous component of software and how we interact with it, and our infrastructure. AI will be there to pave the way. .
With our experience delivering multiple ExaOP supercomputers to Azure customers around the world, customers can be confident that they can achieve true supercomputer performance with our infrastructure. For Microsoft and organizations like Inflection, NVIDIA, and OpenAI that have committed to large-scale deployments, this offering will enable a new class of large-scale AI models.
“Our focus on conversational AI requires us to develop and train some of the most complex big language models. Azure’s AI infrastructure gives us the performance needed to efficiently process these models reliably at scale. We are excited about the new virtual machines in Azure and the increased performance they will bring to our AI development efforts..”—Mustafa Suleyman, CEO of Inflection.
AI at scale is built into the DNA of Azure. Our initial investments in research into large language models, such as Turing, and engineering milestones, such as building the first AI supercomputer in the cloud, prepared us for the moment when generative artificial intelligence became possible.
Azure services like Azure Machine Learning make our AI supercomputer accessible to customers for model training, and the Azure OpenAI Service enables customers to harness the power of generative AI models at scale. Scale has always been our lodestar in optimizing Azure for AI. We are now bringing supercomputing capabilities to startups and companies of all sizes, without requiring the capital for massive investments in physical hardware or software.
“NVIDIA and Microsoft Azure have collaborated across multiple product generations to bring leading AI innovations to businesses around the world. NDv5 H100 virtual machines will help power a new era of generative AI applications and services.”—Ian Buck, vice president of hyperscale and high performance computing at NVIDIA.
Today we are announcing that ND H100 v5 is available for preview and will become a standard offering in the Azure portfolio, enabling anyone to unlock the potential of AI at scale in the cloud. Sign in to request access to the new virtual machines.
