revealed NVIDIA announced the H200, a GPU designed to train AI models powered by the Generative AI surge.
The new GPU is an upgrade over the previous H100 GPU, the chip that OpenAI used to train the large GPT-4 language model.
Big companies, startups and government agencies are competing for limited supplies of chips.
The H100 chip costs between $25,000 and $40,000, and you need to connect thousands of them together to train large language models.
The growing interest in NVIDIA’s AI-supporting graphics processing units has led to an increase in the company’s shares, which have risen so far in 2023 by more than 230 percent.
Topics related to what you are reading now:
The company expects to achieve revenues for the fiscal third quarter of about $16 billion, an increase of 170 percent over last year.
The main improvement in the H200 is that the chip has 141 GB of next-generation HBM3 memory that helps it perform inference, or use a large language model after training it to generate texts, images or predictions.
GPU memory bandwidth rises to 4.8 TB/s from 3.35 TB/s across the H100, improving its ability to handle the power-intensive work of generative AI.
The company indicated that the H200 is approximately twice as fast as the H100 chip in generating output. This chip is expected to reach customers in the second quarter of 2024, which competes with AMD’s MI300X graphics processing unit.
AMD’s MI300X chip has additional memory compared to its predecessors, which helps make it suitable for large models for running inference.
The H200 is compatible with the H100, which means that AI companies that use the H100 to train large language models will not need to change their server systems or software to use the H200.
Nvidia points out that the H200 is available in quad-GPU or eight-GPU server setups across the company’s full HGX systems, as well as in a chip called the GH200, which connects the H200 GPU to an Arm-based processor.
