NVIDIA CEO Jen-Hsun Huang announced the launch of a new generation of GPUs, Blackwell, at GTC, with the first Blackwell chip, called the GB200, due later this year.
Blackwell has 208 billion transistors and utilizes TSMC's 4nm process. The previous generation GPU "Hopper" H100 utilized a 4nm process and integrated 80 billion transistors.
Jen-Hsun Huang said, "Hopper is great, but we need a bigger GPU. blackwell is not a chip, it's the name of a platform."
NVIDIA says Blackwell-based processors, such as the GB200, provide a huge performance upgrade for AI companies, with AI performance of 20 trillion floating-point operations per second, compared to the H100's 40 trillion floating-point operations per second. The system can deploy a model with 27 trillion parameters. GPT-4 is said to have used about 1.76 trillion parameters to train the system.