NVIDIA H100 GPU Price Guide

The NVIDIA H100 GPU, announced on March 22nd, 2022, is the first processor to make use of NVIDIA's Hopper architecture, featuring significant performance increases over its predecessor, the NVIDIA A100.

NVIDIA H100 - Hopper Architecture

NVIDIA’s Hopper architecture, the successor to Ampere, marks a significant improvement in computational performance for advanced AI workloads. Hopper introduces the Transformer Engine for accelerated LLM performance, along with fourth-generation tensor cores and higher memory bandwidth.

Specs & Performance - NVIDIA H100 vs. A100

The H100's most significant improvement over the A100 is the introduction of 4th generation tensor cores with the Transformer Engine, providing up to 6x faster training and 9x faster inference for LLMs.

The NVIDIA H100 also provides:

• 80GB of VRAM, supporting larger model sizes over the A100's 40GB.
• 80 billion transistors, a 60% increase over the A100’s 54 billion
• Up to 3.35 TB/s memory bandwidth, a 67% increase over the A100's 2.00 TB/s.
• FP8 data format support, improving efficiency and throughput for AI workloads.
• • The A100 only supports FP16.

Which Use Cases is the H100 Best Suited For?

The increased inference performance and addition of FP8 support makes the H100 particularly well suited for inference applications that require fast or real-time throughput.

Examples include:

• Chat and Voice Assistants
• Drug Discovery
• Computer Vision

H100 GPU Pricing - Direct Purchase

If you want to buy one of these GPUs for yourself, it'll cost you ~$30,000 per chip.

Typically, these GPUs are bought in factors of 8 and assembled using reference architectures like the NVIDIA DGX H100 SuperPOD, which consists of several interconnected 8 x H100 nodes.

These architectures can range between ~300,000 to several million dollars.

H100 GPU Pricing - Renting On-Demand in the Cloud

A more cost efficient way to get access to these GPUs is through cloud rental or GPUaaS providers.

These providers charge developers by the hour for remote on-demand access to a number of different GPU types.

Each provider prices their H100s slightly differently. Here's a look at H100 rental pricing across providers.

Find the Best H100 Prices on Shadeform

With Shadeform, you can compare and deploy H100 GPUs from over 20 different providers in one platform. You'll have access to the best prices in over 150 regions.

Think the H100 is right for your workloads? Start deploying in minutes with Shadeform.