CPU vs GAUDI2

The GAUDI2 is generally better for large language model training due to its higher throughput, which allows fitting larger models or larger batch sizes in a single pass. For smaller models or fine-tuning tasks where cost matters more, both GPUs can be effective.

Pricing for the CPU and GAUDI2 varies by cloud provider, region, and contract type. Shadeform aggregates pricing from 30+ GPU cloud providers so you can compare and find the best rate. Use the instance table above to see current on-demand prices.

The GAUDI2 has more VRAM at 96 GB, compared to 0 GB on the CPU. Higher VRAM allows you to run larger models without quantization, use longer context windows, and process larger batch sizes — all of which improve throughput and reduce latency for memory-bound workloads.

The CPU is currently available across 2 cloud providers on Shadeform's network, compared to 1 for the GAUDI2. Shadeform lets you deploy either GPU across all available providers from a single platform, so you can always find available capacity without manually checking each cloud.

Mixing different GPU types in a single training cluster is generally not recommended, as it creates performance bottlenecks where faster GPUs wait for slower ones. For best results, use a homogeneous cluster of either CPU or GAUDI2. Shadeform supports on-demand clusters of up to 64 GPUs of the same type with no commitment required.