June 3, 2026
The H100 defined a generation of AI infrastructure. For two years, it was the default answer for serious LLM training workloads, reliable, well-supported, and benchmarked to death. But the hardware landscape has moved on, and the NVIDIA B300 (Blackwell Ultra) represents a meaningful architectural shift, not just an incremental spec bump.
Understanding what changed, and what that means in practice for teams training large models, matters more than the headline numbers.
The H100 brought 80 GB of HBM3 memory and introduced FP8 precision, which doubled compute throughput compared to FP16. That was already a significant step forward. The B300 goes further in every dimension that training workloads care about.
Memory capacity jumps to 288 GB of HBM3e per GPU, 3.6x more than the H100. Memory bandwidth reaches 8 TB/s per chip. In an 8-GPU system, that's 2.3 TB of total GPU memory and 64 TB/s of aggregate bandwidth available for gradient synchronization, activation storage, and optimizer states.
The compute picture is equally different. The B300 delivers 15 petaFLOPS of dense FP4 compute per GPU. The H100 topped out at roughly 4 petaFLOPS in FP8. The gap is not marginal, it's architectural. Blackwell Ultra's fifth-generation Tensor Cores support FP4 natively, with a Transformer Engine that automatically manages precision across layers to maximize throughput without sacrificing accuracy.
The interconnect has also been upgraded. The dual NVLink Switch System delivers 14.4 TB/s of aggregate GPU-to-GPU bandwidth in an 8-way configuration, which matters enormously for tensor parallelism in models with hundreds of billions of parameters.
What This Means for Long Training Runs
LLM pre-training is not a single computation, it's a sustained workload measured in days or weeks, where bottlenecks compound and small inefficiencies accumulate into significant delays.
Three changes from the H100 to B300 have direct consequences for training at scale.
Memory per GPU reduces model parallelism overhead. With 80 GB on the H100, anything above roughly 30B parameters in FP16 required tensor parallelism across multiple GPUs, adding communication overhead on every forward and backward pass. The B300's 288 GB changes that equation. A 70B parameter model in BF16 fits comfortably on a single GPU. A 400B+ model fits across a single 8-GPU node without model parallelism tricks. Less parallelism means less inter-GPU communication, which means cleaner scaling and more predictable training throughput.
FP4 training reduces iteration cost. Mixed-precision training with FP4 reduces the memory footprint of activations and optimizer states by roughly 1.8x compared to FP8, while maintaining comparable accuracy for most transformer architectures. For long training runs where checkpoint storage and memory pressure are real constraints, this is operationally significant, not just a benchmark metric.
Bandwidth eliminates the memory wall for large batches. At 8 TB/s per GPU, the B300 removes the bandwidth bottleneck that limited effective batch sizes on H100 for models with large embedding layers or long context windows. Training on long-context data, 32K, 64K, or 128K token sequences, becomes viable without the memory management complexity that constrained those workloads on Hopper.
The B300's architectural improvements are measurable in isolation. On shared cloud infrastructure, many of them are partially offset by the environment itself.
Multi-tenant GPU clusters introduce variable latency on NVLink and NVSwitch paths when other workloads compete for interconnect bandwidth. Noisy neighbor effects on storage and network I/O create inconsistent throughput during data loading, which stalls the GPU pipeline and fragments training runs. Hyperscaler egress fees apply to every checkpoint written off-node, and long training runs write a lot of checkpoints.
On dedicated bare metal, none of those variables apply.
The GPU resources are yours entirely. NVLink bandwidth between cards is not shared or throttled. Storage I/O is consistent. Network throughput is predictable. Checkpoints move at infrastructure speed, not at metered cloud rates.
For a GPU as capable as the B300, this matters more than it did with previous generations. The higher the peak throughput of the hardware, the more damaging any source of inconsistency becomes. A training job that achieves 70% GPU utilization on shared infrastructure might reach 90%+ on bare metal, and across a weeks-long training run, that gap is the difference between hitting your compute budget and exceeding it.
1Legion's B300 bare metal servers are available now, dedicated hardware, no shared tenancy, no egress fees, no hyperscaler overhead. If you're planning a training run on Blackwell Ultra or want to evaluate performance before committing to a workload, talk to an engineer today. Get started here.
