September 2, 2026
There are now two credible current-generation nodes for serious AI workloads, not one. Most teams still default to whichever name is trending in their feed rather than running the actual math against their own workload, and on dedicated bare metal, where the node you pick is the node you're running for the length of the contract, that's an expensive way to make a GPU infrastructure decision.
The short version: H200 is a memory upgrade on proven Hopper silicon. B300 is a new architecture built around FP4 precision and a much larger memory pool. Which one is right depends less on which is newer and more on what your workload is actually bottlenecked by.
H200 keeps the same GH100 compute die as the H100, according to NVIDIA's own datasheet, so peak tensor throughput is unchanged. What it swaps in is 141GB of HBM3e running at 4.8TB/s, up from the H100's 80GB at 3.35TB/s. Every H200 gain over the H100 traces back to memory, not compute, which is why NVIDIA's own H200 launch benchmarks put the inference gain at up to 1.9x on large language models, a number that narrows considerably for workloads that aren't already memory-bandwidth-bound.
B300, officially Blackwell Ultra, is a different generation entirely. NVIDIA's published Blackwell Ultra specifications, compiled independently by pi3g's 2026 GB300 breakdown, put it at 288GB of HBM3e per GPU and roughly 8TB/s of bandwidth, twice the memory of the H200, with native FP4 tensor cores that neither the H100 nor the H200 support at all. On dense FP8, the same specifications put the B300 at roughly 10 PFLOPS versus the H200's approximately 2 PFLOPS, close to a 5x gap. On FP4, it isn't a multiple at all, it's a capability the H200 simply doesn't have.
If your model fits comfortably inside 141GB and the workload is memory-bandwidth-bound rather than compute-bound, long-context inference, large-batch serving, training in the sub-70B range, H200 remains the more pragmatic choice for dedicated bare metal GPU infrastructure on three separate fronts.
It's air-cooled at 700W SXM, which means it drops into existing Hopper-based infrastructure without a cooling redesign. Pricing has also matured into a genuinely competitive market as Hopper-generation supply has scaled, with reserved and committed-use rates dropping well below on-demand list price for longer commitments. And it's a known quantity. The software stack, the failure modes, and the performance characteristics are well understood after nearly two years in production, which matters more than it sounds like it should when something breaks at 2am.
The B300's case is specific, not general, and it's the same case whether the node sits in a hyperscaler region or on dedicated bare metal. For models in the 70B-plus range, its 288GB of memory holds a full model in FP16 with headroom to spare, something the H200 can't do without careful memory management or a quantization tradeoff it wasn't necessarily designed around. For inference workloads built on FP4 quantization, reasoning models and mixture-of-experts architectures especially, the B300's native FP4 tensor cores unlock throughput the H200 has no path to at any price. And for trillion-parameter or long-context agentic workloads where the entire model state needs to stay resident in memory, the B300's memory ceiling settles the question before compute even enters the conversation.
Memory and throughput numbers get the attention. The deployment requirements are where the decision gets real. NVIDIA's own DGX B300 documentation splits the GPU into two power bins: a 1,100W configuration that ships air-cooled, and a 1,400W configuration, along with the full GB300 NVL72 rack-scale system, that requires direct liquid cooling. The air-cooled option exists, but it leaves meaningful performance on the table, since the higher power bin is where the throughput numbers above actually apply. H200 tops out at 700W and deploys air-cooled in standard SXM configurations at full spec, no bin trade-off involved. Either way, a B300 deployment at the power bin that delivers its real advantage carries infrastructure cost and complexity, cooling loops, power delivery, facility requirements, that an H200 deployment on existing Hopper infrastructure simply doesn't.
Supply is the other constraint worth weighing before the spec sheet. Public procurement commentary through mid-2026 has pointed to multi-week lead times on DGX and HGX B300 orders, longer for full GB300 NVL72 rack allocations, with the 288GB memory tier specifically described as running in heavier demand than supply. Worth confirming current lead times directly with a provider rather than planning against a number that's likely already stale. If your timeline is measured in weeks rather than quarters, that constraint alone can settle the decision regardless of what the throughput numbers say.
Start with what's bottlenecking the workload today, not what the roadmap might need eighteen months from now. If the model fits in 141GB and the workload is bandwidth-bound rather than FP4-bound, H200 delivers comparable real-world throughput at lower deployment complexity and a more mature price curve. If the model exceeds what H200 can hold comfortably, or the workload is built around FP4 quantization for reasoning or MoE architectures, the B300's memory ceiling and throughput advantage justify the added power, cooling, and lead-time cost.
The wrong move, in either direction, is picking based on release date instead of running the workload math first. On dedicated bare metal, that decision holds for the length of the contract, which is exactly why it's worth getting right before signing rather than after.
Talk to an engineer about which node actually fits your workload, not just which one launched most recently.
‍
