Nvidia unveils custom NVHBM memory architecture to boost AI performance

Nvidia has introduced NVHBM, a custom high-bandwidth memory architecture designed to address performance bottlenecks in frontier AI models. By moving the memory controller from the accelerator die into the base die of the HBM stack, Nvidia claims a potential 30% increase in bandwidth, 15% lower power consumption, and 25% more usable compute area compared to the upcoming JEDEC HBM4E standard. The technology, which replaces traditional wide parallel buses with serialized die-to-die links, is currently gated behind Nvidia’s NVLink Fusion program. Amazon’s Annapurna Labs is the first partner named to utilize this ecosystem, though specific integration details remain limited. While the approach mirrors recent industry efforts by Marvell and others, Nvidia’s implementation is tied to its proprietary rack-scale platform. As the industry anticipates the mass deployment of HBM4E in 2027, NVHBM represents a strategic move to optimize memory management for future high-performance computing and AI hardware.