• 3 Ways NVFP4 Accelerates AI Training and Inference L T F R E The latest AI models continue to grow in size and complexity, demanding increasing amounts of compute performance for training and inference-far beyond what Moore’s Law can keep up with. • That’s why NVIDIAengages in extreme codesign. • Designing across multiple chips and a mountain of software cohesively enables large generational leaps in AI factory performance and efficiency. • Lower-precision AI formats are key to improving compute performance and energy efficiency. • Bringing the benefits of ultra-low-precision numerics to AI training and inference while maintaining high accuracy requires extensive engineering across every layer of the technology stack. • It spans the creation of the formats, implementation in silicon, enablement across many libraries, and working closely with the ecosystem to deploy new training recipes and inference optimization techniques.NVFP4, developed and implemented for NVIDIA GPUs starting with NVIDIA Blackwell, delivers the performance and energy-efficiency benefits of 4-bit floating-point precision while maintaining accuracy on par with higher-precision formats.
Article Summaries:
- NVIDIA’s new NVFP4 4‑bit floating‑point format boosts AI training and inference by delivering up to 15 petaFLOPS on Blackwell Ultra GPUs-roughly three times the throughput of FP8-while maintaining accuracy comparable to higher‑precision formats. The format improves token throughput on large models such as DeepSeek‑R1 and enables faster, lower‑cost training, as shown by a 64.6‑minute Llama 3.1 405B pre‑training run on 512 Blackwell Ultra GPUs. NVFP4 has passed MLPerf training and inference accuracy thresholds for multiple large‑language‑model tests. NVIDIA’s upcoming Rubin platform is expected to raise NVFP4 compute to 35 petaFLOPS for training and 50 petaFLOPS for inference, a 3.5‑ to 5‑fold increase.
Sources: