NVIDIA H100 NVL Tensor Core GPU 94GB HBM3
Graphic Cards And Accessories\ Datacenter GPUs
The NVIDIA H100 NVL card is a dual-slot 10.5 inch PCI Express Gen5 card based on the NVIDIA Hopper™ architecture. It uses a passive heat sink for cooling, which requires system airflow to operate the card properly within its thermal limits. The NVIDIA H100 NVL operates unconstrained up to its maximum thermal design power (TDP) level of 400 W to accelerate applications that require the fastest computational speed and highest data throughput. The NVIDIA H100 NVL debuts the world’s highest PCIe card memory bandwidth of nearly 4,000 gigabytes per second (GBps). This speeds time to solution for the largest models and most massive data sets.
Specifications
| GPU Features | NVIDIA H100 NVL |
| GPU Memory | 94 GB HBM3 |
| Memory bandwidth | 3,938 GB/s |
| FP64 Tensor Core | 67 TFLOPS |
| TF32 Tensor Core | 989 TFLOPS |
| FP16 Tensor Core | 1,979 TFLOPS |
| FP8 Tensor Core | 3,958 TFLOPS |
| INT8 Tensor Core | 3,958 TOPS |
| Max thermal design power (TDP) | 400W |
| Multi-Instance GPUs | Up to 7 MIGS @ 12GB each |
| NVLink | 2 -way 2-slot or 3-slot b |
| Form factor | PCIe dual-slot air-cooled |
| Server options | Partner and NVIDIA Certified Systems with 1–8 GPUs |
Explore the technology breakthroughs of NVIDIA Hopper
NVIDIA H100 Tensor Core GPU
Built with 80 billion transistors using a cutting-edge TSMC 4N process custom tailored for NVIDIA’s accelerated compute needs, H100 is the world’s most advanced chip ever built. It features major advances to accelerate AI, HPC, memory bandwidth, interconnect, and communication at data center scale. Transformer Engine
The Transformer Engine uses software and Hopper Tensor Core technology designed to accelerate training for models built from the world’s most important AI model building block, the transformer. Hopper Tensor Cores can apply mixed FP8 and FP16 precisions to dramatically accelerate AI calculations for transformers. NVLink Switch System
The NVLink Switch System enables the scaling of multi-GPU input/output (IO) across multiple servers at 900 gigabytes per second (GB/s) bidirectional per GPU, over 7X the bandwidth of PCIe Gen5. The system supports clusters of up to 256 H100s and delivers 9X higher bandwidth than InfiniBand HDR on the NVIDIA Ampere architecture. NVIDIA Confidential Computing
NVIDIA Confidential Computing is a built-in security feature of Hopper that makes NVIDIA H100 the world’s first accelerator with confidential computing capabilities. Users can protect the confidentiality and integrity of their data and applications in use while accessing the unsurpassed acceleration of H100 GPUs. Second-Generation MIG
The Hopper architecture’s second-generation Multi-Instance GPU (MIG) supports multi-tenant, multi-user configurations in virtualized environments, securely partitioning the GPU into isolated, right-size instances to maximize quality of service (QoS) for 7X more secured tenants.
DPX Instructions
Hopper’s DPX instructions accelerate dynamic programming algorithms by 40X compared to CPUs and 7X compared to NVIDIA Ampere architecture GPUs. This leads to dramatically faster times in disease diagnosis, real-time routing optimizations, and graph analytics.