Colfax Experience Center is offering a test drive program that provides you remote access to a Colfax server with four (4) NVIDIA® H100 NVL Tensor Core GPU cards for evaluation. It is free of charge.
About the NVIDIA H100 NVL GPU
The NVIDIA H100 NVL Tensor Core GPU is the most optimized platform for LLM inference with its high compute density, high memory bandwidth, high energy efficiency, and unique NVLink architecture. It also delivers unprecedented acceleration to power the world’s highest-performing elastic data centers for AI, data analytics, and high-performance computing (HPC) applications. NVIDIA H100 NVL Tensor Core technology supports a broad range of math precisions, providing a single accelerator for every compute workload. The NVIDIA H100 NVL supports double precision (FP64), single precision (FP32), half precision (FP16), 8-bit floating point (FP8), and integer (INT8) compute tasks.
The NVIDIA H100 NVL card is a dual-slot 10.5 inch PCI Express Gen5 card based on the NVIDIA Hopper™ architecture. It uses a passive heat sink for cooling, which requires system airflow to operate the card properly within its thermal limits. The NVIDIA H100 NVL operates unconstrained up to its maximum thermal design power (TDP) level of 400 W to accelerate applications that require the fastest computational speed and highest data throughput. The NVIDIA H100 NVL debuts the world’s highest PCIe card memory bandwidth of nearly 4,000 gigabytes per second (GBps). This speeds time to solution for the largest models and most massive data sets.
The NVIDIA H100 NVL card features Multi-Instance GPU (MIG) capability. This can be used to partition the GPU into as many as seven hardware isolated GPU instances, providing a unified platform that enables elastic data centers to adjust dynamically to shifting workload demands. As well as it can allocate the right size of resources from the smallest to biggest multi-GPU jobs. NVIDIA H100 NVL versatility means that IT managers can maximize the utility of every graphics processing unit (GPU) in their data center.
NVIDIA H100 NVL cards use three NVIDIA® NVLink® bridges. They are the same as the one used with NVIDIA H100 PCIe cards. This allows two NVIDIA H100 PCIe cards to be connected to deliver 600 GB/s bidirectional bandwidth or 10x the bandwidth of PCIe Gen4, to maximize application performance for large workloads.
Supercharge Large Language Model Inference
For LLMs up to 175 billion parameters, the PCIe-based H100 NVL with NVLink bridge utilizes Transformer Engine, NVLink, and 188GB HBM3 memory to provide optimum performance and easy scaling across any data center, bringing LLMs to mainstream. Servers equipped with H100 NVL GPUs increase GPT-175B model performance up to 12X over NVIDIA DGX™ A100 systems while maintaining low latency in power-constrained data center environments.
Technical Specifications
Click to View Technical Specifications
| Model | H100 NVL¹ |
| FP64 | 68 teraFLOPs |
| FP64 Tensor Core | 134 teraFLOPs |
| FP32 | 134 TeraFLOPs |
| TF32 Tensor Core | 1,979 teraFLOPs² |
| BFLOAT16 Tensor Core | 3,958 teraFLOPs² |
| FP16 Tensor Core | 3,958 teraFLOPs² |
| FP8 Tensor Core | 7,916 teraFLOPs² |
| INT8 Tensor Core | 7,916 TOPS² |
| GPU Memory | 188GB |
| GPU memory bandwidth | 7.8TB/s³ |
| Decoders | 14 NVDEC 14 JPEG |
| Max thermal design power (TDP) | 2x 350-400W (configurable) |
| Multi-Instance GPUs | Up to 14 MIGS @ 12GB each |
| Form factor | 2x PCIe dual-slot air-cooled |
| Interconnect | NVLink: 600GB/s PCIe Gen5: 128GB/s |
| Server options | Partner and NVIDIA-Certified Systems with 2-4 pairs |
| NVIDIA AI Enterprise | Included |
¹ Preliminary specifications. May be subject to change. Specifications shown for 2x H100 NVL PCIe cards paired with NVLink Bridge.
² With sparsity.
³ Aggregate HBM bandwidth.
How the Test Drive Works
Step 1
Complete the registration form
Step 2
If you qualify, you will receive an email message with connection instructions.
Step 3
Run your GPU-accelerated HPC or AI application
The Colfax GPU Test Drive program is intended for evaluation purposes only. Commercial use of computing time on any Colfax Test Drive system is strictly prohibited. We reserve the right to remove any user’s remote access at all times.