{"id":308,"date":"2024-05-02T14:57:34","date_gmt":"2024-05-02T21:57:34","guid":{"rendered":"https:\/\/experience.colfax-intl.com\/project\/nvidia-h100-nvl\/?page_id=308"},"modified":"2024-11-14T13:16:21","modified_gmt":"2024-11-14T21:16:21","slug":"home","status":"publish","type":"page","link":"https:\/\/experience.colfax-intl.com\/project\/cfx-nvme\/","title":{"rendered":"CFX Nvme Test Drive"},"content":{"rendered":"\n

Colfax Experience Center is offering a test drive program that provides you remote access to a Colfax server with four (4) NVIDIA\u00ae H100 NVL Tensor Core GPU cards for evaluation. It is free of charge.<\/p>\n\n\n\n

\n
Request Access to a Test Drive<\/a><\/div>\n<\/div>\n\n\n\n

About the NVIDIA H100 NVL GPU <\/h3>\n\n\n\n

The NVIDIA H100 NVL Tensor Core GPU is the most optimized platform for LLM inference with its high compute density, high memory bandwidth, high energy efficiency, and unique NVLink architecture. It also delivers unprecedented acceleration to power the world\u2019s highest-performing elastic data centers for AI, data analytics, and high-performance computing (HPC) applications. NVIDIA H100 NVL Tensor Core technology supports a broad range of math precisions, providing a single accelerator for every compute workload. The NVIDIA H100 NVL supports double precision (FP64), single precision (FP32), half precision (FP16), 8-bit floating point (FP8), and integer (INT8) compute tasks.<\/p>\n\n\n\n

The NVIDIA H100 NVL card is a dual-slot 10.5 inch PCI Express Gen5 card based on the NVIDIA Hopper\u2122 architecture. It uses a passive heat sink for cooling, which requires system airflow to operate the card properly within its thermal limits. The NVIDIA H100 NVL operates unconstrained up to its maximum thermal design power (TDP) level of 400 W to accelerate applications that require the fastest computational speed and highest data throughput. The NVIDIA H100 NVL debuts the world\u2019s highest PCIe card memory bandwidth of nearly 4,000 gigabytes per second (GBps). This speeds time to solution for the largest models and most massive data sets.<\/p>\n\n\n\n

The NVIDIA H100 NVL card features Multi-Instance GPU (MIG) capability. This can be used to partition the GPU into as many as seven hardware isolated GPU instances, providing a unified platform that enables elastic data centers to adjust dynamically to shifting workload demands. As well as it can allocate the right size of resources from the smallest to biggest multi-GPU jobs. NVIDIA H100 NVL versatility means that IT managers can maximize the utility of every graphics processing unit (GPU) in their data center.<\/p>\n\n\n\n

NVIDIA H100 NVL cards use three NVIDIA\u00ae NVLink\u00ae bridges. They are the same as the one used with NVIDIA H100 PCIe cards. This allows two NVIDIA H100 PCIe cards to be connected to deliver 600 GB\/s bidirectional bandwidth or 10x the bandwidth of PCIe Gen4, to maximize application performance for large workloads.<\/p>\n\n\n\n

Supercharge Large Language Model Inference<\/h3>\n\n\n\n

For LLMs up to 175 billion parameters, the PCIe-based H100 NVL with NVLink bridge utilizes Transformer Engine, NVLink, and 188GB HBM3 memory to provide optimum performance and easy scaling across any data center, bringing LLMs to mainstream. Servers equipped with H100 NVL GPUs increase GPT-175B model performance up to 12X over NVIDIA DGX\u2122 A100 systems while maintaining low latency in power-constrained data center environments.<\/p>\n\n\n\n

Technical Specifications<\/h3>\n\n\n\n
Click to View Technical Specifications<\/summary>\n
Model<\/td>H100 NVL\u00b9<\/td><\/tr>
FP64<\/td>68 teraFLOPs<\/td><\/tr>
FP64 Tensor Core<\/td>134 teraFLOPs<\/td><\/tr>
FP32<\/td>134 TeraFLOPs<\/td><\/tr>
TF32 Tensor Core<\/td>1,979 teraFLOPs\u00b2<\/td><\/tr>
BFLOAT16 Tensor Core<\/td>3,958 teraFLOPs\u00b2<\/td><\/tr>
FP16 Tensor Core<\/td>3,958 teraFLOPs\u00b2<\/td><\/tr>
FP8 Tensor Core<\/td>7,916 teraFLOPs\u00b2<\/td><\/tr>
INT8 Tensor Core<\/td>7,916 TOPS\u00b2<\/td><\/tr>
GPU Memory<\/td>188GB<\/td><\/tr>
GPU memory bandwidth<\/td>7.8TB\/s\u00b3<\/td><\/tr>
Decoders<\/td>14 NVDEC
14 JPEG<\/td><\/tr>
Max thermal design power (TDP)<\/td>2x 350-400W (configurable)<\/td><\/tr>
Multi-Instance GPUs<\/td>Up to 14 MIGS @ 12GB each<\/td><\/tr>
Form factor<\/td>2x PCIe dual-slot air-cooled<\/td><\/tr>
Interconnect<\/td>NVLink: 600GB\/s
PCIe Gen5: 128GB\/s<\/td><\/tr>
Server options<\/td>Partner and NVIDIA-Certified Systems with 2-4 pairs<\/td><\/tr>
NVIDIA AI Enterprise<\/td>Included<\/td><\/tr><\/tbody><\/table>

\u00b9 Preliminary specifications. May be subject to change. Specifications shown for 2x H100 NVL PCIe cards paired with NVLink Bridge.
\u00b2 With sparsity.
\u00b3 Aggregate HBM bandwidth.<\/figcaption><\/figure>\n<\/details>\n\n\n\n

How the Test Drive Works<\/h3>\n\n\n\n
\n
\n
\"\"<\/figure>\n\n\n\n

Step 1
<\/strong>Complete the registration form<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

Step 2<\/strong>
If you qualify, you will receive an email message with connection instructions.<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

Step 3<\/strong>
Run your GPU-accelerated HPC or AI application<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n