Tag Archives: Nvidia

Cooling AI Data Centers

Posted on June 27, 2025 by | 1 comment

How important are AI data centers? In just months, Elon Musk’s xAI team converted a factory outside Memphis into a cutting-edge, 100,000-GPU center for training the Colossus supercomputer—home to the Grok chatbot.

Initially powered by temporary gas turbines (later replaced by grid power), Colossus installed its first 100,000 chips in only 19 days, drawing praise from NVIDIA CEO Jensen Huang. Today, it operates 200,000 GPUs, with plans to reach 1 million GPUs by the end of 2025. [1]

Figure 1 – Elon Musk’s 1 Million Sq Ft xAI Colossus Supercomputer Facility near Memphis, TN. [1]

There are about 12,000 data centers throughout the world, nearly half of them in the United States. Now, more and more of these are being built or retrofitted for AI-specific workloads. Leaders include Musk’s xAI, Microsoft, Meta, Google, Amazon, OpenAI, and others.

High power is essential for such operations, and like computational electronics of all sizes heat issues need to be resolved.

GenAI

A key driver of data center growth is Generative AI (GenAI)—AI that creates text, images, audio, video, and code using deep learning. Chatbots and large language model ChatGPT are examples of GenAI, along with text-to-image models that generate images from written descriptions.

Managing all this is possible from new generations of processors, mainly GPUs. They all draw on higher levels of power and generate higher amounts of heat.

Figure 2 – Advanced AI Processor, the NVIDIA GH200 Grace Hopper Superchip with Integrated CPU to Increase Speed and Performance. [2,3]

AI data centers prioritize HPC hardware: GPUs, FPGAs, ASICs, and ultra-fast networking. Compared to CPUs (150–200 W), today’s AI GPUs often run >1,000 W.  . To handle massive datasets and complex computations in real-time they need significant power and cooling infrastructure.

Data Center Cooling Basics

Traditional HVAC was sufficient for older CPU-driven data centers. Today’s AI GPUs demand far more cooling, both at the chip level and facility-wide. This has propelled a need for more efficient thermal management systems at both the micro (server board and chip) and macro (server rack and facility) levels. [4]

Figure 3 – The Colossus AI Supercomputer Now Runs 200,000 GPUs. It Operates at 150MW Power, Equivalent to 80,000 Households. [5]

At Colossus, Supermicro 4U servers house NVIDIA Hopper GPUs cooled by:

  • Cold plates
  • Coolant distribution manifolds (1U between each server)
  • Coolant distribution units (CDUs) with redundant pumps at each rack base [6]

Each 4U server is equipped with eight NVIDIA H100 Tensor Core GPUs. Each rack contains eight 4U servers, totaling 64 GPUs per rack.

Between every server is a 1U manifold for liquid cooling. They connect with CDUs, heat-exchanging Coolant Distribution Units at the bottom of each rack that include a redundant pumping system. The choice of coolant is determined by a range of hardware and environmental factors.

Figure 4 – Each Colossus Rack Contains Eight 4U Servers, Totaling 64 GPUs Per Rack. Between Each Server is a 1U Manifold for Liquid Cooling. [7]
Figure 5 – The Base of Each Rack Has a 4U CDU Pumping System with Redundant Liquid Cooling. [7]

Role of Cooling Fans

Fans remain essential for DIMMs, power supplies, controllers, and NICs.

Figure 6 – Rear Door Liquid-Cooled Heat Exchangers. [7]

At Colossus, fans in the servers pull cooler air from the front of the rack, and exhaust the air at the rear of the server. From there, the air is pulled through rear door heat exchangers. The heat exchangers pass warm air through a liquid-cooled, finned heat exchanger/radiator, lowering its temperature before it exits the rack.

Direct-to-Chip Cooling

NVIDIA’s DGX H100 and H200 server systems feature eight GPUs and two CPUs that must run between 5°C and 30°C. An AI data center with a high rack density houses thousands of these systems performing HPC tasks at maximum load. Direct liquid cooling solutions are required.

Figure 7 – An NVIDIA DGX H100/H200 System Featuring Eight GPUs [8]
Figure 8 – The NVIDIA H100 SmartPlate Connects to a Liquid Cooling System to Bring Microconvective Chip-Level Cooling That Outperforms Air Cooling by 82%. [9]

Direct liquid cooling (cold plates contacting the GPU die) is the most effective method—outperforming air cooling by 82%. It is preferred for high-density deployments of the H100 or GH200.

Scalable Cooling Modules

Colossus represents the world’s largest liquid-cooled AI cluster, using NVIDIA + Supermicro technology. For smaller AI data centers, Cooling Distribution Modules (CDMs) provide a compact, self-contained solution.

Figure 9 – The iCDM-X Cooling Distribution Module from ATS Includes Pumps, Heat Exchanger and Liquid Coolant for Managing Heat from AI GPUs and Other Components. [10]

Most AI data centers are smaller, and power and cooling needs are lower, but essential. Many heat issues can be resolved using self-contained Cooling Distribution Modules.

The compact iCDM-X cooling distribution module provides up to 1.6MW of cooling for a wide range of AI GPUs and other chips. The module measures and logs all important liquid cooling parameters. It uses using just 3kW of power, and no external coolant is required.

These modules include:

•         Pumps

•         Heat exchangers

•         Cold plates

•         Digital monitoring (temp, pressure, flow)

Their sole external component is one or more cold plates removing heat from AI chips. ATS provides an industry-leading selection of custom and standard cold plates, including the high-performing ICEcrystal series.

Figure 10 – The ICEcrystal Cold Plates Series from ATS Provide 1.5 kW of Jet Impingement Liquid Cooling Directly onto AI Chip Hotspots.

Cooling Edge AI and Embedded Applications

AI isn’t just for big data centers—edge AI, robotics, and embedded systems (e.g., NVIDIA Jetson Orin, AMD Kria K26) use processors running under 100 W. These are effectively cooled with heat sinks and fan sinks from suppliers like Advanced Thermal Solutions. [11]

Figure 11 – High Performance Heat Sinks for NVIDIA and AMD AI Processors in Embedded and Edge Applications. [11]

NVIDIA also partners with Lenovo, whose 6th-gen Neptune cooling system enables full liquid cooling (fanless) across its ThinkSystem SC777 V4 servers—targeting enterprise deployments with NVIDIA Blackwell + GB200 GPUs. [12]

Figure 12 – Lenovo’s Neptune Direct Water Cooling Removes Heat from Power Supplies, for Completely Fanless Operation. [12]

Benefits gained from the Neptune system include:

  • Full system cooling (GPUs, CPUs, memory, I/O, storage, regulators)
  • Efficient for 10-trillion-parameter models
  • Improved performance, energy efficiency, and reliability

Conclusion

With surging demand, AI data centers are now a major construction focus. Historically, cooling problems are the #2 cause of data center downtime (behind power issues). With the high power needed for AI computing, these builds should carefully fit with their local communities in terms of electrical needs and sources, and water consumption. [13]

AI workloads will increase U.S. data center power demand by 165% by 2030 (Goldman Sachs), with nearly double 2022 levels (IBM/Newmark). Sustainable design and resource-conscious cooling are essential for the next wave of AI infrastructure. [14,15]

References

1. The Guardian, https://www.theguardian.com/technology/2025/apr/24/elon-musk-xai-memphis

2. Fibermall, https://www.fibermall.com/blog/gh200-nvidia.htm

3. NVIDA, https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip?ncid=no-ncid

4. ID Tech Ex, https://www.idtechex.com/en/research-report/thermal-management-for-data-centers-2025-2035-technologies-markets-and-opportunities/1036

5. Data Center Frontier, https://www.datacenterfrontier.com/machine-learning/article/55244139/the-colossus-ai-supercomputer-elon-musks-drive-toward-data-center-ai-technology-domination

6. Supermicro, https://learn-more.supermicro.com/data-center-stories/how-supermicro-built-the-xai-colossus-supercomputer

7. Serve The Home, https://www.servethehome.com/inside-100000-nvidia-gpu-xai-colossus-cluster-supermicro-helped-build-for-elon-musk/2/

8. Naddod, https://www.naddod.com/blog/introduction-to-nvidia-dgx-h100-h200-system

9. Flex, https://flex.com/resources/flex-and-jetcool-partner-to-develop-liquid-cooling-ready-servers-for-ai-and-high-density-workloads

10. Advanced Thermal Solutions, https://www.qats.com/Products/Liquid-Cooling/iCDM

11. Advanced Thermal Solutions, https://www.qats.com/Heat-Sinks/Device-Specific-Freescale

12. Lenovo, https://www.lenovo.com/us/en/servers-storage/neptune/?orgRef=https%253A%252F%252Fwww.google.com%252F

13. Deloitte, https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2025/genai-power-consumption-creates-need-for-more-sustainable-data-centers.html

14.GoldmanSachs, https://www.goldmansachs.com/insights/articles/ai-to-drive-165-increase-in-data-center-power-demand-by-2030

15. Newmark, https://www.nmrk.com/insights/market-report/2023-u-s-data-center-market-overview-market-clusters

1 Comment

Posted in Artificial Intelligence, Cabinet, Chassis, Cold Plates, datacenter, Heat Exchangers, Heat Sinks, Jet Impingement, Liquid Cooling, NVIDIA, Thermal Design, thermal management, Uncategorized

Tagged , , , , , , , , , , ,

#WeCoverTheBoard: ATS Has Thermal Solutions to Cover the Whole Board

Posted on February 16, 2017 by | 6 comments

We Cover The Board

Advanced Thermal Solutions, Inc. (ATS) has an extensive line of heat sinks and board level thermal solutions that allow ATS engineers to work with industry-leading components and solve the industry’s toughest thermal challenges. (Advanced Thermal Solutions, Inc.)

Advanced Thermal Solutions, Inc. (ATS) has an extensive product line of innovative, off-the-shelf and custom heat sinks and attachments that provides the broadest range of designs to meet the demanding thermal challenges presented by today’s high-powered electronics. Led by its patented maxiFLOW™, which provides the highest thermal performance for physical volume it occupies compared to other heat sinks on the market, ATS has a solution to meet any thermal problem.

In addition, ATS engineers have world-renowned expertise in thermal management and are capable of designing liquid and air cooling solutions using heat sinks, heat pipes, heat exchangers, fans, and cold plates. ATS has more than two decades of solving the industry’s toughest thermal challenges and have a proven record of success in handling the industry’s leading components.

From the latest generation of Intel processors to Altera’s high-powered Stratix FPGA to Qualcomm’s ARM processors to Texas Instruments, Nvidia, NXP, Cavium, and many more, ATS has the experience, the analytical capability, and the products to provide you with the necessary thermal management.

Board Level Solutions

maxiFLOW™ – maxiFLOW™ heat sink design provides the highest thermal performance for the physical volume that it occupies as compared to other heat sink designs. maxiFLOW™ heat sinks are ideally suited to meet the thermal requirements of a broad range of electronics packages, including: BGA, QFP, LCC, LGA, CLCC, TSOP, DIPs and LQFP.

Straight Fin – ATS offers a large variety of high performance Straight fin heat sinks that can be used in many applications where the direction of the airflow is clearly defined. The straight fin heat sink can be utilized in areas where the maxiFLOW™ flair-fanned cannot be used, providing an excellent alternative for cooling thermally sensitive devices.

Cross-Cut – Electronics packages are numerous and range from BGA, QFP, LCC, LGA, CLCC, TSOP, DIPs, LQFP and many others. ATS offers a large variety of cross cut heat sinks that can be used in a variety of applications where the direction of the airflow is ambiguous. The cross cut allow for the heat sink to receive air from any direction.

Pin Fin – Electronics packages are numerous and range from BGA, QFP, LCC, LGA, CLCC, TSOP, DIPs, LQFP and many others. ATS offers a large variety of cross cut heat sinks that can be used in a variety of applications where the direction of the airflow is ambiguous. The cross cut tape on allow for the heat sink to receive air from any direction and can be easily attached to the device by a thermally conductive tape.

fanSINK™ – In many electronic systems, such as telecomm and datacom chassis, or 1U, 2U servers and blades, the system air flow rate is not adequate for cooling of high power devices. Therefore, additional air flow introduced at the device level is required. ATS offers a large family of fanSINK™ products for applications where FPGA or ASICs in BGA packages are deployed. ThefanSINK™ can be either clipped on to the device by maxiGRIP™ or superGRIP™ heat sink attachment technologies or taped on.

Power Brick – DC/DC power converters are an essential part of PCB design and their performance requires a stable temperature for optimum performance. ATS has produced a broad array of high performance power brick heat sinks, based off of the patented maxiFLOW™ design, to effectively cool DC/DC power converters and power modules deployed in a host of electronics applications. ATS’ power brick heat sinks are available in full, half, quarter and eighth packaging.

pushPIN™ – With over 108K different push pin heat sink assembly configurations, ATS offers the largest push pin heat sink offering in the market. Select from fine and ultra-fine pitch heat sinks designed for high velocity air flows and coarse pitch heat sinks for low velocity air flow conditions. Offered in straight fin, cross-cut and the ultra performance maxiFLOW™ fin geometries, ATS pushPIN™ heat sink line is suited to meet a wide variety of applications for components ranging in size from 25mm-70mm. Push pins are offered in brass and plastic and are packaged with different compression springs to achieve precise force required for secure attachment.

blueICE™ (Ultra Low Profile) – In many electronics systems, such as Telecomm, Datacomm, Biomedical equipment and others, card-to-card spacing is small, yet stringent thermal requirements remain the same. Electronics packages such as BGA, QFP, LCC, LGA, CLCC, TSOP, DIP, LQFP are commonly used with stringent thermal requirements in a tight space with limited airflow. Ultra low profile heat sinks offered by ATS range from 2 to 7mm in height and are ideally suited for tight-space application electronics since they offer the best thermal performance. Their thermal resistance is as low as 1.23° C/W within an air velocity of 600 ft/min.

Standard Board Level – ATS’ high quality, low cost, aluminum stamped heat sinks are ideal for low power thermal management solutions. The simple design and manufacturing of these heat sinks allows high volume manufacturing and reducing assembly costs. Stamped heat sinks are ideally used for TO packages and other power devices.

Extrusions – Aluminum extrusions are the most cost-effective solutions for the majority of electronic cooling applications. ATS offers a wide variety of aluminum profiles used for heat sink fabrication and other aluminum applications. Whether you are seeking a standard extrusion profile or the expertise from our design team to create a new and innovative profile, ATS has the capabilities and expertise to meet your requirements.

Heat Sink Attachments

superGRIP™ – superGRIP™ is a two component attachment system which quickly and securely mounts heat sinks to a wide range of components, without needing to drill holes in the PCB. superGRIP™ provides a strong, even attachment force with minimal space required around the components perimeter, making it ideal for densely populated PCBs. superGRIP™ is available with ATS maxiFLOW™ heat sink and straight fin heat sinks.

maxiGRIP™ – maxiGRIP™ is a unique, two component attachment system which quickly and securely mounts heat sinks to a wide range of components, without needing to drill holes in the PCB. The steady, even attachment force provided by maxiGRIP™ allows the heat sink and thermal interface material to achieve maximum thermal performance. maxiGRIP™ is available with ATS maxiFLOW™, straight fin, fanSINK™ and device specific heat sinks.

Thermal Tape – The interface material plays a pivotal role in transporting the heat from the component to the heat sink. The tape is applied to the base of the heat sink and then the heat sink is attached to the component. For tape to work well, proper cleaning of the component surface and the base of heat sink is required. Also, it is usually necessary to apply the tape with a certain amount of pressure.

6 Comments

Posted in Heat Sink Attach, Heat Sinks, PCB, Product Highlights

Tagged , , , , , , , , , , , , , , , , ,