Category Archives: datacenter

Cooling AI Data Centers

Posted on June 27, 2025 by | Leave a comment

How important are AI data centers? In just months, Elon Musk’s xAI team converted a factory outside Memphis into a cutting-edge, 100,000-GPU center for training the Colossus supercomputer—home to the Grok chatbot.

Initially powered by temporary gas turbines (later replaced by grid power), Colossus installed its first 100,000 chips in only 19 days, drawing praise from NVIDIA CEO Jensen Huang. Today, it operates 200,000 GPUs, with plans to reach 1 million GPUs by the end of 2025. [1]

Figure 1 – Elon Musk’s 1 Million Sq Ft xAI Colossus Supercomputer Facility near Memphis, TN. [1]

There are about 12,000 data centers throughout the world, nearly half of them in the United States. Now, more and more of these are being built or retrofitted for AI-specific workloads. Leaders include Musk’s xAI, Microsoft, Meta, Google, Amazon, OpenAI, and others.

High power is essential for such operations, and like computational electronics of all sizes heat issues need to be resolved.

GenAI

A key driver of data center growth is Generative AI (GenAI)—AI that creates text, images, audio, video, and code using deep learning. Chatbots and large language model ChatGPT are examples of GenAI, along with text-to-image models that generate images from written descriptions.

Managing all this is possible from new generations of processors, mainly GPUs. They all draw on higher levels of power and generate higher amounts of heat.

Figure 2 – Advanced AI Processor, the NVIDIA GH200 Grace Hopper Superchip with Integrated CPU to Increase Speed and Performance. [2,3]

AI data centers prioritize HPC hardware: GPUs, FPGAs, ASICs, and ultra-fast networking. Compared to CPUs (150–200 W), today’s AI GPUs often run >1,000 W.  . To handle massive datasets and complex computations in real-time they need significant power and cooling infrastructure.

Data Center Cooling Basics

Traditional HVAC was sufficient for older CPU-driven data centers. Today’s AI GPUs demand far more cooling, both at the chip level and facility-wide. This has propelled a need for more efficient thermal management systems at both the micro (server board and chip) and macro (server rack and facility) levels. [4]

Figure 3 – The Colossus AI Supercomputer Now Runs 200,000 GPUs. It Operates at 150MW Power, Equivalent to 80,000 Households. [5]

At Colossus, Supermicro 4U servers house NVIDIA Hopper GPUs cooled by:

  • Cold plates
  • Coolant distribution manifolds (1U between each server)
  • Coolant distribution units (CDUs) with redundant pumps at each rack base [6]

Each 4U server is equipped with eight NVIDIA H100 Tensor Core GPUs. Each rack contains eight 4U servers, totaling 64 GPUs per rack.

Between every server is a 1U manifold for liquid cooling. They connect with CDUs, heat-exchanging Coolant Distribution Units at the bottom of each rack that include a redundant pumping system. The choice of coolant is determined by a range of hardware and environmental factors.

Figure 4 – Each Colossus Rack Contains Eight 4U Servers, Totaling 64 GPUs Per Rack. Between Each Server is a 1U Manifold for Liquid Cooling. [7]
Figure 5 – The Base of Each Rack Has a 4U CDU Pumping System with Redundant Liquid Cooling. [7]

Role of Cooling Fans

Fans remain essential for DIMMs, power supplies, controllers, and NICs.

Figure 6 – Rear Door Liquid-Cooled Heat Exchangers. [7]

At Colossus, fans in the servers pull cooler air from the front of the rack, and exhaust the air at the rear of the server. From there, the air is pulled through rear door heat exchangers. The heat exchangers pass warm air through a liquid-cooled, finned heat exchanger/radiator, lowering its temperature before it exits the rack.

Direct-to-Chip Cooling

NVIDIA’s DGX H100 and H200 server systems feature eight GPUs and two CPUs that must run between 5°C and 30°C. An AI data center with a high rack density houses thousands of these systems performing HPC tasks at maximum load. Direct liquid cooling solutions are required.

Figure 7 – An NVIDIA DGX H100/H200 System Featuring Eight GPUs [8]
Figure 8 – The NVIDIA H100 SmartPlate Connects to a Liquid Cooling System to Bring Microconvective Chip-Level Cooling That Outperforms Air Cooling by 82%. [9]

Direct liquid cooling (cold plates contacting the GPU die) is the most effective method—outperforming air cooling by 82%. It is preferred for high-density deployments of the H100 or GH200.

Scalable Cooling Modules

Colossus represents the world’s largest liquid-cooled AI cluster, using NVIDIA + Supermicro technology. For smaller AI data centers, Cooling Distribution Modules (CDMs) provide a compact, self-contained solution.

Figure 9 – The iCDM-X Cooling Distribution Module from ATS Includes Pumps, Heat Exchanger and Liquid Coolant for Managing Heat from AI GPUs and Other Components. [10]

Most AI data centers are smaller, and power and cooling needs are lower, but essential. Many heat issues can be resolved using self-contained Cooling Distribution Modules.

The compact iCDM-X cooling distribution module provides up to 1.6MW of cooling for a wide range of AI GPUs and other chips. The module measures and logs all important liquid cooling parameters. It uses using just 3kW of power, and no external coolant is required.

These modules include:

•         Pumps

•         Heat exchangers

•         Cold plates

•         Digital monitoring (temp, pressure, flow)

Their sole external component is one or more cold plates removing heat from AI chips. ATS provides an industry-leading selection of custom and standard cold plates, including the high-performing ICEcrystal series.

Figure 10 – The ICEcrystal Cold Plates Series from ATS Provide 1.5 kW of Jet Impingement Liquid Cooling Directly onto AI Chip Hotspots.

Cooling Edge AI and Embedded Applications

AI isn’t just for big data centers—edge AI, robotics, and embedded systems (e.g., NVIDIA Jetson Orin, AMD Kria K26) use processors running under 100 W. These are effectively cooled with heat sinks and fan sinks from suppliers like Advanced Thermal Solutions. [11]

Figure 11 – High Performance Heat Sinks for NVIDIA and AMD AI Processors in Embedded and Edge Applications. [11]

NVIDIA also partners with Lenovo, whose 6th-gen Neptune cooling system enables full liquid cooling (fanless) across its ThinkSystem SC777 V4 servers—targeting enterprise deployments with NVIDIA Blackwell + GB200 GPUs. [12]

Figure 12 – Lenovo’s Neptune Direct Water Cooling Removes Heat from Power Supplies, for Completely Fanless Operation. [12]

Benefits gained from the Neptune system include:

  • Full system cooling (GPUs, CPUs, memory, I/O, storage, regulators)
  • Efficient for 10-trillion-parameter models
  • Improved performance, energy efficiency, and reliability

Conclusion

With surging demand, AI data centers are now a major construction focus. Historically, cooling problems are the #2 cause of data center downtime (behind power issues). With the high power needed for AI computing, these builds should carefully fit with their local communities in terms of electrical needs and sources, and water consumption. [13]

AI workloads will increase U.S. data center power demand by 165% by 2030 (Goldman Sachs), with nearly double 2022 levels (IBM/Newmark). Sustainable design and resource-conscious cooling are essential for the next wave of AI infrastructure. [14,15]

References

1. The Guardian, https://www.theguardian.com/technology/2025/apr/24/elon-musk-xai-memphis

2. Fibermall, https://www.fibermall.com/blog/gh200-nvidia.htm

3. NVIDA, https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip?ncid=no-ncid

4. ID Tech Ex, https://www.idtechex.com/en/research-report/thermal-management-for-data-centers-2025-2035-technologies-markets-and-opportunities/1036

5. Data Center Frontier, https://www.datacenterfrontier.com/machine-learning/article/55244139/the-colossus-ai-supercomputer-elon-musks-drive-toward-data-center-ai-technology-domination

6. Supermicro, https://learn-more.supermicro.com/data-center-stories/how-supermicro-built-the-xai-colossus-supercomputer

7. Serve The Home, https://www.servethehome.com/inside-100000-nvidia-gpu-xai-colossus-cluster-supermicro-helped-build-for-elon-musk/2/

8. Naddod, https://www.naddod.com/blog/introduction-to-nvidia-dgx-h100-h200-system

9. Flex, https://flex.com/resources/flex-and-jetcool-partner-to-develop-liquid-cooling-ready-servers-for-ai-and-high-density-workloads

10. Advanced Thermal Solutions, https://www.qats.com/Products/Liquid-Cooling/iCDM

11. Advanced Thermal Solutions, https://www.qats.com/Heat-Sinks/Device-Specific-Freescale

12. Lenovo, https://www.lenovo.com/us/en/servers-storage/neptune/?orgRef=https%253A%252F%252Fwww.google.com%252F

13. Deloitte, https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2025/genai-power-consumption-creates-need-for-more-sustainable-data-centers.html

14.GoldmanSachs, https://www.goldmansachs.com/insights/articles/ai-to-drive-165-increase-in-data-center-power-demand-by-2030

15. Newmark, https://www.nmrk.com/insights/market-report/2023-u-s-data-center-market-overview-market-clusters

Leave a comment

Posted in Cabinet, Chassis, Cold Plates, datacenter, Heat Exchangers, Heat Sinks, Jet Impingement, Liquid Cooling, NVIDIA, Thermal Design, thermal management, Uncategorized

Meeting the thermal management requirements of high-performance servers

Posted on April 3, 2019 by | 10 comments

High-performance servers are devices specially designed to handle large computational loads, a huge amount of communication signals, fast data processing, etc. Due to their task-oriented nature, high-performance servers must have high reliability, interchangeability, compact size and good serviceability.

Photo by Taylor Vick on Unsplash

To achieve high computational speed, high-performance servers generally have dozens of CPUs and memory models. They also have dedicated data process modules and control units to ensure seamless communication between CPUs and parallel data processing ability. To reach higher speeds, the power dissipation of high–performance CPUs has been increasing continuously in the past decade for its use in high-performance servers.

Cooling dozens of kW servers brings a unique challenge for thermal engineers. To deal with the ever-growing high heat flux issue in high-performance servers, it will need the cooperation of electrical, mechanical and system engineers to solve the problem. The job to remove the high heat flux from CPUs to ambient requires chip level, board level and cabinet level solutions.

Wei [1] described Fujitsu’s thermal management advancements in their high-end UNIX server PRIMEPOWER 2500. The server cabinet is shown in Figure 1. Its dimension is 180cm × 107cm × 179cm (H×W×D) and has a maximum power dissipation of 40 kW. The system configuration of PRIMEPOWER 2500 is shown in Figures 2 and 3. It has 16 system boards and 2 input/output (I/O) boards installed vertically on two back-panel boards. The two back-panel boards are interconnected by six (6) crossbars installed horizontally.

Figure 1. PRIMEPOWER 2500 Cabinet [1]
Figure 2. PRIMEPOWER 2500 System Configuration [1]
Figure 3. PRIMEPOWER 2500 System Board Unit [1]

To cool the electrical components inside PRIMEPOWER 2500, 48 200-mm-diameter fans are installed between the system board unit and the power supply unit. They provide forced air cooling for system boards and power supplies. In addition, six 140-mm-diameter fans are installed on one side of crossbar to cool the crossbar boards with a horizontal flow. The flow direction is shown in Figure 3. Each system board is 58 cm wide and 47 cm long.

There are eight CPU processors, 32 Dual In-Line Memory Modules, 15 system controller processors, and associated DC-DC converters on each system board. The combined power dissipation per system board is 1.6 kW at most.

Figure 4. PRIMEPOWER 2500 System Board [1]

To cool the electrical components inside PRIMEPOWER 2500, 48 200-mm-diameter fans are installed between the system board unit and the power supply unit. They provide forced air cooling for system boards and power supplies. In addition, six 140-mm-diameter fans are installed on one side of crossbar to cool the crossbar boards with a horizontal flow. The flow direction is shown in Figure 3. Each system board is 58 cm wide and 47 cm long.

There are eight CPU processors, 32 Dual In-Line Memory Modules, 15 system controller processors, and associated DC-DC converters on each system board. The combined power dissipation per system board is 1.6 kW at most.

Forced air-cooling technology is commonly used in computers, communication cabinets, and embedded systems, due to its simplicity, low cost and easy implementation. For high-performance servers, the increasing power density and constraints of air-cooling capability and air delivery capacity have pushed forced air cooling to its performance limit.

For high power systems like PRIMEPOWER 2500, it needs a combination of good CPU design, optimized board layout, advanced thermal interface material (TIM), high-performance heat sinks, and strong fans to achieve desired cooling.

The general approach to cool the multi-board system is first to identify the hottest power component with the lowest temperature margin. For the high-performance server, it is the CPUs. For multiple CPUs on a system board, generally, the CPU located on downstream of a board or other CPUs has the highest temperature.

So, the thermal resistance requirement for this CPU is:

Where Tj,max is the allowed maximum junction temperature, Ta is the ambient temperature, ∆Ta is the air temperature rise due to preheating before the CPU, and qmax is the maximum CPU power.

The junction-to-air thermal resistance of the CPU is:

Where Rjc is the CPU junction-to-case thermal resistance, RTIM is the thermal resistance of thermal interface materials, and Rhs is the heat sink thermal resistance. To reduce the CPU junction temperature, it is critical to find intuitive ways to minimize Rjc, RTIM, and Rhs, because any reduction in thermal resistance is important in junction temperature reduction.

The CPU package and heat sink module of PRIMEPOWER 2500 are shown in Figure 5. The CPU package has an integrated heat spreader (IHS) attached to the CPU chip. A high-performance TIM is used to bond the CPU chip and IHS together, see Figure 6. The heat sink module is mounted on the IHS with another TIM in between.

Figure 5. PRIMEPOWER 2500 CPU Package and Heat Sink Module [1]
Figure 6. CPU Package [1]

The TIM used in between the CPU chip and the IHS are crucial to the CPU’s operation. It has two key functions: to conduct heat from the chip to the IHS and to reduce the CPU chip stress caused by the mismatch of the coefficient of thermal expansion (CTE) between the CPU chip and IHS. Fujitsu developed a TIM made of In-Ag composite solder for the above application. The In-Ag composite has a low melting point and a high thermal conductivity. It is relatively soft, which is good for absorbing thermal stress between the chip and the IHS.

Wei [2] also investigated the impact of thermal conductivity on heat spread performance. He found a diamond composite IHS (k=600 W/(mK)) would result in a lower temperature gradient across the chip and low temperature hot spots, compared with aluminum nitride (k=200 W/(mK)) and copper (k=400 W/(mK)). The simulation results are shown in Figure 7.

Figure 7. Heat Spreader Material Comparison [2]

In high-performance servers like the PRIMEPOWER 2500, the thermal performance gains by optimizing the TIM and the IHS are small, because they compose only a small portion of the total thermal resistance. Heat sinks dissipate heat from the CPU to air and have an important role in the thermal management of the server. In a server application, the heat sink needs to meet not only the mechanical and thermal requirements, but also the weight and volume restraints. Hence, heat pipes, vapor chambers, and composite materials are widely used in place of high-performance heat sinks.

Koide et al [1] compared the thermal performance and weight of different heat sinks for server application. The results are shown Figure 8. They used the Cu-base/AL-fin heat sink as benchmark. Compared with the Cu-base/AL-fin heat sink, the Cu-base/Cu-fin heat sink is 50% heavier and gains only 8% performance.

If the heat pipe is used in base, the heat sink weight can be reduced by 15% and the thermal performance increases by 10%. If the vapor chamber is embedded in the heat sink base, it reduces the heat sink weight by 20% and increases the heat sink performance by 20%.

Figure 8. Thermal Performance and Weight Comparison of Different Heat Sinks [1]
Figure 9. (a) USIII Heat Sink for Sun Fire 15K Server, (b) USIV Heat Sink for Sun Fire 25K [3]

Sun Microsystems’ high-performance Sun Fire 15K Server uses USIII heat sink to cool its 72 UltraSparc III (USIII) processors. In Sun Fire 25K Server, the CPUs are upgraded to UltraSparc IV (USIV), which has a maximum power of 108 W. To cool the USIV processor, Xu and Follmer [3] designed a new USIV heat sink with copper base/copper fin, see Figure 9. The old USIII heat sink has 17 forged aluminum fins, the USIV heat sink has 33 copper fins. Both heat sinks have the same base dimensions and height.

Figure 10. Thermal Resistance Comparison between USIII Heat Sink and USIV Heat Sink [3]

Figure 10 shows the thermal resistance comparison between the USIII heat sink and the USIV heat sink. The thermal resistance of the USIV heat sink is almost 0.1°C/W lower than that of the USIII heat sink at medium and high flow rates, which is a huge gain in thermal performance. The thermal performance improvement of the USIV heat sink is not without penalty.

Figure 11. Pressure Drop Comparison between USIII Heat Sink and USIV Heat Sink [3]

Figure 11 shows the pressure drop comparison between the USIII heat sink and the USIV heat sink. For the same air flow rate, the pressure drop of the USIV heat sink is higher than that of the USIII heat sink. That means the Sun Fire 25K Server needs stronger fans and better flow arrangements to ensure the USIV heat sinks have adequate cooling flow.

The design of the cooling method in high-performance servers follows the same methodology used in the design cooling solution of other electronic devices, but at an elevated scale. The main focus is to identify the hottest components, which in most cases is CPUs. Due to extreme high power of CPUs, memory modules, cheat spreader, TIM, and heat sinks to achieve desired cooling in the server. The goal of thermal management is to find cost-effective ways to maintain the junction temperature of the CPU lower than specifications and ensure the continuous operation of the server. Wei [1] has proved a 40 kW server can be cooled by forced air cooling.

However, it requires highly integrated design and a huge amount of air flow that the 54 fans inside PRIMEPOWER 2500 can generate. In the near future, it would be very difficult for a forced air-cooling method to cool cabinets with more than 60 kW power. It would require bigger fan trays to deliver huge amounts of air flow and large size heat sinks to transfer heat from the CPUs to air, which makes it impossible to design a reliable, compact and cost-effective cooling system for the server.

We have to find alternative ways to deal with this problem, Other cooling methods, such as air impinging jets, liquid cooling and refrigeration cooling systems, have the potential to dissipate more heat. But it will require intuitive packaging to integrate them into the server system.

References:

  1. Wen, J., Thermal Management of Fujitsu’s High-performance
    Servers, source: http://www.fujitsu.com/downloads/MAG/vol43-1/paper14.pdf.
  2. Koide, M.; Fukuzono, K.; Yoshimura, H.; Sato, T.; Abe, K.;
    Fujisaki, H.; High-Performance
    Flip-Chip BGA Technology Based on Thin-Core and Coreless Package Substrates    ,
    Proceedings of 56th ECTC, San Diego, CA, USA, 2006, pp.1869-1873.
  3. Xu, G; Follmer, L.; Thermal Solution
    Development for High-end System    , Proceedings of 21st IEEE SEMI-THERM
    Symposium, San Jose, CA, USA, 2005, pp. 109-115.

For more information about Advanced Thermal Solutions, Inc. (ATS) thermal management consulting and design services, visit https://www.qats.com/consulting or contact ATS at 781.769.2800 or ats-hq@qats.com.

10 Comments

Posted in datacenter, Servers

Tagged , ,

Thermal Management of High-Powered Servers

Posted on March 14, 2019 by | 9 comments

While power demands have increased, engineers are tasked with placing more components into smaller spaces. This has led to increased importance for optimizing the thermal management of servers and other high-powered devices to ensure proper performance and achieve the expected lifespan.

This article brings together content that ATS has posted over the years about cooling high-powered servers from the device- to the environment-level. (Photo by panumas nikhomkhai – Pexels.com)

With server cooling taking on increased priority, there are several ways of approaching the problem of thermal management, including device-level solutions, system-level solutions, and even environment-level solutions.

Over the years, Advanced Thermal Solutions, Inc. (ATS) has posted many articles related to this topic. Click the links below to read more about how the industry is managing the heat for servers:

  • Industry
    Developments: Cabinet Cooling Solutions     – Although their applications vary,
    a common issue within these enclosures is excess heat, and the danger it poses
    to their electronics. This heat can be generated by internal sources and
    intensified by heat from outside environments.
  • Effective cooling of high-powered CPUs on dense server boards – Optimizing PCB for thermal management has been shown to ensure reliability, speed time to market and reduce overall costs. With proper design, all semiconductor devices on a PCB will be maintained at or below their maximum rated temperature. 

For more information about Advanced Thermal Solutions, Inc. (ATS) thermal management consulting and design services, visit https://www.qats.com/consulting or contact ATS at 781.769.2800 or ats-hq@qats.com.

9 Comments

Posted in Cabinet, datacenter, Servers

Tagged ,

Technology Review: Data Center Cooling

Posted on December 20, 2018 by | Leave a comment

(This article was featured in an issue of Qpedia Thermal e-Magazine, an online publication dedicated to the thermal management of electronics. To get the current issue or to look through the archives, visit http://www.qats.com/Qpedia-Thermal-eMagazine.)

Qpedia continues its review of technologies developed for electronics cooling applications. We are presenting selected patents that were awarded to developers around the world to address cooling challenges. After reading the series, you will be more aware of both the historic developments and the latest breakthroughs in both product design and applications.

Data Center Cooling

This article explores recent patents and technical advancements from the data center cooling industry. (Wikimedia Commons)

We are specifically focusing on patented technologies to show the breadth of development in thermal management product sectors. Please note that there are many patents within these areas. Limited by article space, we are presenting a small number to offer a representation of the entire field. You are encouraged to do your own patent investigation.

Further, if you have been awarded a patent and would like to have it included in these reviews, please send us your patent number or patent application.

In this issue our spotlight is on data center cooling solutions.

There are many U.S. patents in this area of technology, and those presented here are among the more recent. These patents show some of the salient features that are the focus of different inventors.

Hybrid Immersion Cooled Server with Integral Spot and Bath Cooling

US 7724524 B1, Campbell, L., Chu, R., Ellsworth, M., Iyengar, M. and Simons, R.

An immersion cooling apparatus and method is provided for cooling of electronic components housed in a computing environment. The components are divided into primary and secondary heat generating components and are housed in a liquid sealed enclosure. The primary heat generating components are cooled by indirect liquid cooling provided by at least one cold plate having fins. The cold plate is coupled to a first coolant conduit that circulates a first coolant in the enclosure and supplies the cold plate. Immersion cooling is provided for secondary heat generating components through a second coolant that will be disposed inside the enclosure such as to partially submerge the cold plate and the first coolant conduit as swell as the heat generating components.

The shortcomings of the prior art are overcome and additional advantages are provided through the provision of a method and associated hybrid immersion cooling apparatus for cooling of electronic components housed in a computing environment. The components are categorized as primary and secondary heat generating components and are housed in a liquid tight enclosure. The primary heat generating components are cooled by indirect liquid cooling provided by at least one cold plate having fins.

The cold plate is coupled to a first coolant conduit that circulates a first coolant in the enclosure and supplies the cold plate. Immersion or direct liquid cooling is provided for secondary heat generating components through a second coolant that will be disposed inside the enclosure such as to partially submerge the cold plate and the first coolant conduit as Well as the heat generating components.

In one embodiment, the cold plate and the coolant conduit each comprise extended external surfaces used to transfer heat from the second coolant to the first coolant. In one embodiment the second coolant cools the electronics via free convection or a combination of free convection and boiling while an alternate embodiment provides a sub merged pump to aid circulation. In yet another embodiment, the electronics are housed on an electronics board that is tilted at an angle to also aid circulation of the second coolant.

Thermal Caching For Liquid Cooled Computer Systems

US 7436666 B1, Konshak, M.

The present invention involves supplementing the liquid cooling path within a given rack of electronic components in order to reduce the temperature rise of critical components before damage or data loss can occur in the event of a primary cooling system failure. Liquid cooled racks generally have piping and heat exchangers that contain a certain volume of liquid or vapor that is being constantly replaced by a data center pump system. Upon a data center power or pump failure, the How stops or is diminished. The coolant, however, is still present within the rack components. Upon detection of a lack of How, an unexpected rise in the coolant temperature, or a significant reduction in How pressure, and before any liquid can be drained away from the racks, an electrically or hydraulically operated valve bypasses the supply and return of the data center cooling path, creating an independent closed loop supplemental cooling system within the rack itself.

One embodiment of a system for mitigating a failure of a data center’s liquid cooling system is shown. A high level block diagram of one embodiment of a system for thermal caching in a liquid cooled computer system is presented. Under normal circumstances, coolant is delivered to a rack housing a plurality of liquid cooled components by a data center liquid cooling system supply and return lines. The supply and return lines are coupled to the rack by quick disconnecting and the coolant is pumped through the conduits by one or more data center pumps. Within the rack a secondary cooling loop is established that is in fluid communication with the data center cooling loop.

Upon entering the rack the data center supply line enters a monitoring device. In one embodiment of the present invention the device monitors fluid flow, pressure, and/or temperature. Other characteristics of the fluid that can also be used to identify a failure of the data center cooling system as imposed on the supply line. Thereafter, and before allowing the coolant to access any of the electronic components housed within the rack, a one way valve is placed on the supply lid entering the rock. The valve allows fluid to pass from the data center cooling supply line into the rack but prevents flow from regressing toward the supply line should the flow stop and/or an adverse pressure gradient is experienced.

Similarly, a two-way valve is placed on the coolant return line that returns heated coolant from the electronic components to the data center coolant return line. The two way valve is in communication with the monitoring device and capable of receiving a signal that indicates a failure in the primary or data center liquid cooling system.

Upon receiving such a signal from the monitoring device, the two-way valve closes either electrically or hydraulically and diverts the return coolant from the electronic devices to the supplemental or secondary liquid cooling loop. The supplemental liquid cooling loop, which is housed entirely within the rack, thereafter operates independent of the data center liquid cooling system.

Modular High-Density Computer System

US 7688578 B2, Mann, R., Landrum, G. and Hintz, R.

A modular high-density computer system has an infrastructure that includes a framework component forming a plurality of bays and has one or more cooling components. The computer system also has one or more computational components that include a rack assembly and a plurality of servers installed in the rack assembly. Each of the one or more computational components is assembled and shipped to an installation site separately from the infrastructure and then installed within one of the plurality of bays after the infrastructure is installed at the site.

Historically, a room oriented cooling infrastructure was sufficient to handle this cooling problem. Such an infrastructure was typically made up of one or more bulk air conditioning units designed to cool the room to some average temperature. This type of cooling infrastructure evolved based on the assumption that the computer equipment is relatively homogeneous and that the power density is on the order 1 -2 Kilowatts per rack. The high density racks mentioned above, however, can be on the order of 20 Kilowatts per rack and above. This increase in power density, and the fact that the equipment can be quite heterogeneous leads to the creation of hot spots that can no longer be sufficiently cooled with simple room-oriented cooling systems.

By separating the computational components of a high density computer system from the requisite cooling, power distribution, data l/O distribution, and housing infrastructure components, significant advantages can be achieved by embodiments of the invention over current configurations of such computer systems. The computational components can be assembled and tested for proper operation while the infrastructure components are shipped separately in advance of the computational components. Thus, the infrastructure cooling components, data and power distribution components, and framework components can be delivered and their time-intensive installation completed more easily and prior to receiving the tested computational components.

Moreover, the shipping of the computational components separately from the infrastructure components achieves less laborious and less costly delivery and handling of the components. Finally, it makes it easier for the data center to upgrade or add additional infrastructure and infrastructure components to a current installation.

For more information about Advanced Thermal Solutions, Inc. (ATS) thermal management consulting and design services, visit https://www.qats.com/consulting or contact ATS at 781.769.2800 or ats-hq@qats.com.

Leave a comment

Posted in datacenter

Tagged

Industry Developments: Cabinet Cooling Solutions

Posted on September 24, 2018 by | 2 comments

Critical electronics are routinely housed inside metal cabinets of different dimensions. Although their applications vary, a common issue within these enclosures is excess heat, and the danger it poses to their electronics. This heat can be generated by internal sources and intensified by heat from outside environments.

Cabinet Cooling

The trends toward compact, multi-function electronic controls, variable speed drives, programmable logic controllers, and tightly-packed processors and server racks can also cause thermal problems. Excess heat can adversely affect digital displays, controls, breakers, ICs and PCBs. In most cases this heat can’t be prevented, so it must be removed to ensure the proper function and service life of components and boards.

Issues with excess cabinet heat have been around for decades, and many cooling approaches have been utilized. Among the most popular are air conditioners, vortex coolers and heat exchangers. Each method has benefits and shortcomings, and improvements are continually made by the engineers who design these cooling systems.

Air Conditioners

As cabinet designs adapt to new needs, air conditioners are being designed for tighter spaces, higher performance and lower costs. Today’s ACs include traditional vapor-compression-refrigeration technology, as well as new thermoelectric systems.

IceQube is providing the Qube Series of air conditioners, which the company described as the world’s smallest compressor-based air conditioner and an ideal cooling solution for compact enclosures with high heat loads. The compact air conditioners are available in power coated and stainless-steel housings. [1]

Figure 1. The Qube series of vertical mount air conditioners from IceQube come in widths as narrow as six inches. [1]

The Blade air conditioners series, also from IceQube, is specially-designed for door mount applications on electrical enclosure cabinets. They have a space-saving, ultra-thin designs for use in NEMA type 12, 3R, 4 and 4X cabinet designs. Cooling performance from the Blade ACs is up to 50,000 BTU/hr. [2]

Thermoelectric ACs, also called Peltier ACs, work without compressors or refrigerants. Some feature efficiently-designed fans as their only moving parts to provide effective internal cabinet cooling. These models typically provide lower cooling performance, but enough to meet cabinet cooling requirements.

TECA recently introduced internally mounted thermoelectric air conditioners for enclosure cooling where there can be no external protrusions from the enclosure. Available in five sizes, the new air conditioners can be horizontally or vertically mounted inside an enclosure. Performance ratings range from 155 BTU/hr – 390 BTU/hr. These air conditioners use no refrigerants or compressors and have no moving parts other than their fans. [3]

Figure 2. Internally mounted TECA air conditioners are suited for use where space requirements prohibit external protrusion. [3]

When high levels of temperature drop are needed inside a cabinet, EIC Solutions offers an alternative to compressor-based air conditioners. EIC’s new High Delta T thermoelectric air conditioners provide a maintenance-free, solid-state solution for applications that require a large ΔT in any environment. ΔT is the difference between return air temperature and supply air temperature.

The new ThermoTEC 142 and 146 series air conditioners feature high ΔT capabilities to achieve greater drops from ambient temperatures compared to standard models. [4]

Figure 3. New thermoelectric air conditioners have high ΔT capabilities for greater drops from ambient temperatures than other TEC models. [4]

The 142 series (500 BTU/hr) and the 146 series (1000 BTU/hr) feature rugged, type 304 stainless steel, and NEMA 4X construction.

Another compressor-based air conditioner is the SpectraCool from Hoffman. Its filter-free design reduces clogging that can cause system failures. SpectraCool units feature an energy-efficient compressor and earth-friendly refrigerant. Models are available for up to 20,000 BTU/hr cooling performance.

The Hoffman SpectraCool ACs can also be controlled remotely. Access comes via a unique IP address to each equipped unit. This allows monitoring and control of cooling, heating, alarms, the compressor and the ambient fan. [5]

The company’s Easy Swap adaptor plenums provide a quick and easy way to upgrade the SpectraCool systems and deliver up to 23 percent greater energy efficiency.

Vortex Cabinet Coolers

Vortex enclosure cooling systems work by maintaining a slight pressurization in the cabinet to keep electrical and electronic components clean and dry. Most vortex systems are thermostatically-controlled to keep cabinet temperatures within a specified temperature range.

The core of these coolers is composed of vortex tubes, mechanical devices that separate compressed gas into hot and cold streams. Air emerging from the cold end can reach -50°C, while air emerging from the hot end can reach 200°C. The tubes have no moving parts. [6]

EXAIR Cabinet Cooler systems use vortex tube technology to create a cold air outlet flow which is pumped into an electronic cabinet. As air is pushed into the cabinet the Cabinet Cooler system also provides its own built-in exhaust. There is no need to vent the cabinet. This creates a positive purge on the cabinet to keep out dirt, dust and debris.

Figure 4. The Exair dual cabinet cooler system minimizes compressed air use and produces 20°F air for cabinet cooling. [7]

EXAIR Cabinet Cooler systems are unaffected by vibration, which can cause refrigerant leaks and component failures in traditional air conditioners. They are UL listed for NEMA 12, 4 and 4X integrity and are marked CE for conforming to European Union safety standards.

ITW Vortec, a leader in vortex tube technology and enclosure cooling provides the UL-listed Electric Vortex A/C. The Electric Vortex A/C is an electric thermostat cabinet cooler with plug-and-play functionality. Unlike traditional electric thermostat enclosure coolers which require additional wiring and piping to properly install, the new Electric Vortex A/C comes pre-wired, requires no additional wiring and just needs an outlet within six feet of the unit.

This new solution eliminates the need for an external solenoid valve and the piping traditionally used to install other enclosure cooler solutions. An electric thermostat allows the user to set the desired temperature to be maintained in the enclosure. The cooler will only turn on when necessary, conserving energy from compressed air usage.

Figure 5. The Electric Vortex air conditioner from Vortec eliminates the need for an external solenoid valve. [8]

Heat Exchangers

Using heat exchangers as cabinet thermal solutions can provide an enhanced solution in terms of performance, cost-effectiveness and smaller size designs. There are both air-to-air, and liquid-to-air models.

Air-to-Air Exchangers

Air-to-air heat exchangers are a proven and dependable cooling method that relies on passive heat pipe or folded fin impingement cores to disperse the heat from within cabinet enclosures to the outside ambient air.

Figure 6. Air-to-air heat exchangers transfer heat without moving parts. [9]

The Stratus line of air-to-air heat exchangers from AutomationDirect includes 120 VAC and 24 VDC models. The series has a closed-loop cooling system, using the heat pipe principle to exchange heat from inside to outside the cabinet.

Each heat pipe has an evaporator section and a condenser section. These are separated by a permanent baffle to provide a closed loop. The coil systems use aluminum end plates and baffles, which improve conduction and reduce corrosion for longer life. The Stratus heat exchangers are available in models for NEMA 4 and 4X enclosures. Units come in three frame sizes (compact, deep, and tall) with up to 72 watts capacity. They are equipped with two circulating fans with sealed overload protectors. [9]

Liquid-to-Air Exchangers

Liquid-to-air heat exchangers provide cooling through a closed-loop system. They are designed for use where heat dissipation needs are too great for natural or forced air convection systems, or where remote heat dissipation is required. Much of their higher cooling performance comes from using fluids with much higher thermal conductivity than air. Typical applications include cabinets, MRI and process cooling.

Figure 7. The WL500 water-cooled, liquid-to-air heat exchanger has a high-pressure pump for fast flow rates. [10]

Conclusion

Cabinet-housed electronics are susceptible to excess heat generated from within along with heat from outside environments. Thus, keeping the electronics cool inside cabinets is essential to maximizing internal device life cycles. Numerous cooling methods are available, including air conditioning, vortex cooling, and heat exchangers.

Each of these methods has its own methodology, such as the choice of air or liquid cooling, to provide options for meeting cooling requirements. A thorough awareness of options, application requirements, and resources should lead to the best cabinet cooling solutions

References
1. IceQube, http://www.iceqube.com/air-conditioners/qube-series-mm/
2. IceQube, http://www.iceqube.com/blade-series-products/blade-series-air-conditioners/
3. TECA Corporation, http://www.thermoelectric.com/2010/ad/internal-mounted.htm
4. EIC Solutions, Inc., http://www.eicsolutions.com/
5. Hoffman, http://www.pentairprotect.com/hoffman/
6. Vortex Tube, https://en.wikipedia.org/wiki/Vortex_tube
7. EXAIR, http://www.exair.com/ https://www.youtube.com/watch?v=CoxzwJmbyxY
8. ITW Vortec, https://www.vortec.com/p-284-electric-vortex-ac.aspx
9. AutomationDirect, http://www.automationdirect.com
10. Laird, http://www.lairdtech.com/products/wl-500

2 Comments

Posted in datacenter

Tagged , , ,