{"id":9940,"date":"2019-03-26T13:33:38","date_gmt":"2019-03-26T17:33:38","guid":{"rendered":"http:\/\/www.qats.com\/cms\/?p=9940"},"modified":"2019-03-26T13:33:44","modified_gmt":"2019-03-26T17:33:44","slug":"immersion-liquid-cooling-of-servers-in-data-centers","status":"publish","type":"post","link":"https:\/\/www.qats.com\/cms\/2019\/03\/26\/immersion-liquid-cooling-of-servers-in-data-centers\/","title":{"rendered":"Immersion Liquid Cooling of Servers in Data Centers"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">A data center is a large infrastructure used to house large quantities of electronic equipment, such as computer servers, telecommunications equipment, and data storage systems, etc. The data center requires non-interrupted power, communication and internet access to all equipment inside, it also has dedicated environment control system which provides appropriate working conditions for the electrical devices hosted inside. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"682\" src=\"http:\/\/www.qats.com\/cms\/wp-content\/uploads\/Data-Center-Blog-1024x682.jpg\" alt=\"Immersion Cooling\" class=\"wp-image-9941\" srcset=\"https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Data-Center-Blog.jpg 1024w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Data-Center-Blog-300x200.jpg 300w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Data-Center-Blog-800x533.jpg 800w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Traditional\ndata centers use cold air generated by a room air conditioner system (CRAC) to\ncool the servers installed on the racks. Cooling the electrical devices by cold\nair generated by an air conditioner is an easy method to implement. However, it\nis not a very efficient method in terms of power consumption. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The\ninefficiency of the method can be contributed to several causes: generating and\ndelivering cold air from a chiller to servers is a multiple heat transfer\nprocess, such as the mixing of warm and cool air in the room, which reduces the\nefficiency and power consumption of cooling hardware such as chillers, computer\nroom air conditioners (CRACs), fans, blowers and pumps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Data\ncenter designers and operators have invented many ways to improve the data\ncenter\u2019s thermal efficiency, such as optimizing the rack layout and air\nconditioner location, separating cold aisles and hot aisles, optimizing the configuration\nof pipes and cables in under-floor plenum, introducing liquid cooling to\nhigh-power severs. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">While\nthe above methods can improve the data center heat load management, they cannot dramatically reduce the Power\nUsage Effectiveness (PUE), which is a measure of how efficiently a datacenter\nuses its power and is defined as the ratio of total datacenter power\nconsumption to the IT equipment power consumption.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An\nideal PUE is 1,0. A better way, proposed and used by some new data centers, is\ndirectly bringing the outside cold air to the servers. This method can\neliminate the computer room air conditioners (CRACs). To achieve this, the data\ncenter has to be located in a specific area where cold air can be provided for\nall four seasons and the servers have to have higher operating environmental temperature.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another\ndramatic solution proposed and used by some companies is liquid immersion\ncooling for entire servers. When compared with traditional liquid cooling\ntechniques, the liquid immersion cooling uses dielectric fluid as a working\nagent and open bath design. This eliminates the need for hermetic connectors,\npressure vessels, seals and clamshells. There are several different liquid\nimmersion cooling methods. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This\narticle will review the active single-phase immersion cooling technology\nproposed by Green\nRevolution Cooling (GRC) [1] and a passive two-phase immersion cooling\ntechnology proposed by the 3M Company [2].<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Green Revolution Cooling has designed a liquid-filled rack to accommodate the traditional servers and developed dielectric mineral oil as the coolant. Figure 1 shows the liquid cooling racks with chiller and an inside view of a CarnotJet cooling rack from GRC. The racks are filled with 250 gallons of dielectric fluid, called GreenDEF&#x2122;, which is a non-toxic, clear mineral oil with light viscosity. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"819\" height=\"287\" src=\"http:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-1-17.jpg\" alt=\"\" class=\"wp-image-9942\" srcset=\"https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-1-17.jpg 819w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-1-17-300x105.jpg 300w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-1-17-800x280.jpg 800w\" sizes=\"auto, (max-width: 819px) 100vw, 819px\" \/><figcaption>Figure 1. Server racks and chiller (left) and inside view of the server rack. [1]<\/figcaption><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">The\nservers are installed vertically into slots inside the rack and fully submerged\nin the liquid coolant. Pumps are used to circulate the cold coolant from the\nchiller to the rack. The coolant returns to the chiller, after removing heat\nfrom the servers. Because of its high heat capacity and thermal conductivity,\nthe GreenDEF&#x2122; can cool the servers more efficiently than air. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The server racks are semi-open to the environment and the coolant level is constantly monitored by the system. Figure 2 shows a server motherboard is being submerged in the coolant liquid inside a server rack from GRC.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"517\" height=\"353\" src=\"http:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-2-17.jpg\" alt=\"\" class=\"wp-image-9943\" srcset=\"https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-2-17.jpg 517w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-2-17-300x205.jpg 300w\" sizes=\"auto, (max-width: 517px) 100vw, 517px\" \/><figcaption>Figure 2. A Server Motherboard Being Immersed in Liquid Coolant in A Server Rack. [1]<\/figcaption><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Intel\nhas conducted a year-long test with immersion cooling equipment from Green\nRevolution Cooling in New Mexico [3]. They have found that the technology is\nhighly efficient and safe for servers. In their tests, Intel tested two racks\nof identical servers \u2013 one using traditional air cooling and the other immersed\nin a Green Revolution enclosure. Over the course of a year, the submerged\nservers had a partial Power Usage Effectiveness (PUE) of 1.02 to 1.03, equaling\nsome of the lowest efficiency ratings reported using that metric. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The 3M\nCompany is also actively engaged in immersion cooling technology and has developed\na passive two-phase immersion cooling system for servers. Figure 3 illustrates\nthe concept of the immersion cooling system developed by 3M. In a specially\ndesigned server rack, servers are inserted vertically in the rack. The servers\nare immersed in 3M\u2019s Novec engineered fluid, a non-conductive chemical with a\nlow boiling point. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The elevated\ntemperature of electronic components on the sever boards will cause the Novec\nengineered fluid to boil. The evaporation of the fluid will remove a large\namount of heat from the heated components with small temperature difference.\nThe evaporated fluid travels to the upper portion of the server rack, where it\ncondenses to liquid on the surface of the heat exchanger cooled by the cold water.\nThe condensed liquid flows back to the rack bath, driven by the force of\ngravity. In 3M\u2019s server rack, the liquid bath is also semi-open to the outside\nenvironment. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Because the cooling method is passive, there is no pump needed in the system. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"604\" height=\"600\" src=\"http:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-3-18.jpg\" alt=\"\" class=\"wp-image-9944\" srcset=\"https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-3-18.jpg 604w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-3-18-150x150.jpg 150w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-3-18-300x298.jpg 300w\" sizes=\"auto, (max-width: 604px) 100vw, 604px\" \/><figcaption>Figure 3. Passive Two-phase Immersion Cooling System from 3M. [2]<\/figcaption><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">By\nutilizing the large latent heat of Novec engineered fluid during evaporation and\ncondensation, the coolant can remove heat from servers and dissipate it to\nwater heat exchanger with small a temperature gradient. To enhance the boiling\non the component surfaces, 3M invented special coating for electronic chips\ninside the liquid bath. The boiling enhancement coating (BEC) is a 100 mm thick porous metallic material. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The application of the BEC is illustrated in Figure 4. The coating is directly applied to the integrated heat spreader (IHS) of the chip. Tuma [2] claimed that the coating can produce boiling heat transfer coefficients in excess of 100,000 W\/m<sup>2<\/sup>-K, at heat fluxes exceeding 300,000 W\/m<sup>2<\/sup>. <\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"500\" height=\"672\" src=\"http:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-4-18.jpg\" alt=\"\" class=\"wp-image-9945\" srcset=\"https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-4-18.jpg 500w, https:\/\/www.qats.com\/cms\/wp-content\/uploads\/Fig-4-18-223x300.jpg 223w\" sizes=\"auto, (max-width: 500px) 100vw, 500px\" \/><figcaption>Figure 4. Application of boiling enhanced coating (BEC). [2]<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In his paper,\nTuma [2] discussed the economic and environmental merits of the passive two-phase\nimmersion cooling technology for cooling data center equipment. He concluded\nthat liquid immersion cooling can dramatically decrease the power consumption\nfor cooling relative to traditional air-cooling methods. It can also simplify\nfacility construction by reducing floor space requirements, eliminating the\nneed for air cooling infrastructure such as plenum, air economizers, elevated\nceilings etc. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Green\nRevolution Cooling and 3M have demonstrated the feasibility and applicability\nof using immersion cooling technology to cool the servers in data centers. The main\nadvantages of immersion liquid cooling are saving overall cooling energy and\nmaintaining the component temperature low and uniform. However, both immersion\nliquid cooling technologies require specially designed sever racks. Specially\nformulated coolants are needed for both cooling technologies, too, and they are\nnot cheap. For the traditional air-cooled data center, the air is free,\nabundant and easy to deliver. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In both immersion\ncooling technologies, the servers have to be vertically installed inside the server\nrack, which will reduce the date center footprint usage efficiency. Because the\nliquid baths used in immersion cooling are open to the environment, coolant is\ngradually and inevitably lost to the ambient during long term service. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The\nenvironmental impact of the discharge of a large amount of coolant by data\ncenters has to be evaluated, too. The effect of the coolant on the connectors\nand materials used on the PCB is not also very clear. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Immersion liquid cooling is a very promising technology for cooling high-power servers. But, there are still obstacles that need to be overcome before their large scale application is assured. &nbsp;&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>References<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><a href=\"http:\/\/www.grcooling.com\">http:\/\/www.grcooling.com<\/a><\/li><li>Tuma, E. P., \u201c<a href=\"https:\/\/ieeexplore.ieee.org\/document\/5444305\">The Merits of Open Bath Immersion Cooling of Datacom Equipment<\/a>,\u201d 26th IEEE SEMI-THERM Symposium, Santa Clara, California, USA &nbsp;2010.<\/li><li><a href=\"http:\/\/www.datacenterknowledge.com\">http:\/\/www.datacenterknowledge.com<\/a><\/li><\/ol>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p class=\"wp-block-paragraph\"><em>For more information about Advanced Thermal Solutions, Inc. (ATS) thermal management consulting and design services, visit <\/em><a href=\"https:\/\/www.qats.com\/consulting\"><em>https:\/\/www.qats.com\/consulting<\/em><\/a><em>&nbsp;or contact ATS at 781.769.2800 or&nbsp;<\/em><a href=\"mailto:ats-hq@qats.com\"><em>ats-hq@qats.com<\/em><\/a><em>.\ufeff<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A data center is a large infrastructure used to house large quantities of electronic equipment, such as computer servers, telecommunications equipment, and data storage systems, etc. The data center requires non-interrupted power, communication and internet access to all equipment inside, &hellip; <a href=\"https:\/\/www.qats.com\/cms\/2019\/03\/26\/immersion-liquid-cooling-of-servers-in-data-centers\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1001004,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[37,687],"tags":[914,923,107,922],"class_list":["post-9940","post","type-post","status-publish","format-standard","hentry","category-liquid-cooling","category-servers","tag-data-centers","tag-immersion-cooling","tag-liquid-cooling-2","tag-servers"],"_links":{"self":[{"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/posts\/9940","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/users\/1001004"}],"replies":[{"embeddable":true,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/comments?post=9940"}],"version-history":[{"count":2,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/posts\/9940\/revisions"}],"predecessor-version":[{"id":9948,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/posts\/9940\/revisions\/9948"}],"wp:attachment":[{"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/media?parent=9940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/categories?post=9940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.qats.com\/cms\/wp-json\/wp\/v2\/tags?post=9940"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}