Meta considers liquid to cool its hard drives

A joint effort by immersion cooling firm Iceotope and Meta, the parent company of Facebook, found cooling hard drives with a dielectric liquid is safe and a more effective means of cooling than using fans.

Hyperscalers like Meta deploy thousands of HDDs in their data centers, and while the heat given off on an individual basis is tiny, it adds up, especially since the drives are in constant use and are close together. The drives are stored in server racks that hold nothing but dozens of hard drives and are referred to as a JBOD (Just a Bunch Of Disks).

A JBOD can overheat without cooling, which up to now has been done with fans, but some drives were further away from fans than others, causing uneven cooling.

One thing that helped the experiment was a change in disk-drive manufacturing. Newer, high capacity drives are filled with helium to reduce vibration when spinning, and to keep the helium in, the drives are hermetically sealed. What keeps helium from leaking out also keeps liquid from leaking in.

The test took a standard air-cooled, high-density JBOD storage system that held 72 3.5-inch hard drives in a 40U rack, along with two single socket nodes, two SAS expander cards, a NIC, and a power distribution board, and it was retrofitted for single-phase immersion cooling.

Iceotope doesn’t dunk the gear into tanks full of coolant. Instead, it works at chassis-level to precision deliver a small amount of dielectric fluid to the hot spots first before cascading down in the chassis to remove the heat from other components. Think of it as partial immersion using a fraction of the liquid required by tank immersion.

Iceotope used a fluorinated dielectric fluid with very high boiling point, however it has successfully trialled synthetic hydrocarbon dielectric fluids since this study was conducted. Iceotope says its technology is liquid agnostic, and the company uses a range of dielectric fluid types and suppliers.

Meta found the liquid-cooled system had a 5.4°F (3°C) variance in temperature among the 72 HDDs regardless of their location inside the rack, a more uniform result than with air cooling.

Iceotope said the power draw for air cooling and liquid cooling are fairly similar at low load, but as the load increases, immersion draws less power than air cooling.

Meta is evaluating the benefits of Iceotope’s immersion cooling across its IT infrastructure. Server racks can be retrofitted to support liquid cooling with a minimal a cost and effort, typically representing trivial percentages of the server cost, according to Iceotope, but liquid cooling still requires hermetically sealed drives.