
3M Novec 7100 Fluid Used in Largest Two-Phase Immersion Cooling Project (2015) - deegles
http://investors.3m.com/news/press-release-details/2015/3M-Novec-7100-Fluid-Used-in-Worlds-Largest-Two-Phase-Immersion-Cooling-Project/default.aspx
======
arcaster
I did some research with Novec back in high school, really interesting stuff.
It was a hassle to get my hands on, but after going through a chemical
supplier that actually flew out a staff chemist to ensure my high school had
proper facilities to handle the vapors from Novec I purchased 1 gallon haha.
Here's a video summary of my project, essentially just comparing efficiency
differences between immersion cooling and air cooling with server hardware.
[https://vimeo.com/37127378](https://vimeo.com/37127378) (Yes, I know the
"efficiency savings" numbers at the end are incorrect)

One of the coolest attributes of Novec 7000 (now outdated haha) is that it's
viscosity is so low near it's boiling point that it can actually slip through
teflon gaskets and even dissolve thermal paste between a CPU and heatsink.

Not sure if Zack Glander is still running Vapor Phaze, but back in 2011 his
startup was one of the only private companies I knew of developing hardware to
use this cooling tech at scale. He had some cool demos at CES for a few years.

------
virtuallynathan
Wow, 250kW per rack is pretty much unheard-of. The highest density stuff I
know of is 40-50kW per rack @ Ebay. They use 480v 3ph PDUs!

~~~
semi-extrinsic
Boiling, like they do here, can remove a ridiculous amount of heat. Witness
your ordinary electric kettle dissipate 2kW into a single 20 sq.in. surface,
which it can do indefinitely without damage if you just hold the "on" button
down.

------
agnuacc
I have wanted to get my hands on this stuff as a hobbyist computer Builder
since about 2015 it's a shame because I live not even a few miles away from 3M
Oakdale where the fellow Phil Tuma makes videos of delidded CPUs cranking away
submerged in this stuff.

~~~
arcaster
A few years back I was able to purchase through a commercial chemical
distributor. It's hard to find in 1 gallon glass containers which on their own
sell for $300 each. I know hobbyists on bitcoin forums have managed to
purchase without an academic institution to back them.

------
wtallis
3M and Gigabyte had a demo of an immersion cooled server this year at CES:
[http://www.anandtech.com/show/11046/gigabyte-server-shows-
tw...](http://www.anandtech.com/show/11046/gigabyte-server-shows-two-phase-
immersion-liquid-cooling-on-a-2u-gpu-g250s88-using-3m-novec)

The 3M guy at the booth said they were partnering with Gigabyte because the
latter can build servers with the requisite non-standard high-density form
factors necessary to not have excess empty space being filled by the expensive
cooling fluid.

~~~
arcaster
So cool to see the "expanded copper" heat spreaders on these servers. In many
cases those heat spreaders are fused to the CPU with indium solder. Cool
stuff.

------
eltoozero
So nobody here is surprised this massive cooling infrastructure is getting
built for custom ASICs for blockchain computation?

Allied Control was recently acquired by BitFury Group, a leading Bitcoin
Blockchain infrastructure provider and transaction processing company, which
builds its own fully custom Application Specific Integrated Circuits (ASICs),
PCBs, servers and data centers.

Novec fluid [is utilized] in the Georgia data center to deploy 28 nm and 16 nm
ASICs more efficiently.

“250 KW per rack"

I sort of realize this isn't being done on people's basement GPU rigs these
days but with the drama with the ASIC vendors a bit ago I thought the bitcoin
hardware market crashed.

I guess only on the hobbyist end.

------
anon363764
Similar approach: [http://www.grcooling.com](http://www.grcooling.com)

------
pdq
2015.

Are there updates on the project results?

~~~
gcb0
everyone realized its impractical to drain a rack to replace hdd and psu every
day and moved on.

~~~
semi-extrinsic
You can pull SATA cables 1m, and PSU cables much longer. Upgrade to SAS, and
you can do 8m cable runs.

So why not build your system such that only the parts that a) produce lots of
heat, and b) break rarely, are immersed? I.e. just the CPUs, motherboard,
memory, GPUs and ASICs? Sounds like the logical thing to do.

~~~
gjem97
This is what I came here to ask. Seems a bit silly to fill the whole tower
case with cooling fluid. My guess is that the incremental benefit you get from
the redesign doesn't outweigh the costs. It may at very large scale, and I
would guess that Amazon, Facebook and Google have all experimented with it.

