
Want an energy efficient datacenter? Build it underwater - Qworg
http://spectrum.ieee.org/computing/hardware/want-an-energyefficient-data-center-build-it-underwater
======
jefflombardjr
I'm all for exploring new approaches, however:

a) Salt water is highly corrosive. Wouldn't maintenance costs be high?

b) Isn't marine biology highly sensitive to heat pollution?

For high latency services like Amazon Glacier, wouldn't it make sense to host
in a place like Iceland? Really cheap hydrothermic/clean power. Highly
educated local talent pool, and relatively consistent cool temperatures. If
you're maintaining 80F
([http://www.datacenterknowledge.com/archives/2008/10/14/googl...](http://www.datacenterknowledge.com/archives/2008/10/14/google-
raise-your-data-center-temperature/)) the ambient temps outside should provide
ample cooling year round.

For lower latency requirements, wouldn't it be worth it to install efficient
cooling powered by electricity? (preferably renewable) With cheap solar who
really cares about grid power loss and associated inefficiencies?

Essentially, I think it makes sense to get better at solar than to have Steve
Zissou as a server admin.

~~~
longerthoughts
> b) Isn't marine biology highly sensitive to heat pollution?

Impact on the marine environment is briefly mentioned in the last couple of
paragraphs of the article, although they're a little dismissive and don't
provide any evidence supporting their claim of negligible impact.

~~~
nkoren
Nuclear power plants usually dump their tertiary cooling water into the ocean.
That's not a great situation for the local sea life (within a few hundred
meters, IIRC), but doesn't have much significance beyond that. That is several
orders of magnitudes more heat output than could conceivably come off a data
centre. So their dismissiveness is likely warranted.

A bit more concerning is the notion of relocating these data centre pods.
Long-term "reefs" that are relocated around the world could perhaps be a new
and different vector for transmitting invasive organisms. I'm not an expert,
but I'd want to hear a few of them opine as to whether this creates any risks
that ordinary shipping does not.

~~~
refurb
_Nuclear power plants usually dump their tertiary cooling water into the
ocean._

Nuclear plants also use cooling towers to lower the temperature before it
dumped back into a lake/river/ocean. You can do a Google search, but those
towers are hollow on the inside. Water runs down the inside walls and
convection currents move up.

~~~
bnegreve
Yes, but not always. When the body of water is big enough, they don't need
cooling towers.

------
deftnerd
The article mentions that one of the benefits is that they don't have to deal
with building codes being different in different places. It's truly an "off
the shelf" data center that doesn't have to be customized for the laws,
zoning, and codes in that area.

But who is in charge of the seabed around the United States? I imagine that I
can't just drop a big metal container on the ocean floor without informing
someone or asking permission, otherwise there would be a lot of things just
dumped out there.

I also wonder about jurisdiction. If the data center is outside of the
Territorial Zone (12 nautical miles from shore) but is inside of the 200nm
exclusive economic zone (EEZ), which country would jurisdiction fall under?
The country that owns the EEZ? Would the data center fly a flag of its owning
country? Perhaps the country where the cabling comes ashore?

Some interesting legal questions. Perhaps the datacenter would be treated like
a submarine in terms of jurisdiction. It could allow Microsoft to have their
Irish branch open data centers 13 nautical miles offshore of the West Coast to
provide storage that's harder for the US to legally get into.

~~~
tn13
But what about global warming ? Wouldn't rising water levels be a big threat
to such experiments ?

~~~
DuskStar
Let's say climate change gets absolutely horrific, and we see a 1m rise in sea
levels over the course of a pod's 10 year lifespan. So your underwater
datacenter pod, originally deployed 200m below the surface, is now at 201m.
Somehow I don't think this would be an enormous problem.

~~~
longerthoughts
A similar argument could be made for concerns over rising water temperatures
as a result of climate change - what might be considered a dramatic rise to
climate scientists and marine biologists would likely have little impact on a
datacenter.

~~~
titanomachy
Forget climate scientists and marine biologists -- if ocean temperatures rise
3-4 degrees, we will _all_ have much bigger problems than increased datacenter
cooling costs.

~~~
tn13
Honestly US defaulting on its debts would come first. Having the same
disastrous effect.

------
sathackr
I really can't see the benefit of submerging these -- the complexity involved
with keeping out several hundred PSI of sea water just doesn't seem to make
sense to me.

For the cooling aspect, they're already using pumped water and heat
exchangers. What is the benefit of submersion vs just pumping sea water from a
pipe to a land, or barge-based datacenter and using that to cool?

~~~
cr1895
>the complexity involved with keeping out several hundred PSI of sea water
just doesn't seem to make sense to me.

Use immersed electronics.

ROVs and the like have quite complicated onboard control systems. They're
immersed in mineral oil (or something like that). They don't get wet and
hydrostatic pressure is no longer an issue.

~~~
mnw21cam
Spinning rust hard drives have an air-filled (or helium-filled sometimes)
cavity in which the spinning rust spins. If you're building a computer
immersed in oil, you generally need to leave that part outside the oil.

Provides great thermal conductivity though.

------
jpm_sd
This is ridiculous. Data centers contain thousands of racks like the single
rack shown in the demo "pod" and they all require hands-on human maintenance
from time to time. This becomes quite a bit harder when your server rack is in
a sealed steel tube on the seafloor.

Splitting the data center into thousands of "pods" will also greatly increase
the cost of cabling and connectors for power and data distribution (we're
talking $100 per connector here) and the number of cable failures will
skyrocket. Did you know that fish like to chew on submarine cables? Unless you
get steel-armored cables, which are defended against "fishbite"...

~~~
baking
If one server goes down and they are not able to restore it remotely they will
just write it off. If a large number go down due to some common failure, it
might make sense to haul the whole thing up and refurbish it.

~~~
cr0sh
I've heard that this is what many data centers currently do anyhow. That when
a client "moves on" (moves out, dissolves, etc) - the machines are just left
behind, because it costs more to unrack/transport/etc than it does to leave
them in place.

/source: worked for a small cloud computing VPS provider

------
wtbob
> The reason underwater data centers could be built more quickly than land-
> based ones is easy enough to understand. Today, the construction of each
> such installation is unique. The equipment might be the same, but building
> codes, taxes, climate, workforce, electricity supply, and network
> connectivity are different everywhere. And those variables affect how long
> construction takes. We also observe their effects in the performance of our
> facilities, where otherwise identical equipment exhibits different levels of
> reliability depending on where it is located.

In the context of the discussion about cost disease, this is quite remarkable
for demonstrating how regulation and local differences have raised costs to
the point that a large company is seriously considering building data centres
beneath the waves.

------
jedberg
This is great. A long time ago I had this crazy idea that if we built data
centers at the poles, we could solve a lot of problems. Each one would sit
idle 1/2 the year, but you could power it with solar and cool it with the
outside air.

Of course the big problem was latency. It was one I couldn't conceive of a
good solution for.

This is _much_ better. It takes advantage of the same idea but you can park it
right next to a population center and move it around the world fairly quickly
as demand requires.

I love this idea!

~~~
lithos
There was an article posted about one of the Antarctic data centers. And it's
a horrible place to put one. Air too dry, too cold, parts being months away,
and rubber/cardboard losing the properties that it needs. The shear terror
when power goes out which will destroy a server if it's out too long.

~~~
jedberg
Yeah that was one of the many reasons it was a terrible idea. :)

------
kortex
I wonder what the cost differential is (any given rack, not just an underwater
one) for a system which is designed to NEVER be maintained by people over its
service life (or duty). Hard drive goes bad? Board fails? Just push the load
elsewhere and shut off power to it. No idea what sort of attrition rate these
systems have.

Sure, you might be at 50% capacity after 3 years, but that's just like lithium
ion batteries.

~~~
noir_lord
Some of the big cloud providers (Google and I think Facebook) do exactly this
in their DC's, when hardware dies they just leave it dead until enough has
failed in that rack to make it worth swapping out the whole thing.

When you have a 100,000 of anything electrical MTBF will hit you every day.

~~~
lostlogin
I'm guessing that someone has done an equation and shown that to be cost
effective. Does that calculation apply when the cost of building the data
centre and running it are considerably higher due to the location you built
in?

------
dougmany
I see the makings of a great story. In 300 years, treasure hunters search for
a lost data-center that drifted away in a storm.

~~~
jlgaddis
... and sell the treasure (data) they find to advertisers?

------
elihu
Seems like it would be simpler and easier to just build a datacenter near the
ocean and pump seawater to it.

Regardless of how it works, I wonder if it's possible to get significantly
more performance out of chips that are explicitly designed to operate at a
much lower maximum temperature? Let's say you want to build a datacenter
somewhere like the Northern coast of Alaska or Antarctica, and you have year-
round reliable and abundant access to sea water that's barely above freezing,
and suppose that you're willing to order enough CPUs / GPUs / bitcoin mining
ASICs or whatever to justify your supplier to design and manufacture custom
chips to operate in a lower temperature range. Is there some potential boon to
performance or reduced manufacturing cost (due to looser tolerances) that
might make such an effort worthwhile?

~~~
akira2501
> Is there some potential boon to performance or reduced manufacturing cost

Guessing, but in the balance, I would assume no. Power generation costs are
higher and service is less reliable. Staffing is more costly. Transport for
all other resources is also more costly and at a much higher delay than in
many other areas that are traditionally used.

From my own experience with high power (~25kW) FM radio transmitters, some of
which we have at the top of mountains in the middle of ski-resorts: while
cooling costs are reduced throughout the winter, they /aren't/ totally
eliminated, so you still need to size and maintain the equipment for summer
loads anyways. Getting to the site to do work and maintenance is an absolute
chore; replacing the transmitter was a 10 man job where similar rigs in other
locations only required 3.

Although, if it was the summer, after we finished our work, we got to take the
alpine slide down the hill. As to why we put it there in the first place,
signal coverage was excellent, and they already had the power we required
available due to the chair-lifts being nearby. If we could have put it
somewhere else and got the same coverage, we would have.

------
ChuckMcM
Imagine if you built a data center was on a river that was fed by mountain
snow melt, and you created a diversion that took some of that water, ran it
along heat exchangers in your racks and the out through a hydro-power unit and
back into the river on the other side. You might be able to build the first
negative PUE data center in the world.

But you wouldn't tell anyone about it lest you lose your competitive
advantage!

~~~
learc83
The Google datacenter a few miles from my house my house is right next to the
Chattahoochee river. They take water from the treatment plant that was going
to be pumped into the river to use in their heat exchangers. Then they clean
it and pump it into the river.

------
_wmd
This is a fun concept with potentially even more interesting consequences.
Using the ocean as a giant heatsink would result in a water temperature
slightly higher than elsewhere, I wonder if over time it would suffice to
attract the attention of non-local aquatic life, or affect the migratory
habits of the existing species

~~~
Scaevolus
Power plants already vent excess heat into rivers and oceans. Changes in
temperature are usually bad for aquatic life, unfortunately. :-(

[https://en.wikipedia.org/wiki/Thermal_pollution](https://en.wikipedia.org/wiki/Thermal_pollution)

~~~
ryao
The same effect occurs when getting into a hot car. You gasp for air.

------
jsilence
Why dissipate the residual heat into the environment when you could use it for
residential heating and/or heating of green houses. This concept seems to be a
design for waste of resources to me.

~~~
mnw21cam
Thermodynamics. The exhaust temperature of the data centre has to be
significantly higher than the temperature of the thing you want to heat,
especially if that heat is to be transported any appreciable distance.
Otherwise, the heat will not flow in the right direction.

With data centres these days trying to keep chip temperatures as low as
possible, this really makes the temperature gradient very low.

An alternative would be to pump in a little bit more energy and use the
exhaust heat as a source for a heat pump.

~~~
jsilence
With the temperature level of a datacenter you can easily operate a heat pump
with a COP of 5 to 6. Or mix in the CHP to raise temperature level to motor
exhaust levels. Use the electricity for the datacenter. Lot of potential.

------
adamwi
Always fun with people that run with exotic ideas =)

I might be missing some aspects but would it not be much easier to locate the
datacenter on land but close to the ocean (or other large body of water), with
similar principles as used for cooling nuclear reactors? Then you would get
the benefits but would not have the same strict requirements on reliability of
the components.

If you locate it on land you would also have the benefit of using the excess
heat for house heating (if located in cold climate and the temperature
difference to outside is big enough). This is done in a number of cases in
Sweden with excess heat from large industrial plants [1].

[1] Report in Swedish;
[http://www.svenskfjarrvarme.se/Global/Rapporter%20och%20doku...](http://www.svenskfjarrvarme.se/Global/Rapporter%20och%20dokument%20INTE%20Fj%C3%A4rrsyn/Ovriga_rapporter/Energitillforsel_och_Produktion/Spillvarme-
fran_industrin_till_fjarrvarme_2004_5.pdf)

~~~
nxc18
I would be concerned with the cost of real estate at that point. They didn't
talk about it, but I expect they're not paying rent/buying land off shore.
Seaside real estate is expensive afaik.

~~~
cr1895
>They didn't talk about it, but I expect they're not paying rent/buying land
off shore. Seaside real estate is expensive afaik.

I don't know how it'd work for something like a submerged data center, but
blocks of area are indeed sold off by the government bodies that administrate
them to O&G producers. This is also not cheap.

------
Shubley
Wouldn't it be a lot more feasible to just make the datacenter on the shore
and make a liquid coolant loop that cools in the water?

Like the Toronto office towers.

~~~
frozenport
I had the same thought. Maybe temperature?

------
alkonaut
I can see the problem of delivering effective data centers to warm climates,
but apart from the latency aspect, my guess is that Facebook's strategy of
building a traditional mega-data center close to cheap hydro power in a region
with year round cool air is more economic than this.

I hope to see them succeed though, especially if the data centers can be self
sustained with wind or ocean (wave, tidal, ...) power.

------
rascul
Doesn't work. DigitalOcean has been working on this for some time and their
Atlantis datacenter still isn't open.

[https://www.digitalocean.com/company/blog/announcing-
atlanti...](https://www.digitalocean.com/company/blog/announcing-atlantis/)

~~~
sgk284
For those who might miss the date on that blog post, it's an April Fool's
joke.

~~~
Groxx
You say that, but it sounds like a win/win from where I'm sitting:

> _Our investigation was a great success: not only were we able to reduce our
> electricity costs by 35%, but we discovered our high-density SSD storage was
> even more dense at 87atm!_

Time to replicate / H2O-as-a-service it.

------
tomohawk
I wonder if they would power these with DC instead of AC, given the issues of
transmitting AC through underwater cables.

[http://www02.abb.com/global/seitp/seitp161.nsf/0/0eed46e5302...](http://www02.abb.com/global/seitp/seitp161.nsf/0/0eed46e53027b407c125708b0059213e/$file/Electrical+supply+for+offshore+by+use+of+VSC.pdf)

------
baking
The article doesn't mention crush pressure at depths of 200M so I'm thinking
the plan is to replace the air with mineral oil which is non-conductive and
absorbs heat well. But it will also mean the containers will probably not be
bouyant.

------
bhauer
Sadly, the Natick web page [1] hasn't been updated in quite some time and now
has a TLS/SSL problem.

[1]
[https://natick.research.microsoft.com/](https://natick.research.microsoft.com/)

------
jchen11
I would be interested knowing 1) % of total operating cost saved from cooling
savings 2) % of building costs saved from standardized building and simplified
regulations.

Also, it wounld be awesome if we could moor these to offshore wind turbines.

------
n2dasun
Perhaps you could use a non-reactive medium like Green Revolution and use the
seawater as a heat sink for it?

[http://www.grcooling.com/](http://www.grcooling.com/)

------
kristianp
How long before our computation needs require LEO datacenters with huge solar
arrays that always face the sun? Consecutive requests might be sent to
different satellites as they move past. More difficult to dump heat in space.

------
walrus01
This is cool and all, but salt water is basically cancer. If you want to
experience first hand entropy at a rapidly accelerated rate, buy a boat and
moor it in salt water. With all of the associated repair/maintenance costs and
headaches that go with it. Now add electronics and server equipment.

There are ways to use cold water for cooling without submersing things. Many
of the buildings in downtown Toronto (including the 151 Front IX point and
datacenter) are cooled from deep lake loop cooling/heat exchange:
[https://www.google.com/search?q=toronto+deep+lake+cooling&ie...](https://www.google.com/search?q=toronto+deep+lake+cooling&ie=utf-8&oe=utf-8&client=firefox-b)

------
Nostromov
Lots of comments already, but 100% this is complete bullshit. If they manage
to make it a diverse project (branching into other possible fields) and even
if not, this is part of a cat-and-mouse game of placing live electronics for
purposes of underwater warfare, espionage and such.

For example smart mines that will be powered, but indistinguishable from the
signature of these "server farms", then hydrophones and other such technology
for locating and tracking submarines and so on.

"About the Authors" makes clear how no thoughts other than the clown game were
entertained by the article, but aren't we past the times that things get
written about 20-30-50 years down the line. :)

------
frandroid
So hum, how do you prevent your data center from being stolen.

~~~
Florin_Andrei
Same as on land?

~~~
sithadmin
Securing a datacenter from intruders on land doesn't require military-grade
underwater sensor equipment and response techniques.

Securing a datacenter from intruders underwater would.

~~~
smallnamespace
To be fair, it's also a lot harder to penetrate an underwater datacenter so
the risk of attack is likely much lower.

------
andrewguenther
I'm surprised that they're so dismissive of heat pollution. Google's Finland
data center, which is cooled by ocean water, has a holding tank used to cool
the ocean water before pumping it back into the ocean[1] in order to prevent a
similar problem. I would like to see additional research here rather than
baselessly dismissing it.

[1]:
[https://www.youtube.com/watch?v=VChOEvKicQQ](https://www.youtube.com/watch?v=VChOEvKicQQ)

------
drdeadringer
Expiration of 1970s SciFi technology. I'm for it if only to rule it out. I see
no problem in this R&D.

------
ben_says
I'm all for this... I think it might be one of the few ways to protect against
the sun's future EMP.

------
archiides
Would it be feasible to use the heat generated to power a stirling engine?

------
z3t4
I think it would be easier to just use solar and wind plus huge batteries.

------
aaron695
"I think there is a world market for maybe five computers."

Amazon, Microsoft, Google.

Off by two.

------
biggio
Yeah screw the environment

------
ecthiender
Isn't it obvious and common sensical that it is a red flag to operate a data
center underwater? I mean there might be short term benefits, but in the long
term you'll be polluting and damaging the underwater ecosystem.

------
jonbarker
DigitalOcean should have named it "sealab".

------
tlow
What about environmental impact?

------
FreekNortier
We have found a new way to make the oceans even warmer.

------
sandworm101
Energy efficient maybe, but not cost efficient. Computers+salt water = bad
things. Ask anyone trying to maintain anything within sight of an ocean. Salt
gets into everything eventually.

Build on the land. Put a heat exchanger in the ocean. Pump nice friendly
coolant to and fro. Forget the idea of setting up shop literally underwater.

~~~
jedberg
The pod is sealed, it doesn't even have oxygen inside.

~~~
sandworm101
And how does data get into this data center? Where is the power comming from?
All those connections still must go through penetrations. It can be done, i
just question the worth.

~~~
ryao
Penetrations are not necessary. It is possible to use electromagnetic
radiation to transfer both data and power.

~~~
xjwm
EM isn't going to travel very well or very far through water. You would want
something exposed on the surface, and preferably in very close proximity.
Every time you transform power, you're going to incur losses.

------
stephengillie
Edit: Great points! Space would be a terrible location for data center
hosting, not only due to latency and heat radiation issues, but also due to
the difficulty in mitigating solar radiation.

 _What about putting the data center in some kind of low Earth orbit?
Something within the magnetic field, but far enough out to radiate heat? Would
the latency still be a factor if your web server and CDN were hosted "in a
cloud" on numerous communications satellites?_

~~~
tshannon
Someone correct me if I'm wrong, but I believe that radiating heat is actually
really hard in space. It seems counter intuitive, because space is so cold,
but it's mostly a vacuum.

To transfer heat, you need a medium for the heat to transfer through. I
believe some heat can be lost due to radiation, and so they have these big
metal fins as heat sinks to radiate as much heat as possible, and reflective
material on the solar facing side to prevent more heat from being absorbed.

The water, on the other hand, is a fantastic absorber of heat, and I'm
guessing it's easier to drop one in the ocean, rather than launch one to
space.

~~~
noir_lord
> I believe some heat can be lost due to radiation.

In hard space all the heat is lost as radiation (minus and incredibly tiny
loss from hitting the very few particles in space), the dark side can be treat
as a blackbody for a rough approximation.

One of the reasons that the side exposed to the sun is mirrored is to reflect
as much of the radiation falling on that side as possible as the heat _is_ so
hard to get rid off, people think the shiny part is gold but it's actually
usually mylar or something stacked in a layered insulation.

Space is a horrible environment to make things work, metals spontaneous weld
themselves (vacuum welding), radical differentials in temperature depending on
orientation, your electronics are constantly hammered by high energy
radiation, micro impacts by things travelling at velocities that make a sniper
rifle look like a slingshot, it's fascinating as a layman what it takes to
keep stuff working reliably up there at all.

