
Microsoft open sources its next-gen cloud hardware design - andytolt
https://techcrunch.com/2016/10/31/microsoft-open-sources-its-next-gen-cloud-hardware-design/
======
jabl
Seems we've come full circle, eh?

Back in the original dot-com boom ~15 years ago 1U pizza boxes were the
workhorses. Same for things like HPC clusters.

Then the big-name vendors started introducing various blade server
architectures, where many servers shared power/cooling/BMC (out of band
management), often with things like integrated switches in the back of the
blade chassis. (Of course, many thought the real reason behind blades were
that standard rack servers were rapidly becoming commoditized, so the vendors
needed something with more lock-in..)

And then we had things like "scale-out" designs, with somewhat bare-bones
servers, often in some simple chassis with shared power. And lately (Intel at
least?) there has been talk of "rack-scale" architectures.

But now MS goes against this "trend", going back to basics so to speak. What
gives?

~~~
wmf
One thing that MS can't say yet is that a Skylake-EP server is "rumored" to
have a larger socket and 24 DIMM slots instead of 16 so it needs to be
physically wider.

~~~
walrus01
There's the old telecom standard for 23" racks... Other than facebook I wonder
who is building new hot aisle/cold aisle separated environments for wider-
than-19" standard.

~~~
wmf
I mean the old MS servers are 9.5" wide and the new Skylake servers need to be
wider than that so they're going to 19". Facebook/Google/Rackspace use a 21"
rack but MS and LinkedIn are using 19".

------
IBM
Can anyone tell me why Intel, Cisco, Juniper, etc are part of Open Compute
Project? It doesn't really make sense to me because this project is
effectively designed to commoditize their business. It makes sense for
Facebook, Google, Microsoft, Apple, etc to be in OCP because server and
networking hardware is purely a cost and isn't the part of the value chain
that they profit from.

~~~
kchoudhu
The networking giants at least aren't planning on selling hardware for much
longer: they want to become purveyors of SDN technology that runs on commodity
OCP-style hardware. I am guessing they are part of these initiatives so that
they have some influence over what their code runs on.

~~~
devonkim
Who's going to sell hardware if not for the networking giants though? While
hardware may be a declining business in terms of revenue it doesn't mean that
there's no business either. After all, the big cloud providers will all have
to source hardware from _someone_ even if everyone goes whole hog into the OCP
ecosystem with cheap, large scale vendors based primarily in low cost of
business regions. Who's better equipped to handle economies of scale better
than large companies in our world? Did I miss something about manufacturing
trends being disrupted in the past 15 years?

If anything, these vendors would want some say on the SDN systems so that
their hardware can better integrate with them and offer more compelling value
than their competitors.

~~~
mcpherrinm
Network switches are increasingly being based on silicon from companies like
broadcom with their Trident and Tomahawk line (what Facebook's six pack use),
Intel's Fulcrum chips, Mellanox, or Cavium's Xpliant chips.

An ODM vendor like Quanta makes the equivalent of a "motherboard" for those,
and sticks it in a box.

Then you can get software from a vendor like Cumulus which has all the SDN
goodness.

Vendors like Cisco, Juniper, or Arista sold all of those as one big product.
Whitebox switching is about breaking those up so you can buy them more like
you buy your servers.

------
wilhil
Now, how can I actually use these?

I say it every time I see "company X releases Y" or open compute project...

They are lovely and I want to use it, but, any enquiry to the manufacturers
show that you need a minimum order that is too big for small scale companies
and I don't see many of the larger companies actually using each other's
designs.

So, it is all well and good being open, but, I seriously wonder who benefits
from it?

~~~
discodave
The main beneficiary is Facebook and other hyper-scale operators for whom
datacenter operating costs is not a competitive advantage.

Google and Microsoft are arguably in it to look "open" and/or they are trying
to reduce their hardware R&D costs (like how they use Linux).

AWS are staying out because they consider their hardware proprietary and a
competitive advantage. I don't think Google has made their latest and greatest
stuff open (particularly network infra).

If you don't have the scale to order your own hardware then you should
probably be considering one of the cloud providers (AWS, GCP, Azure).

------
gorodetsky
I wonder what are hardware specs on Google Cloud. I didn't manage to find
anything similar for GCE.

E.g. what's the max speed of internode network connectivity within a region
zone? What's the spec of local-ssd? Could be interesting to know underlying
hardware.

------
mlinksva
Are there differences between the datacenter and consumer device markets that
make something like OCP gaining traction much harder for the latter, besides
the lack of huge buyers (in terms of both quantity and making a difference in
buyer's operations) in the latter? Are there any potential huge buyers in the
latter that could get something like OCP for some type of consumer devices
started?

~~~
bluedino
Smaller buyers want features that get stripped out like HP or Dell's
management tools. They always want easily replaceable hard drives in caddies
you can reach from the front of the rack. Some designs use a 21-inch instead
of 19-inch width rack. 277 volt power connections instead of 120v.

------
tkinom
If someone can put 32 / 64 ARM (Raspberry Pi) in the same 1U Chassis with each
has its own SSD +1,4,8 GB RAM.

Would you want to use that? Why and Why not?

~~~
wmf
Looking at the failures of Calxeda, SeaMicro, Moonshot, Group Hug... no, you
wouldn't want to use that.

------
kylek
Sockets look LGA 3647 (xeon phi) shaped, no?

------
tekism
As someone who just saw (from another HN post) the documentary "Before The
Floods", I'm curious if they take into account energy saving and lower carbon
footprints into the design as well.

~~~
eightysixfour
That is one of the major design goals of every data center, electricity on
that scale is very expensive.

~~~
molecule
And all of that electricity is turned into heat that needs to be dissipated,
so cooling is also very expensive.

