The 19 inch rack is one of the oldest standards in computing. ENIAC used 19 inch racks. Open Compute, though, uses wider, and metric, racks. 19 inch rack gear can be mounted in Open Rack with an adapter.
The Open Compute spec says that shelves of IT gear are provided with 12 VDC power. There's power conversion in the base of the rack. Facebook has standards for distribution to the racks at 54VDC, 277VAC 3Ø, and 230 VAC (Eurovolt). Apparently Google wants to add 48VDC, which was the old standard for telephone central offices.
Facebook's choice of 54VDC distribution is strange. Anyone know why they picked that number?
A "48V" rectifier will output 54VDC to the power bus (feeding batteries and systems) since it's battery float voltage. If the rectifier goes offline (loss of AC power) the batteries will power the bus keeping systems online.
I'm curious what the actual difference in practice is between Facebook's 54VDC and Google's 48VDC.
And why -54 VDC? Well typical lead acid chemistry actually yields 2.25 V cells -- 24 of those is 54, so it's another legacy artifact. Modern battery chemistry is different, but so much of the other tech is too. Here's a modern power supply I just found (http://www.tdipower.com/PDF/archive/10A_dcdc_conv.pdf) that supplies between -47 and -54 ... hmm, wonder why?
Thankfully, telecom is fortunately one of the most backwards-compatible industries in the world, which is why the whole rickety edifice continues to work well (and support tons of innovation) even when people use it in ways never intended by its designers.
BTW I'd be disappointed to learn if any reasonable 21st century EE education didn't still include this stuff.
How is that a benefit for anyone other than hardware makers?
My pet issue w/IT infrastructure is the management modules. Finding a server w/a management module that works everytime is nigh impossible. Do google and Facebook design their own or do they somehow just work around their quirks?
All kidding aside, if it's a random rare issue (some bit flipped somewhere), chances are that power cycling it may fix the issue (upon initialization). If it's anything more serious, the issue will likely persist, in which case taking the machine out of production is likely the best immediate course of action.
And then it took them a while to decide that leveraging the open source ecosystem would be better than maintaining those various proprietary elements.
And to your second point, it is interesting that OCP (some of it perhaps influenced by people who had worked at Google and then later worked at Facebook) appears to have minimized some of the advantages Google had, and now they find themselves possibly getting behind (one of the nice things, and bad things, about open projects like this are that you get a lot more resources applied to making things better than one company can muster).
For similar reasons I wonder if ARM will displace x86 if only because there are probably 15 to 20 independent teams of smart people with ARM architecture licenses making better ARM processors, and perhaps 4 such teams at Intel making improvements in the x86 architecture. At some point it seems "open" seems to eventually overwhelms "walled garden." Although I'm totally open to counter examples where that hasn't been the case.
Rackspace has largely abandoned the project after initially opening it up, and public clouds based on Openstack seem to be getting shut down left and right (I could go on a long rant about why that is the case, since I am an Openstack contributor :)
The reason, apparently is because Google can no longer sit comfortably behind the scene to watch others play the catch up. Previously Google do not care, because they dont want to share internal Cloud with you. Now they need the profit from Cloud. Google needs to show that it is still the leader. Previously, Google does not need to show anything and all believe they are the leader. After missing the Cloud movement in the past years, Google's reputation has been tainted, and they have to show hard proofs.
For example, when choosing cloud providers, AWS is still the clear no1. People probably still believe that GCP is better than AWS. But GCP does not have the proof. However, if Google somehow can catch up and beat AWS in a decisive manner overnight, then Google still does not need to share anything; and no one will question that Google is the leader. Apparently, Google is not able to do that. It might be that Google is not focused enough. But Google cannot let people have the other impression "Google's technologies might not be as good as we thought" grow in people's mind.
Can you imagine what would happen if Google started the Cloud thingy at the same time as AWS? Then Google certainly do not need to share a damn thing with outside world...
To address your last 'if' scenario - notice that Amazon, as a large online retailer, had a compelling reason to offer up the use of the infrastructure they had to build anyway, which was over-provisioned for most of the year. I don't have numbers, but I would be surprised if Google has anything close to the kind of seasonality Amazon experiences.
Once it became clear that Amazon was onto something with this whole utility computing/cloud thing, the rest had to play catch up, and continue to do so to this day.
And last month Spotify did the same thing.
The price differences between EC2 and GCE, especially with Custom VMs, is massive. Up to 40% according to this:
I've run the numbers, AWS isn't cheaper AT ALL unless you're talking bursting workloads that run for less than a month, or a company that only needs one or two servers but still needs the reliability of a larger environment. Anything outside of that is cheaper to do on-prem 9 times out of 10.
Either he's got his head in the sand, has bought into the "cloud is cheaper hype", or he has other justifications he fails to list.
We recently did the math on some cloud database solutions vs. warehousing data ourselves and using Glacier as a last-resort backup. Turns out it's orders of magnitude cheaper to DIY and would remain so unless we needed -- as you say -- a whole lot of burstable compute. In that case we could always upload (or ship) all the data to the cloud if we wanted.
Cloud is cheaper for some work loads and use cases but not others. You have to analyze your own problem and see what works best.
What cloud vendors have managed to do is put a ton of marketing hype out there to the point that cloud has captured the entire discourse. Today if it's not cloud it's not cloud and that means it's not cloud, because cloud. I've heard many stories recently of companies spending 2-4X what they currently spend on IT to put it in the cloud in order to save money. It's hilarious.
Of course our app is probably different from your app. For us we are doing real-time network testing and diagnostics of a bunch of distributed networks and dumping the data into a series of tables for analysis and diagnostic use. We can also use that data to improve our software in the long term.
A lot of data is generated, but if we lose a little bit it won't kill us. The analytics we want to do right now are fairly consistent in terms of load.
For this work load the cheapest option by far is to buy a RAID array and warehouse it locally. It can also be backed up very cheaply in Glacier so that if we do lose some we can get it back if we want it.
But of course like I said our app is probably not like yours. My point was that you have to do the analysis for your exact specific problem to decide what approach -- cloud or not -- is the best fit.
And then, not directly, roughly saying that scaling up on cheap commodity type hardware is only possible because of the efforts of Amazon/Google/Facebook.
He doesn't directly compare AWS versus, say Rackspace.
I ask every time, and this project is amazing, but, it feels just for the big guys!
When Facebook started OCP it seemed like an awkward initiative, after all, who would manufacture OCP-compatible stuff, when you can't find anything OCP-compatible at all? (It needs custom racks, for shame! Madness!) But slowly, it turned out, that there are multiple groups working on custom designs, trying to get out of the world of half-assed firmwares, crazy sales calls and useless support vectors associated with traditional vendors.
So initially it only made sense if you had at least a few thousand servers and a team to work on that extra few percentages of efficiency. Nowadays, it seems a much more diverse club. And especially after the network stuff got opened up - the timing with SDN was right anyhow - they seem doomed to succeed.
You can still get OCP hardware in standard 19" form factors for these environments, but I don't personally see much point in it. You're seriously limiting your vendor and hardware selections, and if you aren't seeing the purported power efficiency gains then it's not worthwhile. The allure of eliminating the pain of classic vendors is still there, but there are paths through the Dell / HP / Supermicro maze that are tolerable without burning it down and going to OCP.
There are other challenges with OCP's overall design in standard facilities beyond the server cabinet level too. Their (optional) battery cabinet design -- where you save on the facility cost by omitting the UPS bank in favor of battery cabinets for every triplet rack -- is explicitly forbidden at basically every top tier datacenter provider that I'm aware of. No batteries allowed on the floor because of the fire suppression system. You can definitely get a provider to build you a room where they'll let you have batteries on the floor, but that is a multi-megawatt sort of problem, not a 50kw sort of problem.
But that means you need to basically run your own DC operations. Cables, batteries, monitoring, hot spares, and good luck finding a DC that even allows you to speak about putting fire hazardous stuff "there".
Yet it can be done. Money talks, hence the seemingly amazing success of OCP out of nothing.
... ah I should have read your last paragraph, before writing anything :)