

Cray CS-Storm Delivers High-Performance Computing In Million-Dollar Package - marklittlewood
http://techcrunch.com/2014/08/26/geek-dreams-cray-cs-storm-delivers-high-performance-computing-in-million-dollar-package/

======
ajdecon
The Cray brochure for this system has more information, including detailed
specs: [http://www.cray.com/Assets/PDF/products/cs/CrayCS-
Storm.pdf](http://www.cray.com/Assets/PDF/products/cs/CrayCS-Storm.pdf)

A few fun points from the brochure:

\- The internal network ("interconnect") is Infiniband in either a torus or
fat tree topology

\- Up to 512 GB RAM per node

\- Power usage is up to 63 kW per rack

\- You have the option of including liquid-cooled rear door heat exchangers,
if your datacenter air cooling isn't up to the task ;-)

\- Compute nodes can have disks installed, but the system supports diskless
booting

\- Each compute node weighs 88 lbs

One thing to remember about these machines is that the internal high-
bandwidth, low-latency network is a _big_ part of why they get such good
performance. Most applications that run on these machines are long-running
computations consisting of many processes spread across the entire machine,
with lots of inter-process communication throughout the lifetime of the job.
Think weather prediction, simulations of fluid dynamics, or complex mechanical
systems.

(I work on similar systems to these every day. Huge amounts of fun, but quite
different needs than your typical web service, or even most Hadoop-ish data
analysis problems.)

------
laxatives
In what cases would this be better than AWS/other IaaS? Seems like a steep
investment. Do labs use stuff like this?

~~~
jff
Real scientific computing needs high-bandwidth low-latency networks.

~~~
collyw
Go on then, let me know what "real scientific computing" is.

I work with bioinformaticians, which is an area with lots of research going
on. I don't see much that is complex there. Split up your jobs and send them
to a cluster. I take it that's not "real scientific computing" though?

~~~
sp332
From the article: _planet-wide weather forecasting, seismic measuring for oil
exploration, financial analysis or crash simulations._ You can't split up a
physical simulation.

~~~
jff
You actually can split up a physical simulation. Imagine you're trying to
model heat diffusion through a piece of metal. You split the chunk of metal up
into many small pieces and model temperature in each one. Each node in your
cluster is simulating the behavior of some portion of the model; at each
timestep, nodes simulating chunks that border each other must swap the state
of their "edges", sharing the outer edges of their own state with their
neighbors.

Edit: unless you meant that you can't split up a physical simulation on
loosely-coupled nodes and still get good performance. Then, well, yeah.

~~~
sp332
Well I guess the simulation would have to be broken up to run on multiple GPUs
at once anyway. But even if you had a good interconnect, the GPUs in this box
(K40) have almost 2x as many processors and 3x as much RAM. (Edit: compared to
AWS GPU instances.) If you're doing a 3D sim, each node needs to communicate
with 26 others, so just having 1/3 as many nodes is going to help a lot.

------
Aardwolf
But does it come in a beautiful form factor with bench seats?

------
m_mueller
I'm a bit confused, is the described 1PFlop system really only one million
dollar, including cooling and power? That would be amazingly good value. The
single digit PFlop systems I know of have cost two to four orders of magnitude
more than this, even when powered by GPUs.

~~~
leephillips
The computers cost 10 billion dollars? Who buys them? That would be above the
noise floor of even the NSA's budget.

~~~
m_mueller
Sorry, I meant one to three orders of magnitude. The most expensive computer I
know of is the Japanese K computer at ~ 1.25 billion and 10 something
Petaflops. This was mostly politically motivated (Japanese chips) and too
costly for what it does IMO. A typical new supercomputer in the Top 500 today
is in the order of 100 million.

------
marklittlewood
So going to get one as my new laptop. Might have to invest in a wheelbarrow
though.

