

Supercomputers surprisingly link DNA crosses to cancer - Galeno
http://www.deepstuff.org/supercomputers-surprisingly-link-dna-crosses-to-cancer/

======
TTPrograms
I wonder if there are scientists trying to understand why certain genes are
correlated with cancer thinking that it has something to do with proteins
being made, but in the end it's the result of those genes affecting the
physical stability of the DNA strand. Pretty crazy.

~~~
i000
Certainly there are such scientists. There is a very strong incentive for
researchers to be novel and make discoveries "out of the box" \- how much of
these discoveries will be published, and will stand the test of time is a very
different thing. Almost every single cancer case we see in our clinical
sequencing effort has one or more mutations in a limited set of the "usual
suspects" <500 protein coding genes. We are at a stage, where it is more
difficult to find a new oncogene or tumor suppressor gene than to suggest an
"out of the box" mechanism based on pre-clinical data.

~~~
acbart
The pursuit of novelty in research drives me crazy. You don't get published as
easily with replications and tiny iterative developments. I wish they'd
incentivize those kinds of research...

~~~
danieltillett
We certainly undervalue replication (or refutation) in the pursuit of novelty.

What is more of a problem is that scientists are being pushed by the system to
get their results out as fast as possible so they are not doing the hard work
of carefully checking their own work before publication. The incentive is to
publish if an experiment works once or twice and hand-wave away the other
dozen times it didn’t. Combine that with the enormous pressure PostDocs and
PhD students are under to generate results it is amazing anything of value
comes out of the lab these days.

~~~
Sven7
The number of people capable of producing good science, is much less than the
number of people in science. The pressures emerge because of it.

~~~
danieltillett
Even the people capable of producing good science are pushed into bad
situations. The basic problem is we have more good scientists than the funding
to support them - when the success rate of grants is less than 15% then the
pressure gets extreme and also sorts of bad behaviour results.

As a bit of an aside grant reviews were apparently brought in initially to
help scientists avoid working on bad areas. The idea was that everyone would
get funded who put up a reasonable grant and they just wanted to help screen
out the bad ideas so people didn’t waste their time.

We have since twisted this into the current system where we try to determine
which of the top few percent of grants are the best. Of course the whole
ranking process is so noisy that it is little better than chance for anyone in
the top 25%.

A much better system would be to rank the grants as worth funding or not and
then put all those worth funding into a lottery to choose who gets funded. At
least this way even the losers would know they were on the right track and
they were just unlucky. It would save an enormous amount of grantmanship and
other activities that are the antithesis of everything science should be
about.

------
jacquesm
I'd love to work on the computational side of problems like this.

~~~
kaybe
Subscribe to the right mailing lists; once in a while there are job postings
for things like this.

(e.g. this one: [http://listserv.uni-heidelberg.de/science-jobs-
de/sjd-e.html](http://listserv.uni-heidelberg.de/science-jobs-de/sjd-e.html) ,
and subscribe for IT and biology/medicine - I'm sure there are tons more of
these lists out there.)

------
kevin_thibedeau
This would really work better implemented in an FPGA. You could construct a
circuit that exhaustively searches for all repeats within a maximum distance
rather than just checking a relatively small number of random sequences.

------
dnautics
Hm. Surprise is not really my response, but "huh. Neat. Makes sense."

------
laurencerowe
I wish people would stop using the term 'supercomputer' for large scale batch
processing on clusters. This seems like just the sort of problem that could be
run more cheaply using EC2 spot instances.

~~~
mitchty
I work for a supercomputer company. There is almost no way you'll get anywhere
near the performance you need on EC2.

Supercomputers today are built to transfer data between nodes using locality
of the node as well as its physical location. Ok EC2 could do the same thing
or similar sure, but, it won't have fibre optic links that allow direct DMA of
memory from one node to another.

The interconnects and the overall use of a supercomputer is drastically
different than what you have on EC2. Additionally, your process is just that,
another process running on a bare metal system. No hypervisor to get in the
way.

Keep in mind that some compiled code can care about if an avx instruction is
used or not. A tiny detail like that means things like a 5% reduction in cpu
cost. When your job/process overall takes months to run, this is a significant
reduction.

Supercomputers exist because they're more economical than regular hardware. No
really, it is true. These things can generate terabytes of data a second and
shunt it around efficiently. Until you can demonstrate the same on EC2 you
might want to read up on the architectures of modern supercomputers.

As a fun note, today cpu power is mostly a gimme, now most simulations care
more about memory. For the memory you need you effectively get the cpu for
free.

~~~
lmeyerov
You can roughly mimic the Oakridge Titan cluster (#2 in the world) by using
AWS: 4 GPUs / node, 10 GE across nodes, and you can spin up a _lot_ of them.
Best of all, you don't need $60 mil.

This is what we're pushing towards at graphistry ;-)

Edit: link:
[https://en.wikipedia.org/wiki/Titan_(supercomputer)](https://en.wikipedia.org/wiki/Titan_\(supercomputer\))

~~~
mitchty
Mimic sure, but it won't be quite the same. I won't deny you can't do a one
off job possibly cheaper on aws. The thing is, these simulations run for weeks
or months on end and these clusters are run and setup to run at 100% at all
times.

They really are efficient power wise, which is the true test for things like
this. I'm hitting towards nda things but needless to say keep an eye out for
gpu technology in supercomputers. As well as what that top 500 chinese
supercomputer has.

Note, its at the top mostly due to its accelerator cards. Most simulations
don't need those.

I know 60 mill seems like a lot but when it runs for 5 years it really isn't
that expensive. For a one off job sure go with aws, it'll take longer, but i'm
not kidding when you get down to the detail of the cpu architecture and keep
track of things like how cache misses affect a program.

It really is night and day difference in how things run. The avx comment is
not a joke, reducing even cpu cost by 5% over months is a huge deal. These are
just a few of the differences in how supercomputers operate. It is far less to
do with the cpu and memory and more how they communicate amongst themselves.
One node may compute a value, but another adjacent node then takes on that
output to calculate more. This is the key differentiator, and if you aren't
compiling fortran (not joking), its likely its not a very great simulation.

If you'd like to know more, i'd recommend getting a job at a supercomputer
company. They've solved a lot of the overall cluster problems aws has many
years ago. And it is really fun stuff where details matter. Something I can't
say about most aws deployments. PS: I'm working on something that will make
Titan look like a kid. :) Every 2-3 years this stuff gets pushed aside its
fun.

~~~
lmeyerov
We handwrite PTX ;-)

Cost-wise, public cloud destroys traditional supercomputers, and the gap will
just keep increasing. However, the economics enabling it, e.g., sufficiently
non-interfering multitenancy, are only now reaching the needs of HPC.
Likewise, only recently did public clouds embrace non-wimpy hardware: SSDs,
multiGPU, starting to have FPGAs, etc. Exposed interconnects are 10 gige, so
not the best, but like you said, scale requires designing for minimizing
communication to beginwith.

RE:long jobs, we focus on interactive exploration (< 100ms). Spark etc. give
up latency to ensure resiliency on long jobs.

