

Ask HN: what problems in technical and scientific computing are worth solving? - mynegation

TL;DR - if you work with high-performance computing what problems would you like to be solved in that area?<p>I work in finance and while there are things that demand a lot of expertise in high-performance computing, low-latency computing and big data and are technologically challenging and very interesting, I sometimes feel that there could be a better application for my skills.<p>My question is to HNers who work in fields that need a lot of computing horsepower: bioinfo, pharma, aerospace, geological survey, high energy physics, actually any kind of industry that does a lot of analytics.<p>What, in your opinion, are challenges in the area of high-performance computing, that - if solved - would make your life much easier: be it algorithms, infrastructure, tools, or workflows?
======
paulsutter
Generally speaking, high performance computing challenges aren't valuable. And
I say this as a guy whose last startup makes 50 million decisions a second and
processes 10 petabytes a day (Quantcast).

Making the technology scale was the -fun- part. The company gets paid for
solving a specific business problem. A business problem that costs the world
$600B a year (advertising).

It's hard to start with a technical ability and find a startup opportunity.
That is how we decided to start Quantcast - we had a spreadsheet of business
ideas that might leverage our technical backgrounds. But it's a difficult
path.

It's easier to start with basic consumer or business needs and design a
solution generally. The best business ideas you come up with are not likely to
require the cool/fun high performance skills you (we) have.

Murphy's law or something.

~~~
strlen
> It's hard to start with a technical ability and find a startup opportunity.
> That is how we decided to start Quantcast - we had a spreadsheet of business
> ideas that might leverage our technical backgrounds. But it's a difficult
> path.

Really interesting point. I am much in the same boat as you, I am interest in
working on highly technical, mostly systems related challenges.

However, I know that the goal of a company is to generate return for their
shareholder. At the mean time, startup is a lot of hard, thankless, and grungy
work (that makes tooling around with autoconf seem fun by comparison): unless
you're genuinely passionate about the problem(s) you're solving, it isn't
worth it.

So until a deeply technical idea strikes me, I'll hold off on taking that
jump.

~~~
paulsutter
Here's one that came up in another thread: rewrite the core engine of MongoDB,
yet keep all the existing API's and libraries.

You could call it BongoDB. B for better. You can capture all their market
share based mostly on technical prowess.

Some of the problems are documented here:
<http://news.ycombinator.com/item?id=3837772>

~~~
bobak
Here is one example: <http://www.infoq.com/news/2011/12/mongograph-qa>

------
ssylee
Paul Graham wraps it up pretty well in PyCon 2012 a few weeks ago:
<http://www.youtube.com/watch?v=R9ITLdmfdLI>

