
Frighteningly Ambitious Startup Ideas (2012) - sillysaurusx
http://www.paulgraham.com/ambitious.html
======
sillysaurusx
Although it's cliche to submit old pg essays, "a decade or so" is a nice time
to revisit this list. The last big discussion seems to be 3 years ago:
[https://news.ycombinator.com/item?id=14381699](https://news.ycombinator.com/item?id=14381699)

Prior to that was the initial submission, 8 years ago:
[https://news.ycombinator.com/item?id=3686840](https://news.ycombinator.com/item?id=3686840)

The reason I'm submitting it was that #6 turns out to be solved, and it was
quite surprising:

 _6\. Bring Back Moore 's Law

...

It would be great if a startup could give us something of the old Moore's Law
back, by writing software that could make a large number of CPUs look to the
developer like one very fast CPU. There are several ways to approach this
problem. The most ambitious is to try to do it automatically: to write a
compiler that will parallelize our code for us. There's a name for this
compiler, the sufficiently smart compiler, and it is a byword for
impossibility. But is it really impossible? Is there no configuration of the
bits in memory of a present day computer that is this compiler? If you really
think so, you should try to prove it, because that would be an interesting
result. And if it's not impossible but simply very hard, it might be worth
trying to write it. The expected value would be high even if the chance of
succeeding was low._

\--

If you read [https://cloud.google.com/tpu/docs/spatial-
partitioning](https://cloud.google.com/tpu/docs/spatial-partitioning) you may
notice it does exactly that. It's the "sufficiently smart compiler" for ML.
You can literally trade TPU cores for memory: twice the cores => you can train
2x larger models. And I was shocked that it seems to be transparent. Other
than twiddling a few options to TPUEstimator, I did zero extra work, yet we
were able to train a full 1024x1024 StyleGAN2 model. (Previously the maximum
was 512x512 before we ran into per-TPU-core memory limits.)

That made me curious which other ideas on this list were quietly (or loudly)
solved since 2012.

