
AI speed test shows clever coders can still beat tech giants like Google, Intel - optimusrex
https://www.theverge.com/2018/5/7/17316010/fast-ai-speed-test-stanford-dawnbench-google-intel
======
jremmons
I feel like this article does not substantiate its core claim, "clever coders
can beat tech giants." The main evidence given by the authors is that because
Google lost to a much smaller company, Fast.ai, on a toy problem (training a
model to 93% accuracy on the CIFAR-10 dataset) that small-time AI researchers
can beat the big guys at their own game. This feels like a straw man argument
and I'm not convinced.

That said, I do think smaller, independent AI research groups can be
__impactful __even if they aren 't going to beat Google in a head-to-head
competition; they just have to find problems the tech giants aren't working on
yet :).

~~~
jph00
Jeremy from fast.ai here. I think it's tough to really substantiate this claim
fully in a mainstream tech publication, without getting into details that are
likely to be rather dull for much of the audience. However, I _do_ think that
the claim is a reasonable one, and there's more details to substantiate it in
this article I wrote: [http://www.fast.ai/2018/04/30/dawnbench-
fastai/](http://www.fast.ai/2018/04/30/dawnbench-fastai/)

Training small-ish datasets quickly is definitely not a toy problem. Most
folks I know using neural nets in practice are using datasets of ~100MB. Some
people do have giant datasets, but it's not the norm. And being able to train
them quickly and cheaply is great for running experiments, as well as making
the field more accessible to people with fewer resources.

If you're only interested in training larger datasets, this competition showed
that the fastest Imagenet training on a single machine, and the fastest on
publicly available infrastructure, also was done by fast.ai. The only better
results were on TPU Pods, which are TPU machine clusters that are not publicly
available. (It wouldn't be terribly hard to show similar results with a
cluster of GPU machines, although it wasn't something that we have spent time
on ourselves as yet.)

I strongly disagree with your claim that small groups can only compete with
tech giants on stuff that tech giants aren't working on. There is a large
amount of empirical evidence both now and throughout history that this isn't
correct, and you have shown nothing to substantiate this claim. (I heard
similar claims when Google first appeared: "a small group of Stanford
researchers can't beat Yahoo.")

~~~
narvind
Well said, Jeremy!

------
niroze
Talent isn't the problem, capital is.

