Dead end. Train researchers to be engineers. Not the other way around.
I wish you luck though. We’ll see in ten years whether programmers are as effective as researchers, or whether researchers are as effective as programmers. In 60 some years of computing, no one has achieved the latter, despite many attempts.
Also, I was surprised that this webpage is basically a waitlist and nothing else. No discussion of technique, no docs, no substance. Just a “you like pytorch? Pytorch rules!” type hype.
I do like pytorch, but I also like knowing one or two substantive points about what the proposal here is. If you want to train a model from your laptop, it’s a matter of applying to TFRC and kicking off a TPU.
The whole ecosystem is in need of massive overhaul. I like the ambition. But I dislike trying to pretend we aren’t programmers. ML is programming, and pretending otherwise will always cause massive, avoidable delays.
It is much harder to do research then it is to do engineering. I do both at work where we have about 15 ML engineers and only 2-3 of them are capable of doing something novel. Maybe you mean something different by “research” but it is very hard to teach someone how to come up with an idea or a novel method publishable in a top conference. Teaching engineering on the other hand usually goes like this: “Here’s how this works. Here are your options, tradeoffs, pitfalls.” After you’ve done it a few times you mastered it.
I forgot who said it: “In engineering, if you don’t know what you’re doing you shouldn’t be doing it. In science, if you know what you’re doing you shouldn’t be doing it.“
Very little of the work done in machine learning is truly novel versus simple extensions of what has been done before. There's probably the same amount of novel work in machine learning and engineering except the ML crowd publishes it while the engineering version stays in closed source code bases. There's also a distinction between doing novel machine learning work and publishing successfully on novel machine learning work.
I agree with you. However very few "ML engineers" do any novel work, let alone "truly novel" work. Typically they just take the existing code and apply it to their specific problems with some tweaking. Sometimes they would try to reproduce papers when no code is available. Very few engineers are capable of the NeurIPS quality research - it does not matter if it get published or stays in a private repo. That's much harder to do and to teach than what 90% of typical ML engineers do. That was my point.
There's also a ton of bad research being published in second rate conferences, but I don't consider that "research". You won't get "simple extensions of what has been done before" published in a top level conference, the acceptance rates have been extremely low recently.
I would not have touched TensorFlow (or AI in general) at all if it weren't for Keras, and I wouldn't be happy with PyTorch if it weren't for PyTorch Lightning.
Easy onboarding for ML tooling is very valuable for the industry as a whole.
As a scientist: The amount I struggled with getting distributed training on a HPC cluster to work vs. how easy it was with Lightning was eye opening. Almost no code change and finally I can run across 20 nodes with 4 V-100 each :). Plus the automatic SLURM checkpoints and restarts <3.
Not just a marginal improvement on that experience but a 10x completely different approach.