
Deep Learning with PyTorch - saranshk
https://pytorch.org/deep-learning-with-pytorch
======
iamspoilt
To those who don't want to submit their email:
[https://pytorch.org/assets/deep-learning/Deep-Learning-
with-...](https://pytorch.org/assets/deep-learning/Deep-Learning-with-
PyTorch.pdf)

~~~
fermenflo
Damn, I should read the comments first

------
Traktor
Shouldn't [0,2,1] = 5 on page 17 and not 2? Or do i misunderstand how pytorch
handle tensors?

~~~
dfan
You are correct.

------
mroche
For those in the ML area, what are people’s opinions on PyTorch and its use in
comparison to its competition? I don’t have any experience with PyTorch or ML
tech besides having to package and provide PyTorch containers for our
university’s HPC cluster and running the helloworld.py against it for
validation.

~~~
sseveran
So we have been a TensorFlow shop since well before 1.0. We are keeping tabs
on PyTorch with an eye on moving in that direction as certain things get
filled, notably serving and something that is really comparable to tf.data.

The fact that our team in particular is looking at moving is probably a
disaster for TF since a number of the criticisms typically leveled at it we
didn't see as issues. We have a couple of really strong people with haskell
backgrounds so static graphs and laziness we didn't find to be problematic.

However the embrace of Keras in 2.0 has left us dumbfounded. On one hand
having a consistent layer interface is nice. On the other hand having a base
class for the loss function that is not sufficiently general, the fact that
all non-toy models we build seem to need model subclassing and a custom
training loop with GradientTape and the number of issues we ran into while
trying to port a couple of models has led me to conclude that the release was
not ready. So while we like the tools around the model (tf.data, tensorboard,
serving, tfx, etc...) building actual models I think has gotten worse.

Now my opinions on PyTorch are not from shipping production models but mostly
porting to TF and keeping tabs on what they are doing. PyTorch also makes it
easy to define reusable units. It does not try and expose a higher level
interface that requires a significant investment in learning to express
complex or unusual models. It seems a bit less opinionated on what the user
should do.

A couple of other notes, PyTorch is being used inside Google for research I
think. They have now written several papers (including one with Jeff Dean as
5th author) that have had their code released in PyTorch. PyTorch I think (it
might already have) will end up with better governance but I would be
interested in others opinions. They have at least one person listed under the
project maintainers who does not work for FaceBook. A reason for adopting
PyTorch may be that one company just does not decide to radically change the
project to fit their view of the world.

This last bit is purely conjecture. PyTorch I think has already won over TF
and it is going to take a couple of years for it to play out. If I had to bet
today I would bet that PyTorch will become the dominate framework for both
research and production. Of course something could happen to derail that but
if things continue on their current trajectories I think its inevitable.

~~~
acollins1331
I echo this. While 2.0 initially made me happy building some models from
scratch, as soon as I needed to write my own optimizers, losses, class
weights, etc. it became a nightmare. Also not to mention the data pipeline for
large imagesets required you to serialize the data first into tfrecords.
Please just stream AND shuffle images in a folder tensorflow.

------
misslinda
"Essential Excerpts"

------
manojlds
It's not the full book and more of a preview or as it calls it "essential
excerpts". Can someone add this detail to the title?

~~~
nabla9
It has 127 pages. The final book will contain 400 pages (estimated).

------
Apreche
The form isn't working. Not being able to make a web form work doesn't inspire
a lot of confidence in the neural network part. lol.

~~~
iamspoilt
You can get it from here: [https://pytorch.org/assets/deep-learning/Deep-
Learning-with-...](https://pytorch.org/assets/deep-learning/Deep-Learning-
with-PyTorch.pdf)

------
Mirioron
What kind of knowledge should you have before getting into this book? I've
been meaning to try to learn ML and have been looking at some university
courses that have all the material available.

~~~
madenine
Goodfellow’s book on deep learning[0] is a good starter - the first chapters
give a solid overview of ML theory as well. Elements of Statistical Learning
is another.

[0][http://www.deeplearningbook.org/](http://www.deeplearningbook.org/)

~~~
earthnail
I don’t think it is a good book. From a didactic point of view, I actually
found it one of the worst resources out there. The math intro at the beginning
is too superficial - either you know it and skip the chapter, or you need
another resource to learn. The rest of the first part is okay, but parts 2 and
3 are really not very helpful to someone who doesn’t already understand it.

I strongly recommend fast.ai instead. Although often looked at as the resource
for people who can’t deal with the math, I actually found it to be extremely
good at explaining the math. Compare, for example, the deep learning book’s
explanations on various gradient descent methods with Jemery Howard’s
explanation - in the book it looks very complex, whereas in the course it’s
actually really intuitive. And Jeremy doesn’t gloss over things, he actually
implements the various gradient descent methods in Excel (!).

~~~
Mirioron
I finally started looking into fast.ai and the setup seems to be _a lot_ of
hassle. It doesn't help that the course dismissively just says "just buy
server time even if you have a GPU for this."

~~~
Breza
I feel the same way! My workstation can easily handle the fast.ai projects but
they insist that people use cloud servers.

------
hn2017
Anyone have a ballpark estimate of Tensorflow models in production vs PyTorch?
Is it something like 90/10?

~~~
Jwarder
Looks like 3:1 ish.

[https://thegradient.pub/state-of-ml-
frameworks-2019-pytorch-...](https://thegradient.pub/state-of-ml-
frameworks-2019-pytorch-dominates-research-tensorflow-dominates-industry/)

~~~
dahart
Which estimate are you looking at? The main thing I see is:

“PyTorch and TensorFlow for Production

Although PyTorch is now dominant in research, a quick glance at industry shows
that TensorFlow is still the dominant framework. For example, based on data
from 2018 to 2019, TensorFlow had 1541 new job listings vs. 1437 job listings
for PyTorch on public job boards, 3230 new TensorFlow Medium articles vs. 1200
PyTorch, 13.7k new GitHub stars for TensorFlow vs 7.2k for PyTorch, etc.”

That suggests 1:1 for jobs, 2:1 for github stars and 3:1 for articles on
Medium. Really hard to say if any of those reflect uses in production in any
meaningful way, but if so, I’d suggest that the jobs and/or github stars might
be more useful than articles on Medium.

~~~
Jwarder
The jobs numbers suggests where the trend is going. The github and article
counts seem more useful to indicate the current state. When looking at those
numbers I'd rather pick the more conservative number.

------
p1esk
Is there a full table of content somewhere?

~~~
flakiness
It seems like based on this: [https://www.manning.com/books/deep-learning-
with-pytorch#toc](https://www.manning.com/books/deep-learning-with-
pytorch#toc)

~~~
laichzeit0
I only see examples of classification. Regression? Time series?

