
PyTorch v0.2.0 released - evc123
https://github.com/pytorch/pytorch/releases/tag/v0.2.0
======
cs702
I LOVE PyTorch for experimenting with _dynamic_ deep neural nets (DNNs) --
that is, DNNs that can have different graphs for different input samples. I
find it much, MUCH easier to create and tinker with dynamic DNNs using PyTorch
than, say, TensorFlow Fold. PyTorch is great for R&D experimentation.

For example, here's how easy it is to construct a fully-connected neural net
with a _dynamically random_ number of recurrent hidden layers in PyTorch. Yes,
it's a silly example, but it shows how easy it is to construct dynamic DNNs
with PyTorch:

    
    
      import random
      import torch
      
      class MySillyDNN(torch.nn.Module):
          def __init__(self, input_dim, hidden_dim, output_dim):
              super(MySillyDNN, self).__init__()
              self.input_layer = torch.nn.Linear(input_dim, hidden_dim)
              self.hidden_layer = torch.nn.Linear(hidden_dim, hidden_dim)
              self.output_layer = torch.nn.Linear(hidden_dim, output_dim)
    
          def forward(self, x, max_recurrences=3):
              hidden_relu = self.input_layer(x).clamp(min=0)
              for r in range(random.randint(0, max_recurrences)):
                  hidden_relu = self.hidden_layer(hidden_relu).clamp(min=0)
              y_pred = self.output_layer(hidden_relu)
              return y_pred
    

It would be a hassle to do something like this with other frameworks like
TensorFlow or Theano, which require you to specify the computational graph
(including conditionals, if any) before you can run the graph.

PyTorch's define-the-graph-by-running-it approach is sooo nice for
quick-n'-dirty experimentation with dynamic graphs.

You can even create and tinker with dynamic graphs _interactively_ on a Python
REPL :-)

~~~
rayuela
Have you run anything on multiple-GPUs or scaled to multiple nodes? My biggest
hesitation for using pytorch is what appears to be the limited distributed
compute support. Being able to easily scale a dynamic graph to arbritrarily
large size across a cluster would make pytorch an easy sell for me.

~~~
JustFinishedBSG
Well it was literally added this version:
[http://pytorch.org/docs/0.2.0/distributed.html](http://pytorch.org/docs/0.2.0/distributed.html)

Does it meet your needs?

~~~
rayuela
Ooooh this is exciting. Thanks for pointing that out!

------
senko
For those of us who don't know what this is:

 _PyTorch is a Python package that provides two high-level features:

Tensor computation (like NumPy) with strong GPU acceleration

Deep neural networks built on a tape-based autograd system

You can reuse your favorite Python packages such as NumPy, SciPy and Cython to
extend PyTorch when needed.

We are in an early-release beta. Expect some adventures and rough edges._

------
alexcnwy
There was a great podcast with Soumith Chintala on the O'Reilly data show a
couple of days back with more info on PyTorch and how it differs from Theano
and Tensorflow:

[https://www.oreilly.com/ideas/why-ai-and-machine-learning-
re...](https://www.oreilly.com/ideas/why-ai-and-machine-learning-researchers-
are-beginning-to-embrace-pytorch)

------
kbullaughey
Also of note is that the lua version of Torch seems to be in maintenance mode
now.

[https://twitter.com/soumithchintala/status/89417353800895692...](https://twitter.com/soumithchintala/status/894173538008956928)

------
toisanji
it is my favorite neural network library as well. Tensorflow is very heavy for
testing.

