
PyTorch-ProbGraph – Hierarchical Probabilistic Graphical Models in PyTorch - korbip
https://github.com/kpoeppel/pytorch_probgraph
======
DiffProg
This still doesn't fix the fact that PyTorch libraries like torchdiffeq tend
to benchmark orders of magnitude faster than other libraries (1) and then devs
just write off the performance issues (2)

(1):
[https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e...](https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320)
(2): [https://discuss.pytorch.org/t/why-torch-jit-is-so-
slow/36616...](https://discuss.pytorch.org/t/why-torch-jit-is-so-slow/36616/2)

~~~
t-vi
A more charitable view on the PyTorch forums thread is that Ed's explanation
"torch.jit, at this point in time, is not designed to take pointwise loops as
you’ve written here, and compile them into machine code directly." is a
description of "this is the feature you're implicitly asking about, and it
isn't implemented in PyTorch yet". What would have been a better answer in
your view?

It would seem btw that 1.5 years later, people are working on implementing
generated kernels for reductions, even if it is still somewhat experimental.

The speed comparison, which is about 9 months old, is marginally related. The
issue here is optimization of pointwise operations, which would be handled by
the JIT fuser, except it is disabled by default on the CPU. The latest version
of the benchmark code seems to not run on recent PyTorch versions as given. It
still still a fair comparison in terms of it is what a user will get by
default. I won't be the first PyTorch developer to say that Julia and its libs
do a great job at JITed optimizations. Nonetheless I'm relatively certain that
a determined PyTorch user would find ways to get that a better optimization of
that ODE step using some of the disabled by default features.

The other truth, of course, is that for PyTorch, there still is more emphasis
on GPU when it comes to implementing optimizations.

(Disclaimer: I'm one of the people on that thread.)

------
person_of_color
Where are HMMs / CRFs?

~~~
korbip
Feel free to add them. :)

