
OCaml bindings for PyTorch - ghosthamlet
https://github.com/LaurentMazare/ocaml-torch
======
nafizh
At the heart of the machine learning that I do with pytorch, most of the
errors come from wrong sizes of matrix multiplications, or some matrix input
that has a wrong size somewhere in the middle of the net.

How much an extra layer of wrapper of a functional language help with that?
Pytorch is not much helpful in terms of the error messages. You have to print
out the matrix sizes generated from intermediate operations to find out what
is really happening.

~~~
l-m-z
(I'm obviously biased as being the author of ocaml-torch) Using functional
programming is not of much help for size mismatches, the current bindings
don't even use the type system to check the number of dimensions - although
that should be reasonably easy to add. Maybe a better approach to help with
this would be tensors with named dimensions
[http://nlp.seas.harvard.edu/NamedTensor](http://nlp.seas.harvard.edu/NamedTensor)
It's possible that a strong type system would help here but I don't think
there have been much attempts.

However when using the Python api I often have errors because of: \- Unused
variables when refactoring my code, which are just me forgetting to use some
parameters. \- Comparing things of different types (and Python does not report
any error and just return that they are different). \- Making changes to some
helper functions without adapting all the projects where I'm using them. Using
a good Python linter probably helps with these, but that's a place where
languages like ocaml naturally shine.

------
breatheoften
Where are cutting edge type systems at for statically verifying the
“dimensionality” of computations? I remember a long time ago reading about a
language from sun called “fortress” which was going to include the option of
assigning units to values in such a way that scientific domain model
operations could be defined via a kind of type-sugar ... seems like that
technology would work really well for ML (and data science in general) if it
were effective.

I remember reading some lengthy hype about fortress and then never hearing
about it again ...

~~~
nestorD
Note that units are implemented in F# :
[https://fsharpforfunandprofit.com/posts/units-of-
measure/](https://fsharpforfunandprofit.com/posts/units-of-measure/)

------
Gormisdomai
Do bindings like this help much with the day to day work of machine learning
research? Are the kinds of errors you encounter the kind that could be avoided
with a smarter type checker?

~~~
yawaramin
I would say so. There are 4,500 results on Stack Overflow for '[pandas]
typeerror':
[https://stackoverflow.com/search?q=%5Bpandas%5D+typeerror](https://stackoverflow.com/search?q=%5Bpandas%5D+typeerror)

~~~
Barrin92
I'm not sure that's a good metric. After all type checking does not really
answer a question a developer has about type errors, it just informs you of a
type error before you run your program.

In fact I'd wager your average Haskell type error has generated more than a
few frustrated stackoverflow questions.

~~~
whateveracct
Right, but learning about the errors without having to run and hit every
codepath in your program has a lot of value. So in that way, seeing how many
people hit pandas runtime type errors is a useful thing to measure.

------
dunefox
Does this support all of the features of PyTorch?

~~~
l-m-z
Bindings author here. This should support most of the PyTorch ops - this is
done by automatically generating the binding code as there are more than a
thousand of these. Most of the models available in torchvision should also be
there (with pre-trained weights) see [https://github.com/LaurentMazare/ocaml-
torch/tree/master/src...](https://github.com/LaurentMazare/ocaml-
torch/tree/master/src/vision) Finally it's also possible to export some python
defined models and run them from ocaml.

That being said there are some rough edges compared to the PyTorch api, e.g.
no parallel data loader, not much tooling and only a couple tutorials...

~~~
dunefox
Sadly, a lack of material is to be expected for anything but Python...

