
A general-purpose probabilistic programming system with programmable inference - espeed
https://github.com/probcomp/Gen
======
DigitalTerminal
Sounds spiffy. Also, I must find time to learn Julia. One problem with these
'out of a box' cookbook systems is that you only know approximately what you
are doing. If it works, that may be fine, but for statistical systems the
outcome depends heavily on the formulation, is strongly influence by random or
irrational correlations and is more art than science. If you remove the
element of experience that animates the "art", you get nonsense. Big box
programming tools can make it hard to know what correlations are being
established, and this is true with even simple neural networks. By the time
they have many layers and filters in them, all hope is fled. You buy the
result or you don't: simple as that.

~~~
xiphias2
I have nowhere experienced more looking under the hood feeling then when I am
using Julia.

The source code of the packages sit on my computer, I can (and sometimes do)
modify it, and I can still compile the same function to SIMD or GPU.

~~~
GorgeRonde
Although I have never used Julia, this is something that is possible in Ruby,
and I think this feature is seriously underrated – i.e. to be able to jump
quickly to the definition of external code, modify it and run it. When you
work with multiple libraries/repo this is very valuable when debugging, saves
a lot of time.

It should also be possible with Node.js I think (node_modules).

~~~
xiphias2
Don't get me wrong, I love Ruby, and my code was originally in Ruby, but the
20x speedup and better math libraries in Julia without sacrificing the speed
of development too much made the switch for me very easy.

Actually translating Ruby code to Julia was much easier than I thought (the
only real difference is the indexing).

The 1 based indexing really sucks in the PTX assembly output of Julia as well,
I see a lot of useless increment and decrement operations when I don't expect
it.

------
zwaps
Makes sense to write it in Julia. Pretty much the only non commercial language
that deals natively with mathematical objects in a way that makes sense to non
CS people doing statistics. Plus, it is fast out of the box.

Like, when I first saw how you work with matrices in Python, I had a stroke.

I dont really get why we use Python for everything. The matlab style syntaxt
is infinitly better for ML, stats and math.

~~~
valw
> The matlab style syntaxt is infinitly better for ML, stats and math.

Syntax is a secondary concern for all but short-term, small-scale programming.

When it comes to making scalable software systems (scalable in every sense of
the term, not just performance but also feature growth etc.), syntax is really
not the biggest concern; as such, optimizing for syntax can be detrimental to
other more important aspects (e.g composability, reusability,
programmability), and focusing on syntax means you're not paying attention to
the really critical stuff.

I mostly work in Clojure where _all_ operations use prefix syntax (e.g you
write (+ A B C) instead of (A + B + C) for addition, and + is just a
function). Trust me, it's unfamiliar at first, but it makes reasoning about
the program MUCH saner than all the special cases and irregularities that
language designer bake into the language just to make some operations infix.

~~~
bayareanative
RPN FTW. Good ol' LISP. :)

In Haskell, you can turn any function into an infix operator with backquotes.
As a qualitative, nonfunctional stylistic, ergonomic opinion, I find Haskell
much more flexible and beautiful than the ugliness of most brace-, paren- and
prolixity-heavy languages. It can present too many features for pragmatic use,
orthogonal to something in another domain like Rust or Go for effective
software production where features are intentionally constrained or
disallowed.

~~~
gumby
> RPN FTW. Good ol' LISP. :)

Actually Lisp uses ordinary old polish notation. Reverse Polish Notation is
used by stack-oriented systems like HP calculators and Forth.

~~~
kazinator
Not really; the motivating point of Łukasiewicz's notation was specifically to
eliminate parentheses.

~~~
klipt
Well choosing between

(- (+ 1 2 3 4 5) 1)

and

\- + + + + 1 2 3 4 5 1

The first one seems much more readable to me...

------
Vanit
Seems to be a Julia lib unless I'm missing something?

~~~
tejtm
Github says it is

    
    
        Julia 100.0% 
    

(click colored bar at the top of a repo)

------
baalimago
The problem with frameworks like this is when you try to do something which
isn't strictly standard, or lack understanding on whatever the framework is
actually doing. Things will no doubt break, and then you have a 15 layered
framework stack trace to try to debug.

You can't dodge knowledge with frameworks, only use frameworks to effectivize
your knowledge.

~~~
seanmcdirmid
This is exactly the premise of the paper, which isn’t presenting a framework,
rather a language for general open ended computation for ML.

------
ChrisRackauckas
I'm excited for the probabilistic programming symposium at JuliaCon :). We had
so many submissions around PP languages and approaches that we decided to
bunch them all together to get people talking about the pros/cons of various
approaches and find out what kinds of problems to collectively tackle. PP
seems like a very promising field.

------
amval
Could anyone point me in the direction of resources to learn more about
probabilistic programming?

(e.g.: text books, practical applications, introductory articles)

~~~
espeed
MIT Probabilistic Computing Project
[http://probcomp.csail.mit.edu/](http://probcomp.csail.mit.edu/)

The Design and Implementation of Probabilistic Programming Languages
([https://dippl.org](https://dippl.org)) by Noah D. Goodman
[https://cocolab.stanford.edu/ndg.html](https://cocolab.stanford.edu/ndg.html)

Stanford CS 228: Probabilistic Graphical Models
[https://cs228.stanford.edu](https://cs228.stanford.edu) and book by Daphne
Koller
[http://openclassroom.stanford.edu/MainFolder/CoursePage.php?...](http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=ProbabilisticGraphicalModels)

ProbTorch: Library for deep generative models that extends PyTorch
[https://github.com/probtorch/probtorch](https://github.com/probtorch/probtorch)

Anglican: Probabilistic programming language integrated with Clojure and
ClojureScript
[https://probprog.github.io/anglican/index.html](https://probprog.github.io/anglican/index.html)

Discussion:
[https://news.ycombinator.com/item?id=18585465](https://news.ycombinator.com/item?id=18585465)

------
langitbiru
I think this is an exciting time for programming language designers. As we can
see in blockchain space, on which there are new programming languages like
Solidity and Vyper, I believe there will be new programming languages to make
it easier for the programmers to express themselves (something different
compared to Julia, R, Swift, or Python) in AI.

Right now people solve the problem with frameworks like Keras so the
programmers have easier time to express themselves (for example designing
neural networks). Imagine you have a new programming language on which it is
much clearer and easier to create a neural network (in Deep Learning).

------
xvilka
There is a missing piece in the Julia subsystem - first class support for GNN
(Graph Neural Networks). So if someone wants to help - he can take an issue[1]
at the GitHub repository of FluxML[2].

[1]
[https://github.com/FluxML/Flux.jl/issues/625](https://github.com/FluxML/Flux.jl/issues/625)

[2] [https://fluxml.ai/](https://fluxml.ai/)

~~~
ChrisRackauckas
There's a JSoC developing this IIRC.

------
dmix
There seems to be an opportunity for a language to fill the data science/AI
space for people who want something more performant and safe (ie, statically
typed) than Python. I see people coming into /r/Haskell and other forums all
the time looking for something that fits this category but there's never a
really good answer that is better than the compromise that is Python with some
C or whatever based optimizations.

~~~
GavinMcG
Do you know how raw Julia and Nim stack up on those fronts?

~~~
dmix
No I don't sorry I'm only a spectator to those threads.

I'd personally at a minimum try out Julia if I was doing data science often.
Don't know enough about Nim enough yet.

------
pizza
Video abstract:
[https://www.youtube.com/watch?v=DImI6l_0yiM](https://www.youtube.com/watch?v=DImI6l_0yiM)

------
adamnemecek
Github repo [https://github.com/probcomp/Gen](https://github.com/probcomp/Gen)

~~~
applecrazy
Bunch of tutorials using the language:
[https://probcomp.github.io/Gen/tutorials.html](https://probcomp.github.io/Gen/tutorials.html)

Paper:
[https://dl.acm.org/citation.cfm?id=3314221.3314642](https://dl.acm.org/citation.cfm?id=3314221.3314642)
(is this article paywalled? I'm on my university's network so I don't have a
paywall)

The syntax doesn't seem to be changed compared to Julia (so is it actually a
new language or just a library? Not sure, haven't seen Julia code in a while.)

~~~
byt143
>The syntax doesn't seem to be changed compared to Julia (so is it actually a
new language or just a library?

And the semantics are the same with some additions. This is a good thing as it
allow interop with Julia code, low mental overhead and shows the power of
Julia :)

------
sgt101
Very exciting; I've found using STAN really hard, I really struggle to model
real problems, perhaps this will make it easier - and I far prefer Julia to R
in project work (not analytics or episodic data science; but when developing
systems I find it hard to structure and decompose).

I wonder if the HN crowd think this system is AI ?

~~~
sgt101
Random comment : by episodic data science I mean one of question answering
quests. These are often created by incidents or by new stakeholders asking
unexpected questions. Basically - collect, curate, explore, answer, explain,
dump.

------
ipsum2
Pretty clickbaity, this looks like another probabilistic programming
framework. Others include Pyro (pyro.ai) and Edward
([http://edwardlib.org/](http://edwardlib.org/)).

~~~
6gvONxR4sf7o
Maybe the title's clickbaity, but steps forward in probabilistic programming
are great. From the paper, it looks interesting.
[https://dl.acm.org/citation.cfm?id=3314221.3314642](https://dl.acm.org/citation.cfm?id=3314221.3314642)

------
sandGorgon
This press release is super weird. They are trying really really hard to not
mention Julia.

A "new language" versus "a DSL built on Julia".

"Swift for Tensorflow" is still Swift.

I wonder why.

~~~
kgwgk
Maybe the link has changed, but now one can read:

“Building off concepts used in their earlier probabilistic-programming system,
Church, the researchers incorporate several custom modeling languages into
Julia, a general-purpose programming language that was also developed at MIT.”

------
scottlocklin
Typical MIT press release balognia; their publicist really should get a job in
industry and stop misleading the scientific community.

What they really mean is "new probabilistic graphical model language." Yet
another BUGS/JAGS/Stan like system.

~~~
Fishysoup
It's not a PGM package. It's designed to work with arbitrary Julia programs
and can be used for program induction etc. In that regard it's more like
WebPPL or Church than Stan.

~~~
scottlocklin
OP originally linked it to some breathy MIT press release which said

"New AI programming language goes beyond deep learning General-purpose
language works for computer vision, robotics, statistics, and more"
[https://news.mit.edu/2019/ai-programming-
gen-0626](https://news.mit.edu/2019/ai-programming-gen-0626)

I'm pretty sure most sane people agree this is a giant pile of meaningless
marketing horse pookey, which is par for the course with MIT these days.
That's what I was responding to. And characterizing it as "like stan" is a lot
closer to descriptive than anything in that article. I picture whoever writes
this stuff for MIT as wearing tap out t-shirts with neck tattoos.

~~~
Fishysoup
Sure, the article is largely meaningless. But I've spent a couple months
playing with this package and it is a lot more general than Stan etc.

------
cocochanel
The title is confusing. Can anyone please explain if this is Julia or a new
programming language?

~~~
dagw
It's a set of macros and helper functions build on top of Julia to provide a
sort of probabilistic programming DSL. The actual project describes themselves
as a "programming system" rather that a programming language.

------
wtdata
The only place were they make it clear this is a Julia package is in the
installation instructions:

"The, install the Gen package with the Julia package manager. From the Julia
REPL, type ] to enter the Pkg REPL mode and then run: pkg> add
[https://github.com/probcomp/Gen"](https://github.com/probcomp/Gen")

[https://probcomp.github.io/Gen/](https://probcomp.github.io/Gen/)

------
nerdponx
Less-clickbait dupe:
[https://news.ycombinator.com/item?id=20302158](https://news.ycombinator.com/item?id=20302158)

------
linux_devil
I was really looking forward to probabilistic programming but then I saw this
project is based on Julia. Not to keen on leaning Julia as I already know
Python/C++/Java/Scala/R. Wish it was based on c++/python to at least get
started with. Please let me know if there are other interesting probabilistic
programming frameworks available or tutorials to get started with.

Edit: [https://medium.com/tensorflow/an-introduction-to-
probabilist...](https://medium.com/tensorflow/an-introduction-to-
probabilistic-programming-now-available-in-tensorflow-
probability-6dcc003ca29e) There is support in tensorflow for probabilistic
programming. How is this any different?

~~~
espeed
Julia is fast becoming one of the great programming languages of our era.
Julia is not going away, its usage is only becoming more pervasive. Imagine if
you had learned Python 20 years ago. Julia's founding sponsor is Dr. Jeremy
Kepner [1] who is the founder of MIT Lincoln Laboratory Supercomputing Center
[2] and the founder of GraphBLAS [3]. Kepner's got a good eye for things, he
knows what's coming down the pipe and what will be important in a language of
the future. Learning Julia now means you'll be ahead of the curve for years to
come. Your time will not be wasted.

[1]
[https://en.wikipedia.org/wiki/Julia_(programming_language)#S...](https://en.wikipedia.org/wiki/Julia_\(programming_language\)#Sponsors)

[2] [http://www.mit.edu/~kepner/](http://www.mit.edu/~kepner/)

[3] [http://graphblas.org](http://graphblas.org)

~~~
langitbiru
Julia needs something like PyTorch or Tensorflow equivalents.

~~~
ChrisRackauckas
[https://fluxml.ai/](https://fluxml.ai/) ?

~~~
sischoel
And also

[https://github.com/denizyuret/Knet.jl](https://github.com/denizyuret/Knet.jl)

[https://github.com/malmaud/TensorFlow.jl](https://github.com/malmaud/TensorFlow.jl)

