
GitHub and Jupyter IPython Notebooks - benn_88
https://github.com/blog/1995-github-jupyter-notebooks-3
======
RogerL
I am utterly giddy about this. I've spent a lot of time in the book I am
writing to generate PDFs, to create TOCs for links to nbviewer, managing all
these links and production every time I made a change. Now people can just
view it directly in github. Life is as it should be.

There is just so much innovation and hard work coming out of this group, which
is quite tiny. Any time I've raised an issue, even if it turned out to be my
own dumb fault, gets the immediate attention and support of a core developer.

If you want to communicate with programmers there really isn't a better
platform out there. The old 'write code, run code, save results, write latex,
gen document, find error, repeat, opps code is out of sync with results....'
is pining for the fjords.

------
jlrk
Yes! A million times, yes! This lowers the barrier to sharing research results
stored in a private repo with collaborators. Also, for those who have been
looking for ways to display equations in markdown[1], you can embed the
equations in markdown (Ipython uses Mathjax) within the notebook.

[1]
[https://github.com/github/markup/issues/274](https://github.com/github/markup/issues/274)

------
mvermaat
This is great! We use IPython Notebook a lot in our group, for exploratory
data analysis, teaching, sharing experimental results, and as an electronic
lab notebook. Inline GitHub rendering is very nice for all our public work.

If you run a private GitLab server, you could have a look at my patch which
adds similar functionality to GitLab (in a rather ad-hoc way):
[https://gist.github.com/martijnvermaat/6926070](https://gist.github.com/martijnvermaat/6926070)

~~~
sytse
GitLab CEO here, thanks for maintaining your patch for GitLab. If you want to
discuss contributing jupyter support to GitLab itself I'd be happy to discuss.

------
Myrmornis
This is amazing. I think ipython notebook is fantastic in all regards.
However, I feel there's a bit of an elephant in the room. The main tools I use
as a programmer are shell and a text editor. I'm not about to start writing
code in a web browser[1]. Is there any hope that there will be a good workflow
for somehow editing .ipynb from a text editor or is that antithetical to the
design? Could markdown perhaps be used as a primary editing format to target
ipynb?

Looks like there's at least one project doing that.
[https://github.com/aaren/notedown](https://github.com/aaren/notedown)

And there's the emacs-specific project [https://github.com/tkf/emacs-ipython-
notebook](https://github.com/tkf/emacs-ipython-notebook)

[1] EDIT: I mean, not right now anyway. I've got a feeling that statement
could seem a bit dated later in my life...

~~~
idunning
It is an interesting point. In my usage, I don't view it as a one-or-the-other
proposition. I use Julia, so if I'm doing some exploratory work and
experimentation, or making something to present results, then the IJulia
notebook is great. If I'm writing some serious longer-running stuff, or a
package, I'm in an editor. Sometimes I'll write code in a separate file and
call it from a notebook just to keep the notebook focussed on communicating
something, and "hiding" the details.

~~~
ngoldbaum
Yeah, notebooks are great for experimentation but not so much for anything
that needs to be reused.

I tend to write prototype code in the notebook then copy/paste it into a
package as I become satisfied that the code is correct.

------
munro
So weird, I was playing with IPython Notebook with PyCharm yesterday. Then I
went on to think about how it could be integrated into Github without building
the compiled files into the repo. :)

Really there's no solution for extending Github except wait for them to
integrate hooks. It would be nice if they made this more generic, and created
a file render hook, that would allowed devs to render whatever types of files
they want into Github Markdown.

~~~
hcrisp
How does iPython integrate with PyCharm?

~~~
usef-
PyCharm allows you to edit and run ipynb files directly – see here:
[http://blog.jetbrains.com/pycharm/2014/12/feature-
spotlight-...](http://blog.jetbrains.com/pycharm/2014/12/feature-spotlight-
ipython-notebook-inside-an-ide/)

------
bigredhdl
Just a general comment, but in my opinion IPython/Jupyter is worth learning
Python for. It is such a great platform for tinkering with data or really any
idea you might play with in code.

~~~
RogerL
The notebooks support a very large number of kernels, including Julia, R,
bash, Lua, Erlang, Perl, OCaml, and so many more[1].

[1] [https://github.com/ipython/ipython/wiki/IPython-kernels-
for-...](https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-
languages)

~~~
pash
And in case you were wondering, GitHub does render notebooks using other
language kernels. For example, here's a notebook in Haskell:
[https://github.com/gibiansky/IHaskell/blob/master/notebooks/...](https://github.com/gibiansky/IHaskell/blob/master/notebooks/Homophones.ipynb)

------
rcpt
Hey this is great! I just took a look and saw that a notebook [1] I was
tracking in github is now rendering beautifully - and I didn't have to do
anything!

[https://github.com/rcompton/ml_cheat_sheet/blob/master/super...](https://github.com/rcompton/ml_cheat_sheet/blob/master/supervised_learning.ipynb)

~~~
voltagex_
They've really relaxed the filesize restrictions for these - I don't think a
7mb PDF will render in GitHub but this notebook will.

------
batbomb
There's a new problem which still hasn't been resolved w.r.t. iPython
notebooks and reproducible results, and that's the problem of private,
ephemeral, or mutable data. At the very least, iPython theoretically helps
with the audit trail for private and mutable data, but it doesn't necessarily
foster (re)producible results or alternative analysis.

This is a more general problem overall, but potentially something like magnet
URIs and bittorrent could really help with part of the problem. (I don't
really believe git as a system nor GitHub as a platform to be the appropriate
place to solve this either).

~~~
tmarthal
Are you saying that people aren't storing data on github? How large of
datasets are you talking about?

It's a shame that Github's large file storage (LFS) mechanism isn't publicly
released yet, because I am sure that will appease your problems. (This is
assuming your datasets don't fit into a csv (note that the current hard limit
for github files is 100MB, and a 100MB CSV file is a _lot_ of data)).

~~~
Fomite
It should be noted that there are reasons beyond file size not to store data
on GitHub, though I do regularly generate simulation data that's way larger
than 100 MB CSV files.

~~~
sciencerobot
Just curious: why not just provide the code that generates the simulation
data?

~~~
Fomite
Because it's an absolutely massive code base that requires access to some
serious HPC resources, which basically makes it difficult if not impossible to
reproduce for most people. Putting something out there like that also implies
something of an obligation to maintain and support it, which isn't what we do.

There's also a weird data rabbit hole. Is the code that generates the
simulation data enough, without the underlying data that code uses? Some of
that is either protected or proprietary, so even with the code, it's utterly
useless.

------
Fomite
This also means, at GitHub repos can get DOIs via FigShare, you can have DOIs
for an iPython Notebook, which is _awesome_.

------
saboot
Okay, what is the best format to store images in? My notebooks contain lots of
graphs and output but I'd rather avoid saving the raw pixels. So far I've been
using SVG with mixed success. Does anyone use anything else to save space?

~~~
rcpt
I just tried out
[https://pypi.python.org/pypi/ipynbcompress/0.3.0](https://pypi.python.org/pypi/ipynbcompress/0.3.0)
and it worked pretty well. Need to use 'png' to get nice rendering on github.

The last cell here has my use
[https://github.com/rcompton/ml_cheat_sheet/blob/master/super...](https://github.com/rcompton/ml_cheat_sheet/blob/master/supervised_learning.ipynb)

------
stevetjoa
Great idea! Although it appears that the Audio widget does not display
correctly. Compare:

[http://nbviewer.ipython.org/github/stevetjoa/stanford-
mir/bl...](http://nbviewer.ipython.org/github/stevetjoa/stanford-
mir/blob/master/notebooks/onset_detection.ipynb)

[https://github.com/stevetjoa/stanford-
mir/blob/master/notebo...](https://github.com/stevetjoa/stanford-
mir/blob/master/notebooks/onset_detection.ipynb)

------
huac
Hope support for R-markdown is next!

~~~
haddr
I was also a bit dissapointed for no mention about R :( Anyway, i think it's
possible to get notebook for R using this:
[https://github.com/IRkernel/IRkernel](https://github.com/IRkernel/IRkernel)

look at this demo:
[https://github.com/IRkernel/IRkernel/blob/master/Demo.ipynb](https://github.com/IRkernel/IRkernel/blob/master/Demo.ipynb)

------
shepardrtc
So they're only displaying the notebooks, but not actually allowing you to
work in them, right? Because for a minute there I was pretty excited!

~~~
carreau
you cannot work on it __yet __.

~~~
rspeer
It sounds like you expect this to change. If you could actually run the
commands, where would the computing power come from?

~~~
takluyver
If you go to [https://try.jupyter.org/](https://try.jupyter.org/) , you'll get
a temporary notebook server in a Docker container, courtesy of Rackspace. It's
automatically destroyed after a few minutes of inactivity.

------
cabhishek
Very thoughtful and awesome. Now is there an easy way to save notebooks
directly on GitHub?(git as storage system?). That would make the whole flow
seamless and make sharing and versioning easy :)

------
yes_or_gnome
Screen has both 'c' and 'C-c' set to new window by default. tmux only has 'c'
by default.

From
[https://www.gnu.org/software/screen/manual/html_node/Default...](https://www.gnu.org/software/screen/manual/html_node/Default-
Key-Bindings.html):

    
    
      C-a c
      C-a C-c
      (screen)
      Create a new window with a shell and switch to that window. See Screen Command. 
    

Edit: d'oh. I had to reread the original comment. You already know this, but
by adding 'C-c' I don't have to release the Ctrl button.

You can add this to tmux.conf to get screen-like behavior:

    
    
      # New Window
      bind C-c new-window

~~~
wyldfire
The wires are crossed on HN or your tabs. Your tmux response is in the github-
ipython notebook thread.

------
agrothberg
It would still be nice if IPython Notebook had an option to save the code
separate from the output, for example as two files in the same directory. This
would make source controlling easier.

~~~
c0lin
Running `ipython --notebook` with the additional option `--script` may be what
you are looking for. This saves a `.py` file with the same name next to your
`.ipynb` file (the non-code sections are present, but they are commented out).

------
vhost-
I just started using these for my data science work and it has been absolutely
awesome. Even better now that they display on Github and apparently (based on
a comment below) Gitlab too.

------
andersonfreitas
Would be nice to have .ipynb preview on Gist also!

------
calebm
This is awesome! Similar to nbviewer.ipython.org functionality.

------
bruce487
Awesome, thanks for sharing.

