

Now Serving: Full-Text Sampler in IPython Notebook Format - ptwobrussell
http://miningthesocialweb.com/2013/10/25/now-serving-full-text-sampler-in-ipython-notebook-format/

======
carreau
That's really amazing, I think I can speek for the all IPython team by saying
that we are __Really __happy with this.

We've cross a few authors that were writing books for O’Reilly with IPython
Notebooks in the last few month and we have plan to help convert those to (and
from) many format. We would be extremely happy to work with people from
O'Reilly to simplify the conversion process.

We also have plan to integrate with publication in academic world (see
[https://github.com/ipython/ipython/issues/4119](https://github.com/ipython/ipython/issues/4119))
but we really need more people involved than the few of us (Core Team).

Even if IPython seem like a big complicated project, moving forward in area
like document conversion is not that hard, but just needs lots of eyes and
tester to catch all cases.

As for people wanting to convert notebook to other format than those currently
supported, it is perfectly possible using nbconvert with template. We can spit
the TeX that is just between begin/end document so you can include in your
preferred dissertation template. It should be pretty easy to also just write a
converter that export to asciidoc or doocbook.

For those interested of the idea behing how nbconvert I wrote this quite some
time ago:
[http://nbviewer.ipython.org/urls/raw.github.com/Carreau/post...](http://nbviewer.ipython.org/urls/raw.github.com/Carreau/posts/master/06-NBconvert-
Doc-Draft.ipynb) and [https://github.com/ipython/ipython-in-
depth](https://github.com/ipython/ipython-in-depth) Offer some example too.

Hope this will push some people to contribute, we'll always be happy do guide
you in your first step.

~~~
IanOzsvald
I'm working on an O'Reilly book using IPython Notebook, I'm basing my workflow
on Olivier Grisel's script (with a couple of tiny fixes):
[https://github.com/ianozsvald/ipynbhelper](https://github.com/ianozsvald/ipynbhelper)
which extracts the Notebook's 'markdown' blocks (which contain asciidoc which
obviously won't render in the browser) & code blocks, these get exported as
asciidoc for O'Reilly.

It is clunky but I'm hoping we'll get better control as nbconvert evolves, so
we're experimenting with this approach.

Most of the code examples are not 'live', they're pasted in along with
analysis results (the book is about high performance and parallel computing:
[http://shop.oreilly.com/product/0636920028963.do](http://shop.oreilly.com/product/0636920028963.do)
), as lots of the examples are best run from a fresh VM.

------
mherradora
It is a very practical idea. How many times do you want to test an example?
With pdf and other formats you could make a copy-paste but many things could
go wrong. But the really important improvement is the use of Vagrant and a
virtual machine to give an accurate workspace.

~~~
jyeee
+1 mherradora - I got the first version right after twitter changed the API
>_< and busted one of the examples in chapter 1. Not anymore with Mining the
Social Web 2e :). Bonus style points for Vagrant so that the author doesn't
have to host (i.e. pay for) a VM and the massive bandwidth.

------
badhairday
I purchased Mining the Social Web, 2nd Edition (O'Reilly, 2013) a few weeks
ago and have been loving the IPython Notebook setup they're using to
distribute the code examples. Anyone can clone the following git repo and play
around with the code:

[https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-
Ed...](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition)

------
julienchastang
I work with scientists that love the IPython Notebook technology. Some claim
the IP[y]: Notebook to be the best thing since the Mosaic web browser and the
most important development in scientific computing in a decade. They are
replacing a variety of tools (Perl, Matlab, R, etc.) with the notebook
technology. I tend to agree, it is a revolutionary technology and the idea of
executable papers is tantalizing. But I also see some limitations. Setting up
a Python environment with all the necessary libraries can be a real pain in
the neck even with technologies like pip. Also taking advantage of multicore
parallelism is not trivial because of the Python Global Interpreter Lock. I
have also worked in JVM environments where parallel computing is becoming
significantly easier and I don't see that happening in Python anytime soon. I
would love to be proven wrong, of course.

~~~
mcburton
It should be noted that the IPython Notebook is being developed to support
multiple backends. People are working on backends for Haskell[1], Scala [2],
and Julia[3]. I'd love to see a Clojure backend.

All of this does not obviate the challenges of setting up IPython, but that
could also be (somewhat)resolved via cloud services[4][5][6].

[1]
[http://gibiansky.github.io/IHaskell/](http://gibiansky.github.io/IHaskell/)

[2] [https://github.com/mattpap/IScala](https://github.com/mattpap/IScala)

[3]
[https://github.com/JuliaLang/IJulia.jl](https://github.com/JuliaLang/IJulia.jl)

[4] [http://www.windowsazure.com/en-
us/develop/python/tutorials/i...](http://www.windowsazure.com/en-
us/develop/python/tutorials/ipython-notebook/)

[5] [https://www.wakari.io/](https://www.wakari.io/)

[6] [http://blog.picloud.com/2012/12/23/introducing-the-
picloud-n...](http://blog.picloud.com/2012/12/23/introducing-the-picloud-
notebook/)

------
jyeee
$1.15M (plus my opinion) says that IPython is the bee's knees:
[http://ipython.org/sloan-grant.html](http://ipython.org/sloan-grant.html)

------
mcburton
YES!

I would also love to see a publishing workflow that could render IPython
Notebooks for print on dead trees.

~~~
ptwobrussell
The 1.0.0 version started bundling nbconvert, which does a pretty good job of
exporting to html, pdf, latex, etc. Have you tried it out yet?

~~~
mcburton
I haven't started meaningfully playing with nbconvert yet, but I think one of
the really promising features are the nbconvert templates[1]. From what I
gather you can make custom templates with jinja and invoke them on the command
line. It would be interesting to create a docbook, msword, specifically
formatted pdf templates which would allow IPython Notebooks to flow into
existing print publication workflows (if I recall O'Reilly uses docbook,
yes?). I haven't been able to find documentation about the nbconvert templates
so I'm dragging this out of memory from an email or a tweet I saw once upon a
time.

One thing I'm hoping to figure out is a way to transform IPython Notebooks
into PDFs that satisfy my university's dissertation formatting guidelines (one
inch margins, 12pt times, etc).

[1]
[https://github.com/ipython/ipython/tree/master/IPython/nbcon...](https://github.com/ipython/ipython/tree/master/IPython/nbconvert/templates)

~~~
ptwobrussell
Wow, you definitely have some great ideas here, and given the enthusiasm about
ipynb, I would not be surprised to see these things happen in the next 6-12
months.

O'Reilly does use docbook internally as part of the toolchain, but asciidoc is
coming into style as well. I've done most of my work in docbook with a docbook
editor, but did some of my latest boook in asciidoc, which was much easier.
Just use a standard text editor of choice (like Vim) and get right to it. Much
easier for me, anyway.

One thing I'd really like to see is ipynb supporting asciidoc instead of or in
addition to markdown. You can go to/from asciidoc and docbook in a lossless
way, IIRC.

~~~
mcburton
asciidoc support would require an ansciidoc renderer written in javascript,
which does seem to exist[1](although I haven't used it). This also highlights
some of the problems with markdown because it is an insufficiently rich
language for generating semantically meaningful markup for use-cases beyond
blogging. There has been some really interesting work on "scholarly
markdown"[2][3], but given the social and technical complexities of markdown
implementations and the community[4] seems like adding asciidoc is easier than
fixing markdown.

[1] [http://asciidoctor.org/news/2013/05/21/asciidoctor-js-
render...](http://asciidoctor.org/news/2013/05/21/asciidoctor-js-render-
asciidoc-in-the-browser/)

[2][https://github.com/scholmd/scholmd/wiki](https://github.com/scholmd/scholmd/wiki)

[3][http://blog.martinfenner.org/2013/06/19/citations-in-
scholar...](http://blog.martinfenner.org/2013/06/19/citations-in-scholarly-
markdown/)

[4][http://www.codinghorror.com/blog/2012/10/the-future-of-
markd...](http://www.codinghorror.com/blog/2012/10/the-future-of-
markdown.html)

------
hesdeadjim
I would love to see this start to happen in the academic world too. Being able
to play around with a very complicated algorithm you are trying to wrap your
head around would be a godsend.

~~~
ptwobrussell
There are some awesome notebooks out that demonstrate exactly that. Here are a
couple of good links with combinatorial optimization and TSP:

[http://nbviewer.ipython.org/urls/raw.github.com/fonnesbeck/B...](http://nbviewer.ipython.org/urls/raw.github.com/fonnesbeck/Bios366/master/notebooks/Section3_2-Combinatorial-
Optimization.ipynb)

[http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipy...](http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipynb)

~~~
sillysaurus2
This is the first I've looked into IPython Notebook, so apologies for the
newbie question: Is a notebook an html page with Python code snippets
interspersed? Are these supposed to be interactive at all? Or do you have to
copy the Python locally, then install some packages before you can finally run
it?

I guess I'm asking, what's the difference between an HTML page that shows a
Python implementation of an algorithm vs those links? Or what's the appeal?

I'm certain IPython Notebooks are valuable, since everybody seems to love
them. It's not clear from glancing at those examples what the big deal is,
though, so I thought I should ask in case more newbies come along and
incorrectly conclude that there's nothing especially interesting about those
examples beyond their content. (To be clear, the content is excellent, but why
is IPython Notebook to thank rather than the author? Could those examples not
have been done via some other method, or is the purpose of IPython Notebook
just to make it really easy to generate such pages?)

~~~
mcburton
The IPython Notebook is part of the IPython interactive computing
environment[1]. You must to install IPython locally(or in the cloud) and run
the notebook web server to load and execute the notebook's code live in the
browser. These are static HTML representations of the notebooks, think of them
as previews, but you can download the JSON .ipynb file and load it up in your
IPython Notebook environment and execute the code.

The benefit of the IPython Notebook is sharable JSON file and interactive
REPL-like environment that blends code, results, and narrative prose.

[1] [http://ipython.org/ipython-
doc/stable/interactive/notebook.h...](http://ipython.org/ipython-
doc/stable/interactive/notebook.html)

edit: I should have refreshed the page before I wrote this redundant reply.

------
ptwobrussell
I think it's your opinion that is really the deciding factor there ;)

~~~
jyeee
lol, don't just take my word for it: [http://blog.fperez.org/2012/09/blogging-
with-ipython-noteboo...](http://blog.fperez.org/2012/09/blogging-with-ipython-
notebook.html)

~~~
leephillips
Really interesting blog. I would love to follow it but I don't see an RSS (or
atom) feed: did I miss it?

~~~
jyeee
On bottom right:
[http://miningthesocialweb.com/feed/](http://miningthesocialweb.com/feed/)

One of the fun examples shows how to mine an RSS feed ;)

[https://rawgithub.com/ptwobrussell/Mining-the-Social-
Web-2nd...](https://rawgithub.com/ptwobrussell/Mining-the-Social-Web-2nd-
Edition/master/ipynb/html/Chapter%205%20-%20Mining%20Web%20Pages.html)

