
Jupytext: Jupyter notebooks as Python scripts - JA7Cal
https://github.com/mwouts/jupytext
======
turingcompeteme
This is neat. If you are looking for something a bit simpler: put your code in
a python package, install it, and then use %autoreload

You can now develop in your favorite editor, and changes are automatically
reloaded.

More detail here: [https://blog.godatadriven.com/write-less-terrible-
notebook-c...](https://blog.godatadriven.com/write-less-terrible-notebook-
code)

~~~
xaedes
Well, thank you! No more annoying kernel restarts, that really bugged me a
long a time.

------
Grumbledour
This seems neat, but also a bit strange. Are Jupyter notebooks so prevalent
that it makes sense to ghave an exporter that makes them into markdown and
back again? I mean, I always thought this workflow was neat, but isn't one of
the selling points of jupyter the WYSIWYG Interface? And if you like working
in code, would it not be easier to create a markdown extension for embedded
code (Or does something like this already exist?) and create editor plugins or
command line tools to render said document or provide a live preview?

Personally, I totally want that functionality but don't want to mess with
running jupyter kernels or using my browser to work in my notebook. So where
is my markdown with embedded code WYSIWYG editor that saves into plain text
files? ;-)

~~~
stochastic_monk
As an academic, ipython notebooks are becoming the lingua franca of
reproducible research. I don’t like it or them, but they’re excellent for
didactic purposes. Extracting the code from the markdown is very helpful for
me, as I prefer working directly with it most of the time.

~~~
hobo_mark
I'm sure notebooks are used a lot for interactive exploration, but I would not
use them for _reproducible_ research, so long as they lack things like
dependency tracking:

[https://twitter.com/joelgrus/status/1033035196428378113](https://twitter.com/joelgrus/status/1033035196428378113)

~~~
makmanalp
The bar in academia (over-generalizing here) was so low for so long that
anything around including your code with your publication still gets placed
under the historical title of "reproducibility" \- that's probably where the
confusion here is arising. I agree with your sentiment, but grandparent
probably just meant "a lot of people release their code as notebooks".

~~~
stochastic_monk
It’s a step, but you’re both right. Things that containerize notebooks would
count as reproducible, but they’re unwieldy and disgusting. I agree that while
better than nothing they’re still not there.

------
dec0dedab0de
_The Python script can be opened with PyCharm_

This is going to be a bit pedantic, but for anyone that doesn't realize.
Pycharm can already open ipynb files, and it will connect to a running
notebook. I don't do it very often, usually just to quickly find my way around
some library code.

This project looks great though. I will likely use it, as I tend to use
Jupyter to work out code before copying it over to my main project.

~~~
Seanny123
In my personal experience, PyCharm really mangles Jupyter Notebooks. Has it
gotten to better recently?

~~~
sgillen
I have not experienced much mangeling, but the newest CE edition still doesn't
do notebooks very well IMO. to the point where I use jupyter-lab for my
notebooks/prototypes and pycahrm for packages.

------
thomasfedb
This is really cool stuff. Editing notebooks in the browser is pretty good,
but it can't (yet) match the experience of a well configured (and extended)
text editor.

~~~
subw00f
I've been using vscode with the jupyter extension
([https://github.com/DonJayamanne/vscodeJupyter](https://github.com/DonJayamanne/vscodeJupyter))
and it's exactly what you describe, experience of a good text editor and the
visual notebook workflow.

~~~
thomasfedb
Neat, I'll give that a shot. Does it handle all kernels?

~~~
kylebarron
It doesn't appear to. Helpfully, Hydrogen [0] for Atom was designed to be
entirely language agnostic.

[0]:
[https://github.com/nteract/hydrogen](https://github.com/nteract/hydrogen)

------
drej
If you're worried about diffs, check out
[https://github.com/jupyter/nbdime](https://github.com/jupyter/nbdime)

~~~
julienchastang
That's true but ideally you would like the native format to be easily diffable
(e.g., plain text) and not require special tools.

------
hcrisp
Could this be the answer to "I don't like notebooks" [0]? Or does calling
scripts from within a notebook still do horrible things to the managed state?

[0]
[https://news.ycombinator.com/item?id=17856700](https://news.ycombinator.com/item?id=17856700)

~~~
zimablue
I've never thought this criticism was that relevant. Notebooks have been
incredibly successful, and if it's possible to call your code in the wrong
order and create a mess, it's probably a problem with your code not your
coding environment. The more functional your code, the less of a problem
jupyter notebooks are. They encourage better code not worse imo.

~~~
mikepurvis
The "managed state" is important, though. For example I deal with cases where
I'm doing some big expensive operations upfront (eg, processing a bunch of
data coming in over a network share— ROS bag files, which are also bzipped,
adding more upfront processing cost), and I'm interesting in tweaking the
plots and analysis coming out the other end without having to re-run the
processing each time.

Even as a relatively experienced developer, I've found it hard to reason about
what exactly it is that Jupyter is doing under the hood; basically things will
break in weird ways, and rerunning everything will mysteriously fix it, or I
have to use hacks to force it to re-import certain modules in order to get
things the way I expect.

Basically, it ends up being easier to just manage this myself, like doing the
processing and pickling the result, then using other, standalone notebooks to
load and render the pickle. But this shouldn't be necessary.

~~~
mynewtb
That sounds really weird. The only time I noticed such weirdness was when a
colleague and me were simultaneously editing a notebook on Google Colab. The
code was shared but not the state, super weird...

------
micheda
Nice! similar to
[https://github.com/minodes/pynb](https://github.com/minodes/pynb) , that
converts Jupyter Notebooks to Python scripts you can run and back, preserving
Markdown cells.

------
kankroc
Can someone provide me a usecase for this when compared to executing jupyter
straight into VSCode/Atom with vscodeJupyter/Hydrogen?

~~~
enriquto
The idea is that you can edit your notebooks with your favorite text editor,
use git for version tracking them, and no need to mess with a web browser _at
all_.

------
enriquto
This is fantastic!

Now, they just need to adapt jupyter itself so that it reads the python files
directly, dispensing with the need of the silly ipynb!

~~~
dfee
I think that’s called IPython. But I agree with your sentiment.

~~~
enriquto
I was not being ironic. Notebooks are lovely as an output format for showing
your research and allowing others to fiddle with your code. However, for
people who do not want to edit inside a web browser window, jupytext is a
godsend!

~~~
dec0dedab0de
At that point you should put most of your work in a separate file, and just
import the stuff you need for your demo.

~~~
PeterisP
In such a use case, the whole point is showing your work in a 'literate
programming' manner i.e. having the appropriate code that's being run be right
there between the descriptive text, instead of just running do_something()
that you imported from somewhere.

------
azag0
I have written
[https://github.com/azag0/knitj](https://github.com/azag0/knitj) to solve a
similar use case, but with an alternative interactive browser interface

------
xaedes
I really like the markdown conversion! This should make it easy to import
ipynb notebooks into markdown based documentations.

