Hacker News new | past | comments | ask | show | jobs | submit login

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book.

Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming idea where prose takes precedence over code, which is usually the reverse of the way we usually program; code first, comments second.

Item #2 Don't use notebooks for general purpose programming. Notebooks are supposed to have an audience and clearly explain something.

Item #3 Keep code cells simple and clear. If needing a comment in the code block, consider putting that verbiage in a markdown cell instead and elaborating on the idea the notebook is trying to convey.

Item #4 Don't make notebooks a long series of extended code cells, or even worse, just one long cell. Explain what is going on or see Item #2.

I disagree.

While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing.

Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply:

* I'm not quite sure what or how.

* It will likely be a one-off

* I need it now.

* Someone is watching me, to learn how I do it.

I would go a step further and say that notebooks should probably not be shared directly most of the time, and that you should wrap up functionality into modules/packages for other people to use in their own notebooks.

Edit: Just thought of a corny pun to make my point: They are notebooks, not textbooks. Notebooks are personal, and while it may be useful/insightful to compare notes, it's not the primary function of a notebook.

Agreed. I've been increasingly doing that at work with org-mode. Sometimes I'll just open up a source block under a TODO item, other times I'll create a new heading or a new file for some stream of thoughts mixed with code blocks in various languages. It feels better than doing it in a pure source code file, with my thoughts noted as comments (though I do plenty of that too).

Indeed, when the alternative is the interpreter, the structure notebooks offer and the (loose) process they encourage makes for a vastly better solution for recording and capitalizing on your work.

I agree with with your disagreement. I've always approached notebooks as a prototyping tool first and a presentation tool second. I see Jupyter more as an example of an early Interactive programming tool that just happens to be useful for presentation and teaching purposes. (https://en.wikipedia.org/wiki/Interactive_programming)

And regarding sharing -- particularly with notebooks you should be really cautious about sharing and opening them. The security model is not exactly bullet proof.

> I see Jupyter more as an example of an early Interactive programming tool

Nitpick, but I hope you meant "young", not "early". Interactive programming is as old as (and today still primarily featured in) Lisp - i.e. twice as old as most of us here on HN.

While I generally agree with everything you say for a particular audience, i.e., someone who is writing something to explain an idea to someone else, I think that at the same time this scoping limits the utility of Jupyter too much.

I believe that there is an audience for Jupyter beyond just "data scientist", or "teacher". I think that what Jupyter encourages is _experimentation_, and there are lots of folks who could benefit by using Jupyter to experiment with ideas. If you consider programmer as another audience, we all at different points in time experiment with things. Think about all of those random console applications that we've all created at one point or another to learn a new API, or to try out an idea. Jupyter can (and does) excel in these scenarios and handles the mundane task of "writing things down". This is why adding a Jupyter experience to a text editor is something that we're experimenting with as well.

I disagree massively with this, to me Jupyter is a repl+ development environment + rapid prototyping in ETL/visualisation. There's no rule that they have to be used for exposition, or any writing at all. I don't know why you think this is a rule, or what you'd use as an alternative data-repl.

For example Peter Norvig does an exemplary job with jupyter notebooks for expository writings: https://github.com/norvig/pytudes

See also "computational essay"[1]. It is written with Mathematica notebooks in mind, but many of the ideas carry over to Jupyter notebooks.

[1] https://blog.stephenwolfram.com/2017/11/what-is-a-computatio...

and a more jupyter-centric agree/reply by Tony Hirst:


For #2, I have used notebooks for personal data analysis. I am my own audience and need to remember what I did and why.

This is a really good idea, and I've been planning to write said guide for a while.

I think there is a lot to do with organising and formatting the notebooks, and the %run command works a treat for me when breaking a project into multiple notebooks

Check out: Ten Simple Rules for Reproducible Research in Jupyter Notebooks, https://arxiv.org/abs/1810.08055 and https://github.com/jupyter-guide/ten-rules-jupyter

While I don't follow #2, I suggest two more item:

Take the time to learn some good programming practices. If you're lucky and fall in love with scientific programming, your programs will grow to a point where they are unmanageable without some techniques borrowed from computer science and software engineering.

Have someone show you the "out of order execution" and "hidden state" failures.

Obligatory Joel Grus: slides from his 'I don't like notebooks' talk. He has good follow-up talks too.


...and a well-written response to that by Yihui Xie: https://yihui.name/en/2018/09/notebook-war/

Since Xie's post discusses a "meta" layer above Joel Grus, I recommend people read it before JG. This actually helps one understand the underlying thinking in JG's slides.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact