
Jupyter notebooks in the IDE: VS Code vs. PyCharm - JA7Cal
https://towardsdatascience.com/jupyter-notebooks-in-the-ide-visual-studio-code-versus-pycharm-5e72218eb3e8
======
oefrha
Anyone else noticed that when editing a notebook in VS Code, one could lose
unexecuted edits to a cell with accidental arrow key input (that defocuses the
current cell and erases changes)? Too many times I have lost a paragraph worth
of Markdown text midst typing, it’s kind of maddening.

~~~
derriz
Yep - it's very annoying but for me it only happens with markdown cells? So
now I generally select all and copy now before leaving a markdown cell if I've
edited it.

The other issue is the language server seems to get into a funk regularly -
the completion stops working as does the display of docstrings so I regularly
use the "Developer: reload window" feature to reset everything.

Also the plots view is poor - it's basically just an SVG you can magnify -
there's no proper zoom or rotation for 3d plots.

I was also irritated by the "smart" environment discovery stuff - just let me
specify which python install or vent I want to use. Don't make be google to
see what names/locations I have to use in order for an environment to
magically appear in the list.

Also the editor extension (I use vi mode) is ignored. And even saving the
notebook doesn't work the way every other editor pane works in vscode.

Hmm... writing this list makes me wonder why I haven't looked at alternatives.

~~~
greazer
Disclaimer: I am the engineering manager for the jupyter support supplied in
the Python extension.

I LOVE this feedback! Keep it coming (though entering issues
[https://github.com/Microsoft/vscode-
python](https://github.com/Microsoft/vscode-python) would be ideal)

Yes, as conner4312 VS Code is investigating how to better incorporate a
juypter notebook experience within it, natively. As has been pointed out here,
there are a number of limitations that the python extension notebook support
has run into. We have been working closely with the VS Code team to find
solutions to these limitations. The custom editor work
([https://github.com/microsoft/vscode/issues/77131](https://github.com/microsoft/vscode/issues/77131))
is one such mechanism. We plan on rolling out our use of that work soon after
it is officially released (sometime in February). However, this will not
address all of the problems mentioned in this thread. Those issues will be
addressed as native jupyter support arrives in VS Code.

Here's some additional info: \- Find/replace. (Personally, this drove me NUTS!
Just ask my team! :) ) That being said a simple find (ctrl-f) is avaiable in
the current official release. Unfortunately, it does not include replace.
While we recognize that this is a severe limitation for some, implementing a
full find and replace solution would end up requiring the extension to
duplicate most of the functionality provided by VS Code itself. Therefore, we
currently planning to piggy back off of that work when it becomes available.
If that winds up taking an inordinate amount of time, we will consider further
stop gap work.

\- The markdown loss issue has been addressed and will be in the February
release.

\- For the language server problem discussed above, we need more info. I would
encourage you to enter a new issue at [https://github.com/Microsoft/vscode-
python](https://github.com/Microsoft/vscode-python). In the meantime, there
are known related issues that will be fixed the Feb release. One in
particular, was caused by the backend jupyter server inexplicably taking a
long time to return us a full completion list. So when you don't see
completion pop up, it may not be that it's broken, it may just be taking a
long time. We are working around this issue for the February release. Also, if
you're having docstring issues, you may find that turning off the python jedi
language service in settings will help.

\- Plots view. Yes, it is limited and honestly can cause rendering slowdowns.
We are working through ways of addressing both problems. A good solution for
supporting "proper" zoom will require adding ipywidget support overall, which
is coming soon.

\- "Smart environment stuff". I'm not following completely. Is this a general
python extension issue or does it have to do with the jupyter kernel selection
support we have for notebooks? Again, feel free to enter additional details
at: [https://github.com/Microsoft/vscode-
python](https://github.com/Microsoft/vscode-python)

\- Supporting global editor extensions like "VI mode", isn't something that we
can support without help from VS Code itself. One of the main reason's VS Code
is pursuing native notebook support is to be able to handle issues exactly
like this.

\- Saving the notebook weirdness will be addressed in the February release. We
were waiting for the custom editor support I referred to above to arrive.

Hope this is useful information!

~~~
derriz
Great response - I really appreciate the detail and also commend the tone you
use. It makes me a little worried that I came across as too negative earlier.

To balance that: I haven't forgotten the "WOW WHAT IS THIS" feeling I
experienced the first time opened an ipynb file from vscode when the notebook
support kicked in and asking my colleagues "did you know about this?" as I
excitedly started exploring the feature. Since then (and I'm a relative vscode
noob), I've never been tempted to go back to working with notebooks through a
browser. Despite my gripes, the overall experience is still miles better (in
my opinion) than accessing notebooks through a browser. It's the future for
sure.

And thanks for all those updates - I can't wait for the Feb release - and it's
very reassuring that these issues are not only well recogonized but also are
being aggressively addressed.

Regarding some of your feedback to the feedback:

\- Re. the language server issue - it could well be caused by a slow down.
I'll test that hypothesis the next time I experience it.

\- The "smart environments stuff" is, as you say, actually a general python
extension issue. But actually, I withdraw that complaint. I've just checked
and I can point the extension at any python interpreter by editing
settings.json - it's just the GUI which restricts you to selecting from a list
of discovered interpreters.

Thanks again for your response and your engineering efforts in general.

~~~
greazer
Glad to hear, derriz!

I think environment selection would be considered a bug. I theory any python
environment that's on your machine should be selectable in the UI. Could you
enter an issue at that same location? I can for you, but I'd prefer that the
issue comes directly from an outside user (sometimes we filter bugs based on
from where they came).

------
mstump
I'm surprised that nobody mentioned that find/replace doesn't work for
notebooks in VSCode. It's by far my biggest annoyance.

[https://github.com/microsoft/vscode-
python/issues/7903](https://github.com/microsoft/vscode-python/issues/7903)

~~~
greazer
See my reply below.

------
mweatherill
Microsoft have released another IDE with notebook support called Azure Data
Studio. It uses the same underlying code base as VS Code but is targeted at
data driven tasks such as connecting to databases.

Strangely, it uses a different jupyter notebook implementation to VS Code. I
found it avoids some of the annoying bugs mentioned in another comment.

It also introduces a SQL kernel which I’ve found useful for organising my
queries.

~~~
wayneftw
> SQL kernel

I've never heard this term before and a brief search for it did not bear any
definitions.

Can you expand on what exactly makes an SQL kernel?

~~~
sachdevap
Not GP, but I think they are referring to
[https://github.com/jupyter/jupyter/wiki/Jupyter-
kernels](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels)

------
taeric
Needs more mention of emacs and org mode... :)

I am curious to see the general workflow expanded on. I hesitate to think
there is a single correct way to keep a notebook. Seems flexibility is the key
feature. And is why text is so well used.

~~~
radarsat1
I haven't even gotten into org mode, I just edit Python using emacs and it
works great. I can evaluate parts of the code and jump to the interpreter when
needed. I keep a matplotlib window open with plt.ion() when running locally,
or use plt.savefig() when running remotely and have a qiv window open that
automatically detects new images. Works great.

The only thing I would like is to be able to use the Python interpreter
_within_ the context of a function sometimes, it's annoying when an error
occurs inside a function and I can't access the local variables.

I like jupyter notebooks too, but one of the things stopping me is that I
can't use emacs features for editing ;)

~~~
cjbillington
To interactively inspect within a function, you could use the debugger with
`breakpoint()`. But it's not a regular Python interpreter, which annoys me
having to learn the different syntax. You can also embed IPython interactively
with:

    
    
        import IPython; IPython.embed()
    

though IPython kinda doesn't clean up after itself, so it's a bit messier
(your prompt is messed up afterwards).

Finally, you can:

    
    
        import code; code.interact()
    

The downside of this is there's no history or autocomplete by default.

I have a custom function `embed()` [1] that calls `code.interact()`, but with
history and autocomplete configured. This is the closest I got it to looking
like a regular Python interpreter at an arbitrary place in the code.

[1] [https://pastebin.com/WDuheBfV](https://pastebin.com/WDuheBfV)

~~~
radarsat1
Thank you! I didn't know about code.interact, but IPython.embed I have used in
the past. My point was more that I would like a way to drop to an IPython
prompt when my program crashes, I really don't like the solution of having to
modify the code to insert a statement where I _think_ my program is _going_ to
crash.

In any case I think there was a way to examine variables from up the stack of
a stack trace or something like that.. bit too lazy to look it up right now.
Regardless, working with a REPL in emacs is pretty great ;)

I think there is a way to do jupyter from emacs (or at least there was a way
to do ipython notebooks), but I haven't used it extensively, just tested it
once I think. I guess I'm pretty satisfied with the REPL and haven't found a
need to have intermediate output during the running of a program.

What I do like about the matpotlib interactive approach is being able to watch
the progress of a loop very easily, which is hard to do from a notebook,
although there are some ways, but they are either hacky or require some pretty
sophisticated things like custom widgets.

~~~
cjbillington
Yeah I considered extending it to automatically pop into the frame where the
exception happened, but this is actually not as useful as it sounds, since the
exception is usually much deeper in some other library and not in your code
where you actually want to inspect variables. You would either need to declare
the boundary between 'your' code and other code (by filepath or something), so
the tool could know where it should run, or you would need to include commands
to push and pop through the stack, which complicates things a bit and is on
the path to a full debugger (so perhaps one should just use the debugger - as
you've mentioned in the sibling comment).

So I just add a try: except around code known to be crashing and call my
`embed()` function on except. This has served me well enough that I haven't
bothered with anything else.

Plus, if I really want to inspect unplanned crashes, I would want to be able
to get them when running, and not just developing, the app. So there might not
even be a terminal - there are more complications than just 'not having to
declare where you think it will crash', so I don't think I'll bother solving
the more specific problem unless it would help me debug unexpected crashes in
a broader context.

------
swiley
Why are you using a notebook _and_ an IDE? I thought the notebook UI was an
alternative to the IDE UI.

~~~
skwb
Frankly I HATE web based notebooks. I do deep learning research, and I write a
lot of complex functions to handle pre-processing. And with that there's a lot
of debugging I need to do, particularly visual debugging of images. The
interactivity of notebooks and the features of an IDE has made my programming
experience much better.

~~~
_coveredInBees
I'm curious, what interactivity do you enjoy about notebooks that you find
missing in IDEs? I find my situation to be quite the opposite. I love
developing/writing code in Pycharm because apart from all the things it excels
at (introspection, auto complete, linting, documentation, debugging, etc), it
lets me easily prototype things in the interactive python interpreter (usually
set to IPython), has an excellent variable viewer and even allows you to
attach a debugger to your python interpreter and debug things on-the-fly.

I find myself able to prototype code a lot faster in that setting compared to
a Notebook. Even with plots, plotting in Pycharm provides me an interactive
plot I can zoom around in, which is not the default behavior in Jupyter. The
only situations where I ever use a Notebook is when I need to share a Python
"document" that isn't just a python script + results, but rather some form of
extensive self-documenting code that could benefit from Markdown integration
and co-location of output plots, etc. The other situation is some of their
built in animation/interactive widgets (perhaps that is the interactivity you
are referring to?). In most other cases, I've found it quite limiting compared
to a good IDE like Pycharm/Code.

That being said, I think the efforts on Pycharm and Vscode's part to bridge
this gap is commendable and quite interesting.

~~~
sansnomme
Microsoft Visual Studio level of Edit and Continue has never been well
implemented in open source tooling (with the exception of certain Common Lisp
implementations that predate even visual studio, I suspect Turbo Pascal might
have this feature too but I have never really used it). This is why most
people have to resort to hacks like notebooks and repls with bad UX that spend
most of the time spewing errors at the user. It is starting to come back with
Flutter, React, and all the livereload stuff but you need a ton of work to get
it to work on Python.

See:

[https://news.ycombinator.com/item?id=16289856](https://news.ycombinator.com/item?id=16289856)

[https://news.ycombinator.com/item?id=21646240](https://news.ycombinator.com/item?id=21646240)

~~~
pjmlp
Turbo Pascal never had it, which barely mattered as it was already damm fast
on a 4 Mhz MS-DOS PC.

Not sure about Delphi though.

However, I would also point to Java, Smalltalk, Eiffel as another set of
examples.

------
OldGuyInTheClub
Spyder has a Jupyter notebook plugin. I was able to get a notebook going and
link the kernel to the IDE. The variable editor worked and I could debug code
in Spyder and then paste it into the notebook. The notebook piece of it was
good for graphics, widgets, markdown, etc.

~~~
CinchWrench
Is spyder still super slow? I gave it up back in 2018 because of the
(perceived) performance problems.

~~~
OldGuyInTheClub
Hard to say as I am not a power-user/coder by any standard least of all those
of this forum. I've tried Python on and off over the years and the Spyder
3.x/Jupyter combo was the closest I could get to the kind of UI that Matlab
offers: Variable inspector, debugger, persistent/docked plots, documentation,
recordkeeping, etc. It wasn't clear whether the Spyder/Jupyter plugin was
going to be maintained in future Spyder releases and the combo, although
close, was not as smooth as I would have liked.

Ultimately, Matlab is a better fit for the things I want to do in technical
computing. My employer offers it and Simulink with a wide variety of toolboxes
and blocksets so I can get farther faster for the problems that interest me or
that I am asked to solve. The new table datatype and the steadily improving
Livescript make exploration and notetaking very easy. I've also been able to
rely on Mathworks documentation to get up-to-speed in new areas.

------
JA7Cal
Also available at
[https://github.com/mwouts/notebooks_in_vscode_and_pycharm_ja...](https://github.com/mwouts/notebooks_in_vscode_and_pycharm_jan_2020)

------
robenkleene
I'm trying to implement these same features in my app, Repla, but without
requiring an editor plugin, and instead having a view in a separate app that
automatically updates via file-system events when you save. There's some more
information here [https://repla.app/live-coding.html](https://repla.app/live-
coding.html)

It's an approach that I think has somes pros and cons relative to integrating
these features into an IDE. The main argument for moving them to a separate
app, is that IDEs are already really complicated.

~~~
tvb12
I'm not on macOS, so I can't try it right now, but I've bookmarked this and
will keep an eye on it. Basically the report/output view of Iodide/Jupyter
without the "editor" view? I was really hoping something like this would come
along.

~~~
robenkleene
Yes, that's exactly right. Great to hear other people are also interested in
this approach! The technology I developed has ended up being pretty flexible,
you can read about it here ([https://blog.repla.app/2020/01/13/what-is-
repla/](https://blog.repla.app/2020/01/13/what-is-repla/)) and here
([https://blog.repla.app/2020/01/13/repla-use-
cases/](https://blog.repla.app/2020/01/13/repla-use-cases/)) if you're
curious. But yeah, that's exactly the use case I developed the technology for.
I wanted a browser engine with a plugin system so that I could use to display
output of systems like Jupyter and Observable HQ, while still editing the
source code in my exiting text editor, the same way I edit other source code.
I sort of discovered along the way that this system is also a nice way to run
a local development web server, and that was a bit easier to implement, so
that's what I've launched with first, the homepage has more info on that
([https://repla.app/](https://repla.app/)).

I'd like to support Linux and Windows eventually too, but it's easier to start
with supporting just one platform while I'm just testing out whether there's
other people interested in the idea.

------
jonnycomputer
I see this, and I cannot help but think, hey, one step closer to achieving the
plain-text goodness magic of the R Notebook. But maybe I'm being too sassy.
Its just that every time I pick up Jupyter I think, what is this clumsy thing?
I know people love it, but that's how I feel. I wish pycharm had the in-line
capabilities of the R Notebook, because I really like pycharm.

------
MayeulC
KDE's Cantor isn't too bad either, but I am not sure how well it interoperates
with Jupyter notebooks.

~~~
ptrott2017
Overall its pretty good but some known issues. See:
[https://sirgienkogsoc2019.blogspot.com/2019/08/cantor-and-
su...](https://sirgienkogsoc2019.blogspot.com/2019/08/cantor-and-support-for-
jupyter.html)

------
andykx
I don’t do data science and I really don’t understand what Jupyter notebooks
are used for. Are they just a way to explore ideas with immediate feedback?

~~~
fulafel
You can publish them on github or mybinder as persistent self documenting
interactive repl sessions. Then others can see what the code did, the
intermediate results, and can easily play with it.

(No "data science" here either, but into repl based instant feedback
development & collaboratiove programming tools)

~~~
andykx
Oh, very cool. Sounds like something I’d love to use. I always loved the
Clojure REPL.

~~~
gimagon
Notebooks also inline visualizations more naturally than the repl, which makes
them really nice for the data analysis workflow.

------
codydh
This misses a pretty stellar entrant in the field, Atom + Hydrogen. Works
nicely with multiple kernels (I use R and Python), allowing notebook-like
functionality across multiple languages.

~~~
Jedd
What do you mean misses? The author mentions that exact combination in
paragraph 4.

~~~
codydh
You're right, I missed that, thanks for pointing it out. I wish it had ranked
high enough for a comparison though, seems to solve some of the issues raised
(though perhaps causes others).

~~~
figo22
I strongly second the use of Atom + Hydrogen.

------
teddyuk
I can’t help thinking that notebooks would be less relevant if people embraced
unit testing more

~~~
prepend
I don’t think they are very related, aside from both being useful.

I use primarily for doing stepwise development. I also use them for a decent
way to see the output of my code, usually tables and charts.

I have unit tests for the various modules and classes created.

