SoS Notebook relaxes this restriction and allows us to use a different kernel for each cell of the notebook so that we can use the most appropriate language (tool, library etc) for each step of the analysis. More importantly, SoS Notebook provides a mechanism to transfer variables among live Jupyter kernels so that we can, for example, clean data in Python, analyze them in R or MATLAB, and plot the results in JS.
We have also tried to improve the Jupyter frontend to create a more comprehensive work environment for interactive data analysis. For example, SoS Notebook provides a side panel that allows you to execute cell content line-by-line using shortcut Ctrl-Shift-Enter. It also provides magics to, for example, render output from any kernel in Markdown or HTML, and clear non-informative output after the execution of the cells. We are very excited about our work and would really love to get your feedbacks.
Maybe should look at collaborating with BeakerX:
I personally prefer the legacy "beaker notebook" which did similar things in (I think) a nicer interface than Jupyter.
I agree that all other notebooks (e.g. BeakerX, Zeppelin, R Notebook) have nicer interface than Jupyter but Jupyter excels at its simplicity and JupyterLab (to which SoS Notebook will be ported eventually) is making great progress there. With the frontend enhancement that SoS Notebook provides, especially the line-by-line execution feature, we are pretty satisfied with the frontend and do not really miss the fancy frontend of other notebooks.
1. Extended from Python 3.6 to make SoS an easy and yet powerful workflow language.
2. Embedding workflows in SoS Notebooks allows you to annotate the workflows with detailed descriptions (markdown cells) and results (of demo runs).
3. Supports both forward (sequentially numbered) and makefile style (patten matching) workflows.
4. Execution signatures to avoid re-execution of long steps.
5. Magics to execute workflows in SoS Notebooks so you can, for example, execute cells of a notebook conditionally and repeatedly.
6. A task system that sends parts of workflows to remote host (a more powerful workstation), cluster (PBS/Torque/LFS/Slurm systems), or RQ task queue, even if the remote systems are on different file systems (SoS automatically maps paths and synchronize files).
However the idea of multi-kernel is useful.
I am going to have to try this out.
Can you explain what is the architecture of SoS kernel?
The architecture of the SoS kernel, if we ignore the SoS workflow engine part, is just a Python3 kernel that sits between Jupyter and all other kernels. The SoS kernel starts the subkernels, collect user inputs (interpolate them if needed), send them to the subkernels, and collect and display outputs from the subkernels. Data exchange between subkernels are implemented by executing hidden statements in SoS and subkernels, with assistance from SoS language modules for supported languages. This simple setup introduces minimal changes to Jupyter users: they can use multiple languages in a SoS notebook but can also use the SoS kernel as a wrapper to their kernel for an improved frontend, and they can enjoy all the tools that Jupyter provides (e.g. JupyterHub, template, conversion tools) with SoS Notebook.
Finally, the name SoS Notebook comes from the fact that it is a frontend to the SoS (Script of Scripts) workflow engine. We know it does not sound great and places SoS well behind the real "SOS" in a google search, but let us just hope that some day SoS would appear in the first page when you search for SoS. :-)
2) Will it support `JupyterLab`?
2). That is certainly on our radar after JupyterLab matures and provides a stable API. The core of SoS Notebook (namely the data exchange model) should be easy to port but the entire frontend would need to be rewritten.