
SciPy 1.0: fundamental algorithms for scientific computing in Python - Anon84
https://www.nature.com/articles/s41592-019-0686-2
======
dragonsh
> Our policy for evolving the API over time is that new functionality can be
> added, while removing or changing existing functionality can only be done if
> the benefits exceed the (often significant) costs to users and only after
> giving clear deprecation warnings to those users for at least one year. In
> general, we encourage changes that improve clarity in the API of the library
> but strongly discourage breaking backward compatibility, given our position
> near the base of the scientific Python computing stack.

This is one of the nuggets out of it. I have been using Python with scipy and
more extensively with it's sci-kit library since 2014. Started with it by
solving discrete traveling sales man problem for solution of sitting
arrangements and was astonished with its ease of use of matrix algebra with
these libraries and got hooked to it. It helped me to do scientific computing
and create a user friendly user interface on web to help end-user to just
focus on siting arrangement and conference Organization. Can solve it with
Matlab or GNU octave, but couldn’t have developed easily the whole system with
user interface in them.

If Swift wanted to be used in scientific computing it needs to follow the
similar policy of managing its scientific stack.

Julia is good for scientific computing, but it’s UI and web development
libraries are isn’t that powerful. So will see how it develops.

I constantly see on HN criticism of Python and something rewritten in Rust as
something revolutionary or ground breaking that it will change the whole
world. Earlier this fad was with go language. Probably in time to come Rust
will be used for some core library development which brings substantial
benefits like scipy, not something rewritten as wrapper around C library with
unsafe code which can be done with Python much better.

~~~
gigatexal
That discrete traveling salesman problem of seating arrangements ... how did
you model that? I have a similar problem with organizing committees where
committees have members that are in other committees and so committees with
overlapping members can’t meet together etc etc.

~~~
bhl
Not exactly TSP, but something similar on solving seating arrangements: it’s
with simulated annealing; it might help with formulating your optimization
problem.

[https://news.ycombinator.com/item?id=16548012](https://news.ycombinator.com/item?id=16548012)

~~~
gigatexal
Danke!!!

------
maxnoe
Shameless plug:

Here is a grand tour of scipy as a jupyternotebook me and a colleague used as
intro to scipy for PhDs in astrophysics:

[https://github.com/maxnoe/scientific_python_notebooks](https://github.com/maxnoe/scientific_python_notebooks)

You can try it without installing the software by clicking on the binder
button.

~~~
complex_pi
There is also [https://scipy-lectures.org/](https://scipy-lectures.org/) :-)

------
tobmlt
SciPy, and especially numpy: the “free drugs”of the scientific computing
world. In the best possible way.

~~~
enriquto
> (...) In the best possible way.

I do not see it that way. A whole generation of students is growing with the
wrong impression that writing a "for" loop is inevitably inefficient. Also,
they believe that it is OK for large arrays of numbers not to be a core
language construct, requiring an external library like numpy. These two
incorrect beliefs are incredibly damaging in my view.

In a sane programming environment (e.g., with jit compilation), writing a
matrix product using three loops should be just as efficient than calling a
matrix product routine. The matrix product should also be rightly available
without need to "import" anything.

~~~
visarga
Well, you got to think about optimisation - loop unrolling, parallelisation on
all available cores, efficient data layout that works well with caching, doing
it on GPU, adapting to the type of GPU you have. It's not as easy as dumping
three for loops.

~~~
enriquto
The fact that you write three nested loops on your program does not mean that
the compiler/interpreter cannot do all these nifty things when running the
code.

~~~
zo1
You're talking about the wrong language, then. This is not c-python (or python
too perhaps) if you want such ridiculous compiler-level optimizations. If you
want the languages for loops to be almost on-par with the hyper-optimized
implementations being utilized in numpy and the layers below it then you
should use something like C++ and hand-roll those for-loops yourself.

The reason python is so amazing in this regard is because that has all been
done for you on some level and you just need to use them. Sure, could the
numpy/scipy interfaces into lower-level code be more closely aligned to plain
for loop implementations of the algorithms they represent? Perhaps.

------
stelfer
I remember going to the first SciPy conference at Caltech in 2002. IIRC Most
of the talk was about needing better package management. What happened since
is just amazing.

------
ddavis
Congratulations to all involved with this paper, an awesome accomplishment for
the SciPy community-- pretty wild that it's almost 20 years old.

~~~
timkam
Good that they got a paper in Nature. It's important such accomplishments get
rewarded and acknowledged in the traditional academic way. I hope these people
pile up citations. The engineers who build the foundations of modern
scientific research get far less credit than they deserve.

~~~
enriquto
Indeed! This publication is very good news, and I am happy that free software
acquires such a prominent place in the academic citation space.

On the other hand, notice that scipy is in large part an interface. The
underlying solvers (e.g., numpy.linalg.solve) are often routines written in
Fortran several decades ago, and already duly cited and academically
recognized.

~~~
mattip
There is tremendous value that often goes unrecognised in taking best of brand
routines and packaging them so real people can use them, consistently and
reliably over decades.

~~~
enriquto
Sure, that's why people keep doing that: taking the same exact Fortran routine
and giving the world access to it from whatever different languages and
frameworks are fashionable.

------
neonate
[https://web.archive.org/web/20200207035109/https://www.natur...](https://web.archive.org/web/20200207035109/https://www.nature.com/articles/s41592-019-0686-2)

------
mud_dauber
I learned about SciPy via Numerical Python (Apress/GitHub:
[https://github.com/apress/numerical-
python](https://github.com/apress/numerical-python)). That was a ton of fun,
especially after I recovered from freshman calculus flashbacks.

------
valgor
Great read!

I'm ignorant of numerical computing, so I have to ask: is Fortran still used
because it is that much faster than C or any other specialized language? I
assume that is the case because I'm sure someone would have rewritten those
routines by now otherwise.

~~~
coliveira
Fortran still is the language for numerical computing, most algorithms in this
area have been implemented in Fortran. And this is very specialized code that
has been test by experts in their areas. That's why a project like SciPy
cannot simply replace it with something written in C or Python.

------
xianwen
May I ask what the status of parallel computation is with SciPy?

~~~
fock
OpenMP threaded blas-routines

------
nrclark
Isn't Scipy on 1.4.1?

~~~
geoalchimista
Because it took half a year for the paper to get accepted. You can see that
the manuscript was sent to Nature Methods on July 28, 2019.

~~~
philshem
For the record, July 2019 is still at least 18 months after the release of
SciPy 1.0 (late 2017)

[https://docs.scipy.org/doc/scipy/reference/release.1.0.0.htm...](https://docs.scipy.org/doc/scipy/reference/release.1.0.0.html)

