
NumPy receives first ever funding, thanks to Moore Foundation - happy-go-lucky
https://www.numfocus.org/blog/numpy-receives-first-ever-funding-thanks-to-moore-foundation/
======
chollida1
Wow, I'm surprised that this is the first funding they've ever got.

It wouldn't be a big stretch to say that 90% of quantitative hedge funds use
Numpy in some fashion, whether its directly, or via a library that sits on top
of it like pandas or tensorflow.

I can't think of a more ubiquitous library in the financial space, maybe
QuicFix ([http://www.quickfixengine.org/](http://www.quickfixengine.org/))...

Maybe numpy's problem is visibility?

Possibly it does its job so well that people don't know they are using it when
they use library libraries like scikit learn and Pandas?

~~~
lordnacho
I didn't know it needed funding until I read this.

It's just an assumed resource in quant finance, like air or water. You do
realise you're using it, though. When you're using scikit or pandas it's very
normal to do "import numpy as np". And you get the odd np.nan reminding you.

Do people still use QuickFix? I dropped it years ago, it was noticeably slower
than alternatives when I tested it.

~~~
jzwinck
Tons of people still use QuickFIX. As I think you know, people very concerned
with speed try to avoid using any kind of FIX for trading but still use
QuickFIX for things outside the hot path such as drop copy (duplicate stream
of trades for reliability).

QuickFIX is open source too, so you can make it somewhat faster without
abandoning it. I did.

------
csaid81
It's great that the Moore Foundation provided funding for open source data
science tools in Python. Good for them!

That being said, I do wonder if numpy is the most appropriate recipient. In my
experience with data science, the tool that would benefit the most is not
numpy, but pandas. While data scientists rarely use numpy directly, every data
scientist I know who uses pandas says they are constantly having to google how
to do things due to a somewhat confusing and inconsistent API. I use pandas at
work every day and I'm always looking stuff up, particularly when it comes to
confusing multi-indexes. In contrast, I rarely use R's dplyr at work, but the
API is so natural that I hardly ever need to look things up. I would love if
pandas could make a full-throated commitment to a more dplyr-like API.

Nothing against pandas -- I know the devs are selflessly working very hard
hard. It's just that it seems there is more bang for the buck there.

~~~
abuckenheimer
If you look at the design documents for pandas 2 there is a good illustration
of how a lot of pain points in pandas 1 spring from numpy ( [https://pandas-
dev.github.io/pandas2/internal-architecture.h...](https://pandas-
dev.github.io/pandas2/internal-architecture.html)). I think any significant
development effort numpy would probably greatly benefit both libraries.

Will have to check out dplyr :) love to see how they master the magic that is
multi-indexes.

~~~
has2k1
In many cases, the use of multi-indexes in Pandas is (I think) a result of
culture/style or expectation that the cells of a dataframe should have scalar
values. If that would change and it became common to have nested dataframes,
the use of multi-indexes would diminish.

The tooling to support nested dataframes (and maybe even lists) is simple to
create, It can even be a third party library. I find that multi-indices though
may be an accurate conceptual way of thinking about certain data, they tend to
be practically more inconvenient than nesting the dataframes. In all cases I
have encountered only single level of nesting is required.

~~~
shoyer
If you're excited about non-scalar values in DataFrames, you should take a
look at xarray ([http://xarray.pydata.org](http://xarray.pydata.org)), which
implements a very similar idea in its Dataset class.

------
carreau
Just to note that if you know of anyone who is interested in working on NumPy
and potentially to move to UC Berkeley then tell them they probably should
contact Nathaniel – if NumPy got funding they'll likely hire
developers/community manager/technical writer ... etc . UC BIDS is a fantastic
place to work at, and Nathaniel is an extraordinary person to work with. I'm
going to assume there is also some opportunity for remote work.

------
rectangletangle
Really surprised there wasn't already funding for this.

Numpy is an amazing library, and it's basically Python's "killer app." The
fact that you can seamlessly blend numerical/data science computing with more
general web applications is what makes Python great.

------
gigatexal
Imagine if .1% of wall street profits from shops that use numpy were donated
to the project. Or some similar scheme for the other OSS projects used for
profit by large firms.

~~~
danjoc
Why would they do that? Numpy gave it to them. It's theirs now. "Screw you
nerds" is their opinion. Don't like that? Then dual license. Copyleft and a
commercial license. They can pay, break the law, or go leech off something
else.

~~~
Radim
The funny thing is, open source projects _scramble_ to offer the most
permissive license possible. It's either BSD/MIT or go home. I've seen people
shit on projects that use copyleft (GPL, even LGPL, etc).

Sounds a little schizophrenic to complain about the (lack of) money at the
same time?

I personally think this may be connected to the "academic" origins and mindset
of much OSS. Everything's "free" in that world, which can make a software's
transition into the world with real economic constraints and sustainability
challenges somewhat painful.

~~~
TeMPOraL
I would think that's because most open source projects don't start with money
in mind, so their first goal is users. And as a user I can tell you GPL or
even dual-licensing frustrates me, because it _limits my options_.

Consider: today I might be working on FOSS. But maybe tomorrow a friend asks
me to help him with his (small) business, by e.g. adding a little bit of
automation. Suddenly, all my knowledge and experience of GPL-ed libraries goes
to waste, as I won't be able to use any of that to help my friend.

Given two equivalent libraries, one on GPL and one on MIT, I'll always go for
the one on MIT. MIT, BSD, etc. seem to be the libraries that give you most
_options_ (I'd say freedom, but that's not how GPL sees freedom) while still
maintaining the integrity of the library itself. Those are the licenses that
best and at the same time satisfy the needs of developers who are not in it
for money, and users who don't want to waste their brain cycles on going
through possible legal scenarios around all their actual and imagined use
cases.

~~~
srean
> Suddenly, all my knowledge and experience of GPL-ed libraries goes to waste,
> as I won't be able to use any of that to help my friend.

Help me understand this please. In what legal way is GPL obstructing you as
opposed to say an MIT or a BSD license.

GPL license has noting against use in a proprietary setting, its only if
GPL'ed software is being sold/distributed, that it is required that the source
and the changes be made available as GPL. Google uses GPL'ed software all the
time and is far from the only one.

~~~
TeMPOraL
Last time I checked, the GPL, as opposed to LGPL or GPL with Classpath
Extension, basically forced your product to become open source.

~~~
srean
Only if you are distributing/selling the software to others. If you and your
friend are using it as a company internal tool, GPL has no issues with that.

------
ykler
I wonder what they plan to use it for. Numpy kind of seems finished already.

~~~
ris
That's not to say there aren't things NumPy doesn't do that it _could_. How
about lazy evaluation or even just matrix chain multiplication? Either could
save a _lot_ of computation.

~~~
shoyer
Matrix chain multiplication was added to NumPy recently, see
numpy.linalg.multi_dot:
[https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/...](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.multi_dot.html)

~~~
ris
Hooray! Still, it's not exactly transparent in a way that you could get with
lazy array evaluation.

------
thearn4
I could have sworn that Continuum had gov't funding for numpy development, but
maybe that was just for Blaze?

~~~
joejev
[https://www.continuum.io/blog/news/continuum-wins-darpa-
xdat...](https://www.continuum.io/blog/news/continuum-wins-darpa-xdata-
funding)

It was blaze and bokeh. The description of blaze given in the article is
pretty out of date at this point.

~~~
travisoliphant
Continuum co-founder here and previous NumPy developer. We received funding
for Bokeh and also explorations around scalable "NumPy-like" work. The efforts
around Blaze produced Numba, Dask, and Datashape as well as a more limited in
scope library called Blaze.

While we continue to develop those things, we are working on taking the core
ideas in NumPy and creating a set of lower-level C and Python libraries that
could be used by pandas, xarray, Numba, dask, arrow, and potentially NumPy:
[https://github.com/plures](https://github.com/plures). This work is not
directly related to the NumPy funding.

~~~
thearn4
Thanks for the clarification, Travis, that's pretty exciting to hear.

------
visarga
Are they going to make Numpy work on GPU? There is a library called Cupy (from
Chainer) that does that but not quite well enough. In fact on my attempt to
swap Numpy with Cupy, my program ran slower.

[https://github.com/cupy/cupy](https://github.com/cupy/cupy)

~~~
glup
If you are looking for performance optimizations beyond Numpy (GPU or CPU)
take a look at Numba by Continuum

------
metalliqaz
They've come a long way without funding. Good for them. Mathworks taking
notice, I'm sure.

------
brian_herman
You can donate here. [http://www.numpy.org/](http://www.numpy.org/)
[https://www.flipcause.com/secure/cause_pdetails/MzUwMQ==](https://www.flipcause.com/secure/cause_pdetails/MzUwMQ==)

------
marmaduke
Slightly off topic but you can use most of the Python stack from MATLAB since
version 2014b. The syntax is a little funny but it works well outside of ABI
mismatches in shared libs.

------
santaclaus
Wasn't Google funding the lead dev on NumPy for a while?

~~~
travisoliphant
No. I'm not aware of this happening.

NumPy built on Numeric which was primarily written by Jim Hugunin while he was
a grad student at MIT. While I was a professor at BYU, I wrote the the core of
NumPy with a lot of community input -- outside of my regular job. I sold a
book "Guide to NumPy" for a while that I used to replace the grants I should
have been writing and pay for a grad student to help write iterative solvers
for SciPy. Chuck Harris joined the NumPy effort early and has been steadily
contributing ever since without direct funding. Many others have contributed
volunteer time since then.

A major reason I started Continuum was to help create places where people
could get paid to write open-source. I am happy to say we have been doing this
for 5 years though mostly outside core NumPy itself (Numba, Dask, Bokeh,
conda, etc.) We are working to support many more open source projects more
generally -- and our devs have now made additional contributions to NumPy
itself. We have a thriving 40 person Community Innovation team at Continuum
supporting many open source projects. I expect this funding to help bring more
new people to the NumPy development ecosystem.

The community also started NumFOCUS at this same time to be a community-run
foundation that could be a focal point for donations and support to projects
including NumPy.

Nathaniel Smith wrote a great proposal and put the effort into securing this
funding. It is real work to secure funding. I look forward to NumPy getting
better for the benefit of all because of this work.

------
theprop
Congratulations!! Nice work...looking for lots more math libraries :-D!

------
digitsman
Does anyone have a link to the text of the proposal?

------
ahmedfromtunis
I really wish I could help!

~~~
arikr
What's stopping you?

------
gregjw
About time.

------
in9
$645020 is good for what? 4 jr developers or 3 slightly experienced
developers, working full time on numpy for 2 years?

~~~
oh_sigh
What's your point with this post?

~~~
xj9
free/open/libre software takes time and skill to create. more companies should
probably be paying some kind of support or at least donating since they are no
doubt using it to extract some kind of value from the market.

~~~
et2o
This is a charitable foundation that gives scientific grants, not a company.

~~~
xj9
my point stands. more public money should be going to free/libre projects too.

