
Numpy Clone in Common Lisp - lelf
https://github.com/numcl/numcl
======
gumby
Nice work: not a skin over the python one but an actual, optimizable
implementation.

~~~
varjag
I concur, first rate work and good use of the language.

~~~
pmiller2
I think this is great and interesting, but do CL compilers optimize numerical
code anywhere near as well as Fortran compilers?

~~~
gumby
I would be surprised if so for vector arithmetic given the opportunities for
modern FORTRAN to tell the compiler what to do. OTOH the whole point of
metasyntactic macrology and the function object abstractions is to permit such
things. But modern FORTRAN compilers are very good at what FORTRAN is good
for.

The MACLISP compiler was as good and in some cases better than its
contemporary FORTRAN competition. But FORTRAN tech has moved on since then.

------
jxy
IBM doing Common Lisp. I wonder what this is actually used for inside IBM.

~~~
guicho271828
Author here. IBM is currently focusing on the combination of machine learning
and reasoning, also called neural-symbolic AI [1], to combine the best of both
worlds (noise robustness and explainability, etc.). I am from Tokyo and will
join MIT-IBM lab soon to perform this line of research.

I have been working on Latplan [AAAI-2018, 2,3,4,5,6], a neural-symbolic
classical planning system that learns symbolic representations of noisy images
and perform efficient symbolic planning in the latent space. Later, several
other groups have started working in this field, like [7].

The problem with python is the lack of _sequential execution speed_ , which is
necessary for solving symbolic tasks, e.g. SAT solver, classical planning,
theorem prover. At the same time, traditional languages like C++, typically
used for writing solvers, are too inflexible to accommodate the workflow in
machine learning. For a neural-symbolic task, personally, Lisp is best in this
space (I don't try to repeat why, because there are plenty of articles on
it.), with Julia being a contender.

I am also interested in adding the compile time type/shape checking on the
deep learning code, since python debugging is painful (just a personal opinion
-- not intending to start a war here).

By the way, I made the first prototype of this lib in 3 days, and am working
on improving it for 3 month (for other ML projects).

[1] [https://medium.com/@MITIBMLab/the-shared-history-and-
vision-...](https://medium.com/@MITIBMLab/the-shared-history-and-vision-
behind-the-mit-ibm-watson-ai-lab-405ce4032013) [2]
[https://github.com/guicho271828/latplan](https://github.com/guicho271828/latplan)
[3] AAAI18 presentation at : [https://asai-research-
presentation.github.io/2018-2-5-aaai/](https://asai-research-
presentation.github.io/2018-2-5-aaai/) [4] AAAI18 paper : Classical Planning
in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary,
[https://arxiv.org/abs/1705.00154](https://arxiv.org/abs/1705.00154) [5]
Follow-up papers (ICAPS2019) :
[https://arxiv.org/abs/1903.11277](https://arxiv.org/abs/1903.11277) [6]
Follow-up papers (ICAPS2019), extension to first order logic :
[https://arxiv.org/abs/1902.08093](https://arxiv.org/abs/1902.08093) [7]
Learning Plannable Representations with Causal InfoGAN,
[https://arxiv.org/abs/1807.09341v1](https://arxiv.org/abs/1807.09341v1)

~~~
_ph_
Is there a specific reason you choose LGPL for the library? Are you aware of
the issues to use that license with Common Lisp libraries? Common Lisp doesn't
use shared library linking as C does, so some terms of the plain LGPL are not
directly applicable to Common Lisp, Franz tries to compensate for this with
the LLGPL, but it still more difficult to use correctly compared to a more
permissive license.

~~~
kazinator
Loading a Lisp .fasl is very much like loading a shared library. (Shared
library loading can be regarded as a Greenspunned version of what goes on in
Lisp.) There is code in there, referenced by symbols, which resolve. _dlsym_
is like a hacky, low-level version of _symbol-function_ or _symbol-value_.

C header files also have a ready analogy in Lisp: they are like Lisp files
that provide macros and constants.

Where the LGPL says "The object code form of an Application may incorporate
material from a header file that is part of the Library." we can understand
this "header file" to refer to a Lisp file that provides macros that generate
code within the Application's file.

The LGPL doesn't define what is a "header file"; it doesn't say that it's a C
thing with typedefs and #defines. In that regard it has a flaw whether we
apply it to a C shared library to a collection of Lisp object files.

~~~
_ph_
I know how Lisp loads files (disclaimer, I am professionally using Common
Lisp) and consequently are aware of the LGPL issues. First of all, does LGPL
only allow using fasl files or also building applications? Can you create a
fasl file of a modified library and load it instead? Especially, if the user
does not have the source code to the rest of the application?

The answers depend on the Lisp implementation used. There is a good reason
that Franz decided to augment the LGPL (at their time version 2) with a
license preamble to clear up the ambiguities in the context of Common Lisp.

~~~
armitron
There are no ambiguities in LGPL. "Franz did LLGPL" is not by itself a good
enough reason to suspect there are or to use LLGPL.

~~~
_ph_
_There are no ambiguities in LGPL._

You are a lawyer? You are a professional Lisp programmer? How does the LGPL
deal with Common Lisp macros?

 _" Franz did LLGPL" is not by itself a good enough reason to suspect there
are or to use LLGPL._

As Franz is the most prominent Lisp vendor, employing some of the greatest
Lisp programmers and undoubtfully having a good legal department, I see no
reason why I shouldn't take their legal opinion seriously. Especially as it is
not part of any commercial offerings of theirs, but just provided as a clearer
license for Lisp libraries. It also coincides with everything I heard from my
lawyers at the company I work for.

~~~
jackdaniel
You don't have to be a lawyer to read legal documents and use licenses. There
is no reasonable interpretation of LGPL in Lisp context which would lead to a
need of additional clarifications.

Here is my stance on this: [https://common-lisp.net/project/ecl/posts/ECL-
license.html](https://common-lisp.net/project/ecl/posts/ECL-license.html).

TL;DR if someone wanted to have Lisp code to be licensed under GPL terms they
would license it that way. I'm a professional Lisp programmer and I've spend a
fair share of hours on reading licenses and analyzing them.

~~~
_ph_
This might work for ECL which is build as a shared library. But what if e.g.
an SBCL user wants to use a LGPL Lisp library? How can the developer provide
the ability to "link" against a modified version of the LGPL library? What
about macros exported by the library? The LGPL talks about the szenario of
linking shared libraries. This is a clearly defined process in the C world,
which has no exact counterpart in the Lisp world. You claim that if the author
of a library had other intentions, the author would have chosen the GPL. That
might be true, but certainly isn't sufficient for legal certainty. Perhaps the
author wanted allow only shared libraries where it is possible. The LLGPL is
an easy amendment to make those intentions clear. Even easier, and better
suited for Lisp environments (even following the LLGPL to the letter can be
technically difficult) are licenses like BSD, which in this case would be the
obvious one in my eyes, as Numpy itself is under the BSD license.

~~~
jackdaniel
> This might work for ECL which is build as a shared library. But what if e.g.
> an SBCL user wants to use a LGPL Lisp library?

If you had read the post you'd have my answer to that question.

There is no need to spread fear and uncertainty about LGPL and other copyleft
licenses, I've seen too much of that over last few years (and I have a strong
intuition that it is not an accident).

P.S. "are you a lawyer?" argument is very cheap (and often happening on
threads about copyleft licenses) given it is HackerNews not LawyerNews -- I
know of lawyers working for corporations who say that they wouldn't touch
anything with GPL in the license name with a five-meter pole (not knowing a
difference between LGPL, GPL and AGPL) -- there goes your "in-depth legal
analysis". Same lawyers doesn't bat an eye when they give a green light for
eula-crippled code from their "technical partners". Cargo cult is present
everywhere, not only in software development.

~~~
_ph_
_If you had read the post you 'd have my answer to that question._

I have read your post, and just reread it, and I don't see where you answer
that question. How does an SBCL user deploy a program with an LGPL library so
that the reciever of the program can relink it as required by the license.

 _P.S. "are you a lawyer?" argument is very cheap_

I used this question on a post where the poster just makes a claim, not
supporting it by any reasoning or linking to supporting documents. The
question must be allowed how he justifies his statement. So for legal
questions, this would be: are you a lawyer? as I would ask a random poster
making definitely medical statements: are you a doctor?

And yes, I know, beeing a lawyer per se doesn't guarantee a correct
evaluation, I have worked with far too many corporate lawyers, which are
usually not experts on open source licenses. But I also had no doubts on the
competence of those, who looked into the details of the licenses in question.
I have been able to get their OK on LLGPL software for example, though GPL is
of course limited to very isolated usages.

------
agambrahma
Good to see someone fill in this missing link! Could become a viable
competitor to Julia (uniform language stack, as opposed to the Python+C
combination currently being widely used).

~~~
ddragon
I think it's more than a uniform language stack, after all C++ could be
considered one as well but most people still use python to interface it as
opposed to just using it (at least for exploratory data science). It's a
combination of interactivity to experiment, conciseness to stay close to the
pseudo-code/math, easy hackability since you need to stay beyond the cutting
edge and performance so you're not restricted in what you can do.

Common Lisp (as does Clojure and Julia to a lesser extent) has unparalleled
interactivity (besides maybe smalltalk), you can make it as concise as you
want, ultimate hackability and really good performance. Research (not
deployment) is the one area CL could probably surpass every other, but
unfortunately it lacked a concentrated effort (by a company or community) in
this current AI resurgence (maybe because of the Lisp Curse). Though there
will always be other opportunities.

------
_0ffh
It's a bit sad that Lush seems to be kind of dead since YLC has moved on...

~~~
eggy
I agree. Lush 2 was very promising. On the bright side, now Lisp is back in
what seems a good fit of research and practicality.

------
fartcannon
This is an excellent project. Is there a developer blog? I'd like to try
reimplementing numpy I'm various other languages (or part of it, since it
seems like it's a huge task).

~~~
enriquto
You can re-implement a much better interface that numpy if your purpose is
doing math comfortably. Numpy has a wonky interface to ease the transition
from matlab/octave (while amazingly producing a much less comfortable system).

~~~
nemoniac
Could you point to a "much better interface" to numerics?

~~~
enriquto
Well, for one, octave or scilab are perfectly accetable solutions. Or even
"julia" if you pressure me.

Or fortran, where there is no stupid overhead in "for" loops. Or C. Hell, or
even luajit if you do not need fancy stuff.

Honestly, python/numpy is possibly the slowest and less convenient way to do
math (assuming that you _only_ want to do math, and no need any
strings/dictionaries and other useless data structures).

~~~
mkl
With the possible exception of Julia (which I don't know), all of those
languages have much clunkier and more primitive numerics interfaces. Even just
considering array indexing Numpy is more capable, convenient, and comfortable,
and Numpy's array-wise maths operations are fast, as they're implemented in C
and Fortran.

~~~
newen
Coming from Matlab, it took me a couple of hours with the documentation to
learn about the indexing mechanisms in numpy. It's highly unintuitive. Numpy
is very awkward to use all around.

~~~
mkl
That's surprising to me. I came from Matlab too, and I found Numpy's indexing
immediately simple and (to me) clearly superior. It seems largely a superset
of Matlab's, but nicer (e.g. a[-3:] instead of a(end-3:end)). The biggest
differences are the 0 base, and that slices are half open. What specifically
did you find difficult?

Edit: I'm asking as I teach this stuff to undergrads and need to understand
potential obstacles.

~~~
newen
This was a few years ago but here's what I somewhat remember. First was
obviously the 0-based indexing, non-inclusive ranges, and the row-major array
ordering.

Then, there are a few issues when you get past non-trivial indexing of 1D
arrays like a[3:]. For matrices, the semantics of numpy says that a 2D array
is a 1D array of 1D arrays. So if A is 2D array, then A[3] is the 4th row of
the array. Not great.

Another is that given 2D array A, and indexing vectors r and s, A[r,s] returns
a 1D array while Matlab returns a 2D array with completely different
semantics.

Another is negative indices.

Then you have things like A[(1,2)] being different from A[[1,2]]] and the
whole concept of slices objects like Ellipsis, np.newaxis, and their
combinations like (Ellipsis, 1, np.newaxis).

Another is that indexing returns a view and not a copy.

This is just indexing. There are tons of other issues with numpy like the
multiply operator, the seemingly random way that they split methods (as in
A.sum()) and applicable functions (as in np.sum(A)), and other nonsense.

Numpy is basically a library stuck on top of Python with Python not being away
of numpy or multi-dimensional arrays at all. It's ridiculous that numerical
and scientific programming has devolved into working with numpy.

~~~
enriquto
> Numpy is basically a library stuck on top of Python with Python not being
> aware of numpy or multi-dimensional arrays at all. It's ridiculous that
> numerical and scientific programming has devolved into working with numpy.

This! So much this!

Many people have lived through different phases of numpy appreciation. First,
when it appeared, it was amazing to be able to access huge arrays from within
a script. Then when it evolved, there was a slight suspicion that things were
going a bit out of hand. Later it became a caricature of itself when it began
to replace matlab. Today, we are in the tragic state where many young people
think that "the only possible way to multiply two matrices on a computer is by
using numpy.dot".

------
chunsj
Maybe this will need C based highly optimized backend as well, however, the
code looks very clean and beautiful to me. I hope someday differentiable
programming becomes possible in common lisp.

~~~
jlarocco
SBCL is actually really good at optimizing this type of code, and I _think_ it
can even do some vectorization.

If it's not not good enough, there are a few CL bindings and wrappers around
BLAS and LAPACK.

A good one is Matlisp, which is available in QuickLisp.

[http://quickdocs.org/matlisp/](http://quickdocs.org/matlisp/)

~~~
fiddlerwoaroof
Sbcl has all the pieces in place for vectorization, but I don’t think anyone
has actually hooked them up. However the great thing about lisps is that the
compiler and optimizer are written in the language they compile, so an end
user can do this sort of thing themselves:
[https://www.pvk.ca/Blog/2013/06/05/fresh-in-
sbcl-1-dot-1-8-s...](https://www.pvk.ca/Blog/2013/06/05/fresh-in-
sbcl-1-dot-1-8-sse-intrinsics/)

------
zmmmmm
Wonder how it compares to using something like ND4J with Clojure? That would
(as I understand it) have the advantage of not sacrificing as much speed.

------
noobermin
Sympathetic, but as long as lisp requires one to write math as function
composition it won't catch on with computational types. Of course, I imagine
that sort of thing could be a set of macros away...then that would be awesome,
familiar infix mathematical expressions with common lisp in the rest of the
code.

~~~
linguae
I'm not 100% sure about this. A counterexample is the popularity of RPN. HP's
RPN calculators with postfix notation are well-loved among some engineers,
scientists, and accountants, precisely because of the benefits of RPN over
infix notation. I have a HP-48SX calculator, and I regularly use the `dc`
command on Unix machines and in WSL whenever I need access to a calculator
while I'm on a computer. Of course, I am proficient in prefix, infix, and
postfix notation, but I like postfix notation the most.

Of course, though, the marketshare of TI's infix calculators is much higher
than HP's RPN calculators, and I haven't heard of much recent developments
regarding HP's calculator line (the last I heard was HP had a limited-edition
re-release of the famous HP 15c model sometime back in 2011, and also earlier
this decade HP released a graphing calculator with a color display). But
nevertheless there are people who prefer RPN to infix.

~~~
mcabbott
The thing about RPN is that it worked well as a sort-of "spoken not written"
language. It's a clear and usable way to give instructions one-at-a-time to a
calculator with a 1-line display.

But written out on paper, it's very hard to see what's going on at a glance,
which is why nobody does this on paper.

Almost anyone programming in mathematical expressions of reasonable complexity
today will spend much more time reading (and triple-checking!) these
expressions than typing them. That's a strong reason to like infix notation,
and in general notation close to what you would use on paper.

