
Is APL Dead? Not anymore - lelf
https://www.sacrideo.us/is-apl-dead/
======
codesections
I investigated APL pretty seriously at the end of 2019, after a very inspiring
conversation with Aaron. I discovered that everything in this post is true,
and (dyalog) APL is a pretty great language.

However, I also decided that it is still a bit too dead, at least for me.
Specifically, it does not work well with other programs, at least on *nix.
This is reflected not only in the lack of external libraries, but even in some
basic areas – both FFI and even reading/writing to standard input and output
are surprisingly challenging. Summing all this up, writing a basic Linux CLI
app (read from stdin, take and parse arguments, print to stdout) is still a
pretty large challenge.

As great as APL is, for me, it's hard to overcome the ability to work with the
whole existing Linux/FOSS ecosystem. (I might feel differently if I did any
Windows development, which does feel like much more of a first-class citizen
for Dyalog APL)

~~~
gandercrews
gnu apl is an often overlooked but extremely solid APL implementation that
plays nice w/ unix.

slightly verbose example but heres gnu apl as a pipe-like interface. Often
this would be abstracted to a library that gets included when you call it:

    
    
      printf "1,2,3,4\n5,6,7,8" | apl --eval "f←{4⎕CR ⍵}◊s←{⍺~⍨¨⍵⊂⍨1++\⍺⋸⍵}◊f ⊃{⍎⎕UCS ⍵}¨¨44 s¨10 s⊣⎕FIO[41] 0"
      ┏→━━━━━━┓
      ↓1 2 3 4┃
      ┃5 6 7 8┃
      ┗━━━━━━━┛
    
    

DESCRIPTION:

    
    
        --eval ≡ evaluate the text
    
        f ≡ formatting function for printing nested values
    
        s ≡ partition a line of text and remove the partition character
    
        ⎕FIO ≡ read stdin as byte stream (see FILE_IO.apl)
    
        ⎕UCS ≡ convert bytes to characters
    
        ⍎ ≡ execute an expression (convert characters to numbers)
    
    

SUMMARY:

    
    
      read stdin as a bytestream
    
      partition at newlines and commas
    
      convert to characters
    
      interpret characters as numbers
    
      disclose into a rank-2 array
    
      format to stdout

~~~
kick
GNU APL is pretty slow and makes some controversial design decisions. Fun to
play with, though, and remarkably polished.

~~~
snapdangle
I'd be very interested in reading more about the differentiating features of
GNU APL. There is a great big announcement at the beginning of the GNU APL
docs that talks of a decision needing to be made around a gap in the ISO
specification. Is that the primary source of controversy or is there more
discussion I could read somewhere else?

I've been looking for a good "diff" of the language differences between Dyalog
APL and GNU APL, as much as to understand the extent of the progress since
'APL2' (or whatever the ISO standard name is considered equivalent to it) as
any for any specific list of what GNU APL can't do relative to it's modern
commercial compatriots.

(EDIT: Fixed final sentence to be a complete thought/sentence).

~~~
gandercrews
dyalog has a few more operators. for example @ & ⍠. It lets you use statement
separators in lambdas. In gnu apl you can simulate that with '⊣'. It allows
multiline lambdas. Dyalog uses ⍺⍺ & ⍵⍵ for operators vs ⍶ & ⍹. Dyalog lets you
use ∘ as a composition operator. Dyalog generally has better performance,
which becomes noticeable with large arrays. It has a lot more libraries
available. Gnu APL behaves more like a traditional unix based interpreter. Gnu
APL has erlang, python, lua, and C interfaces.

------
snapdangle
I'm awaiting delivery of a printed Mastering Dyalog APL book while reading
this! I landed on kdb+ after seeking a reasonable alternative to the so called
"best in class" Elastic Search/Kibana tooling, fell in love with K once I
understood that the syntactic terseness is all for the sake of fitting the
entire interpreter into L2 cache, and have now landed at the decision that
learning Iverson's classic is the only way to satisfy my desire to live a life
free of them dang stinking loops!

~~~
dan-robertson
I don’t believe that the terseness of k is necessary to fit into the I$. I
think you could have reasonably longer operators and do fine.

I think it’s partly about style, partly about having a small number of
operators (which compose well together), partly on using simple data
structures (it doesn’t take much code to iterate an array.

I’m not even particularly convinced that fitting in the instruction cache is a
trick that magically makes everything fast anyway. Most of your memory
accesses in a typical data processing program (ie the kind of program one
would write in k) will be to the data and hopefully these will be nearly
always linear and mostly sequential accesses.

~~~
snapdangle
The original K design was laid out in the 1980s when the constraints were even
tighter than what they are today. The utilization of very short operators
means not only the interpreter easily fits into cache but also the custom
function definitions you have written will as well.

When dealing with high performance computing or real time processing of high
volumes of data, any fetch to RAM for loading a function call to dispatch is
going to have _some_ impact in a tight loop. Add that up for all the libraries
you have loaded for your application verses a ground up implementation in K...
Does that whole thing live in L3 along with the VM or intepreter +
dependencies underneath it? It's doubtful.

My experience was simply using their Kx's free Developer IDE and experiencing
the performance differential on datasets myself. YMMV but my (admittedly
limited) experience leads me to believe that there is a serious case to be
made for the performance advantages of having all your computational logic
living as close to your computational cores as possible.

See also the PhD by author of the OP article where he presents language where:

"The entire source code to the compiler written in this method requires only
17 lines of simple code compared to roughly 1000 lines of equivalent code in
the domain-specific compiler construction framework, Nanopass, and requires no
domain specific techniques, libraries, or infrastructure support."

Linked from the article, available here:
[https://scholarworks.iu.edu/dspace/handle/2022/24749](https://scholarworks.iu.edu/dspace/handle/2022/24749)

~~~
dTal
Surely none of that is an argument for terse _user-facing_ syntax? Anything
the user types can trivially be converted into K's "bytecode", ahead-of-time.

~~~
beagle3
No, it’s not.

The arguments for terse user facing syntax are related, though:

The ability to see a whole program (17 lines vs 1000 lines) means you need
much less human “working memory” or whatever the biological equivalent of
cache is, to reason about the program.

It also means you can use your visual system’s pattern matching abilities
because patterns are 5-10 characters rather than 5-10 pages as they often are
in C++.

Totally different hardware, but it’s still about the L1 and registers.....

~~~
dan-robertson
I agree with these points. Also keeping a language small allows it all to fit
in your brain (though of course there are all those idioms in apl) which isn’t
really true of larger languages

------
zests
I don't really understand these array based J/K/APL languages but from a
distance it would seem like they would primarily be useful for numerical based
work such as finance or calculating patterns on 2d arrays (game of life). The
languages seem more mathematical than software engineering based.

It reminds me about when I was learning my first programming language. The
book had a ton of example with simple numerical computations (e.g., calculate
leap years) and I soon became competent with the language's facility for basic
mathematical operations. But as a programmer now I have very little use for
these operations so when I learn a new language I have only a rudimentary
understanding of how to do math in that language.

Is this correct or am I misunderstanding the usefulness of array based
languages?

~~~
beagle3
It requires a different mindset. You might not need leap year calculation
often, but many things you do have a short, simple, efficient J/K/APL
implementation entirely different from how you’d do it in other languages.

E.g. to check if a text string s has balanced parantheses in C/Java/Python,
you would have a loop to increase/decrease depth based on character, error if
negative at any point, and balanced if zero at end.

In k, you would compute the running balance at each point

    
    
        b:+\(s=“(“)-(s=“)”)
    

And then check the minimum of it &/b to see if it is negative, and the last
element *|b To see if it is zero.

This is a text parsing problem, but has an elegant vector solution. Most
things do, whether it is graphics, database, whatever - but it’s rarely
intuitive if you are not used to thinking in vectors.

Also, the expression for b above can be half as long, but you would really
need to be versed in k to understand it.

~~~
clarry
Will the vector solution thing still be elegant if parentheses that are
escaped with a backslash don't count? Oh, and backslashes can also be escaped,
so the paren in "\\\\(" still counts. Oh and you can have a quoted segment in
the string where parens don't count. As expected, quotes can be escaped..

I bet there _are_ elegant solutions, and perhaps they are obvious to those who
are well versed in array programming. What I think the world needs is a
book/tutorial/whatever that demonstrates elegant solutions to things that seem
ugly and not amenable to vector processing (at least in the obvious and naive
way). I think we have enough (too many) tutorials showing how to calculate
means and such :(

~~~
beagle3
Elegance is in the eye of the beholder, of course - a state machine is easier
to code in K than in almost any other language, and is likely the most elegant
overall solution.

A common kludgy solution you’d see would probably be to delete or zero out
characters that shouldn’t be counted, and then do the original thing; some
would consider that elegant and others won’t.

There are many examples if you look for them, many well explained, e.g.
[https://nsl.com](https://nsl.com) ; the problem is APL/K is almost as
different from C/Java as Japanese is from English ; there’s no amount of
chewing anyone will do for you that will save you from going deep, and if
you’ve decided to go deep, the material is already available.

~~~
clarry
Sounds good, I love state machines and I often write them in C where other
people would just go with ad-hoc loops and branches and temporaries. I'll keep
an eye out for examples..

~~~
avmich
Recently there was a link on simdjson, a C++ parser for JSON which aims to be
record fast. It uses vector instructions of modern processes, but if you'll
watch the video presentation, Daniel Lemire will also explain how they handle
finding string boundaries - with all those escapes needed - without branching,
purely logical operations.

Could be a good example for how APL approaches things.

------
kimi
Even if it was the best thing since sliced bread, I still would not use it:
[https://www.dyalog.com/prices-and-
licences.htm](https://www.dyalog.com/prices-and-licences.htm)

A run time license for multi-user applications? you must be joking, right?
does still anyone produce single-user applications, but the odd shell script?

So no, I won't use Dyalog APL in my company and won't invest my time on it.
Sorry.

------
clarry
What would getopt look like in APL (also K, J, what have you)? I couldn't find
any examples. Similarly, I'd like to see some examples of handling multiple
I/O endpoints (sockets, stdio, files?) asynchronously (or should I say without
blocking) using whatever mechanism works best in APL.

~~~
srazzaque
It's literally 4 characters in Q.

.z.x

But in reality you'd probably use:

.Q.opt

(which is 6 characters!)

Update: reference to .Q.opt

See [https://code.kx.com/q/ref/dotz/#zx-raw-command-
line](https://code.kx.com/q/ref/dotz/#zx-raw-command-line)

~~~
clarry
Uhh.. no, that's just argv. Getopt is something you'd use to actually parse
flags and arguments out of argv.

EDIT: Ok, .Q.opt looks a little better, although it's still not close to
getopt (where -abc would be shorthand for three flags, -a -b -c; but other
flags might be defined to take (possibly optional) arguments). Is it a built-
in implemented in whatever K/Q is written in? How would you implement .Q.opt
in K if it were not built in?

[http://man.openbsd.org/getopt.3](http://man.openbsd.org/getopt.3)

[http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?...](http://cvsweb.openbsd.org/src/lib/libc/stdlib/getopt_long.c?rev=1.31&content-
type=text/x-cvsweb-markup)

~~~
srazzaque
Yep you're right, it's definitely not doing as much as getopt, though it's the
same general use case. It's output is a dictionary representing your command
line arguments, which you'd then use in your program to act accordingly.

It's implemented in K and comes standard with the distrubution. If you've got
a KDB+/Q distribution you can view the actual function definition by executing
it without an argument:

    
    
      q).Q.opt
      k){[o]x::$[#i:&o like"-[^0-9]*";i[0]#o;o];((`$1_*:)'o)!1_'o:i_o}
    

Disclaimer: I cannot read K, and I'd consider myself a beginner in Q, so the
above is completely alien to me. But what that does, in words: any "-single-
dash-params" is treated as an option. It splits your 'argv' by those options.
Everything between the options become the 'values' for the prior most recently
preceding option. If you've got a no-arg option, it just becomes a member in
the dictionary without a corresponding value:

    
    
      $ q -opt1 foo bar -opt2 baz -opt3
      q).Q.opt .z.x
      opt1| ("foo";"bar")
      opt2| ,"baz"
      opt3| ()
    

EDIT: formatting.

------
robomartin
Having used APL professionally for ten years from the early 80's to early 90's
I am a big fan. I used it for a wide range of applications, from business
systems to DNA sequencing. I even published a paper at a conference eons ago.

So, yeah, I am an "APL guy". And, yeah, I think it's dead.

Well, not really, but, yes really at the same time.

Let's start with the most glaring issue:

[https://www.dyalog.com/prices-and-
licences.htm#devlicprice](https://www.dyalog.com/prices-and-
licences.htm#devlicprice)

Yeah, $1,600 USD for a language license? No. Thanks. Go away.

In addition to this:

"In addition to a current Developer Licence, a Run Time Licence is necessary
if:"

Yeah, no. Go away.

And then...

"When several users use one or more applications on a shared server, a Server
Licence is required."

No. Definitely no. Please, pretty please, with sugar on top...go away.

I could go on. Just have a look at that page and see how you could possibly
justify getting into this monster of a business case for your particular
business or any other business who's financials you understand.

If you are still interested after that, take a moment to go browse the source
here...

Oh, wait. Yeah. No. Go away.

That said, I still do love APL. I love the language, the ideas, what it
represents and the superpowers it gives you. It truly enables the creation of
computational solutions for complex problems at the speed of thought once you
learn to think in APL (which likely takes about a year to fully internalize).

During the 80's I often dreamed of designing a purpose built computational
engine (hardware) specifically designed to execute APL commands in one clock
cycle. Hey, I was young and stupid. While I was using APL I was also coding
embedded systems in C and assembler. I would often wonder "wouldn't it be
amazing to be able to execute this APL statement in one clock cycle, like some
of these assembler instructions on this tiny little MCU?".

Anyhow, let's end this on a positive note. Here's a list of language
improvements I authored a long time ago. This list isn't exhaustive (it was a
five minute stream of consciousness exercise). Also, some of what's on this
list might already exist in modern variants of the language. Feel free to
criticize, no problem, and comment. Understand I am issuing advance notice
that this is not a perfect list or even a current one.

    
    
      APL should go FOSS or adoption will forever be limited to esoteric high-ROI
      applications where it might make sense or legacy APL code already exists.
      
      The option for compilation would be very interesting.
    
      Within a function, variables are global by default.  Huge pain in the ass.
    
      Objects. Yet be careful to avoid the mess OOP can (and often does) become.
    
      A language-defined interface to C (or some other suitable language) for 
      when you need to speed things up.
    
      Inline C or Python, or both, as a first class citizen. Make it first 
      hybrid high level language.
    
      An extension to nested arrays that allows you to create C-like structures.
      I am going after the self-documenting nature of having structure members 
      have name-based access.
    
      An extension to function definition to allow for more than two arguments. 
      This would require a little language re-thinking.
    
      I would also look into making a hybrid that includes some C-like syntax
      with an APL core.  C/C++ -like comments and conditionals with bracketed
      groupings of APL code would go far in making code easier to read and 
      organize.
    
      Native C-like switch() primitive with APL-inspired extensions.  For example,
      switch could take a vector argument.
    
      APL-run-time-definable comparison operator.  Sometimes I want to apply a complex
      evaluation function to data rather than the simple equal, not equal, grater/less
      than, etc. operators.
    
      The ability to enforce data types if desired.
    
      A means of telling the compiler/interpreter to optimize code 
      --at the statement level-- for speed or resource (memory footprint) and 
      other criteria.  Certain operations can explode into huge multidimensional
      memory-sucking arrays which isn't always the best idea.
    
      Real-time extensions integrated into the language.
    
      Ability to use and request GPU resources for computation.
    
      Ability to use and request CPU cores for computation.
    
      A simple-minded example of the above two would be wrapping a code
      section with "CPUn{}" or "GPUn{}, where "n" is the core or GPU number.
    
      Standard interface for custom hardware-based acceleration (read: custom 
      FPGA boards). FPGA's are amazing, make them first class cititzens.
    
      True binary vectors and arrays.
    
      More flexibility in multidimensional array indexing.
    
      A rethinking of the workspace model to better support multi-developer 
      environments.
    
      Genetic and Evolutionary computing primitives built into the language.
      
      Neural Network primitives built into the language.
      
      Language native image processing (think OpenCV integrated right into the language as a first class citizen)
      
      Same with NumPy. APL should be able to do all of this and more.
    
      Built in primitives for multi-threaded/multitasked computing.
    
      Built in primitives for network access and processing.
    
      Built in primitives for multi-core and distributed computing.
    
      Built-in primitives for data exchange.  For example, ingest or output 
      nested array data from/to JSON, SQL, XML or other modern data formats.
    
      Better pattern-matching primitives.  I'm thinking at least regular 
      expressions. 
    
      A general cleanup of the notation to remove lame text-based "fixes" 
      from an era when rendering non-ascii characters required custom ROMs 
      on the graphics card.  All APL notation should be symbolic.  ASCII 
      should be limited to strings, function, variable, object and other 
      constructs that require text. I'm talking about a lot of the quad
      stuff, etc.
    
      Make it open source and make the open source version far superior to 
      any available commercial version.
        
    

I'll end with a slightly edited portion of one of my HN posts from about seven
years ago:

One of the reasons I abandoned languages such as APL, Forth and LISP,
languages that I used extensively for many, many years is that they became
less and less practical and relevant.

I can apply C to nearly everything from embedded to workstations/servers and
even in modern hybrids where FPGA's are integrated with capable
microprocessors. I can hire people, even entire teams, both locally and around
the world to work on anything from embedded to web projects in many of the the
C-like languages. Trying to do the same with APL, Forth or LISP is very close
to impossible (and the quality and capability choices you have are much
reduced).

On the hardware front languages like Verilog are very reminiscent of C and, as
long as you understand that you are actually describing hardware and _not_
writing software, are easy to pick-up with the appropriate background.

You move up to languages such as C++ and other layers open up. PHP, Python,
Objective-C when I absolutely must and Java if I have no choice.

All of these are very flexible and relevant tools that have, for the most part
remained relevant and useful for years. This range of applicability will never
be achieved with something like APL.

If an APL-like language is going to come to the forefront it will be for very
specialized applications where it makes sense. It will not be to run a
shopping cart on a website or control a servo on a robot. That's just reality.

I love APL. I devoted a huge chunk of my professional life to it. I can't see
using it or the wanna-be variants for anything today. Sorry.

~~~
teleforce
Hi Martin, thanks for sharing your insightful comments and the improvements
list. I think if anyone can do that, the impact will be probably nothing less
than the impact of introduction of spreadsheet to do PC industry, and arguably
it is still the killer computer application to beat until today.

Just sharing about the improvements of array based programming language from
the academia inspired by APL. Two of the most promosing approaches are the
Single Assignment C (SAC) from Hertfordshire University, UK (proprietary) and
Furthark from Copenhagen University, Denmark (open source). SAC try to the
introduce better syntax (as in Algol C) for functional programming array based
language with concurrent multi-core CPU support. Furthark also try to better
the syntax for programming array language for GPU parallelism by being the
more intuitive intermediate language.

From non academic, I would say the closest to your improvements list will be
the D programming language. Unlike Python and NumPy, where there is a clunky
impedance mismatched, using D it is feasible to create a seamless array based
integration by virtue of its CTFE and metaprogramming capabilities including
native support for object oriented, imperative and functional programming
paradigm. For initial work check the Mir GLAS library for the native D
implementation where it has managed to outperform the venerable BLAS/LAPACK
Fortran based linear algebra library where normally Python or other
programming languages depend on [1].

Additionally D also support native nested functions that can be handy for
processing nested multi-dimensional arrays.

I have got the feeling that for seamless CPU, GPU, FPGA and TPU programming
integration, there will be based on the Static Single Assignment (SSA) form
that is currently being proposed independently by MLIR (from LLVM team) and
LLHD (from ETH Zurich). If you have noticed that the SAC is also utilizing
this compilation technique as well.

[1]
[http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...](http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-
gemm-benchmark.html)

~~~
robomartin
Thanks for the comment. I'll look into the items you suggested and see what I
can learn.

My perspective on APL going forward is that it _must_ retain symbolic
notation. This is a very important element of the value delivered by this
language and one of the aspects that tends to be difficult for non-APL-ers to
understand (the value) from what amounts to casual contact with the language.
You have to live APL for a while before the "tool for though" realization
--and the importance of the symbols-- can be understood.

In the '90's Iverson developed J to get around the difficulties technology of
the era had with symbols. This was a huge mistake and something that went
exactly opposite the "Notation as a tool for thought" idea he promoted for
years when describing APL. I watched him give this lecture in person at one of
the APL conferences in the mid 80's.

The problem with any language --APL being no exception-- is that a business
case must exist for wide adoption. Engineers tend to get lost in the technical
minutiae, which is important when you are doing the job but it isn't a
requirement for the job to be done. Or put more precisely: IF there is no
quantifiable need for APL and a mechanism for better ROI, it is hard to
justify shifting entire teams to this, or any other language, simply because
we like the technical aspects of it.

At the end of the day things are simple: It either makes sense or it does not.
I would LOVE for APL to become relevant again and for it to find mass
adoption. I love the language. And yet I understand business and know the
probability of this happening is so close to zero it might as well be zero.
Sadly.

------
itsajoke
As long as a language can still run, it's not dead. It might just be napping.

Anecdote: when I saw that Python 2.x was reaching end of support, the
contrarian in me thought, "finally it'll be stable enough to use"

------
carapace
Ah! This is arcfide. Awesome. His co-dnfs compiler is a revelation.

(Ooo! He has _merch_ now! Co-dnfs on a t-shirt!? Maybe everything isn't so bad
after all!)

------
hota_mazi
> The result is a very solid language with some of the best tooling out there.

Hard to take an article seriously when it makes this kind of claim about APL.

> It turns out, OOP and software engineering are not the Godsends we thought
> them to be, particularly in the presence of rapid change.

I don't know what the criterion is to be qualified as a "godsend", but the
fact that an enormous majority of the code that powers our civilization today
is OOP is a part of the answer that can't be waved off that easily. It doesn't
mean it's the best possible approach, but it at least contradicts the claim
about OOP not being adaptable to rapid change.

And, really, claiming that APL is better at rapid change? A language that
requires a specialized keyboard that's almost impossible to find these days?

> We live in an international world full of non-English scripts.

Source for this?

As far as I can tell, most languages in existence and being created every day
use English keywords.

> APL is designed as executable math notation

That is not true at all. Only a very tiny number of APL symbols are
mathematical (and some even use these symbols wrong). Most of the APL alphabet
is made of invented symbols.

> Here again, APL has proven itself far ahead of the curve.

The opposite. APL tried to experiment with brand new symbols for a programming
language, which was a commendable attempt to innovate, but since no languages
in existence today do the same, it's clearly an experiment that completely
failed.

The only reason why APL can still be considered not dead is that once a year,
somebody posts the one liner Game of Life on a public forum and everybody
marvels at it while being secretly happy APL never took off.

~~~
Tistron
I generally agree with your sentiment, but I feel that you are a bit
uncharitable in your interpretation of what you are responding to.

The way I read "We live in an international world full of non-English scripts"
is that the scripts meant are not computer code but things like the cyrillic
script or the devanagari script.

I also read "executable math notation" not as meaning that it's a way of
executing some specific other math notation (I don't think there really is a
single standardised math notation) but a way of writing the specific
executable math notation of APL.

------
victorthetester
Test comment please ignore, will delete later, thanks

