Extensibility and composability is a better abstraction.
Say that our modern day programmer is comfortable with + × > < and can learn in a few moments that max 3⌈5 and min 3⌊5 and reverse ⌽values have their own symbols. By learning the three patterns of / as replicate, reduce and n-wise reduce, they get a large amount of composable patterns to play with, covering sum(), min(), max(), reverse(), [::-1], functools.reduce(), functools.filter(), Numpy overloads of +, Numpy .sum() and .prod(), and more.
In Python, that is many standalone disconnected patterns which do not compose, In APL the "and more" is because composability means there are lots of ways of putting these operations together.
values.prod() is what you have to do without composability, you can't re-use the builtin multiply without making a separate wrapper or overload. The result is visually different to sum(values) and conceptually different because it is a method call on an object, and the resulting prod() does not compose with anything else in the language.
Show me how you present those arrays as a graph in APL. In "wordy, verbose" python it's [three lines, another third party install, an import and rename, a two stage operation and a bizarre array show() call passing itself as a parameter(?)]. Is visualizing the data you're manipulating so basic?
Yes, in Dyalog APL it's:
]chart array
And that doesn't just show a bar graph, it also loads a GUI for customizing the look of the chart, and the chart library is SharpPlot for .Net, which ships with it. (This is not in ISO standard APL, like matplotlib is not part of pure CPython. APL is not giving up extensibility, you can write your own functions which hide things behind names, or in different implementations of APL call out to OS/.Net/library features).
when all your examples are array operations is going to make the general purpose language look funny.
Yes, true. But is it not the everyday task of programming to process chunks of data in collections?
I don't want a DSL for array operations I want a programming language. And when you aren't doing array-math, APL isn't so great.
Strings are character vectors, like Python lets you treat strings as iterables and slice them, so you can do many array transforms on text. APL's array operations aren't limited to numeric math like +5 or A×B, they also work on something I don't know what to call it - geometric patterns, maybe? Like, indicate where 5 is less than integers to ten:
5 < ⍳10
0 0 0 0 0 1 1 1 1 1
Visually patterned half and half. Or this:
¯3 ⌽ 3 < ⍳9
1 1 1 0 0 0 1 1 1
Visually, spatially, patterned into thirds. You can feed this into filter() to make combinations of things more complex than "items greater than five", but "items in this pattern", the filter is not a single lambda function which takes an element and decides whether to keep or remove it, the filter-reduce is more powerful and composable than that, and building the patterns from composing the same basic primitives.
And yes you could pull in Numpy and fill an array with values and rotate it, but you wouldn't think to do that to apply it to a string, because it's /so much work/ and so far away and distant from the provided black-box string methods.
That is, APL is so great for things more than array-math. Albeit not everything more than array math. I sure have my own skepticism and questions about how well it scales up to larger programs and where its practical and pragmatic limits are.
But, take some imaginary pixels in one array and brightnesses in another and (50<brightness)/pixels will get you the pixels brighter than 50. Try that in Python and you get something like [p for i,p in enumerate(pixels) if i in [i for i,b in enumerate(brightness) if b > 50]]. The APL is "dense and unreadable" and the Python is "clear and composable". "Oh you wouldn't do that in Python", no indeed you wouldn't, you'd have to put stuff in a tuple or object to work around the fact that Python won't let you keep simple things simple.
That's still just pixels[brightness >50] with numpy. Which brings me back to what I said before: if I want a powerful array manipulation dsl, I still have it, but I'm not limited by it.
To your statement "you can't use the built-in multiply without adding an overload": yes, but that's because apl forces everything to be an array. If I want to work with something with non-array syntax (as an example, where addition uses a l2 norm), if my objects support that, that's just x+y, while in apl you have to do something more complex.
Note also that you're being unfair to Python. List comprehensions, array index notation, and numpy methods cover pretty much everything you've mentioned in apl. And they funny require matching weird syntax to operations.
> Note also that you're being unfair to Python. List comprehensions, array index notation, and numpy methods cover pretty much everything you've mentioned in apl.
All of that to cover ~ten symbols. Is it unfair to Python to point out that this is a huge difference in complexity that someone needs to learn to be able to do those things from scratch?
> Which brings me back to what I said before: if I want a powerful array manipulation dsl, I still have it, but I'm not limited by it.
Which is fine. It's just that the few operations of array manipulation DSL go so much farther than I expected they would. I do know that APL is not going to be the language to implement WireShark or Halo vNext.
The big catch in paragraph two is "if my objects support that"; yes you would have to do something more complex in APL. But not /much/ more complex. In Python you'd need to understand classes and magic methods and overloading before you could write that overloaded addition - and understand CPython internals, C and NumPy to add it to NumPy objects, I imagine. I doubt you can do that all in APL, but then again if I'm correctly reading what l2 norm addition is, Pythagorean square-root-of-sum-of-squares including complex numbers, it's not a lot of code to write that anyway:
or without the temporary variables which don't add much clarity:
( +/ (|values) * 2 ) * 0.5
⍨ is a cool operator which lets you swap arguments around, so instead of having to read from inside nested parens out, you can remove parens and read serial code left/right instead:
0.5 *⍨ +/ 2*⍨ |values
Name that with a lambda/anonymous function/dfn:
l2NormAdd ← {0.5 *⍨ +/ 2*⍨ | ⍵}
l2NormAdd values
Which .. isn't so bad that you'd wish for overloading, if the cost of writing the overloading was so much higher, is it?
> That's still just pixels[brightness >50] with numpy.
That is cool, I didn't know you could do it. But it is completely separate from the normal Python list comprehension style, apparently a different use of > (?), won't compose with the normal Python sort(key=) to sort the pixels by brightness. At what point does learning one-off skills for every task start to get annoying? (From my personal experience, it never does get annoying, and that seems weird to me now).
----
But then a tiny amount more APL and here's a depth first recursive tree traversal with accumulator function, projecting a tree structure onto a nested array:
⊃∇⍨/⌽(⊂⍺ ⍺⍺ ⍵),⍺ ⍵⍵ ⍵
│└┬┘ └─┬──┘ └─┬──┘
│ │ │ └──── (possibly empty) vector of sub-trees.
│ │ └──────────── new value of accumulator.
│ └────────────────── left-to-right reduction →foldl←
└──────────────────── recursive call on (⍺⍺ trav ⍵⍵).
- https://dfns.dyalog.com/n_trav.htm
I sure could bash out a depth-first tree traversal in Python, with dictionaries or a dedicated tree-node class, and it would take me way less time than understanding this will take me. Yes this may be 20 characters, but it seems a shame to make "few characters" the main focus of why this is interesting. Each of these primitives in the line is almost trivial to learn on its own, none of them are complex magic not even omega-omega. But an expert combining them together carefully makes them do something way more than the sum of their parts, and way more than the shortness suggests they will do. (Here's John Scholes, founder of Dyalog APL, building on this to solve the N-Queens problem: https://www.youtube.com/watch?v=DsZdfnlh_d0 the commonly linked Conway's Game of Life in APL is more approachable, but this is more amazing because of what it's doing to treat arrays as trees, but way harder to follow and more "magic")
> I sure could bash out a depth-first tree traversal in Python, with dictionaries or a dedicated tree-node class, and it would take me way less time than understanding this will take me.
That's my point. APL is interesting, but its enforced structure doesn't fit things intuitively (perhaps there's an implied "for most people" here). Yes, omega combinators or whatever it is that's doing is neat and perhaps pedagogically useful. But
> That is cool, I didn't know you could do it. But it is completely separate from the normal Python list comprehension style
That's because you're not using list comprehensions, you're using ndarrays, which do poweful things to python's already powerful slice notation, and as a result get all of the nice broadcasting things that you get in APL. Its why a + b and a * b just do what you expect in numpy-land.
Slice notation in python is already powerful: a[:], a[5:], a[:5], a[::2], and a[::-1] are things I'd expect someone relatively new to understand intuitively (those are copy, head(5), tail(5), every_other, and reversed).
Adding the ability to customize it: `a[:,:,::-1,:]` for example inverts the 3rd axis of a 4d array, similarly you can pull out a subarray, strided subarray, etc. very declaratively. And numpy further extends that by allowing the argument to be a mask (which is what I showed you in the last comment), so a array[boolean_mask] does the kind of thing you'd expect.
>At what point does learning one-off skills for every task start to get annoying?
When the one-off skills are better abstractions for the task than the "consistent" thing, never, as it seems you're realizing.
You make very good points, but unfortunately most people won't seriously consider APL as it's not a general-purpose language. It's just too alien to put much effort into.
I do believe in the benefits of powerful notation. I also find some concepts useful outside array-oriented languages, eg. verb rank [0]. My qualms with the APL family is the difficulty of choosing language and implementation. J or K seem strictly better than APL (eg. forks and hooks) except they use line-noise ASCII notation. Implementations tend to be proprietary and require a license.
Say that our modern day programmer is comfortable with + × > < and can learn in a few moments that max 3⌈5 and min 3⌊5 and reverse ⌽values have their own symbols. By learning the three patterns of / as replicate, reduce and n-wise reduce, they get a large amount of composable patterns to play with, covering sum(), min(), max(), reverse(), [::-1], functools.reduce(), functools.filter(), Numpy overloads of +, Numpy .sum() and .prod(), and more.
In Python, that is many standalone disconnected patterns which do not compose, In APL the "and more" is because composability means there are lots of ways of putting these operations together. values.prod() is what you have to do without composability, you can't re-use the builtin multiply without making a separate wrapper or overload. The result is visually different to sum(values) and conceptually different because it is a method call on an object, and the resulting prod() does not compose with anything else in the language.
Show me how you present those arrays as a graph in APL. In "wordy, verbose" python it's [three lines, another third party install, an import and rename, a two stage operation and a bizarre array show() call passing itself as a parameter(?)]. Is visualizing the data you're manipulating so basic?
Yes, in Dyalog APL it's:
And that doesn't just show a bar graph, it also loads a GUI for customizing the look of the chart, and the chart library is SharpPlot for .Net, which ships with it. (This is not in ISO standard APL, like matplotlib is not part of pure CPython. APL is not giving up extensibility, you can write your own functions which hide things behind names, or in different implementations of APL call out to OS/.Net/library features).when all your examples are array operations is going to make the general purpose language look funny.
Yes, true. But is it not the everyday task of programming to process chunks of data in collections?
I don't want a DSL for array operations I want a programming language. And when you aren't doing array-math, APL isn't so great.
Strings are character vectors, like Python lets you treat strings as iterables and slice them, so you can do many array transforms on text. APL's array operations aren't limited to numeric math like +5 or A×B, they also work on something I don't know what to call it - geometric patterns, maybe? Like, indicate where 5 is less than integers to ten:
Visually patterned half and half. Or this: Visually, spatially, patterned into thirds. You can feed this into filter() to make combinations of things more complex than "items greater than five", but "items in this pattern", the filter is not a single lambda function which takes an element and decides whether to keep or remove it, the filter-reduce is more powerful and composable than that, and building the patterns from composing the same basic primitives.And yes you could pull in Numpy and fill an array with values and rotate it, but you wouldn't think to do that to apply it to a string, because it's /so much work/ and so far away and distant from the provided black-box string methods.
That is, APL is so great for things more than array-math. Albeit not everything more than array math. I sure have my own skepticism and questions about how well it scales up to larger programs and where its practical and pragmatic limits are.
But, take some imaginary pixels in one array and brightnesses in another and (50<brightness)/pixels will get you the pixels brighter than 50. Try that in Python and you get something like [p for i,p in enumerate(pixels) if i in [i for i,b in enumerate(brightness) if b > 50]]. The APL is "dense and unreadable" and the Python is "clear and composable". "Oh you wouldn't do that in Python", no indeed you wouldn't, you'd have to put stuff in a tuple or object to work around the fact that Python won't let you keep simple things simple.