
J Language – New Website - 4thaccount
https://www.jsoftware.com/#/
======
kyllo
Have any data scientists here who were primarily working in R or Python,
learned J? What did you think?

I love programming languages that teach you new ways of thinking about
programming and I'm very curious about the APL family--thinking of learning J
or q/kdb+.

I do most of my work in query languages on column-store databases, then some
visualization and hypothesis testing on top. So I'm wondering if I could find
J more expressive than R/Python + some SQL-like language.

~~~
4thaccount
I use Python a bunch and have used R in the past. I've dabbled in Julia and
both APL & J. I know just enough to say that APL & J are much more expressive
as long as Python doesn't have a fix_problem() method built-in. I believe
ScottLocklin says something similar below, but he has a lot more experience in
R and J than I.

Now q/kdb+ might be expensive, but I can only dream of doing analysis using
it. SQL is great, but having the full power of K And Q-SQL would be pretty
awesome.

~~~
kd0amg
_I know just enough to say that APL & J are much more expressive as long as
Python doesn't have a fix_problem() method built-in._

This is pretty much the same impression I got, though I came from the other
direction. I was kind of surprised how many StackOverflow posts ask about
adapting a fix_problem() method to fix some very_similar_problem.

~~~
4thaccount
Yep. It's one reason why I prefer to write my own code where possible instead
of using 3rd party libraries. When you need to change something, it is nice to
not read tons of other code.

------
etatoby
I've long been a big fan of APL, since I learned it many years ago. I started
by studying on the classic book by Gilman and Rose and trying to use the
scarce freeware/shareware implementations available at the time. At the time
it was hard or impossible to get anything working for free on a Linux
computer.

Nowadays there is a great, free, and actively maintained implementation: GNU
APL. That, combined with the widespread use of Unicode and the APL support in
Linux/XOrg keyboard definitions, make it so much easier to get started than in
the past.

I have used APL mostly for fun, that is, for learning the language itself,
solving mathematical puzzles, and taking part in "code golf" games. I find the
language exquisite, especially its programming model based on multi-
dimensional arrays, its graphical symbols, and its grammar. But it is somewhat
"impractical" to use nowadays.

J is a great successor (designed by the same mastermind behind APL) but I
found it harder to learn (and to read) due to its ASCII "line-noise" look and
to the prevalence of point-free or tacit function definitions, which I find
inferior in readability.

I'm only now learning NumPy and TensorFlow, and I bet their authors were
inspired by APL.

Does anybody have any benchmarks of J's performance against NumPy?

As for K, I wouldn't recommend picking it up. It's closed-source and
documentation-less, meaning good look if you run into any trouble; its error
messages are notoriously infuriating; it overloads its operators with a ton of
meanings, depending on context, even more so than APL and J; and its
fundamental programming paradigm is not based on multi-dimensional arrays.

On the other hand, writing an APL-like frontend for TensorFlow looks like a
nice project!

~~~
etatoby
Update: I ran a quick benchmark (large matrix multiplication) between GNU APL,
J, and NumPy (standard pre-compiled linux amd64 packages, all running single-
core.) Here are the results.

GNU APL (1.7)

    
    
        ⍴+.×⌿?2 3000 3000⍴1e10
    
        - size 3000: 65s, 9GiB RSS
          (crash on bigger sizes)
    

J (8.07)

    
    
        $(+/ .*)/?2 3000 3000$1e10
    
        - size 3000: 2s, 0.2GiB RSS
        - size 10000: 60s, 3.7GiB RSS
          (crash on bigger sizes)
    

NumPy (1.13.3, blas/lapack 3.7.1)

    
    
        import numpy
        a=numpy.random.randint(0, 1e10, (2,3000,3000))
        print((a[0,] @ a[1,]).shape)
    
        - size 3000: 25s, 0.2GiB RSS
        - size 10000: (>15m, I killed it) 2.3GiB RSS
    

Conclusions:

J's implementation is surprisingly performant! Easily beating NumPy on speed
alone by a factor of 10 or more! (And I thought blas/lapack were already
heavily optimized libraries!) J's memory usage is comparable to that of NumPy.
GNU APL had the worst memory and cpu profile of them all.

~~~
etatoby
Just thought I'd add the same benchmark on the latest Dyalog APL.

Dyalog APL/S-64 (17)

    
    
        ⍴+.×⌿?2 3000 3000⍴1e10
    
        - size 3000: 6.6s, 0.5GiB RSS
        - size 10000: 101s, 5.4GiB RSS
    

Conclusions:

J's implementation is still the fastest at this particular task (matrix
multiplication of huge matrices on a single CPU thread--granted, not the most
significant of benchmarks.) Dyalog APL comes close behind. GNU APL and NumPy
lag much more behind that.

~~~
Volt
I'm trying this with Dyalog 16 (Mac OS X) and I'm getting WS FULL. Do you know
why that might be?

BTW, what kind of machine are you testing on?

------
snek
why does no programming language website ever include an example of the
language? just a hello world or whatever would go a huge way. this annoys me
so much.

~~~
4thaccount
In APL and J, hello world is just 'Hello, World' I think if I recall
correctly.

What makes the languages (APL, J, K) impressive is that you can simplify what
is sometimes pages of code into a single line of symbols (APL) or standard
ASCII characters (J and K).

Keep in mind that these languages are the definition of line noise to most
coders as having prior knowledge of things like C, Python...etc will do you
zero good as the building blocks are completely different from a high level.
They might have if-statements in the language, but they're hardly idiomatic. J
emphasizes tacit programming which is kind of like a data flow model made up
of function application. The languages are quite mathy in some ways, but still
approachable.

I find them to be lots of fun, very interactive, thought provoking, and pretty
cool. I've only written a little APL & J, but enjoyed how I built my command
up from a few built-in primitives.

A problem to me involves reading other people's code which uses primitives I
haven't learned yet, or common idioms (Ex: you can make your own average
function with a few characters avg=.+/%# so no need to add it to the stdlib).
The average function on the tryapl.org site looked a little funny to me...then
I realized it worked for not only lists, but multi-dimemsional data structures
of n dimensions like tables or lists of tables. I imagine it takes awhile to
become proficient.

Surprisingly, both J and APL have some decent support for things such as
graphics, but they are still small communities compared to languages we know
and love like Python.

In short, give it a try. J has some built in documentation called "J labs"
that is submitted by the community. Dyalog APL has a free version for non
commercial use and the excellent tryapl.org. Try not to be turned off by the
bizzare symbols in APL. They have a long history and are easy to enter with
either the IDE or keyboard (especially if you buy one from Dyalog with the
symbols on the keys). The inventor of APL got the Turing Award awhile back
(later invented J). It was also a big IBM product back in the day and is still
used by companies like Volvo.

~~~
snek
I'm not discounting the language at all, by your comment here perhaps some
application of a common mathematical function such as an average or the
fibonacci sequence would be a good front page example.

~~~
4thaccount
That would make sense. Maybe someone on their mailing list will take note (I'm
just a lurker). If you want to see something really cool, look at the APL
video for Conway's Game of Life on YouTube. If I recall correctly, Jon Scholes
made that video a few years ago. I don't know him, but he was supposed to be
an APL wizard and an all around good guy from what I've read. I was sad to
read that he passed away recently.

~~~
kwhitefoot
That was brilliant! Thank you for making me aware of that video.

It would be great if someone could translate that into J, then I might have a
chance of understanding it properly.

~~~
jodrellblank
I tried, although I don't speak J so it's a rough transliteration. I couldn't
do the last GUI animated part without more knowledge of J's tools, but it can
compute multiple generations.

Code summary:

    
    
       r =. (3 3 $ i. 9) e. 1 2 3 4 7
       R =. _1 |. _2 |."1 (5 7 {. r)
       RR =. 15 35 {.  _10 _20 {. R
       life =. verb : '+./ > (1;y) *.each 3 4 =each +each/ +each/ 1 0 _1 |.each/  1 0 _1 |."1 each <y'
       gen =. dyad : 'life ^: y x'
    

The 20th generation:

    
    
        (RR gen 20) { '.X'
    
        ...................................
        ...................................
        ...................................
        ...................................
        .........X....XXX..................
        ........XX.....XX..................
        .......XX.....XX...................
        ........X.X...X.X..................
        .........XX....XXXX................
        ..........X...X..XX................
        ..............X.XX.................
        ..............XXX..................
        ...................................
        ...................................
        ...................................
    

Transcript of the whole talk with line by line translation here:
[https://dpaste.de/OoEM/raw](https://dpaste.de/OoEM/raw) (expires in 7 days).

~~~
kwhitefoot
Thank you! That was great. Thanks for putting in the effort.

------
jodrellblank
NB. You can get J for Apple iOS, free:
[https://itunes.apple.com/us/app/j701/id1255235993](https://itunes.apple.com/us/app/j701/id1255235993)

and it's apparently got a new in-app keyboard to make it easier, since last
time I looked.

------
bithavoc
For a second I thought this was the return of J Sharp
[https://en.m.wikipedia.org/wiki/J_Sharp](https://en.m.wikipedia.org/wiki/J_Sharp)

------
haolez
I like to play with algotrading bots and I'm really tempted to try out J or K
in the near future. It has been in my backlog for a while :)

------
snazz
I suggest they update the logo since it now looks very out of place.

Does anyone use J for anything aside from experimentation (and math)? I know
the same question is asked of everything that deviates from the mainstream,
but the APL (and by extension J) world seems really far out from my mental
model of computers.

~~~
steve19
I struggle to understand why you would want to use it instead of array
orientated languages like R, Julia, Matlab or Python/numpy.

When is terseness a feature? It must make maintenance a nightmare.

~~~
scottlocklin
Those are only partially array oriented languages. I use J for most of my
personal projects because it's terse and highly productive. From empty emacs
buffer to something that works; 5x faster than developing from raw R, and I'm
bad at J and decent at R. Of course if R has a package which does what I need,
it's easier to use that. There's also the matter that for medium big data
problems, J can do plenty that R will never be able to do.

Terseness is a feature. It's also a good design in general, even though it
breaks most people's brains.

~~~
4thaccount
When you say that J is faster than R, are you referring to using Jd versus
dealing with R's read_csv, or are you just talking about the calculations in
general for the same size dataset and just using vanilla native language
constructs?

~~~
tluyben2
Not OP but I think he is referring to speed of development, not execution?

~~~
4thaccount
Thanks, that seems obvious now. I should not skim so much.

------
asaph
J Is free for non-commercial use under the GPL. What does a commercial license
from jsoftware cost?

~~~
4thaccount
Jd the J database is free for non commercial use and listed as $600 for a
single user to use commercially on their site somewhere last I looked. They
have fancier enterprise support contracts as well.

As for the language itself, it is free to use in commercial projects I think
(someone please correct me if I'm wrong), but the source code to the J
language is either GPL3 or you can pay. I think that means if you wanted to
use the J source as part of a commercial project, you can pay or make the
source available under the GPL3 license. I'm honestly not sure though and
would appreciate some clarification.

------
aboutruby
Seems to be a lightweight shell that links to their wiki:
[https://code.jsoftware.com/wiki/Jd/Overview](https://code.jsoftware.com/wiki/Jd/Overview)

~~~
4thaccount
Yea...sorry! I got excited and posted the link before realizing the majority
of the site was still using the old wiki layout. Please forgive. I think the
new landing page does look nicer and more modern.

------
0db532a0
Learn J here:

[https://www.jsoftware.com/help/learning/contents.htm](https://www.jsoftware.com/help/learning/contents.htm)

