
A look at the J language: the fine line between genius and insanity (2012) - tosh
https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-j-language-the-fine-line-between-genius-and-insanity/
======
scottlocklin
The TSDB is now called 'jd'[0] FWIIW. Funny, I've been using J on and off
since then and I'm still not sure if it's genius or insane. It, along with the
other APLs, is definitely its own thing, just as much as Lisp is (my previous
favorite weird language was Yann LeCun and Leon Bottou's "Lush,"[1] which I
continue to think is an amazing design). Once you get a little bit proficient
at it, you finally realize what an actually high level programming language
could be.

I think it's an idea whose time has come in some ways: hardware is more array-
like than ever. J has evolved a lot from 806 to the present 902 beta; lots of
performance increases from moving away from the single core paradigm (where it
excelled anyway). Lots more to unlock there as well, and some interesting
J-based startups brewing[2]. I also think it's one of the best user
communities I know of.

Oh yeah, Art's new language Shakti[3] should probably be mentioned. Lots of
people excited about this.

[0][https://code.jsoftware.com/wiki/Jd/Overview](https://code.jsoftware.com/wiki/Jd/Overview)

[1][http://lush.sourceforge.net/](http://lush.sourceforge.net/)

[2] [https://www.monument.ai/](https://www.monument.ai/) for example

[3] [https://shakti.com/](https://shakti.com/)

~~~
marmaduke
I started with Lush a long time ago before being obliged to switch to MATLAB
and then C.

Recently I have been trying to learn Futhark which is a ML like language
generating parallel code (E.g. OpenCL or CUDA) from pure functional code with
nested array structures. I find this exciting because when using the GPU with
a thin layer like OpenCL directly, you have to write kernels from scratch
without much abstraction possible. Higher level APIs like PyTorch or Jax give
you easy ways to flexibly structure the computation but you just have to hope
for kernel fusion. In Futhark this is in principle explicit and well handled.

ISPC is another example of a C like language where SIMD style parallelism is
unambiguous while staying relatively high level.

~~~
p1esk
If your primary interest is in parallel code that runs on a GPU, and not
autograd features of Pytorch/Jax, take a look at CuPy:
[https://cupy.dev/](https://cupy.dev/)

~~~
dTal
That's extremely cool. Julia has something very similar, in the form of
CuArrays [0].

I do wish we had some better cross-platform solutions though, instead of
cementing nVidia's hegemony. There's no technical reason why GPGPU should be
any more difficult on AMD hardware.

[0]
[https://github.com/JuliaGPU/CuArrays.jl](https://github.com/JuliaGPU/CuArrays.jl)

~~~
eigenspace
> I do wish we had some better cross-platform solutions though, instead of
> cementing nVidia's hegemony. There's no technical reason why GPGPU should be
> any more difficult on AMD hardware.

Then look no further than GPUCompiler.jl. Tim factored out all the platform
agnostic parts of CUDA.jl (successor of CuArrays.jl and CudaNative.jl) into
GPUCompiler.jl and that is now being used by the AMDGPU.jl package. It works
today!

~~~
dTal
Thanks! I had hit some of those pages before, but their ontology isn't
immediately obvious, so thanks for the clarification. In particular I've
definitely looked at AMDGPU.jl recently, but as the page says: "not all
features (and especially performance) are up to par with CUDA.jl" \- hence my
impression of AMD being a second-class citizen. Still, it's great that we have
anything at all, and I'm extremely grateful for the hard work of those in the
Julia community. It's a remarkable achievement.

~~~
eigenspace
> In particular I've definitely looked at AMDGPU.jl recently, but as the page
> says: "not all features (and especially performance) are up to par with
> CUDA.jl" \- hence my impression of AMD being a second-class citizen.

That just takes developer time to fix. Currently it’s basically just a one-man
show.

------
Verdex
I've learned a bunch of different languages[1], but J (and its friends APL and
K) has been both the one that I don't feel I have the patience to learn AND
the one that I feel like I might fall in love with if I actually learned it.

The incomprehensible programs just seem to hint that there is some sort of
maximal information density where every keystroke provides maximal
productivity.

There's also something artistically appealing about starting at a program and
making a deliberate one character change that makes all the difference ...
although on the other hand, that's kind of scary too.

Here's a video that really illustrates the power of this paradigm:
[https://www.youtube.com/watch?v=a9xAKttWgP4](https://www.youtube.com/watch?v=a9xAKttWgP4)

[1] - I learned (in this approximate order): C, Java, Lua, C#, Ruby, Python,
Lisp, Forth, Prolog, Clojure, OCaml, Haskell, Erlang, F#, R, Racket, and Rust.
I also took a look at languages like: ATS, Cyclone, C++, Idris, Agda, D, and
Perl 6. Of all those languages it really feels like APL/J/K are the strange
ones.

~~~
TurboHaskal
I struggled a lot too. I think the main issue is the problem domain in which
these languages excel at.

I like compilers, systems programming, distributed systems... but math?
statistics? I'm admittedly terrible at that and find it really uninteresting.
It doesn't help that most literature on J tries to teach its language concepts
showing you how to solve math problems. I have no need for a fancy calculator.

For what it's worth, I like K and Klong and found them very approachable.
Basically because you can get away by writing Scheme with m-expressions.

~~~
ashtonkem
My general belief is that these languages would be far more useful as embedded
languages; occasionally there is a need for the core of some complicated
system to take advantage of their power, but the same design that makes them
good at math makes them horrible for writing APIs or database access.

~~~
beagle3
They work better than databases for accessing databases - nsl.com has examples
like tables.k , a reasonably efficient in memory ordered-and-mostly-relational
database in all of 14 lines of K.

And a variety of spreadsheets, from 2 to 100 lines.

They were sort-of-ok for API access, but Shakti’s FFI is actually incredibly
straightforward - comparable to writing a C extern definition only much terser
(‘cause it’s Arthur...)

------
ethagnawl
Someone who was _very competent_ with J gave a demo of it at one of the
Recurse Center (ne Hackruiter) _Brain Dump_ meetups back in the day and broke
the crowd's collective brain. It might as well have been Brainfuck to all of
us, but the proof was in the ... STDOUT. I believe they were demonstrating the
standard factorial, map/reduce, etc. sorts of solutions you'd expect, but I
remember being shocked by the small amount of code required to achieve the
desired outcome. To the point the article appears to make, this may or may not
be a good thing in terms of team productivity and maintainability, but it was
damn impressive.

~~~
Archit3ch
There's a very elegant Haskell quicksort implementation that misses out on
performance because it's not in-place. It's frequently used as a
demonstration. The performant version looks as bad as C.

I'd be more interested in a J/K demonstration that plays to the language's
strengths. How about live-commentating e-sports?

* Player gets first blood. Commentator executes search query in J across DB. *

"We've seen $PLAYER_NAME contribute to 65% of first blood situations across
their last 10 matches."

~~~
beagle3
Idiomatic K solutions (and from what I hear, J as well) tend to be faster than
idiomatic C solutions to the same problems.

It originally seemed like magic to me, but really - there is no magic. It is
simpler in K and J to process more things, in more stages, but with completely
predictable and very pipelineable operations. The J/K program often does 2 or
4 times as many operations, but has virtually everything prefetched to L1
before it’s needed so it runs as fast or even faster.

CPUs are constantly converging towards the K model (GPUs were always there).

~~~
smabie
If there was a time for array languages, it would be now. I would argue that
most programming nowadays is about manipulating arrays of data.

------
mwexler
Funny, that phrase can also describe k itself... It's a mindset, or cult,
depending on who you chat with about it. Very powerful, and somewhat obtuse,
which often go together.

Fwir, Kdb+ is written in k which is sorta APL, and then there's q for writing
kdb+ programs which is easierish. But still an adventure.

~~~
rscho
People always compare J, K and APL as if they were almost identical apart from
the character set, but that's far from the truth.

[https://aplwiki.com/wiki/Array_model#:~:text=In%20APL%20it%2...](https://aplwiki.com/wiki/Array_model#:~:text=In%20APL%20it%20is%20the,multidimensional%20FORTRAN%20or%20C%20arrays).

~~~
ggrrhh_ta
APL and J are different between themselves (I started with APL but jumped into
J) like FORTRAN and C are... What is your point?

~~~
systems
But do they implement the same programming paradigm, or is there any
fundamental differences

For example, C# and Java implement the same paradigm , they are different,
many argue which provide a better set of feature, but I would not call the
differences fundamental

C#, F#, implement completely different paradigms

F# and Clojure, same paradigm, but have fundamental differences, F# being
static and come from the ML family , Clojure dynamic and come from the Lisp
family

So APL to J, is more like what to what, C# to Java, or F# to Clojure ??

~~~
rscho
APL and J are similar but for a pretty fundamental difference: APL has nested
array theory while J is flat array theory. In practice, this does not make
them that different to the user, except APL's model is arguably more elegant
since it's "turtles all the way down" (J array elements are not array
themselves).

K on the other hand, is really different! K doesn't have true multidimensional
arrays, but lists of vectors. So it's ideal for its use case: finance (1d/2d
numeric tables).

~~~
ggrrhh_ta
I see now your point. And as you say it does not make a lot of difference to
the user. I have never used K.

------
Buttons840
J seems like a language I should learn using flash cards. Make a flash card
for each character, for each operation, and just learn all the operators, and
then I will know J.

Any truth to this?

I suppose it might be like telling someone, "you know conditionals and loops,
now you know how to program!"

~~~
scottlocklin
Definitely no; I tried something similar with the J dictionary. The right way
to learn is to go through "J for C programmers" (it comes with the language),
or just grab NuVoc[0] and start writing code for something non trivial. Labs
can also help for areas you're not familiar with. Its the composition in J
which carries meaning; and stuff like hooks and forks (more or less, writing J
AST's directly) make it non-obvious in early days.

[0]
[https://code.jsoftware.com/wiki/NuVoc](https://code.jsoftware.com/wiki/NuVoc)

~~~
recursive
I couldn't find "J for C programmers" in my installation, but there seems to
be more info here.

[https://code.jsoftware.com/wiki/Books#J_for_C_Programmers](https://code.jsoftware.com/wiki/Books#J_for_C_Programmers)

~~~
scottlocklin
Memory failing me; it used to be part of the IDE, with a copy of it and the
dictionary stored on local storage, but I guess J has changed enough recently
it needed to go up on the wiki where it can be changed. You can always find it
from nuvoc.

Usr Pri JfC LJ Phr Dic Voc !:

All these are interesting; some primers, some the old version of the
dictionary (I think it's depreciated for now). I end up !: a lot for the
foreigns.

------
andybp85
I stumbled across J years ago when I was messing with Supercollider[0]. I wish
I had a use for J, it's very cool.

Slight tangent, Supercollier is one of the best langs I've ever played with.
Smalltalk + J. Totally blew my mind when I first wrapped my brain around it!

[0] [https://supercollider.github.io/](https://supercollider.github.io/)

------
grimgrin
Here's the response they got asking their n00b question:

[http://jsoftware.2058.n7.nabble.com/Spatial-trees-
td25879.ht...](http://jsoftware.2058.n7.nabble.com/Spatial-trees-td25879.html)

Link in article was dead

~~~
scottlocklin
Thanks; updated

------
RyanHamilton
Some of these languages are an excellent fit for specific problems. I still
think an array based language is best for sql queries rather than set based.
Any time you hear something like "select the first....", you don't want a set,
you want an ordered list.

Plug: I wrote an open source version of q/kdb that runs on the JVM:
[https://github.com/timestored/jq](https://github.com/timestored/jq)

You can give it a try online in the browser here:
[http://www.timestored.com/jq/](http://www.timestored.com/jq/)

~~~
mdaniel
It appears you forgot some references to your home directory in the code:
[https://github.com/timestored/jq/blob/1f4de6def8b7c12c6d6254...](https://github.com/timestored/jq/blob/1f4de6def8b7c12c6d6254296dcd24a2e7f1eca6/genie/src/main/java/com/timestored/jdb/codegen/Genie.java#L23)

Also, Java's "File" object is -- for the most part -- by its nature cross-
platform, which is why the forward-slash version right below your change
already worked as expected:
[https://github.com/timestored/jq/commit/4b94ce214452cf2b2fb8...](https://github.com/timestored/jq/commit/4b94ce214452cf2b2fb8e915035dd6ea25f51904#diff-
bb81c0f14ea5ae146dbb8f5feba4a4ccR67)

------
JRaspass
Shameless plug: [https://code.golf](https://code.golf) supports J if you'd
like to compete with cryptic line noise :-)

I don't claim to understand J, I just added it to the site.

------
djmetzle
I would see J solutions to Project Euler problems a lot, and they would often
be the most terse and cryptic solutions.

Good introduction. Its always nice to see a "community review".

~~~
aloe_falsa
Shameless plug: I'm currently working through Project Euler problems using J,
and the language is obscenely well-suited to this kind of problems. The
terseness has grown on me, as for the other thing - well, I guess code in any
language can look like line noise, it's all up to the programmer.

Please take a look at
[https://ciumei.ca/blog/2020-07-25/j.html](https://ciumei.ca/blog/2020-07-25/j.html)
and do let me know if the code is at all cryptic.

~~~
klysm
I found them surprisingly readable as someone with no prior J
/APL/OtherBlackMagic experience :)

------
dang
If curious see also

2018
[https://news.ycombinator.com/item?id=16393873](https://news.ycombinator.com/item?id=16393873)

Discussed at the time:
[https://news.ycombinator.com/item?id=4543038](https://news.ycombinator.com/item?id=4543038)

------
smabie
If someone is interested in array languages, I would start with learning q,
then k, J, and finally APL. q and k are much more conventional and less
complicated than J/APL. They also tend to be more practical, and less
mathematical. They also don't have multi-dimensional arrays, so the code tends
to be both less general and less confusing. Once you've gotten familiar with
them, J and APL won't seem as weird.

Moreover, it's entirely possible to write regular procedural code in q and k,
so it's easier to transition to the array language mindset.

------
haolez
My intuition for array-based languages is that you can use the "obscure" code,
which heavily uses the standard vocabulary, for the low-level details of the
business logic, but that you would still maintain a set of high-level verbs
(DSL?) to describe in more clear terms what the hell the code is doing.

However, the few real world projects that I've seen in these languages won't
use any kind of DSLs and the argument usually is "for array-based language
programmers, this is actually very clear and concise". I need to improve my J
to verify that :)

------
sally1620
Is J fast enough for image processing? I have to do all of my experiments in C
because every other language is just too slow.

~~~
teleforce
Perhaps you should try Single Assignment C (SAC), it's a fast array based
language with C/Algol syntax but it's proprietary though.

[http://www.sac-home.org/doku.php](http://www.sac-home.org/doku.php)

