
A look at the J language: the fine line between genius and insanity (2012) - jxub
https://scottlocklin.wordpress.com/2012/09/18/a-look-at-the-j-language-the-fine-line-between-genius-and-insanity/
======
scottlocklin
This is a really old blog of mine. I was so influenced by J I tried to do a
startup in 2015/2016 making a more legible APL/TSDB. At present I'm more or
less running a small cryptocurrency HF in J and using the J database engine
jd.

~~~
throwaway7645
Hey Scott...didn't know u were on HN. I've enjoyed most of your writing
tremendously and am sad Kerf didn't work for you and your business partner. I
hope you find time to write more about J in the future. Your comment on not
wanting to write another single For-Loop has really resonated with me. I've
played with J/APL/CommonLisp/Clojure and nearly another dozen languages, but
Python has traditionally been my daily driver. Recently though I've been using
Mathematica and have been very impressed with the terseness. There are
zillions of functions that are very easy to combine to get things done. It is
kind of like a Lisp on the backend as most functions consume and create lists.
I've found having primitives for charting, graph-theory, numerical solvers,
and OS level stuff to be very enjoyable with no explicit looping. They also
have plenty of primitives for things like FFT and Linear Algebra...the price
is quite high though.

~~~
scottlocklin
I guess I use R for the sorts of things I might use Mathematica for.
Mathematica is kind of the opposite of J; with J you only have a few
primitives. Mathematica is a swiss army knife for data and math. Python is
fine; I was an early adopter, I just don't have any reason to prefer it to R
(package quality perhaps).

Kerf was a tough slog, but Kevin and I landed on our feet. We'll do something
with the code one of these days.

~~~
throwaway7645
Agreed on that front. I've used R & Mathematica as a swiss-army-knife for
those things as well, although the stats stuff I do is a lot more basic than
the stuff you do professionally. I'd like to get more proficient at J & APL,
but that is a long road I've found.

Glad to hear y'all found something after Kerf. I would've guessed there would
be some market for a tsdb cheaper than kdb+ (what I assume y'all were trying
to do). Your blog post mentioning overhyped tech really gave me a chuckle. In
college I started to focus on nanotechnology before going to a career fair and
talking to some companies. Thousands of students getting degrees focused on
that and there are only a handful of startups hiring researchers and only so
many grad-student type jobs. I had a physics professor tell me a niche field
comes up like that every decade which gets a lot of funding, but essentially
under delivers and dies out. I think there is some benefit to nano materials
like high strength cables, but not the miracle sauce that is getting preached
everywhere.

~~~
scottlocklin
Material science is real; nanotech, not so much. Glad you found your way out
of that dead end. I have a friend with a Ph.D. in nanotech. He is a pretty
good GPU programmer.

------
ktamura
Programmers continue to discover and re-discover APL and array-oriented
programming.

If someone is serious about array programming/APL and wants to make a
lucrative career in it, study KDB+/Q/K. There's a small but active community
around them.

Just to relate an anecdote: I used to be a fairly active KDB+ programmer in
finance but left both finance and programming ~8 years ago. Just yesterday, I
got an email from my friend who's building a statarb fund around
cryptocurrencies. They were going to use KDB+ and offered me a job for
"300-500k base with incentives"...

My friends clearly don't know that I now work in sales & marketing :D

Aside: Roger Hui, the creator of J, is a programming sibling of Arthur
Whitney, the creator of K/Q and the founder of www.kx.com

~~~
qznc
I understand the advantages of array-oriented programming, but why the J/APL
syntax? There are plenty of column-oriented databases available.
[https://en.wikipedia.org/wiki/List_of_column-
oriented_DBMSes](https://en.wikipedia.org/wiki/List_of_column-oriented_DBMSes)

Also related is range-based programming.
[https://wiki.dlang.org/Component_programming_with_ranges](https://wiki.dlang.org/Component_programming_with_ranges)

    
    
        return
            // Start by generating all dates for the given year
            datesInYear(year)
            // Group them by month
            .byMonth()
            // Group the months into horizontal rows
            .chunks(monthsPerRow)
            // Format each row
            .map!(r =>
                    // By formatting each month
                    r.formatMonths()
                     // Storing each month's formatting in a row buffer
                     .array()
                     // Horizontally pasting each respective month's lines together
                     .pasteBlocks(colSpacing)
                     .join("\n"))
            // Insert a blank line between each row
            .join("\n\n");
    

Much more readable than something like "(~R∊R∘.×R)/R←1↓ιR".

~~~
ken
_Is_ it more readable? I'm even less familiar with D than APL, and that D code
only looks more readable to me right now because it's mostly comments.

Wouldn't adding 8 lines of comments help newcomers to APL, too? This seems
like a rather unfair contest.

Familiarity counts for a lot. 25 years ago when my primary language was 6502
assembly, I would have said 20 pages of assembly code look more readable than
either of these.

In the long run, though, in every field I've worked in, being able to turn 20
lines of code into 20 characters has turned out to be a huge benefit. Once you
familiarize yourself with the vocabulary, you can operate at a higher level.
Nobody fears concise names and symbols when it's in a context they understand.
I think even the most APL-phobic would admit that "a←b+c" is more readable
than "assign(a,sum(b,c))".

------
aplorbust
Heres the .q file I use to read HN. Hobbyist programmer;
corrections/improvements welcome. Better yet, how would I do this in J?

cat 1.q

    
    
       \c 2000 200
       k)-23!t:select from .:`:t.kdb
       k).Q.fs[{`t insert +:`url`title`user`item!("SSSS";",")0:x }] `:hn.csv 
       k)-23!t:?:select from t 
       `:t.kdb set t 
       \\
    

As for how to get hn.csv, I use shell scripts to download all the pages of HN
html (in accordance with robots.txt specified delay). Then I use a simple
lexer (made with flex) to transform the html to hn.csv with the above selected
columns.

After some years, the file hn.csv can grow quite large, larger than available
memory.

To read and select articles from t.kdb, I use less(1) and some small shell
scripts.

This lists all the articles, starting after a given offset, or zero by
default:

    
    
          echo -e '\\c 2000 2000\nk)select i,title from .:`:t.kdb where i>'${1-0}'\n' \
          |exec q > 1;
          exec less 1;
    

Then, to select and read an article or its comments I use the ":!cmd" feature
in less(1) to invoke an appropriate shell script (actually, an execline
script). Quit, and I am returned to less.

Have seen lots of systems for reading HN made by HN readers, using various
databases I guess. This is mine, using kdb+.

~~~
adrianN
Neat, but why don't you use lynx?

------
prezjordan
Shameless plug but I recently picked J back up as a hobby and it's been super
fun. You can read some of my findings here:
[http://thatjdanisso.cool/j-fibonacci/](http://thatjdanisso.cool/j-fibonacci/)
[http://thatjdanisso.cool/j-pascal/](http://thatjdanisso.cool/j-pascal/)

I encourage you to give this language a shot - it's got some really cool
ideas.

~~~
nickpeterson
These are cool, but how practical would it be to produce a blog post akin to,
"Here is a web app running J in the back end"?

------
zengid
Somewhat tangential, but some concepts and operators from J have been
integrated into the audio programming language sclang for Supercollider [1].
Pretty neat! (I'm currently learning SC [2]).

[1] [http://doc.sccode.org/Guides/J-concepts-in-
SC.html](http://doc.sccode.org/Guides/J-concepts-in-SC.html)

[2] [https://github.com/Engid/superfun/blob/master/first-
impressi...](https://github.com/Engid/superfun/blob/master/first-
impressions.md)

------
javiermaestro
I've read a bit about APLs and I've seen them popping up in HN a few times.
They do seem incredibly awesome and the products around it (K/KDB+ etc)
amazing.

However, in case I ever felt like learning one of these, I'd always wanted to
know: the syntax crazyness, "a la brainfuck", is it a feature? a bug? Is it
really needed? Can't they make an APL "look like Python" but using the same
concepts, etc? Or is it "a thing" / something ultra-core to these languages,
to be extremely terse / cryptic and intensely dense (each line packs a ton of
punch!)?

e.g. just read
[http://www.jsoftware.com/help/dictionary/didot.htm](http://www.jsoftware.com/help/dictionary/didot.htm)
and some of the stuff in there can be translated quite well:

i. 5 --> list(range(0, 5))

i. _5 --> list(range(0, 5))[::-1] or list(range(4, -1, -1))

i. 2 _5 --> [list(0 + j, 5 + j)[::-1] for j in range(0, 10, 5)]

P.S. any pointers to content / books about the philosophy and ideas behind
these programming languages are welcome! :D

~~~
throwaway7645
Certain things translate, but not everything. In J when almost everything is
an array and you're using points free form, you can pretty much combine
everything without having to build classes...etc. Those python examples are
much longer even for simple stuff. As far as books go, Dyalog APL has
"Mastering Dyalog APL" for free and although Dyalog costs money, a hobby
license is free. There is a guy who writes programming languages and books for
a living. He recently wrote an array language called "Klong" and a book on it.
J has several books. There's not a whole lot on the philosophy, but using it
teaches you what you need. It looks like line noise at first, but you start to
recognize idioms and phrases. Aaron Hsu did two HackerNews discussions on here
and a YouTube broadcast on his "co-dfns" GPU compiler for Dyalog APL. That was
pretty amazing. He talks about trying to write it in Scheme, Java,etc...but
APL lets him put it all on a few pages to where he sees everything at once. At
his level, abstractions aren't necessary.

~~~
matthewwiese
I did a cursory Google search for Klong after your mention. The author of the
language (and its book) posts his writings to t3x.org

Thank you for the brief mention -- I've been perusing his website and writings
and am enamored. There's something about niche, obscure tools (and their
authors) that give me some profound joy.

~~~
throwaway7645
I was thoroughly impressed as well. It would take me years to read all his
stuff. Klong us actually written in his other language. That is impressive.

------
useful
As someone who has written APL professionally, Numpy and Nd4j are much better
choices to spend your time on. It might not be exactly the same thing, but
thats the point. I only miss the linear algebra from APL and there are very
awesome libraries that do better.

~~~
throwaway7645
Numpy does have much better scientific capabilities, so you can use vanilla
Python for the basic coding and Numpy routines when you need them.

APL uses arrays for everything though. I just wish it had taken off more on
the scientific front instead of the financial side as the linear algebra stuff
isn't nearly as fleshed out as Matlab/Numpy/Mathematica.

------
doug1001
read through the OP and scanned a few examples on the J Project Wiki

i suspect that, to capture the full value of J, one needs to already know the
array-oriented paradigm (or at least have a strong motivation to learn the two
in parallel)

knowing the AOP paradigm also helps understand the programming patterns as
well as the syntax

still, the dense syntax has extraordinary power

for instance, n-queens in J (from Rosetta Code) is just 4 short lines:

    
    
        perm   =: ! A.&i. ]             
        comb2  =: (, #: I.@,@(</)&i.)~    
        mask   =: [ */@:~:&(|@-/) {
        queenst=: comb2 (] #"1~ mask)&.|: perm
    

a fundamental AOP patterns is evidenced here:

line 1: create the 2D array (n x n chessboard)

line 2: populate the array with candidate solutions

line 3: eliminate invalid solutions ('mask' them)

~~~
kd0amg
_i suspect that, to capture the full value of J, one needs to already know the
array-oriented paradigm (or at least have a strong motivation to learn the two
in parallel)_

Yes, that's the power that's specific to J. Having a library of functions like
anagram index, nub, antibase, matrix inverse, etc. can be done in pretty much
any language. So can using really short names for everything (have a look at
the J implementation). What J has that isn't so easy to export is the way
first-order functions work on arbitrarily high-rank arguments, and even a lot
of APLs don't let you mix argument ranks in the way that J does (e.g., vector-
matrix addition). Without the inherently rank-polymorphic function application
semantics, it would just be an idiosyncratic pointfree programming syntax for
people who don't want to use anything higher than second-order functions (and
maybe have a grudge against static parsing).

------
danidiaz
The talk "APLicative Programming with Naperian Functors" shows an approach to
APL-like programming in Haskell
[https://www.youtube.com/watch?v=D1sT0xNrHIQ](https://www.youtube.com/watch?v=D1sT0xNrHIQ)

Many APL idioms, when translated to Haskell, are related to applicative
functors. A smidget of dependent types is needed as well, to express the
dimensions of vectors, arrays, and the like.

------
nickpeterson
Writing J code feels a bit more like you're interacting with a database, in
that the code is being executed in a way that isn't immediately obvious based
on the logical instructions you give it. It seems like the higher level and
terser you can make the instructions, the faster it will generally run,
because J can more ideally optimize the operations.

