
An introduction to Kerf - jmount
https://scottlocklin.wordpress.com/2015/12/15/an-introduction-to-kerf/
======
srpeck
Great description from the reference manual
([https://github.com/kevinlawler/kerf/blob/master/manual/refma...](https://github.com/kevinlawler/kerf/blob/master/manual/refmanual.pdf)):

"Kerf is a programming language built on pragmatism, borrowing ideas from many
popular tools. The syntax of Kerf will be familiar enough to anyone who has
programmed in C, Python or VBA. Data is described using syntax from JSON
(JavaScript Object Notation), a text-based data interchange format. Queries to
search, sort and aggregate data can be performed using SQL syntax. Kerf’s
built-in commands have aliases which allow programmers to use names and terms
they are already used to."

------
yummyfajitas
I don't think the authors claim that "The only thing comparable to it from an
engineering standpoint are the other APL based databases such as KDB and Jd"
is correct, at least not based on this blog post. Numpy+pandas also seems
pretty similar, as is breeze, and Julia has similar features also.

Let me emphasize that while I'm about to implement most of his blog post in
numpy/pandas, I don't mean to come off as critical or dismiss building a new
one. Kerf looks pretty cool. I just don't think this post gives much reason to
believe there is nothing comparable to it.

Idiomatic numpy:

    
    
        def wavg(x,y):
            return sum(x*y) / sum(x)
    

Less idiomatic, but analogous to kerf or J:

    
    
        def wavg(x,y):
            return add.reduce(x*y) / add.reduce(x)
    

arange(1,6) is equivalent to his range(5), and full((5,), 1) is equivalent to
repeat(5,1).

The SQL-ish engine looks fairly similar to pandas:

    
    
        df = pandas.DataFrame({'a' : arange(5), 'b' : ['a', 'b', 'c', 'd', 'e']})
        #Result is:
           a  b
        0  0  a
        1  1  b
        2  2  c
        3  3  d
        4  4  e
    
        df[df.a > 2]
        #Result is
           a  b
        3  3  d
        4  4  e
    

Pandas similarly has joins, averages and other aggregates, etc. E.g.:

    
    
        pandas.merge(df1, df2, how='inner', left_on='a', right_on='b')
    

is equivalent to:

    
    
        SELECT * FROM df1 INNER JOIN df2 ON df1.a = df2.b;
    

Syntactically it's a bit different (embedded SQL is pretty cool) but from this
blog post I don't see any major semantic differences.

However, if there are important differences under the hood, I'd love to hear
about them. Again, I'm really not trying to criticize Locklin for rolling his
own, I'm just disputing a minor point.

~~~
scottlocklin
There are lots of "table" like solutions out there: R's data.frame and
data.table are obvious examples. Speed, the ability to persist data as an
actual database system, and the ability to work with data "out of core" would
be the advantages over Pandas. I'll be releasing various performance
comparisons over coming weeks to demonstrate these advantages.

~~~
yummyfajitas
Cool, any chance you'll write a blog post discussing that? Being a good column
oriented DB combined with a decent vectorized language is definitely not
something so easy to find - blaze purports to be that system (sort of), but it
isn't really living up to the hype.

~~~
scottlocklin
Of course we'll blog on this topic! Just not today.

That's what the APL world brings to the table.

FWIIW, you can read about my comparing Jd (another APL related columnar
database) to a Blaze like tool (Spark bigDF) on the J list:

[http://jsoftware.com/pipermail/database/2015-May/000076.html](http://jsoftware.com/pipermail/database/2015-May/000076.html)

------
brudgers
Kerf homepage: [http://www.kerfsoftware.com/](http://www.kerfsoftware.com/)

Kerf github:
[https://github.com/kevinlawler/kerf](https://github.com/kevinlawler/kerf)

~~~
zem
seems vaguely parasitic to be using github to distribute closed-source
binaries, not just for the bandwidth hit, but more so for trying to use the
open source ecosystem to gain mindshare for a closed product.

~~~
scottlocklin
Why? We pay them. Many companies do. This is why github exists as a company:
because people give them money for things.

~~~
zem
right, my mistake. sorry about that.

------
svan99
Good intro Scott. I just start playing with Kerf and it seems to me a pretty
well written piece of software with potential. I like the json style
assignment. Will keep an eye on this.

------
th0br0
Why do these products always have to be closed source... especially when the
market's already fairly saturated, this seems to make little sense...

~~~
srpeck
Market segmentation, premium pricing, signalling, etc. Basically, when your
product is good, you can charge more for it. Free is a great price to get
exposure, but unnecessary for extraordinary profitability (see Kx).

~~~
fra
Charging enterprise makes sense to me, but I would have thought that allowing
free non-commercial use would help with adoption.

------
mynegation
Ha! I've asked the question about existence of something like Kerf quite a
while ago[1]. Good to finally have an answer other than "it is not really
needed".

[1]
[https://news.ycombinator.com/item?id=7890931](https://news.ycombinator.com/item?id=7890931)

~~~
scottlocklin
Nial is a good example of a Kerf-like language, but I don't think you can
download it at present.

There's a way to make J look more readable, but by the time you're able to
accomplish this, you can read the J runes, and have realized the syntax sugar
just slows down the parser.

~~~
avmich
Syntax sugar also slows down the REPL experimenting way of "growing" J
scripts.

Yes, it's harder to read... You can combine wise naming scheme with good use
of spaces - and also short lines, which anyway carry a lot of work - to make
programs more readable. Comments are also available :) - in fact, I found that
J really encourages the literate programming style. As they say, "in APL you
solve a problem in 5 minutes, then spend an hour trying to write a most
elegant and short solution". Indeed.

------
placeybordeaux
Any word on what the license will look like?

Gonna grab the current binary to play around with, I love learning new styles
of programming and kerf seems rather approachable compared to the others in
it's family.

~~~
otterley
It's a commercial product, closed-source.

------
mosquito242
Slightly unrelated question - where would I find an interpreter for K so I can
start playing around with the language?

~~~
srpeck
32-bit version of q/k4/kdb+: [http://kx.com/software-
download.php](http://kx.com/software-download.php)

This version is technically unrestricted, but cannot be used for commercial
purposes.

~~~
mosquito242
thanks. I've seen that before, but for some reason I kept assuming that it was
different than an actual k interpreter because none of their docs refer to the
K language.

~~~
srpeck
q is a language written in k - when in the q interpreter, you can drop into k
by typing a single '\', and then return to q using the same method.

------
jerf
Nice name. Pity about the 8th bit, though.

