
Carp: a statically typed lisp, without a GC, for high performance applications - adamnemecek
https://github.com/eriksvedang/Carp
======
junke
Is there an explanation of the memory allocation used? There seem to be a GC
([https://github.com/eriksvedang/Carp/blob/master/src/gc.c](https://github.com/eriksvedang/Carp/blob/master/src/gc.c)),
maybe used in the compiler only, coupled with lifetime analysis
([https://github.com/eriksvedang/Carp/blob/8665a7f9a19d9347cc0...](https://github.com/eriksvedang/Carp/blob/8665a7f9a19d9347cc0f5785cee946fc25d1f645/lisp/calculate_lifetimes.carp)),
which as far as I know is used once
([https://github.com/eriksvedang/Carp/blob/6107e619d4f632acace...](https://github.com/eriksvedang/Carp/blob/6107e619d4f632acacef9c9d2cf808e06353158c/lisp/builder.carp#L177))
in the compiler. This analysis seems tied to the manual "reset!" special form
which apparently frees any existing data bound to a symbol. So, what happens
when lifetime analysis cannot determine lifetime? what about stack-allocation?
What should I worry about when programming in Carp w.r.t. memory?

~~~
larsbrinkhoff
As far as I understand from listening to Erik talk about Carp, there is an
interactive interpreter which does use GC. In contrast, the compiler generates
GC-free code.

~~~
tom_mellior
> In contrast, the compiler generates GC-free code.

Yes, but the question was _how_ it does that. Does it reject code where it
cannot reliably detect lifetimes? Or does it generate code that just does not
deallocate (i.e., that leaks) objects whose lifetime the compiler can't
determine? Both of these are "GC-free"...

~~~
larsbrinkhoff
The compiler does lifetime analysis, and there is a "reference borrowing"
concept like in Rust.

~~~
madgar
So it rejects code for which it can't prove lifetimes. Why not just say so?
There's an uncomfortable degree of beating around the bush in this thread.

~~~
akavel
Curious if semantics of this subdomain of "allowed/legal" programs will be
easy to understand, or will it be "try and see on case-by-case basis if your
code compiles". Still, this way or the other, probably user will have to
gradually learn and build intuition, as with learning any other aspect of any
programming language.

Also: will be/is the lifetime-checking done in the same way in the
interpreter, even though it uses GC? Otherwise, those are in effect two, I
think significantly different dialects of the language.

~~~
eriksvedang
It will require roughly the same amount of practice as when learning Rust, so
a pretty significant learning effort I'm afraid.

I'll try to do as much checking as possible in the interpreter so that the
transition from interpreted to compiled code is smooth. Right now there is
none though.

------
akavel
I understand this is marked as "a research project", so any answer here is
possible and must be accepted; that said I'd really love to learn at least
about plans with respect to the following questions:

• Regarding FFI, is it possible to pass pointers to carp functions/closures as
arguments to external dynamically loaded functions? are those features
available in REPL?

• What's the story/approach regarding structs inheritance vs. composition?
also custom types and implicit type casting?

• What are the plans regarding multithreading/parallelism? Esp. interesting
with respect to memory handling.

At its current state of claims, the language sounds awesomely close to the
"holy grail" for me at first glance — though with a fair bit of vagueness and
uncertainty.

~~~
eriksvedang
1\. Yes, and yes!

2\. No plans for inheritance at the moment, probably some kind of interfaces
but that's not a top priority right now. Custom types are limited to structs,
I will add union types soon

3\. This has not been decided yet, I want to do more research before settling
on any particular solution. It will be a high priority though, since I want to
use it for the games I write

Thanks, Erik

~~~
cbebdhhd
Could you pretty, pretty please allow for an option to have automatic
parentheses insertion, similar to how javascript will insert semicolons,
making them optional? This is almost exactly the language that I've been
planning for the last six months, and if you would be so kind as to implement
that one feature, it would save me a lot of reinventing the wheel.

I was originally envisioning having the compiler being able to infer
parentheses based on code indentation.

Also have you considered compile time AST evaluation and simplification?

~~~
imtringued
If you don't want the parenthesis then why not use reverse polish notation?

~~~
blastrat
unless you're thinking of something that I'm not thinking of? RPN only solves
this when the operators are known and used exclusively in operator context and
can be relied on as implied close-parens

------
eggy
Looks like just what I have been trying to find, since I am incapable of
writing my own. I did start with 'Learn C * Build Your Own Lisp' book [1], and
I will finish it for educational purposes; it is a great, short book to work
through, and should help with grokking Carp.

The closest I have come to finding something similar for livecoding visuals
and audio, or games is Extempore [2], which has xtlang, a Scheme, with manual
memory management and types. The only two criticisms I have, and they are
small, are the build and number of dependencies, and how to distribute
standalone executeables. In all fairness there are binaries now available for
all platforms, but standalones are still problematic.

I curse Fluxus [3] for starting me down this crazy road many years ago in
search of the best creative coding environment. Shame it is not refreshed
every so often, because the interface is brilliant.

I'll have to look over Carp this weekend more closely. I am intrigued how it
is like Rust's semantics. And if it is manual memory management all the way,
then I guess that is where it differs from Rust's safety features?

Now I don't have to try Clojure again! I don't like the JVM, and prefer going
to C.

[1] [http://www.buildyourownlisp.com/](http://www.buildyourownlisp.com/)

[2] [http://extempore.moso.com.au/](http://extempore.moso.com.au/)

[3] [http://www.pawfal.org/fluxus/](http://www.pawfal.org/fluxus/)

~~~
ddrdrck
You should have a look at Hypergiant library for Chicken Scheme. It compiles
to C as well.
[https://github.com/AlexCharlton/Hypergiant](https://github.com/AlexCharlton/Hypergiant)

~~~
eggy
I did. I'll have to revisit it. I gave it up, since I work on Windows and
Linux mainly, and I think I couldn't get one or more of the dependencies
working on Windows, and I gave up. I really like Chicken, but with Chez Scheme
having gone opensource, I use it now when I am working in a Scheme.

I've also looked at Lobster, which is now statically typed by default, not
optional.

------
jlarocco
I saw this a month or two ago and submitted a pull request that I still need
to fix up to have it accepted. My original intention was to benchmark a bit
against SBCL, but I got sidetracked with the pull request, then lost time and
then forgot about it. I need to get back to it and see about getting it
submitted.

Anyway, it sounds neat, but even after playing with it a bit, I'm not sure how
necessary it is.

SBCL (and supposedly the commercial Common Lisps) are pretty fast nowadays.
Not C fast, but I can usually get about 1/2 - 1/3 C speed and 4-8x faster than
Python without doing anything special.

For example, I made an animation app ([https://github.com/jl2/qt-fft-
viz](https://github.com/jl2/qt-fft-viz)) that reads an MP3 and generates an
animation while it plays back, and I get 90+ fps. It's not a big graphics/CPU
intense video game, but it's doing a lot of FFT calculations and some basic
graphics.

So, not to diminish the Carp project, but I think regular old Common Lisp is
faster than most people think, is better supported, has more libraries, and
works on more platforms and operating systems.

~~~
larsbrinkhoff
I believe one major motivation behind Carp is the lack garbage collection. Or
rather, avoiding GC pauses. So, maybe you could benchmark Carp vs SBCL in that
area?

------
killercup
Interesting.

> Carp borrows its looks from Clojure but the runtime semantics are much
> closer to those of ML or Rust.

> –
> [https://github.com/eriksvedang/Carp/blob/c762e9ff07544b40c8d...](https://github.com/eriksvedang/Carp/blob/c762e9ff07544b40c8db2c32584f1d1f6e9ea92e/LANGUAGE.md)

------
_pmf_
To the author: please, please, please port (or reuse) LuaJIT's C-header parser
for FFI; it's the best FFI interface specification method if seen among JNI,
Python, V8 and even C# (because you can just use the C header, with support
for typedefs, unions and packing).

It really looks like a great language!

~~~
eriksvedang
Interesting, thanks a lot for the tip!

~~~
lemming
Also check out Pixie's - towards the end of
[https://www.youtube.com/watch?v=1AjhFZVfB9c](https://www.youtube.com/watch?v=1AjhFZVfB9c).
It's a really interesting technique, and for a Lisp too.

------
acron0
This is fantastic. AFAIK there are no Lisps our there that are designed with
games in mind which is a huge shame because I often think how perfect it'd be
for game dev (Arcadia and Nightmod spring to mind but imo they're shoehorns.
Good ones, but still).

~~~
rurban
There are a couple of high-performance, low-latency lisps, either with a low-
latency GC or refcounted or compiling to C as Carp does.

ECL (also typed), Vicare, Larceny, Ypsilon, the new guile, Gambit-C, Chicken,
Chez, Bigloo, stalin, Corman CL, or fast typed CL: sbcl, franz, lispworks and
few more.

Or the ones compiling to LLVM or JVM or .NET. Most prominent Clojure with a
very unlispy syntax.

~~~
rurban
And note that you simply get a low-latency GC with libgc (Boehm-Weiser) by
using the incremental GC with GC_enable_incremental() and SMP with parallel
marking phases -DPARALLEL_MARK.
[http://www.hboehm.info/gc/scale.html](http://www.hboehm.info/gc/scale.html)
This is no voodoo nobody ever managed to do, even if most blog posts for
popular not-invented-here languages state the opposite.

Of course libgc is pretty simple and slow, good for foreign code (i.e. ffi's
needed in games), slower than Cheney-2 finger copying allocators which I got
down to 10ms, compared to 100-300ms for mark-sweep. But copying collectors
need 2x memory, so not usable for small devices such as phones. And a bit
complicated for foreign memory, which prefers conservative mark & sweep.

------
hamidr
Hmm I actually like the fact that the arguments of a function are defined
using square brackets. It just reminds me of Clojure, especially the "defn"
macro, which sounds pleasant to me.

I wonder what Lispers would say about that since I heard that not having a
homoiconic syntax might cause some troubles at metaprogramming(macro) level.

~~~
junke
It is homoiconic, because the data structures used to describe code are also
data structure provided by the language. However, brackets mean that you pass
arguments as arrays instead of lists: there is little reason except aesthetics
to have this distinction in source code. I prefer parenthesis everywhere, but
some people really dislike that.

~~~
calibraxis
Yes, it's still homoiconic.

I think vectors for argument lists is mainly a usability affordance. Which
gives you aesthetics for free. (If you happen to find it more aesthetic.)

> _Common LISP and Scheme are not simple in this sense, in their use of parens
> because the use of parentheses in those languages is overloaded. Parens wrap
> calls. They wrap grouping. They wrap data structures. And that overloading
> is a form of complexity by the definition I gave you._
> ([https://github.com/matthiasn/talk-
> transcripts/blob/master/Hi...](https://github.com/matthiasn/talk-
> transcripts/blob/master/Hickey_Rich/SimpleMadeEasy.md))

Elsewhere, he critiques Clojure's use of vectors for argument lists. Because a
vector implies order, so every caller must put arguments in the right order.
(To decomplect, you'd use maps instead of vectors. But vectors win you
brevity. Many notice that with longer argument lists, maps increasingly become
more attractive than long argument lists.)

~~~
junke
Take C:

    
    
        struct {
           int x;
        } ...;
    
    
        int foo () {
            return 0;
        }
    
    

Should braces in both cases represent the same internal data structures?
Probably not, the first one is for structure members and the other one for a
block of statements. But in Lisp, characters are used to parse the same kind
of data, which means they are same structure in all contexts. The Lisp reader
has a simple approach to parsing, you don't generally change the readtable's
binding based on which form you are reading (but strings, comments and code
are not read the same way). Using different characters for semantics has its
limit.

In Clojure, take [a b] out of context. Which element is evaluated, a, b, none
or both? The fact that it is a vector does not help you, because it could be a
binding, an argument list or a vector constructor.

Also, _how_ something is stored inside the AST has nothing to do with its
meaning at runtime. Argument lists could be passed using the stack, but you
don't actually write a stack inside the source code. The fact that they are
written as vectors or lists does not matter either, you still have to know the
semantics. And semantics is almost never context-free.

------
Pxtl
That's brilliant... I'd always assumed lisps were so tightly bound to the GC
that any other memory model was infeasible.

~~~
toolslive
you might enjoy this classic then:
[http://www.pipeline.com/~hbaker1/LinearLisp.html](http://www.pipeline.com/~hbaker1/LinearLisp.html)

------
makufiru
This looks amazing and very close to a project I've been working on myself - a
clojure-ish lisp-1 with Rust style memory management. Very cool work!

------
krige
I'm fishing for a pun in there but just can't catch a big one.

------
Roze
Is there a compiled binary for Windows yet??

------
pritambarhate
Are all those round brackets necessary? I understand the goal is Lisp like
syntax. But while reading the OpenGL sample, I found the enclosing brackets
quite unintuitive. Since this is still supposed to be new language may be
"statically typed Lisp with python-like blocks" will appeal to a larger
audience?

~~~
chriswarbo
One of the nice things about s-expressions ("all those round brackets") is
that they're well suited to meta-programming.

For example, you can automatically convert back and forth between
s-expressions and something else, like i-expressions
[http://srfi.schemers.org/srfi-49/srfi-49.html](http://srfi.schemers.org/srfi-49/srfi-49.html)

