

Nimrod: A new approach to metaprogramming - ryeguy
http://nimrod-lang.org/talk01/slides.html

======
progman
Some very interesting features of Nimrod which I would like to emphasize:

\- clean Pythonic syntax (whitespace relevant but tabs are forbidden)

\- native Perl syntax for regular expressions (slide 42)

\- subrange types as in Ada

\- set types as in Pascal

\- strings as case (switch) selectors

\- easy C interface with automatic type conversion (only functions as
parameters need additional compiler pragmas)

\- typed macros (templates)

\- object code: native, C, Javascript (probably more to come)

I am working with Nimrod right now, and it is really one of the most
productive languages that I ever encountered. It seems to mix the best
features of other popular languages. Nimrod actually is where Python should
be. I wonder if it is suitable for small embedded systems also.

By the way, one important difference to Perl regular expressions is that
re"..." always parses the whole expression where Perl parses partially.
Fortunately Perl's behavior can easily be simulated with a template.

~~~
TylerE
For embedded use it's probably ok, anywhere C is ok. It statically links the
bits of the stdlib you use but dead code elimination seems to work pretty
well.

A simple word counting script I just wrote compiles down to about 52k on 64bit
OS X.

(Src:
[https://gist.github.com/tylereaves/7711302](https://gist.github.com/tylereaves/7711302))

~~~
progman
Thanks - but your example is Python code ??

~~~
TylerE
No, it's Nimrod. I set it to python in gist because that's the closest to
nimrod for syntax highlighting purposes.

Edit: Apparenetly not...there are alot more modes in there but the UI exposing
them is easy to miss

------
rdtsc
A GC with a deadline is what caught my attention, seemingly can specify a max
pause time (in ms) and GC wont' take longer than that. That is very good for
soft realtime stuff -- games, audio processing.

Also, Nimrod has a very nice library. Definitely has enough "batteries" in it
to get started.

[http://nimrod-lang.org/lib.html](http://nimrod-lang.org/lib.html)

Just a few impressive ones: http client, server, json parsing, actor support,
redis db driver, zmq, cairo graphics, OpenGL wrappers, even Lua and Python
support.

That looks very appealing, anyone using, what is your experience with it?

~~~
rbehrends
I have been using Nimrod for some personal projects and internal tools since
about version 0.9.0. It is quite usable -- especially the current Github
version -- but as you may expect from a 0.x project, it still has some rough
edges. These haven't stopped me from being productive with it, but you should
be prepared for the occasional "huh?" situation (the most common problem I've
had was that when I experimented with and learned the language, malformed code
that I inadvertently wrote would trigger a compiler error, e.g. an illegal AST
exception).

If you want to do some serious experimentation with Nimrod, I recommend using
0.9.3 (the Github version). It is considerably more mature and stable than
0.9.2.

The one oddity that I'm still getting used to is Nimrod's preference for value
types. I.e., the string and seq[T] types use copying for assignment [1] by
default, when most other languages assume reference semantics (you can use
reference assignment instead, but have to say so explicitly). This may create
inadvertent overhead if you are unaware of it, but also avoids some pitfalls
you otherwise may run into with mutable strings.

As for the default GC, it's not that you can actually guarantee a maximum
pause time [2]. The GC uses deferred reference counting (i.e., eliding most RC
operations and batching the ones that are needed). As long as your data
structures are acyclic, that allows for a very low upper bound on GC pause
times. Pause times for dealing with cyclic structures depend on the size of
the cycles involved (and the size of data structures attached to those
cycles). A practical workaround for when you have to deal extensively with
cyclic data structures is to make use of the fact that each thread has its own
heap: thus, putting all time-critical stuff in one thread where cyclic data
structures aren't used, and the rest in another thread.

[1] Copying is not used for passing arguments to a procedure or for returning
a result, only actual assignment.

[2] You can influence it to some extent by changing the value of ZctThreshold
in system/gc.nim, which specifies the size of the zero count table. Each GC
iteration will still have to scan the stack, of course.

Edit: Oops, to correct myself, Nimrod does actually have a GC_setMaxPause()
procedure, though I still don't think it can _guarantee_ hard upper bounds on
pause times.

~~~
pcwalton
Wait, if the RC scheme has a cycle collector, how do you ensure there are no
CC pauses for acyclic data? Every CC algorithm I know of uses heuristics to
determine when to scan the heap for cycles (usually when an RC drops from >2
to 1) and those can result in pauses proportional to the size of the live set
on the heap, even when there are no cycles.

~~~
rbehrends
I am not sufficiently familiar with the inner details of Nimrod's GC
implementation to give you a full answer, but, no, you don't have to scan the
entire live heap to reclaim cycles.

The technique is called "trial deletion" and only has to traverse potential
cycles. Strictly speaking, a type-agnostic implementation may have to traverse
all objects reachable from an object whose reference count was decremented
since the previous pass, but in a strongly typed language you can skip that
for all objects that can't be part of a cycle. Nimrod at least makes use of
that information in _asgnRefNoCycle()_ in system/gc.nim.

Obviously, if everything on the heap is part of one big cycle, then, yes,
you'll have to scan the entire heap.

~~~
pcwalton
It's still proportional to the live set in the general case. In practice cycle
collectors tend to have to scan a lot, leading to some severe pause times
(30ms in Firefox used to be common until the ad-hoc ForgetSkippable was
added). Heuristics that cycle collectors use tend to fall down a lot in
practice, unfortunately.

------
eliah-lakhin
Very interesting work. By the way, I'm planning to work on the project with
the close purposes, but I have started with a more low-level tool first. My
first work is a library to construct incremental parsers:
[https://github.com/Eliah-Lakhin/papa-carlo](https://github.com/Eliah-
Lakhin/papa-carlo) The idea is that the language's compiler based on this
library will be easier integrated into the exist code editors and IDEs. Maybe
we should somehow unite our efforts with the author of Nimrod?

------
ibudiallo
Impossible to navigate the site on mobile . For some reason it says swipe so
it is supposed to work well on mobile device.

~~~
twotwotwo
It's using [http://nimrod-
lang.org/talk01/Slidy2/scripts/slidy.js](http://nimrod-
lang.org/talk01/Slidy2/scripts/slidy.js) \-- that does indeed seem to try to
support swipe navigation on mobile, but I had trouble with it on Chrome Beta
for Android.

------
ericb
This looks really neat!

Looks like a cross-platform language thats an even more acceptable lisp than
ruby (hello macros!), with multiple build-language targets including
C/javascript/C++ and a fast garbage collector? Like Go, but even cooler?

~~~
ericb
My enthusiasm is greatly lessened. The IDE doesn't compile on Mac OS X. Even
worse, the developers don't care, and keep closing the issue.

[https://github.com/nimrod-code/Aporia/issues/37#ref-
issue-17...](https://github.com/nimrod-code/Aporia/issues/37#ref-
issue-17624074)

As a rule, I avoid any project where the developers have too much of the it-
doesn't-work-and-I-don't-care attitude, mainly because you can't build on top
of a foundation that isn't maintained fastidiously--you'll waste all your time
fixing the foundation and won't get productive work done. Culture matters.

~~~
mortyseinfeld
Don't worry. You'd be better off using a normal text editor anyway. GTK+ is
flaky on non Linux/BSD systems.

They should note what some of the OpenDylan folks are doing and write a plugin
for Intellij.

------
wwwtyro
Cool project. How does it accomplish the realtime GC max pauses of 1-2 ms
after compiling to javascript? Wouldn't the js implementation's GC have the
final say on GC pauses? Thanks.

~~~
RyanZAG
Just being optimistic, probably. You could write a fairly trivial program that
builds and discards massive trees that will force GC to take greater than
1-2ms simply on freeing memory. Maybe they mean in regular operation or
something - but then what is regular? I guess they have to claim impossible
feats to get interest given just how many languages there are already.

~~~
pcwalton
It's deferred reference counting, so the 1-2ms claim is perfectly reasonable
because all it has to scan is the stack (though if they want to break cycles
they will have to do something costlier). The downsides are all of the
downsides of reference counting, plus the fact that last I looked Nimrod's GC
is not thread safe. That last one is a huge downside IMHO. I think DRC is
interesting but I'm not much of a fan of it, honestly: it doesn't solve the
big downsides of RC (cycles and thread safety being super expensive) while
giving up RC's biggest advantage (prompt reclamation).

~~~
RyanZAG
Deferred means you eventually have to do the work. Same for reference counting
- when you free that last element it will have to actually do the freeing. If
you've filled up ram and are allocating objects fast then you can either crash
or do some garbage collection and if the tree is large it will have to take
more than 1-2ms.

~~~
rbehrends
_Deferred means you eventually have to do the work._

Not quite. What you save with deferred reference counting is what is generally
the biggest cost of reference counting: assigning references to local
variables and having local variables go out of scope. Deferred reference
counting only requires an RC update if you store a reference in a heap object
or in a global variable.

You will, of course, necessarily incur the overhead for allocation and
deallocation, which one can make cheaper (bump allocation in generational and
copying GC, memory regions) but can't entirely get rid of, especially in non-
copying/compacting schemes.

~~~
pcwalton
But you lose the promptness of deallocation. Furthermore, I think the biggest
overhead of reference counting is not the count manipulation (especially if
you aren't making it thread safe), it's the inability to handle cycles, which
DRC does not address.

~~~
rbehrends
You are correct that you won't get the immediacy of reference counting; that's
a problem of all memory schemes that aren't naive reference counting. But no,
the biggest constant time overhead for naive reference counting comes from the
RC updates for literally _any_ pointer manipulation. Not just the overhead of
having increments, decrements, and possibly failed branch predictions for
every single pointer assignment, but also what it does to cache locality.
Deferred reference counting essentially only needs a write barrier not very
different from generational or incremental forms of garbage collection. This
is why tuned versions of deferred reference counting come pretty close to
other tuned GC schemes, but naive reference counting falls short, even in the
absence of cycles.

Cycle detection primarily affects pause time, not overhead.

~~~
pcwalton
Pause time is a form of overhead. It reduces both latency and throughput.

Furthermore, the RC updates needed for DRC have to be thread-safe in a
concurrent scenario. I understand that Nimrod has chosen not to have a thread-
safe GC, but I don't think that'll scale for systems programming: too many
algorithms want shared memory, and manual memory management is much more
difficult in a concurrent setting. Once you go thread-safe, RC updates become
extremely expensive, while concurrent GC is well-studied and performant.

~~~
rbehrends
_Once you go thread-safe, RC updates become extremely expensive, while
concurrent GC is well-studied and performant._

This is simply false. In fact, concurrent versions of deferred reference
counting have been known and studied for years [1]. The simple solution is to
store the RC updates in a buffer and execute them only during a collection
phase.

 _Pause time is a form of overhead. It reduces both latency and throughput._

Yes, but for many applications (e.g., HPC) only amortized cost is relevant.

Note also that you _can_ do cycle detection concurrently with the mutator's
operation (again, see [1]). Concurrent cycle detection makes the
implementation more complicated, but no more so than a concurrent tracing
scheme. Moreover, if your program is indeed performance-critical, RC schemes
allow you to minimize the impact on cycle detection by designing your code so
as to avoid cycles (either by using acyclic data structures exclusively or by
using weak pointers). Conversely, tracing schemes make it difficult to
programmatically influence worst case pause time.

[1] E.g., [http://researcher.watson.ibm.com/researcher/files/us-
bacon/B...](http://researcher.watson.ibm.com/researcher/files/us-
bacon/Bacon03Pure.pdf)

~~~
pcwalton
> Note also that you can do cycle detection concurrently with the mutator's
> operation (again, see [1]). Concurrent cycle detection makes the
> implementation more complicated, but no more so than a concurrent tracing
> scheme.

I disagree—it's much more complicated. Concurrent GC is not trivial, but it
doesn't have _seven_ different colors and two different tests (sigma and delta
test).

A good concurrent reference counting system is just as complex as a good
tracing garbage collector, because it requires the same runtime
machinery—precise stack maps, write barriers, etc., to perform backup tracing.
But it also has the reference counts to deal with. It ends up being more
complex overall.

> Moreover, if your program is indeed performance-critical, RC schemes allow
> you to minimize the impact on cycle detection by designing your code so as
> to avoid cycles (either by using acyclic data structures exclusively or by
> using weak pointers).

No, that's my point—even if you use acyclic data structures, there is no
guarantee that you won't have the CC run because you had multiple references
to an object and you dropped one, making the cycle collector suspect it was
part of a cycle. This is the entire reason why ForgetSkippable was added to
Firefox: this happened a _lot_ in practice. The solution was to add a bunch of
ad-hoc code to make the cycle collector drop objects from the purple buffer,
much like the "acyclic" pragma does in Nimrod—but now you have the possibility
of leaks unless this is done very carefully.

> Conversely, tracing schemes make it difficult to programmatically influence
> worst case pause time.

In collectors like Metronome, it's extremely easy: set the pause time to what
you want it to be. That's the only solution that's really scalable in my view:
working around the collector by manually telling the CC not to scan certain
types feels too error-prone.

------
twotwotwo
It has a lot of cool stuff, and I'm intrigued by its approach to threading.
(Per-thread heaps and message passing, with an unsafe shared heap if you need
it.)

How does dynamic typing work in Nimrod, if it has it? What's the equivalent of
being able to call Write() on any io.Writer in Go, for example? [Edit: just
found the explanation at [http://nimrod-lang.org/tut2.html#dynamic-
dispatch](http://nimrod-lang.org/tut2.html#dynamic-dispatch) but have not yet
read and digested.]

I'd love to see some open-source projects that've been written in it--the
Nimrod tools and stdlib are obviously a good start. I think one of the things
that really helped Go was having lots of n00b-friendly content (the tour, a
short "spec", Effective Go doc, blog posts, talks); you could call it
"marketing", but it really helped me, at least.

------
fxtentacle
nimrod sounds even better than google go. why did you start it? what's the
desired program that made you design nimrod?

------
msellout
The html slideshow fails to work on Chrome on Samsung Galaxy S4. It tells me
to swipe left to move to the next slide, but nothing happens.

------
humpty44
This language looks very very interesting. Expressive and efficient languages
are hard to come by. I really like the name too.

------
catwell
Every time I see that name (Nimrod) I think it is about
[http://www.messagelab.monash.edu.au/Nimrod](http://www.messagelab.monash.edu.au/Nimrod)

Why can't people find original names for software? :(

------
rcthompson
It says the GC is implemented in Nimrod. Does the GC code avoid using the GC
itself?

~~~
drakeandrews
I presume by allocating memory yourself. It does mention that if you don't use
the GC, it is removed from the final program.

------
eonil
Nimrod is the one which is closest to my hypothetic ideal language in
features.

~~~
dirkk0
Yes, that's what I thought, too.

My only complaint is that my internal Python (awesome) vs. JavaScript (not so
awesome but becoming standard) war gets tackled from a third side now. Another
language to care about - not exactly what I needed.

Having said that: Nimrod looks beautiful and maybe it's time to hop on the
bandwagon.

------
discreteevent
I think I could get along with this language.

------
goggles99
Am I in a time warp? Nimrod is over 5 years old. Yet most comments here think
it's new and groundbreaking with statements like _nimrod sounds even better
than google go_. Nimrod is a cool language, but am I missing something here?

Original site with almost same description verbiage:
[https://web.archive.org/web/20110704041631/http://force7.de/...](https://web.archive.org/web/20110704041631/http://force7.de/nimrod/)

~~~
gfodor
Why is it confusing that an obscure programming language, on any given day,
will be seen for the first time by a lot of people when it gets onto the front
page of HN?

