It's extremely interesting, but it has a to be noted that, even though the main page claims 100% compatibility with Clojure, the status page lowers the claim down to 42%, while the GitHub page states you can't even build it.
Everything on the landing page talks about jank's goals and aspirations. 100% Clojure compatibility is a big one. The progress page starts with this disclaimer:
> jank is under heavy development. It's safest to assume that any feature advertised is partially developed or in the planning stages. There is no sales pitch here; just a lot of work and some big plans. All development happens on Github, so watch the repo there!
The usage of "jank" here refers to its design; the concept of jank. This is different from its current implementation. jank is designed to be many things, as the landing page shows, and the implementation is catching up with that.
No matter how you spin it, this makes the landing page very misleading.
Nobody cares about the "design" of a language when there is no implementation that fully realizes the design. Otherwise, anyone can say "my language is 100% compatible with clojure, but oh, yeah, the clojure compatibility part isn't fully implemented yet".
If your landing page said "jank's goal is to be 100% compatible with clojure", or "jank is designed to be 100% compatible with clojure (when fully implemented)" that would be fine. Right now the statement is just false, because right now, the language isn't 100% compatible with clojure.
I really think you should change the wording -- and I say that as someone genuinely interested in the project. I hope you achieve 100% clojure compatibility, that would be really cool. With the way things are worded now the first impression you are giving people is that the language implementation is way more complete than it is, which I believe will be a big negative for you in the long run.
Most libraries I tend to use are generally pure Clojure, with some reader conditionals at the edges. jank will also support reader conditionals, so those libraries can grow to support C++ interop where needed.
You could probably make everyone happy just by copying that text to the top of the landing page, as an additional paragraph just above where the "Current progress" button is, right after "..ensure clean and correct multi-threaded designs."
Can you explain the type checking a bit more? Is this occurring within the JIT for compilation efficacy or is the goal here that you would get gradual type inference across a codebase facilitated by some kind of LSP?
Type checking gets wonky in the REPL world, since things are changing a lot. During JIT usage (i.e. REPL usage or running a program right from source), type checking happens for each new form, taking into account what it knows of the types so far. In this context, some things, such as redefining vars, don't retroactively type check all previous usages of the var. So that sort of analysis is disabled.
During AOT compilation, though, jank does whole program analysis and uses the type information to box/unbox, monomorphize, compute at compile-time, etc. I'll be publishing documentation on each of these designs (the type system, the JIT, the interop support) on the website as things move forward. Stay tuned. :)
It might build, but can you actually execute something? According to https://github.com/jeaye/jank/issues/7, you won't be able to compile `(if true 1 0)` at the moment even.
I've been dreaming of something like this for a long time. While you can also make native executables with Clojure, using GraalVM, they tend to weigh at minimum around 10MB, and setting up the whole build system is also something I never want to deal with, so this is very awesome.
Yes, when the same functionality could be achieved in like 200KiB, 10 MiB is too much. It adds up and suddenly you may have to bundle a whole browser with your ap...oh
Bear in mind that this 10mb is not a fixed multiplication factor. Like, doubling you code size doesn't double your binary size with native-image because that base contains a lot of stuff that's generic. Not just GC but stuff like support for timezones.
It also contains a cached heap (in fact most of it is cached heap) which is one reason these binaries start as fast as C programs do. Especially important for Clojure where the runtime and libraries are written in such a way as to do lots of unnecessary work at startup. Native images can cache all that so you get rid of the startup overhead of Clojure itself.
That begs the question: if Clojure is so inefficient, that a program written in it needs 10 MB of some "cached heap" just to start as fast as a C program, and it does a lot of unnecessary work at startup, why use Clojure and this Jank (great name lol) in the first place? We have Rust, after all. I mean, creating an optimizing native compiler for any language is not an easy task, there must be some seriously powerful stuff in Clojure to warrant such an undertaking. Or just madness.
Rust and Clojure address completely different domains of problems.
Rust is competing with C++ for writing low level systems, whereas Clojure is designed for writing business applications.
Low level systems tend to have stable and well understood requirements, whereas the same cannot be said for business applications. Efficiency is not a primary concern of business applications, but flexibility is. The efficiency of Clojure is often "good enough" for business application, and there is no peer in flexibility. This explains the many fintech companies using Clojure, as well as many other B2B companies.
A dynamically typed Lisp has virtually nothing in common with Rust. A cached heap can benefit literally any language which is why it's an optimization found in some programs written in C or C++ as well.
Also, Rust binaries have been of comparable size in the past. Check out this question where someone asks why Hello World is 3mb even in release mode:
This 3mb doesn't have any cached heap, doesn't have any garbage collector, doesn't have support for high quality exceptions and doesn't have much in the way of a cross platform abstraction, whereas the Clojure-as-native binary has all of those and more.
There's a giant pile of things you can do here to try and reduce Rust's binary size:
Rust gets great press and HN is flooded with "let's do it in Rust" type comments. But I think sometimes we need a reality check. Rust is an extremely different language to most others, even C. The combination of the borrow checker+everything async is not obviously better than everything else, not even in the low level domains even though Rust is deservedly picking up loyal users there. The moment you care about development speed i.e. for the sort of apps Clojure was designed for, you're going to be better off with a GC and a strong runtime that abstracts the platform well.
It's like they have not seen how much engineering resources and talent have been put into V8 and JVM GCs, and it's only because of these state of the art GCs (and JITs) that Clojure is bearable and practical for day to day use.
I'm almost sure the author would drop the project as soon as he realizes and lives the frustration of not being able to come anywhere near close to the performance of Clojure in the JVM.
Love this. As a Lisper I’m always intrigued by Clojure but as someone with no knowLedge of or desire to learn to JVM I stayed away since I figured eventually I’d hit JVM questions/issues I frankly didn’t feel like dealing with.
Is it really 100 percent compatible with Clojure or is that just aspirational.
Also, is it the case that lots of libraries/workflows outside the core language in Clojure assume Java integration? I always thought a big Clojure selling point was the awesomeness of Lisp while having full access to the libraries of Java.
> Is it really 100 percent compatible with Clojure or is that just aspirational.
Aspirational. The project is really early still, and in the middle of a grand rewrite so basically it doesn't build at the moment, see https://github.com/jeaye/jank/issues/7
The official Clojure (JVM) project has an official sister project: ClojureScript, which is designed to be compiled to JavaScript (and maybe Webassembly in the future?). There is a lot of code that runs with both languages and therefore I don't think you should worry too much about using Jank with existing Clojure/ClojureScript libraries.
Clojure is a language without a platform so all the Clojure dialects have the different semantics of the underlying platform, be it the JVM or JavaScript.
For example ClojureScript has all the “wat” semantics from JavaScript primitive types.
You can build libraries that try to hide the differences but the leaky abstraction is inherently still there.
In my not so extensive Clojure experience, there's almost nothing you need to know about JVM to do real things with Clojure. If you're building something big/complex enough that you need to know something about the JVM, then you probably already discovered what you needed along the way.
Basic Java interop is pretty simple (and concise!). But unless you're specifically needing a Java library, you won't even do this much. You _can_, but it tends to be more for performance optimization in my experience.
In other words, don't be scared of JVM. I would say it's kind of like Elixir/Erlang. You can do real things with Elixir without knowing hardly anything about Erlang. Of course, later on you can do more awesome things if you know how and when to leverage the Erlang VM. Same probably goes for Clojure/JVM.
I'll be curious to see what the performance comparisons of typical Clojure programs will be. Persistent data structures tend to benefit a lot from highly optimised garbage collectors like those packaged with the JVM. And Clojure's highly dynamic style probably benefits a lot from JIT techniques too.
Still, it's a fascinating project, which I will watch closely.
This is a very very early project but mostly I think it'll be interesting to see how well they manage to utilize type annotations and how well the Clojure ecosystem provides this? (ie, is general Clojure code mostly untyped or is there enough for compilers to deduce most actual types?)
Why would this be important? Well because they are relying on LLVM for optimization, and while that's good if you know type info (via annotations or lower tiered profiling), there's a lot of code to start using this and those optimizations can be fragile.
If you read the articles posted here in the past year on the Erlang JIT they mirrored this sentiment, earlier "advanced" JIT attempts built on classic static language techniques like LLVM prefers failed because they didn't have enough type information, whilst the "simple" JIT that mainly removed overheads and concerned more with local types was showing good promise without too much engineering headache.
Very much inspired by Clojure, with a lot of time put into benchmarking and profiling.
It's too early for me to share numbers on jank vs Clojure itself, especially due to the rewrite, but my earlier jank versions were definitely competitive with AOT compiled Clojure jars.
> Clasp is a new Common Lisp implementation that seamlessly interoperates with C++ libraries and programs using LLVM for compilation to native code. This allows Clasp to take advantage of a vast array of preexisting libraries and programs, such as out of the scientific computing ecosystem. Embedding them in a Common Lisp environment allows you to make use of rapid prototyping, incremental development, and other capabilities that make it a powerful language.
Exciting idea, I love Clojure, but every time I tried Clojure, I found myself butting heads with the Java ecosystem, and I always wished it had its own ecosystem and interpreter/compiler. I will caveat that that's just because I've never spent long in the Java ecosystem, so it's not a criticism of the current Clojure + Java ecosystem, more my own lack of experience.
The big issue here is that (most) Clojure libraries that interact with the rest of the world do wrap some high-quality Java libraries (JDBC, web servers, etc) so Jank would start afresh here. CLJS has the same issue, of course, while Graal doesn't.
If they could pull off easy integration with the C and C++ world that would be an absolute killer feature.
They are basically trying to make my new dream language.
The only thing I am worried about is compile time. It seems many projects using LLVM have trouble with that. See Julia with its LLVM JIT, though they did manage to improve it somewhat. I guess it is simply to early to tell, lets hope for the best.
LLVM is not a particularly slow compiler (though it is progressively getting slower). There are faster compilers out there, but it’s fine.
The reason julia has so much compiler latency is not that we have a slow compiler, it’s that we compile a LOT of code.
Julia is very very (runtime) performance oriented, and so we do everything we can to speed up execution time. This means that we have very aggressive code specialization, inlining, and unrolling heuristics, and people take heavy advantage of them. It’s a very normal thing in julia to try and write very high level generic code that performs like low level optimized code by taking advantage of versions code generation techniques.
In addition to all of that, we end up recompiling a lot of code that was already compiled before because we have very strict heuristics to make sure that when a method body is altered, all downstream dependants of that method get invalidated and must be recompiled, and sometimes the heuristics overeager.
Julia compile times have been dropping rapidly over the past couple years mostly because people have put a lot of work into avoiding invalidations, and being smarter about how compile-time code generation is approached. Almost none of that improvement has to do with LLVM, it all happens well before we hand things off to LLVM.
Compile times will be slower than Clojure's, for sure. That also means run-time speed will be affected, due to JIT compilation time. This will not affect AOT compiled run-time speed, though. I don't have numbers yet, but you're right that compiling C++ with LLVM could be much faster. :)
How so? Last commit: June of this year. State: "very early (pre-alpha) stages, but it has progressed far enough to load clojure.test, allowing the test suite to actually be written in Clojure." and the author is still a very active lisper. So, pretty nice for an experiment, I guess.
It does look like it has some technical similarities to Julia, with the LLVM JIT and such. It'd be great to see some benchmarks.
Also see the GitHub[0], which looks like the author put a ton of work into this so far. Very impressive.
That said, looking at the progress page[1], this is still far from achieving 100% feature parity. I'll be following it closely with my fingers crossed though.
If you want to support this, it looks like the author is also accepting GitHub sponsorships[2].
The difference is Clojure's persistent data structures. You don't add an element to an array, you create a new copy of the array with the element attached, and the old array is accessible.
Internally this is done with structural sharing, so it's way faster than copy on write, but it's still potentially allocating a half dozen objects for what would usually be a trivial operation that does not involve allocation.
On the plus side, Clojure in theory has some pretty nice guarantees that older objects can't point to newer objects. From memory Haskell takes advantage of these guarantees to accelerate its version of eden. There may be a way to get a fast GC that's not nearly as complex as the full JVM GC.
> The only reason Clojure is viable for production use is because it relies on the really good JVM GCs. How will Jank try to solve the the GC problem?
I think this original statement is wrong because people are fine with poor performance (see Python, Ruby) if the language (ecosystem) is effective. And Clojure is effective (IMO) moreso because of the JVM ecosystem not the JVM's performance.
I think you're underestimating how slow a naive implementation would be. The only Clojure implementations that have gained traction are built on top of both great JITs and GCs.
"Jank" describes undesirable, problematic, glitchy UX. I genuinely thought that the title was denigrating Clojure on LLVM. Maybe it's intended to be funny or ironic, and if it catches on it won't matter, but it sure seems like a footgun.
Unless I'm reading it wrong, it sounds from the license[1] like programs made with Jank that you want to distribute must have their source available. If the intention is that distributions of the _Jank compiler itself_ are the only things that need source available, I think it'd be good to spell that out.
This applies to the compiler itself, in the same way that GCC is under the GPL but that doesn't mean all programs you compile with it are also under the GPL.
I'm curious if there is a reason you went with (what I believe is) a derivative of the Sleepycat license, instead of one of the more familiar licenses?
Your comment here is at odds with my understanding of how the Sleepycat license works and how Berkeley DB is licensed.
Interesting project! I have been wondering why there isn’t a LISP on top of LLVM yet (except Common Lisp port, Clasp). Good that someone started to work on this.
Jank looks like a project to check in on its status a few times a year, exciting possibilities.
For your thinking about Clasp and alternatives for Common Lisp: I don’t think there is a gap to be filled with Common Lisp deployment. For free, build SBCL from source enabling heap saves compression. Then creating SBCL Common Lisp apps produces a fairly small app size, easy to build and distribute. Or, pay for something like LispWorks and creating compact standalone applications is perhaps even easier.
