Idiomatic Clojure is also really slow compared to SBCL-compiled Common Lisp or one of the faster scheme implementations. The compiler's fairly basic and it relies on the JVM for optimisations, which isn't great at the kind of optimisations a compiler aware of higher level semantics can do. It is possible to achieve decent performance by using it as basically a Java wrapper, but that's much uglier than idiomatic code, compared to e.g. SBCL Common Lisp where idiomatic code can still be fairly performant.
Do you happen to know what the present state of SBCL optimization is? The old CMUCL manual had quite a lot of guidance on writing high performance code. I can't seem to find it though, mostly I remember the bits about what can and can't be open coded and how to give enough basic type hints to help the optimizer out (and using DISASSEMBLE to check that you really did). In fact I think you could even ask for warnings for probably inefficient constructs.
Obviously CMUCL was a mess in a lot of other ways though.
One great thing about sbcl is you can put (declare (optimize (speed 3))) at the top of a function and the compiler will give useful feedback when it can’t optimize something and suggest type hints to improve performance.
I’m not a optimization wizard, but I’ve been able to get some very impressive speed ups this way, in combination with some attention to how my code is written.
Tangentially, I think the future of optimization in CL and similar languages is run time optimization/compilation. For example, the goal should be to make (typical cases of) generic function dispatch as fast as inlined structure field access, even in the presence of the possibility of redefinition of the functions and classes. This is not easy, since these may get redefined while calls to functions using them are on the stack.
Since code is data, some kind of functional persistence might be the right path forward here. I lack the expertise to develop the idea further though. I don't much care about the exact behavior vis-a-vis functions on the stack, so long as it's well-defined.
I would add cryptic error java stacktraces to this list but, as someone who recently overcame the "java is gross" bias, I'd say benefits of JVM and its vast ecosystem outweigh the cost.
I've had a number of great experiences working on toy projects using Clojure but nevertheless feel resistance to adopt it fully "because of the JVM".
I've pinned my own resistance down to three things:
- Slow startup times. (Most of my use cases are not long running servers.)
- Not "unixy". (I write programs to run on linux and OSX exclusively and am used to using non-portable APIs maybe?)
- I've been lead to believe Java is "gross" and "enterprisy".
Really, of those three reasons only the first one has merit.
It kind of sounds silly when I put it this way: When I'm hacking on fun projects, I enjoy using a "hacker" language and Clojure doesn't feel like one.
One implementation you didn't mention that's pretty neat is Chicken Scheme [1].
[1] https://www.call-cc.org/