
Crystal in Production: Diploid - sdogruyol
https://crystal-lang.org/2017/10/27/diploid-and-crystal.html
======
sdogruyol
Crystal is a Ruby inspired compiled language, allowing it to run blazingly
fast with a very low memory footprint. It uses LLVM for emitting native code,
thus making use of all the optimisations built into the toolchain.

I've been using Crystal for more than 2 years have some projects in
production. Can't wait for 1.0 :)

~~~
cel1ne
Everytime I read a HN comment claiming a language to be „blazingly“ fast, i
wish they posted a link to statistically sound benchmarks, including vm-
warmup, GC collection times, etc.

Otherwise I just throw these adjectives away. I argue a new
compiler/interpreter will always lose against the JVM, which has thousands of
man hours of optimization built-in.

The JVM in turn will always lose against a clever memory-conscious lowlevel-
implementation in Rust or C or assembler.

Please don‘t advertise speed without any studies or comparison to back up that
claim.

~~~
pjmlp
Everytime I read such comment in HN, I smile and remember the days C programms
on 8 bit micros and later MS-DOS, were made of 80% inline Assembly statements,
because the compilers were quite lousy.

Fran Allen is the opinion that the adoption of C delayed the field of compiler
optimizations research back to pre-history (Coders at Work).

It took 40 years of optimization research and clever use of UB defined in the
standard, for C compilers to achieve the code quality generation they have
nowadys.

~~~
groovy2shoes
> Fran Allen is the opinion that the adoption of C delayed the field of
> compiler optimizations research back to pre-history (Coders at Work).

Yes! The entire book is wonderful, but as a compiler writer myself, Fran's
interview really stuck with me.

The relevant passage, for the curious:

———

Seibel: _When do you think was the last time that you programmed?_

Allen: _Oh, it was quite a while ago. I kind of stopped when C came out. That
was a big blow. We were making so much good progress on optimizations and
transformations. We were getting rid of just one nice problem after another.
When C came out, at one of the SIGPLAN compiler conferences, there was a
debate between Steve Johnson from Bell Labs, who was supporting C, and one of
our people, Bill Harrison, who was working on a project that I had at that
time supporting automatic optimization._

 _The nubbin of the debate was Steve 's defense of not having to build
optimizers anymore because the programmer would take care of it. That it was
really a programmer's issue. The motivation for the design of C was three
problems they couldn't solve in the high-level languages: One of them was
interrupt handling. Another was scheduling resources, taking over the machine
and scheduling a process that was in the queue. And a third one was allocating
memory. And you couldn't do that from a high-level language. So that was the
excuse for C._

Seibel: _Do you think C is a reasonable language if they had restricted its
use to operating-system kernels?_

Allen: _Oh, yeah. That would have been fine. And, in fact, you need to have
something like that, something where experts can really fine-tune without big
bottlenecks because those are key problems to solve._

 _By 1960, we had a long list of amazing languages: Lisp, APL, Fortran, COBOL,
Algol 60. These are higher-level than C. We have seriously regressed, since C
developed. C has destroyed our ability to advance the state of the art in
automatic optimization, automatic parallelization, automatic mapping of a
high-level language to the machine. This is one of the reasons compilers are
…_ [sic] _basically not taught much anymore in the colleges and universities._

Seibel: _Surely there are still courses on building a compiler?_

Allen: _Not in lots of schools. It 's shocking. There are still conferences
going on, and people doing good algorithms, good work, but the payoff for that
is, in my opinion, quite minimal. Because languages like C totally overspecify
the solution of problems. Those kinds of languages are what is destroying
computer science as a study._

———

(pp. 501-502)

I recommend that any programmer who hasn't read this book give it a read. In
fact, I think I might give it another read this week :)

------
interfixus
I have dedicated this very weekend to digging into Crystal. Uncanny
coincidence seeing it popping up here, today of all days.

It is a tremendously nice looking language - and I say this as a pythonesque
guy who never wrote one line of Ruby. The feeling I get from community and
projects is that it's a very up-and-coming thing, about to take off in a major
way. There is just too much enthusiasm and too many things done right for
Crystal not to earn some solid share within a very foreseeable future.

~~~
look_lookatme
Crystal really does feel like Ruby without the annoying parts, plus the macro
system feels like a _better_ meta programming facility.

On the other hand I've read reports that sufficiently large codebases begin to
hit machine memory limits for the compiler because they have to scan and
construct every possible union type in the codebase. Not sure what there is to
do about that other than bisecting the codebase into shared libraries over
time.

~~~
RX14
The actual memory usage is coming from the number of instantiated methods. If
you write a method, it gets an entirely new implementation of it copied and
compiled for every different list of argument types. For example,

    
    
        def foo(bar)
          bar + bar
        end
        
        bar1 : Int32
        foo(bar1)
        
        bar2 : String
        foo(bar1)
        
        bar2 : Int32 | String
        foo(bar1)
    

The above code gives you three methods to generate code for: foo(Int32),
foo(String), foo(Int32 | String).

This isn't really a problem for small projects, but with large projects (60k+
loc), you end up with some methods such as puts, Array(T)#<<, etc with
thousands of instantiations.

We have some edge-case semantics (we don't cast arguments to their restriction
types, which we plan to remove before 1.0) which also prevent us from allowing
only instantiating foo(Int32 | String) when we change the definition to `def
foo(bar : Int32 | String)`. Hopefully in the future, we can allow the compiler
to take advantage of method argument type restrictions to reduce the number of
instantiated methods and speed up the compiler.

That doesn't mean the compiler will be super slow if you don't annotate your
methods, it just means that the stdlib and shards' public API will start
annotating it's method arguments more (which gives you better docs and error
messages _anyway_ ).

For more info: [https://github.com/crystal-
lang/crystal/issues/4864](https://github.com/crystal-lang/crystal/issues/4864)

~~~
nerdponx
They also have a very detailed blog post about how their compiler works:
[https://crystal-lang.org/2015/03/04/internals.html](https://crystal-
lang.org/2015/03/04/internals.html)

------
curtis
Crystal reminded me of Mirah [1], a Ruby-like programming language which is
statically typed but relies heavily on type inference in order to compile
efficiently to JVM bytecode, while retaining the appearance of being a
dynamically typed language.

(I've got no real point here other than to say I had a strong feeling of deja
vu while reading about Crystal.)

[1]
[https://en.wikipedia.org/wiki/Mirah_(programming_language)](https://en.wikipedia.org/wiki/Mirah_\(programming_language\))

~~~
nerdponx
Why are people so interested in running things on the JVM? I'd much rather
compile and ship binaries than force someone to install Java (or Ruby or
Python).

~~~
grzm
For one, the JVM is available on multiple platforms. Rather than requiring a
project to target each of these platforms individually, they can target a
single platform—the JVM.

There are plenty of people who dislike the JVM for any number of reasons
(performance often being cited as one) and prefer to target specific platforms
individually. Like many other things, it's a tradeoff, and people may arrive
at different decisions depending on their priorities.

~~~
ricardobeat
Modern languages like Crystal target LLVM IR, which if, I'm not mistaken,
supports even more platforms than the JVM.

~~~
jerrre
But it would need to be recompiled right?

~~~
michaelmior
Correct. But due to the separation of the front-end and back-end, the language
developer doesn't have to do extra work to support different architectures.

------
piyushpr134
Crystal is one of the most exciting new languages out there. I have been using
it for my one off tasks at Ola and it works pretty well. It is huge plus point
that you have safety harness of statically typed languages and speed
comparable to java and scala(unscientific benchmark here
[https://github.com/kostya/benchmarks](https://github.com/kostya/benchmarks))

------
marcrosoft
> Features that we take for granted in Ruby or other languages, are not
> available in Go. Examples include operator overloading or keyword
> extensibility, as well as true OOP.

These are literally the top reasons why I love Go. To each their own?

------
WJW
Crystal is a super cool project and doing websockets with Kemal (Crystals
Sinatra equivalent) was one of the easiest ways of hosting websockets I've
encountered so far.

That said, I'm eagerly waiting for true paralellism support since the use case
we have in mind would greatly benefit from that. Some of the testing tools are
also not quite as polished as Rspec (yet).

~~~
sdogruyol
Kemal: Fast, Effective, Simple web framework for Crystal Website:
[http://kemalcr.com/](http://kemalcr.com/)

P.S: I'm the author of Kemal :)

~~~
WJW
So cool! I really enjoy working with Kemal. :)

The State of Crystal at v0.21 article ([https://crystal-
lang.org/2017/02/24/state-of-crystal-at-0.21...](https://crystal-
lang.org/2017/02/24/state-of-crystal-at-0.21.html)) stated that multithreading
with work stealing was coming "soon" and that you already managed to run kemal
in parallel. Can you share anything about the current state of that project?

~~~
sdogruyol
Even though it's experimental I've successfully compiled Kemal with multi-
thread support. The throughput was OK. It's promising and will definitely help
CPU bound apps :)

You can check the wiki for more info [https://github.com/crystal-
lang/crystal/wiki/Threads-support](https://github.com/crystal-
lang/crystal/wiki/Threads-support)

------
cyberferret
I've been using Crystal for only a few months, but impressed with its speed
and low memory footprint. Starting to build some small services with it now,
but using Kemal or the Amber framework, I can see some medium sized projects
coming out of it soon.

Was a cinch to swap over to it from Ruby. It is a little fussier with type
definitions, but I guess that is to be understandable with a compiled system.

The third party ecosystem is still a little thin on the ground, or immature,
and I hope it will grow. The Crystal community has also been really friendly
and responsive. The couple of questions I have asked on Reddit or SO have been
answered quickly and with lots of useful info.

------
cyberferret
Here is another (sort of) real world app I wrote using Crystal and Kemal - a
real time race telemetry display app for an F1 racing game [Blog post link] -
[http://devan.blaze.com.au/blog/2017/10/28/racing-along-
build...](http://devan.blaze.com.au/blog/2017/10/28/racing-along-building-a-
telemetry-system-using-crystal-rethinkdb)

------
humanrebar
> With Crystal, data scientists could have the ease-of-use of Python/Ruby
> combined with the performance of C.

Big if true.

Are there any benchmarks to back up this claim? There is some mention of
experiments, but I'm not seeing any numbers or code.

~~~
mverzilli2
[https://manas.tech/blog/2015/10/30/linear-algebra-in-
crystal...](https://manas.tech/blog/2015/10/30/linear-algebra-in-crystal-from-
lapack.html)

------
maxpert
Still waiting for v1; Ruby being my fav in terms of syntax, Crystal is really
going to lift my experience in terms of syntax. This interview describes low
traffic usage, what I am looking for is high traffic usage scenario and how
crystal’s GC behaves running for longer periods of time.

~~~
faaq
Check latest benchmark (not the best but shows good performance)
[https://www.techempower.com/benchmarks/previews/round15](https://www.techempower.com/benchmarks/previews/round15)

~~~
yxhuvud
These benchmarks show raw throughput though, and not latency distribution. A
well functioning GC should not be too bad in either, and the current solution
in the GC area is very much stop the world, so I expect the latency tail to be
pretty bad even if raw throughput is damned good.

------
blunte
"...OOP. Moving from Ruby to Go sometimes feels like ignoring 20 years of
progress made in language design."

If OOP is so important to them, why did they bother including Elixir in their
list of possibilities?

Also, given their problem domain, I'm doubtful of the fit of OOP (vs
functional). But regarding Crystal, it is nice to see a potential performant
Ruby replacement.

~~~
coldtea
> _Also, given their problem domain, I 'm doubtful of the fit of OOP (vs
> functional). But regarding Crystal, it is nice to see a potential performant
> Ruby replacement._

Part of their problem domain was to easily port their code from Ruby, which
would presumably be OOP.

------
mooreds
As mentioned in the article, library support in Crystal is growing but still
far from complete. This GitHub repo might be a good place to start searching
for useful shards: [https://github.com/veelenga/awesome-
crystal/blob/master/READ...](https://github.com/veelenga/awesome-
crystal/blob/master/README.md)

~~~
fridgamarator
also [http://crystalshards.xyz/](http://crystalshards.xyz/)

------
faitswulff
Is there an easy way to get interop working between crystal and ruby? It would
be nice to be able to simply require a crystal module that calls out to a
compiled binary in the middle of my Ruby code.

~~~
faaq
See [https://es.slideshare.net/AnnaKazakova/how-to-write-ruby-
ext...](https://es.slideshare.net/AnnaKazakova/how-to-write-ruby-extensions-
with-crystal)

~~~
faitswulff
This is the full talk, I assume:
[https://archive.fosdem.org/2016/schedule/event/ruby_extensio...](https://archive.fosdem.org/2016/schedule/event/ruby_extensions_crystal/)

------
vram22
Crystal looks interesting based on the comments in this thread by those using
it. Anyone know why it seems to not be available for Windows (without WSL)?
Does it have some POSIX dependencies?

~~~
dingdingdang
Yep, proper Windows support is a must for easy to use dev setup - Go is OK'ish
in this sense but Crystal is currently really difficult to get running (either
vm or Win10 Linux sub-system or Cygwin hack needed!)

~~~
vram22
Thanks. Still wondering what the reason is though - but based on what you
said, it seems like POSIX-like APIs being needed may be the reason. Maybe for
the concurrency features or other language / library features of Crystal. But
then again, Go has concurrency features but works on Windows, and of course
Windows has threads anyway, which are usable from C/C++ and other languages.

~~~
faaq
Here is an installer waiting for Crystal support on Windows
[https://github.com/faustinoaq/crystal-windows-
installer](https://github.com/faustinoaq/crystal-windows-installer) :)

~~~
vram22
Thanks.

------
singularity2001
I love crystal yet it puzzles me how the heroic developers can make rapid
progress when each compilation of the compiler takes half a minute.

------
tambourine_man
How is the compile time on larger projects?

~~~
faaq
See [https://groups.google.com/forum/#!topic/crystal-
lang/xG5MQXT...](https://groups.google.com/forum/#!topic/crystal-
lang/xG5MQXTOyj8) and [https://github.com/crystal-
lang/crystal/issues/4864](https://github.com/crystal-lang/crystal/issues/4864)
Crystal is improving everyday :)

------
faaq
Crystal is very good language, seems very promising

Waiting for parallelism & Windows support +1

------
todd3834
I’ve always wondered why Crystal doesn’t get more attention.

~~~
devmunchies
Because it's not production ready yet. It's getting there.

------
quickthrower2
That's a nice looking language. The inferred type system on an procedual / OO
language with protection against nulls is something I had wished existed. I
didn't know it had been done.

~~~
thesmallestcat
Heard of Swift?

~~~
quickthrower2
I've heard of it but not investigated. So many programming languages. Is it
general or tied to Apple?

~~~
faaq
general

------
hetman
I'd be curious to know what the compile times are like? Does ever compile run
recompile the entire code base or is there some kind of incremental
compilation?

------
dsign
Is there a place with the distribution of the number of developers per
programming language? I have the feeling it has a very long tail.

------
lobster_johnson
I'm surprised Swift was outperformed by Ruby in their testing. I wonder what
their benchmarking methodology was.

------
gtt
On a related note, how well Crystal is suited for number-crunching?

------
cutler
Crystal is currently no faster than Ruby when parsing a 19Mb Apache log file
on OS X Sierra:

    
    
      fh = File.open("logs1.txt")
      fh.each_line {|x| puts x if (/\b\w{15}\b/.match x) }
      fh.close

~~~
theodorton
I guess it depends, for one-off tasks you'll have to compile and run the
Crystal code. I assume you ran the code above with `crystal run`. If you first
compile the code with `crystal build --release` and run it, you should see a
dramatic improvement over Ruby.

On my machine, parsing a 374MB file took:

    
    
        - 25.63 using Ruby
        - 19.97 using Crystal (run w/o optimizations)
        - 19.01 using Crystal (compiled w/o optimizations)
        - 18.68 using Crystal (run w/ `--release` flag)
        - 11.54 using Crystal (compiled w/ `--release` flag)

~~~
cutler
No, it was compiled but without --release as I couldn't get it working on OS X
Sierra.

------
eXpl0it3r
Having no idea what Crystal is, my mind replaced it with crystal meth and it
made the introduction text a lot more entertaining.

------
m3kw9
I thought this is a new type of drug

