
Racket-on-Chez Status - ausimian
https://blog.racket-lang.org/2020/02/racket-on-chez-status.html
======
jnxx
What I wonder is whether the Chez base might massively improve Racket's
concurrency capabilities.

In Common Lisp, one has pthreads-like concurrency capabilities which seem very
well suited for fine-grained parallelism as well as for server tasks. I am
thinking in such things like parallel backtracking and optimization algorithms
for robotic control or board games, for example.

However, threads are notoriously difficult to handle cleanly and safely in
larger programs. Here, Clojure (which in my eyes is a very Scheme-like Lisp)
offers a very elegant solution with its concurrency primitives of Futures,
Atoms, Agents, and STM. After some experimenting, I believe they are very,
very attractive for many concurrent server tasks (like a web server), but not
that well suited for computing-intensive parallel algorithms like the ones I
mentioned above, which I am highly interested in. Part of the reasons are that
such algorithms can become quite GC-heavy. Also, the Clojure compiler has
limits on how much primitive types can be passed as parameters in one
functions. This means that a complex backtracking algorithm in Clojure can
still be two orders of magnitude slower than in C (which is somewhat
disappointing, but one has to remember that Clojure was not designed for
this).

Racket has had, so far, only limited concurrency capabilities. It had Futures,
however they could easily become blocked by GC. It also has Places, which are
a very safe and clean solution of splitting parallel computations into
separate processes. However, I think that places are not the first choice for
heavily parallel algorithms with strong interdependencies.

Now, Racket can run on top of Chez, and Chez has fine-grained concurrency
capabilities on top of pthreads, which seem to be on par with Common Lisp.
Also, Racket has strong support for functional and side-effect free
programming, including some data structures. In my impression, this seems to
open a wide range of new possibilities, including providing look-alike
primitives for Clojure's Futures, Agents, and Atoms. I would be very
interested to know more whether this impression is correct.

------
xvilka
Will it work on top of GNU Guile[1] as well? It became very fast since 3.0
version with the introduction of JIT[2].

[1] [https://www.gnu.org/software/guile/](https://www.gnu.org/software/guile/)

[2] [https://www.gnu.org/software/guile/news/gnu-
guile-300-releas...](https://www.gnu.org/software/guile/news/gnu-
guile-300-released.html)

~~~
gus_massa
It is difficult because there are a lot of minor details that has been added
to Chez Scheme or to the patched version of Chez Scheme that is used by
Racket.

For example ephemerons, that need some magic to cooperate with the garbage
collector.
[https://cisco.github.io/ChezScheme/csug9.5/smgmt.html#./smgm...](https://cisco.github.io/ChezScheme/csug9.5/smgmt.html#./smgmt:s26)

------
danieldk
Question from someone who knows barely anything about the Scheme ecosystem:

It seems that the one of the motivations for starting this effort, besides
performance, was to move away from a C code base. Is Chez Scheme primarily
written in Scheme?

~~~
zelphirkalt
I think the reason is more maintainability, when basing on a solid and faster
other Scheme. It's much easier to write in terms of that other Scheme's
primitives, than writing a C core, avoiding all kinds of C typical bugs.
Furthermore, many improvements in Chez Scheme will carry over to Racket and
the 2 communities might join forces in improving Chez and thus Racket in
effect.

What that other Scheme is based on is a secondary consideration. Chez Scheme
is also said to be implemented very well, however.

~~~
bjoli
Iirc Idris2 went straight to running on chez since Edwin was so impressed by
chez scheme and it's runtime.

~~~
logicchains
The result is super fast: if you use if after using Idris1 you'll be blown
away by how much quicker it type-checks. I'd never have thought a Scheme could
perform noticeably faster than Haskell, but maybe most of the improvements are
just due to algorithmic improvements made during the rewrite.

~~~
bjoli
I never played with Idris2, but I recall running some idris1 programs using
the chez scheme backend instead of thebc one and got instant 5x speedups.
Across the board.

Not that it wouldn't be possible to make the C code fast. It is just that the
chez backend was a one man job.

~~~
kryptiskt
Thank the Chez Scheme developers for an absolutely fantastic compiler, and
also Idris for being designed so that it's possible to feed any of the IRs to
the particular backend. There's not much of an impedance mismatch between
Scheme and Idris lambda output, so the overhead is small. I could have made it
a bit better if I had represented Idris lists and bools as the corresponding
scheme types, instead of being mapped as all the other types by their
structure.

------
kamaal
Is there an update on what Racket decided to do with the proposal to deprecate
the existing language syntax?

Racket was a great tool to teach people lisp and initiate them into that
paradigm.

Is the main syntax still good for long term use?

~~~
kryptiskt
They changed the name of the project from Racket 2 to Rhombus to make it clear
that it was not about replacing Racket.

And even in the original announcment it was stated " * `#lang racket` is not
going away and will always have its current parenthesis-oriented syntax. In
the same way that Racket still supports `#lang scheme` and `#lang mzscheme`
and even `(module <name> mzscheme ....)` and even top-level programs, the
Racket compiler and runtime system will always support `#lang racket`
programs. We believe that Racket's `#lang`-based ecosystem makes it uniquely
positioned for trying new language variants while preserving and building on
our past investments."

They have only made that statement stronger since then.

~~~
kamaal
So will the current Racket language continue to be developed, as in new
features, bug fixes and regular releases _after_ Rhombus is released?

Renaming doesn't help the case much here, if Racket's future is abandonware.

~~~
kamaal
A bit of Googling landed me here:
[https://groups.google.com/forum/#!msg/racket-
users/-x_M5wIht...](https://groups.google.com/forum/#!msg/racket-
users/-x_M5wIhtWk/V47eL30HCgAJ)

From there:

Phase 1: Brainstorming (months)

Phase 2: Iterative Design (years)

Phase 3: Conversion (months or years)

Phase 4: Transition (years)

I am a nobody in front of people like Matthew Flatt, but this feels like
Seconds Systems Effect taken to its extreme definition:
[https://en.wikipedia.org/wiki/Second-
system_effect](https://en.wikipedia.org/wiki/Second-system_effect)

Plus as a former Perl programmer having watched Perl 5 lost almost everything,
chasing a never to have come to reality Perl 6. I can say, given all this, the
future of Racket as a language, for its core uses and users is pretty much
_dead_ in the years to come. Like dead totally. Racket doesn't even have as
much the share of dev mind share or a resource like CPAN at its disposal.
Languages like Racket will fade away into oblivion a lot more quickly.

It can be hard to see this in all the enthusiasm in the early stages of the
project. But when you embark on multi year project journeys. People's
priorities change, people get into health crises, lose jobs, move on to better
projects, recessions happen. Core teams that started these projects change so
much, newer one's pretty much give up and see no point in it after a while.
Many people prefer a perfectly working tool improved over time, than a pie-in-
the-sky idea that will never come to see the light of the day. While all this
is happening, your existing language, libraries, dev mindshare, tooling
suffers. Merely supporting small time fixes means nothing, because no one
likes to use a tooling in hospice care. Your core dedicated set of users, who
did most of the evangelism for your cause move on to newer tools and
languages, for the obvious reasons that they don't see their future with
existing tool they like, the newer one is taking for ever and isn't even the
same goodness as the current one. Once you lose those users, the newer users
you wish to attract from the crowd of Java and Python programmers won't even
bother to try, why should they? They have something proven to work in
production with tooling, libraries, books, and production success stories for
decades.

That is when you realize you lost both your old and new language.

Racket was awesome for what it was. An awesome lisp, a top successor for
Common Lisp and a playground for experiments and initiating people into Lisp.

It's not late. Probably it's time to stop right now.

~~~
kryptiskt
Racket (and PLT Scheme before it) has always been a bunch of languages,
keeping a few different syntaxes and semantics around isn't anything new. I
don't get why having another take on the surface syntax should cause all that
much anxiety.

~~~
kamaal
Because you are not exactly running Rhombus as a side toy language, but as a
successor to Racket. This requires Rhombus to eventually replace Racket.
Anything less than this and it makes Rhombus a non-successor to Racket. While
at the same time much needed resources to make Racket win, will go to making
Rhombus happen. Even if Rhombus comes along, it won't have any of the current
Racket goodness to begin with. Rhombus will take years to just exist.
Meanwhile precious resources for improving Racket to a better Racket over the
years will be taken by Rhombus.

Net result is a stagnated Racket, and a non-attractive Rhombus.

Plus C based languages have their winners already. Racket is a kind of a
winner in the Lisp family. With Clojure it makes the only other choice. Now if
you change the syntax, you are going after users who don't really care what
you have to offer, at the same time, you are taking away what your existing
users like.

~~~
dTal
Racket is not like other languages.

Racket is a language with _first class support_ for building languages on
_top_ of it. And so no, Rhombus does not need to be a "successor" to Racket.
It does not need to "replace" Racket. It _will_ have all the current Racket
goodness to begin with, because _all_ of its semantics will come from Racket -
they will be the same language. Nor will it steal valuable developer attention
- a bug fixed in Rhombus is a bug fixed in Racket, anywhere except the syntax
parser. They will share all core libraries. The whole point of Racket is that
#langs are mutually compatible. And the stress test of using Racket in earnest
for what it is actually _designed_ for will make Racket better, not worse.

Consider PyonR [0]. This is a similar and yet _much_ more ambitious goal than
Rhombus - Python as a Racket #lang, with 100% compatibility with Racket _and_
Python including external modules. And yet, it's a one-man project, written as
a thesis and now pretty much abandoned. Writing Rhombus, a mere surface syntax
for Racket, should be much easier. The hard part isn't writing it, it's
deciding what it should be.

The Racket team are the only ones that I can see who are taking all the "write
your own DSL" Lisp hype and actually trying to apply it in a structured way.

[0]
[https://github.com/pedropramos/PyonR](https://github.com/pedropramos/PyonR)

~~~
kamaal
Well PyonR is not a replacement for Racket. It's something totally a different
project altogether. Rhombus is supposed to replace Racket. We could have
PerlRacket or AwkRacket or RubyRacket and none of it should effect Racket,
because you are using Racket as a language to build things, not exactly using
RubyRacket as a replacement for Racket.

This is how it rings to my ear. Racket as is, is in some way frozen when it
comes to features and big changes. Rhombus is where the next set of
awesomeness will happen. Lispers who happen to be core users of Racket have no
use at the time for non-existent Rhombus, and its not Lispy so their whole use
case ends here. Racket won't be continued to developed in the same breath as
Rhombus, so Racket is also dead here. C based language devs have a lot of good
stuff already, so they don't need Rhombus either.

Eventually this could lead to the exact situation Perl was in the 2000s. The
core development was frozen for 5+ years. The language usability just fell
year over year. Perl 6 took forever to even start with the implementations.
Pugs, then Neitza, then Rakudo. Eventually they came to a point, where they
announced Perl 5 development would just run parallel as a separate project
compared to Perl 6. By then a lot of dev mind share was lost to Python/Java.
Now neither Perl 5 nor Perl 6 have significant mind share. Perl 5 certainly
has lost a lot of its users.

Also in general trying to be everything to everybody is a bad idea in general.
Ask Parrot VM people. It doesn't work the way you expect. You won't solve
anyone's specific problem, and it takes forever to get done.

~~~
wtetzner
> This is how it rings to my ear. Racket as is, is in some way frozen when it
> comes to features and big changes. Rhombus is where the next set of
> awesomeness will happen.

You still seem to be misunderstanding how Racket and #lang's work. Rhombus and
Racket will be the _same_ language, but with different parsers. Big features
of Rhombus will be added by adding those features to Racket.

Languages in Racket are not developed independently of each other, they all
share the underlying language machinery.

~~~
kamaal
>>Rhombus and Racket will be the same language

Exactly the point, one will have to bring Rhombus to Racket's feature parity,
which by the link I posted, and the Racket people themselves acknowledge will
take several years to happen. That much dev resources have to be dedicated to
build a new language which will have the same feature parity as Racket. This
effort could be expended to taking Racket forward. This is non trivial effort.

At the same time, you are hoping to attract users from Java/Python land(Who
very likely won't come, given where your language will be(absence of killer
libs, frameworks, books, q&a support etc)). While turning away existing lisp
users. This ultimately becomes a disaster.

One of the biggest selling points of Racket was that it was continuously being
worked on, unlike CL whose spec is frozen(putting it mildly, the real word
would be _abandoned_ ).

Looks like Lisp has a lot of self destructive tendencies than anything else.
Competition seems to be non existent. But Lisp communities just can't agree to
work on things. A super massive pivot that just won't fix any real problems to
existing users, adding very few incentives to new users, plus act as a
resource drain, and cause your perfectly fine existing language to stagnate.

[https://news.ycombinator.com/item?id=20503742](https://news.ycombinator.com/item?id=20503742)

A similar argument is made in the above thread.

Regardless, of all this serious software also commits itself to things like
stability and takes backwards compatibility seriously.

~~~
gus_massa
It will take years (if it is even completed) but it doesn't mean that Racket
will not be advancing meanwhile. Most of the discussion is in angry[1]
comments in GitHub written while drinking coffee in the morning. In these 6
months, I guess the lost development time was only 1 or 2 weeks distributed in
small chunks. If I can extrapolate linearly, the delay will be only 6 months
(but some parts may take more time, for example rewriting the documentations).

The idea is that the libraries written in Racket will be usable in Rhombus and
that the libraries in Rhombus will be usable in Racket. Racket already ships
with multiple internal languages, so one more is not a big difference. [2]

Note that some constructions that are special cases that must be implemented
by the language are just libraries in Racket. For example most of the
implementation of `for` and `match` are in their own library, and even more
weird things like at-expressions is just a library [https://docs.racket-
lang.org/scribble/reader-internals.html](https://docs.racket-
lang.org/scribble/reader-internals.html) Some of them may need some tweaking
to make them more idiomatic with the new syntax, but most of the
implementation will be useful as is.

[1] Most comments are nice, but let's add some drama to make the story more
interesting.

[2] Some languages have more impedance mismatch and need some macro magic to
make sharing easy. In some languages sharing libraries is straightforward.

------
orsenthil
Why is it called "Chez" scheme?

~~~
uselpa
Because it’s now based on « Chez Scheme »
([https://scheme.com](https://scheme.com)).

~~~
orsenthil
I meant, why was "that" scheme called "Chez" scheme?

~~~
pjmlp
All about it on "The development of Chez Scheme" paper, written on its
twentieth anniversary.

[https://legacy.cs.indiana.edu/~dyb/pubs/hocs.pdf](https://legacy.cs.indiana.edu/~dyb/pubs/hocs.pdf)

~~~
catalogia
I don't see the name explained in that document. It does mention ' _C_
-Scheme' and ' _Z_ 80 Scheme' so maybe _C_ he _z_ is a sort of interpolation
of those?

~~~
onemoresoop
In French chez means "at the home of (used in imitation of French, often
humorously)." This always rings this bell in my mind when I hear this word.

An example: Je suis allé chez Steele et Sussman roughly translates to: I went
to the Steele's and the Sussman's homes

