
Dylan: the harsh realities of the market - kryptiskt
http://logicaltypes.blogspot.com/2014/08/dylan-harsh-realities-of-market.html
======
BruceM
I'm the Bruce that he mentioned in the post.

For better or worse, I've been pushing Dylan forward heavily over the last few
years and am effectively the primary maintainer.

Over the last couple of years, we've made a lot of progress. We've completely
revived the documentation from 1990s era FrameMaker files and have it
published via a pretty modern system. We've converted from SVN to Git and
moved to GitHub. We've done 4 actual releases. We've improved our platform
portability. We've provided some basic debugging integration with LLDB. We've
fixed some long standing issues in the compiler and tool chain. We've improved
the GC integration on all platforms.

But there's a lot to do. We need to fix our Unicode support. We need to
improve the type system in at least minor ways if not major ways. We need to
improve how parse failures are handled as the errors are not always friendly.
We need more libraries. Some of this is really easy, some isn't. But for
pretty much everything, there are bite-sized pieces of work that could be done
in a couple of hours/week that would lead to significant gains.

I've wanted to just flat out use Dylan for something and have built some small
prototypes with it and while they've worked out well enough, the actual
projects themselves didn't go anywhere (unrelated to the use of Dylan).

I think this blog post was triggered by a comment that I'd made publicly
yesterday that I'm feeling rather discouraged at this point. There was also a
private email that I sent to 19 people who have been involved with Dylan
recently, but the author of this post didn't get that email.

I view Dylan, not as a language from the past, but as a stepping ladder
towards building a better language for the future. We don't have to get bogged
down in a lot of the minutiae involved in creating a new language as a lot of
the work has been done. We get to focus on things at a different level and
those things are just as important. People bring up Goo often when Dylan comes
up. Goo is interesting, but the implementation is nothing close to being
industrial enough to survive an encounter with the real world.

I came to Dylan because I saw the mess that Scala and other languages were. I
didn't like where they were going and following some people on Twitter like
[https://twitter.com/milessabin](https://twitter.com/milessabin) and others
seems to show that I'm not alone.

And that's why I'll probably keep at it with Dylan. I want a better future and
I'm going to keep trying to build it.

~~~
justinpombrio
It sounds like you care not only about Dylan but also about advancing
programming language construction in general. Have you considered writing,
e.g., blog posts about design decisions you make as you work on Dylan? Maybe
it would help other people learn from your work.

~~~
BruceM
I do ... and I post on [http://dylanfoundry.org/](http://dylanfoundry.org/).

I don't usually bother to post them on HN as I don't have the time to try to
get something on the front page (otherwise, no attention). I do post them on
r/lisp though or lobste.rs usually.

I've got a couple of posts in draft stage now that I hope to publish this week
or next.

------
georgegeorges
If you're going to rant, you might want to actually have a clear point to
make.

> college kids on comp.lang.lisp asking for the answers for problem-set 3 on
> last night's homework

Surely not during the Naggum days. CLL was a hostile wasteland.

> That is the lesson of perl and python and all these other languages. They're
> not good for anything. They suck. And they suck in libraries and syntax and
> semantics and weirdness-factor and everything.

What? How have you not heard of CPAN? There is not a single language in the
world that can touch Perl's libraries. I'm not sure why you feel the need to
toss either Perl or Python under the bus to make some petty point about
Dylan's lack of popularity. Python replaced Scheme at MIT. It's time to move
on. I know I have.

You have to have your head pretty far up your own ass to not see how much
Common Lisp _sucks_. It's a language designed by committee, and it _looks like
it_.

I've used Erlang too. For everything Erlang does well, there are countless
areas that make you want to bang your head against the desk.

Languages don't matter. Platforms matter. APIs matter. Playing nicely with the
rest of the world _fucking matters_. Common Lisp wouldn't.

~~~
chrismonsanto
> What? How have you not heard of CPAN? There is not a single language in the
> world that can touch Perl's libraries.

That's a really outdated meme. CPAN is small, pretty much every language you
hear about regularly has a larger package space than CPAN. Even Go's package
space is bigger than CPAN's. JS and Java each have package spaces roughly 3x
CPAN's size.

[http://www.modulecounts.com/](http://www.modulecounts.com/)

~~~
Diederich
[https://metacpan.org/recent](https://metacpan.org/recent)

There are a couple of CPAN module installers, and every one of them, by
default, will not install the target module if there are any test failures.

Honest question: are there any other languages that do that? I last looked a
few years ago, and it didn't seem to be the case.

~~~
lmm
Maven will not, by default, allow you to perform a release if there are any
test failures. That seems like a better model, at least for a VM language - if
something works on the release machine and not on the user's machine, you have
bigger problems.

~~~
Diederich
That's great to hear!

I'm not too strong on Java; does what you're saying imply that some/all/most
of the freely available Java 'modules' or 'packages' that are built with Maven
will end up running and passing associated test suites in most all of the
organizations that end up using the code in question?

~~~
wtetzner
No, the tests are run as part of the deploy. So before deploying a library, if
the tests don't pass, the deploy fails.

However, the artifact that actually gets deployed is a .jar file containing
.class files. Users of libraries don't rebuild the libraries.

~~~
lmm
Just to clarify, "deploy" in this sense means "upload to your organization's
maven repository (or maven central)".

------
eudox
The article hits rather close to home, as someone who, for better or worse, is
committed to improving Common Lisp's ecosystem. I often feel this existential
dread of "Is it worth it? Whose lives will it change? Will I spend years
labouring in obscurity for nothing?". And while the answer to all those
questions is probably not what I'd like to hear, I still do it.

This is why ecosystems are, for most people, more valuable than intrinsic
language features: Tribalism along the lines of "We have X thousand people
backing us up"/"X thousand devs can't go wrong". People don't care about
monads or macros, they care about feeling like they're part of a large
community.

~~~
seanmcdirmid
Its even worse for the PL designer inventing something new without even a
small ecosystem in place; I ask myself everyday "is it worth it?"

~~~
tetha
The important question isn't necessarily: Is it worth it, without the
ecosystem? The more interesting questions is: What does this language add?

For example, I need to deal with some unix process management tasks. I'm not
crazy enough to do them in straight C, so I'm using python for it's strong C
bindings, so I can use system calls in C with python level control flow.
However, there's a lot to be desired, because dealing with all this
concurrency and inherent raciness of the system in a language like python is a
royal pain in the rearside.

Similar things with java and performant programming. Some of my major programs
at work don't use the java ecosystem outside of log4j, junit, apache commons
lang and apache commons collections. They use java, because java is darned
fast without being as brittle as C or as nuts as C++. So java adds a lot of
really cheap development security.

Designing yet another language that puts the existing features together with
just a tiny difference doesn't add anything. I've enjoyed looking at
experimental languages, and I've enjoyed tinkering around with ideas myself
some time ago. But by now, I've accepted I'm not smart or talented enough to
make something worthwile, and it will be some time until an interesting new
breakthrough occurs.

~~~
seanmcdirmid
My languages tend to be very innovative or even inventive; e.g. I recently
posted this to HN: [http://research.microsoft.com/en-
us/people/smcdirm/managedti...](http://research.microsoft.com/en-
us/people/smcdirm/managedtime.aspx)

My problem is quite the opposite: when the languages are so different from
what already exists, people (and even myself) have trouble thinking about how
adoption would even occur, at least in the short term.

~~~
tetha
A lot of languages have made one major mistake: It's not easy to get a
development environment going, and some even wanted me to not use vim.

The first step to some kind of adoption is that I can get it running with an
"aptitude install crazy-foo-language-dk", maybe after adding a repository.
After that, crazy-foo or crazy-fooc should be interpreter and/or compiler, and
that's that.

~~~
seanmcdirmid
Ya, well, my languages include their own development environments (I see no
different from language and IDE, actually). Smalltalk was great in this regard
also, to the chagrin of many developers who wanted to use vim.

------
rdtsc
> These languages are defined, right there in the dictionary.

Erlang: see 'career wrecker.'

Please. Someone, wreck my career some more.

Unlike Dylan Erlang was created by a company for a purpose with very clear
goals and it did and still excels at meeting those goals, and nothing out
there gets close to the qualities it has. Not everyone needs those qualities,
but sometimes nothing will do. Erlang is at the core of many solid industrial
applications -- mobile to internet gateways, message queues, trading systems,
large databases, handling millions of concurrent connection and billions of
messages per day for WhatsApp.

What does Dylan do? This is the second time I heard about Dylan. I've played
with Mercury, Prolog, Nimrod, Curry (Haskell + Logic programming) and other
rather obscure languages but haven't heard about Dylan much.

Some languages just don't make it, sometimes it is just luck. However I don't
like the disparaging and angry remarks thrown around at other languages and
ecosystems. That does nothing to promote Dylan it only pushes people away.

~~~
Tomte
> What does Dylan do?

I'm totally unfair here, because I really don't know much about Dylan, but my
view maybe explains a bit why Dylan does not even "win" with people like me,
who really like those strange languages like Lisp or ML.

To me, Dylan was never about creating something new and great. It was taking
Common Lisp and "fixing" its syntax.

Unfortunately, I can't stand Lisp without prefix notation and lots of
parentheses. And people who hate prefix notation and lots of parentheses
aren't interested too much in Common Lisp.

The resulting set of people interested in Dylan was... well, I've heard of
Bruce. Andreas Bogk was doing some _heavy_ advocacy in Germany, to the point
where people just didn't want to listen anymore.

Let's say about ten people in the world cared. And I just don't see how that
set could grow by orders of magnitude.

~~~
nostrademons
Dylan started out with a prefix syntax; the infix syntax was added on late in
its development at Apple, because it was believed the market wasn't willing to
adopt a prefix-syntaxed language. Turns out it didn't matter very much.

Dylan also standardized and simplified _a lot_ of the dark, twisty corners of
Common Lisp. Generic functions became a core part of the language, rather than
something bolted on by ad-hoc macro packages. The collections API was unified
to use generic functions (so no more mapc/mapcar/mapcan/mapl/maplist/mapcon
mess), and the language syntax's infix operators were all defined in terms of
generic functions as well, so you could eg. define a matrix type and have +
and * work on it as expected. The superclass linearization was fixed to be
monotonic. Macros are hygienic. It's a Lisp-1, and everything in the language
is an object bound to a name.

Honestly, I think its lack of success came down to a Worse-is-Better problem.
Dylan is a _really_ nice language, designed by a committee with a combined
total of over 2 centuries of Common Lisp experience. But it doesn't excel in
any one domain. It's a large language, fairly complicated, which tried to
mainstream a number of features with complex interactions and limited
applicability. To an outsider, who doesn't understand those features or how
they might be useful, it's too hard to pick up in a weekend. And so simpler
languages like Scheme, Go, Python, or Erlang steal its thunder - they lack the
total power of Dylan, but you can point to Scheme and say "Small composable
building blocks", or Go and "Fast concurrency with quick, simple deployment",
or Python and "Prototype anything, with batteries included", or Erlang and
"Reliable message-passing concurrency." If you point to Dylan and try to
describe it in one sentence, you get "Generic functions all the way down",
which means nothing to the average programmer.

It's much the same problem that Scala faces, except that Scala features JVM
interop and so Scala advocates can say "Tomorrow's language features with
yesterday's Java code."

------
nostrademons
Dylan was my favorite language back in college. I remember following PG's
essays to Lisp, and then Lisp to all the newer dialects like Dylan or Goo.
Dylan had it all: a metaobject protocol, generic functions, optional static
typing, _infix macros_. I even got started working on an Eclipse plugin for
it, which I ended up shelving after like 3 weeks when I lost interest.

Unfortunately, there are large network effects to programming languages, and
the stuff that really makes you productive - libraries and tooling - Dylan
just lacked. It wasn't practical to write anything larger than an ICFP contest
entry in it. So I went from Dylan to Python, which lacks many of the really
cool language features and is a lot slower, but at least comes with so many
batteries included that you can whip up a prototype for anything really
quickly.

~~~
wisty
I don't really think it's purely tooling that's the issue. If it were so easy
to use Dylan, then the tooling would be easy to write.

I think it's performance.

"Scripting languages" tear compiled languages apart when it comes to
iterating. Write a "hello world" web app page in Python and Scala, then see
which one takes 30 seconds and about a gig of RAM to actually display in the
browser.

On the other hand, if you needed performance scaled across millions of users,
you needed C or C++.

The big languages are C and C++ and Fortran (if you need performance), a long
laundry list of "scripting" languages, and Java. Java is the only one that
actually gets a spot because of its tooling, most other languages only got
tooling because people loved the languages.

There's a lot of compiled languages coming out (or re-emerging) these days
(Scala, Haskell, Swift, Julia) with funky features that require compilation
that don't get you closer to the metal. I suspect that's largely because
computers now have the RAM and CPU power to actually make them fun to use.

~~~
nostrademons
Dylan is compiled. It was done by the same folks who did CMUCL, which was the
original optimizing compiler for dynamic languages, built in the 80s, well-
before Strongtalk begat Self begat Hotspot begat V8.

~~~
leoc
Typo: Self begat Strongtalk.

------
Fice
_You could think of it as putting a low-pass filter on some of the good ideas
from the ’60s and ’70s, as computing spread out much, much faster than
educating unsophisticated people can happen. In the last 25 years or so, we
actually got something like a pop culture ... So I think the lack of a real
computer science today, and the lack of real software engineering today, is
partly due to this pop culture._

A Conversation with Alan Kay, ACM Queue, 2004,
[https://queue.acm.org/detail.cfm?id=1039523](https://queue.acm.org/detail.cfm?id=1039523)

~~~
djur
The history of mass computing involves numerous 'bottleneck' events where an
increase in usage was bought with a curtailment of hardware and software
quality. You had the first blast of cheap minis available outside military and
academic environments. Then you had cheap home computers with limited BASIC
implementations. That was followed by web apps scripted with half-assed '90s
JavaScript and served from cramped mass hosting servers. Then you had
smartphones -- iOS using a somewhat spruced-up but constrained version of the
decades-old NeXTSTEP, and Android using an outdated and pared-down version of
Java.

This is pretty much what "Worse Is Better" is about. Cheap, readily-available
software that runs on cheap, readily-available hardware is always going to
have a huge head start.

~~~
Joeri
Unix is another example of a worse reimplementation of an earlier system to
fit the hardware constraints of the day.

The thing is though, the security model of multics would be a much better fit
for today's security needs, but we don't have it because the hardware that
could run it was too expensive 40 years ago, which seems crazy when I think
about it. Sometimes it feels like the industry as a whole is no longer
ambitious, that building fundamentally better systems is no longer considered
important. It's nice that you can run unix on your phone, but i would like to
run something better than unix on my desktop. Where are the OS's that are
ambitious enough to eventually turn into scarlett johansson in the movie
'Her'?

------
loup-vaillant
> _Algebraic types? Dependent types? You 'll never see them. They're too ...
> research-y. They stink of academe, which is: they stink of uselessness-to-
> industry._

One may think that because closures are finally entering the mainstream (after
what, 5 decades?), we have hope for those things to come as well.

But then I saw Swift. Built-in support for an Option type, so one can avoid
null pointer exceptions. At the same time, this languages manages to recognize
the extreme usefulness of algebraic data types, _without_ using them in their
full generality. Like, why bother with a generic feature when we can settle
for an ad-hoc one?

I'd give much to know what went so deeply wrong in our industry that we keep
making such basic mistakes.

~~~
cwzwarich
> But then I saw Swift. Built-in support for an Option type, so one can avoid
> null pointer exceptions. At the same time, this languages manages to
> recognize the extreme usefulness of algebraic data types, without using them
> in their full generality. Like, why bother with a generic feature when we
> can settle for an ad-hoc one?

Swift has the syntax to define arbitrary algebraic datatypes, even if it
doesn't yet work in the beta versions of the compiler.

~~~
tel
It seems to be likely that 1.0 will hit before this really holds true. In Beta
6 I'm told you can easily crash the compiler with recursive generic ADTs and
most have to travel "through" some heap type to compile at all.

Furthermore, Swift doesn't support enough laziness/deferral/coalgebraic
formulation to have, say, an infinite stream type without breaking GCD. These
will probably be fixed in time, but Swift's ADT support is still pretty
experimental to say the least.

~~~
tempodox
Wow, where did you get Beta 6? I only have 5.

~~~
tel
This is based off comments someone else had on trying to compile some of my
experimental Swift modules.

------
rayiner
Dylan is also, unfortunately, an example of "worse is better" in action. The
extant Dylan implementations were incredibly ambitious. CMU's d2c built on the
experience with CMUCL. Harlequin Dylan (AKA OpenDylan) was "Dylan all the way
down" with a sophisticated native code compiler and an IDE written in Dylan.
Multithreading, generational GC, etc.

And what filled the dynamic language niche? Interpreted languages like Ruby
and Python that have yet to achieve 1970's levels of implementation
sophistication. But simplicity made them agile and portable and allowed
resources to be spent on libraries.

~~~
lispm
Dylan wasn't really a 'dynamic language' in the classical sense. It was
thought to bring the power of dynamic languages to application developers.

Dylan was made obsolete by Objective-C at Apple, not Ruby or Python. Outside
Apple Dylan was made obsolete by Java.

Some of the early Dylan users complained that too much time was wasted on the
IDEs, while the language implementation and libraries were lacking.

------
enduser
I appreciate this article by someone who is serious about contributing useful
solutions to the world--not just the social aspect of programming--and
appreciates a language that empowers him to develop those useful solutions as
readily as possible.

When I'm not programming I like to get some distance from my work and hang out
with people who have diverse interests. When I'm serious about programming I
use Common Lisp. When I'm serious about connecting with other people I use
English. Many people seem to confound these pursuits and end up with languages
that compromise weakly between talking to people and talking to computers.

For me, programming is about solving business problems ASAP in a manner that
is amenable to a long series of minor improvements over many years. Having a
stable language standard with language improvements happening as add-on
libraries is a huge win. My old code keeps working, so I can stay focused on
improvements instead of bailing water.

Also Lisp has the seemingly magical property of being one of the easiest
languages to read, understand, and reason about by programmers who have the
aptitude to learn it--and it scares the pants off of people who don't. With
all of the "expert programmer" pretenders out there it's helpful as an
employer to have something that separates the serious programmers from the
pretenders.

~~~
tempodox
I fully agree. Maybe it's just me, but that sounds much more hands-on to me
than the typical HN comment.

------
nickbauman
If you want to gauge a language look at one thing and one thing only. Who are
these people using it and what are they using it for? The answer to that
question is more important than anything else on the list after that. Amen.

~~~
xplothar
Good one. Used this heuristic to select Erlang and very happy since that time
:-)

------
davidw
There are a lot of things that strike me as being wrong about this essay. Most
importantly, to understand the success or failure, you have to think about it,
and its economics, like any other product on the market. Granted, programming
languages have different characteristics as products than lighthouses or tv
shows or telephones, but they do share things with them that we can learn
from. This is a somewhat dated attempt of my own to share a bit of that
thinking:

[http://www.welton.it/articles/programming_language_economics](http://www.welton.it/articles/programming_language_economics)

A few other things:

* Like rdtsc says, Erlang does not fit the mold in a lot of ways. It had a large corporate sponsor from the get-go, which was good for it in some ways (money for developers), and perhaps bad in others: lots of production code early on means it's not possible to change stuff that is less than optimal.

* As per my article, new/small/unpopular languages need a niche, a beachhead if they are to gain traction. You can't create a new language and platform from scratch with an ecosystem as big as Java's (indeed, piggybacking on the JVM is a popular strategy because of this), so you'd better have one thing where you absolutely kick ass. Erlang has this in spades, for instance. Ruby had Rails. Tcl had Tk and a few other "killer apps". PHP was _way_ easier to get started with than mod_perl, back in the day. I don't see this for Dylan, particularly, but then I don't know much about it, so maybe it's there somewhere, and BruceM will figure it out and the language will gain a following.

------
ZenoArrow
I don't know that much about Dylan, but isn't there some consensus that Julia
is the "new Dylan"?

------
RichardFord
I really like Dylan, but it has a huge problem that never gets discussed -
verbosity. The syntax is entirely too verbose for a modern language. My
suggestion would be to first complete the Intellij plugin and then actually
change the syntax so it can compete with the scripting languages.

------
pjmlp
Very interesting read.

I have a soft spot for Dylan from magazine articles back in the Newton days.

Here we have a Lisp like language, with a more approachable syntax for the
average Joe/Jane developers, AOT compilation and Apple kills it.

I think it is also important to bring out the paper from Erik Meijer about
using Visual Basic to hook the typical enterprise developer into FP (via
LINQ).

[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118....](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118.2025)

------
jbergens
You must find the corporate developer who cares ;-) [http://steve-
yegge.blogspot.se/2010/12/haskell-researchers-a...](http://steve-
yegge.blogspot.se/2010/12/haskell-researchers-announce-discovery.html)

And then maybe try to get more like him/her.

------
AndrewKemendo
Something I never quite understood about languages was why they are restricted
to one or another domain. For example the author writes:

 _Algebraic types? Dependent types? You 'll never see them. They're too ...
research-y_

Why can't those features be baked into C++ or Java?

~~~
peterkelly
I think C++ is full now.

~~~
Symmetry
People keep saying that yet the standards committee keeps nailing new
tentacles on.

[http://c2.com/cgi/wiki?ExtraLegsOntoaDog](http://c2.com/cgi/wiki?ExtraLegsOntoaDog)

------
yarrel
I love Dylan. We should all be using it instead of C++ and Java. Particularly
C++. Each time I type "friend" in C++ or have to use the STL, I miss Dylan.

------
cschep
just say fuck! we're all thinking it!

