
Clojure: If Lisp is so great, why do we keep needing new variants? - mhb
https://blogs.law.harvard.edu/philg/2015/10/12/clojure-if-lisp-is-so-great-why-do-we-keep-needing-new-variants/
======
mbil
Clojure rationale:

"Customers and stakeholders have substantial investments in, and are
comfortable with the performance, security and stability of, industry-standard
platforms like the JVM. While Java developers may envy the succinctness,
flexibility and productivity of dynamic languages, they have concerns about
running on customer-approved infrastructure, access to their existing code
base and libraries, and performance. In addition, they face ongoing problems
dealing with concurrency using native threads and locking. Clojure is an
effort in pragmatic dynamic language design in this context. It endeavors to
be a general-purpose language suitable in those areas where Java is suitable.
It reflects the reality that, for the concurrent programming future,
pervasive, unmoderated mutation simply has to go.

Clojure meets its goals by: embracing an industry-standard, open platform -
the JVM; modernizing a venerable language - Lisp; fostering functional
programming with immutable persistent data structures; and providing built-in
concurrency support via software transactional memory and asynchronous agents.
The result is robust, practical, and fast."

via [http://clojure.org/rationale](http://clojure.org/rationale)

~~~
zeveb
Of course, one can run Common Lisp on the JVM and one can integrate CL with
the JVM; one can add immutable persistent data structures to CL (but all the
mutable stuff would still be available under the covers). I imagine that it'd
be easier to add that stuff to CL than it would be to create a new language
from scratch.

I honestly don't know how hard it would be to add software transactional
memory and asynchronous agents to CL. I don't think it would be bad, but
perhaps the end result would be so unlike CL that it wouldn't be pleasant to
work in.

~~~
PuercoPop
FYI: STM is available for Common Lisp, including in the JVM

[https://github.com/cosmos72/stmx](https://github.com/cosmos72/stmx)

~~~
zeveb
That's pretty cool. I imagine that one could use that with (say) SBCL's
threads and or with ABCL and the JVM's threads and have a decent environment.

------
lisper
> And why hasn’t the C world turned into a similar Tower of Babel?

Because it's much harder to produce a variant of C than it is to produce a
variant of Lisp. Producing a variant of Lisp in Lisp (a.k.a. writing a macro)
is a trivial exercise that anyone can do in a matter of minutes. Producing a
variant of Lisp from scratch is an elementary exercise that an undergrad can
do in a couple of hours. Producing a variant of C is an advanced exercise that
would take even the most skilled programmer many days.

~~~
tormeh
... Because so many languages are made in a matter of hours or days. Can we
please drop the machismo? Making a language takes about a decade, with
multiple people involved. A prototype one without a library can be made in a
matter of months, but nobody would want to use those. I find it plausible that
making a lisp is easier, especially if it's interpreted, but it's not _that_
easy.

~~~
mlieberman85
I think you misunderstood. A dialect of lisp is a fairly trivial exercise. We
were able to do it in a single lecture back in college. Now that dialect had
trivial and essentially useless functionality but allowed us to easily build
on and create new languages, but lisp was designed to allow for these dialects
to be made and to be made easily.

~~~
tormeh
Did you make an interpreter in one lecture? Or did you just add a new macro to
an existing language? S-expressions are tidy, tidy things, but just the
underlying tech used to build a lexer, parser and interpreter takes a good
while to understand, much less learn to write that quickly.

Lisper just reminded me of some HN commenter who once wrote that "any idiot
can build Youtube, it's making it scale that's hard", and that guy gets on my
nerves. Sorry about projecting that onto lisper.

~~~
nickpsecurity
There's tutorials to manually or automatically derive parsers for whatever you
can clearly define. LISP's syntax makes this trivial. From there, you just
need a few functions and tree operations for interpreting/compiling. LISP is
designed for tree operations with tutorials and code for those. From there, a
GC and I/O with many examples in papers and on the web with doc's plus code.
Additionally, there's a ton of LISP/Scheme interpreters to draw on.

Hard to imagine an easier one to interpret. Compiling isn't that difficult if
you target C as it basically takes two techniques. Takes a few more to target
native and you don't get that in 90 minutes like one Scheme-to-C compiler
presentation.

What is your counter-example in terms of mainstream languages that's easier to
lex, parse, interpret, compile, or link than LISP? Or even as easy? Even
Wirth's languages are more complex and difficult to compile despite being
brilliantly simple.

~~~
tormeh
Oh, I do agree Lisp is the simplest. No contest there. Hm, I guess it may be
because building a new language (with some interesting properties) was my
master thesis and it was really unpleasant. I guess I am reluctant to realize
_how_ much easier it would have been with a lisp.

~~~
nickpsecurity
That's a fair point. It's never easy and I had that rough experience, too. I
mean, I wasn't a real expert on these things so mine was a reimplementation of
existing work but still hard.

Curious, what was your language's interesting properties?

"I guess I am reluctant to realize how much easier it would have been with a
lisp."

It might help if you see a modern example. The recent language impressing me
the most with its features is Julia:

[http://julialang.org/](http://julialang.org/)

Wondering aloud about how they pulled all that off, esp macro's, led someone
here to tell me it's actually femtolisp internally:

[https://github.com/JeffBezanson/femtolisp](https://github.com/JeffBezanson/femtolisp)

So, they _appear_ to have built a simple LISP, then used it to incrementally
build a compiler for a complex language. They just represent the syntax
internally in a LISP form and work with it from there. Don't know much more
than that but it shows the power of the concept.

Better demo is the one below as it goes step-by-step in stages. One commenter
(Orion63) pointed out the author was re-using the proven cheat: "build a LISP,
do it all in LISP, profit." Haha. I've considered duplicating that work with
different language options.

[https://news.ycombinator.com/item?id=9699065](https://news.ycombinator.com/item?id=9699065)

~~~
tormeh
My language's main feature was deterministic multithreading, of the
synchronous school. It also forced a certain structure: All threads started
and the shared variables they can write to have to be in the same area,
arguments are always named, and sequence doesn't matter (optional in Python),
shared variables can only be written to by one thread, but read by all. In
contrast to some other deterministic multithreading solutions, the sequence in
which IO is effected is also deterministic, not only the sequence of
application to shared variables. The compiler had a typechecker and compiles
to C++, using GCC to do the rest. I guess I should submit it to HN sometime...

~~~
nickpsecurity
That sounds interesting. Might be a decent stab at real-time programming and
concurrency as it would need most of that.

"All threads started"

In many languages, the threads kind of sleep unless activated explicitly. Do
you mean just the activated threads started or did you choose to have them all
run at once for some reason?

"shared variables can only be written to by one thread, but read by all"

Good idea. BeOS also had read-many, write-one in form of benaphores:
semaphores that only locked for writing. Strategy made it really fast.

"the sequence in which IO is effected is also deterministic"

What do you mean by that?

"compiles to C++"

The compile to C++ was my trick, too, since I knew I couldn't out-code GCC
team. Although I wrote mine to avoid using C++ while being compatible lol...

"I guess I should submit it to HN sometime..."

You should. Either that or a forum where people discuss various language
tradeoffs.

------
pekk
It's not a binary, either/or choice between "Lisp is perfect" and "we need a
new variant of Lisp". Here's a map of possible attitudes toward Lisp. Take
your pick.

On the Lisp-affinity axis,

1\. Lisp is almost perfect as-is, we don't need new variants which will only
adulterate the purity and greatness of Lisp as it already exists.

2\. Lisp is great, and my favorite variant (e.g. Scheme) is the greatest
variant because it is the purest available representation of the Lisp concept.

3\. Lisp is mostly great, it just needs features x, y, z... which are provided
by my favorite variant (e.g. Clojure).

4\. Lisp is great, but really only for particular things, and its variants
might be great for slightly different things.

5\. Lisp isn't so great at all, and making new variants is just putting
lipstick on a pig. Use a functional language other than Lisp (e.g. Haskell).

6\. Functional programming as a whole sucks. Use a mixed-paradigm or
imperative language (Java, C#, Perl, etc.)

7\. High-level programming sucks. Use something without mandatory garbage
collection (C++, Rust, C, Assembly).

On the intolerance-to-tolerance axis,

1\. Lisp is almost perfect as-is, we don't need new variants which will only
adulterate the purity and greatness of Lisp as it already exists.

2\. Lisp is great as-is, so we don't really need new variants, but since they
are also Lisp, they are also great.

3\. Lisp is great, but some other languages are also great.

n. It doesn't matter what language you use, all language differences are only
cosmetic.

~~~
kaffeemitsahne
This could be a nice xkcd-style infographic hehe.

------
bunderbunder
I think the question is framed poorly, and on that ground I'd challenge it
from two angles:

First, one could argue that there are all sorts of additional C-like languages
that weren't listed - Java, C#, Rust, Go, etc.

One could also argue that "Lisp" is a more fundamental category of languages
than "C", so it's not an apples-to-apples comparison. Exhibit A: the author's
list of Lisps included both dynamic and static languages, whereas the list of
C's included only statically typed ones. It captures the idea of a functional
language with a unified representation for both code and data. C-like, on the
other hand, seems to specifically mean C and its superset languages (a much
smaller family than its descendants) according to the author's usage.

------
codemac
S-expressions != "lisp"

There is the Common Lisp standard, and those variants really are managed like
C compilers are. It's pretty great, and the differences are tiny.

Scheme has standards -- but yes, due to the language's easy implementation
many people have made implementations of varying adherence to that standard.
This is a _good_ problem, it means people are able to implement their own
languages.

Finally - clojure's rationale is pretty straight forward, and has obviously
met an incredible niche in the PL space. I don't think it's that controversial
of an implementation of S-exps.

~~~
Skunkleton
I have always been a doubter of lisp likes and even I think this article makes
no sense. If you are going to lump CL, Scheme, Clojure, etc together, then the
list of "C" like languages is much much longer than the three they have in the
article.

------
weavejester
This is more nomenclature than anything. We have a broad definition of what a
lisp is, but a fairly narrow definition of what a C variant is. I'd argue that
C++ and Java have more in common with each other than, say, Shen and Emacs
Lisp.

------
davidrupp
The major contribution of Clojure is not that it's a Lisp; it's that its model
of immutability-by-default eliminates whole categories of errors due to shared
mutable state. I personally also enjoy having non-parenthesized literals for
things like maps and sets, but those are incidental benefits.

------
prospero
Clojure isn't a minor variation on earlier Lisps, it's something that uses
s-expressions with a significantly different set of default data structures,
core library functions, and platforms.

This is like asking why we needed C when we already had ALGOL.

~~~
nickpsecurity
First point is true. Second point isn't: BCPL was created solely because they
couldn't build an improved ALGOL (CDL) on a 1950's computer. Took out every
good feature the hunk of junk couldn't compile. C was modified BCPL/B to do
that on a PDP-11 w/ UNIX a watered-down MULTICS for same reason.

A comparison would be LISP 1.x vs modern Scheme or Common LISP. And only using
LISP 1.x because you had to work with a microcontroller instead of a PC. ;)

~~~
prospero
My point was that treating a language as just a few implementation details
atop its syntactic heritage is overly reductive (as you demonstrated).

------
dragonwriter
Lisp is not a language like C (Common Lisp is, but Lisp is broader).

Lisp is a language family, sort of like you might have a family of languages
with Algol-derived syntax.

And, you know, we see new members of the latter family all the time; they're
just so dominant among programming languages that we don't even talk about
them as a related group.

------
rubiquity
The difference is that Lisp is a proud heritage for a language to have. So
when you make a Lisp, you naturally tell everyone you made a Lisp. When you go
out and make the next twist on C, which is what Java and Go originally
were/are, you keep that skeleton in the closet.

------
copsarebastards
It's the thinking and core idea of Lisp that is great, not the implementation.
Every implementation includes tradeoffs, hacks and compromises.

Clojure's success is largely due to the fact that they've made a set of
tradeoffs, hacks, and compromises that makes it usable in a lot of business
contexts. Running on the JVM means giving up some really core Lisp ideas (like
tail call elimination) but it also means that you can distribute a JAR file
and nobody will know you didn't write it in Enterprise SafeAndTrustworthy
Java.

We don't keep needing different variants, we keep needing different
implementations.

------
kazinator
The thing about Lisp-like languages is that they have some superficial
similarities which get them lumped together, because programmers expect to be
able to classify programming languages based on superficial similarities. If
it has symbols encased in parentheses, it's some kind of Lisp (even if the
authors don't use that word in the name).

They are not all the same any more than languages which have braces for block
structure and f(x, y) function calls are the same!

Languages which have a complicated surface syntax tend to distinguish
themselves sharply on that syntax. Thus we can regard PL/I, Pascal, C, Ada,
Rust, Go and Java to be semantic offshoots of the Algol family: new dialects
of Algol.

If you invent a new dialect of Algol with new keywords and new syntactic
gadgets for declaring, calling, throwing, catching, structuring, flowing, and
so on --- old semantics with a new skin --- nobody accuses you of making just
a dialect of Algol.

If you make a new language with S-expressions for its syntax, but new
semantics under the hood, it is "just another Lisp" that some university kids
can throw together in an afternoon.

Syntax is semantics is the popular view; you can't possibly have new semantics
if you don't _look_ different. And the presence of some old semantics
exhibited in a few examples of course confirms that you don't have new
semantics. Hey look, functions look the same, and there is let and lambda.
Same old junk!

------
lispm
Since Clojure is not a Lisp in the narrow sense, this is a useless question.

If Lisp means something broad, then we would list very different languages:
Logo, Dylan, Julia, R, Javascript, Racket, ACL2, ... - again that would be a
useless question.

Also, if GNU/Linux is so great, why do we keep needing new distributions... if
Perl is so great, why do we keep needing new scripting languages? ... if the
Porsche 911 is so great, why do we keep needing new variants?

------
fiatmoney
"If English is so great, why do we keep inventing new slang?"

------
brudgers
In a sense, this is like asking "If Turing machines are so great, why do we
keep needing new variants?"

Lisps are lisps because to some degree they express a particular model for
computation. What makes them seem more alike than the varieties of stack based
system is that the first pass at a notation for ordering application
(parenthesis) is recommended by its similarity to the mathematical notation
for ordering application. In other words, languages modelled [more or less] on
lambda-calculus look like "a thing" by the accident of similar notation,
whereas languages modelled [more or less] on Turing machines don't look like
"a thing" by virtue of the diversity of notation.

In practice, this means that Scheme and Common Lisp and Clojure get lumped
together while Forth and Cobol and C# don't, even though there is as much
variation between the execution model in the first group as the second.

------
rayiner
Nobody bases their "something better" on something they don't think is already
great.

------
kazinator
Lisp inspires some people to work at the implementation level.

My TXR Lisp ([http://www.nongnu.org/txr](http://www.nongnu.org/txr)) has
strong ties to Common Lisp, but it does certain things which I've been able to
do easily due to working at the implementation level.

For instance, I have regular cons cells as well as lazy cons cells, both of
which satisfy the consp function, and are recognized by all the library
functions, from car and cdr onward. If this were develpeed in an existing
Lisp, it would require rewriting the library from scratch. E.g. not using the
standard car, cdr, mapcar and so on.

Another example is that forms can optionally express Lisp-1 style evaluation
for those arguments which are symbols. This is sugared over with square
brackets.

    
    
      ;; list is in the function namespace, but resolved
      [mapcar list whatever]
    

Implementing this correctly means writing your own code walker; it's not just
"add a macro and hand off the code to Lisp's evaluator/compiler". The exact
rule is that the closest lexical binding of the identifier is used, whether it
is a function or variable; and if there is no lexical binding, then the global
one is used. If both a function and variable of the same name exist globally,
then preference is given to the variable binding. It doesn't end there: symbol
macros have to be handled properly: in [a b], a or b or both could be symbol
macros. But they get shadowed by function bindings which are lexically inner:

    
    
      (defsymacro ((m whatever))
        (flet ((m () ...))
           (m ...) ;; unambiguously function
           (list m) ;; expanded as symbol macro
           [m ... m ...])) ;; function everywhere 
    

Lisp has always been a family of languages. From almost the very beginning,
there were numerous dialects. Every major institution that had Lisp activity
had its own dialect!

It is in recognition of this that the late John MacCarthy expressed his wish
that no dialect be just called "Lisp".

Forty years ago you could have said, "if Lisp is go great, why do we have
MacLisp, InterLisp, BBN Lisp, ..."

Part of the reason Lisp is great is that it teaches programmers to fish.

~~~
klibertp
Hi, I just wanted to tell you that I'm learning TXR right now
([https://klibert.pl/output/this_week_on_polyglot.html](https://klibert.pl/output/this_week_on_polyglot.html))
and I like it very much. My current goal is to write a tutorial for it: the
docs are good and pretty detailed, but they aren't exactly friendly to new
users. I hope I'll manage to finish this, I think TXR deserves much wider
recognition.

Anyway, thanks for your work on TXR and thanks for making it available. Great
job!

~~~
kazinator
Hi. I think your code snippet:

    
    
      @  (block cd)
      @    (bind v @(get-string (open-command `cd @path && git remote -v`)))
      @    (next :string `@v`)
      @      (skip)
      vps@a(fetch)
      @      (bind url @(trim-str a))
      @  (end)
    

probably just wants to be something like perhaps this:

    
    
      @(next `!cd @path && git remote -v`)
      vps@\t@url (fetch)
    

Match directly into the output of the git command with @(next ...), and
extract the url directly: it is delimited by tab from the remote name (which
we can use literally or represent using @\t) and by a space from the (fetch)
or (push) keyword. Thus we can eliminate the trim-str extra step and variable.

Also if v holds a string, `@v` simplifies to v!

Another thing is that if we are just marching through data to produce output,
and don't actually need to collect anything to analyze, we can convert this
structure:

    
    
      @(collect)
      ...match...
      @(end)
      @(output)
      @  (repeat)
      ...spew...
      @  (end)
      @(end)
    

into:

    
    
      @(repeat)
      ...match...
      @  (output)
      ...spew...
      @  (end)
      @(end)
    

It's not just shorter, but efficient too if the data is large (it doesn't hang
on to collected data as it marches through it).

Cheers ...

~~~
klibertp
Thanks! I rewrote the example according to your suggestions. It's true that
the part with @v, open-command, etc. wanted to be written differently. I
realized that there has to be a better way of calling external command - I
just couldn't find it then. Looking at the docs now, the meaning of ! is
mentioned briefly:

> The @(next) directive supports the file name conventions as the command
> line. The name - means standard input. Text which starts with a ! is
> interpreted as a shell command whose output is read like a file. These
> interpretations are applied after variable substitution. If the file is
> specified as @a, but the variable a expands to "!echo foo", then the output
> of the "echo foo" command will be processed.

So it looks like I just missed it on my first reading.

------
sbochins
It seems like the author is conflating two things, Lisp not being popular and
there being many Lisps. As others have stated, the reason there are many Lisps
is because it is very easy to make a Lisp. You can check on github for Lisp
implementations and find very many. The reason why it isn't popular is
something I don't really want to get into. But, I will say that modern
languages are transitioning much closer to Lisp than they are backtracking to
languages like C.

------
reilly3000
There is a difference between language and execution. I can put together a
clojure server and have it deployed virtually anywhere in about 10 minutes. I
have never tried to deploy a CL app but I imagine it must be a little more
complicated than making an Uberjar. The semantics of Lisp are beautiful. Why
do we need to force them into a single language when it's best strength is
portable symbolic logic?

~~~
mateuszf
I'm not Common Lisper, but actually CL distribution is trivial compared to JVM
(for user applications, not servers). It's possible to create native
executables, which don't need any frameworks to exist on the target systems.
So deploying CL may take more work (making multiple executables), but the
programs itself don't need users to have some frameworks installed before.

------
time_is_scary
I'm getting hung-up on the word "need".

~~~
pekk
In what way? Either there is a need for variants, or there is no need for
variants but it is perfectly okay to make them, or there is no need and it's
not okay to make them.

~~~
time_is_scary
If there is a need, then they should be made. If there is no need, then they
can still be made (maybe even revealing a new need in the process).

I guess it seems like a non-issue to me.

Then again, applying my own reasoning:

There is no need to ask this question, but it's fine to ask it anyways.

------
Mikeb85
Not sure I'd say new variants are 'needed'. But the fact that Lisp languages
are so easy to write encourages people to write them.

Besides, Clojure has some fairly modern and novel stuff going on when it comes
to concurrency, in addition to being a wholly modern Lisp.

------
nickpsecurity
Well, you first need to understand the history of C to fully get why it's such
a terrible language. I have link (below) with both a long video and a brief
summary in numbered points. I think the arguments against using it _today_ are
solid and conclusive if you're not forced to for some pragmatic reason.

[http://pastebin.com/UAQaWuWG](http://pastebin.com/UAQaWuWG)

Nonetheless, C got a lot of use via network effects. There actually _have_
been many C dialects out there: compiler variants, extensions for specific
systems, safe/secure versions of it, parallel/concurrent versions, hardware
languages... the list goes on. Probably dozens of C variants appeared and
mostly failed. The main reason is legacy: whatever replacement exists must
support the existing C language usage and codebase. Any fixes that break apps
or require rewrites are rejected. Marketing and community effects are another
issue.

Now, we have to ask what is LISP. I think Graham said it best in that it's an
axiomatization of computation itself:

[http://www.paulgraham.com/rootsoflisp.html](http://www.paulgraham.com/rootsoflisp.html)

So, LISP is that concept and core. Its nature lets you express various
constructs however you want to. You break the problem into smaller pieces
while simultaneously modifying the language to better express their solution.
What's best way of expressing a problem naturally varies from programmer to
programmer. The LISP variants therefore are the many ways different people (or
communities) looked at solving a problem within the LISP core and philsophy.
Some stay purely in it, some diverge wildly from it, and some are in the
middle. The fundamental concept and approach, though, with many
interpretations on expressing it are why there are so many LISP's, though.

Clojure, a new expression of the LISP concept/core, has advantages expressed
well in the first paragraphs of the rationale:

[http://clojure.org/rationale?responseToken=894e644e3f4488b17...](http://clojure.org/rationale?responseToken=894e644e3f4488b17953fe43bbd586c0)

This philosophy and use of DSL's for expression is no longer unique to LISP.
One can see it in REBOL/Red quite directly with some takeup in languages such
as Haskell and Julia. Forth also does this sort of thing with stack
operations. So, we're seeing uptake of the idea even in mainstream with them
recognizing the benefits. All started with McCarthy's reducing all computation
down to some primitive, consistent elements that were shown to express or
adapt to anything.

------
amirouche
“If Lisp is so great, why did this guy have to build a slightly different
version instead of building a popular application program in an existing
version of Lisp, such as Common Lisp?”

------
kazinator
> * We accept the proposition that C is feeble and yet there are only three
> major variants of C: C, C++, and Objective-C. *

C90, GNU C, C99, C11, C++98, C++05, C++11, C++14, ...

------
GigaMos
because "LISP can learn a lesson from all the languages that borrowed ideas
from LISP. It is nature's way."

from [http://blog.samibadawi.com/2013/05/lisp-prolog-and-
evolution...](http://blog.samibadawi.com/2013/05/lisp-prolog-and-
evolution.html)

------
atomicbeanie
And if antibiotics are so great, why do we keep needing more of them?

------
macmac
The question is its own answer.

~~~
mod
So why are we trying to differentiate lisps, given that they're excellent?

Are they each better for some reason?

Why innovate on perfection? Or, why not improve the existing lisps rather than
create new ones?

I understand what you meant, but I don't know the answer to these questions--
before or after reading the article.

(I have given clojure a shot, though, and I enjoyed myself!)

~~~
zem
don't think of "lisp" as a language, think of it as another branch in the
evolutionary tree of programming languages. the "lisp is great" people are not
saying that any given lisp-based language is the be-all and end-all of
programming languages, they are saying that the lisp branch is the best branch
to explore.

~~~
macmac
That's just scary...

~~~
KurtMueller
Hi macmac. Can you expound a bit as to why you this line of thinking is scary?

------
atomicbeanie
And if antibiotics are so great, why do we keep needing new kinds?

------
TazeTSchnitzel
Clojure isn't a lisp, it doesn't have conses.

------
redahs
Because Lisp doesn't have a flat memory model, you can't directly produce a
binary array suitable for direct copying to output device memory (ex. network
card, GPU, etc) by simply expanding expressions using the built in syntax.

This means it's necessary for LISPers to continually reinvent the input-output
layer and add non referentially transparent features to the language borrowed
from Java or C in order to do any useful work.

Doing so is also contrary to the ideals of functional programming however, so
they will continually overthrow the established LISP to invent a new one.

edit: to downvoters, replace "you" with "language runtime implementers", and
understand that this is not a normative assertion, but a descriptive theory to
explain why we have seen a huge number of non-interoperable Lisp\Scheme
implementations with minor differences over the years.

~~~
zeveb
> Because Lisp doesn't have a flat memory model, you can't directly produce a
> binary array suitable for direct copying to output device memory (ex.
> network card, GPU, etc) by simply expanding expressions using the built in
> syntax.

What are you talking about? Common Lisp has built-in support for vectors:

    
    
        (let ((vector (make-array length :element-type 'unsigned-byte)))
          ;; do stuff with that flat array of bytes
          ;; …
          (replace ext:gpu vector))
    

(In this example, EXT:GPU is a chunk of memory which is overwritten by VECTOR;
one might conceivably use COPY-SEQ instead of MAKE-ARRAY, and one might of
course just operate on EXT:GPU)

> Doing so is also contrary to the ideals of functional programming

Lisp isn't about functional programming; Lisp is a multi-paradigm language,
supporting imperative, functional and OO programming out of the box (and can
of course be extended to support aspect-oriented programming, table-oriented
programming or what-have-you).

~~~
redahs
I would argue that a significant number of people were attracted to
Lisp\Scheme variants over the years because of the promise of functional
programming and the theoretical advantages of being to design programs using
functional principles.

The problem with your example is that evaluation of the expression performs a
side effect on an internal machine specific to the implementation of Common
Lisp, and that it is not a portable and abstract expression defining a return
value.

Ideally we would be able to define a return value as a variable length
sequence of constituent elements through recursive evaluation, where the
process of evaluation is guaranteed to compactly allocate all of the
constituent elements contiguously on the same location of the stack.

This would allow the runtime implementation to directly copy the contents of a
value of an arbitrary size to a necessary output device, so long as the
runtime had access to a portable and functional definition of the value to be
copied.

Because the Lisp language family does not provide for this, there has been a
need to continually invent new implementation variants which each model
different internal machines for performing these tasks.

~~~
zeveb
> The problem with your example is that evaluation of the expression performs
> a side effect on an internal machine specific to the implementation of
> Common Lisp, and that it is not a portable and abstract expression defining
> a return value.

I don't get what you're saying: REPLACE is bog-standard Common Lisp; it's not
implementation-specific. You complained that it's not possible to produce a
'flat binary array suitable for direct copying to output device memory'; I
demonstrated that it is.

> Ideally we would be able to define a return value as a variable length
> sequence of constituent elements

You can do that with '(simple-array unsigned-byte)…

Seriously, Common Lisp was built to be an industrial-strength language: it can
bit-blit with the best of 'em, as well as supporting high-level dynamic
concepts. It's pretty cool!

~~~
redahs
> _complained that it 's not possible to produce a 'flat binary array suitable
> for direct copying to output device memory'_

I believe my mistake was using the term 'array' rather than 'string'.

This should read it's not possible to define a 'flat binary string, of
variable length, in a pure functional manner, through implicit concatenation,
with the guarantee that every return value reduces to a binary string, using
the commonly accepted semantics of s-expression evaluation common to all
Lisps'.

It certainly possible to build and process fixed size binary strings by
adding, as I mentioned, "non referentially transparent features" to your
language implementation, including Common Lisp's "replace" operation.

However, since such semantics are procedural and imperative in nature, they do
not lend themselves well to the production of concise and declarative
definitions suitable for describing the encoding of formatted binary strings
adhering to a specific protocol, such as what one might find in the
documentation of protocols for mass communication, encryption, and long term
storage.

If they did, then we would most likely expect to see Scheme\Lisp\Clojure
programmers of all varieties directly importing the same open protocol
definitions from an established sources directly into their application
codebases, and not repeatedly rewriting this functionality or relying on
bindings to C\Java libraries.

> _Common Lisp was built to be an industrial-strength language_

I am not discussing the limitations of Common Lisp in particular, I am
discussing the limitations of Lisp family of languages as a whole, and
describing why there will most likely always be someone implementing a new
slightly different variants of it, of which Common Lisp is but one example,
and why it is unlikely that Lisp programmers will ever converge on one single
language.

