
Homoiconicity Revisited - vanschelven
https://www.expressionsofchange.org/homoiconicity-revisited/
======
fiddlerwoaroof
I think this is close, but sort of missing the point: it’s possible to extend
Common Lisp to take code in the form of Python source code (clpython),
JavaScript source code (cljs) or many other textual syntaxes: originally, the
iconic s-expression syntax of lisps was intended to be a sort of IR dump for
m-expressions.

What makes CL homoiconic has nothing to do with the textual syntax, but rather
the existence of a function READ that takes _some_ textual syntax and produces
the code to be evaluated in the form of the language’s ordinary
datastructures: EVAL and friends consume a representation of the code that is
neither just a blob of bytes nor some exotic AST objects, it’s lists, hash-
tables, strings, vectors, and all the types of things programmers manipulate
every day.

The implication of this is that intercepting and modifying the code between
READ and EVAL doesn’t really require any special knowledge: you have to know
the semantics of your language, and how to manipulate its basic
datastructures, but you don’t need to understand any special “metaobjects” for
representing code.

~~~
dTal
Sounds like any random language can become homoiconic with the addition of a
parser library.

~~~
fiddlerwoaroof
If it parses to primitive data structures (and actually produces a structure)
and allows you to intercept the code between parsing and
evaluation/compilation, this is exactly what I’m saying.

~~~
pjc50
This sounds like LINQ would definitely qualify?
[https://docs.microsoft.com/en-
us/dotnet/api/system.linq.expr...](https://docs.microsoft.com/en-
us/dotnet/api/system.linq.expressions?view=netcore-3.1)

I think your best insight here is the importance of the "lid off" intermediate
representation; plenty of languages have an eval function of some sort, but it
does both READ and EVAL in your terms with no intermediate access to the
parsed representation.

~~~
fiddlerwoaroof
The issue I’ve had with things like that is that I have to learn a whole new
API for manipulating syntax: while there are definite advantages to this,
representing code as everyday types that very generic functions can operate on
has the advantage of making metaprogramming look like normal code. e.g. if I’m
doing try...finally... stuff alot, I can right a macro that transforms:

(with-db [conn (connect ...)] ...)

To:

(let [db (connect ...)] (try ... (finally (close db)))

Just like:

    
    
        (defmacro with-db [[sym expr] & body]
          `(let [~sym ~expr]
            (try ~@body
              (finally (close ~sym)))))
    

The backtick/tilde notation isn’t macro-specific: it’s a generic way to
template readable datastructures that I can use anywhere, so when I see it in
a macro, it isn’t some strange API to learn, it’s just a handy way to build up
the datatypes I use all the time.

------
tom_mellior
I think this definition makes sense, though I agree with
[https://news.ycombinator.com/item?id=23426047](https://news.ycombinator.com/item?id=23426047)
that the notion of homoiconicity, and therefore a precise definition of it, is
not really useful.

What I'm disappointed by is the analysis of the examples and counter-examples,
which even for a cursory analysis is somewhat sloppy. For example:
"S-expressions are trees, and the nesting of items is immediately apparent: (
denotes the start of a child, and ) its end." Not really, as I would say that
"(f x y z)" either has four children, or one distinguished function symbol and
three children. But there is only one pair of parentheses. Even if you take
the reductionist view that there really are two children, namely "f" and "(x y
z)", there is only one pair of parentheses in the original.

Further: "JavaScript has strong language support for composable, tree-like,
structures (bullet 1) and the structure is immediately apparent by looking at
the pairs of { and } brackets (bullet 3)." It's more complicated than that,
since "f(g(x + y * z))" is also highly structured, and much of the structure
is immediately apparent, but some of it isn't because it uses precedence
rules. Also, no {} in sight. And then: "Java has no strong language support
for literal representation of simple composable data in the language (bullet
1) and also fails to meet the other criteria." I don't see how Java fails
bullet 3 if JavaScript is claimed to pass it. The syntax uses {} and other
grouping and expression forming mechanisms in essentially the same way. If
anything, Java makes structure a tiny bit more explicit since it requires
explicit semicolons to delimit statements.

~~~
cies
> I don't see how Java fails bullet 3 if JavaScript is claimed to pass it.

I understood it as Java not having a data structure that can be used as a tree
available as literal. JS has that. This is valid JS but not valid Java: [[1,
2, [1, 2, []]], [[], [], 0], 1, 2, 3]

~~~
tom_mellior
It might not be what the article meant since it only mentioned { } braces in
connection with JavaScript, but this is a very good point!

------
foobar_
I wonder where tcl falls on that list. It's sorta like everything is a command
/ list of commands and a command is a list of strings ?

------
DonaldPShimoda
> let’s say a word means what the people who use it mean by it.

This is how words work, so I don't know why the author says "let's say". There
is no "correct" definition of a word; the dictionary which you might consult
to obtain such a definition is itself just recording what is the most
commonly-used meaning of the word among people who use it most natively.

I make this point not out of pure pedantry, although that may play a part, but
also because nobody seems to know what "homoiconicity" actually is — nor does
it seem to actually be very important for us to find out! When people cannot
agree on what a word means, I think it's reasonable to say that that word
doesn't really _have_ a meaning.

Some time ago, I had the opportunity to talk to Matthias Felleisen — the
progenitor of the Racket project (a derivative of Scheme, itself a derivative
of Lisp). When I asked whether he felt homoiconicity was an important quality
of a language, he indicated something to the effect of (heavily paraphrasing)
"I don't know what that word means, and you don't know what that word means,
but whatever you think it means is not important. It doesn't matter."

Shriram Krishnamurthi, a former student of Felleisen's and one of the original
members of the Racket project, can be found to disparage the word whenever he
finds it on Twitter. He also consistently refers to it as h11y (that's
"H-eleven-Y"), not just to save characters in tweets, but also, I think,
because the word itself is not very important. I've seen multiple exchanges
where he challenges peoples' attempts at defining the term precisely, and
invariably they give up. (One such exchange is visible at [0].)

Another Racketeer, Matthew Flatt, recently started a push to build a new
Racket language (now called Rhombus, I think) which abandons s-expressions
almost altogether, but without losing any of the benefits of Racket.

If these people, who are all everyday Lispers, and who are widely known and
respected in the programming language community, don't have a solid definition
of "homoiconicity" and, further, don't think it's worth investing the time to
define it... maybe we should stop getting articles like this, and maybe we
should instead tell people to abandon their search for the One True Meaning.

(I do want to say that I think this is one of the better articles about the
topic that I've read, and the author did a good job. I just think the topic is
not as interesting as the author expects their readers to find it.)

Homoiconicity doesn't matter. Nobody can define it very precisely, so no two
people will have the exact same definitions in their head when discussing it,
so any talk regarding it will be necessarily lossy (or, in the worst case,
completely non-productive). If the literal representation of your language
happens to evoke some imagery in your mind of the secret machinations of the
underlying forms or something, that's great! But I don't think it's worth
pursuing this quality specifically. What syntax works this way for you may not
work well for others, or vice versa.

I think that, instead, it is better to focus on more specific goals. Maybe
your language is very tree-oriented, so you make it easy to represent trees
directly. Something like that. But the specific goal of "homoiconicity" —
whatever that means to _you_ — is perhaps not worthwhile, nor should we be
seeking to define it. We should leave it by the wayside and move on!

[0]
[https://twitter.com/ShriramKMurthi/status/104694495053182976...](https://twitter.com/ShriramKMurthi/status/1046944950531829760)

~~~
openfuture
I have a different view:

Everyone knows what homoiconicity is supposed to mean, however, it is not
actually a feasible thing to achieve completely and so people use the word
approximately..

In my mind I transform "lisp is homoiconic" into "lisp is more homoiconic than
most other languages" that is to say, they are closer to achieving this
impossible idea.

Really you could let go of this particular "fancy word" (after all it's not
like it's the only needlessly complex programming language theory jargon...)
and just call it "syntax consistency" or something along those lines.

I sympathize with your stance, precision is important, but people generally
don't treat words with any respect so it's somewhat of a futile endeavor.

~~~
galaxyLogic
That makes me think of "pure languages". Some languages are "pure" because
they have no side effects? Yes maybe but which practical programming languages
have no side-effects?

So similarly as you state about homo-iconicity some languages are simply purer
than others.

~~~
DonaldPShimoda
The use of "pure" in this case isn't to say "better", but rather to do with a
mathematical viewpoint.

In math, if I say "x = 3" and then "y = x + 2", it is true that "y = 5" and
nothing else (assuming our regular understanding of addition is given).

In programming, it might also have come to pass that some data was output to
the console, and some logging took place, and maybe an HTTP request or two
were carried out in the interest of making that evaluation. These are not
"pure" mathematics; side-effects are _impure_ because they are not accounted
for directly in the semantics.

Purity is not on a scale. Either a language is pure, or it isn't. Nobody made
any claim that purity is something to strive for in a language. Haskell has it
because it was designed specifically to be a language where academics could
carry out their "What if?" fantasies (which is catalogued in "A History of
Haskell", a paper from HOPL III (2007)).

I will say that purity is useful in _most_ practical programming — just not at
the top level. The more purity you have in your code (by which I mean, the
fewer side-effects that are not evident in the type system), the easier it is
to make changes to your code and to use it with abandon. The more side effects
you have, the harder it is to restructure code and reason about how things
work. So within the implementation of a particular application, there might be
regions of purity and then (encompassing) regions of impurity. But a given
function is either pure or impure — no gradient to it at all.

~~~
tome
> Purity is not on a scale. Either a language is pure, or it isn't.

This is not quite right. "Purity" mean that with respect to some unspecified
by fine-grained semantics various pairs of programs are equivalent. For
example, in the denotational semantics of Haskell the following are equivalent

    
    
        let x1 = rhs
            x2 = rhs
        in body
    

and

    
    
        let x  = rhs
            x1 = x
            x2 = x
        in body
    

But they are not equivalent under the operational semantics. The former
allocates two closures, the latter one. The "scale" of purity then is measured
by how fine-grained the semantics is. Haskell is pure at the "scale" of
denotational semantics, but not at the scale of operational semantics.

~~~
DonaldPShimoda
Ooh I had not heard "purity" defined this way previously in such explicit
detail, but it makes perfect sense and I really really like it. Thank you so
much for giving me better words for this going forward!

Edit: I think, though, that my point kind of still stands in that, with
respect to a specific level of semantic detail, a language is either pure or
it isn't. Further, I think it isn't useful to compare Language A at Semantic
Level 3 to Language B at Semantic Level 1 to compare purity; you're generally
going to be sticking to one level of semantic detail at a time, and within
that lens all languages will either be pure or impure. Right? Hmm maybe I need
to think on this more.

------
remolacha
Maybe off topic, but I find the styling of this website so pleasant. The
colors and fonts really improve my reading experience.

~~~
vanschelven
Thanks, though credit where credit is due: it's basically
[https://github.com/coletownsend/balzac-for-
jekyll/](https://github.com/coletownsend/balzac-for-jekyll/) with some minor
tweaks

~~~
macintux
Not sure why, but the list numbers in your working definition are vertically
chopped on my iPad (but not my iPhone). Only the right half of each index
number is visible.

------
empath75
It seems like there's two ways to go from here -- abandon it, or formalize it.
Just because formalizing a concept is difficult, that doesn't make it
impossible. I think everyone agrees that there is _something_ that
distinguishes lisps from other languages, even if it's hard to say exactly
what it is.

------
nemoniac
An article about homoiconicity without once mentioning either the word
"transform(ation)" or "macro". I can't quite make my mind up whether this is a
tour de force or if it misses the point entirely.

------
galaxyLogic
Isn't the idea of homoiconicity in its simplest form that every data-structure
can also be interpreted as a program?

~~~
cies
I'd reverse it: every program can also be interpreted as a data-structure.

And this is what the article is about, it's just a lot more in depth
dissecting all the ingredients needed to get there.

~~~
tom_mellior
Every program in every programming language _is_ interpreted as a data
structure. Otherwise we would not have compilers and interpreters.

------
mirekrusin
Maybe it's just: 1\. code is written in ast, and 2\. (macros) you can
construct objects that can be interpreted as ast

------
DonHopkins
PostScript is homoiconic, and semantically quite Lisp-like, when it's not
being syntactically somewhat Forth-like.

[https://news.ycombinator.com/item?id=21968842](https://news.ycombinator.com/item?id=21968842)

>[...] PostScript and Lisp are homoiconic, but Forth is not. The PSIBER paper
on medium goes into that (but doesn't mention the word homoiconic, just
describes how PS data structures are PS code, so a data editor is a code
editor too).

[https://medium.com/@donhopkins/the-shape-of-psiber-space-
oct...](https://medium.com/@donhopkins/the-shape-of-psiber-space-
october-1989-19e2dfa4d91e)

Also, here is a metacircular PostScript interpreter, ps.ps: a PostScript
interpreter written in PostScript! Since PostScript is homoiconic and so much
like Lisp, it was as easy as writing a metacircular Lisp interpreter (but
quite different in how it works, since PostScript and Lisp have very different
execution models).

[https://donhopkins.com/home/archive/psiber/cyber/ps.ps](https://donhopkins.com/home/archive/psiber/cyber/ps.ps)

The heart of the metacircular interpreter, "iexec", uses some dynamically
defined macros like MumbleFrotz, PushExec, Popexec, containing embedded
literal references to data (a dictionary representing the interpreter state,
and ExecStack, and array representing the execution stack).

    
    
        % interpretivly execute an object
    
        /iexec { % obj => ...
          100 dict begin
            % This functions "end"s the interpreter dict, executes an object in the
            % context of the interpreted process, and "begin"'s back onto the
            % interpreter dict. Note the circularity.
            /MumbleFrotz [ % obj => ...
              /end load /exec load currentdict /begin load
            ] cvx def
    
            /ExecStack 32 array def
            /ExecSP -1 def
    
            /PushExec [ % obj => -
              /ExecSP dup cvx 1 /add load /store load
              ExecStack /exch load /ExecSP cvx /exch load /put load
            ] cvx def
    
            /PopExec [ % obj => -
              ExecStack /ExecSP cvx /get load
              /ExecSP dup cvx 1 /sub load /store load
            ] cvx def
    
            /TraceStep {
              iexec-step
            } def
    
            PushExec
    
            { ExecSP 0 lt { nullproc exit } if % nothing left to execute? goodbye.
    
              ExecStack 0 ExecSP 1 add getinterval
              TraceStep pop
    
              % pop top of exec stack onto the operand stack
              PopExec
    
              % is it executable? (else just push literal)
              dup xcheck { % obj
                % do we know how to execute it?
                dup type
                //iexec-types 1 index known { % obj type
                  //iexec-types exch get exec % ...
                } { % obj type
                  % some random type. just push it.
                  pop % obj
                } ifelse
              } if % else: obj
    
            } loop % goodbye-proc
    
            currentdict /MumbleFrotz undef % Clean up circular reference
          end
          exec % whoever exited the above loop left a goodbye proc on the stack.
        } def
    
    

It also has a "vexec" function that executes arbitrary PostScript code, and
prints out another text PostScript program to draw an animated trace diagram
of the operand stack depth -vs- execution stack depth. That's another simpler
kind of macro that produces text instead of structures. So I was using the
metacircular PostScript interpreter to visualize one structural PostScript
program's execution, by producing and executing the resulting text PostScript
program in another PostScript interpreter!

I don't currently have the PS output or rendered images decoded and online,
but here's a readme:

[https://donhopkins.com/home/archive/psiber/cyber/twist.readm...](https://donhopkins.com/home/archive/psiber/cyber/twist.readme)

This is a plot of the execution stack (x-axis) and the operand stack (y-axis)
during the execution of the PostScript /quicksort routine. Each successive
picture is more twisted than the last. Twisting is accomplished by rotating
the coordinate system clockwise slightly around the center of each of the dots
as they are drawn. The rotation around each plotted point accumulates to make
the whole drawing curl up. The more twisted away from the original orientation
a point is, the later it occurred in time. In the first picture, the untwisted
version, up corresponds to a deeper operand stack (pushing things on the stack
moves you up), and right corresponds to a deeper execution stack (procedure
calls and control structures move you right). The lines follow changes in the
state of the stack between steps of the interpreter. (This was made possible
by a PostScript interpreter written in PostScript.)

To see the twist animation, run monochrome NeWS, type "psh" to the shell, then
type "(/wherever/you/put/it/twist.ps)run". The reason you can't just psh the
file directly is that NeWS 1.1 psh does $1 $2 $3 arg substitution, even on
binary data! (X11/NeWS psh should work, so you can just go "psh twist.ps")

What the file twist.ps contains is a short header defining the function "c",
which reads a canvas and displayed it on the framebuffer. That is followed by
a series of "c"'s each followed by a 1 bit deep 1152x900 sun raster files.

-Don Hopkins (don@brillig.umd.edu)

------
vanschelven
somewhat off-topic, but as both the author and original poster of this
article, it's interesting to note that the article appears here as though it
was posted 3 hours ago, even though I actually posted it 23 hours ago (when it
gathered no attention).

~~~
dang
That's because it got put in the second-chance pool - see
[https://news.ycombinator.com/item?id=11662380](https://news.ycombinator.com/item?id=11662380)
and the links back from there.

~~~
SV_BubbleTime
Dan,

Emailed, still shadow banned, should I email again?

~~~
dang
I wouldn't. Each time you do you put yourself at the top of the inbox, and we
work through the inbox from the bottom up—this seems fairest, plus is a
natural form of comeuppance for the most demanding emailers.

Please don't post off topic things like this to HN. I'm sorry it takes a long
time to answer emails sometimes, but those are the constraints we're under.
Believe me, I don't like it either, especially on nights when I'm up late
answering them. You're not being treated worse than others, and it's a tad
off-putting to be harangued.

