
Programming Bottom-Up (1993) - ibobev
http://www.paulgraham.com/progbot.html
======
jasim
"Like the border between two warring states, the boundary between language and
program is drawn and redrawn, until eventually it comes to rest along the
mountains and rivers, the natural frontiers of your problem".

I wish pg programmed even today and kept writing about it.

------
mannykannot
So how do you know that your low-level construct is on the path to where you
want to be?

Quite often, you can see immediately that you need to represent X, that
representing X requires foo() and bar(), and you can confidently make these
things with a reasonable expectation of their usefulness. This already
depends, however, on having done some sort of preliminary high-level design
thinking. How much you need to do to be confident that you can identify an
efficient set of primitives depends on what you are trying to do and how
familiar you are with its problems. Writing primitives and utilities can
simply be a way to avoid tackling the difficult bits.

For a cautionary tale providing a counterpoint to this article, there is this
somewhat well-known case involving writing a Sudoku solver:

[http://ravimohan.blogspot.in/2007/04/learning-from-sudoku-
so...](http://ravimohan.blogspot.in/2007/04/learning-from-sudoku-solvers.html)

~~~
montrose
I find it's a valuable heuristic simply to try to make your code short (in
tokens, of course, not characters).

That rule means that it's worth defining a new operator if you use it enough
to save more than the amount of code you need to define it.

There are exceptions, of course. I have a higher standard for macros. But
overall the principle of optimizing for brevity has held up through decades of
programming of various types.

~~~
tom_mellior
I disagree. I recently read through Real World Haskell
([http://book.realworldhaskell.org/read/](http://book.realworldhaskell.org/read/)
though it seems down right now?) as a refresher, and it devotes a _lot_ of
time to examples of the form "here is a three-line function, but aargh, it is
_too long_ , let's turn it into a _two_ -line function by peppering it with
<$> and <*> and <@!^!@> operators!". The savings in code size is small, but in
every single case I found their "shorter", "simpler" program harder to read.

~~~
tome
I suspect that's a pedagogical device to motivate introducing a style that can
make code shorter and more readable, not a prescription to always convert
three lines to two by using sigils.

~~~
tom_mellior
Sure, but IMHO it tends to fail even on the "more readable" front.

~~~
tome
Yes, for converting three lines to two, but often not when converting thirty
lines to twenty.

------
alexashka
It seems like what Paul is describing, is the niche filled by
libraries/frameworks.

When you have a website to build, you reach for an existing web framework,
because it has already done the bottom-up steps for you.

The same could be said of choosing a language, OS, hardware, etc.

Existing technologies allow for increased productivity, at the cost of it
becoming increasingly difficult to understand the bigger picture. The 'bottom-
up' part at this point, is choosing which Ruby/Java/Python/etc libraries to
use. The productivity bar has been set high, leaving little time for bottom-up
decision making or education. Hence MongoDB and the never-ending tales of
mediocre solutions gaining adoption due to marketing hype.

Curiously enough - academia isn't able to solve this problem well, because
they're encouraged to produce 'novelty', rather than sit down and digest the
mess we're in, to help clarify matters a little. They're in the same rat-race,
of getting things done on a 6-12 month schedule.

There's a quote that I really like - "Prolific programmers contribute to
certain disaster." [0]

Ah well :)

[0]
[https://youtu.be/-I_jE0l7sYQ?t=21m55s](https://youtu.be/-I_jE0l7sYQ?t=21m55s)

~~~
forkerenok
> When you have a website to build, you reach for an existing web framework,
> because it has already done the bottom-up steps for you.

As I see it, it is from bottom to a greatest common denominator though. The
tools you choose are not likely to have everything up to your "up", you still
have to build your domain specific (but still abstract) "primitives".

~~~
wvenable
Almost everyone should build their own framework on top of existing frameworks
for their own domain. At work, we have a common look and feel for all our
applications so that's build into the framework. We have specific domain
specific data types -- those would never go into someone else's generic
framework but they are in ours.

I find this idea of building your own framework to be pretty rare -- I do it
because I've actually had to build frameworks for platforms that didn't have
them -- but most programmers are content to use what they're given but never
think of extending it.

------
jasode
Reading some of the replies that interpret pg's essay as talking about _"
libraries"_ or _" frameworks"_ or _" metaprogramming"_ doesn't look quite
right to me.

I argue that instead of those 3, the most applicable terminology to map to
pg's idea would be a "DSL" ("Domain Specific Language".)[1] The "DSL"
terminology seemed to have gained currency around 2008 so this old 1993 essay
wasn't able to use it. In any case, his point is to program Lisp in such a way
as to build a "DSL" inside of Lisp and you then program your higher level
concepts on that DSL. His key quotes:

 _> a principle which could be called bottom-up design-- changing the language
to suit the problem. In Lisp, you don't just write your program down toward
the language, you also build the language up toward your program._

 _> Instead of a single, monolithic program, you will get a larger language
with more abstract operators, and a smaller program written in it. _

One can also pursue the "DSL" concept with other programming languages. In
C++, you can design the classes to be a "DSL". In fact, one of the derided
features of C++ such as _operator-overloading_ was motivated by allowing a
custom "DSL" to integrate into the language better.

Lisp is an interesting programming language for writing an embedded DSL
because the custom "mini language" you write _looks like other builtin Lisp
syntax._

Why the other proposed terms don't seem to fit:

\- _" metaprogramming"_ : this is usually describing programs generating other
programs either at compile time (e.g. templates) or runtime (e.g. the eval()
function in Javascript/Python)

\- _" libraries"_ : these are often non-domain-specific reusable code... such
as a string library, matrix math library, or a TCPIP library

\- _" framework"_ : this is usually a source-code runtime (basically an
application "loop") with customization points to call your custom code

PG's essay like many other programming essays suffers from misinterpretation
because _it doesn 't include any explicit source code_ to illustrate the
ideas. Therefore, writing in metaphors invites misunderstanding.

[1] [https://en.wikipedia.org/wiki/Domain-
specific_language](https://en.wikipedia.org/wiki/Domain-specific_language)

~~~
lispm
> currency around 2008 so this old 1993 essay wasn't able to use it

Embedding languages in Lisp is a concept since around 1962 (Daniel Bobrows
METEOR Lisp extension for pattern based string processing). There are a bunch
of Lisp articles written before 1980 which describe it and advocate using it.

> Lisp is an interesting programming language for writing an embedded DSL
> because the custom "mini language" you write looks like other builtin Lisp
> syntax.

Not really. Lisp is useful for DSL development because it provides the
necessary extension features: which allows one to reuse a lot of the Lisp
machinery (reader/parser, interpreter, compiler, macros, object-system, memory
management, ...).

Common Lisp already comes with DSLs as part of the standard: format strings
and LOOP are examples. They actually don't look like Lisp syntax - format
strings are radically different and LOOP statements use more algol like
syntax.

example for a FORMAT string:

    
    
        CL-USER 2 > (format t
                            "~{~#[~;~a~;~a and ~a~:;~@{~a~#[~;, and ~:;, ~]~}~]~}"
                            (list 1 2 3 4 5))
        1, 2, 3, 4, and 5
        NIL
    

example for a LOOP form:

    
    
        CL-USER 3 > (loop for i in '(1 324 2345 323 2 4 235 252)
                          when (oddp i)
                            do (print i)
                            and collect i into odd-numbers
                            and do (terpri)
                          else
                            collect i into even-numbers
                          finally
                            (return (values odd-numbers even-numbers)))
    
        1 
    
        2345 
    
        323 
    
        235 
        (1 2345 323 235)
        (324 2 4 252)
    
    

> "metaprogramming" : this is usually describing programs generating other
> programs either at compile time

metaprogramming means to be able to program on the language level. It has not
much to do with program generation. For example in Lisp you can program new
syntactic constructs via macros. One can also program the 'reader' which
allows one to program the token reader. There are various other ways to
program at the language level - the most complex is the Meta-Object Protocol
of CLOS, the Common Lisp Object System - which allows one to program the
object system system itself in itself.

metaprogramming facilities are tools to develop DSLs.

~~~
jasode
_> Embedding languages in Lisp is a concept since around 1962 _

You misunderstood what I wrote. I was talking about the _terminology_ PG used,
not the concept. PG couldn't time-travel to the future and write "DSL" in his
1993 essay so more readers would instantly understand what he was talking
about.

 _> it provides the necessary extension features: which allows one to reuse a
lot of the Lisp machinery_

I agree. The "looks built in" is a characterization that some Lisp programmers
(maybe not you) would call the ability to add "new syntax" to Lisp. It's a way
of contrasting with something like C++. You can't _" extend the C++ language"_
with new syntax for your custom project unless you fork the GCC or Clang
compiler and add new keywords to the parser. All new so-called capabilities to
embed a "DSL" has to be done as "functions()" with obvious parentheses to call
it. Lisp ("code is data!") doesn't have this hard boundary.

 _> metaprogramming means to be able to program on the language level._

I agree. There's parsing/manipulating its own AST as a data structure but
"metaprogramming" also has lot of currency usage to describe programs writing
programs and the usage of dynamic eval().

~~~
lispm
> I was talking about the terminology PG used

What now is called 'embedded domain specific language' was called 'embedded
language' in Lisp. The name is slightly shorter, but the concept is/was the
same.

~~~
jasode
_> What now is called 'embedded domain specific language' was called 'embedded
language' in Lisp._

Exactly! If you do "Ctrl+F" search on PG's essay, you'll see it does _not_
have the phrase _" embedded language"_ nor _" embedded domain specific
language"_.

What exactly are you nitpicking?

~~~
lispm
That he could have used the term 'embedded language', which was used in Lisp
decades before he wrote his article. So the article is probably not about
embedded DSLs.

What he describes is actually extending the language to express the domain -
but that does still not mean that the language is 'specific'. It's not
necessarily even an embedded language.

~~~
jasode
_> What he describes is actually extending the language to express the domain
- but that does still not mean that the language is 'specific'._

This appears to be a subjective labeling because "extending the language" is
described by some other Lisp programmers as creating a "DSL". Some examples
out in the wold:

 _> "Lisp is different, it is a shape shifting creature. It can be made to
adapt to any environment [...], one can build a DSL (domain specific language)
within Lisp to cater to that domain."_ From: [https://www.quora.com/According-
to-Paul-Graham-in-2002-langu...](https://www.quora.com/According-to-Paul-
Graham-in-2002-languages-can-be-ranked-in-power-order-and-Lisp-is-the-first-
on-that-ranking-Is-this-still-accurate/answer/David-Lin-211)

> _" Extending the syntax the way you're talking about allows you to create
> domain specific languages."_ From:
> [https://softwareengineering.stackexchange.com/a/164674](https://softwareengineering.stackexchange.com/a/164674)

 _> ", we can extend it in Lisp itself [...] We've just created a very small
and limited language for managing to-do lists embedded in Lisp. Such languages
are very specific to a particular problem domain and are often referred to as
domain specific languages or DSLs._" From:
[http://www.defmacro.org/ramblings/lisp.html](http://www.defmacro.org/ramblings/lisp.html)

What PG is describing may not be a complete and formal DSL according to your
definition but others are describing his technique _today_ as writing a
<scarequote>DSL</scarequote>.

~~~
lispm
> "extending the language" is described by some other Lisp programmers as
> creating a "DSL"

Lisp can be extended in many ways without creating a DSL. That's the basic
error you make. Paul Graham talks about extending/changing Lisp and you assume
that this must be a DSL - while it is just a larger Lisp.

Typically bottom-up programming does not mean developing a DSL. Though it can
be that one develops one or more DSLs during bottom-up programming. But it's
not necessarily so.

Remember DSL means 'Domain specific language' \- a language (that means
syntax/semantics/pragmatics) specific (means offering a set of operations and
data structures) to a domain (which has a problem which needs to be solved).

A larger Lisp is neither specific to a domain, nor a new language.

To give you an example: For some engineering problem (process control of
chemical plant) one needs to express fuzzy values and needs to compute with
them. PG advocates to implement these fuzzy data structures and operators. One
then constructs further parts using these new building blocks. This could be a
bunch of classes and functions. Just an extension of the existing Lisp - it
would not only work with boolean values, but also with fuzzy values. While
using the fuzzy operators, one extends them until all needed operators are
available.

With the new set of operators large parts of the solution can be written in a
shorter way. One can do such bottom-up programming in many languages and now
also in many popular languages. One thing that made Lisp attractive early on
for this style is the interactive development mode. One creates a new building
block and one can immediately try it out - improving it and testing it again
was always very fast. Thus one can extend the running program piece by piece
and play around with the code in combination with domain data. Something which
would not be typical in C, C++ and Java. Still, if one wants, one can do
bottom-up programming in those languages too.

The result is neither specific to a domain (because all the other
functionality is still there), nor would be a 'language' \- it's just more
vocabulary in an existing language - probably not even new syntax. One could
also use the extension points of the language. For example Lisp would use T
and NIL as constants for boolean values. One could extend the s-expression
reader to add a new syntax for fuzzy values. That would not make a new
specific language, but just extend the existing language along the prepared
extension points.

~~~
jasode
_> Lisp can be extended in many ways without creating a DSL. That's the basic
error you make. Paul Graham talks about extending/changing Lisp and you assume
that this must be a DSL - while it is just a larger Lisp._

You completely misunderstand my point. We're talking past each other. I
absolutely do not assume PG's technique _must be a DSL_. I tried to be very
careful in my wording in the very first comment to _avoid_ claiming such a
strong conclusion. Let me try again to clear up the confusion.

My perspective is about _communication_. A lot of people out there already
have "DSL" in their head to describe what PG is doing. It's irrelevant whether
it is right or wrong. It's the _closest_ mapping people have. Others in this
thread tried mapping it to "libraries" or "framework". I argue that "DSL" is
much closer to what PG is describing to set off the "aha!" lightbulb moment in
readers' minds.

Your perspective is about "DSL" being a _formalized definition_ that PG's
technique doesn't match and therefore it's not a DSL. _I totally understand
your point but it 's not relevant to what I was explaining._

Yes, a "larger Lisp" may not be a DSL in your view but that's what many
programmers are calling it when they extend Lisp to more closely match a
domain. When PG writes _" changing the language to suit the problem"_, many
will naturally rewrite that in their head as _" changing the language to suit
the domain"_, and adding an operator is therefore _" extending the language to
more suit the domain"_. That's how people out in the wild are describing it
even though there's no full-blown embedded Domain Specific Language there with
formal semantics and grammar/syntax.

Tldr: you're taking "DSL" very literally instead of observing how programmers
are using that phrase in the real world.

~~~
kazinator
Whether DSL or not, Paul Graham does emphasize the use of macros, in more than
one of of his essays. Not just the stock of macros that you get from an
implementation, but the program's own. For instance, I read something of his
some fifteen years ago where he claimed that it was typical for a Lisp program
to be about 25% macros.

lispm is right that a bunch of macros don't necessarily make a domain-specific
language. If I use PG's anaphoric if macro, or "with-gensyms", I'm not in a
different domain; I just have some shorthand for what I'm already doing.

~~~
jasode
_> I'm not in a different domain;_

The "domain" doesn't have to necessarily mean different disciplines such as
"game programming" is one domain and "online shopping" is another domain.

Thanks to montrose's reminder, I cite PG's _" On Lisp"_ page 5 (which appears
immediately after the _" Programming Bottom-Up"_ section):

 _> We transform our software from a mere program into a _programming
language_, and advanced users can build upon it the extra features that they
need. [...] The simplest bottom-up programs consist of two layers: language
and program. Complex programs may be written as a series of layers, each one
acting as a programming language for the one above. If this philosophy is
carried all the way up to the topmost layer, that layer becomes a programming
language for the user._

Also in section "1.4 Extending Lisp" page 7:

 _> Lisp is an excellent language for writing compilers and interpreters, but
it offers another way of defining a new language which is often more elegant
and certainly much less work: to define the new language as a modification of
Lisp. Then the parts of Lisp which can appear unchanged in the new language
(e.g. arithmetic or I/O) can be used as is, and you only have to implement the
parts which are different (e.g. control structure). A language implemented in
this way is called an _embedded language_. Embedded languages are a natural
outgrowth of bottom-up programming._

Yes, the 1993 time period predates today's "DSL" meme but now that "DSL" has
permeated into widespread programmers' awareness, is there a better label to
describe those PG paragraphs? I argue that today, people would rewrite PGs
technique in their mind as "layering DSLs".

------
superlopuh
I feel that the C ABI is local maximum, with a massive associated opportunity
cost. Imagine if all the operators that have been written, "bottom up", in one
language could be easily accessible from others. And the only way to leverage
that today is to go through the crusty, unsafe API that no-one really wants to
export to unless it's required in the spec of a project.

Swift soon being able to call python is a massive step forward, but a lower
level API that wouldn't require a runtime yet still was superior to C's feels
like a massive win for the industry that's just waiting to happen.

~~~
jasode
_> , but a lower level API that wouldn't require a runtime yet still was
superior to C's feels like a massive win for the industry that's just waiting
to happen._

A lower-level binary API was tried before in the 1990s by various players.
Microsoft's effort was COM/COM+ to unify programming languages across binary
boundaries.[1] Another initiative was CORBA.[2]

Those technologies are still alive in various legacy forms but they haven't
taken over the landscape like the proponents thought they would. (Although the
whole MS .NET ecosystem introduced in 2002 is arguably a continuation of COM+
by a different name.)

[1]
[https://en.wikipedia.org/wiki/Component_Object_Model](https://en.wikipedia.org/wiki/Component_Object_Model)

[2]
[https://en.wikipedia.org/wiki/Common_Object_Request_Broker_A...](https://en.wikipedia.org/wiki/Common_Object_Request_Broker_Architecture)

~~~
adrianratnapala
I see what you mean that COM and Corba tried to unify programming across
language boundaries. But they were not a low-level ABI type thing. They
explicitly pushed a particular idea of what objects are and what they are for.

In any case, judged by the standard of becoming a universal ABI, they were
also designed to "fail" in the sense that MS wasn't going too sit around and
standardise it across operating systems, while the CORBA was a (feeble)
response from the Unix world that no one expected would take off on Windows.

------
brlewis
I don't think bottom-up is a good way to describe the capabilities Lisp gives
you.

You can go totally top-down in your design, which is the pattern I was
encouraged to use by SICP ("wishful thinking" = top down). But Lisp doesn't
make you stop when you reach the language level. "I wish my language would
give me a concise way of saying whether variable x has the same value it had
in the last iteration of this loop" is not a wish you can fulfill in a
language without macros, for example.

~~~
kazinator
Bottom up is also C and Unix:

 _It is the way I think. I am a very bottom-up thinker. If you give me the
right kind of Tinker Toys, I can imagine the building. I can sit there and see
primitives and recognize their power to build structures a half mile high, if
only I had just one more to make it functionally complete. I can see those
kinds of things. The converse is true, too, I think. I can’t— from the
building—imagine the Tinker Toys. When I see a top-down description of a
system or language that has infinite libraries described by layers and layers,
all I just see is a morass. I can’t get a feel for it. I can’t understand how
the pieces fit; I can’t understand something presented to me that’s very
complex. Maybe I do what I do because if I built anything more complicated, I
couldn’t understand it. I really must break it down into little pieces._ \-
Ken Thompson, _Unix and Beyond: An Interview with Ken Thompson_ , 1999

------
tabtab
I've found a similar process by sketching up the domain logic, or typical
steps of it, in as high level of pseudo-code as I can. Play around with it to
try to optimize (factor) expressiveness. One then starts to see commonalities,
and thus a domain-specific language(s) emerges. This language(s) may resemble
or even become an API(s).

From a different direction, form a relational model, even if not using an
RDBMS, to clarify the relationships between the domain parts.

Once you do both of these, then a bigger picture starts to form, and you
refine it to have the API's work properly from the data relationship
perspective.

------
sriku
On the practical front, this approach is tenable when programming for yourself
or by yourself. How do you articulate this if you are consulting, your client
isn't tuned to this thinking and want you to describe architectures and
timelines? A statement like "I'll be working to create some primitives and
based on what I see I'll choose to build or modify structures when
appropriate." \- doesn't fly. Though we might do that at the end of the day,
I'm curious to hear here about what HNers think would be some good ways to
talk about this process .. or not.

~~~
sriku
Btw Peter Nair's "programming as theory building" is the best description of
this approach I know of.

~~~
m1n1
Naur

------
studentik
> Few would dispute, at least, that high level languages are more powerful
> than machine language.

I would! =) How is power of language defined?

Level of abstractions, high-level vs low-level? Attractive language syntax
(DSL/macro/syntax sugar)? Tools, support and libraries? Community traction,
popular, paid-more? Optimized and effective code? Low entry-barrier?

Depending on definition on what powerful language is, winners might become
losers and vice versa.

~~~
kluck
This.

Be a highlevel programmer: know enough languages to pick the right one for the
task at hand. Do not take part in language wars.

------
jaequery
bottom up programming is actually what a lot of jr developers do when starting
out their careers and i would advise against it.

a top-down approach would be clean and simple and the complexities are hidden
away from interface:

User.create(data)

whereas, a bottom-up would be look like you just hacked things on top of each
other:

create_user( validate_user_data(format_user_data(data) ) )

but, i do know lisp kinda forces you to code that way, so pg's article may
have a lot to do with that.

~~~
quikoa
Interesting, I got the exact opposite experience with junior developers. They
often try to design a system top-down. Fail to do so well enough because they
don't have enough experience from the bottom up.

~~~
sriku
Concur. Same experience. And I did the same mistakes when I was one of them
junior devs too :)

~~~
quikoa
Yes me too and so many times!

------
kluck
It seems programming bottoms up is the only way to go - consider coding with
your bottoms down, quite embarrassing.

------
ZainRiz
His bottom up design sounds like the libraries and frameworks of today.

Need to sort a list? There's a sorting function in this library. Need to host
a website? There's a framework over there for that.

The main takeaway seems to be the need to create utility level libraries for
your own work that is specialized to your needs.

------
TimmyMustGo
I believe the "bottom part" should be interchangeable, like switching to a
different CPU architecture, or switching to a different framework. The bottom
part is an implementation detail. The API is important since that's what you
interact with.

I think it's best to have as few dependencies as possible, try to keep
everything independent and interchangeable. I like to view my code as a graph
structure, try to keep the functions and its dependent functions as a
Directed-Acyclic-Graph and avoid cycles.

~~~
jlg23
Beware, he was using the term "bottom up" in a meaning that diverges from
current mainstream usage of the term: the bottom is the _language_ , not the
architecture.

