
SICP Goodness – Why you don't need looping constructs - lvguowei
https://www.lvguowei.me/post/sicp-goodness-looping/
======
sametmax
Actually you don't need another contruct than goto. You don't even need
functions.

But for some reason given the choice, people use them.

It makes things easier to reason about for most day to day problems.

I train people in programming for a leaving. "Map" is harder to understand
than "for" to most of my students. "Reduce" harder than manual looping.
Recursion harder than iteration. Immutable harder than mutable.

Now I always end up teaching the entire toolbox. Those are all very useful
concepts that I use in production and I do want my students to have a large
solution set to their future problems.

But there is a clear pattern in my teaching experiences.

~~~
CodeArtisan
>Actually you don't need another contruct than goto. You don't even need
functions.

>But for some reason given the choice, people use them.

There is no choice when it come to functional programming because the paradigm
is based on Alonzo Church model of computation known as Lambda calculus which
is declarative: Everything is an expression or, more specifically, a function.
Because of that, statement like goto are forbidden. Goto is from the Alan
Turing model of computation known as the Turing machine which is imperative;
everything is a statement.

Guy Steele said that the lambda expression in Scheme is the ultimate GOTO
because it's like an unconditional jump while remaining declarative and
functional.

------
YeGoblynQueenne
Beware the conclusions drawn from the first read through the writings of our
functional programmer forefathers (and the second, and the third), for "a
little knowledge is a dangerous thing". The unprepared mind of the imperative
programmer is especially prone to following the revelations of the old masters
before achieving real understanding.

A while ago, I was working in company X, that sold a web platform for job
boards, with multiple big and famous clients. One time, there was a bug in our
platform, that had to do with a for-loop with an always-true condition. The
loop -which was in our core platform code and so deployed in every single
website we maintained- caused untold havok, as resources disappeard down a
black hole and all our servers dropped to their knees and asked for mercy.

In the aftermath of the incident, when an explanation had to be given to those
UpStairs running the show, it was put to them that the cause was an
(effectively) unconditional loop. Now, as a lowly junior dev, I was not
present in the deliberations but, given what followed, I can imagine the
conversation:

    
    
      Those UpStairs: "How can we ensure that the same thing never happens again?"
    
      Senior Software Person: "Well, there is no way to ever be 100% safe. A for-
      or while-loop can always go wrong."
    
      Those UpStairs: "Is there any alternative to for- and while-loops?"
    
      Senior Software Person: "Not really. I mean, there is recursion of course..."
    
      Those UpStairs: "Recursion? What is that?"
    

Shortly after, we had a big emergency meeting to discuss What Should be Done
to Avoid the Same Problems in the Future, where we were instructed to not use
the looping constructs of the langauge in which our platform was written,
anymore. Instead, we should only ever use recursion to write loops.

The language of our platform was C#, mind. A language that (to my knowledge,
and certainly at the time, around 4.0) does not even support tail-call
optimisation. Not to mention, it's just as easy to go into an infinite
recursive loop as it is to go into an infinite imperative loop, if not easier.

Suffice to say that nobody followed the instructions given from On High,
either because they found it too much work, or because they found it too
stupid, or because they were just busy pulling their hair out.

~~~
discreteevent
It's just as easy (if not easier) to make a mistake with a recursive call such
that it never terminates. Why did your team suggest it as an alternative and
why was it accepted? TBH the whole story seems very hard to believe.

~~~
maaark
Imagine "Those UpStairs" is angry, panicked and shouty. Imagine "Senior
Software Person" is put-upon and long past arguing with "Those UpStairs".

I can see it.

~~~
a3n
Been there, had that rammed up good and hard. I can see it.

------
bjoli
I would say that schemes looping constructs are too limited for comfortable
use, but that tail call elimination makes implementing your own constructs
very easy. I have reimplemented racket's for loops for guile using named let
loops [0] which was not only pretty easy, but it was also easily understood
and optimised by the guile compiler. The end result is as fast as a hand
rolled named let for almost all cases (I still haven't found one where it is
measurably slower, but I have only tested it in the repl where the compiler
can make lots of assumptions)

For that reason alone I think tail call elimination is a must, especially in
today's transpiling world.

[0]: [https://bitbucket.org/bjoli/guile-for-
loops](https://bitbucket.org/bjoli/guile-for-loops)

~~~
kccqzy
ES6 used to mandate TCE but it was removed. No browser other than Safari
implements TCE for JavaScript. Apparently the main reason is an inaccurate
stack trace on exception.

~~~
hajile
The latest spec still talks about tail call optimization. The big objection
was to replace implicit tail calls with a syntax to specify functions that
might tail call.

With TCO being part of the spec and the potential replacement spec killed off,
browsers really need to start getting with the program.

JavascriptCore (safari), Node v6, duktape, jerryscript, XS6, etc all have
proper tail calls. It breaks the community to not implement them. It's not a
standard if its not standard.

~~~
soegaard
Explicit or implicit doesn't matter for compiling to JavaScript. Explicit tail
calls are much better that no tail call support at all.

~~~
hajile
I agree, but the companies behind explicit calls dropped it, but are refusing
to implement the current spec too. Their stupid politics endanger the js
ecosystem.

------
goalieca
I made it a goal of mine about 10 years ago to avoid explicit for-loops. It
turns out using declarative style makes things more readable and safer and
better optimized by the compiler. I can usually tell the purpose of the loop
by just reading the function name (map, filter, ..) and the lambda. Sometimes
parallel comes for free. Plus, all those off by one errors, iterator errors
from erasing elements, etc. are all eliminated :)

~~~
jstimpfle
> It turns out using declarative style makes things more readable and safer
> and better optimized by the compiler.

I don't think most of my loops are straightforward maps or filters of a single
iterable. I need to reference various data sources and execute actions with
side-effects. To do that using a functional framework, something like
Haskell's mapM ("monadic map") is required. But that brings quite some mental
overhead and I find it way too constraining for the typical case.

> more readable and safer and better optimized by the compiler.

I doubt these are in general qualities of non-builtin constructs vs builtin
ones. But yes, too much builtin and non-orthogonal functionality leads to
detrimental complexity.

In any case I don't think it matters much to most compilers whether you write
a loop using syntactic sugar or as labels and jumps.

~~~
goalieca
I haven’t really encountered a loop that there isn’t a function for. There
ought to be some exhaustive list of all the different kinds of loops in
computer science textbook. But Most of what you need is in Zip, fold, map,
reduce, filter, find, sort, reverse, transpose, permute, concat, any, all,
max, min, cycle, take, ..

I’ve periodically done microbenchmarks of inner loops over the years and the
various gcc outputs of the c++ loop functions are always performing slightly
better than if I hand rolled. Perhaps with gcc7 or clang this is different.

~~~
jstimpfle
> Zip, fold, map, reduce, filter, sort, reverse, transpose, permute, concat,
> any, all, max, min, cycle, take, ..

Cool, so if that's all we should just use those. But wait, what about those
dozens of megabytes of data, should we really zip/transpose/concat them? We
should use non-strict evaluation then...

~~~
tuukkah
You can use lazy lists / streams even with strict evaluation, you just have to
be explicit.

~~~
jstimpfle
Point being, I'd rather not have this complexity. Do you mean to explicitly
take each next value from the stream? And then possibly throwing all this
"lazy" work away? Or memoizing the work and writing extra code that checks if
the work was already done when the data is accessed? And then probably
forgetting to get rid of the data when it's no longer needed?

No thanks, that's even worse. It's honest, but that doesn't make it a good
idea. I like to do one thing at a time, not jump around like a wild horse.
Turns out optimizing for not jumping around too much also makes programs
simpler overall.

I've recently cleaned up some code of this kind. Defines objects with
unsuitable lifetime and tries to be smart about reusing work instead of
planning out in which order to do things in the first place. While the actual
task was not difficult, the code was incomprehensible, to state it in a
diplomatic way.

~~~
xfer
Not sure where you get all this idea of complexity from, lazy streams are used
in functional languages a lot for data processing and they are not
incomprehensible, also imperative languages like rust has iterators that are
more used than for/while loops.

~~~
jstimpfle
Well, I described the situations from which I get my ideas of complexity...
Regarding iterators, I'm fine with their usage depending on what they _do_.
However I don't think it's a good idea to iterate over an abstract stream that
under the hood does sorts and zips and folds. Because you lose control and
understanding how much intermediate data will be produced (constant small
amount? 3 times the amount?), and in what order code gets executed.

And if you think consciously about the intermediate data you probably can use
it for a whole lot of other things!

In Haskell, unexpected memory expansions and duplication of work can happen.
And these problems are not nice to debug.

I might be generalizing too much, though. Maybe you're talking about short-
lived web applications that never see a large amount of data, or something
like that.

------
pjc50
However, if iteration and tail-recursion are equally powerful constructs, the
implication that recursion is morally superior to iteration seems misplaced.

~~~
noelwelsh
You can use tail recursion to create abstractions in a compositional manner
that cannot be created compositionally with iterative constructs.

For example, if you can express a finite state machine (FSM) as a number of
mutually (tail) recursive functions. Then you can reuse this FSM as part of a
larger FSM via a tail call. You can't do this with the iterative constructs in
C et al---the whole FSM must be defined in one spot.

~~~
burfog
So do the tail recursion in C.

Any decent compiler will handle it just fine. If you want to include crappy
compilers, well, then we need to include the same for LISP and Scheme.
Whatever the language specification may claim about tail recursion is not
relevant when you sit down to produce code for the actual system at hand.

~~~
kazinator
A Scheme compiler that doesn't do tail recursion isn't crappy; it is
incorrect/nonconforming. A C compiler that doesn't do tail recursion still
conforms to the abstract language semantics.

In Scheme, local functions are used for looping (via tail recursion). C
doesn't even have those (other than as a GNU C extension).

------
Const-me
I prefer looping constructs over recursion, almost every single time.

Loops are closer to what CPU actually does. A compiler is therefore simpler to
implement and will likely contain less surprises.

It’s easier to use a debugger on procedural-style or OO code (step
in/over/out, local variables) compared to recursive functional style
advertised by SICP.

In most runtimes, call stack is limited to a small size, couple of megabytes.
Stack overflow exception is hard to handle in reasonable way. When instead
using a loop with a stack container for state, it can be much larger, also
easier to handle overflows i.e. implement soft and hard limits.

~~~
willtim
Explicit recursion is really a foundational primitive upon which other
abstractions should be built, including looping constructs. It is similar to
goto, in that it is a form of unstructured programming. Functional programmers
prefer using maps and folds.

Loops are not necessarily close to what a CPU actually does either, especially
one supporting vector operations or out-of-order execution. Loops imply an
ordering which may or may not be actually exist and many compilers _are_
complicated by their presence.

Debuggers are a problem for any suitably high-level abstract code. Would a
step-by-step word-at-a-time debugger help to debug a SQL query or even a
complex arithmetical expression from Fortran? There is no getting away from
the fact that high-level domain-specific code needs high-level domain-specific
debuggers to be built alongside.

~~~
Const-me
> functional programmers prefer using maps and folds.

I know and I don’t think these are always good things.

> Loops are not necessarily close to what a CPU actually does either

Indeed, not necessarily, but very often. Unless automatic vectorization is
involved, long loops are very often quite close to the machine code uploaded
to CPU. Inside the CPU it’s free to do whatever, but on the public API, it’s
still loop instructions.

> the fact that high-level domain-specific code needs high-level domain-
> specific debuggers to be built alongside.

That’s not a fact, that’s your opinion. I don’t code SQL nor Fortran, mostly
C++ and C#, sometimes Python and others. The debuggers I have in corresponding
IDEs are already built, they are sufficiently high-level for domain specific
code, but the debuggers themselves aren’t domain specific, they are pretty
general purpose.

~~~
willtim
> I know and I don’t think these are always good things.

Maps and folds do make structure explicit though. For example, it is very easy
for a compiler to vectorise a "map" operation. Folds will always terminate.

> The debuggers I have in corresponding IDEs are already built, they are
> sufficiently high-level for domain specific code.

But does your debugger allow you to visualise the intermediate steps of an
arithmetic expression? Does it understand Iterables/Enumerables and allow you
to capture and inspect them? Can it handle concurrency and parallelism, or
help you make sense of callbacks for non-blocking IO etc? My point is that the
debugger is simply not as general purpose as most claim it to be. It sees the
world word-at-a-time and only allows you to step statement-by-statement. It
does not understand the composite data structures or control structures that
you have built. This is less of a problem with low-level code, as typically
there is limited abstraction. But it will be a problem for any sufficiently
high-level abstract code (functional or otherwise).

~~~
Const-me
> it is very easy for a compiler to vectorise a "map" operation

For a general map function, it’s insanely hard problem. It’s only trivial if
the map function is trivially simple e.g. `return x*2`.

> does your debugger allow you to visualise the intermediate steps of an
> arithmetic expression?

No, and I write my code in a way so the expressions aren’t too complex. When
they grow to be that complex, I break them into multiple steps, keeping
intermediate results in separate variables. BTW my primary motivation is
readability, debugging is secondary.

> Does it understand Iterables/Enumerables and allow you to capture and
> inspect them?

I think so.

> Can it handle concurrency and parallelism, or help you make sense of
> callbacks for non-blocking IO etc?

Sure, debugging async-await code in .NET is almost as simple as old school
non-scalable blocking IO code.

> But it will be a problem for any sufficiently high-level abstract code

Not a huge problem in practice. There’re simple ways to implement support for
your custom composite data structures, e.g. natvis files in VC++. Also, in
.NET you can run any code in debugger’s immediate window, while execution is
paused. These simple methods aren’t effective in 100% cases, but often they
are sufficient to debug, therefore saving a lot of time because no domain-
specific debuggers are required.

~~~
willtim
Wow that's fantastic that Microsoft have extended the debugger to understand
streams and async/await code, this of course only works because they develop
both the abstractions and the debugger. It's probably not so easy for the rest
of us to extend it for our own abstractions.

~~~
Const-me
> It's probably not so easy for the rest of us to extend it for our own
> abstractions.

Depends on the abstraction. For idiomatic stuff it’s relatively easy to extend
the debugger, see e.g. [1] about writing custom debug visualizers for very
complex objects (for reasonably complex there’re much simpler ways).

For non-idiomatic stuff such as extending the language by modifying the
compiler, or e.g. post-processing the compiled output, I don’t know because I
never needed debugger support.

[https://docs.microsoft.com/en-
us/visualstudio/debugger/creat...](https://docs.microsoft.com/en-
us/visualstudio/debugger/create-custom-visualizers-of-data)

------
mattfrommars
Does anyone know a good time in one's career to take time out and spend it on
going through SICP and videos lectures with it?

I have a STEM (non-CS) degree but very recently, started working as a
programmer. Doing my first internship at a 'techy' corporate company. One of
the things I'd be doing on the side is to go through
[https://mitpress.mit.edu/books/elements-computing-
systems](https://mitpress.mit.edu/books/elements-computing-systems). The video
introduction to it just gives me goosebumps to this day. I don't know why.
It's fascinating.

Than there is other paradigm books such as "pragmatic programmer", "Thinking
Like A Programmer", "Design Patterns: Elements of Reusable Object-Oriented
Software", etc.

So far, I've done plenty of reading on Algorithm & Design Structure and
leetcode here and there. In the past, I didn't qualify for internship at
Google, Amazon or M$ as a software developer/engineer. I feel the reason why I
didn't was because I didn't have a "computer science" keyword mentioned
anywhere on my resume. Hence, my hope is after getting through "Element of
Computing System" & Design Pattern, I'll be able to shove in "computer
science" keyword one way or another on my resume.

~~~
paidleaf
> Does anyone know a good time in one's career to take time out and spend it
> on going through SICP and videos lectures with it?

I guess any time is good. It's generally a class you take as a freshman or
sophomore. People who attend top engineering schools generally played around
with these topics in high school or even in elementary school.

Also, it looks like MIT stopped teaching SICP.

[https://news.ycombinator.com/item?id=11628080](https://news.ycombinator.com/item?id=11628080)

> In the past, I didn't qualify for internship at Google, Amazon or M$ as a
> software developer/engineer. I feel the reason why I didn't was because I
> didn't have a "computer science" keyword mentioned anywhere on my resume.

Or you are competing against the best of the best CS students from around the
world. People who aren't blown away by introductory stuff like elements of
computing systems or who are just looking into SICP. I really don't think
missing a keyword is why you didn't get the internship at google, amazon or
microsoft.

> Hence, my hope is after getting through "Element of Computing System" &
> Design Pattern, I'll be able to shove in "computer science" keyword one way
> or another on my resume.

I've never had computer science on my resume though I majored in it. For my
internships, I just listed the courses/projects I worked on. For my first job,
I just listed the degrees I had ( BS and BA and the gpas for both ), along
with the projects I worked on and the internships I had. After that, I just
listed the university and degrees and my work experience. "Computer science"
isn't something I ever listed in any resume. But then again, I was never
really good at writing resumes.

I think you are really overthinking things but feel free to do what you want.
If you are so intent on putting computer science on your resume, why not just
major in computer science?

------
davidgrenier
I believe this is the place to ask a question I've been wondering about.
WebAssembly currently doesn't support Goto labels and several people are
pushing for it:
[https://github.com/WebAssembly/design/issues/796](https://github.com/WebAssembly/design/issues/796)

Now I'm aware that mutually recursive functions can be gracefully used to
implement state machines and that the issue of jumping into the middle of a
loop from a goto seems to be prized.

The former (mutually recursive function) seem to solve that problem, not
necessarily in almost a general senses, however perhaps preventing some of the
most nasty things Goto can do.

I'd like to know if mutually recursive function requires the underlying
machinery to provide gotos. In my view, that would fully justify them. If not,
then tail-recursion might be the only thing missing from WebAssembly.

~~~
simias
Why would you be worried about the "nasty things Goto can do" in an assembly
language? IMO assemblies are about exposing the underlying machine's
capabilities (be it physical or virtual) in the most straightforward way
possible. I don't expect an elegant abstraction, I leave that to the higher
level languages.

I don't know much about WebAssembly so I don't know if gotos have a place or
not but if they're useful and the only reason for not adding them is cargo-
culting "gotos are teh 3v1l" it's a bit silly.

~~~
willtim
I respectfully disagree that Web Assembly really "exposes the underlying
machine capability" in any meaningful sense. The situation we are in today, is
that many assembly ISAs are no longer fit for purpose. A modern C compiler
will spend a lot of work recovering data-flow dependencies (e.g. SSA form)
only to then remove them when outputting e.g. x86 assembly. The CPU then has
to recover much of this information again for out-of-order execution etc.

IMHO, the Web Assembly folks should focus on a simple, elegant core language
that is easy to define and easy for a JIT to optimise. The machine
architecture, whatever that may be now or in the future, should be secondary.
Personally, I would never have called it "Web Assembly" either.

~~~
simias
That sounds very reasonable to me but if my (very limited) understanding of
WebAssembly is correct then it's mainly meant as a target for compilers, not
something you'd routinely write yourself unless you wanted very tight control
over the code or maybe micro-optimizations, as such it doesn't make a lot of
sense to object to certain feature from a code maintainability/language
ergonomics point of view IMO.

I'm not saying anything about what WebAssembly should or shouldn't look like,
I haven't got the faintest idea about that, I was just criticizing the
argument of the parent about the "nastiness" of goto that felt out of place in
this context. If 99.99% of all wasm code is generated by compilers then it
should mostly aim at providing a target that lets compiler output efficient
code, not something that would be nice for humans to write or add arbitrary
limitations to prevent coders from writing "nasty" code.

~~~
willtim
> it's mainly meant as a target for compilers, not something you'd routinely
> write yourself

Yes absolutely. That's why they should not cave in to pressure to add
semantically redundant features just to make a particular project or
application slightly more convenient.

In order to meet the objectives of easy to define and easy to optimise, it
will be necessary to agree on a core set of foundational primitives and stick
to them.

------
User23
Loops are theoretically interesting and rich and with the right tools even the
most complex cases can be written clearly and reasoned about easily.
[https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/E...](https://www.cs.utexas.edu/users/EWD/transcriptions/EWD03xx/EWD316.4.html)
is a good example of how mathematical reasoning is just as applicable to
"imperative" programming as it is to "functional."
[https://en.wikipedia.org/wiki/Hoare_logic](https://en.wikipedia.org/wiki/Hoare_logic)
describes the syntax used.

------
madmulita
Luckily I had been exposed to recursion previously, what blew my mind while
watching the SICP videos was the implementation of car and cdr using only
lambda and the realization that the language could go without variables.

------
LfLxfxxLxfxx
Lisp only replaces looping with recursion, it doesn't eliminate it. APL does.

~~~
kazinator
Lisp idioms like (mapcar function list) replace both looping and recursion
with an aggregate operation, similar to APL; it's just not obsessively
condensed into a single character syntax.

------
erikb
The title should be "why you don't need looping constructs if your language
supports tail recursion".

------
tzahola
Technically you don’t need loops, that’s true.

But from a pragmatic standpoint, it’s just too easy for someone to turn a
tail-recursive function into non-tail recursive, causing a stack overflow
eventually.

So when working in a large team, with run-of-the-mill programmers, I’d stick
with loops.

~~~
chriswarbo
Would an annotation help, e.g. a `tailcall` keyword, which will trigger an
error for non-tailcalls?

~~~
tasuki
Fwiw, Scala does exactly that with its `@tailrec` annotation.

------
Double_a_92
Yes, you can use recursion to loop... What was this post trying to tell me?

