
Holding a Program in One's Head (2007) - dasmoth
http://paulgraham.com/head.html
======
segmondy
I'll leave this here

“I believe that the clue to his mind is to be found in his unusual powers of
continuous concentrated introspection. A case can be made out, as it also can
with Descartes, for regarding him as an accomplished experimentalist. Nothing
can be more charming than the tales of his mechanical contrivances when he was
a boy. There are his telescopes and his optical experiments, These were
essential accomplishments, part of his unequalled all-round technique, but
not, I am sure, his peculiar gift, especially amongst his contemporaries. His
peculiar gift was the power of holding continuously in his mind a purely
mental problem until he had seen straight through it. I fancy his pre-eminence
is due to his muscles of intuition being the strongest and most enduring with
which a man has ever been gifted. Anyone who has ever attempted pure
scientific or philosophical thought knows how one can hold a problem
momentarily in one's mind and apply all one's powers of concentration to
piercing through it, and how it will dissolve and escape and you find that
what you are surveying is a blank. I believe that Newton could hold a problem
in his mind for hours and days and weeks until it surrendered to him its
secret. Then being a supreme mathematical technician he could dress it up, how
you will, for purposes of exposition, but it was his intuition which was pre-
eminently extraordinary \- 'so happy in his conjectures', said De Morgan, 'as
to seem to know more than he could possibly have any means of proving'. The
proofs, for what they are worth, were, as I have said, dressed up afterwards -
they were not the instrument of discovery.”

― John Maynard Keynes

~~~
icc97
This reminds me of G. Polya's book "How to Solve It" [0], there he talks about
the process that teachers can go through to get students to think through
problems. But more than that it's trying to build in a habit, so that you
tackle problems in the same way each time and can train your brain pathways to
hold more of the process in your head.

[0]:
[https://www.math.utah.edu/~alfeld/math/polya.html](https://www.math.utah.edu/~alfeld/math/polya.html)

~~~
vram22
That's a very good book. I first read it while in college or high school,
IIRC. Should buy a copy again, it's worth it.

BTW, for others who might not know, it is not applicable just to math
(although Polya was a mathematician). It's good for general problem solving.

And there is a book that was inspired by it, called How to Solve it by
Computer. I had and read that one too. It is very good too. See descriptions
of both books below.

[https://en.wikipedia.org/wiki/How_to_Solve_It](https://en.wikipedia.org/wiki/How_to_Solve_It)

[https://en.wikipedia.org/wiki/How_to_Solve_It#Influence](https://en.wikipedia.org/wiki/How_to_Solve_It#Influence)

[https://en.wikipedia.org/wiki/How_to_Solve_it_by_Computer](https://en.wikipedia.org/wiki/How_to_Solve_it_by_Computer)

------
3pt14159
He's close to how I think about it, but not quite there.

On regularity:

Take the handling of what is false-y. In Ruby only nil and false are false-y.
If you use an && the only way it can come out false is if one of the two sides
is nil or false. Python, on the other hand, tries to treat emptiness as
false-y. It leads to irregularities because assumptions are broken. The
clearest example was that the exact second of midnight was considered false-y.
Fixed in 3.5 if memory serves.

On restrictiveness:

I agree with PG that a pro working on a project with a small set of other pros
prizes unrestrictive languages. On large projects it is usually the opposite.
This is why Go is great for Google, but Ruby is great for startups.

On conciseness vs readability:

Regex is concise, but it is shit at readability. The reason conciseness is
less important than readability is that conciseness is about the _character
count_ while readability is about _the time it takes to understand_. They're
related, but only to a point. Ruby is much more concise than assembly, and
that helps its readability, but further gains in conciseness don't really
impact speed to understanding.

Regex is fine for a grep of your own filesystem because you're the one writing
it so you know you understand it. But for complex form validation it's usually
not the first tool I reach for because I tend to assume that the person that
comes in behind me might not know more than the basics. Even when I do use
them, I break them up as strings into variables that name what they do before
concatenating them.

~~~
xamuel
Nontrivial regexes should be treated as their own language. I.e., either the
team agrees to use them freely, in which case, team members are obligated to
actually learn them. Or, if not, then team members agree not to use them.

If you were working on a Python-only codebase and someone opened a merge
request to add a C++ file, you'd smack them down. Complex regexes should be no
different.

~~~
mrfredward
I disagree. Regexes can can be done in a neat and modular way. I can write a 3
line function IsValidEmailAddress() with a nasty regex in it, and everyone on
the team who doesn't know regexes will understand immediately what I'm up to
when I call IsValidEmailAddress() and program around it.

The alternative is to write a function hundreds or thousands of lines long
that verifies one part of an email string at a time. Of course, that function
will be too long and will have lots of duplicated code, so it will be
refactored into smaller and more generic functions over and over until I've
got a half baked text matching library on my hands. And then I'll have to
document the library, and write unit tests for it, and if there is a bug or
assumptions change about what to accept as an email, someone will have to read
and understand my pages of code. And then someone will want to find email
addresses in a file, and there won't be an efficient way to do it.

If it's not trivial to do a pattern match with regexes, it's probably not
trivial to do it without. The best you can do is keep the ugly implementation
hidden in some well named functions that make your intent obvious.

~~~
thrmsforbfast
Regexes get a bad rap mostly for two reasons:

1) they are abused by people who have never written a parser but need to write
a parser, often to parse languages that are highly nonregular, and

2) they get thrown at problems where the underlying grammar is actually
unknown and iterated on until they're uneditable black boxes.

The first problem already has a really good solution: hire people who know the
standard undergraduate CS material and you'll never end up with a regex-from-
hell when a simple and easy recursive descent parser would've done the trick
with ease.

The second problem is, of course, more difficult. I've only seen regexes used
successfully for a problem like that once. And that's only because the
structure was very easy to think about as an NFA. By itself, the regex was a
freaking nightmare to look at. But the documentation included a picture of the
automata and a great description of what was going on. From there, the
monstrous regex was easy to reason about and modify.

~~~
wahern

      hire people who know the standard undergraduate CS material and you'll never end up with a regex-from-hell when a simple and easy recursive descent parser would've done the trick with ease
    

The number of programmers whom I've personally worked with that actually
_applied_ proper parsing principles in their work are exceedingly few--count
on one hand, few; too few to make generalizations. But suffice it to say not
all had CS degrees, nor even a degree of any kind.

By contrast, most of the people I've worked with professionally have had a CS
degree, and regex abuse is rife. I'm sure the reasons are varied and complex
(never really grasped it, unable to recognize appropriate situations, can't
apply it in larger code bases, or are simply unable to resist easy fixes when
under pressure), but the end result is the same.

~~~
thrmsforbfast
_> But suffice it to say not all had CS degrees, nor even a degree of any
kind._

You will notice that I very carefully worded that post. The word degree wasn't
used.

Among all the coding quiz questions that get asked in interviews, the parsing-
related questions make a ton of sense to me. Being able to bang out a
reasonable parser -- and more importantly, knowing which type is needed for
which problems -- is an actually useful skill and an even better indicator.

 _> most of the people I've worked with professionally have had a CS degree,
and regex abuse is rife._

IME this can be shortened to simply:

 _> regex abuse is rife._

~~~
perl4ever
"> But suffice it to say not all had CS degrees, nor even a degree of any
kind.

You will notice that I very carefully worded that post. The word degree wasn't
used."

I sympathize with those shaking their heads at people using regexes to parse
XML, for instance. I have fought the battle of trying to get people to do it
right. However, in (my version of) the real world, people will give you data
that is invalid according to a canonical parser or the language standard, so
sometimes you might as well use regexes anyway. A lot of professional
programming is more like being an auto mechanic than being an automotive
engineer. HN gravitates towards the latter mindset.

~~~
thrmsforbfast
_> However, in (my version of) the real world, people will give you data that
is invalid according to a canonical parser or the language standard, so
sometimes you might as well use regexes anyway._

This is very true, and I mention this in my top-level reply:

 _> Regexes get a bad rap mostly for two reasons: ... 2) they get thrown at
problems where the underlying grammar is actually unknown and iterated on
until they're uneditable black boxes._

When the (non-fuzzy) version of the input language is regular, then
robustifying a regex can be the right way to go. And blaming the person who
chose regexes on the basis that they used regexes doesn't make sense if the
robustification isn't good enough. The failure to sufficiently robustify is a
failure of domain knowledge about that particular fuzzy parsing task, not a
failure of training.

However, if a developer is using regexes to parse XML then they're Doing It
Wrong (TM), _even if_ the XML files they're parsing are not always valid XML.
It's much easier to robustify the right solution to the idealized problem than
it is to robustify the wrong solution to the idealized problem.

 _> A lot of professional programming is more like being an auto mechanic than
being an automotive engineer. HN gravitates towards the latter mindset._

That's true. But also, a lot of professional programming is more like being an
auto mechanic _even though it should be like being an automotive engineer_.
Including, unfortunately for a hundred deceased Toyota vehicle owners,
automotive software engineering.

------
daenz
I wish there was a way to send this to my programmer coworker without seeming
like a jerk. He insists we "work" together, meaning, we're in the same
physical space, working on two different projects, and he interrupts my train
of thought every 2 minutes with an arbitrary question about the thing he's
working on. I'm close to a snapping point and don't know how to address it
tactfully. What I want to say: "us working together may enhance your
productivity by 50% but slows mine down by 80%."

~~~
snoman
The next time it happens, have a conversation with him about time-boxing. That
is, say something like this:

Him: <some question> You: "Hey, that's an interesting question. Time-box an
investigation into that to 20 minutes, and then we'll dig in after that if
you're still not sure about the solution."

If he asks what time-boxing is, then you can have the discussion about
actively investigating a solution for a specific amount of time. If, at the
end of that time, you don't have a solid lead on a solution, then you reach
out to your peers. That means, you have input to answer questions if your
peers have any, and you know some simple solutions that don't work, and why.

That's what I've done with jr engineers for a number of years now and it works
great. You have to do it a couple of times before they turn it into a habit,
and internalize it, but eventually they bother you less and become better
engineers in the process.

~~~
daenz
Thanks. Admittedly, I've been giving positive re-enforcement to his questions
by answering them immediately. Which I realize is doing us both a disservice.
It's now easier for him to ask me than to spend time investigating, and it's
easier for me to answer quickly to get back to my work. But it's just making
the problem worse. It's become programming by proxy.

------
Jach
I think being able to be productive without having the whole thing in your
head is one of the important skills of working at BigCo. It's definitely more
awkward and I'm sure leads to an increased number of bugs, but it lets
shifting teams and priorities move forward. Some people struggle a lot with
this though, especially if they've only ever worked on their own programs or
programs designed in a small and tight group (e.g school projects). It's
definitely a skill.

After an intern experience I changed up one of my phone screen sections -- we
have to do a phone screen (I modeled my favorite variation on Yegge's), I try
to change it up from time to time though to make it somewhat interesting for
both people, test different ideas of what questions produce what signals and
what signals are valuable, and at least never make it a whiteboard or
accurately-code-well-known-freshmen-problems-like-sorting-algos hazing. I used
to reserve a section for discussing program design in a hypothetical board
game program, I took that out to instead give the person a description of a
simple web app with an accompanying ~100 LoC describing the architecture and
component hooks (simplified and mocked a bit from our actual code, it has a
startup that creates top level components like the PageBody component and the
Header component, the Header component creates a Navigation component and
whatever else, the Navigation component creates navigation elements...) and a
problem statement that involves threading some new Component data from the
root to a leaf using the existing framework in order to make a proposed code
change work. So it's not that hard objectively, it's mainly adding a new
function param in a couple component initializing functions and updating call
sites, but it requires being able to read a chunk of code, ignoring most of
it, but understanding just enough of it in context to make the change. The new
intern who got through this (took ~20-25 mins) ended up being pretty solid,
and I think the test would have negatively differentiated the prior intern
with other candidates we had thought of as about equivalent...

------
edoo
I'm a talented programmer but wasn't always so. The most important thing I
ever learned was how to plan a program into abstract components that are as
isolated as possible BEFORE ever touching any code.

If you plan an abstract hierarchy correctly you don't have to try and keep the
entire swarm in your head. You can focus on the individual components and when
they are done, forget about the implementation and refocus on the next level
in the hierarchy. If you run into issues step back out into the planning
phase. Never write code without a plan.

When I was younger I could barely get 1k lines of code down without it being a
mess. Now I can build complex applications with multiple libraries in the
20k-40k SLOC range without much mental effort. It is almost like entering and
leaving functions in your brain.

~~~
jpmoyn
Do you write the plan down? Like an outline? I want to get better at what you
are describing.

~~~
edoo
Better put you build the components with the least dependencies first after
your software architecture is decided, up to and including pseudo code, to the
point you are confident in the design and are not required to think about it
while implementing it in real code. If a design issue comes up stop
programming and fix the architecture. I'm a graph paper guy myself.

Say a racetrack library. We have a 'racetrack', it has a 'list' of 'cars',
they have a 'list' of 'tires', and 'list' is a custom class and we know how it
is all going to fit.

Now normally you might start with a 'racetrack' class, start a car class,
start the list class so you can have the cars on the racetrack, start the
tires class. Add functionality to the list class to get something to work. You
leave a trail behind you that if you try to keep in your head can get pretty
awful in big projects.

If you start with the objects with least dependencies you can forget about
them once you are done. 'list' has no dependencies, you can make the class
(and all test code if desired) and forget about it. Now you can focus entirely
on a tire (and all test code if desired), and forget about it. Then you do the
car class fully, then the racetrack. At every level you have everything below
it done so you can fully complete the implementation in one shot instead of
bouncing all over the place . It isn't always perfect but it really cuts down
on the brainpower you have to expend.

This is best suited with unit testing. At first it can feel ungratifying to
program this way, even scary. You could be 85% done with your planned
architecture and have almost no visible behavior of your final product, with
that last 15% being the actual high level abstractions that really tie
everything together and the project just seems to pop into completion.

------
nivexous
A couple months ago I quit my software job to incubate a startup idea. I’m
working out of home now and this definitely rings true. I’ll add a couple
additional thoughts. One is that “whole program” thinking is tiring, and I can
only muster about 6 hours a day. After that, my brain resists thinking deeply
until the next day. Naps help somewhat, but I’ve learned to just switch gears
and either enjoy some entertainment, or work on less demanding tasks. I keep a
queue of lite tasks now for this purpose. Another discovery is how helpful a
warm shower is to creative thought (which includes refactoring IMO). It sounds
rediculous, but when I’m working on a tough problem now, I may take 3 or more
showers a day. And it works! Like magic, I don’t think I’ve ever _not_ come
out of a shower with the problem solved. There’s nobody to judge me :) But if
I do create a company someday, I will have onsite personal showers and
encourage their use.

~~~
dasmoth
I find nearly anything that gets me away from the screen can help with
creativity. For me, gardening is one of the best.

An interesting property of the shower is that it’s one of the few places where
hardly anyone takes electronics. Even if you’re someone who reaches for a
smartphone without a second thought while walking, the chances are you put it
down before showering (the garden has a similar property —- I don’t want to
handle electronics with muddy hands or gloves...)

~~~
taneq
> An interesting property of the shower is that it’s one of the few places
> where hardly anyone takes electronics.

I've been finding plane flights are good for that kind of unstructured free
association problem solving too, for precisely that reason.

------
saagarjha
> Oddly enough, scheduled distractions may be worse than unscheduled ones. If
> you know you have a meeting in an hour, you don't even start working on
> something hard.

This hit a little too close to home, since I'm procrastinating working on
something right now because lunch is scheduled to be in ten minutes and won't
be able to start anything meaningful…

~~~
dasmoth
See also:
[http://paulgraham.com/makersschedule.html](http://paulgraham.com/makersschedule.html)

------
CamTin
One thing that this list of prescriptions would probably contain if this were
written today is: write tests. This lets you make a lot of classes of change
without actually having to hold the whole thing in your head. You can make an
educated guess as to what needs to change, write a test for it, and confirm
that you made the right change. More importantly, if the tests you wrote
months ago are complete enough, you can be confident that your recent changes
didn't fuck up anything up that you didn't intend.

~~~
cle
Depends on how you write the tests. If your tests are very low-level, making
cross-cutting changes can be extremely painful, to the point that you'll avoid
doing them b/c of the cost of fixing the tests.

Unfortunately, such low-level tests are becoming increasingly common as
managers and engineers focus on optimizing metrics like code coverage, and
those kinds of tests are very easy to write to improve code coverage.

~~~
alttab
As a manager, I haven't bothered to look at code coverage for years.

What I care about is functional coverage for the boundaries and exception
cases of our use cases or APIs. Run them CD in your pipeline.

Cover customer expectations at the highest level of reasonable abstraction (I
also tend to avoid ui testing, and frontend JavaScript too).

Low level tests or unit tests only matter insofar as they make it easier to
trust your dependecies. Breaking abstraction or encapsulation to test is a
hard stop NO.

------
mark_l_watson
That is also how I work when I am at my most effective. As a Lisp programmer,
I especially like: “You can magnify the effect of a powerful language by using
a style called bottom-up programming, where you write programs in multiple
layers, the lower ones acting as programming languages for those above. If you
do this right, you only have to keep the topmost layer in your head.”

I work for a company right now in an office and the only way I get
uninterrupted time slots is to either start work very early in the morning or
occasionally go home for two hours when I need quiet heads down work time.
When I work at home as a remote consultant I usually do 4 to 5 hour sprints
and my wife and my pet parrot usually don’t interrupt me. A five hour sprint
starting early in the morning followed by four hours of exercise and
relaxation, then another hour or two of work is just about perfect for me.

------
lukev
This is certainly true when solving hard problems.

A counterpoint would be that most software used to run businesses is
complicated, but not hard (or the hard parts are very localized.) And for
these problems it's best to factor your codebase such that it is not
_required_ to hold the whole thing in your head.

Because if you do, at some point in the project's history, someone is going to
blunder in and make a change that fractures your crystal palace without you
even realizing it.

~~~
hateful
I'd say the second part of #3 is addressing this.

> You can magnify the effect of a powerful language by using a style called
> bottom-up programming, where you write programs in multiple layers, the
> lower ones acting as programming languages for those above. If you do this
> right, you only have to keep the topmost layer in your head.

~~~
felipemnoa
This is pretty much how you should write programs regardless of the
programming language. Otherwise the program would become virtually impossible
to understand and maintain as it grows.

------
tarikjn
I think the fundamental is the description of software as an idea.

It is very important to understand, because it helps explains that the
essence/idea part of software writing will never be automated and that any
repetitive part in software must be automated to make progress.

This is why the concept of software sweat shops/and some bad flavors of pair
coding/agile development are stupid, it's all built around systematizing what
should have been expressed in higher-order software anyways. They're only
effective at still making money for the shop while the market hasn't caught
up.

------
commandlinefan
Amazing how modern corporate "catch the cheaters and slackers" management
structures are designed exactly the opposite of this.

------
alex_young
#2 Work in long stretches.

This is wrong or at least worded incorrectly IMHO. I believe the author knew
it wasn't quite right when they wrote it, since the second paragraph is an
attempt to qualify it away.

The rule would perhaps be better stated as "Think in a distraction free
environment".

Plenty of productive programming is done at rest, and long stretches of "work"
produce subpar code.

~~~
codingdave
I do think working in long stretches is effective, but I'm not talking about a
single "butt-in-seat" marathon coding session. I think of it more as a focus
on solely one project for a number of days, without multi-tasking on 3+
projects at once giving each one only a couple hours per day.

The processing that you do when not sitting at a desk, and those eureka
moments that come along with it happen more often when you are working one
thing at a time. Or, at least they do for me.

~~~
dasmoth
Yep. The tricky thing is that "help a colleague on some little problem" can
end up being an expensive context switch, but can also be pretty valuable. A
degree of asynchrony, and perhaps some kind of "office hours" type mechanism
to handle questions in batch mode seem like good solutions. But that goes
against processes which treat "resolving blockers" as top priority.

------
userbinator
As someone who does this with a lot of programs I've written, I'll say that
avoiding _ab_ stractions is just as important as _di_ stractions --- if doing
the simplest of tasks takes half a dozen layers of abstraction and
indirection, it makes it all the more difficult to see the whole program at
once. "Too many moving pieces makes it hard to see what's going on", as the
saying goes.

The amount of "fluff" introduced by "modern" languages and their culture, like
Java and C#, is certainly an obstacle to this; it seems like they _encourage_
overcomplexity and the creation of huge complex software, so that all programs
written in them naturally take on that style, no matter how trivial they
actually should/need to be.

~~~
kungtotte
The languages themselves sort of forces people's hands depending on how rigid
they are and how easy it is to change your mind afterwards.

Take Python for example. Firstly they have a culture of "we're all adults
here" and don't really bother with public/private distinctions in classes, and
secondly the language makes it easy to turn a variable into a property with
the @property decorator.

I'm not that familiar with Java or C#, but I guess turning a variable into a
property after the fact is non-trivial. In Python all the calling code remains
unchanged (`val = object.field`)

------
CarVac
I manage to keep my main side project (~20,000 lines of code) in my head
despite not meeting conditions #1 (I'm always distracted), #2 (I work in my
free time at home), and #3 (C++, woo) because I have extensive documentation
about the _why_ of code, and this helps me put myself in the same state of
mind I was in when I wrote it.

It does help that all the others apply to me, though. I've rewritten it kinda
twice, I already mentioned my comments, I work basically alone on it, so
nobody else edits anything, and it started out very small.

------
phirschybar
"Sometimes when you return to a problem after a rest, you find your
unconscious mind has left an answer waiting for you."

I love when this happens. For me it's usually after a good sleep.

------
RickJWagner
That's the best programming article I've read in a long time.

I can't say all that is true for me, but some of it is.

It's truly interesting to see how others write and understand programs. It
does seem a lot like art (and less like engineering) sometimes.

------
everybodyknows
>Maybe we could define a new kind of organization that combined the efforts of
individuals without requiring them to be interchangeable.

Epihany: Tolerance for uniqueness is one way the open source trumps closed
source.

------
blumomo
That's a wonderful article describing the work of a programmer, why bigger
companies cause them trouble and how good programmers reason about code which
is not even yet written. Paul Graham, which is a notable figure in the
software engineering world if you didn't hear his name yet, names 8 steps that
make you able to load a whole program into your head. I'm proud to say that I
have mastered all 8 steps multiple times, though from my experience in my
current engineering team I struggle with step 7. I do actually believe that
what he describes shouldn't be done, is actually feasible - through pair
programming.

------
gypsy_boots
> 7\. Don't have multiple people editing the same piece of code.

Do others find this pretty crucial? We do this at my org and I’m interested in
knowing if it’s detrimental.

~~~
marcosdumay
I think it's such a basic issue that one shouldn't even need to point. It's
like remembering people to breath.

And now you just said you don't breath, and I am really questioning my
understanding of the world. Do people at your org get anything done?

------
mmartinson
Paul Graham's essays are a consistent joy to read.

------
zubairq
I totally agree with PG here. This is also how Steve Wozniak designed the
first Apple hardware

