
A significant amount of programming is done by superstition - mwcampbell
http://utcc.utoronto.ca/~cks/space/blog/programming/ProgrammingViaSuperstition
======
anon4
I'll have to admit something about myself -- I've never been able to learn how
to do anything from just reading its first principles. I need to have a
working example that I can then adapt and change and observe changes in the
result.

This has been especially true in programming. I've been writing programs in
some capacity for 18 years now and not once in that time have I been able to
simply read the reference manual of anything and following that write working
code on my first try. Well, obviously I've been able to do it for trivial
things, like after the Python 2->3 switch when map/filter/range started
returning iterables instead of lists, simply being told "map/filter/range/etc.
now return an iterable and not a list" is sufficient to explain the change in
behaviour, but that's only because I know all the concepts behind it. You
could say the explanation is only one level removed from my intuitive
knowledge. Things start getting exponentially harder the more levels removed
they get, i.e. if it's a new concept explained using new concepts previously
defined, but for which I have no intuitive knowledge what they mean.

~~~
GauntletWizard
Almost nobody can. Some people think they can, and charge forward with it, but
rarely do they produce something that actually works. Usually, they bang on it
and test it and cargo-cult it until it works, then convince themselves that
it's from 'first principles'.

Even if it were a common ability to produce engineering work from written
design principles, I'm not certain that's what we want. Experimentation is the
key to science. You have to observe, change, guess, run, observe, change,
observe until you can successfully predict what will happen for each change
you make. The scientific method is taught to everyone, even if most ignore it
and skip steps. 'Guess/check' is the _correct_ way to do real-world
engineering and design, not pure-logic simulation, because we still don't know
all the first principles.

~~~
avmich
> Experimentation is the key to science.

Computer science is a branch of mathematics. Here you build models, reason
about them, find properties, then prove them, then rely on them. Tests -
a.k.a. experiments - are usually not exhaustive and don't provide guarantees
of correctness.

Having said that, I agree that experiment is of crucial importance - how else
you would validate your models in the first place? However, both experimenting
and reasoning can be flawed, and part of the trick is to learn how to do that
properly. Computer science is difficult.

~~~
GauntletWizard
Programming is a branch of engineering. Very little programming is done by
proof; It's done by experimentation, debugger, and tests are human-reasoned
and hopefully cover all codepaths and critical junctures (for every x > 0, you
test x=1, x=0, x=-1).

Proof is of critical importance - It's how you find those junctures, for
example, how you optimize, how you design and how you reason about runtime,
etc. But programming as practiced in the field is engineering first,
mathematics second.

~~~
dasil003
Reminds me of the famous Knuth quote:

> Beware of bugs in the above code; I have only proved it correct, not tried
> it.

Deep thinking and mathematical proofs are tremendously useful when thinking
about a well-defined and scoped problem at a more or less fixed level of
abstraction.

The problem with software engineering is that in practice all abstractions are
leaky, and so you inevitably find yourself dealing with issues that are too
far-flung and random to be mathematically tractable. The best you can do is be
rigorous about the core problem you are solving, but there is always some
amount of hammering and duct tape to build it into a non-trivial real-world
system. It's possible to attack this problem asymptotically with a rigorous
engineering process such as NASA employs, but we don't because it's simply not
cost effective for the majority of software.

~~~
Osiris
_because it 's simply not cost effective for the majority of software._

That's a great point. We utilize best practices, like automated tests, to
narrow the gap between pure experimentation and mathematical proof. Proofs in
most code would be difficult to impossible because most of use libraries that
use other libraries, etc., so we just do the best we can given the time and
monetary constraints that we have.

------
krick
That's right, but I guess we're missing the _real_ problem here.

The real gold are these words:

> You've really read the LSB specification for init scripts and your
> distribution's distro-specific documentation? If so, you're almost certainly
> a Debian Developer or the equivalent specialist for other distributions.

Many of us would agree, that it's true. But _why_ is it true? Shouldn't _that_
make us suspicious in the first place? It's almost paradoxical, even funny:
making the work done is easier, than finding out, _how_ to do that work. You
see, init scripts or makefiles aren't exactly rocket science. I mean, things
we are willing to achieve with them are pretty simple: just _start_ some
process, just _build_ some project (which means "execute bunch of shell
commands in the right order", and not implementing out own superoptimizing
compiler).

So when we arrive to situation like one we've got today, the right message
wouldn't be "come on guys, you see what we're doing because of being lazy? we
should stop being so lazy!". That is, yeah, we all are lazy and maybe we
should stop it, but that's not the point. What this situation actually means
is that our tools for doing simple stuff are overcomplicated shit, and our
documentation for these tools is shit. If anything, it's pretty obvious that a
couple of lines of code, showing how to do the work is better documentation
than complete list of possible flags and parameters with no examples
whatsoever. So no wonder people are reading stackoverflow instead of man-
pages. And so on.

~~~
throwawayaway
> it's pretty obvious that a couple of lines of code, showing how to do the
> work is better documentation than complete list of possible flags and
> parameters with no examples whatsoever. So no wonder people are reading
> stackoverflow instead of man-pages. And so on.

the best manpage is a man page with a nice EXAMPLES section you can grep to
quickly. though the permutations offered by many programs prohibit all bases
being covered.

~~~
mgob
[http://bropages.org](http://bropages.org) to the rescue!

~~~
throwawayaway
alias aprobros="bro"

------
rjbwork
I think a better title might be "A vast majority of programming is done at a
level of abstraction that this author is slightly uncomfortable with".

If you work in Java or Ruby or Python or C#, or even C++ all day with say, an
IoC container for argument's sake, you're certainly not constantly thinking
"okay this interface will be resolved to this type, which has these
dependencies which will be resovled to these types, which generate this byte
code, which generates this JITed assembly code, which generates this machine
code, which generates these executions of instructions on the processor and
accesses these memories, which invokes these manipulations of these gates and
flip flops on this particular part of the ALU".

Doing so would be the mark of an unproductive madman.

~~~
dandelany
I don't think this is about abstraction. Abstraction implies having a deep
understanding of the problem domain and encapsulating it in such a way that
the implementation details can be ignored and the domain can be reasoned about
at a higher level.

I think a better title would be "A significant amount of (shipped, so-called
production-quality) code is not fully understood by its authors".

~~~
nostrademons
"A significant amount of (shipped, so-called production-quality) code is not
fully understood by its authors"

Of course. Time spent fully understanding the code before it's shipped is time
that it's not shipped.

This is sort of a Gresham's-Law situation, where "bad code drives out good
code". The code that is fully tested & debugged with all situations understood
isn't available to the user at first. As a result, they go with the "good
enough" solution, build stuff off of it, and by the time "the right thing"
comes along, all the momentum has shifted to its crappy competitor. A bunch of
crappy software gets built on top of that, and as a result, we end users get
the worst thing that could possibly work.

But work it does. Maybe it's worth a shift in perspective to be grateful for
the stuff that works rather than worry about the stuff that doesn't quite work
right.

~~~
Swizec
Anyone can build a bridge that stands up, but only an engineer can build a
bridge that _barely_ stands up.

Good Enough is our main design goal. Anything more than good enough means
you're wasting resources.

~~~
coldnebo
This sounds like Engineering, but be aware that in mechanical engineering the
requirements are well-defined for materials and load. Consider the famous
bridge resonance problem and you will see why wind resonance is now a required
subject for structural engineering.

ME has an advantage over CSE in this regard because in CS the requirements are
usually very poorly stated. About the best and most rigorous requirements you
get are from mathematicans, but most devs and managers now certainly view that
as wasting resources.

To make not to fine a point of this, I always like to turn to Alan Kay's
comparison: TCP/IP vs the web. When was the last time any human technology
like TCP/IP scaled so well and so invisibly, it's like air we breathe without
even thinking about it. The web in contrast was the work of rank amateurs.

Alas, mathematics is really the "good enough" standard we in CS should strive
for just like physics is the "good enough" standard behind ME and EE.
Unfortunately as CS opened to the mainstream, I think a deep fear of
mathematics led us to view this as "over engineering" even when it wasn't. The
results are that the majority of the web is woefully _under_ engineered,
requiring far more money and time for inferior products.

We _know_ they are inferior, because even the simplest gui application has
more consistency than a web variation of it. And that's what marketing
constantly compares things too when they can't understand why the web sucks as
much as it does.

"Good Enough"? Please! For the last 20 years we haven't even come _close_!

~~~
olavk
The web might have been created by amateurs, but it works incredibly well.
Sure, we have browser incompatibilities and whatnot, but this is an effect of
having multiple independent implementations of the standards, which is part of
what makes the web work in the first place.

~~~
coldnebo
no, it doesn't. Having multiple independent implementations is a PITA, but
that's why you have testing & validation labs along with standards. Take
windows graphics driver labs... They test for pixel perfect compliance of
output across hundreds of vendor implementations. Contrast that to the web
where it took a separate group outside the w3c to embarrass browsers with the
ACID2&3 tests. Now separate browsers look a lot closer in output.

Devs are trying to fix these ecosystems: why does react use a virtual dom? Why
do we need css resets? Why do we need js shims and polyfills? Because its the
only way to come close to normalizing the platform.

But have you ever wondered why you expect no two browsers display the same
image? Postscript met that bar and is just as old as the web. Why didn't the
w3c base the web on device independent coordinates instead of this confusing
and unpredicatble layering of partial scalars and "angle subtended by a pixel
on a 96dpi surface at a nominal arms length from the surface" crap? No one
could have made a reference implementation off those requirements, much less a
consistent verification & validation suite.

And no offense to TBL, but HTTP didn't even survive first contact with
netscape's vision of shopping carts. Cookies? An elegant solution? Or simple a
new hell of tunneling client/server state over a supposedly stateless
protocol. HTTPS everywhere requires long lived sessions as the basis?!? No
wonder people are heading towards web sockets, etc. webapps are client/server
apps -- HTTP was always grossly misapplied to them.

Webdev is hard, not because I'm building beatiful bridges in the sky that are
"good enough" poetic balances of constraints while coming in on time and on
budget... Webdev is hard because of all the underlying assumptions I
constantly have to check and recheck because I can't rely on them as an ME
would (or hell, even as a backend J2EE engineer would). This is why some of us
lament that people don't know the stack all the way down, because we have to
in order to solve real problems. Every abstraction leaks, but hell, web
abstractions are flipping sieves!

No, the thing that "works incredibly well" is not the web, but whats under it
that lets us make so very many mistakes and yet keep on trucking.

~~~
coldnebo
Regarding Postscript vs web: I'm only talking about device independence.
Specifically in the case of pixel perfect layouts. I am not talking about the
layout constraint problem, which is extremely challenging no matter the
technology. But layout constraints depend on a solid notion of coordinate
system, which the web lacks. Device independence gives you that in postscript
and SVG.

Besides, windows faces a similar problem of multiple resolutions and devices.
How do they v&v? They set the resolutions the same for certain tests! Even if
you do this for browsers, they cant pass the test. Yes, it would be nice if we
could have device independent layout constraints as well, but even the
simplest most constrained test not involving layouts fails. At least now, its
close. Before ACID it wasn't even close.

If webdevs can't even rely on their browser coordinate system in the most
heavily constrained case, how can they hope to trust it when they try to solve
challenging problems of dynamic layouts across multiple resolutions?

------
nicpottier
I actually really like the content of the article, but I'm not convinced
superstition is the right word here. Yes, absolutely I have copied an init
script / makefile or whatnot and if it worked after I tweaked it then I went
on my business. But I didn't use it as is because of superstition, but rather
because it now worked I believed it correct.

Yes, quite often that came without a perfect knowledge of every single option,
certainly usually without referring to the manual for every one of them.

Perhaps this is semantics. But the definition of superstition is:

 _a widely held but unjustified belief in supernatural causation leading to
certain consequences of an action or event, or a practice based on such a
belief_

Which ya, isn't quite the right thing. Anyways, agree with the sentiment of
this 100%, just not the title. :)

~~~
ptx
I think this is more commonly called "cargo cult programming", defined by
Wikipedia[1] as: "the ritual inclusion of code or program structures that
serve no real purpose".

[1]
[https://en.wikipedia.org/wiki/Cargo_cult_programming](https://en.wikipedia.org/wiki/Cargo_cult_programming)

~~~
nicpottier
I think there's a significant distinction here to be made between
"configuration" and "programming".

Do I copy/paste swaths of code that I don't understand and put them in my
programs? No, absolutely not, because each line is one I can reason about and
want to understand.

But do I copy/paste swaths of configuration files / init scripts / etc,
without understanding the implications of every single configuration? Yes,
absolutely, because the depth behind each of those options is often irrelevant
to me. (until I discover otherwise late at night when an alarm goes off (!!))

I think it is important to distinguish these. I don't think any dev that's
done ops work would claim they aren't guilty of the latter, but at the same
time I think very few good devs ever, and I mean ever, do the first.

~~~
lil_cain
Init scripts _are code_ though. I mean, they're code that's mostly boiler
plate, but they're still code - I can do arbitary things in them, reasonably
unconstrained by their stated purpose.

------
santosha
I've always enjoyed what I call uncoding, which is taking a working piece of
code and taking stuff away until things break, and examining how they break.
It's really helped me understand what each piece does, and be more efficient
when I'm writing it myself.

~~~
eric_h
Amusingly, that's also more or less how we've learned to understand the
various functions of the different parts of the human brain. We don't know
what it does until it's been removed/damaged.

------
jnem
I hate to call myself out, but I envy most of you. It sounds as if many here
get the opportunity to work on interesting problems from end to end. As such,
first principles matter, and approaching something from scratch makes sense. I
however have been stuck in a segment of the market that doesn't appreciate the
right solution, but rather just wants a solution as fast as possible. To be
clear, I'm talking about a more consumer audience. My clients expect things
done quick, and quick means leveraging cms, ecommerce platforms (magento, woo
commerce, etc.). To be frank I find such work unsatisfying, and often need to
hack things together to meet the requirements set forth within the constraints
of the budgets I'm working with. I'd love to be able to work in an environment
where doing things right mattered, but the plain fact is the deliverables
aren't measurable based on quality, but expedition.

While I'm on a soap box, I'd like to point out to those in the positions where
they're work is evaluated by professionals and peers how lucky they are. I'd
kill for a job/client where my code is evaluated by other developers. This
community sometimes forgets they are the 1% of the development world. Those
who work for the googles, facebooks, and visionary startups with leaders who
came from the same cloth. The rest majority of us are laboring away under
management that have never produced a lick of code, or even design, for that
matter.

I guess the Tl;DR is: appreciate the fact that you even get to consider the
finer points, and pray for the rest of us.

------
Animats
I've referred to this for years as "ritual-taboo programming". Ritual-taboo
societies are ones in which things are done in certain ways because they
worked in the past, but no one knows why. Too much of programming is like
that.

If you work from the specifications for a library, you'll probably find that
some of the documented features don't work. If they weren't used by some
"framework", or mentioned in a how-to book, they probably haven't been
exercised well.

------
depsypher
A nice example of this is yoda expressions in, say Java.

The initial reasoning for them was to prevent accidental assignment, since a
C/C++ compiler will complain on this: if (null = x)

but not on this: if (x = null)

In Java, neither is allowed, so the whole yoda construct is almost pointless.
The exception is boolean assignment which does benefit from this idiom, but
the loss in readability is a trade off to consider. In any case, I suspect
most uses of this construct in languages like Java fall into the
"superstition" category, where the people using it aren't really considering
why.

~~~
Peaker
A lot of us find Yoda conditions easier to read:

    
    
      if(0 == very_long_function_name_here(param1, param2)) {
    

Tells me a lot after I've read a little of the line, whereas:

    
    
      if(very_long_function_name_here(param1, param2) == 0) {
    

Makes the == 0 part easy to miss, and buried behind a lot of clutter.

~~~
klibertp
Why would you _ever_ have a function with a name that long in the first place?

Also, there are many more ways of dealing with readability: try different
formatting and indentation rules, wrap lines manually where you think it makes
sense and so on. As for your example, even the lowly C let's you do something
like this:

    
    
        int (*short_name)(int) = &veeeeery_long_and_ugly_and_unnecessary_function;
        if(short_name(1) == 2) {
          printf("\n\nYay, it worked!");
        }
    

which completely solves the problem, no matter the order of compared objects.
I'm not sure, but I suspect things like this are being optimized away by the
compiler anyway, which would mean that you can use it anywhere you want
without worrying about costs of indirection.

In higher level, modern languages you have even more, much more sophisticated
tools for doing this kind of things.

~~~
ericb
Explicit function naming requires slightly longer function names, but you can
actually read a program and get a sense of what is happening. Give me long
function names over short ones any day. In my book, 5 words is a little long,
but not out of bounds.

~~~
klibertp
If you take a closer look at my example you'll see that the long name is there
of course, it's still very close to the point where it's used. It's only
aliased for a moment in this single section of code and because of lexical
scoping there's no danger of name collision. Of course, doing this in a global
namespace makes no sense.

How long identifiers are acceptable depends on a language. With languages
without namespacing or modules you obviously have to use some naming
convention so that there are no name conflicts and this makes your names
longer. I'd say 5 words is ok in this case.

But I'm not arguing against long names of things: on the contrary, I like
having names as descriptive as possible. OTOH, too long names are also bad,
because they make the code less readable and harder to work with. Like almost
always it's a matter of balance: you need to know when to stop adding words to
a name. I think that "name is too long when it starts making other names in
the same line much harder to spot" is a good heuristic for this.

~~~
Peaker
The alias is just adding confusion. If I see such an alias in code, I would
wonder where else that function is used in the function, and be confused that
it isn't.

Not to mention it becomes more verbose once you make that alias a const ptr
(which we do on all of our local variables).

~~~
klibertp
> The alias is just adding confusion. If I see such an alias in code, I would
> wonder where else that function is used in the function, and be confused
> that it isn't

You'd be confused once or twice, then you'd learn the technique and you'd stop
being confused. Every code pattern was unfamiliar to you at first. And
confusing, until you internalized it. It's unrealistic to assume that you can
ever _stop_ learning new patterns - try switching to another language and you
immediately have dozens of unfamiliar, confusing patterns to learn. (it gets
better after a certain amount of languages known (like
[https://klibert.pl/articles/programming_langs.html](https://klibert.pl/articles/programming_langs.html))
because you start noticing meta-patterns)

My C is rather rusty nowadays and using function pointer here may not be the
best option, but as someone else said, there are other language tools for
doing this kind of aliasing, like #define. I'd go for function pointer
probably, because it reveals not only a name, but also a type of function and
it's guaranteed not to escape the current scope (unless explicitly returned)
while #define has no knowledge of scopes at all. In languages which support
real macros, and preferably lexically scoped macros (like Racket) I'd use
those. In languages with closures and first-class functions I'd probably do it
in yet another way. But in general, if I find myself working with a name so
long that it makes it hard to spot other names on the same line I _will_ alias
it locally.

Have you read
[http://shop.oreilly.com/product/9780596802301.do](http://shop.oreilly.com/product/9780596802301.do)
? It's a good, short book on the topic and it discusses exactly this issue at
length in one of the chapters.

------
venomsnake
Since I began programming I don't believe in miracles. I count on them.

------
mpdehaan2
A very sigifigant part of Ansible service module code was done to deal with
misbehaving init scripts. They are very much cargo-culted just like RPM or
debian package formats - borrow the ones from your previous project.

They are hard to get right and many apps don't daemonize correctly or return
OK before they are ready to be running - and then are actually running much
later. Upstart/systemd didn't neccessarily make that more clear for people to
understand either.

While I haven't tried it, I recently encountered a reference to
[https://github.com/jordansissel/pleaserun](https://github.com/jordansissel/pleaserun)
and it sounds promising.

I don't know how much of this applies to programming in general, really, but
init scripts are.. yes... special things.

~~~
GauntletWizard
I dislike tools that claim to automatically generate your configuration for
you. At Google, one of the worst abuses that anyone had to deal with was auto-
generated configuration; a tool purported to deal with configuration for you,
but really just blutrted a boilerplate mess into your directory, many of the
properties and setting set by cargo-cult or overriding sane defaults with
baked-in versions of the same. Facebook is proceeding down this path; There's
a new tool that will do all of your service stuff for you! I look forward to
what a boondoggle those services that use it will be.

There was a discussion I recently stumbled upon about init systems in
containers. Many processes spawned lots of zombies! You need a full init!
Well, no; If your process is forking new processes, it should probably wait
for them. If it's forking other processes that fork other processes, and
they're not cleaning up zombies when they're done, _that is a bug in those
processes_. If all else fails, your root-level process should probably
implement a simple init on it's own (a wait() on loop in the background,
dumping wait objects into a LRU map that other processes can do an equivalent
of waitpid() on). Or just run under bash.

------
ggchappell
Maybe a better term than "superstition" might be "validated experience". We do
what has worked in the past.

It is true that there are standards that should be read and followed; the ones
referred to in the article are probably among these.

But other standards have a fair amount of wishful thinking in them. (How many
of you JS developers have read the latest ECMAScript standard?) Sometimes,
what _has worked_ in the past is a better guide for practice than what _ought
to work_.

------
perlgeek
sysv init scripts in particular are so pointless to write.

In the end, the whole script usually revolves around four lines of actual code
(starting a process, stopping it, querying the status, optionally sending it a
SIGHUP to make it reload its configuration), but the result is a script with
50 lines of boilerplate.

Which, of course, could nearly all be automated away by a sane, declarative
format.

Leaving all the political mess aside, that's the part that systemd got right.
You write sometthing like

    
    
        [Unit]
        Description=some human-readable text here
    
        [Service]
        Type=simple
        User=...
        ExecStart=/path/to/deamon --with=options
        KillSignal=SIGKILL
    
        [Install]
        WantedBy=multi-user.target
    

and that's it.

Quick comparison of sysv init script lengths on Debian Wheezy vs. systemd
service files on Debian Jessie: ssh: 162 vs. 15 lines, cron: 92 vs 11 lines,
dbus: 122 vs 9 lines

My point being: it's a bit too much to demand that people carefully engineer
their sysv init files, when about 90% of those very same init files is simply
boring, repetitive boiler plate code. Also, what happened to the "don't repeat
yourself" principle?

I tried writing a sysv init script from scratch. I stopped because of sheer
boredom, and resorted to copy&paste to save both time and nerves.

~~~
vezzy-fnord
The issue isn't so much in the medium (shell), but the underlying program
itself (sysvinit) being overly primitive. See the daemontools family where
runscripts are largely as short or shorter than systemd unit files:
[http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/ru...](http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/run-
scripts-and-service-units-side-by-side.html)

In addition, the BSDs have much shorter rc scripts mainly by putting all the
boilerplate functions into a single library file that is then sourced.
Gentoo's OpenRC framework, as well.

~~~
digi_owl
Franly i find myself thinking that moving the boilerplate elsewhere is also
what systemd and upstart does. But those implement it in C rather than shell
script.

------
ff_
While the article is dramatically true (credits to StackOverflow for some of
my production code), there's a factor that isn't taken into account here:
time.

Yes, because as engineers our _time_ is more valuable than a minimal and
perfectly-standard-compliant solution. I would happily read all the RFCs, and
all the significant papers in CS, and all...etc.

The reality is that we don't get paid to read, but to ship.

------
copsarebastards
I'm surprised by the number of people who don't see this as a problem.

I've worked pretty recently as the most senior developer on a team, and I saw
tons of issues caused by copy/pasted code. It results in a lot of issues.
Subtle bugs around `==` versus `===` in JavaScript, unecessary variables that
obfuscate the code, Python that isn't Pythonic, C that was written as an
example and therefore doesn't include the error handling that is _constantly_
necessary in C (how many C examples check the return value of malloc?).

The biggest problem, I think, is that often you end up with 10 different ways
to solve the same problem in the code because instead of looking at the
existing code, people Googled for a solution with subtly different keywords
than the previous person and found a different blogger. This kind of
duplication is very hard to identify and remove, and this _does_ cause bugs.

Of course a more experienced developer can work around these issues, but a
more experienced developer also knows off the top of their head how to write
the code themselves and usually does it.

~~~
superuser2
>doesn't include the error handling that is constantly necessary in C

I mean, I'm sure there are some cases, but how many C programs are capable of
gracefully recovering from a malloc failure? Probably very few. If the program
is going to crash anyway...

~~~
copsarebastards
I don't agree at all, but even if I did, that's only one of _tons_ of cases.
There are plenty of very recoverable errors which aren't handled by example
code you copy off the internet, but which you really _should_ be handling.

Take networking code for example: do you really want to be dying without a
visible reason because you didn't check `errno` and do a retry? This is really
basic stuff.

------
frandroid
The problem here is that if it works, how is it superstition? When I knock on
wood or throw salt over my shoulder, I don't have a way of verifying that I've
averted disaster. However, I can test the code I don't understand, and if it
does what it says it does (or does what I want it to do, since that's why I've
selected it in the first place), then we're a step removed from superstition.
There might be unintended consequences if I truly don't understand the code,
but that is something else than superstition.

~~~
rbanffy
If you do not understand the code, you cannot say you tested it. It may have
corner cases you did not anticipate. The test may be incomplete and only cover
what you think the program should do.

~~~
mreiland
why write unit tests when we don't understand the microcode in the CPU? We
can't have "tested" it because we don't understand said microcode.

Or wait... no, maybe your characterization is wrong.

Yeah... that's probably it.

~~~
rbanffy
Where exactly did I say you should understand the code down to how electrons
flow through the chip's gates?

I said if you do not understand what you are testing (not the compiler, not
the runtime libraries, not the OS, not the instruction decode, not the
microcode, not the gates), you can't say you actually tested it. You merely
observed what could be a side-effect.

~~~
mreiland
you're drawing an arbitrary line, that was the point.

We specialize and abstract for a reason. No one understands everything about
everything surrounded their code, and that includes other software. Don't tell
me you've personally been through the code of every single project your
projects touch.

You haven't and that makes you a hypocrite.

~~~
rbanffy
A line that encompasses the object you are testing but that you don't fully
understand (or that is hidden from you) and nothing more is not arbitrary. The
best you can do in these circumstances is say it adheres to a spec for the
cases you tested (which could, for all you know, be the only ones that yield
correct results).

~~~
mreiland
It's an arbitrary line. Too many people come up with an opinion and then cast
about for a way to make it seem valid.

~~~
rbanffy
This is, BTW, just what you did by claiming the line is arbitrary.

~~~
mreiland
yes and I'm racist when I point out someone is being racist.

I'm also a troll when I point out someone is trolling.

do you know how I turn female? By pointing out that someone is female.

It's amazing how making an observation suddenly means you've become the thing
you're observing.

Or not, but who am I to judge, amirite?

------
danschumann
Such as, people are taught not to use global variables, but maybe they weren't
taught the reasons why not. The reasons (some of) would be polluting the
global namespace, having unexpected things defined in unexpected places, and
so on. At some point, it could save the programmer oodles of time to just use
global variables. It could be a project with limited scope where polluting and
things won't be an issue. The reason for superstitions at all are stopping
people from doing something when they can't understand the reasons yet. It's
hard to explain to a child all of the reasons why they should be a good
person, and much easier to tell them Santa will give them presents, until they
can understand why being good is inherently its own reward.

------
nickysielicki
I'm a proud cargo culter.

Truth of the matter is you can't just stare at a man page or textbook all day
and try to understand, then code. You'll code some mess you don't fully
understand, sure, but later on you'll come back and either realize it was
wrong and fix it, or you'll understand why it was right. But you will
eventually understand.

The people trying to understand it will have gotten nowhere in the mean time.

------
haberman
I am someone who tends to work from first principles, to a fault. It slows me
down a lot. But it's like a compulsion.

StackOverflow has improved my programming life dramatically, because it makes
it a lot easier to "memoize" the search from what you're doing down to first
principles. Whatever question you're trying to get clarity on, there is
probably a StackOverflow answer out there that builds a positive case for a
particular option, going all the way back to primary documents like standards.

------
danbruc
This does not resonate with me at all. I can not stand having things in my
code I don't fully understand. Without reading the source code of libraries,
without reading relevant standards, without understanding the mathematics
behind algorithms, there is always this feeling I may make really dumb
mistakes. Really understanding all the bits and pieces obviously does not make
your code bug free, but I am usually pretty confident that I did not do
something horribly wrong.

~~~
ackalker
This piques my interest. When was the last time anyone read the source code of
(g)libc "cover to cover" before using it? Does anyone know the precise
implications of (not) defining any of the myriad sensible permutations of its
"feature test macros"? Is anyone even sure that those implications are
actually implemented correctly before relying on them?

~~~
danbruc
I obviously do not and can not trace the entire code path from a print
statement through the class library, the operating system, the display driver
and the hardware, some things I will still take for granted. But if there is a
really critical aspect to what I am doing, say some thread synchronization, I
will definitely not just follow some sample code, but read the relevant parts
of the memory model of the language specification detailing why this works as
expected. If the underlying hardware matters, I will grab the reference manual
for it, too.

A lot of this is surly curiosity and you can get away without it most of the
time, but at least as soon as I run into the first problem it really helps me
a lot to have at least a basic understanding of what is going on in the layer
of abstraction I am interacting with and maybe a layer or two below. And it
gives me a really comforting feeling, that I know what is going on better than
describing it as some black magic.

~~~
kazinator
The computer doesn't care which code you consider to be in the scope of your
concern, and which you take for granted.

If you examine why you're concerned about some code more, you will probably
come to the realization that you're less confident in it because it is less
old and mature, has fewer users, and has had fewer developer eyes on it and
such.

I.e. whether explicitly or not, you're focusing your understanding where it
will be needed and that coincides with where there is more risk if you don't
understand.

~~~
danbruc
It's not about the quality or maturity of the code I am relying on, it's about
the quality of my code. I want my code to do the right thing in the simplest
and most elegant way possible and part of this is knowing really well how
everything works.

------
boomshucka
A significant amount of everything is done by superstition.

~~~
swyman
Still, important to be occasionally reminded of your own assumptions and
corner-cutting.

~~~
kremlin
Though as the author himself implies, it's not always appropriate to allocate
the time necessary to do anything but corner-cutting.

"If you say that you don't take this relatively fast road for Linux init
scripts, I'll raise my eyebrows a lot. You've really read the LSB
specification for init scripts and your distribution's distro-specific
documentation?"

------
mwsherman
A significant amount of programming is done without articulating either the
problem or the solution.

------
throwawayaway
as a concerned cancerian, i am glad somebody is willing to admit it. i leave
my sub conscious do the heavy lifting. "this is the general shape". making a
list before starting is probably the extent of my preparation. if i can copy
what's gone before without understanding it, all the better.

only when i get stuck do i engage critical thinking. maybe this has been to my
detriment.

very often i notice myself and colleagues making up nonsense explanations for
why c++ behaves the way it does without any real basis other than intuition
and truthiness.

------
jrochkind1
What that means is that if you're writing the system you want someone to do
'right' \-- you better provide lots of good examples in the docs.

A 'good' example is one that you've actually tested for real to make sure it
works, that follows the intended principles for the system in question, that
meets actual use cases developers using it will have, etc.

A significant amount of software is released without good docs -- examples
being just one part of good docs. So of course developers copy from the
examples they can find.

------
yason
In my view, a lot of this is just trying to avoid learning yet another system.
Learning systems is hard and takes time, and it's only worth it if you're
going to be using the system a lot in the future.

For example, creating a new init script or a Debian package description is
something I don't do all the time. I don't want to learn _all that stuff_
because by doing so I'd basically push out something else from my mind.
Copying an existing configuration or script and making modifications without
trying to create something original is an excellent pattern for things outside
your immediate expertise target.

The "avoiding creating something original" part does indeed carry a sort of
"superstition". It's like relying on a new culture by merely mimicking it,
because you don't really know it well enough to break the rules correctly. You
don't know why things are done the way they are but your best bet is just to
copy and adapt. This creates a sphere of fuzzy knowledge where intents and
black magic seem to alternate.

------
lmm
This is specific to shell scripting and to the history of nix standardization.
No-one follows or cares about the LSB[1]; certification is expensive and based
on buggy test programs that practically force you to "code to the test" and
not to the specification. And when so many implementations ignore the spec, it
would be _more_ superstitious to code to the magical spec and assume it's
going to work, rather than writing something and testing it empirically.

When I'm programming (in Python or Scala) I can and do check the language
specification if something is confusing, and I don't think I'm alone in this.
It may be a "significant" amount of programming, but there are definitely
areas of programming with more rigour.

[1] I blame Debian, usually quixotic in its adherence to standards like the
FHS - but when it came to the LSB they prefer to pretend that "alien" counts
as RPM support. That set the tone for how much other distros tend to care
about the spec.

~~~
the_why_of_y
IMHO "alien" isn't much of an issue since LSB packages are allowed to use only
a small subset of RPM features; the fancier features aren't portable between
different RPM-based distros either.

[http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-
gener...](http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-
Core-generic/pkgscripts.html)

[http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-
gener...](http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-
Core-generic/pkgdepend.html)

A bigger problem is Debian gratuitously using different SONAME than required
by LSB for some libraries - your LSB binary can't use those system libraries
if it wants to run on Debian. But at least it looks like they fixed that
problem in Wheezy.

------
everyone
I have a tendency to not use things I dont feel that I fully understand. This
makes me stick to using the more basic elements of the language. In C# for
example recently, there was a part of a game I am writing in my spare time for
which 'events' would have suited. But I still dont feel I understand what
events in C# is doing exactly. So instead I just put all the things I want the
event to happen to in a list, and when the event happens everything in the
list has a particular function called on it. I will probably get around to
using C# events eventually when I have looked at them more and feel like I
understand them. There is _maybe_ a very small performance cost to what I did
(according to some stuff I read, but I havent tested this so I dont know) But
at least I have an architecture for my game and code that makes sense to me
and that I feel like I understand completely (at least at the abstraction
level of C#)

------
cel1ne
The answer to "Why we can't build software the way we build bridges?" is that
bridges and other physical works disintegrate over time. Code doesn't rot, so
the mistake of 1980 can be freshly reused 40 years later.

------
dblohm7
This explains why so many websites don't validate email addresses properly.

~~~
nulltype
How do you validate email addresses properly?

------
dsugarman
even with code I write myself, I can't remember the line by line
implementation on the entire code base. To be honest I often forget
implementation details on code I wrote last week.

~~~
eric_h
Ditto. When a coworker asks me about implementation details of some code a
wrote just a few weeks ago, my first answer is always "I haven't a fucking
clue". Then I scan the code in question and it all comes rushing back.

~~~
eric_h
I should add that the extent to "it all comes rushing back" is directly and
inversely correlated with how long ago I wrote it, and positively correlated
with how interesting/difficult the problem was to solve at the time.

Also, almost every time I go back to review code I wrote more than a year ago,
my first reaction is "jesus, that's a stupid pattern/implementation. Why did I
do it that way?". I've been a professional developer for about 12 years now
and I suspect that will never change.

~~~
quicklyfrozen
It won't. And sometimes you'll start to change it, hit a wall, and then
realize "oh that's why I did it that way."

------
thoughtsimple
I've worked with developers who seem to code mostly by going to stack
overflow, finding something that seems to fit their problem and then they copy
code into the app. No attempt to find a more recent version or a more correct
solution of the problem is made. I've seen JavaScript code copied from a stack
overflow answer that was dated 2009. There is nothing better than a 6 year old
answer to a common JavaScript problem.

------
Sevzinn
Check .gitignore to see if it appears copy and pasted or actually thought
through--this will invariably indicate the quality of the rest of the code.

------
martininmelb
I think this is what I refer to as SODD - or 'Stack Overflow Driven
Development'. Somebody is not sure how to do something, so they look for an
example on SO and they copy the code without really understanding what they're
doing. (note: this is different to when they get an answer on SO and copy it
knowing what they're doing).

------
anupshinde
The author sometimes confuses the work "superstition" with "abstraction". That
is how Humans work everywhere. And that is why my idle PC cannot create a
Facebook/Twitter/or even a plain website by itself.

------
Sophistifunk
If init scripts are mostly boilerplate that people keep copy-pasting, maybe
the init process is the problem, rather than the poor chumps who have to write
the scripts?

------
gee_totes
What would be an example of good programming primary sources?

------
systematical
Superstition...right I don't shave until a project is done and I have lucky
underwear that I program in...

------
benihana
I realized that the entire piece is hinged upon this incorrect assumption:

> _This isn 't something that we like to think about as programmers, because
> we'd really rather believe that we're always working from scratch and only
> writing the completely correct stuff that really has to be there;_

This is the issue I have with this article - the author assumes everyone works
in exactly the same way they do and so are vulnerable in exactly the same way
to their vulnerabilities. I don't know anyone who actually approaches
programming in such a way that they think they're writing things from scratch
and it's correct. I mean, defensive coding is a concept that has existed for
decades that specifically addresses how incorrect the code we interact with
is. The fact that I started coding in an environment where I had to manage
memory myself and now I don't makes me very aware of the fact that everything
I'm using is an abstraction built upon other abstractions.

I dunno, the day I learned to program in Intro to C and Intro to Computer
Science, defensive coding, not trusting user input, and the completely
understandable at the time weirdness of the guy who came before you have been
stressed as things to pay attention to.

~~~
Retra
Usually when I'm writing code it is "Let's get the infrastructure up and come
back to clean up after I know it works..." Sometimes I never come back. It
doesn't mean I think the code is correct, only that it is not the biggest
problem.

------
mkramlich
I think when I was very young and new to programming I sometimes did coding
via superstition. But it was a long time ago, in say my first year or two,
when I didn't yet have a good mental model of how the computer worked and how
the language compiler or interpreters worked. Once you become solid at that
I've found that 99.9999% of the time that computing is refeshingly rational
and deterministic and amenable to logic.

------
nightski
I am sorry, the only reason I have been as successful as I have been is
because I am not superstitious at all. I know that if something goes wrong,
there is always a logical explanation for it. The quicker I can find that
logical explanation the better I am doing.

I don't think I am alone in this.

~~~
agumonkey
I often wish to have investigation classes. How to systematically reduce the
search space in order to find meaning in explaining a phenomenon, good or bad.

------
johnny99
It's true. All of my programs go from line 665 to 667. I've never made a
commit on Friday the 13th, ever--and the one time I did it caused
irretrievable data loss. Once I walked under a ladder, and it broke the build.

------
rilita
Article: Reinvent the wheel, because if you just mount whatever wheels fit the
axle they will inevitably work poorly for your purposes.

Wheels work just fine. Choose the best wheel from those available and tweak it
as desired. Test it to make sure it doesn't shatter under too much weight.
Apply and move on.

This pervading notion of "if you don't understand you shouldn't do it" is
silly. Sure it is nice to understand, but it doesn't put food on the table.
Sometimes it's just handy to use the gun as is and go shoot your dinner in the
woods. You don't need to understand the formulation of gunpowder or rifling in
order to use a gun properly for it's intended use.

