
Magic in software development - skilldrick
http://skilldrick.co.uk/2011/04/magic-in-software-development/
======
jerf
I'm going to mention what I think the other big definition of magic is,
because I haven't seen it spelled out yet. It's when there is something that
is occurring in code but you have no handle on how to follow it back to where
the magic occurred. For instance, you write a class that looks something like

    
    
        class A:
            thing1 = 0
            thing2 = 0
    

but somewhere in between your definition and your usage, the class is suddenly
an ORM mapping onto a database. As cool as that may be, just looking at class
A, you have no idea where to look for that behavior. If you had to subclass
something to get that behavior, you could have looked there; if you had to
call a function on the class to get this behavior, you could follow it back
from there, but when you have this sort of magic you can't. The forensic trail
has gone cold.

If you think about it, you can see how the two definitions are related; this
sort of magic tends to stay something you don't understand because you don't
just naturally pick it up over time, you must make great effort to figure out
what the problem is. But it's a different problem with a different solution,
certainly.

Generally the two places this magic lives is in some all-encompassing
framework, and in the language itself. Perl has numerous magical constructs,
some of which turn out to be very difficult to even search for in the docs
because it isn't even obvious what the operator is. The flip-flop operator
comes to mind: [http://perl-tricks.blogspot.com/2007/01/flip-flop-
operator.h...](http://perl-tricks.blogspot.com/2007/01/flip-flop-
operator.html) If you encounter that for the first time you can be _really
lost_ trying to figure out how the list construction is occurring and it's not
at all obvious what to search for.

------
wccrawford
I've seen some backlash against 'magic' lately and it surprised me. 'Magic'
doesn't need to be understood to be used.

I don't care how my car works, just that it gets me to work every day. If it
breaks day, I have the choice of learning another craft (car repair) or paying
someone that already knows it.

Likewise, if I find a bug in Ruby's base code, I can choose to learn enough
about Ruby's code to fix it, or write a bug report and let someone else.

I'm not advocating ignorance, I'm saying it's a perfectly valid option given
that we aren't immortal.

~~~
pavelludiq
But i do want to learn more about how my car works, theres very little
technology at home that i haven't seen disassembled(washing machine, microwave
oven, tv, pc, mp3 player, cell phone). I've at least skimmed over parts of the
source of a lot of software i use daily(emacs, slime, sbcl, HN). Even though i
barely understand the electronics in my tv, and even though I've barely
scratched the surface of emacs, i still prefer a more hands-on approach to the
things i use in my daily life. I've been like this since a very young age,
apparently i didn't have any toys that weren't destroyed by my curiosity :) I
had the impression that this was true for a lot of people on this site.

~~~
DasIch
As mentioned in the posts abstractions are leaky. You can't drive a car
without having a rough knowledge about how it works or how forces act upon
your car but if you drive a hybrid you don't have to know how the engines
interact nor do you need to know how energy is stored when you brake. If you
are interested in that you take a look otherwise you accept that it just
works.

~~~
wccrawford
Actually, all I need to know is the controls and the daily maintenance, like
putting gas in it. All the rest can be left to a specialist. I don't have to
care anything about why pushing 1 pedal makes it go forward and the other
slows it down. Or how turning a wheel changes the direction. Or how various
buttons activate various features.

They just do, and that's good enough.

(I'm not actually ignorant of these things, but it certainly wasn't necessary
to learn them. I chose to. It hasn't really helped me much, except to satisfy
my curiosity.)

------
happy4crazy
Nick, I'm reading two really cool books that you might enjoy when you're in a
"bottom-up" phase: The Elements of Computing Systems [0] and Computer Systems:
A Programmer's Perspective [1]

[0] <http://www1.idc.ac.il/tecs/> [1] <http://csapp.cs.cmu.edu/>

CSAPP is unfortunately a hulking $100 hardback, but the authors have put close
to 200 sample pages online. Despite looking a bit fearsome, it's _extremely_
clear. And TECS is just awesome--they take you all the way through building
(in a hardware simulator) a simple 16bit computer!

~~~
skilldrick
Wow, those sound very interesting.

I've been toying with the idea of a series of blog posts (or possibly even a
separate site) that give a bottom-up explanation of all the stacks we work
with - starting with electronics => circuits => computers => programming
languages. It would be a way of teaching myself the material _and_ provide a
useful resource to others.

Feels like a lot of work though :P

~~~
jamii
Using TECS to go from NAND gates to assembly is 4-6 evenings worth of work and
totally worthwhile. I have a rough idea how compilers and operating systems
work but until TECS everything below that was run by little laptop pixies.

~~~
skilldrick
Haha I love "little laptop pixies" :) That's exactly what I was talking about.

I think I'm going to have to get this TECS book (no pun intended) - it sounds
just what I'm after. Need to finish SICP first though :P Not enough hours in
the day.

------
skilldrick
Just so people know, I'm not talking Rails magic here - more the idea that
there's stuff we don't understand that's going on that to all intents and
purposes is magic. In other words:

> anything below your current level of abstraction that you don’t understand

------
gfodor
I hate the phrase "law of leaky abstractions." Where is Joel's peer reviewed
article containing a mathematical proof describing leaky abstractions?

~~~
jerf
"Where is Joel's peer reviewed article containing a mathematical proof
describing leaky abstractions?"

The proof is so obvious that it borders on a "by definition"... an abstraction
is something that has had details removed from it. When those details are
still important for some purpose, as they almost always are, your abstraction
is leaky. When you suddenly have those purposes, you suffer.

The idea of a _non-leaky abstraction_ is almost nonsensical; if it doesn't
leak at all it pretty much _isn't_ an abstraction, it's an isomorphism.

~~~
gfodor
The leakiness of an abstraction sounds like its dependent upon the value of
the details that are abstracted away. Clearly there are cases where these
details don't matter. If those cases constitute 100% of the usages of the
abstraction, then the abstraction doesn't leak.

The point here is we're talking about a "principle" of un-intended abstraction
leaks, not a "law of leaky abstractions", which sounds more authoritative than
it should be and leads to shoddy thinking on the part of people who cite it as
some immovable law.

