
The Wrong Abstraction - rumcajz
http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction
======
wellpast
I can tell an amateur programmer from a professional by looking at their order
of priorities when they grow a code base.

Amateur programmers tend to put code de-duplication at the top of their
priority list and will burn the whole house down to that often-trivial end.

This writer is pointing out that there are other concerns that far, far trump
duplicated code -- and she's right. However she's not elaborating enough on
what it is a "wrong abstraction." We can be more precise.

The real offense when we factor duplicated code is the new dependency that is
added to the system. And this is what amateurs don't understand. Every
dependency you bring into your code architecture costs you and should be
judiciously introduced. De-duplication of code ALONE is _rarely_ a strong
enough reason to add a dependency.

If you want to be a professional programmer one of the most important things
to acquire is a distaste for dependencies. Every dependency you add should be
carefully considered (and lamented as a reluctant necessity if you decide to
introduce it). As a 20 year veteran in this industry having worked on myriad
code bases, I will always prefer a code base with duplicated code and fewer
dependencies than the other way around.

So back to the "wrong abstraction". When we compose systems, we are looking
for fewest dependencies and _stable_ dependencies. What I think the writer
means by "the wrong abstraction" is a "volatile dependency".

I'm trying to be precise here because a common reaction to terms like "the
wrong abstraction" is that wrong/right, its all subjective. The truth of the
matter is that it's not subjective at all -- the higher-quality system is the
one with optimally few dependencies and stable dependencies, these are
measurable qualities.

~~~
curun1r
Dependencies (coupling) is an important concern to address, but it's only 1 of
4 criteria that I consider and it's not the most important one. I try to
optimize my code around reducing state, coupling, complexity and code, in that
order. I'm willing to add increased coupling if it makes my code more
stateless. I'm willing to make it more complex if it reduces coupling. And I'm
willing to duplicate code if it makes the code less complex. Only if it
doesn't increase state, coupling or complexity do I dedup code.

The reason I put stateless code as the highest priority is it's the easiest to
reason about. Stateless logic functions the same whether run normally, in
parallel or distributed. It's the easiest to test, since it requires very
little setup code. And it's the easiest to scale up, since you just run
another copy of it. Once you introduce state, your life gets significantly
harder.

I think the reason that novice programmers optimize around code reduction is
that it's the easiest of the 4 to spot. The other 3 are much more subtle and
subjective and so will require greater experience to spot. But learning those
priorities, in that order, has made me a significantly better developer.

~~~
wellpast
> I'm willing to add increased coupling if it makes my code more stateless.

I like statelessness as a top priority. However I'm not sure how statelessness
ever comes into tension w/ coupling. Aren't they mostly orthogonal concerns?

> I'm willing to make it more complex if it reduces coupling.

Complexity = f(Coupling), in my definition. So an increase in coupling results
in an increase of complexity. Sounds like you have a different definition of
complexity -- I'd love to hear it.

~~~
curun1r
There's a few ways in which state vs coupling can play out. Often they're part
of the architecture of a system rather than the low-level functions and types
that a developer creates. As an example, should you keep an in-memory queue
(state) of jobs coming into your system or maintain a separate queue component
(coupling). By extracting the state from your component and isolating it in
Rabbit or some other dedicated state management piece, you've made the job of
managing that state easier and more explicit.

As for complexity, there are many different types. Coupling is a form of
complexity, but it's not the only one. Cyclomatic complexity is another and
one. Using regular expressions often increases the complexity of code. And one
need only look at the spec for any reasonably popular hash function to see a
completely different sort of complexity that's not the result of either
coupling unique paths through code. The composite of all the different forms
of complexity is how I'd define it since they all add to a developers
cognitive load.

------
tejohnso
"Programmer B feels honor-bound to retain the existing abstraction"

I think it's more often likely that the next person comes along and lazily
forces the most brittle, minimal change possible to make their new requirement
work without thinking about the larger context. Even if that means putting a
complicated conditional into the existing function, while leaving the existing
function name intact.

This isn't about honor. It's simply bad programmer behavior. It might be due
to lack of skill or experience, or maybe laziness, or lack of discipline, or
some other negative attribute.

~~~
ianamartin
Or maybe programmer B is trying to maintain a monolith of pasta for which
there is no documentation, the original programmer has long since left the
company, no one is entirely sure what the business rules of the program are
actually supposed to be, there are no tests, and you don't have time to
actually read and understand all 100k lines of code.

Management just needs this one little feature tweaked, so you dive in to where
you think that needs to go, make the new thing happen while absolutely
changing as little as possible because god help us if anything breaks.

Now imagine that you're not the second or third person to be responsible for
this, but maybe the tenth or so, and it's been going on for decades. And the
last person to work on it was really new to this language and didn't
understand its idioms very well, or vice Versace, and you don't understand the
language very well.

I greatly enjoy these conversations about code quality and maintainability,
but I've encountered it more than once where business constraints don't allow
you to even start to think about these issues. You have no choice but to
respect the abstraction you inherit because your job is to get in, get out,
and pray you didn't break some edge case that no one would ever think of.

Or maybe I'm just an amateur/novice.

~~~
lukaslalinsky
You can always chose to change the surrounding code whenever you are adding
something to an existing mess, to make a bit less mess. I have heard excuses
about not being given time to do it from the management many times before, but
the fact is that you are the one who deals with the code, you need to make the
decision. If you are not comfortable doing some change, you are the one who
needs to be writing the tests or making sure you understand the code. It's not
some kind of separate task. It's essential for doing your job right, so you
should not expect extra time allocated for it.

Just adding your small change without touching the rest of the application is
usually the easiest way, though, so most people just do that instead.

The best rule I have heard is to always leave any code you touch in a better
state than you found it.

~~~
ianamartin
I don't think this is a realistic attitude. I've gotten my fingers burned more
than once by trying to take this approach.

The problem with web applications is that they are not self-contained. You
don't necessarily know who or what is calling what or how for some legacy
applications.

I can agree with the concept of improving code as you see it, but changing the
abstraction--the topic of conversation here--cannot be done willy-nilly for
any non-trivial app. You have to dig up all the people are, or were, or will
be counting on that abstraction, and that's a challenging thing to do
sometimes.

And then you have to provide a business reason for you to allocate the time
necessary to go read those 100k lines of code and understand them or write
tests for them. And you tell your boss this is a mess and needs serious work,
and he says, "Dude, this isn't rocket science, and we're going to burn this
whole thing down in a few years anyway, and all I need you to do is add this
one button in this edge case and don't break our standalone desktop app that
isn't going to get updated to understand this functionality."

You're idealism is admirable, and I don't disagree with it. But the reality of
the world is that if you work for a company whose primary product is not
software (hell, even sometimes when you do!) the priorities are about the
business functionality, not about the state of the code or the right
abstraction.

I just think it's a little disingenuous for people to have these conversations
in a vacuum and attribute anything from malice to incompetence to a person who
takes a different approach.

~~~
Anchor
Maybe I am being idealistic, but I think the sentiment in the discussion
concerns not only the ideal coding techniques themselves, but also the bigger
picture of business needs. That is, if we took the approach of keeping code in
somewhat decent shape, we would end up, in the long run, producing the whole
system more efficiently, and reducing the TCO of the system for the user and
the business.

I certainly hope that the discussion is not only about making the lives of the
programmers easier.

------
akud
The problem was programmers between B and X didn't refactor to update the
abstraction with their new understanding of the domain, and to better meet
likely business requirements.

When programmer A introduces an abstraction, it's not as if they are saying
"this abstraction is the one true way to represent the domain." The
abstraction is a tool. It helps cover some duplication, and ideally express
intent. But the tool needs to be updated when it's not useful.

------
chowells
If you found yourself agreeing with article, yet have ever mocked
mathematically-derived abstraction patterns like monads, it's time to
reconsider.

Math is, at its core, the study of abstraction. Things that have been found to
be good abstractions in math probably are good abstractions in programming.

Learn from history. Mathematically-derived abstractions probably are used
because they are the right abstractions.

~~~
lackbeard
I don't think math is the study of abstraction. It seems to me it's proving
truths about formal systems. Abstraction has the same purpose in math as it
does in programming, a tool to more effectively communicate ideas.

~~~
sebastos
Not sure I agree. Usually, mathematics is developed backwards - you have a
concrete question that you want to answer, and you reason that it can be
answered so long as such and such as true. The formal system is developed
post-hoc to give yourself a language to reason in, but you're really trying to
take the result you already "knew" to be true, and find the least restrictive
system description to which it still applies. Then you look for parallels and
generalizations. The abstractions are in a large sense the most important and
difficult thing to create, because you're creating a schematic and saying "if
you can rephrase your problem into these terms, then I already proved this
intuitively correct thing for you, so you don't have to worry about edge
cases". Algebra, geometry, calculus, etc. all follow this model. A lot of the
conclusions in, e.g. analysis are obvious once you impose continuity.

The object of mathematics is to produce abstractions that make proofs possible
or trivial. Sets, fields, groups, categories, functions, integers, reals,
complex numbers, quaternions - these aren't notational conveniences that are
introduced to make it easier to talk about. They're important because if you
have something with the properties of a group, you know some powerful truths
about it for free. If you change anything about the definition of group, the
set of truths you know changes. People have been grinding on what the
definition of a set should be for a century, trying to build the best possible
abstraction, and they have all of the same problems coders have. You assume to
much, it's not very general. Don't assume enough, there's nothing interesting
to say that's unilaterally true.

So I would argue that characterizing math as the study of abstraction is
largely fair. You "see" a result is probably true some of the time, and then
you try to find out just how general you can make that statement.

~~~
lackbeard
Thanks for the response.

> People have been grinding on what the definition of a set should be for a
> century, trying to build the best possible abstraction, and they have all of
> the same problems coders have. You assume to much, it's not very general.
> Don't assume enough, there's nothing interesting to say that's unilaterally
> true.

This is a great point.

------
amelius
You: "We can implement this new feature but doing it right would mean we have
to design a new architecture for the entire system, and that would take
several months. Or, we could just hack it, change two lines of code and be
done with it, but it might bite us in the future."

Management: "Change those two lines of code."

------
Almaviva
For many decisions in programming, there are tradeoffs, and relying on general
rules of thumb can be much worse than judging the specific case. And
heuristics are often interpreted differently by different people.

Such is this discussion, imo. It's very possible to over-abstract. If
something isn't extended, maybe it shouldn't be designed to be extendable, and
if it isn't configured, maybe it shouldn't be configurable.

On the other hand, abstractions are the heart of programming, and finding a
good one is what can make you much more productive and maybe even have more
fun. The feeling when I've abstracted something and got more out than I put in
is maybe the best feeling there is in this craft.

The difficulty is in finding the balance. I also dislike the rule of 3 as a
hard general rule. There are many times when it makes sense to abstract
something out of 2 uses, if there is a lot of code and clear separation and
commonality involved, or you anticipate more uses later. Maybe even
abstracting sub-problems out of one case makes sense, if separating the
problem cleanly into two parts makes it easier to reason about. There are many
times when it's a bad idea too; it depends on the specifics and it's a
judgement call based on experience.

(And, I would even say, there is room for personal taste in programming about
this; where the sweet spot lies may vary from person to person.)

------
rumcajz
I wrote about the problem here:
[http://250bpm.com/blog:36](http://250bpm.com/blog:36)

The fact that abstractions can often add complexity rather than remove it
should be taught in schools.

The problem with distinguishing between the two cases is that it often
requires business domain knowledge. And programmers, sadly, rarely care about
the domain logic.

~~~
TeMPOraL
> _And programmers, sadly, rarely care about the domain logic._

Oh, but we do. It's the one thing we want to know but we pretty much never get
from our customers. No surprises there - most people are not equipped to
express their "domain logic" in a way that is useful for automating it; quite
often they don't even understand it. Humans can hand-wave their way through
anything, relying on intuition, patterns and methodologies. They don't _need_
to understand what they're doing deep enough to express it as code.

And so what usually happens is that your client/manager gives you a nonsense
spec saying "this thing should do that when clicked, and that thing should do
something else". And, as a programmer, you then have to reverse-engineer the
domain abstractions from it; abstractions the people giving you the spec
probably don't understand themselves.

~~~
mbrock
That's why understanding the domain logic requires cooperation between
programmers and those who understand the domain.

It might not be that they don't understand the domain "deeply enough." Maybe
they understand it in the way that is relevant for their work, which typically
doesn't require formalization.

Making sure this collaboration actually happens is one of the big topics of
_Domain-Driven Design_. We can't just expect to be given a correct model;
teasing that out is part of our job. If the spec is nonsense, we should say
that and try to fix the process.

------
adajos
If only Programmer B had avoided that initial choice to preserve the broken
abstraction rather than abandoning it then and there.

He must have been practicing Deadline Driven Development.

~~~
protomyth
> He must have been practicing Deadline Driven Development.

or The Team Has A Hammer Development

------
nine_k
TL;DR: If you have made a mistake, return back to the previous known working
state, proceed from there.

It does not matter much which was the specific mistake, really: a bad choice
of an abstraction or something else.

 _Good_ abstractions help immensely; programming is all about them. Let's not
forget it, too.

------
mbrock
Richard P. Gabriel wrote about this in connection with the idea of design
patterns, e.g. in _Patterns of Software_. Here's a nice quote from the
beginning of a chapter:

> _" The room fills with cold, conditioned air; outside the heat hazes,
> filtered through greened glass windows: a new building hardly first
> populated. The speaker is wild-eyed, explaining new ideas like a Bible
> thumper. His hair is a flat-top; his mouth frowns in near grimace. He
> strides to my seat, looks down and says in a Texas drawl, 'and the key is
> simply this: Abstractions. New and better abstractions. With them we can
> solve all our programming problems.'"_

------
cpeterso
I'm reminded off ESR's "curse of the gifted" email, schooling Linus Torvalds
on the lack of modularization and code-sharing in Linux's driver code. :)

[http://lwn.net/2000/0824/a/esr-sharing.php3](http://lwn.net/2000/0824/a/esr-
sharing.php3)

~~~
hinkley
Oh my. I recall reading this years ago. I had totally forgotten that Eric
basically tells Linus to grow up and learn to use version control. Now we're
all using Linus' version control.

------
agentultra
I hate needless abstractions for the sake of removing duplication, DRY, etc.
It's mad. Most of these languages don't have proper macros and so you end up
going through these logical contortions to achieve a particular pattern of
behavior... and it's just noise. Anyone who comes along can look at that code
for hours and never know whether it performs any work (or even what that work
is).

A little duplication is fine until you figure out what the real problem is.
The problem is usually in your data. Maybe it's not structured properly or you
need to simplify the steps to transform it at an earlier stage.

A good specification will go a long way to reducing the desire to introduce
hapless "abstractions." An abstraction, in the mathematical sense, will hold
over the domain and introducing a new one merely allows you to manipulate
objects on the lower level using new algebras, predicates, etc.

Code abstractions are often the leakiest abstractions. Especially the kind the
author is talking about. Avoid them. Even if you have a little bit of
duplication. Only start worrying about that duplication when it starts
spanning compilation units/modules/whatever and is actually causing problems.
Then look at the data and figure out how to structure it so you don't need
that code all over the place.

~~~
Chris2048
I think lack of things like macros is the bigger problem. There are a limited
ways to abstract, or represent duplications, in a lot of languages. Sometime a
comment in an issue tracker might help.

------
grandalf
It's funny how often I've made this argument with even fairly experienced
programmers and they seem to have a visceral reaction to the code being "less
dry" than it could possibly be.

Similarly, once a codebase uses several very wrong abstractions, it becomes
significantly more confusing to work on, exponentially increasing cost.

The temptation to use a mature library as a dependency is very strong since
time is initially saved. When that library introduces the wrong abstraction,
the consequences can be severe.

I typically argue for (at least) creating the correct abstraction and having
it wrap the mature library to constrain or properly name its behavior, and to
make it less strongly coupled to the rest of the system.

It doesn't just take experience to realize these sorts of things, it takes a
willingness to question one's own code and imagine how it might have been done
better, or how it might appear to someone who didn't write it.

~~~
TheOtherHobbes
It's all a bit no-true-Scotsman though.

abstraction != dry bad abstraction != anti-dry

Abstraction is primarily about separation of concerns, not about avoiding
repetition. Drying out code that's repeated all over isn't the same as
creating a formal abstraction for some element of the overall logic.

Which is why

>once a codebase uses several very wrong abstractions, it becomes
significantly more confusing to work on, exponentially increasing cost.

And drying out code makes it easier to maintain, but it doesn't guarantee that
the architecture isn't a mess.

The problem is perhaps that CS teaches algos, and sometimes it teaches design
patterns. But there's almost no useful theory of abstraction design.

Design patterns are more or less as good as it gets, and all they do is give
give you a cookbook of stock formulas to try.

Beyond that, there's no useful way to _reason_ about abstractions, test them
for domain fit, or rate them for elegance and efficiency.

~~~
grandalf
Precisely. Very well put.

If anything, some developers use design patterns as a grab bag when solving a
problem... a better approach is to model the solution and then be ready to
"back in" to a design pattern upon noticing strong similarity or observing
that the design pattern is a bit more abstract way of doing the same thing.

Because of the tendency to pick a pattern first and design for the domain
later, many instances of design pattern use in the wild are subtly (or not so
subtly) incorrect.

> Beyond that, there's no useful way to reason about abstractions, test them
> for domain fit, or rate them for elegance and efficiency.

Very true indeed. Looking at a design through the lens of coupling and
testability and modularity is a good start and can reveal many problems, but I
think the real gotcha has to do with naming: Once we name a
concept/abstraction we are likely to reason within that abstraction, and we
rarely consider whether we are stretching it too far, or if it is even that
_thing_ anymore.

~~~
douche
A graphic I found very enlightening in the past year is here:
[http://blog.thecodewhisperer.com/images/age_old_battle/virtu...](http://blog.thecodewhisperer.com/images/age_old_battle/virtuous_cycle.png)

from this blog post: [http://blog.thecodewhisperer.com/2013/12/07/putting-an-
age-o...](http://blog.thecodewhisperer.com/2013/12/07/putting-an-age-old-
battle-to-rest/)

There's a lot of other good stuff there too, when you start digging through
the archives.

Generally, I find that I've got to build a prototype first, which is quick and
dirty and ugly, but works. Then iterate as the structure emerges. Some things
stay in flux, and it's okay to leave them messier, but eventually the code and
functionality settles out and commonality arises. Quite often, I don't
necessarily know what I'm doing going in; I've got a general goal, but until I
experiment a bit, the best way to get there is unclear.

One of the joys and pitfalls of a multi-paradigm language like C# is that
there are so many different ways to skin a cat. You can pick and choose
procedural, object-oriented, functional, or some bastard mishmash of all of
the above and more.

------
lmm
This may be obvious, but when working on Java I've found that this applies not
just to classes but also to methods. When refactoring a poorly-factored class,
often the most effective way forward is to inline all the private methods
(introducing a lot of duplication) and then factor out the duplication.

~~~
mannykannot
It is even more effective when you are faced with a bunch of classes, most of
which do nothing but delegate their responsibilities to others. I have a case
in front of me right now.

Data-flow analysis was a method that was swept away (somewhat unfairly) on the
grounds of being too procedural and not OO. One of its key tenets was that
once you had discovered the essential flows and interactions, you should
discard the problem-space partitioning that helped you find the graph, and use
the graph to partition the solution space (which might well result in the same
partitioning, but not always.)

------
tempodox
There is another koan I would recommend to the interested:

    
    
      Quantity is easier to control than complexity.
    

I propose that “the wrong abstraction” is “complexity without benefits”. In
light of the sentence above, duplication is clearly the lesser evil.

------
mannykannot
While I have no objection to the notion that duplication can be the right
choice in some circumstances, and I completely agree with the author in
thinking that pressing on rather than backtracking and taking a different path
is a source of much complexity, his pseudo-example misses a third alternative:
instead of (re)introducing duplication wherever the current abstraction is
used, derive a generic abstraction covering the common aspects of the previous
and new requirements, and use that in implementing two new abstractions, one
to replace the old, and the other for the new cases.

------
JDDunn9
A 3rd option would be to simply apply good programming standards like a
function doing one thing and only one thing.

~~~
rco8786
At some point, you're gonna have to introduce some logic into your program,
and that logic is gonna have to be in a function.

------
daxfohl
Meh, it's an easy situation to get out of though. Especially with a typed
language and decent IDE: right-click, "inline function"; check your git status
to see what changed, go to those files and remove unused branches (which a
decent IDE should hightlight and provide 2-keystroke fixes for). 2 minutes,
done.

Grotesquely replicated code is a much more difficult situation to resolve.

I'm guessing the OP primarily uses untyped languages. I'd offer that _that_ is
the core of the problem, not the abstractions.

------
protomyth
Isn't a bit of this governed by which programming paradigm your language uses?
I get the feeling the answer and approach are much different when comparing
Forth to Java.

~~~
drostie
Yes. In theory there is a programming paradigm which fixes this, called
"aspect oriented programming." I haven't seen a really accessible aspect-
oriented system however.

~~~
collyw
Decorators in Python are pretty much the same idea. I see them used quite a
lot in Flask and Django.

------
jarpineh
I read somewhere, but have forgotten where, that having doing things same way
twice is ok, only when you're second time repeating yourself, think about
using an abstraction.

Or to say it more succintly, duplication is not a problem, triplication is ;)

I have found that following this simple rule I allow the development to go
forward, and can go back to it when I have better knowledge of the system at
large. At later date I might have a better luck in choosing the correct
abstraction.

------
mojuba
Or, in terms of OOP, wrong abstraction is when you have subclasses with pretty
much everything redefined in each.

I don't think this is very common though. You will more likely see a lot of
duplication in poor legacy code, and not because the coders were so clever,
but quite the opposite: it's when they barely even understand how to extract
abstractions.

~~~
arethuza
Or baroque structures of abstractions where you have a real problem finding
where stuff actually gets done - sometimes combined with a need to base all
application classes on a common shared abstract base class even though they
really don't share anything.

------
guscost
One thing I've put a lot of thought into recently is what parts of a program
_should_ typically be abstracted, because when you come across them they might
be your best bets for big wins.

Top of the list: constants should probably all go in special structures with
the goal of guaranteeing consistency and making them easy to understand at a
glance. Everyone might know that charge code "X" means a check payment, but
what happens when a new developer looks at that code with "X"s everywhere?
It's more verbose to use constants.chargeCodes.CHECK_PAYMENT but nobody will
misunderstand what you mean, and your IDE will be able to verify that your
codes are valid. That's worth an awful lot of extra characters.

Bonus win: when your legacy backend finally gets upgraded, you have the option
to change to a new code for check payments, easy as pie.

------
hex13
This is the old truth - premature abstraction is a root of all evil (I know
original quote was "optimization", but isn't an abstraction just some
"architectural optimization"? ;)

On the other hand: I often see _copy pasted_ code which have these same
characteristics author wrote about: "Another additional parameter. Another new
conditional. Loop until code becomes incomprehensible."

If somebody writes too many conditionals, parameters and creates
incomprehensible loops, then he will do it always, no matter whether in
abstracted code or in copy pasted code...

The problem is that most of programmers are not skilled enough to write good
code.

So sometimes there is a need for refactor wrong abstractions in legacy code,
sometimes there is need for cleaning tons of copy-pasted code and make good
abstractions... Either way - maintaining is hard.

------
dandare
Firstly, there is no such things as perfect abstractions, hence the The Law of
Leaky Abstractions. Secondly, without abstraction we would be writing in
machine code. So unless the authors enlightens us on what makes an abstraction
wrong the whole discussion is a waste of time.

------
bograt
My own observation is that programmers have a marked tendency to assume that
because two methods have similar or identical behaviour, that this implies
they should share an implementation. In large code bases, its common to find
methods that have identical implementations 'by chance'. By which I mean that
the commonality is a side effect of requirements that could readily change.

In my opinion, the best defence against this is good documentation: if a two
methods have clearly documented behaviours, then even if their implementations
have been fused, a subsequent programmer will have more context (and more
confidence) reduplicating the code in response to further changes.

------
ska
On the other hand, moving to the right abstraction is one of the most powerful
ways to improve a code base. Often the right abstraction cannot easily be seen
from existing code - they come from the domain.

~~~
kaeluka
Someone once told me: "Just looking at Java code, you'd never find monads."

------
andreasklinger
In my team we usually work with the principle of 3

"if it's non obvious wait for 3 cases before abstracting it"

------
_pmf_
The same applies to dependencies. I've grown very reluctant towards
introducing a dependency for the sake of deduplication. I'd rather have
isolated modules than a de-duplicated Big Ball of Mud.

------
mattiemass
It's taken me perhaps longer that t should have to get here, but boy oh boy do
I think this is spot on.

------
dreamdu5t
False dichotomy. You can avoid code duplications while using the "right"
abstractions.

~~~
hnbrox
expand?

