
Do We Worship Complexity? - ScottWRobinson
https://www.innoq.com/de/blog/do-we-worship-complexity/
======
wellpast
I was interviewed recently. I was asked to implement a solution in Java. It
was essentially a stack-based computation engine. The interviewer posed a
question and there were enough tacit hints for me to see what he really
wanted: an "OO" approach using a type hierarchy + command pattern. He offered
unsolicited prompts and I took them: I coupled the command objects to the
engine. He liked that.

What we ended up with was O(n) _classes_ (where n = the number of computation
operations).

I remained in Java, but took a functional and dynamic approach yielding a
three-line implementation of the engine where each command was simply a
function. All the tests passed.

I was kind of holding my breath at this point. What will he say? We just went
from this huge implementation to three lines. I thought he was going to love
it. The crazy simplicity of it all.

But to my draw-dropping surprise he muttered something about adding new types
and how he liked that the commands were coupled to the engine.

Is this complexity worship? Or what is this? It's almost as if his brain was
refusing to see the solution. Like it couldn't possibly believe that the
solution could be that simple. So it just ejected it, put a big black
censorship block over it.

I'm currently reading a book on brain hemispheres. Apparently experiments have
shown that the left hemisphere will completely and blatantly reject obvious
evidence even if the right hemisphere 'has a clear counter proof'. Sometimes I
think our industry suffers from an abundance of left hemispheric dominance. Or
something like this...

~~~
exabrial
I was asked in a Google interview to randomize a List. I wrote
Collections.shuffle(list); The interviewer was not pleased.

~~~
harryh
It's normal to not be pleased when a candidate fails to understand a question
(or exhibits smart-ass behavior), though in this case the easy and correct
thing for the interviewer to do is to say "OK, how is Collections.shuffle
implemented?"

~~~
exabrial
I think it's stupid to ask an interviewee to demonstrate something they should
never do at their job.

~~~
xyrouter
I think it's stupid to assume that a decision regarding time complexity would
never come up at their job. Where I work right now, and especially in Google
(not where I work right now), it is important that one understands time and
space complexity issues well. The data sizes are huge and optimization skills
are highly valued.

Sure I can teach constant vs. linear time. But what incentive or reason do I
have to spend time teaching these fundamental concepts when I can just hire an
engineer who demonstrates the understanding of basic CS at the interview time
itself?

Given two candidates with all things equal except that one demonstrates the
understanding of CS fundamentals and other does not, why would I want to hire
the second person and spend our time teaching him those concepts?

~~~
pbhjpbhj
>why would I want to hire the second person and ...

Someone commented on here recently that they would take motivated candidates
over knowledgeable candidates. That's one reason. I can easily think of at
least two others.

There are balance points here that vary according to all sorts of things.

~~~
xyrouter
I said, "Given two candidates with all things equal except ..."

I mean if there are two candidates who are equally motivated and are more or
less equal in all things except that one is strong at CS fundamentals and
another isn't, is there a good reason to reject the candidate who is good at
CS fundamentals.

In many hiring decisions, I am faced with a similar choice, and I go for the
person who is good at CS fundamentals. If two candidates are good in all other
ways, then the understanding of CS fundamentals becomes a tie breaker. I see
no rationale for selecting the guy who does not demonstrate his strength in CS
fundamentals.

------
pcwalton
Honestly, I think we as programmers have the opposite problem. We value
simplicity so much that we neglect planning for complexity, because we think
our simple, elegant solutions will last forever.

It's really fascinating to study modern CPU design. Modern high-performance
CPUs are horrendously complex, with the very notion of superscalar
architectures and pipelining resulting in guaranteed complexity explosion.
Yet, as far as we know, _there is no way around this_. You need pipelining and
superscalar execution to get adequate instruction-level parallelism in real-
world code. Unless you want to use microcontrollers for everything, that
complexity must exist.

Compilers are another example. Many people who go to implement compilers read
the Dragon Book and think that all the complexity in GCC and LLVM is needless
bloat. Then they quickly discover that they can't compete with them in
performance.

It is of course desirable to avoid complexity where possible. But all too
often the response that we as engineers have to discovering that difficult
problems require complex solutions is to become defensive, stick our head in
the sand, and stand by our simple "solutions". This is how we ended up with
crufty Unix APIs, the security problems of C and C++, the pain of shell
scripts, and so forth. We need to learn how to accept when complexity is
necessary and focus on _managing_ that complexity.

~~~
collyw
Not in my experience. I see far more engineered overly complex solutions to
simple problems than the opposite.

~~~
aiCeivi9
And then people say: lets rewrite it, the old solution is overly complex and
has a lot legacy code that no one uses, we can do better. They imagine simple
& perfect solutions because they underestimate complexity of real world
problems, just like parent said.

~~~
collyw
I would say rewriting is OK if you are eliminating code while keeping the same
functionality.

Obviously that is a generalization and there will be exceptions, but as
someone else said in this thread "code is a liability not an asset".

------
ken
It's interesting to me that Parkinson's Law wasn't originally conceived of in
the context of software projects, yet that's the only place I hear it applied
today.

I've been working outside of software recently, and noticed to my delight that
this Law hasn't applied at all. In one case, I was hired for 3.5 days of work,
and we got finished after 2.5 days so we were sent home early -- nobody was
dragging their feet to make it last 3.5 days. In other cases (more common),
we've temporarily had too many people for the job at hand, so the team lead
said "Just wait", and we do nothing until there's more work ready for us.

Why have I never heard of any software team ever saying "There's nothing for
you to do right now, so just wait"? My first thought was the endless supply of
bugs, but that can't be right, because I've never heard of a team lead saying
"We have no work for you today so go fix bugs for a while", either.

It really does seem like every software team manager thinks that the proper
amount of complexity in a system is perennially $(current_complexity + 1). The
cases where program complexity approaches a constant asymptote (like Redis and
perhaps SQLite) are so rare as to be notable. They're also frequently
mentioned as being developer favorites.

Maybe the field just needs another 50 years to mature.

~~~
repsilat
> _Why have I never heard of any software team ever saying "There's nothing
> for you to do right now, so just wait"?_

In addition to Parkinson's Law, there's this one: "A poem is never finished;
it is only abandoned."

There's never nothing to do, because we can always improve things. And to a
business or to a manager, "just wait" costs about as much as "work on
something of little importance" but provides less benefit. It might be
different if programmers were all on zero-hour contracts.

(OTOH... Firing engineers could work. I wonder why sites that seem "done"
don't decrease their payrolls. Pride?)

~~~
ken
I disagree that "we can always improve things". That only works in a few
fields.

When Tom Wolfe was finishing up a new novel, adding more writers wouldn't
improve the story. When a cancer patient is undergoing chemo, adding more
physicians this afternoon won't improve the outcome. Adding cooks to the
kitchen of my favorite noodle shop isn't going to improve the noodles one bit.
Adding more actors to a film's shoot schedule isn't going to make it go any
faster.

In all these cases, even if you gave me 100 more skilled people for free for a
week, I'd tell you that we don't need them, and it would actually hurt us for
them to participate. Changing the plan or going off-plan has a real cost.
Mature fields like civil engineering have a great track record _because_ they
don't just let extra people make contributions at any point in the process.

BTW, according to Wikiqote, the correct (and unabridged) quote is: "A work is
never completed except by some accident such as weariness, satisfaction, the
need to deliver, or death: for, in relation to who or what is making it, it
can only be one stage in a series of inner transformations."

For programming to be a mature field, "the need to deliver" must be a
necessary component, and "satisfaction" should be the goal, not merely an
"accident". We can't utilize the process of a 19th century French poet and
expect to get results like 21st century civil engineers. This was a guy who
(according to his Wikipedia article), "around 1898, he quit writing
altogether, publishing not a word for nearly twenty years."

------
meuk
People definitely fetishize complexity. I often mention that I like an
intellectual challenge at interviews, and the interviewers often mention 'oh,
but this project is _very_ complex' like they are saying something naughty.
(Once, the interviewer followed up with "we're using _algorithms_ ). I suspect
they are usually right, and the codebase is a pile of unnecessary design
patterns (because those are what educated developers use, right?).

Where I currently work we're using Azure to do a shitload of computations. At
the same time, many modules don't even bother to throw away intermediate
calculations. They literally have a _giant_ array of them, and save every
result, for each step, even if it's not necessary.

But hey, the project is seen as a huge success, because it's _so complex_.

~~~
windwake12
More Azure billage to the customer, more kickback from MS.

------
Sharlin
When talking about software complexity I'm always reminded of the old quote by
German general Kurt von Hammerstein-Equord but attributed to various military
leaders:

    
    
      I divide my officers into four groups. There are clever, diligent,
      stupid, and lazy officers. Usually two characteristics are combined. Some 
      are clever and diligent – their place is the General Staff. The next lot
      are stupid and lazy – they make up 90 percent of every army and are 
      suited to routine duties. Anyone who is both clever and lazy is qualified 
      for the highest leadership duties, because he possesses the intellectual 
      clarity and the composure necessary for difficult decisions. One must 
      beware of anyone who is stupid and diligent – he must not be entrusted 
      with any responsibility because he will always cause only mischief.
    

Clever and diligent developers devise complex solutions to complex problems,
which may often be good enough.

Stupid and lazy developers can be entrusted to come up with simple solutions
to simple problems.

Clever and lazy developers are able to find simple solutions to complex
problems, a very desirable trait.

But stupid and diligent developers, given the chance, _manage to implement
complex solutions to simple problems!_

~~~
api
[http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer....](http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer.html)

I have been saying for many years that overengineering is the plague of modern
software. Almost everything seems far more complex than it needs to be.

~~~
Sharlin
Yes, definitely. There are too many diligent and too few lazy programmers. Too
many devs take pride from coming up with complicated, overengineered
abstractions – after all, if they can manage the complexity they must be
really smart, right?

~~~
irrational
>Too many devs take pride from coming up with complicated, overengineered
abstractions

This is what I feel about the current JavaScript ecosystem. I feel like web
development became popular so programmers from other disciplines (C, Java,
etc.) jumped in and found it to be too simple so they've hijacked the ship and
created a new JS ecosystem that is as complex as their old environments.

~~~
TimJYoung
Coming from a (primarily Windows) desktop development background, I can
definitely tell you that JS/HTML/CSS and the DOM is some of the most complex,
over-engineered stuff to ever see the light of day. It _seems_ simple, but
that's because it _was_ simple initially, and now it's grown into this monster
that has very little design at all and is just piles and piles of APIs and
code added on top of other APIs and code.

Desktop development is blessedly simple, in contrast. The problem is that
desktop development APIs and UI toolkits have been neglected for many years in
favor of web and mobile, so now desktop development is _also_ a fractured mess
because much of it is outdated or hasn't kept up with current graphic design
and UI standards. But, at its core, desktop development doesn't struggle with
oddball concepts like promises and other bizarre features that were introduced
to overcome issues with the overall design of the language/environment.

------
klagermkii
I think it could be that rather than worshiping complexity some people find it
hard to value or respect simple things, and that that leads them on a path to
"accidental" complexity. Like when someone find out that a rainbow is just
water refracting light, or the sun is just a big ball of gas... that it takes
some of the coolness out of it. There could be a tendency like that to find
simple systems disappointing.

Or it could just be that people have a certain mental budget for maximum
complexity, and they'll try and make sure they spend it all in the belief that
it will cover more uses.

Or it could be that complex systems tend to stick around because they are far
more immune to random management changes because everyone goes "oooh, we'd
better not touch that". Simple systems might becomes the victim of their own
ease and get subsumed by a more complex monster.

~~~
rossdavidh
I have heard co-workers say that the code where we were working used to be
dissatisfactory because there just wasn't very much of it, but now it's much
better. My mind was boggled. Less code is almost always better (obviously
there are exceptions, but rarely in production, in my experience).

~~~
collyw
I cringe reading this. I take far more pride in removing code while keeping
the same functionality than writing new code these days.

------
GorgeRonde
We also worship simplicity. I like to think of things as quadrants. Imagine a
2x2 matrix with columns and rows titled as such: [simple, complex] [easy,
complicated].

They are quite not the same since one dimension denotes the complexity of the
thing whereas the other denotes the complexity of the act of building,
maintaining and evolving the thing in question.

We should always strive for easiness, however complexity shouldn't always be
avoided. Quite to the contrary, staying away from complexity locally often
leads to that complexity being sprayed on a global level, and that's when it
turns into something complicated.

I have too often seen "complex" code, i.e. code that works on "complicated"
datastructures such as trees and graphs be discarded in favor of solving the
problem "simply" and directly, which means by disseminating the problem's
logic accross the codebase and with multiple, gradual bugfixes because the
problem being intrasequely complex is underspecified and cannot be tested in
isolation of the system.

Of course the same people that are baffled by "complex code" and think
simplicity sums up to looking away and not anticipating future needs, are the
same who advocate it. Actually, they like good design and typography, focus
more on indentation than datastructures and generally have a taste for
nitpicking with as many subtle details as they can come up with, not seeing
past the filter of their own opinion about what simplicity and complexity
entail, and of course that makes the social process of building code a slow
nightmare that does not converge.

I sound harsh I know. What I want to point out here is that by denigrating
complexity in favor of simplicy, we may get rid of medieval savants, but we
open the door to plain idiots disguised as zen masters.

~~~
pdimitar
You actually sound diplomatic.

Wanna hear something harsh? Most programmers I ever worked with for 17 years
of career do not deserve the right to touch a keyboard; they should work on
farms. That would be [somewhat] harsh.

You are on point with everything you said. Ego, strong opinion enabled by a
secure job position where no amount of professional failure will ever get you
booted (because you are isolated from the business outcomes), echo chambers of
fellow bros who think like you, and plain old fear of change is what drives
most humans -- and programmers aren't an exception.

IMO it's high time some formal certification and legal liability to be
introduced to our profession. Also, pick 5 imperative and 5 functional
languages and make them "official". People love their religious language wars
but it has to stop at some point because billions of bucks are being wasted on
25-year old egos.

~~~
GorgeRonde
Keeping on being diplomatic, I don't think it's an ego problem per se:
instead, the egos fight over pointless things such as coding style. Don't get
me wrong, I've nothing against adopting a coding style nor being told I
misindented code by mistake. I'm fine with this, this is teamwork. But when
the coding style keeps evolving, is not the same depending on the code base,
when you have to take into account who your reviewer is in order to know which
style to adopt, and when in addition you have very "diplomatic" ways (or, to
put it more bluntly, when you have a dominated and introverted personality)
you just end up being a punching ball for the whole team and serve as an
indirection layer so that the biggest egos in the team can fight each other
through proxy wars.

As for a formal certification, here in France "engineer" is a state-certified
status. There is no legal liability I can think of (there might be some when
you're engineering bridges, but I have not heard of something similar
concerning software). To be frank, this status is bullshit. At work I'm using
a payment API from a local shop. The CTO comes from a top rated school (the
french Calltech), yet their API does not ensure the reception of async
notifications like if communication failures wasn't the crucial characteristic
of a computer network. In addition to that, they are unable to provide a
dedicated infrastructure to their big customers and ended up implementing a
rate-limiting system on top of their absolutely business critical payment API
(and boasting about it in blog posts). Last time I checked my company
experienced a 80% drop rate in payments due to errors or the rate-limit being
exceeded.

I say "last time" because I haven't been to work in months, my sick leave
being extend month after month by my psychiatrist. I worked a lot these past
couple years. In 2017, I think I averaged 70h per week. Actually, I have no
idea. I just know I worked a lot of 80/90h long weeks that year. What's
certain is that my hourly pay rate dropped under the legal minimum, which
isn't surprising since I have the lowest salary in the team (a little less
than 40k€/y). Everybody comes to me to fix their shitty problems, and when I
push PRs to avoid them code aesthetics and "simplification" takes over in the
code review and it takes foreeeever, and I'm never guarded against bug i may
have introduced, only indentation and inessential shit like that, so I have to
review my code on my own, and these PR never see the light of day and then I'm
suddenly considered the master of unfinished work, almost in a self-satisfied
tone by people who work almost twice as less as I do and earn almost twice as
much. Also I'm not allowed to work remotely (but everybody else in the team
is) and I have to be at work before everybody else, and the fact I sometimes
come two hours earlier is never taken into account. Oh and my sleep and
medication is monitored and I have been subject to very condescending and just
borderline illegal remarks about it.

I'm just insanely butthurt to be honest. I could just as well start
complaining about being "talked to like a dog" (these are not my words, but
what two persons independently told me about the way I was being treated in
the team).

And you know what, they are all very concerned about keeping the project's
complexity low. Coming from a Clojure background I exactly know what this
means and what's wrong with their approach, i.e. they see complexity as
opposed to simplicity, but what's pertinent is to actually oppose it to
easiness. See Rich Hickey (Clojure's inventor) "Simple made Easy" seminal talk
for an in-depth overview of software engineering from this perspective.

To give a concrete example of what this misunderstanding leads to, let me
compare the simplicity of markdown, praised by many minimalism hipsters and
whose main force is to make us forget its limitations, fascinated as we are by
its fixed-width typographical beauty, VS the simplicity of acknowledging that
mixing text with control characters (the old-fashioned unix way) is not a
sound way to build complex things at a large scale.

Anyway, since then, I've become extremely wary of those who advocate
simplicity just like an early or "private" christian would dislike Church (or
what it has become).

I'm now getting back to working on extending Clojure's compiler to experiment
with the idea of integrating the notion of IDE and editor directly at the
language level, but deep down I'm not sure I can continue with this kind of
bullshit career, where ability to handle complexity is not just disregarded,
but is punished.

~~~
pdimitar
Yep, you ended up like the punching ball of the team. Everybody has
frustrations and somehow you ended up being that person who everybody
mistreats and vents to. You have to end it.

Usually I would tell you "leave your job" but me being from Eastern Europe and
not in the cozy enabling environments of most of Europe... I realize you might
not have the choice.

I don't know your situation so what I can recommend is probably misguided.
Still, here goes, in case it can help you (and in case it's not obvious):

1\. Leave the job if you can. You already are not working and are apparently
still collecting some kind of paycheck. I am not sure any amount of medication
of psychiatrists will help you overcome the fear of eventually coming back to
that hell you have been working in. How do you feel about that prospect?

2\. Take a creative break. It seems you already are doing something along
these lines by working on things outside your immediate duties. Do they
fulfill you? If not, definitely just stop. I lied on my ass for 2 months
before actually going back out there and starting to get stuff done. Sometimes
you just need it. Also, programming in your spare time isn't always relaxing.

3\. Talk to people about your situation -- not only to people who are paid to
listen to you though. That's a very broad advice so apply it as you like. ;)

It's OK to be butthurt/salty in this situation. We aren't angels or saints,
these things can and do get to us. And you have been wronged, many times.

I faced the same. I actually come across as quite manly and assertive to a lot
of people but that's mostly my appearance and my attitude of gettings things
done without tarrying on petty differences. I strive to never argue
emotionally or engage in yelling competitions with people and they very often
mistake that for me being a pushover. What I usually do is: if somebody is
becoming an obstacle, I just go to a higher manager and talk to them about it.
If they don't care or have the wrong idea then I just reduce my efforts in the
said work significantly and just move from paycheck to paycheck. Done that
many times and now I am suffering financially for months -- very severely! --
because I want to choose a workplace where things are not like that. It seems
the mythical "cultural fit" is not BS after all...

What I take from your gently shared pain is that you are not a conflicting
persona. That's okay. You don't have to be. Do your best to find an
environment where you don't have to fight daily with people. They do exist.

------
drawkbox
Engineers are supposed to simplify complexity, taking something complex and
making it simple.

There are many engineers though that like to over-complicate and add
complexity because it makes them look smart or they think they are expected to
create complexity.

Complexity through simple parts is ok, but overall the job of an engineer is
to take complexity and break it down to simplicity and simple parts.

In the game dev world for instance, Unreal used to be needlessly complex,
still is a bit, then Unity came along and made things simple, so Unreal then
looked to make things simple.

Or in the web dev world, a framework might abstract away the underlying
standards and add complexity on top to seem simple, but actually make the
domain more complex with more to learn and push for developer lock in to the
framework over basic standards and simplicity. The first version of .NET with
WebForms was an example of this, other web frameworks can be seen as this as
well.

Lots of engineering people like to look smart by managing complexity but it
doesn't always need to be so complex. Sometimes time pressure can create
complexity and thus technical debt due to the complexity. Some simplification
is misguided, a one liner that isn't understood 6 months later is not
simplifying.

An engineer that takes something simple and makes it more complex for no
reason other than job security or to look smart is the worst kind of engineer.

~~~
flukus
> An engineer that takes something simple and makes it more complex for no
> reason other than job security or to look smart is the worst kind of
> engineer.

I don't think most people set out do do this, most complexity comes from
attempting to simplify things. A develop will see two similar bit's of code
and try to stuff the commonality into a base class. In their minds it's
simplified because there is less code but then when someone else picks up the
maintenance it's more complex because now the logic is distributed. Then the
requirements of each diverge slowly and over time each simple change is hacked
into more code paths of this distributed logic. Eventually most simple changes
take time because you have to be sure you're not breaking other potential code
paths.

On a more macro scale, complicated architecture is another symptom of this
over optimistic pattern matching that humans are susceptible to.

~~~
pdimitar
We the programmers sometimes get too caught up on extracting seemingly common
code that we don't stop for a minute and think "but does it really make sense
to extract it?".

It's a very classic problem and it's something that code reviews and pull
requests for anything you do are such a good idea -- provided that your team
is not a total echo chamber of course.

------
tabtab
One word: "Plastics", no, actually it's "Job Security". There is an incentive
in ANY profession to recommend or encourage more of your own labor. For
example, a surgeon is more likely to recommend surgery to solve a problem than
a non-surgeon.

This bias isn't necessarily intentional: human nature (competitive evolution)
just naturally pushes us to interpret the world in a way that makes ourselves
as valuable as possible.

In IT, complex solutions that we create or learn keep out competition. "Only
Bob knows how to fix this monstrosity" is a common pattern.

Simplicity should be added to the employee evaluation process. This includes
parsimony in both features (YAGNI) and in how the features are coded.

Further, avoid "eye candy" UI gimmicks that add complexity and fragility. End-
users often love them, but they are often a longer-term maintenance headache.
Beauty ain't free.

~~~
majewsky
> This bias isn't necessarily intentional: human nature (competitive
> evolution) just naturally pushes us to interpret the world in a way that
> makes ourselves as valuable as possible.

For the surgeon example, the simpler explanation is that, given a particular
ailment, they just know only the surgical treatment, not the alternative drug-
based therapy.

Or if they know, they know it way back in their head and don't think of it
unless explicitly prompted, whereas the surgical procedure is probably recent
experience.

I could also see this apply to IT. Given some problem, maybe there would be a
simpler solution if I implemented this particular problem in, say, Python, but
I'll implement it in Go because I work with Go a lot and all its idioms are
much more salient in my mind.

~~~
tabtab
That's a variation of "if all you know is hammers, then everything looks like
a nail". That's certainly an aspect of it, but I'm not convinced it's the
entire story. It would be interesting to see studies on how prior
careers/specialities affects one's decisions in new specialities.

------
rossdavidh
I was with him until the very end, where he said that the cloud,
microservices, etc. were the simpler approach, that other companies (not
Google or Amazon) are trying to avoid.

What I see is the opposite; companies that would do perfectly well with a
server or three, and a simple monolith, trying to instead use Kubernetes and
an Amazon service salad because that's what Amazon and Google say to do.

~~~
taeric
I think it is worth knowing how you would grow into one of those solutions.
That said, I do not know why you would try and start there. And, yes,
migrating between solutions is always difficult. However, it is also the kind
of problem only successful companies ever should have.

------
starbugs
I think a massively underutilized core strength of developers is that we have
a working thought process in front of us in the form of code, which can be
used for more than just running a program.

In my experience, complexity in the code often reveals not only technical
problems, but frequently also points to product and business issues. Exploding
complexity and long iteration cycles are often a consequence of bad business
decisions. Looking at the points of exploding complexity in the code can
sometimes help identify these issues.

The key is to make the disciplines work together. Business and product
decisions should not trickle "down" to developers. Instead, there should be a
working feedback loop. For that to be achieved people need to be team players
and talk to each other frequently. It also helps to have a 10% or so
generalist in each team member's head to guarantee a shared understanding of a
high level perspective.

I know, I know — this is all a given in agile methodologies. But in reality,
it's unfortunately rarely executed that way.

------
exabrial
One only has to look at any Node.JS codebase for the answer to your question.
On the Java side, the Spring Guys love complexity as well:
[https://docs.spring.io/spring/docs/current/javadoc-
api/org/s...](https://docs.spring.io/spring/docs/current/javadoc-
api/org/springframework/web/servlet/support/AbstractAnnotationConfigDispatcherServletInitializer.html)

~~~
wesleytodd
Complexity is a choice in most programming ecosystems. The most simple
applications I have ever worked on are node applications.

In some ecosystems it is more consistently complex (Java?) but others just
have many approaches with different complexity levels for you to choose from
(node?).

~~~
BigJono
Yeah, agree. I've worked on both very simple and intuitive Javascript
codebases, and absolutely heinous ones that should honestly just be abandoned.

I do think JS has a lot more variation in complexity than other ecosystems
though.

------
Yahivin
Everything is always at the edge of maximal complexity. If it were any more
complex it would collapse under its own weight. If it were any simpler someone
would say "it would be so easy to add..."

------
gambler
Simplification of systems should be a an actual college course, mandatory for
all computer science grads.

Too many people these days _manage_ complexity through ever more complex sets
of tools instead of simply getting rid of it. They use complexity as an excuse
to introduce more complexity and it just keeps growing.

One important thing I discovered for myself is the practice of starting with
_minimum viable representation_ of the problem. If I work on a system, I try
to write down what it needs to keep track of. If I work on a component, I
start with the minimal configuration it requires to instantiate. This helps to
avoid anchoring myself with available tools and "common" solutions.

------
westoncb
There is a kind of complexity worship I've noticed, but my accounting of it is
that it comes from folks who aren't aware of the methods of simplification
employed by those successfully wrangling complex subjects/systems; absent an
alternate explanation, they assume that the mind of the complexity wrangler is
fundamentally different from their own.

In other words, rather than imagining the complexity wrangler inventing layers
of simplifying abstractions, they imagine him/her to just have a radically
extended short-term memory or something on those lines.

The article points out the case of architects worshipping complexity in some
cases by essentially over-engineering. I think this is a mistaken explanation.
I think it's more of a fear-driven thing: the architect is like, "Oh shit,
this problem I'm gonna be dealing with is gonna be _really_ complicated; I'd
better throw everything I've got at it!"

------
AnimalMuppet
Primarily, people worship their own positive self-image. Complexity gives them
the chance to view themselves as smart enough to handle this really
complicated thing.

~~~
TeMPOraL
Yup. Also, they'll reason that the way from junior to senior is knowing and
using _design patterns_. I've seen plenty of inexperienced cow-orkers
overcomplicating things because they thought that's how good programmers build
things. I know I did that too, when I was first learning to program.

(Popular books on OOP and code style definitely do _not_ help keep things
simple.)

~~~
collyw
i agree, I would say that this is where most of my experience comes in
valuable, not knowing the latest framework, but being able to find a level of
abstraction that promotes simplicity and maintainability in the code. Yet
interviews seem to value the opposite.

------
userbinator
_After all, we all are looking for technical challenges and want to implement
interesting projects._

In my experience everyone does, but the difference between the complexity-
worshippers and (for lack of a better term) simplicity-worshippers is that the
former like to take a simple problem and blow it up with complexity, while the
latter like to take a complex problem and make it simple. In other words, some
people like the challenge of making things more complex, while others like the
challenge of making things simpler.

To make a concrete example, complexity-worship would be something like using
half a dozen different new web frameworks/languages/libraries to set up a
personal blog, while simplicity-worship would be more like an H.264 encoder in
a single file[1] or a self-compiling C-subset JIT[2].

One thing that seems somewhat obvious about the complexity-simplicity divide,
and which could also explain the prevalance of complexity-worship, is that it
is relatively speaking much easier to generate complexity than to reduce it,
and unfortunately it seems the majority of developers just don't have the
skill to reduce complexity. It's easy to glue a bunch of existing code
together without really understanding how it all works; it's much harder to
take a spec that dozens of experts have worked on for many years, and condense
it into a concise implementation.

[1]
[https://news.ycombinator.com/item?id=18045494](https://news.ycombinator.com/item?id=18045494)

[2]
[https://news.ycombinator.com/item?id=8558822](https://news.ycombinator.com/item?id=8558822)

------
grandmczeb
> The managers, that this paper describe, worship complexity without realizing
> it. They want as large a team as possible and thereby they make a problem
> complex because a large organization can cause the architecture to collapse.

Maybe we have different definitions of "worship", but this seems like
accidental complexity, not reverence. If anything, people tend to worship
simplicity.

~~~
mannykannot
What people say they want, and what they do, are only loosely related. I have
lost track of the number of in-house / one-off frameworks I have seen that
have been justified on the grounds of making thing simpler, but which have the
opposite result.

Efficiency is another common excuse offered for gratuitous complexity.

~~~
TeMPOraL
Other common excuses are "we won't optimize prematurely" and "we avoid NIH
syndrome". You then end up with something like Node ecosystem.

Simplicity is hard.

------
irrational
Have you seen web development recently? The answer, apparently, is a
resounding YES!

------
titzer
Conway's insight, and the subsequent law that bears his name, was so laser-
precise that it is uncanny.

Local incentives matter too. In any social system there's an incentive to
"flexing your muscles" and showing off our abilities, since that helps to
build prestige and respect.

When coding moves away from "impact", and "scaling" and "agility" and back to
old fuddy-duddy engineering staples like better up-front design, partitioning
problems properly and solving them in toto before shipping (i.e. doing it
right the first time), allocating the right problems to the right people--
difficult to do in a team with junior members that need space to grow and
experts that can dash things out in minutes--then we'll hopefully be sober
enough to see through cargo cult practices down to what works.

------
grownseed
There is "complex", and then there is "complicated", I believe the author
refers to the latter. A complex solution involves a lot of moving parts, but
each of these parts serves a specific purpose, such that it is possible to
break down the solution into relatively simple, understandable blocks. On the
other hand a complicated solution will have inefficiencies, some of its
building blocks may be too big or convoluted to be broken down further,
causing implementations that are hard to understand and/or may be redundant,
feeding back into the complications. I think Rube Goldberg's Inventions are an
excellent example of the difference I'm trying to make.

Social posturing is certainly largely to blame for over-complications, but I
think more deeply, a lot of people hate to let go, to be told that the very
thing they were responsible for, that they worked hours/days/... on isn't
necessary any more, or perhaps never was. And then in the other direction,
there are what I refer to as "exponential requirements", for lack of a better
term. You buy a large bag of coffee that you could simply use as you go and
store in the fridge. Or, you could have a jar with some of the coffee in it,
and the rest of the bag still in the fridge. Then a plate to put the jar on.
Then a new shelf to put those plates and those jars. By the time you've
followed this track long enough, you're looking at buying a new house, even
though your essential requirements haven't changed from the beginning.

------
hirundo
The ability to recognize and manage complexity is surely a strong survival
trait, akin to wisdom. As true for hunting and gathering as for particle
research. Complexity is also somehow deeply related to beauty, or at least
awe. Such emotions motivate us to become more observant of the complex,
possibly therefore out-replicating the more oblivious.

Worship of the complex, or at least a particular focus on it, may be built in
by Chuck Darwin.

------
swayvil
The fetishism of complexity in engineering (and our machine loving culture) is
much like the fetishism of helplessness in some of our religions. Both
intensely value surrender to an unknowable omnipotent power. A power
addressable only by blind ritual.

You might say that in both cases there is a longing for a dissolution of self.
Or death even.

Is the machine-lover a death-lover?

------
nimbius
when i was younger I used to think complexity was some sort of example of
ignorance. I work as an automotive engine mechanic, so my gold standard for
"its easy and works" was stuff like old fox body mustangs and such.

Fast forward to 2018 and stuff like timing chains are buried under a mountain
of engineering sins. I have a honda with the oil pan, air conditioner,
exhaust, and oil drained and in various states of disassembly and for what? a
chain.

As i get older, I start to worship it. its inevitable that engines come with a
spinal cord of three or four dozen sensors you need to carefully disconnect
before doing anything. But what I cant abide by is the inclusion of
demonstrably poor quality parts. Companies hope the engineering complexity
will just baffle people into accepting the idea of disposable plastic valve
covers and plastic water pumps, but I remember when these things didnt get
thrown out.

~~~
Tade0
_Companies hope the engineering complexity will just baffle people into
accepting the idea of disposable plastic valve covers and plastic water pumps,
but I remember when these things didnt get thrown out._

I saw this part:

[https://youtu.be/NZAWeR46z_Q?t=238](https://youtu.be/NZAWeR46z_Q?t=238)

of this review the other day and began to wonder: how prevalent is this
really?

------
taeric
The intro on Conway's law was more fun to read than I expected going into
this.

It is interesting, because I've seen Conway's law used as a weapon against
modularizing something. I think the key is that you need to weigh what you are
getting out of the modules. Even if it is, in some ways, more complicated, if
you can more meaningfully delegate ownership of parts of the problem out
because of the the modularization, it is a good chance it is still worth
doing.

I don't think I can give a brief example. In my case, we had three teams,
rather than trying to put everything on one team, I was trying to ask for what
the three major components of the system were. If we couldn't agree on that,
it seemed unlikely we were using three teams well.

------
nestorD
The "Simplicity: Not Just For Beginners" talk does a great work of touching
the problem from the opposite angle :
[https://www.youtube.com/watch?v=n0Ak6xtVXno](https://www.youtube.com/watch?v=n0Ak6xtVXno)

~~~
starbugs
Indeed a great talk.

Here's the related HN discussion link:
[https://news.ycombinator.com/item?id=18093158](https://news.ycombinator.com/item?id=18093158)

------
aryehof
Surely at issue is that we do not know how to generally handle complex
problems well. Particularly those _outside_ the domain of computing.

The old fallback of functional decomposition into parts that can be reasoned
about and developed independently, is problematic due to the need to
constantly rework and extend the hierarchical decomposition as more about the
domain under consideration is specified.

How to predictably build a payroll system, or air-traffic control system or a
much simpler (at first glance) yet inevitably complex real-world system,
remains out of reach.

Surely we don't worship complexity, but outside of computer science, just
still don't know how to deal with it?

------
seeker61
The more complex a system is, the more people it takes to maintain, write,
test for it. It builds the manager's empire and also increases mindshare for
the complexity-driven approach. I've seen it many times.

~~~
aero142
I've seen an even worse version. If only the person who wrote it understands
it, then when someone else tries to modify it, they will likely fail. Then the
original author can back-channel predict the failure and use that as further
evidence for their brilliance and senior-ness.

------
yason
Any serious programmer simply can not worship complexity because then his/her
work would never get done.

You can start with a trivial problem and grow it into a system so complex it's
hard to even begin grasping what it actually does and how the software
actually works.

If you start with a complex problem the only viable path is down to less
complexity; often we won't reach simplicity but at least we're somewhat
equipped to handle the underlying complications.

------
anonytrary
How many times have you stumbled upon something on Github that took 100x more
lines of code and 20x more contributors than necessary? Probably every day.

------
cpeterso
Another law of complexity: the Law of Requisite Variety: If a system is to be
stable, the number of states of its control mechanism must be greater than or
equal to the number of states in the system being controlled.

[https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_R...](https://en.wikipedia.org/wiki/Variety_\(cybernetics\)#Law_of_Requisite_Variety)

~~~
btrettel
Is there a better introduction to these ideas than Wikipedia? It makes
intuitive sense, but I'd like to see the proof of this statement. Wikipedia
mentions a "good regulator theorem", but the citation is pretty bad: "Conant
1970".

Edit: The paper must be this:
[https://www.tandfonline.com/doi/abs/10.1080/0020772700892022...](https://www.tandfonline.com/doi/abs/10.1080/00207727008920220)

But I'm still interested if there's a better introduction.

Edit #2: Looks like Ashby's 1956 book around page 207 has an argument based on
information entropy. I haven't read it, just skimmed it. I'll look more
closely later. You can download the book here:
[http://pespmc1.vub.ac.be/ASHBBOOK.html](http://pespmc1.vub.ac.be/ASHBBOOK.html)

~~~
sgillen
The whole "cybernetics" field is pretty weird to me, I think the notation and
lingo have sort of moved on (I could easily be wrong and biased without
realizing).

However the theorems you are talking about have useful, well studied analogs
in the field of control theory.

For example the Law of Requisite Variety is very much related to the concept
of under-actuation
([https://en.wikipedia.org/wiki/Underactuation](https://en.wikipedia.org/wiki/Underactuation)).

The good regulator theorem becomes the internal model principle.
[https://en.wikipedia.org/wiki/Internal_model_(motor_control)](https://en.wikipedia.org/wiki/Internal_model_\(motor_control\))

I think you'll have much better luck finding good introductions to these
topics in control theory than in trying to learn more about cybernetics.

------
Double_a_92
Because we can feel smart if we created something that others can't simply
understand.

------
cletus
I very much doubt this originated from me but whether I heard this somewhere
and took a liking to it or thought of it myself, I became fond of saying this
while I was at Google:

"How do you know if you're engineering if you're not overengineering?"

Now I don't direct this at Google specifically. This is more about the mental
traps we, as engineers, can easily fall prey to.

Take interviewing and the FizzBuzz thing that was popularized (if it didn't
originate with) Joel Spolsky. The beauty of FizzBuzz is that it's a quick and
great _negative_ signal. This doesn't mean that if you ace it, you're a superb
engineer but it does mean if you fail it, you're almost certainly not. Hiring
(from the employer's perspective) is a numbers game. Every candidate costs you
time (giving interviews, writing feedback, etc) and is a huge opportunity cost
(other work that could be done, other candidates that could've been
interviewed) so the goal is the filter out the "no"s as quickly as possible.

So the engineer trap when faced with something like FizzBuzz is to think "oh
wait, that's too simple!" and they go on to change the problem to something
you'll pass if you've heard of the particular obscure algorithm and will
probably fail if you haven't. It's a easy fallacy to fall for. Harder is
better. However now you're not optimizing to cull early candidates. Now you've
designed a test that optimizes for people who do well coding under pressure
(with some degree of luck) on a whiteboard. That is completely different from
the original intent and (IMHO) almost entirely useless as a hiring signal.

Another example: another thing I was fond of saying at Google was that there
was a hierarchy of engineers:

\- At the top level (in their minds) were the C++ engineers. You're not
engineering if you're not writing code in C++ (basically)

\- Next tier were the Java engineers. They thought you weren't engineering
unless you were writing in Java or C++

\- Next came Python and Go

\- Last came Javascript

Now at Google I met some engineer who were _superb_ C++ engineers. Like truly
amazing. I also met others who were like walking Wikiepedias for the
C++11/14/17 standards. And no, that's not the same thing.

So one example that springs to mind once was surprised to learn that I didn't
know what perfect forwarding was. This is something that probably only needs
concern you if you're writing a widely used C++ library using templates
(especially template metaprogramming). As a user of such libraries, it's not
often something you need to know.

My point here is that with all the knots C++ has twisted itself into over the
years as a result of its origin and legacy, the trap some fall into is viewing
such complexity as a virtue instead of baggage.

IME these aren't the people you want making design decisions on complex
systems. The people you want making design decisions on complex systems are
the ones who reverently follow Postel's Law.

------
mirceal
people are definitely suckers for complexity.

Complexity of a system actually creates meaning for people that do understand
it and can navigate it.

Complexity also justifies headcount and effort put towards maintaining,
enhancing something.

Finally, people associate complexity with being smart. If you like simple
things you are either a newb or you are [very] senior/old/experienced.

------
erikpukinskis
YES

------
failrate
No.

------
willart4food
Yes.

