

Software Architects Need More Respect: Hacker Culture Is Overdone  - bernardlunn
http://bernardlunn.wordpress.com/2011/05/03/software-architects-need-more-respect-hacker-culture-is-overdone/
Some people would say that I am not qualified to write about software development as I have never written a line of code in my life. But I have spent over 30 years working with some great developers on a wide range of projects and I have seen what works and what does not work.<p>A few years ago the art of architecting systems for scalability and extensibility started to get a bad press. I think Joel Spolsky was the first to coin the phrase “architecture astronauts“, to describe those who were always designing layers of abstraction but forgetting to deliver code that actually did anything.    Then we got the lean start up movement, focusing on the constant iteration that the web enables.<p>This led to the hacker culture, oriented to “just do it” speed. “Hacker” used to be a term of abuse, but like “punk” it was adopted by those accused as a badge of pride.<p>That is all good stuff, but methinks it has gone too far. We now see too many systems where the original system has to be totally thrown away. That may be OK if you raised $ gazillions and can now compete with Google and Facebook for a big team of really great engineers. But for most startups, it is not possible. So most startups limp along with the original system. The upshot is:<p>Hosting bills that really start to bite into cash flow.
Systems that crash too often; a few of these and your users have clicked away to a rival. Yes, we persevered through Twitter fail whales, but is your service really that addictive?
New features take way too long. You either release fast or put in some QA and testing process to avoid brand-killing crashes.
It does not need to be like that. The best developers can do both. They create a scalable and extensible architecture AND deliver quick feature iterations. Sometimes you cannot get one person who can do both – it is rare. One team I worked with solved this by having a 4 person team where:<p>1 person hacked demos, in the full knowledge that the code would thrown away. He was awesome at this and in full synch with his colleague who:
created the architecture AND the working code.   35 years later, the system is still being used.
2 people who did the ancilliary stuff like testing, documentation and so on. Great surgeons have a team, same is true with great engineers.
I think the problem may be with the pure architect model. No engineer likes being told “here is a perfect architecture, just code it as per that spec”.  Going back to the house analogy, we need more architectural engineers who create designs that they know will work and they prove that they work by writing the code.
======
dpritchett
It seems the title doesn't quite match the content and thus I was skeptical
until the end: " _I think the problem may be with the pure architect model. No
engineer likes being told “here is a perfect architecture, just code it as per
that spec”._ "

This is _exactly_ the problem with waterfall management in general. Some elite
management/architecture cabal presents a long-range plan and if things don't
work out it's the fault of the grunts for not "sticking to the plan". This
type of software development management is quite obviously wishful thinking
and intellectual laziness. It puts a dramatic chilling effect on developer
creativity and engagement.

~~~
dkarl
I don't think it has anything to do with waterfall versus agile. I work in an
agile process as a software architect, and "Here's the spec, implement it" is
_exactly_ what I get pressured to do by product owners, scrum masters, and
yes, sometimes even by developers, who naturally feel insulted by the idea of
having an architect "over" them and have no desire to collaborate on design.

The biggest problem with the idea of software architecture is that it's turned
into a job title. It stunts both architects and developers by attempting to
divide the labor of programming. We are all programmers, and programming is
really just design, from the lowest level of designing lines of code up to the
highest level of designing distributed systems. Software architects who don't
write code get stale very quickly. The longer-tenured architects at the
company I work for are sad sacks. I've been on calls where they "reviewed" and
approved "designs" that were actually questionnaires filled in by my product
owner and fact-checked by me. The documents they read, while accurate, did not
allow them to understand the work on any level at all. Their job was to check
for things such as, what browsers will this be tested against? You get on a
conference call with them, they ask some bizarre questions based on
misconceptions they got from inaccurately reading between the very widely
spaced lines, they don't understand the answers, and then they rubber-stamp
the paperwork. That's apparently a significant part of their job.

Meanwhile, the developers are encouraged to remain code monkeys because that
division of labor makes sense to management. To the developers, this seems
like disrespect, and it is, though it isn't as insulting as they think,
because management doesn't realize that they're basically saying that
developers sucks at half their job.

I don't think "architect" should be a title. Everyone should have the same
basic title. Call it "developer" or whatever. Just make it clear that we're
all doing the same kind of work. Gradations are okay: junior, senior,
longbeard, whatever. There are a million differentiations between different
programmers in practice. Senior developers should help junior developers when
they need it. Developers should consult with other developers on any difficult
or momentous decisions. Some people should be focused on the big picture, but
not everyone at the same time. At any moment, some developers will be working
on prototypes and new projects, and other developers will be working on
maintenance. People will develop expertise and ownership. The point is that
every difference is _personal_. One developer may write stellar systems-level
code but have unfortunate taste when it comes to designing distributed systems
that work together over a network. One developer may be slow at finishing new
projects but good at ferreting out bugs.

That's why you need have technical managers with power who make situation-
specific judgments about who does what. And that's what this is all about:
replacing personal judgment with rules and formulas. Who should design and
implement System X, Doug or Sam? Doug, Sam, and their boss know their
capabilities. Doug would like to design and implement System X, but their boss
knows that is a stretch for Doug and is too risky for the given time frame.
Sam knows she can design and implement System X, and so does her boss. So Sam
should design and implement the system. Horrible! Unacceptable! Unscientific!
Everyone should be assigned tasks based on their job title. Since Sam seems
more capable than Doug in this situation, let's give her the title "Software
Architect." She will design System X. Since Doug seems less capable in this
situation, let's give him the title "Developer." He will implement System X.

What happens next release when there's a system that Doug really ought to
design, to stretch his design skills or because it's in an area he knows well?
Well, you can promote him, and then he'll be theoretically "assignable" to
design any project. If you don't want to promote him, then Sam will do the
design and he'll be stuck implementing it. Sam, of course, will be kept busy
with design work and "supporting" Doug, so she won't code much, and pretty
soon she'll be useless and will spend her time in meetings "representing"
engineering, telling sales what's possible and what isn't and hoping that her
answers aren't invalidated by things that have changed behind her back. Doug,
of course, is busy changing things behind her back, because he only gets to
make design-impacting decisions if he doesn't admit that they _are_ design-
impacting.

Wow, this is really a subject that gets me worked up. So much for my
productivity this morning. I'm an architect, and God help me, I do not want to
have this title at the next company I go to.

~~~
dpritchett
I think you're describing what Steve Yegge called "good agile" [1]. Each task
is handled by the person closest to the issue rather than by the person whose
job title best matches the perceived difficulty of the task.

" _With a priority queue, you have a dumping-ground for any and all ideas (and
bugs) that people suggest as the project unfolds. No engineer is ever idle,
unless the queue is empty, which by definition means the project has launched.
Tasks can be suspended and resumed simply by putting them back in the queue
with appropriate notes or documentation.

You always know how much work is left, and if you like, you can make time
estimates based on the remaining tasks. You can examine closed work items to
infer anything from bug regression rates to (if you like) individual
productivity. You can see which tasks are often passed over, which can help
you discover root causes of pain in the organization.

A work queue is completely transparent, so there is minimal risk of accidental
duplication of work._"

[1] [http://steve-yegge.blogspot.com/2006/09/good-agile-bad-
agile...](http://steve-yegge.blogspot.com/2006/09/good-agile-bad-
agile_27.html)

~~~
bernardlunn
"good agile" work queue sounds a bit like how good open source works and that
might be scalable even for commercial and enterprise. The problem with some
agile is it does not scale, so managers invent some process that is a bit
waterfall-like and that turns into a kludgey mess.

~~~
dkarl
Exactly. Agile does not scale to the problems management wants to apply it to,
so they rip it to pieces to make it fit. We have features contractually
committed before development even starts, months-distant deadlines with
predetermined deliverables, products with multiple owners, and scrum teams
where people have the barest understanding of the work that their teammates
do. We even have inter-team dependencies that mean any change of direction by
a team will hang other teams out to dry (and what is more essential to agility
than the ability to change direction?) A well-managed waterfall process would
beat the hell out of the travesty of Agile that we practice now.

But nothing would fix the bogus and stunting idea of a division of labor
between "architects" and "developers."

------
agentultra
I have been warm to the idea of adding the roles "designer" and "finisher"
from the art production world to the software world.

 _1 person hacked demos, in the full knowledge that the code would thrown
away._

This is akin to the character designer in the art production world. They
produce the initial sketches and ideas.

These works are generally too rough or "out there" to be used in the
production and are handed off to a "finisher" who has the technical know-how
to flesh out the character to be used by production artists.

In the software world the designer would throw caution to the wind to pursue
the big ideas. They would produce software that "works" but would in all
likelihood need revision to be useable and maintainable by a team.

The finisher would take this rough code and apply their deep understanding of
good software and revise the code to be maintainable, stable, and scalable.

But maybe this is an artificial division of responsibility. In practice I've
been on both sides. I start with an idea and get it to a point where it works
correctly. Then I switch gears and refine it so that it's stable and most
importantly: _maintainable_. However, that switch is always a big one and
elucidates many different feelings (usually one of excitement to one of dread
and fear of boredom). The work can be done by one person, but I like to think
I could increase my output if I stayed on one side of it.

------
bernardlunn
Thanks for all who commented and sorry for all who got annoyed by this. I am
not a developer and certainly don't understand hacker culture, but I really
enjoy working with smart developers and have a deep respect for the mix of
craft and science. I am absolutely NOT a believer in waterfall management,
that is stifling creativity. But I am noticing a trend too far in the opposite
direction. The only thing I have seen that works is a really, really small
team (2-5) of really, really smart developers who are willing to follow the
lead of one person who is also willing (indeed eager) to code and not just
pontificate.

------
wccrawford
When you work for someone, you have to do what they pay you for. For a
'software architect', that often means doing things someone else's way. It's
tougher, and often less than optimal, but that's what you're paid for.

I think it's one of the reasons so many programmers start their own project
and just work for themselves. It allows them the freedom to do it however they
want.

Personally, I like the structure of a company above and around me. I like
being told what to work on and how it should end up.

In my free time, I work on projects for myself. This allows me that freedom
that I don't get during my paid hours, and I end up with something -I- want
instead. Eventually, I expect one of those projects will become popular, and
I'll deal with that if it happens, but for now I get to do anything I want on
them.

Going back to 'respect'... If you do your job, and you get paid a fair wage
for it, where's the lack of respect?

However, if you're complaining about your paycheck-supplier complaining that
he's not getting his way, you're disrespecting him, not the other way around.

------
ap22213
Well, if the title is the problem, what's would be a better title?

Significant software projects need a technical person at the top, who's not a
manager. If that technical person doesn't have a cool title, he's going to get
implicitly demoted by the other stakeholders. That's the way it is.

Architect is just a cool title that gives some authority, so that person
doesn't get stomped out by the other non-technical people in the mix. I've
been called other things besides Architect (e.g. tech lead, lead developer,
etc.), but those titles don't have enough weight. They imply that technology
is somehow a second rate activity. When the "Architect" walks into the room,
at least they get the attention of the CXO or executive level stakeholders.
Otherwise, those types will just focus their attention on the non-technical
managers in the room.

What else would you call the lead technology person who knows the nuances of
everything about the project, but who doesn't want to be the manager of it?

------
StudyAnimal
Now I am conflicted. I agree with this article just as much as I agree with
this one that says "Architecture has no place in software", also posted today:
<http://hhgttg.de/blog/?p=352>

------
edderly
The problem is that software usually needs a good architecture but solving
this by creating a job for a software architect isn't always the best way of
addressing this.

------
mariusmg
Architects do bridges not software.

------
dspeyer
Is it just me, or does this author have no understanding of hackerdom?
Starting, of course, with the terminology and history, but also in the
assumption that a staged model will always produce a better design than rapid
iteration and refactoring.

~~~
bernardlunn
Guilty as charged your honor! But I do understand punk culture ie just do it,
end result matters more than credentials. Any relation? But I do NOT believe
in a staged/waterfall model. It is not an either/or. Or at least some people
hope there is something that avoids the perils of each,\\.

~~~
dspeyer
If you're interested, I think PG's "Hackers and Painters"
(<http://www.paulgraham.com/hp.html>) does a good job of expressing the
concept, and "The Power of the Marginal"
(<http://www.paulgraham.com/marginal.html>) has some good insights.

Also of interest is ESR's "Basics of the Unix Philosophy"
(<http://www.catb.org/~esr/writings/taoup/html/ch01s06.html>) which gives some
idea of how to produce quality code as you go, without separating out
architecture.

------
swombat
_“Hacker” used to be a term of abuse, but like “punk” it was adopted by those
accused as a badge of pride._

Yowzers. Epic culture disconnect fail.

Unfortunately, the rest of the article is equally misinformed. Hacker culture
predates startups by some 30 years... it certainly did not appear as a result
of Eric Ries' Lean Startup writings!...

There could be some truth to the points this guy is making, but the post is
too misinformed for the points to stand.

