
Why do web sites and software take so long to build? And why is it so hard? - scottporad
http://www.scottporad.com/2012/05/06/why-do-web-sites-and-software-take-so-long-to-build-and-why-is-it-so-hard/
======
trustfundbaby
brilliant comment in the original post, just in case anybody missed it

"When I had my bathroom remodelled, at first I thought it was organized
amazingly well and why couldn’t software be like that? There was a designer
from Expo, a primary contractor and subcontractors for tiling, painting, etc.
a project workbook containing all the documents, including the design, and a
logbook for every contractor to record their visit. But it turned out to be
just like a software project.

As soon as they started ripping up the walls, the painstakingly drawn design
for the tub/shower was hastily and arbitrarily adjusted because the casing for
an outside fire extinguisher was in the way. Many of the components turned out
to be incompatible with each other and many were not the ones originally
ordered (the bathtub was not even the one labelled on its box!)

Communication was terrible – the city inspector would tell the primary
contractor to change something, and then a subcontractor would show up and ask
me what he was supposed to do. When there was a disagreement about whether the
tiling was done properly, one of the contractors disfigured the tiling to
force another contractor to redo it. They all had other projects, so for long
stretches I had a pile of dirt (for the cement) in my patio and a non-
functioning bathroom for months. And toward the end there were some quick
patches, e.g. the walls were slightly curved so edges of the tiles looked bad
and they just painted the patches, which worked but cost me extra. And a
couple of years later when the tub sprung a leak, the plumber couldn’t figure
out how to get in there, said they must have done a shoddy job on connecting
the pipes, then when he finally got a good look at it, realized why they did
it that way. It was exactly like a software project!"

~~~
yaix
Only difference, its really hard to do "git reset HEAD" when the bath tub
didn't fit.

~~~
itmag
I would tell them to "git r dun" :)

~~~
kpennell
made me chuckle, thanks.

------
bigiain
My opinion?

Because as much as the software industry loves to adopt the work "engineer",
almost none of the sort of websites and software discussed here on HN ever
gets anything much like "engineering" in the sense of, say, "civil
engineering" done.

I think in my ~20year career, I've had only 3 projects that were specified
well enough up front that we just "built it according to the plans" and had a
satisfied customer at the end. Overwhelmingy, some (or most) of the design
gets "made up as we go along". Even on projects with several small forests
worth of up-front documentation, there's almost always large areas of
vagueness or outright contradictory requirements, which need decision making
on the developers part halfway into the job.

There are _very_ few websites that couldn't be coded from the ground up in a
few weeks with a few good programmers - _IF_ you had already thought through
all of the things the site needs and all of the consequences and the
contradictions in those consequences. Even the big ugly complex projects, like
Google's search or Facebook or Wikipedia or Twitter - if you had a spec that
answered all the details about how you wanted them, a small team of
experienced web guys could get it up and running in a month or so, and let you
now how much cloud/hardware/sysadmin/support you'd need to budget for as your
customer acquisition kicked in.

It doesn't take very long to "build" the software or the website (obvious
"large scale" projects like OSes excepted). What takes most of the time is
identifying and solving the problem. (And, I suspect a lot of the reason
software/websites have a reputation for taking "so long" is that they're _way_
too often rushed into the "building" phase way before all the things that need
designing are even identified, then all the new solutions take longer 'cause
we try to fit them around all the work that's already been done.)

~~~
fridek
I was working as a freelancer for a long time and I had a quite simple
solution for lack of documentation. Before the first meeting I'd send a
customer 12-page form to fill which covered most of the "what and why" of the
project. It quickly turned out that most of people have no idea whatsoever
about the product they wanted to buy. I know that it's partly our job to guide
people through rough path, but if all customer knows is "I want a website"
it's better to quit early.

Sometimes I waited for a filled form for few months or even more (not paying
much attention), and some of these projects were very successful. If I'd have
started building right away, I'd spent these few months struggling and waiting
for decisions to be made. What is even worse, I'd probably estimate my time
and salary based on wrong assumptions, so I would be mad and underpaid. The
project would take way longer than I estimated, so customer would be angry
too.

Do yourself a favor and never start coding before final documentation arrives.

~~~
eric-hu
I, too would like to see this form. It would provide a lot of guidance for my
company. We're knee deep in the consequences of scope creep right now.

~~~
fridek
It is in Polish, so I guess it won't be very useful. I may translate it and
post somewhere if I have some free time. For now, try googling "website brief
template", it's how I made my first version, then improved it over time. It's
basically a compilation of most of the questions I had to ask during my
previous projects. If you like to look like a funny guy try adding some "would
you like a pictures of your cat on the website". It releases some tension
associated with such formal document (which I usually attach to signed
contract, just in case), but be prepared to hear "yes, I actually would like
to have cats on my website".

It's probably worth mentioning that this approach is correct only for small
and medium projects, like things possible to deploy with Wordpress ninja in
team.

This may be a good place to start:
[http://www.methodandclass.com/article/write-a-web-site-
brief...](http://www.methodandclass.com/article/write-a-web-site-brief/)

~~~
eric-hu
I think I'd be able to get the idea of your questions using Google translate--
but the link helps too. Thanks!

------
peterwwillis
You know, it's really easy to build a car.

All you need is wheels, brakes, a frame, a power source, a transmission, some
fuel, and some crap to connect the power to the transmission to the wheels and
the wheels to the frame. And a seat. Oh, and a steering wheel. Boom. Done.

You can hand-build a car by yourself over a couple days. I mean sure, the
engine, transmission, frame, wheels, tires, seat, gas tank, carburetor,
brakes, etc are all manufactured by hundreds of people and dozens of companies
to get you to that point. But basically, you decide what components to use and
how to put them together. It takes you a relatively minute amount of time to
assemble them and it's much easier than trying to manufacture all the parts
yourself.

Somehow, after building cars for over 200 years, both hobbyists and huge
corporations find new ways to build them (and seem to enjoy themselves). They
even come out with new ones every couple years and keep finding people to buy
them. You'd think the general public would wise up to the fact that it's all
the same thing over and over and demand our jetson cars already.

~~~
RowanH
Interesting analogy.. I'll take it a little further in the custom/kit car
world, it's not unlike the software world. The last 20% takes 80% of the
effort, and you can clearly tell the projects that have had thousands of hours
put in with attention to detail versus those that haven't. And, not unlike
software you can see as a builder where you came from as your craft matures.
Looking at a rolling chassis Lotus Super 7 replica I started building 15 years
ago, is not unlike my first SaxBasic (VBA) script I wrote not far around the
same timeframe. Fast forward 15 years later and it's like really, what was I
thinking ? Contrary to brogrammer theory, this stuff is hard, and takes a lot
of years of experience and breadth of knowledge to do well.

------
maaku
Except that software is written by machines these days. I stopped writing in
assembly language after it stopped being cool in the 90's. Even still, you'd
have to be hand-crafting machine code in a hex editor to really get all the
machines out of the loop.

These days I write in an interpreted language using complex libraries that
handle a multitude of protocol choices for me. Even if you argue that each of
those tools were at one point hand-crafted, even down to the compiler, there
are now development tools that help me write code, from basic code completion
and tooltips to the most complex static and runtime analysis. That is machines
writing software, with a human in the loop.

~~~
michaelneale
Exactly - this article makes the very common (still) (1) mistake of thinking
construction of software == writing code. Construction of software is
compiler/interpreter/all the machinery that kicks in to take some text or
instructions and makes it do something. To correct the analogy with building
physical things - you could compare the architect designing a bridge (and
arguing with engineers about materials and budgets) with writing code - that
would be a fair comparison. The construction of software has been automated
for some time.

This works just fine for construction of physical things because the cost sunk
into to the "non construction" bit is tiny compared to the whole project - so
no one spends too much time thinking about methodologies and automation of an
architect coming up with the concept of a building.

(1) I note in the article the person mention spent time as a consultant in a
big consulting company - this view is still held by them at least with the
last brush I had with organisations like that (and they are incredibly
frustrated by it).

~~~
bigiain
" … construction of software == writing code … "

In one sense there _is_ this part of "constructing software", and _largely_ it
can be done by the software equivalent of stereotypical "construction
workers". (This is what a lot of people who've tried outsourcing to India are
trying to do.)

The problem is, while you can collect a pickup full of Mexicans who can lay
bricks / hang sheet rock / tar roofs on most street corners in the south of
The Mission, and they'll do a great job of it if you give them good directions
- you don't expect those guys to be making architectural or structural
decisions, or zoning or permitting or code decisions.

"Code writers" have to make those sorts of decisions every day - a current
high-profile example is Marius Milner and "his" decision about what data
Netstumbler should collect from the Streetview cars. One of the biggest
software companies ever, having ethical/legal/policy decisions made by the
coder-on-the-spot (at least if you believe Google's representations on the
topic). Or Apple with Lion debug-logging clear text passwords for FileVault,
and having it escape "into the wild".

The "architect" and the "civil engineer" and the "structural engineer" who
have important roles in the world of building physical things, the guys who
sign off on bridges or tunnels or even just-repaired airliners, the guys who
put their careers on the line when they sign the paperwork, the guys with
qualifications and certifications and often indemnity insurance to satisfy
society that they understand the risks - for the vast majority of
software/websites discussed here that's reasonably likely to be a 22year old
college dropout aiming to be "the next Zuck". Even in small and medium
enterprise sized businesses, those roles are largely thrust upon whichever
developer seems to be good (and doesn't duck their head quickly enough). And
if the shit hits the fan, they say "sorry boss, it seemed like the right
answer at the time" (and hopefully doesn't get hung out in the press like it
seems Milner has been…) (And the "big consulting companies" mentioned in the
post I'm responding too, in my limited experience they often seem to want to
make all the architectural/engineering/policy/ethical/legal decisions, then
leave with their paycheck before the "codemonkeys" implement it all, and not
be contactable when their "solutions" turn out to be
incomplete/contradictory/impossible)

I _hope_ government regulation of "software construction" isn't the answer (at
least not for software that'll just cost investors money when it fails, as
opposed to bridges or airliners that'll kill people), but I think lines of
responsibility and authority need to be more explicitly identified in many
software projects, with appropriate authority conferred on the people burdened
with the responsibility. Holding developers to deadlines without giving them
the authority to adjudicate on them or be involved in the determination of
them, is a startlingly common way to have your developers cut corners - and
worse, feel entirely justified in cutting corners and convincing themselves
they're "doing the right thing".

~~~
kragen
The grandparent comment explained clearly why this is. It's because the work
that can be done by "codemonkeys" who will hang sheetrock if you give them
good directions, but can't make architectural or structural decisions, is
already being done by the compiler and other such software, so the only work
that's left for humans is the stuff that involves making those decisions.

You could probably regulate "software construction" and mandate specific
methodologies, but we can see already what the result would be: just look at
the SEI CMM Level 5 Certified software development teams that already exist:
Wipro, R Systems, and so on — technically inept companies that only exist to
rip off clients who don't know any better.

Any country that mandates that kind of development for all software will be
rapidly left behind by the countries that don't as software becomes an
increasingly important part of the 21st-century economy. They'll still have
human beings laying their literal bricks and tarring their literal roofs,
while the rest of us are living in robot-built houses full of fountains and
sculptures, or dynamically-reconfigurable programmable houses.

------
knieveltech
I am compelled to pick the following nits:

"Stuff that is made by hand is hard to make, and even more hard to make well,
and tends to be less sturdy than things made by machines."

The last portion of this statement is inaccurate. Hand tools (for example)
benefit in terms of usability, durability and quality when hand-made. This is
why top end cutlery, wood carving chisels, etc. are typically forged by hand.
This also typically applies to furniture.

"How often do you think two plumbers argue over the right way to plumb a
bathroom? Almost never!"

Also inaccurate. Clearly the author has never worked in the trades.

"Finally, can you think of any job where people are making really complex
things by hand, and which requires a ton of experience and training to be good
at, yet everyone and their uncle has an opinion on how long a project should
take to get done?"

Pick any form of construction known to man. Budget overruns, issues with
building plans, and problems with materials crop up constantly, even with
modern building materials and best practices.

------
goldmab
I call bullshit. If this guy was bored because all of his software projects
were exactly the same, he should have automated their creation and reused his
code so that they took an extremely short amount of time to create. Anyone who
can't do that is either encountering novel problems that require decisions, or
just not a good enough programmer to automate and reuse.

This is the same way I feel when programmers complain about writing "just
another CRUD app." CRUD apps are in fact very difficult to write because of
usability concerns. If all of your "create" screens look exactly the same no
matter what is being created, that means you are making no effort as a UI
designer to anticipate common creation patterns. Even assuming a cookie-cutter
UI, a CRUD programmer has to properly model the concepts in data, which is not
trivial either.

~~~
sreyaNotfilc
I was thinking the same thing. I'm relatively new to professional programming,
but I do create tools and code snippets that solves/facilitates problems that
commonly exists. Isn't that the main function of creating software and using
libraries? Code reuse was one of the things that my professors constantly
advocated when I was in college.

------
tablatom
To date, software engineering has failed to deliver on one critical goal,
which is why we are in this mess.

The world of software development needs to be divided in two -- the component
creators, and the component assemblers.

To stick with the given analogy, component creators are the people inventing
new kinds of plumbing: easier ways of connecting pipes, taps that don't ever
drip. Component assemblers are the people fitting out bathrooms.

Creating new components is high-end engineering. It needs to happen far away
from the day-to-day challenges of making a client happy.

To a large extent, this division is already present, but it doesn't go far
enough. There has been amazing progress - nowadays we work on top of an
incredible stack of technology that we don't have to re-invent, but so far
we've not achieved the "last mile".

We'll know when we're there because component assembly (i.e. making something
for a client) will start to look more like a trade.

Today, just getting a regular been-done-a-million-times-before database-driven
website (or whatever), requires FAR too much low-level code. This, I think is
what is meant here by "hand made". We should be snapping things together, and
often we are, but suddenly you get to an awkward bit and you're back to
forging a new kind of pipe joint that never existed before.

~~~
philwelch
This is one of those classic ideas that's been around forever. It was more or
less implied when the idea of "code reuse" became popular alongside object-
oriented programming, I think. But I don't think it's ever really going to
happen, because one of two things will happen. One is that you'll run into a
set of requirements that no one has run into before, not because any of the
requirements is unusual but because the space of possible requirements is very
big if not infinite in size, and once you combine requirements that space
increases exponentially. The other is that you'll run into a requirement
that's simpler to implement directly than it is to write the glue code for all
the components you _could_ use to solve it. Both of these more or less happen
already; the real question is what would surmount these problems.

~~~
tablatom
> You'll run into a set of requirements that no one has run into before.

Inevitably. Which means you'll have to resort to "real programming" (as
opposed to component assembly) for /that part/ of your project. I don't see
this as a fundamental reason why component-assembly can never become viable.
As we get better at creating flexible components, these situations will get
less common, but they will never go away.

> you'll run into a requirement that's simpler to implement directly than it
> is to write the glue code for all the components you could use to solve it

100% this. For me this pretty much sums up why component-assembly isn't viable
today. For all but the simplest components this turns out to be the case.
(e.g. date-picker, file-uploader, or maybe something a bit bigger with /very/
fixed requirements, like a disqus comment trail). But jumping from this to "I
don't think it's every going to happen" is overly pessimistic. The glue code
is too hard to write? We need a better way to write glue code. That could be a
fundamentally different type of language, or a fundamentally different
conception of what we mean by "component".

(Aside: In my foolishness I am working on such things).

~~~
philwelch
> Inevitably. Which means you'll have to resort to "real programming" (as
> opposed to component assembly) for /that part/ of your project.

It's not just that you'll run into one unique requirement; you might run into
a unique _combination_ of requirements, each of which already has proven
solutions, but with no good way to glue it all together. That's the reason C
programmers still sometimes write their own string handling or memory
allocation code despite that stuff being literally in the standard library.

Writing general purpose software components is _hard_. If you're creating a
product, you know what kind of component you need, and you don't really care
about anyone else. If you're making a general purpose component, you have
almost literally _no possible way_ of even comprehending, much less
fulfilling, _all_ the requirements of _every_ product that could potentially
use your component.

You're always going to notice a difference between something that's been
cobbled together out of spare parts and something that's been designed to fit
an integral product vision. There's a reason we've been hearing about reusing
program components for literally decades. I'm sure some chunk of the problem
will be broken off and solved, some kind of standard solution to the CRUD app
or something, but there are still going to be products out there that need
real engineering, not just component assembly.

------
fierarul
Nothing is made entirely by machines. At some point it's made by hand. The
thing is software is a bit like doing clay pottery when it's not properly
done. Once you add a bit of a systematic approach to it and you test is,
review it, it becomes a bit closer to an industrial product.

The thing is the rest of the world is also in a state of flux. Plumbers
changed materials quite a few times in the past decade which changed the
fixtures a bit. So the concept is the same (ie. design patterns) but the
actual material and its characteristics changed (ie. framework, programming
languages) and there are different ways to put them together: welding, glue,
etc (ie. APIs).

But changing some material, fixtures and binding material doesn't even compare
to the kind of flexibility we have with software!

The solution for plumbers is that they always use what's current. Which makes
their field as much of a fashion-driven field as ours (think about it).

But give them an existing house with multi-layer pipes and tell them to fix it
and they will also comment that it's old tech and you should switch to copper
or whatever.

In conclusion, I don't see how software is any different than other fields
except that we are much more flexible and go with a much greater speed.

Software is at its infancy because science in general is at its infancy.

------
ranebo
My dad would start yelling at the screen if he read this article. He runs
power tool manufacturing business, that requires each product to be tested and
tested to perfection. If there is a fatal flaw the entire line may need to be
recalled.

The fact I can push a software update to a million users in minutes makes him
green with envy.

So yes software is hard... but cost free replication is an enormous upside.

~~~
praptak
Not every piece of software is like that. My updates take time measured in
months from the moment developers declare them "done" till the first time a
line of code is executed in production. And this is assuming all forks on this
path go positive.

Not that I'm complaining, just giving some perspective. Maybe this will make
your dad less jealous :)

~~~
ranebo
Sure I can completely see that being the case in many situations (for example
all Aircraft software engineers have my utmost thanks :-)), but even then the
cost of prototyping changes or testing fixes is made much easier by the
malleable nature of code.

------
roel_v
"How often do you think two plumbers argue over the right way to plumb a
bathroom? Almost never!"

I don't think this guy has ever been to a construction site. The people who
work for me argue about stuff like this _all the goddamn time_. About the most
minute details like making a connection from a wall point to the drain going
first straight then left, or first left then straight, and a million other
things that are completely irrelevant in the grand scheme of things. So just
like software. They also come up with the same justifications for doing it one
way or another - 'oh but it'll be easier later on' (programming language nerd
wars), 'this way saves work', 'this way is more robust'. Get 10 of them
together and you get 11 opinions. And you know what - in the end, the best
ones are the ones who don't come up with different solutions every single
time, but who just get stuff done, good enough, in time and within spec. Just
like software.

~~~
kamaal
>> _the best ones are the ones who don't come up with different solutions
every single time, but who just get stuff done, good enough, in time and
within spec. Just like software._

This looks like some who uses Dynamic/scripting languages to get the job done,
like Perl. Especially 'get stuff done', 'different solutions every single
time'(TIMTOWDI), 'Good enough' seem to perfectly match that definition.

~~~
roel_v
Eh, no, exactly the opposite - what I meant was, the people I prefer are the
ones who don't decide on tech first, then fit the solution to the actual
problem withing the constraints of that tech. People who get the job done
don't care all that much about one type of pipe vs the other or what have you
(I don't know anything about plumbing really), or if they should use Perl of
C++. They use what fits the problem, and don't fret endlessly about the tools,
they just use those tools to get results.

------
dools
I was toying with the analogy of "building things" vs. "building software" the
other day.

I've always felt that comparing software with architecture, building a car
etc. is what people naturally reach for since it's something they're familiar
with in the physical world and much of the same terminology is used.

I don't think that software is at that point, though. Physical things "play
nice" together because they are part of the physical world. Materials have
characteristics that are inherent and don't need to be conceived of where as
the way things behave and interact in software needs to be defined and
constructed entirely by humans.

I guess one could argue that happens to a certain extent in the world of
materials science but ... I don't really think it holds up.

I arrived at a point where I started to think of building software as writing
a novel. Once you start to use that as an analogy, it doesn't seem so weird
that it's hard and takes ages, because so does writing a novel.

In a novel, one must define the entire world, and one can make the choice of
using/re-using story lines or doing something original. People that trot out
formulaic drivel make better money than the tortured geniuses on average, but
the few tortured geniuses that manage to hit, hit it big and serve as an
example to all the others.

I kind of stopped thinking about it at this point and got back to work, but I
think that using that analogy, things really start to make more sense ... what
do you reckon?

------
peejaybee
I think Joel Spolsky had a nice spin on this a few years back:

"Once you've written a subroutine, you can call it as often as you want. This
means that almost everything we do as software developers is something that
has never been done before. This is very different than what construction
workers do. Herman the Handyman, who just installed a tile floor for me, has
probably installed hundreds of tile floors. He has to keep installing tile
floors again and again as long as new tile floors are needed. We in the
software industry would have long since written a Tile Floor Template Library
(TFTL) and generating new tile floors would be trivial."

<http://www.joelonsoftware.com/news/fog0000000337.html>

------
seldo
I think getting a "DB Connection Refused" error when I clicked this link was
wonderfully appropriate.

~~~
ranebo
I actually laughed out loud and was a little disappointed to see that this
wasn't intended. I'll go read the cached link now.

~~~
scottporad
I wish I were that clever because, you're right, that would be funny!

------
swecker
I think of software development more as an art than a science. If those
questions were asked of a professional painter I think we would hear similar
responses. It's all by hand, colors shapes and textures can be amazingly
complex in their combinations, standards would only inhibit a true artist, and
a painter usually has no idea at all when a masterpiece might come forth. So
my answer to "Why is is so hard?" would be, "because it's art." And "why do we
keep doing it?" "Cause we're artists, and people enjoy art."

~~~
apl
That's outrageous self-glorification and utterly misleading. Software
development qualifies as much as an art as other forms of engineering and most
crafts do -- somewhat, at the highest levels, but generally not. Don't justify
the shortcomings of current software engineering by nebulously labeling it a
form of art.

~~~
swecker
I don't consider any of those things as shortcomings, simply aspects of
anything involving human minds using a creative process. You're right that
it's the same as all other forms of engineering, but I would also consider
most of that art as well. With that I mostly just mean; despite the
laws/science/logic involved, creating something new is usually a creative
process.

------
chrsstrm
I think wanting a product cheaper and faster is a pretty universal sentiment.
I think it can also be said that aside from cooking a meal or buying IKEA
furniture, the typical consumer purchases a product that's ready to go with no
assembly or lead time required. I like to think that if my (ordinary non-
commercial) client assumes something is easily done, that's a compliment to my
work. He or she can ask for a new button with a simple function to be added to
a page, but they have no comprehension of the complexity behind adding what
they see to be a simple element. To the ordinary or non-technical person,
there is a lot of "magic" behind what we do (it's just a button, why can't you
just add it in?). That lack of understanding combined with how easily we've
solved other (seemingly similar) problems gives them no way to properly gauge
how long a task should take or what it should cost.

------
eaurouge
This may be somewhat controversial to say on HN, but writing software isn't as
hard as this post makes it out to be. Yes, there are disciplines in computer
science that require a lot of effort, dedication and smarts to master, but the
actual task of writing software isn't that hard. It can be frustrating, but I
won't describe it as "so hard".

After all, you're really just describing to a machine, the idea(s), design or
algorithm(s) you need it to implement. And we have languages, tools,
processes, practices (etc) to make this task easier. Yes, it can get
frustrating at times but a lot of other professions can be just as
frustrating. By the way, if you think building websites is hard, you should
try writing code to run on some electro-mechanical system, like a robot.

------
baddox
I'm not sure I understand the author's definition of "handmade." My Rails web
app runs on my handmade code with a bunch of handmade Ruby libraries, on the
handmade Ruby language, on physical hardware that was made by handmade
machines (or machines made by handmade machines, etc.). How is software any
more handmade than any man-made technology, like a cellphone or automobile?
The distinction sounds necessarily fuzzy.

~~~
jes5199
I'm equally bewildered by that. Everything mass-produced is originally
designed in CAD, by a person. Those things are _more_ like software than the
items made one at a time by craftsmen. It's a false dichotomy.

------
jheitzeb
Wow, thought-provoking post! I had three reactions to this.

1) Software is in it's early days (in the grand scheme of things.) Though
things may seem "hand-made" now, it won't always be the case. In fact I'd say
OO, design patterns, frameworks, cloud technologies and more are the early
equivalents of regulations, well-accepted standards, etc.

2) What other profession is as egalitarian? What other profession has as much
upside? Entire industries can be changed by a few smart people. That kind of
opportunity makes the "hassle" worth it.

3) The hassle of having stakeholders/customers constantly wanting things
faster and cheaper can easily be mitigated by being picky about what company
your work, for, it's business model and it's culture. Not all software
companies face that issue.

------
manmal
The analogy the OP establishes - building a house vs making software - is good
(I've actually used it too in debates), but things are not as tightly
regulated by the government as the OP claims:

* While there are certain constraints a house has to satisfy, architects do have freedom to be creative. They MUST plan such that it will resist certain standard incidents, and they have to plan it such that the floors can carry ~10x the weight you would ever expect there. These regulations vary depending on the region you live in. E.g., in Austria there is the possibility that meters of snow lie on roofs, and architects (or their structural designers) have to keep that in mind when calculating the max weight. You won't find this constraint in Portugal.

* Keeping to these constraints does not guarantee a building won't break. Constraints (like requirements) are prone to grow outdated, or be incomplete. In Austria, roofs will sometimes break down when it snows for some days in a row (4m of snow are very heavy).

* As incidents in e.g. Turkey and China show, plumbers or other construction workers can do such a bad job ("botching") that buildings just break down after some years. Governmental constraints don't assure quality. Some countries like Dubai won't let local workers unguarded near any expensive building for this exact reason - most of the times, German and other Western workforces are hired to oversee the construction process.

I think one could compare governmental constraints on construction work with
requirements. A building's floor must be able to carry a weight of 100 tons,
and a software system should be able to serve 1000 requests/second - I think
that's comparable, and software constraints might even be easier to test.

What I think this really boils down to is that software requirements evolve
much more quickly, and due to this it's not feasible to establish legal
requirements - government would have to issue new requirements every week,
adjusting them for technologies like nodejs. Also, how do you establish best
practices for bleeding-edge technologies within weeks? Not at all. We are in a
constant state of learning and experimenting.

------
njonsson

      Plus, in the history of the world, he said, is there one
      thing you can think of that has been hand-made, and on such
      a large scale as software, that was as complex?  [I cannot
      not.]
    

Yes: musical compositions. The largest software projects are more complex, and
take longer to complete, than symphonies. But the failure rate is probably
comparable.

There are probably plenty of commercial composers who write music mostly
because it’s a job they can do, and they derive a modicum of satisfaction from
it. But the composers who have made history — just like the programmers who
have made history — do it because they must. If they give it up, they cease to
do what they were made to do.

------
Swizec
Building software is often compared to building houses. If civil engineers can
design and create a very complex bridge or dam or whatever, and it keeps
working for 150 years without breaking. Why can't us software engineers make
software that will work without bugs for more than 10 minutes?

Problem is, the two aren't really comparable.

A while ago someone wrote an article that we are Software _Gardeners_ not
engineers[1]. And I think that comparison is much more apt.

Software is maleable, it can change, the layers of complexity build up so high
it's impossible for any one person, or team of persons, to fully understand.
Even the mathematical theory of software isn't fully understood yet, and yet
we are piling those little lacks of understanding on every layer of the
architecture ... they add up and things become monstrous.

BUT! This is a _good_ thing. Software isn't a physical object. I _want_ to be
able to change the specs half way through a project rather than having to wait
another 50 years to design a better bridge. This makes progress quicker.

As always, when progress is quick (and easy), it is also messy.

For example: it took builders thousands of years to invent The Arch and trully
revolutionize the industry. It took software only a couple of decades to go
from machine code to python.

Oh and never forget, the Turing Machine was "invented" in 1948. Our lives
revolve around an industry the theoretic principles of which were first
defined less than 70 years ago.

[1] [http://chrisaitchison.com/2011/05/03/you-are-not-a-
software-...](http://chrisaitchison.com/2011/05/03/you-are-not-a-software-
engineer)

~~~
ExpiredLink
Writing software is like writing books:

<http://www.artima.com/weblogs/viewpost.jsp?thread=255898>

------
Detrus
To use the plumber analogy, we can't agree on how big the common pipes should
be. They have physical limitations set by properties of water and waste.
Software handles a liquid of constantly changing viscosity.

------
arocks
> Plus, in the history of the world, he said, is there one thing you can think
> of that has been hand-made, and on such a large scale as software, that was
> as complex?

It is a testimony to the innovation in any field that leads to tools become
outdated so quickly. Easy commoditisation is directly related to the tools one
has at their disposal. Creative fields like photography or movie-making also
face the same issues.

Interestingly, if we are given a choice to constantly upgrade our tools then
we would always prefer to. For eg: I don't find most people using the cork
screw on a swiss-army knife. I would rather have a lighter multi-tool. But
redesigning, testing and manufacturing a new multi-tool takes a long time.

Software has relatively speaking a much shorter lifecycle. Most of the clients
recognise this and sometimes have higher expectations of the turnaround times.
This leads to unintentionally short timelines or bug-ridden software.

As in any uncertain venture, it is often better to go with iterative and agile
methodologies rather than a big bang approach.

------
fauigerzigerk
I think he's got the hand-made idea wrong. Programs are machines. Writing
software is the act of specifying how those machines work. I don't know any
industry in which this is done by other machines. It's always manual work.

The work of a financial analyst is also entirely manual by the way.

------
TheBiv
This <http://i.imgur.com/UW5rM.png> I truly believe that most software
developers want to solve problems, and establishing a database error is a
problem that people are willing to pay for...thus, there is a class of people
that are willing to do the work!

EDIT: I'm sure the db connection will come back up, but this error is the root
of this discussion, in my opinion. No one wants to be yelled at, so to prevent
this, we revert to what "just works" and your idea of that concept is
different than mine, thus the more we can agree on standards the better off we
will be for the general use case.

~~~
fsniper
I've seen it but seems like it's OK now.

------
evoxed
> To boot, when constructing buildings, the methods are so well known that one
> person can design (the architect) and another person can build (the
> carpenter, plumber, electrician)

As an architect... _easier said than done_.

------
charlieok
I like the analogy of walking through a dark room with your hand on the wall.
As long as your hand is on something you recognize, you know where you are in
the room, what ground you've already covered, and you may have some idea of
how you're going to reach the other side.

I started a rails project very meticulously. I had recently read through the
'Agile' book which closely tracks the development of the framework. I had a
decent idea of what I was going to build and how.

It would be a tight ship, and code would be kept clean from start to finish.
Before even starting the project, I invested a ridiculous amount of time and
effort learning all the tools I would be using, evaluating every case where I
had a choice between popular and well-supported tools. For each tool I chose,
I thought about why using it was a good idea and how exactly it would fit into
the workflow. I was determined that every commit would keep tests and
documentation up to date with any code changes in that same commit.

All that is to say that I came into the project with carefully considered, but
fairly rigid, opinions on most aspects of how to do the project. Perhaps not
surprisingly, the other developers on the project did not share these
opinions. They did things that ran against my idea of how to keep a project
organized. Perhaps I also did things which ran against theirs.

Eventually, the state of disorganization reached a point that felt to me like
letting go of a piece of furniture in a pitch-black room. I no longer had a
sense of how the whole thing was put together. I no longer knew what all the
3rd party libraries we were using did, or why they were included in the
project. I had planned to keep all libraries up to date but, for various
reasons, we were falling behind in that regard and various roadblocks stood in
the way of bringing things current. We had started using esoteric features of
the database I didn't know very much about. I would do a deep dive of research
to try and catch up my understanding, and then I'd fall farther behind in
keeping my finger on the pulse of changes in the project. Soon, the codebase
had dependencies on various servers in our organization that prevented me from
simply running my own isolated instance of it.

I don't know a solution to this, yet. Right now I'm still trying to feel my
way through that dark room.

~~~
jdlshore
Pair programming [1] and collective code ownership [2] are great ways of
getting everyone on the same page and keeping them up to date.

My experience is that only a fraction of developers are interested in keeping
things clean and meticulous. The rest are more interested in cranking out
features. Both are needed. In my experience, a ratio of 1 "meticulous" to 6
"git-r-done" programmers works fine if you're using collective code ownership
& pairing.

(The real trick is getting the rest of the team to agree to it.)

[1] <http://jamesshore.com/Agile-Book/pair_programming.html>

[2] [http://jamesshore.com/Agile-
Book/collective_code_ownership.h...](http://jamesshore.com/Agile-
Book/collective_code_ownership.html)

~~~
charlieok
This is fine, as far as it goes. However, when I do this, I feel like I'm
being yanked around from one area of the code to another. What I really want
is the opportunity to take a few steps back and spend some effort getting
things back in shape. If, as you say, both types are needed, I think it would
help if an organization valued both types and allowed them to do the type of
work they find fulfilling.

If I get a new feature working, but I do so at the cost of adding more
technical debt to the project, I don't feel fulfilled at all; I feel I've done
a net disservice to the world, and especially to whoever comes onto the
project after me.

Being the type of developer interested in keeping things clean and meticulous,
while being pressured to always sacrifice quality for speed, seems like a
recipe for stress and burnout.

~~~
jdlshore
Well, just as it's a fine line between "git-r-done" and "cowboy coder," it's a
fine line between "meticulous" and "architecture astronaut." (See early Java
vs. later Java, for example.) I think both camps win when they're tempered by
the other.

I follow the boy scout rule: the code isn't done until it's at least a tiny
bit cleaner than we left it. I'll generally spend about one hour out of every
four on cleanup (and not budget for it separately). I also focus my cleanup
efforts on what I'm directly working on, and what's causing me the most grief
today. This allows me to keep my code clean _and_ make the parts of the system
that I use the most gradually improve.

------
apinstein
Two comments:

1\. When things don't line up like the plans in an analog project, you can
just "line them up" and the original intent isn't seriously disturbed. That
doesn't work in a hard-logic digital world.

2\. That said, "real" projects still can come together more quickly because
they are better about using interfaces and being loosely coupled than we
(software developers) are. You can slightly move joists, light switches, etc,
because the interfaces of how they interact with other components are better
defined, and the glue code (ie wires, nails, cuts) can be trivially adjusted
to make things fit and still get the desired end outputs.

#2 is partly why software "integration" projects can be hacky as hell and
still work, they're just glue code for clearly defined interfaces. If we
designed our internal applications with such simplicity and clearly-defined
interfaces and wired them together rather than coupling them, it'd be more
predictable.

Unfortunately that's really tough to do for software inventions, as there are
so many new things where the interfaces might not even be able to be clearly
defined.

Construction would look a lot more like software if you ordered wood but when
the stack of wood showed up it was metal rods instead, or if it came pre-
molded to the shape of another house.

There are just too many variables in de novo software. I used to have this
debate with my old boss all the time. You either live with the reality and
create new, valuable stuff, or if you want to build bridges, go write glue
code for legacy systems.

------
agentultra
I don't really find software _development_ difficult. When you're developing
applications be they web applications or some sort of enterprise widget, the
majority of the work is connecting libraries together. Most of the difficulty
is in dealing with the incongruities of the various APIs. As long as there is
a good process in place I've found little trouble in delivering and estimating
software.

As in the construction of any artifact by human means there will be bumps in
the road. No amount of technology we can use today will solve that issue. It's
a matter of communication and flexibility. One might speculate that great
buildings such as the Eiffel Tower or the Empire State Building weren't so
perfectly specified as to allow the construction workers to simply, "put the
pieces together." Problems certainly arise and must be dealt with accordingly.

Software is no different. We still write programs that guide fighter jets to
land on a tiny boat in the middle of the ocean. We've written programs that
drive robots across the martian surface. And we also write programs that share
our inane thoughts and activities with our friends, family, and random
interested strangers. The degrees of failure are still the same and for the
most part assessed accordingly: while Twitter will strive for 100% up time
there is little money or life lost when it does go down. Whereas when a
martian robot plummets into the side of the planet it's a big loss (and has
happened). Mistakes are made in engineering. Unanticipated side-effects arise.
And we do our best to deal with them as well as we can.

What distinguishes software from contemporary engineering? A couple hundred
years of development at the very least. I'd say we've come a long way and have
some ways to go. But if you stick with it you might find it.

------
cbsmith
I think the amazing thing about this post is it suggests an incredibly myopic
view of the world. Even if you look at the most basic function of what a web
site is and compare it with the non-web version, you see complexity.
Billboards, magazines, newspapers, books, etc., all have incredibly complex
and intricate development processes, much of which is done by hand. The tools
are somewhat more standardized, but that is mostly because the technology is
much, much older. You try going back to the first 50 years of those
technologies and you'll find it is pretty messy.

The reality is that these things get complex and intricate because that is
what the system demands. I can make a website in <1 minute. It's not hard.
However, it won't stand out amongst the sea of sites out there, which kind of
defeats the purpose. This is why tools that facilitate the process, while
improving the quality of the final product, don't actually reduce the time it
takes to build a site, because improving efficiency just raises the bar higher
about what a quality site needs to be.

------
j45
It takes so long, because we waste so much time rebuilding logic over and over
that is neither re-usable, nor unique. Login/out, passwords, security, roles,
reporting.

The app logic itself can be written itself but we spend so much time on micro-
integration of libraries, or writing them ourselves that we end up wasting a
lot of time on things that aren't solving a problem, rather helping to deliver
it.

------
cateye
The important question that needs to be distinguished is:

How much R&D is involved in your project?

It doesn't matter if it is a software project or a physical project. If the
percentage of unknown terrain is high, the project will become complex.

When a very progressive and innovative bridge/house needs to be build with new
materials and a unknown construction etc., the project will be complex and you
can't design it (completely) upfront.

For example, if your software project is just a CRUD application (without
complex business logic), a qualified and experienced software developer can
estimate it and deliver it on time. (There are a lot of frame works and tools
already available for this kind of software. So, nobody doesn't need to
reinvent the wheel.)

If the wheel is not invented yet for your problem, yes, it can take a while
and your luck is in the hand of gods.

The other issue is that there are a lot of developers that can't get working
any shit. But incompetent people are everywhere.

~~~
kragen
It does matter if it's a software project, because any work you're doing on a
software project that isn't R&D is wasted time that you should figure out how
to automate. That's not the case if you're building a bridge or a house.

Yet. Robots will get a lot better this decade.

------
mgkimsal
Quite an aside, but... I see many things going wrong with processes in
businesses _all the time_. Simple processes - things like following up with me
on an inquiry that I made, or correcting an invoice and sending it to me. And
those get borked constantly. And almost inevitably, I get something like "oh
sorry, our computers were down" or "oh, the computer made a mistake". Hrm...

But with software development _we are in charge of the computer_. When there's
a problem with the computer, there's no ability to shrug it off or make
excuses - it's our entire purpose on a project is to make the computer do
stuff. Oh, no doubt we _can_ (and do!) make excuses - the compiler versions
are wrong, the tool doesn't work with this other platform, etc... but no one
outside our field can even understand those problems, nor do they want to.
It's just our fault, and we have to fix it.

------
JVIDEL
Regular people can't see software, they really can't: for them it's a bunch of
crazy numbers and letters, and incredibly boring too which is why media like
movies do a montage with meaningless 3D objects and not actual code.

That's also the reason why is so easy for people to justify pirating software:
is something unsubstantial, "how can it be so expensive? how can it cost more
than the computer in which I'm using it? I can see the computer!" is what I
hear the most...

And it gets even funnier when you consider most coders and pro users can deal
with half-baked apps and find a workaround for bugs and glitches, but the
average enduser, the kind that barely appreciates good quality software or how
much it costs to make it is the first one to complaint when something doesn't
works perfectly, even if it actually does but they are too dumb to know how to
use it.

------
betageek
On a fundamental level the tools we have to build software are still pretty
bad. If that wasn't true we wouldn't be seeing the massive increase in
languages & frameworks that try to take the pain away.

It just takes time to make an entire industry work smoothly and software
engineering is still a young discipline. When I look round at the innovation
going on lately - the rise of alternate JVM languages, increased
expressiveness in languages like Ruby, frameworks like Rails, Django,
Bootstrap, Less & SASS, compile-to-JS projects like CoffeeScript &
ClojureScript, renewed interest in visual programming ideas like LightTable
and many more, it seems we've put our foot on the gas again. I'm pretty
optimistic the programming tools of tomorrow will fix the day-to-day
annoyances, leaving us more time to concentrate on the core problems.

------
smtuttle
All this discussion -- and no reference to Fred Brooks?!

A classic opinion piece on why building software will always be hard is Fred
Brooks’ “No Silver Bullet – Essence and Accident in Software Engineering” —
there’s a version of this available at:

[http://people.eecs.ku.edu/~saiedian/Teaching/Sp08/816/Papers...](http://people.eecs.ku.edu/~saiedian/Teaching/Sp08/816/Papers/Background-
Papers/no-silver-bullet.pdf)

And if you don’t agree, one of its rebuttals, “What if there’s a Silver Bullet
… and the Competition Gets it First?”, by Brad Cox — with a version available
at:

[http://www.virtualschool.edu/cox/pub/92ByteWhatIfSilverBulle...](http://www.virtualschool.edu/cox/pub/92ByteWhatIfSilverBullet/index.html)

…even gets into the idea of an “industrial revolution” for software.

Good classic reading… 8-)

------
teyc
I wonder what changes we might see if we insisted on teaching programming in
the large upfront. For instance, instead of teaching 'public static void
main,' we taught people about component oriented programming. This way, people
are equipped with an understanding of the complexity involved.

I also wonder whether teaching practitioners to diagnose and identify what
type of project one is dealing with might avoid blind spots when estimating
projects. For instance, an LDAP project should be identified as an integration
project, where one might have to deal with unexpected schema.

The real problem with software projects at the moment is we lack a proper
framework to think about the nature of each project and therefore don't
understand where the risks are, and properly advise clients.

------
b1daly
A lot of good points, just a couple I thought of: Software is written to
control hardware. In the end that's all a user can use. Computers are a new
category of machine, they are super complex, easily reconfigurable general
purpose machines. So writing softare doesn't really compare with other trades.
In some sense it's like a design, but since the machine is already constructed
the building of the particular machine that is the app in practice is trivial
(usually, click install). If you compare to building a house,let's say the
program is like the complete design. One could design a house that was reused
over and over. In some degree this happens (prefab houses). But since the
construction costs are high relative to the design, there is usually some room
and incentive to customize, just a bit.

For a software app, if it's well written, the end product builds itself. The
cost of this last step is virtually nothing. Since the design can be used
repeatedly, the software development costs can be spread over a lot of end
users. For example, take Excel. A "spreadsheet machine" is pretty useful, it's
complicated. You could compare "spreadsheet machine" to a house (or maybe
something like a tractor). Aside from tweaking, the software is done, the
actual machine (a PC) is a commodity. It's a done deal, anyone who wants a
"spreadsheet machine" can get one relatively cheaply. But now, since the
machines are out there, along with a huge amount of cheaply distributed
software (the OS,languages, compilers) everyone is tempted to make something
new. How about a multimedia playback machine? Brand new things that haven't
even been made before. Even though there are a bunch of general purpose tools
available, it is the "newness" of the app that makes it hard to predict how
long it will take to code. New problems don't have existing solutions, so no
one can predict how long it will take to find them.

Also, since the actual machine is super complex, which is necessary for such a
general purpose device, there is looseness in the designs, leading to many
different ways of doing the same thing and inevitable bugs. There are bugs in
the OS, so you are going to have bugs in your app. Think about how much more
complex a PC Playing back a DVD is compared to a DVD player.

------
dgregd
I would love to get software to visualize source code as 3d structures. As a
main purpose to see its complexity, not to read or understand source code. And
then put on one picture, side by side with proper scale, my system and a
stadium.

------
joedev
This article points out the issues I have with the "software craftsmanship"
movement. By definition, crafting is building by hand. When we treat software
development as a "craft", we attribute too much benefit to hand-building
software and lend little effort to finding ways to be more productive and
efficient at software development.

As long as software is considered a craft, it will always be an expensive,
laborious, and frustrating process for clients and customers. That may be
great for software developers' egos and pocketbooks, but does little to
improve the image of software development projects among the general public.

------
ExpiredLink
Is it?

> meta name="generator" content="WordPress 3.0.1"

Was it hard to create this blog entry? Obviously he has a point but, in
general, the abundance of predefined frameworks and libraries makes it easier
to create "web sites and software". Fortunately for us (developers) at the
same time user demands grow higher and higher. Web-Sites 'need' to become more
dynamic, styled, responsive, ... simple CRUD 'needs' to turn into a 3-trier
distributed middleware architecture.

tl;dr software complexity is a constant. Progress in easing development is
compensated by increasing demands.

~~~
davedx
I think that's a tautology.

In my professional experience (I started programming 20 years ago and worked
on my first 'enterprise' Java project 12 or so ago), software complexity has
become much greater. If you look at just the different areas you need to be
proficient in now to build 'enterprise' software compared to 20 years ago,
it's insane.

What I think has happened is that experienced developers have constantly had
to keep up with learning to account for this. I don't think complexity has
stayed constant - we have expanded our skill set and knowledge to keep up.

~~~
ExpiredLink
> software complexity has become much greater

I'm not sure if your memory serves you well. Anyway, the human mind is limited
wrt complexity. It cannot grow infinitely. Also, complexity shifted from
languages (e.g. C++) to frameworks and tools (e.g. JEE).

------
kpennell
I worked in an oil refinery for about a half year (turnarounds). Turnarounds
involved shutting down one part of the refinery for rebuilding and repairs.

A lot of the work involved pipe welding. Pipe welding pays extremely well (for
a trade) and that's because it requires such consistently high performance. As
a pipe welder, you have to stamp your name into every weld you do. If your
weld fails, it could kill people.

They check pipe welds using an xray. If a guy's weld failed xray more than
once(and usually more than 0 times), he was out of the job.

------
rumcajz
How is the code hand-made? The millions of gmail users are using the same
piece of code. Same thing with Windows users. That's what you call mass
production.

~~~
fsniper
I think this is looking backwards. Who uses an item is not related to it's
producing methods. Say a handmade sword can be used by different swordsmen in
generations. And this would not make it mass produced.

------
efa
The article mentions that the customer always thinks it should be done faster
or cheaper. I often have the opposite experience. The customer who is
completely clueless and has no idea how long things take. I've had customers
ask for a link to something and ask if that would be really hard to do. I
think it would be really easy for a disreputable programmer to take advantage
(and I'm sure many do) of these people.

------
jakeonthemove
Because software is nowhere near as mature as machines - give it a few decades
and the process will be very automated, for sure. Even now you can build
simple apps with drag-n-drop, and you can make your computer do anything you
want using automation tools like Automate/WinAutomation.

Also, the whole post is a big subscribe link for me when NoScript is enabled
(pretty smart, actually, if it's not a bug).

------
sparknlaunch12
Not so sure web coding as unique as author makes out.

Hand made items - essentially think back to before the industrial age. Most
things were hand made. Art, tools, food etc

Now we have machines to make things easier. Or we have off shore labour.

If it is a precise trade, well sorry but you still need to struggle by hand.
You should therefore new charging a premium to do it.

------
anonymous_mouse
Mirrored a copy here, <http://jsbin.com/apawin/3>

If you have an aversion to using Google cache and would like to see a clean
version.

------
Bxstraz
Cuz you're a frickin' amateur? It's not rocket science, it's easy to do, but
it is work!

I think these meatheads are complaining because it's work...

------
ericson578
Those are all legitimate problems with our industry. But instead of quitting,
I'd prefer to try and solve them.

------
jff
Is it standard/acceptable procedure to make every word in your article link to
a subscription page?

------
scotty79
You know what would make my day as webdeveloper?

1\. Language that doesn't require me to connect to the external system and use
new brain-dead language to do such basic thing as persisting and efficiently
accessing data. How do you declare persistent array of objects that you will
be wanting to access by some of their properties in your favourite language
without keeping the whole thing in RAM? I'd love to have my data stored in XML
if I could query it without loading the whole file into memory and if I could
hint it what XPaths I'll be accessing in the future so it can create some
indexes to make those accesses faster.

2\. Tool that I could use to convert photoshop design to sane subset of HTML
and CSS and perhaps JavaScript without manually writing text documents all by
hand. Dreamweaver is just set of buttons to click to insert tags and that's
not what I mean. I think of something more like a GUI designer where I could
load psd, or multiple psd-s and use them as a blueprint and source of bitmaps
while creating a webpage. In MS Visual Studio GUI designer has a nice thing
called Anchor. It allows you to anchor borders of your component at fixed
distances from borders of it's parents. If people who defined HTML standards
embraced that ideas then webdevelopment of recent 10 years would be flowers
and bunnies instead of franticly searching how to make the browsers actually
fill the height of the parent when you say "height: 100%" (hint: you can't).

3\. Tool that could present me with graphical view of what components my
webapp consists of and how they are connected. Something like database
diagrams but for living breathing components. It's really tiring to making
sense of what your app actually does just by looking through a small hole of
the size of letters in the debugger and stack trace.

4\. I'd like to see, while inspecting a method, all the places in my code
where this method is surely called from and all the places this method might
be called from (because of inheritance, function pointers, dynamic calls
etc.). Function call is the connection between component. It's really nasty
that when looking at source point of the connection then you can see the
destination but when looking at destination you have no idea what are the
source points.

What webdevelopment lacks in my opinion are just proper (visual) tools to deal
with its complexity and inconveniences. Decade or two of this crap and you
really are fed up because there are less and less interesting problems to
solve and same drudgery over and over. Standard component interfaces would
also be nice but unless you have a generally adopted tool that favours some
standard there won't be any standard. Current state of the art web-development
tool is text editor with syntax highlighting and auto-complete broken to
various degrees. The only thing it favours is not keeping HTML, JavaScript and
CSS inside strings because then it refuses to highlight it. It does the same
thing about SQL but that really doesn't bother people enough because SQL is
already lost cause.

------
idleloops
Most sites could use pre-made systems with a minor amount of look and feel
tweaks. So I think a lot of it comes down to sales/con men and snobbery. And
possibly programmers getting lost down rabbit holes.

------
gcb
It's all because developers think they are entitled to interesting problems
and amazing coding.

Heck, in my day job i even have to fight to learn my damn goal. Which never is
"to code" something. That's always only the means to the end. But still, every
Coworker and manager start stating that as the ultimate goal of an engineer in
a software department.

