
The shittiest project I ever worked on - m0nastic
http://blog.plover.com/tech/prudential.html
======
redact207
Although it's a fun read, it's a classic example of diving into coding without
giving a project it's due diligence. There's nothing that's derailed my
projects more consistently when I started out as not understanding the user's
needs. They'll never tell you what they want, only what they don't want after
you deliver something.

I think that's the key difference to an experienced dev/BA. One who can
actually sit with the stakeholders and build the system on paper and go
through each of the problems as the diagrams connect. What you end up with is
the stated requirements (tip) and the unstated assumptions (iceberg).

These types of projects are easily spotted as they're often called "quick" or
"easy", which in layman's means no one's really thought about it yet.

~~~
shailesh
Kind of agree with you _and_ OP, so let me add to the discussion with the
caveat that YMMV etc. Some of these points may overlap with comments by fellow
HNers in these threads.

In any organization as a prospective customer, there are three forces at work
that matter to a consultant: users, decision makers and finance folks.
Ideally, the decision makers appoint a dedicated person (or team) who mediates
users and finance guys to create a coherent road map for the system to be
built. What tends to happen in practice is a combination of chance, human
errors and conflicts in priorities.

To survive, we must understand ourselves as consultants, the terrain and the
opposing forces (anything that prevents money getting deposited in the bank).

Consulting is a business => the individual consultant must shoulder quite a
lot on himself:

* Advertising himself

* Finding work

* Meeting prospective customers

* Writing proposals

* Closing the deal

* Writing a contract

* Defining the scope of work

* Nailing what is to be done and what is to be avoided

* Constructing components of software

* (In worst cases) baby-sitting customer's devs / ops

* Testing his modules

* Getting money in his account

So, when we talk with customers, there is inherent pressure in closing the
deal versus precisely defining everything. Icing on the cake: back of the
mind, you might worry about milkman's bills and the birthday gifts to be
brought for your kid's friends. Always have a good runway: six months seems
optimum. Technically, even three months should suffice, but it is insufficient
for the edge case when these three things hold true simultaneously:

* You've to walk away from a customer in the middle of a project, _and_

* Dry sales pipeline, _and_

* Unexpected crisis at the personal front: very close friend in need of money for surgery etc.

God be with you.

Now, back to the main track. The key is to first understand the priorities of
stakeholders. The contradictions are easier to grasp once we put ourselves in
the individual's shoes:

* [Convenience] The users want everything including the kitchen sink,

* [Accountability for the risk] They want it yesterday but won't commit to it on paper since you're in early sales stage,

* [Cost] The finance folks want stability and the most economical price,

* [Availability] The decision maker prefers to spend the least of her time to be spent on this activity, except when it's her pet project.

These roles may be played by different people or the same person.

Again, we don't have to agree, but acknowledge that things are meant to be
imperfect. Most people are actually decent and rational. Now, somehow, the
consultant must figure out a good path. To begin, he should have a good handle
on:

* The industry in which the customer operates, e.g. BFSI, Energy, online marketplaces, food, fashion etc.

* The nature of the market: oligopoly, crowded, emerging et al.

* The level of Govt. regulation: HIPAA? PCI?

* The customer's position: among top tier, mid-size, small firms.

* Customer's customers: helps to track the wind.

* Standard pain areas in the market.

* How customer's competition and partners survive the pain, specifically the differences among the customer's approach and the competitor's ones.

* The financial health of the customer: if possible, try to talk with their existing suppliers in other arenas (e.g. the guy who supplies printer cartridges, the cleaning lady etc.).

Based on that, assuming we've landed a hot lead, say that we want to convert
it to money. There will be a lot of conversations over the phone, e-mail,
chats, face to face; what is to be left out of those conversations is really
context sensitive. The important parts are:

* What problem we're going to solve for all stakeholders?

* How we're going to solve it?

* Communicating our approach and progress on a timely basis.

* Improving Vulcan skills: like money, expression of anger is a great servant but a terrible master.

I leave out the details on the development process, since it depends on the
individual.

Here are a few things which worked for me marvellously in certain contexts and
doomed in others:

* Setting expectations by explaining the cone of uncertainty in the estimation process. Always be prepared for some customers, who deliberately mislead you that they don't understand the mumbo-jumbo, projecting ignorance as a shield so that they can ruin you on payments. Mostly you should refuse such customers, unless you're in an extreme crisis on multiple fronts.

* Writing use cases for the modules / systems I was supposed to work. Most customers say that they don't want any time spent on "documentation," to which I reply that writing a document is not "typing" but "nailing the thoughts to deliver higher quality work."

* Creating mock ups using sketching tools (even Excel/LibreOffice can be good enough, if one is pressed too much for time) in the absence of UX designers on the team.

* Writing test cases before writing my code. Helps to ease the integration.

* Optimizing lazily: progressively refining the artefact (document / mock up / test / code).

* Knowing the expected value of each task: (amount of money * satisfaction * improvement in reputation) / (e-mail tennis * stakeholder's understanding of requirements * payment latency * time spent)

Finally, reducing it to just one sentence: be nice, believe in yourself and
humanity, be multi-dimensional and please heed Guy Kawasaki's advice about
reading voraciously and omnivorously.

Edit: formatting.

~~~
rexreed
Awesome list - deserves a nice little blog post of its own that I can
bookmark.

~~~
shailesh
Thanks for the kind words.

------
kamaal
For those who don't know about the author of this post. His name is Mark Jason
Dominus, the author of one the most awesome programming book that one can ever
read.

Higher Order Perl, is available for free download. If you read it you will see
some amazing insights into programming techniques most people would have never
heard of encountered in MegaCorp jobs. You will also grow a great appreciation
for Perl in general and understand how it can be an amazing language of choice
for a wide variety of problems.

~~~
bionerd
I know it's probably totally unnecessary but still, here's a link to the free
download page of the book [1].

Together with Modern Perl (also free for download [2]) this book completely
changed my view of Perl. It helped me see through the misconceptions built
(most of the times unintentionally) around this language and made me even
switch from Python to Perl.

If you're interested, spend a few minutes browsing through one of these books.
I think you'll like what you see.

[1]
[http://hop.perl.plover.com/book/#PDF](http://hop.perl.plover.com/book/#PDF)

[2]
[http://onyxneon.com/books/modern_perl/](http://onyxneon.com/books/modern_perl/)

------
angrow
>Prudential didn't need an affiliate locator application. They needed a static
HTML page that told people to call the number.

They needed a competitor.

------
fusiongyro
> In 1995 I quit my regular job as senior web engineer

You had the job title "senior web engineer" when the web was 4 years old.
That's pretty cool.

~~~
adamconroy
Sorry if it reduces self-esteem, but we (software developers) are not
engineers.

I've been brainwashed for many years by my brother, who is a Phd civil
engineer, that professions that hijack the term engineer are kidding
themselves.

I know it sounds cool, and my current title is 'Senior Software Engineer', but
it is all a scam.

~~~
AndyKelley
I'm not convinced. Care to make a case?

~~~
briancpotter
The degree of responsibility is vastly different. Engineers are licensed, and
can (and often are) sued for professional negligence. Failing to do your job
adequately means you'll be stripped of your license and prevented from
practicing engineering.

~~~
hackula1
No, only certain fields of engineering are licensed and regulated. Those
include Structural, Chemical, Mechanical, etc.

Others are less rigorous, but can still are the practice of engineering.
Software Development is one of those.

------
narag
I'm surprised nobody, not even the author, has said that the program was
useful anyway. But instead of publishing it in the web, it should have been
used by the toll-free number operators.

------
nekopa
I never cease to be amazed that the more things change, the more they stay the
same. If you removed the dates from this story, you'd be hard pressed to fix
when this happened.

(Except the static HTML idea is a bit of a give away, no web developer would
ever dare suggest a static page in our brave new world of Web X.0)

~~~
thekingshorses
Just say we will load data(phone number) using AJAX in JSON format and
everyone will be happy.

Page is static. JSON is a static file with a phone number. And use jQuery to
load the JSON file.

~~~
misterjangles
My sarcasm detector is showing erratic results from your post. AJAX was 3
years from being invented and John Resig was about 10 years old in 1995.

Perhaps a whooshing sound should be made though, if I didn't get the joke..?!

~~~
fibbery
You missed "If you removed the dates from this story.." in the parent comment.

~~~
misterjangles
Ah, thanks!

------
kabisote
> _These days I would handle this easily; after the first or second iteration
> I would explain the situation: I had based my estimate on certain
> expectations of how much work would be required; I had not expected to clean
> up dirty data in eight different formats; they had the choice of delivering
> clean data in the same format as before, renegotiating the fee, or finding
> someone else to do the project._

Great advice for dealing with issues we did not consider in our estimate.

Had he charged hourly instead of a fixed price, would this project have been
less shitty?

Edit: Fixed grammar. Thanks b0z0. :)

~~~
LordHumungous
An hourly rate has the effect of focusing the client's mind a bit.

~~~
lotyrin
I find that fixed bids they try to take me for a ride, time and materials they
are afraid to spend time with me to figure out what they actually want.

I have yet to find a customer unwilling to try to doom their own project or a
compensation strategy that reliably prevents it - its just tons of work and
communication. Unfortunately I'm just an okay coder and most people seem like
they actually want a code talk therapist.

------
LargeWu
Sounds about right.

I worked at Prudential about 10 years ago, as a FTE. Our small division mainly
ran on a bunch of custom Access reporting applications. It wasn't quite
cutting it, because Access, so it was decided that we would build a portal on
the company's intranet. The only problem was that we, as accidental web
developers, were not allowed to run development web servers on our dev
machines, because they were locked down by corporate. We had to use an extra
PC that, by some miracle, had IIS, and develop against that remotely. Good
times.

------
the_cat_kittles
The title reminds me of the shittiest software job/project I ever worked on-
short and sweet: Got hired off craigslist. It was all php. Their main
competitor was "the spreadsheet". Worked directly next to a cold caller that
repeated the same phrase over and over again, fake laugh and all. They were
all from the same church group and tried to convert me multiple times. Paid
minimum wage.

In addition to being funny in retrospect, it was a good lesson to me to learn
that no matter how shitty your current situation, you can always improve it.

~~~
chetanahuja
oooh... looks like someone has a case of the mondays.

------
danso
The punch line here is great, but even if the specs necessitated something
more than a static page, then it'd still be a hard job.

If the most critical part of a data heavy project is speccing it out, I'd say
the next most important part is the data munging process...and sadly, both of
these things are the most overlooked.

------
dsugarman
I'm sure this is the very tough first lesson any new consultant with limited
experience would learn. The summary is that the specs are incredibly
important, they should be expensive to produce and they should protect you and
your client.

------
mjd
tl;dr

~~~
ovoxo
Ha! I think the people who downvotted this didn't realize that "mjd" is Mark
Jason Dominus aka the guy who wrote the blog post.

~~~
steveklabnik
Even then, the post adds no value to the discussion, so it deserves downvotes
anyway. Its a great blog post, and I upvoted it, but its a bad comment, and I
downvoted it.

~~~
kbenson
Sure it does. It says that mjd is on HN, and reading this article's comments
at _least_ enough to post this, and that he has a sense of humor.

I'm sure if someone so desired, they could ask a specific question regarding
the story either in reply to the root comment or as a root comment and expect
it may be seen by the author, and possibly get a reply.

~~~
steveklabnik
I think the quality of the other responses to the parent indicates exactly how
likely that is. A whole subtree full of gray comments.

Maybe I'm just getting old.

------
Sami_Lehtinen
That's not bad at all, it's sounds more like business as usual. I think it's
my regular workday. Btw. Did the customer require extensive documentation,
escrow, you to fix their data when they can't get it done (like invalid post
area codes linked to wrong addresses, fixing post number / city information
based on post area code) etc or looking it up based on street adress or so.
Been there, done that. Did you spend several weeks in meetings where they
can't decide how their stuff works, and you'll just keep wondering if they'll
ever decide what they actually want etc. Of course they're going to completely
change that week later etc. They don't have a clue how things should
technically work, or even what the actuall business process is. Because they
have bought an integration, everything must just automatically work, right? We
need to know what females in age range 20-25 have bought during last month.
Err, but we don't record customer age or sex? Well, but our management team
needs that information. It's sure alarm sign, that they want to know how much
"this" project will cost, but nobody really knows what "this" is. Also it's
needs to be completed by end of the month. I have declined so many projects,
and clearly told customers why I'm not going to do anything for them at all.
Unless they accept it as "agile" project, with unlimited budget. Then I'll
promise them that I'll personally see that it gets done, but it's going to be
expensive. - 15 years of ERP/POS/BI/CRM integration programming & consulting.

------
donquichotte
I am working on quite a tedious project right now. It involves 10 years old,
quite extensive, Visual Basic 6 programs. No source control was used. In our
company it is practice to hire interns for 3-month periods to work on
production software. A mix of programming styles can be found in this project.
Some functions return 0 when they fail. Others return 1. Or -1. Or False. Or
"False". I love it!

~~~
eterm
Count yourself lucky there was any error handling at all! My current job
involves a lot of old (and new) visual basic 6 and there are parts of our
software which don't work once you remove the magic line:

On Error Resume Next

~~~
hackula1
I cracked open some code the other day to find it riddled with:

    
    
        on e goto end

------
wil421
"In 1995 I quit my regular job as senior web engineer for Time-Warner and
became a consultant developing interactive content for the World-Wide Web,
which was still a pretty new thing at the time."

I am confused the person stated that they were a senior web engineer but quit
to work on a new thing the WWW. How can you be a senior web engineer for
something brand new?

Just nitpicking...

~~~
Moto7451
I believe "interactive content" is the key difference. I.e. non-static pages
served via CGI.

------
ibudiallo
This gave me a good laugh, i didn't know what to expect. Just today i had to
deal with something at a similar level.

------
tobiasbischoff
One of the best examples of consultant failure i've ever seen. Clearly an
example for

\- not enough questions asked \- not listened to the customer

Do not ever start building something or proposing a solution if you don't
understand the customers problem deeply enough.

~~~
chii
i dont think the problem lies with the consultant tho - how could he have
figured out the actual business problem, if no body bothered to tell him? He
was hired to do a particular task, and that task was pre-determined by some
"stakeholders" who dont actually know what they want.

------
sfbsfbsfb
I can highly recommend Flawless Consulting by Peter Block. He covers all these
issues and many others. Self awareness is critical to success. If you are
inexperienced you need to be able to recognize/acknowledge your inexperience.
Then read everything available on the subject and interview every expert you
can identify. Clint Eastwood said it best, "A man's got to know his
limitations".
[http://www.youtube.com/watch?v=_VrFV5r8cs0](http://www.youtube.com/watch?v=_VrFV5r8cs0)

------
jetti
Fascinating! As somebody who is starting to do freelance work a story like
this is not only an interesting read but provides insight on what to avoid and
when to speak up as I start freelancing.

~~~
grumps
hmm yes... until your pipeline slows down.

~~~
jetti
In general or because of tips I would take from the article? I'm not that
concerned about a slow pipeline because at this time, I'm only going to
freelance on nights and weekends.

------
scriptstar
The shittiest project that I worked is just finished where we are not allowed
to write any test cases cause we don't have time. I just can't believe that I
finished fairly big project without writing a single test case. I hate my
company, my role and managers, sales people who negotiate tight deadlines.
Over all don't work for any consulting companies out there. They care less of
the code quality. They just want money and no ethics.

My next job will be a product based company.

------
epa
Consider the price you paid to learn a key life lesson which will pay for
itself multiple times over. If you learn from the situation, you have not
failed.

------
cbp
This is why:

1\. You should spend more time designing (away from the computer) so that you
find a _problem_ and then come up with a reasonable solution. Instead of
putting together a list of features. (see Rich Hickey's talk "Hammock-driven
development").

2\. Think of the sum you're going to charge for your consulting and then
multiply it by 4 and charge that because you have to take risk and other
factors into account.

------
taude
The shittiest project? Hmm...if he got paid, not sure I'd classify it as such.
A waste of time because of business getting in the way of potential efficiency
that they perceived they wanted? Yes.

Also, let's not forget that this was 1995 and most big businesses weren't
really fully aware of the potential of the internet and the disruption on
standard business models that was going to ensue...

------
adamconroy
Sounds like poor contract negotiation more than anything. Fixed quotes are
very tricky things to navigate. I simply don't go there. If the client insists
then I try to negotiate a fixed budget, then when the budget us running dry
they can extend the budget or reduce scope. If they don't agree to that I
walk.

~~~
nedwin
So you would take on a project if they kept paying you per hour, even if the
end result was useless?

~~~
adamconroy
Well to answer that I would need some context. Like :

a) Do I need the cash and is the market quiet at the time? b) Are there some
other opportunities that might result? c) Is the hourly rate way higher than I
could get somewhere else? d) Is the tedium only short term? e) Am I gaining
some significant skills / experience? f) Did I give my word to do the job?

If the answer to any of the above is yes, I may well accept the project or
continue on with the project, otherwise I would move on.

I remember when I was working in London, I was on the same contract for 2
years. About 9 months out from the end they extended my contract and we were
asked to build a prototype of a system that automated a lot of human elements
of the original system we built. We built the prototype, and it was very good,
but the funding was pulled and most people left the project. By that stage I
was about 4 months from my visa running out, and there was no work to do. So I
stayed on and did nothing but about 30 minutes of support work per day and the
rest of the time worked on pet projects and amused myself. I felt a little bad
about it, as I was earning $120/hr, but I may well have been screwed if I
tried to find work with only a few months left on my visa.

------
seivan
Most shitty software is not because of the engineers, but management. Not
putting value on good coded and tests.

------
misterdai
Interesting read. However, I helped build a web based system for the
management of a bowel cancer screening programme. Considering what would be
sent back on the testkits and processed into the system... it's the shittiest
project I've ever worked on, but not for the same reasons :P

------
gcb0
The reason the fee was there was to pay for the 0800 thing.

Some executive rightly saw that paying you to render that useless would free
them of that service and the need for the fee (which i bet was not turning a
profit)

But, since big companies run on cargo cult... that happened.

~~~
PeterisP
In pretty much any business, and extremely so in real estate, referral fees
are many orders of magnitude larger than paying for a free phone line.

I'm not working in real estate, but I'd guess that each referral would earn
them at least something like $100 or much more - so forcing the "customer"
(not really their customer since they don't sell anything to them, but only to
the realtors) to call is the correct business decision.

The only WTF question is why they initiated a project that they don't want to
use.

------
kemofo
If that's the shittiest project you ever worked on then you're a lucky guy.
User's will use a software project for all sorts of political purposes that
have nothing to do with you and it does nothing but f up the process.

------
n00b101
This is how life insurance companies operate on a daily basis. OP had a pretty
shitty deal but he should count his lucky stars that he didn't have to
interact with actuaries, who would have multiplied the same problems ten-fold.

------
iamjealous
wow! I am jealous. If that is the shittiest project he ever worked on, he
really has not too much to complain about.

Add to this that this was back in 1995. Companies had no clue what the
internet was nor what they wanted to do with it, so this kind of clueless
behaviour what the customer wanted was pretty much standard for most companies
up to at least 1998 - 1999.

The guy has either been tremendously lucky, or he has not worked in too many
different projects / companies for the last 18 years....

------
cheshire137
This got infinitely more readable when I loaded it into Pocket and let Pocket
do its formatting on it. Adjustable font size and a more reasonable line
width...

------
hclee
I could see why development service team always want to get more and more
information about a project. Open it more, then you get better return. Fun
reading.

------
elwinmurton
Someone once told me: The client know what he wants, but not what he needs.
Success is discovering that before the project ends.

Great post!

------
yitchelle
Great [http://thedailywtf.com/](http://thedailywtf.com/) material.

~~~
hackula1
In that case... TRWTF is that he did not charge them an arm and a leg for all
of those planning meetings. If you charge appropriately, only necessary
changes get made, and only needed software gets built (most of the time, but
you stay paid and happy either way).

------
digitalmaster
Without a pollyfill to support older browsers this is merely a peak into the
future.

------
etherael
Ask them what they're praying for, not how to pronounce the Latin.

------
paulhauggis
The shittiest project I ever worked on was a php project that was converted
from another language (I don't remember which one). This doesn't sound bad,
except they used software to automatically convert it. The PHP code had no
comments, minimal white space, and the variables were all hex. My job was to
fix bugs.

I worked there for about a week before I quit in frustration.

~~~
josephagoss
How can someone suggest auto convert into another language like that?

In the long run it would be cheaper rebuilding the entire thing.

~~~
nmcfarl
I've actually been involved in at least three projects to do precisely that.
And mostly they worked fine.

In two of the three cases I was converting between "the same language" but
different incompatible versions of it and the original code bases had strict
style guides. At the end of it everything worked brilliantly, and the code was
just as maintainable as it was before the conversion.

In the third case was an full lexer/parser built out to compile VBScript down
to JavaScript. The resulting code was not readable or maintainable but it did
run and that satisfied the needs of the project.

In my experience is that language conversion projects are gnarly, and depend
heavily on style guides and accurate specifications for the languages to
produce good output. But these are not doomed enterprises from the start and
in the end it's the quality of your convertering code that determines the
success of the project.

