
How Experienced Developers Can Handicap a Lean Startup - _pius
http://kevindewalt.com/blog/2009/11/06/how-experienced-developers-can-handicap-a-lean-startup/
======
gcv
In my past life as an "enterprise" programmer, most people at best paid lip
service to doing the "Right Thing." Most code I saw was incredibly bad,
written by programmers who knew they would switch groups or companies, or go
into management, before they had to seriously maintain it. Programmers I met
who worked in an "enterprise" were not into their work. Few of them knew of
anything outside of their prescribed Java or sometimes C++, or followed
anything new or interesting in their field. I could mention something like The
Mythical Man-Month, or talk about looking up an answer in Knuth, and get blank
stares. Explaining closures to all but a couple was like talking to a wall.
This was at a top-tier company, in one of the most prestigious groups in its
IT organization.

I think real experience and talent as a programmer shines in the ability to
find a solution which is both correct and takes the same amount of time to
write as garbage. This is not always possible, of course, but I would caution
against coding without thinking about this trade-off.

In the case of the original article, the author cites the foreign key debate.
Truth is, because reasonable and good programmers disagree on which is better,
this is not even a good example of quick-and-dirty versus the-right-thing. If
anything, it's an example of difficulties in storing data in traditional
relational databases.

------
ulf
I guess an even more experienced developer knows when to use his best
practices, and when to leave them be for the sake of the project...

~~~
bayareaguy
_Unfortunately I found that foreign keys in my Rails migrations were a
constant source of headaches. Moreover, I decided to migrate to Heroku during
the project and had to re-write the foreign keys for Postgres instead of
MySQL. What did I learn about customers in this process? Nothing._

While I agree with Mr. Dewalt's basic message, identifying integrity
constraints as something a "lean startup" shouldn't worry about for a
production application containing _his customers data_ undermines his implicit
claim of data management competency - experienced database developers know how
to design and organize their schema so that details like dbms-specific foreign
key implementation are trivial. When foreign keys cause you serious pain
relative to your other data management problems it means you're doing
something wrong.

What he didn't learn about his customers in the hour at most it should have
taken him to get the keys right (thus helping ensure his customers data is not
corrupted by application, framework or provider infrastructure bugs) pales in
comparison to the lesson he will learn if their data is silently corrupted.

~~~
btilly
The initial success of MySQL is evidence against your assertion.

When MySQL came out it was the first widely used database that explicitly said
that speed of development and operation matters more than correctness.
Experienced DBAs were shocked and horrified. Over time the database has
matured, however its initial success stands as a testament to the principle
that there are a lot of problems (particularly in the web area) where people
care that the site does what they want and is responsive, but are tolerant of
occasional problems.

Yes, I am aware of all of the possible shortcomings. However requiring people
with good ideas to understand more theory before they can get things done does
not help motivated newcomers to get interesting things done. And lots of
newcomers getting interesting things done leads to successful startups. Which
is how a crappy but convenient database (MySQL) got traction in the first
place.

For a classic essay on the tradeoff between convenience and correctness, see
_Worse is Better_ at <http://www.jwz.org/doc/worse-is-better.html>.

~~~
dstorrs
_When MySQL came out it [...] said that speed of development and operation
matters more than correctness. [...I]ts initial success stands as a testament
to the principle that there are a lot of problems (particularly in the web
area) where people [...] are tolerant of occasional problems._

I wonder if this would still be true today? Things are very different from
1994, when MySQL was born.

Back then, the Web was so small it was almost possible to visit every site.
Site designers, site developers, and customers were all trying new things
constantly and learning what worked. There weren't many expectations because
everything was so new.

Today, my guess is that people have a much higher standard for e-commerce. But
that's just a guess.

~~~
btilly
People want to believe that their money is safe. But people are still pretty
tolerant if something like twitter has an outage.

------
omouse
Articles such as this are what fuck up the computing industry.

If you want software engineers to be recognized as actual engineers they have
to conform to a code of ethics and other standards. When the employer says
"let's just cut this corner", the engineer says "NO we cannot!" instead of
trying to justify and rationalise it.

I dislike the term software engineer and I think computing science would be
better and more applicable. A scientist doesn't shy away from his
responsibilities and he sometimes tells the truths that hurt.

We already _always_ ship buggy code. That part is done for you. What this
article advocates is going further and questioning almost everything that can
lead to a proper design.

------
fnid
So basically the author is saying that experience says foreign keys are good,
but in practice foreign keys make development and switching data providers
harder.

Seems like an experienced developer would have written a data access layer
that wasn't so concerned about how the underlying data store managed foreign
keys. Almost all senior level developers know that foreign keys are a
performance inhibitor and plans for eventual denormalization of data.

I'm wondering if this person is _actually_ an experience developer or not. He
starts out saying one of the problems he solved would be to _“figure out a way
to speed up the data processing”_ , yet he doesn't understand how foreign keys
play into this and had to work around normalization?

I also wonder exactly why it took so long to rewrite the application because
of foreign keys. I've never migrated from mysql to postgres but if the process
was so time consuming to warrant a blog post like this, I really question the
amount of experience this individual really has.

~~~
gaius
If the purpose of your foreign key is to ensure data integrity, then no amount
of denormalizing can help. The point of FKs is to enforce business rules in a
central location (e.g. don't ship a part to a non-existent customer, if a
customer's address changes then there's only one place to update, etc).

The alternative to FKs is to write your own "middleware" or "business rules
engine" and guess what, that doesn't get you to market any quicker either.

~~~
fnid
My point was only that it seems the author of the post has limited experience
with foreign keys.

------
chipsy
I am having an experience that is in some ways strikingly similar to the one
described.

I want to iterate on my game's level design abilities to choreograph various
enemy AIs with detailed scripts - things like "this group will spawn over here
and take this path over there, shooting at these intervals."

But figuring out what the best paradigm, the best data format and tooling to
help me iterate on the choreography is - is a guess. I can make a bells +
whistles solution, but it might not be the right one. But neither is it
something that can just wait and becomes obvious at some later date - the
technical solutions and the design solutions are intermingled and have to be
resolved together. So even though I'm my own customer, I'm still in the "lean"
stage of this feature.

So I'm trying to stick hard and fast to "stupidly simple" means of tackling
each problem I encounter. Instead of making one all-powerful choreographer
system I've started with the stupidest, simplest one I could think of, hedging
my bet by allowing multiple systems to coexist, while making the key pieces of
data I already know I need, like spawn positions and trigger targets, sit in a
different layer where they're easy to edit and interchange.

Essentially, I want to ride on the "80% solution" for a while and then
gradually accumulate the rest, and I think the key to making that work is to
know where you are uncertain and start fencing off the most suspicious areas
so that you can build the eventual shift into the architecture - not
modularity so much as "this code will someday become legacy code, so treat as
if it already is."

I'm really sleepy right now so this might seem more profound to me than it is.

------
hello_moto
It's always the developer's fault eh?

Marketing: We need these features now to close $1 mill deal Dev: but... but...
we can't implement them in 2 days... we need more time Marketing: Go figure
out something...

Marketing: Remember that Y features we asked last month? well, sorry, the
customers decided to go with X... Dev: Uh... Uh...

Perhaps once in a while we should switch side: hey
management/marketing/sales/CEO, do whatever you need, without modifying our
software, to close out deals and to grab huge market share... go figure out
something...

~~~
adw
And believe me, if your sales and marketing team are any good they'll be
sweating blood trying to do that. But it may not be possible. In fact, pounds
to pennies it isn't.

(Won't it be a founder selling anyway? I reckon it's a tremendous advantage if
your biz dev person/people know one end of Textmate from the other and can do
prototype-building spadework, too. It's hard to really get how a job feels if
you've never done it. Marketers gotta code and coders gotta market...)

Big, slow thinking sessions ignore this: in an early stage startup you don't
know what you're selling. You don't know until you sell it. So you need to
find out by showing people endless variations on your core theme, and an iota
of effort put into making one which doesn't sell is wasted effort, and wasted
effort is death.

You are going to kill nearly all your features. That's code which gets
discarded, but it's served it's purpose - working out what the business really
is.

------
wglb
I think the title should be "How _Enterprise_ Experienced developers can
Handicap a Lean Startup". This is really just limited experience. I once saw a
very simple project list used by 20 people turn into an enterprise nightmare
with 29 tables and a 3-month development cycle.

Wouldn't it seem to be a mismatch from the very beginning?

~~~
synnik
In most enterprises, 29 tables and a 3-month development cycle ARE quick
hacks.

------
KevBurnsJr
"The problem with efficiency is that it has nothing to do with whether or not
what you are currently doing is the right thing to do."

[http://blogrium.wordpress.com/2009/11/05/why-efficiency-
is-n...](http://blogrium.wordpress.com/2009/11/05/why-efficiency-is-not-as-
important-as-you-think/)

Hacking together a solution may be more efficient in the short term, but it
may also distract you from discovering an effective solution. Diving into a
solution before understanding the problem is often a recipe for wasted time.
Regardless of whether your solution is a hack or a gem, you're committed to
seeing it through. Take time to frame the problem before you commit to taking
a stab at it.

Measure twice, cut once. Teak or balsa.

------
edw519
_Solving problems TRW (the right way) almost always takes longer than quickly
hacking together a solution._

I would rephrase this to:

 _Thinking about_ solving problems TRW (the right way) almost always takes
longer than _thinking about_ quickly hacking together a solution.

Or graphically, with "t" as "minutes thinking" and "d" as "minutes doing":

    
    
      The quick way:
    
      ttttttdddddddddddddddddddddddddddddddddddddddddd
    
      The right way:
    
      ttttttttttttttttttttttttttttttdddddddd
    

If you want to do something right the first time, thinking about it almost
always takes longer. But actually _doing it_ can take less time, often _much_
less time.

When trying to convince my customers to do things the right way, I often tell
them, "It's not how soon we get started, it's how soon we finish."

~~~
davi
Sure -- but I don't think this is the point of the article. Let's say your
ratios are correct, and you want the "t" to "d" ratio to be high in most
cases, for shorter overall "t+d".

The article is saying that you often want to get through each of the following
lines as fast as possible:

    
    
        ttd (release)
    
        tttdd (release different feature set)
    
        td (release totally changed and simplified app)
    
        tttttttttttttdddd (now we understand the problem, do a good job)

~~~
DannoHung
Don't forget that some of those final t's are because you already invested a
lot of other t's and d's into doing it the wrong way.

~~~
shabda
Sure, but

ttttttttttttttttttttttttttttttdddddddd

Startups have areal risk of running out of money, just before they complete
enough d. However with

ttd,

I have a shot at cashflow with that d, which I can invest back into more (nt
+nd)

Established companies dont have that risk so they can do

ttttttttttttttdddddddddddd

While for startups it makes sense to do

ttddttddttddttddttddttdd... etc.

~~~
wglb
For a minute there I thought you were going to do

    
    
      tddtddtdd

------
scotty79
> Bugs in the application resulted in garbage in the database that wasn’t
> discovered until a couple years and terabytes later. One of my colleagues
> had the joy of spending 6 months as “data garbageman” cleaning up the
> database.

Cleaning up should be faster.

It's strange that thing A is optional but without doing thing A up front you
can get yourself in a mess such hard to clean up. If thing A is optional
cleaning up the mess resulting in not doing thing A and having bad luck should
be much easier. Db guys tend to overlook interesting problem how to get out of
the mess. They just have pie in the sky rules and if you don't use them all
and get yourself into trouble then it's your fault so suffer!

------
jsankey
I think the difference between the best practices for "lean startups" vs
"enterprise" is overstated. The benefits of fast feedback still apply to large
projects - the only real difference is that big companies won't necessarily
die from inefficiencies like over engineering. And the fact that, over
anything but a very short timeframe, the right way is also the fastest way
still applies to startups. Sometimes a shortcut is in order - but you can't
sustain that for as long as it will usually take to become established.

Real experience teaches you how to choose the right balance.

------
motters
Warning: unscalable databases and spaghetti code can make your startup fail.

------
evilbit
If Ruby on Rails causes nothing but headaches for those who insist on silly
frilly things like data integrity, I will theorize that perhaps the problem
lies with RoR, not foreign keys.

You probably don't actually need a database at all , given that the integrity
of data stored is not paramount. You don't get any points for "We have this
data. We have no way of assuring it's correct or is what we want it to be -
but we have it!"

On the other hand, given that most start-ups don't bother with things like a
business plan (at least one which doesn't end with "Step 3. ????? Step 4. Get
bought by Google"), perhaps it's only appropriate that such start-ups do away
with other fundamentals of reality, such as data integrity.

------
DanielBMarkham
The problem here is that the devils are in the details.

Should you use small variable and function names because they are easier to
type? Throw away all error-checking because without customers errors don't
matter? Or as this article says, perhaps toss out FKs because they might be
difficult to refactor?

Trying to make the argument that it's one way or the other is really taking a
very complex decision and boiling it down to comic-book level. You need to do
enough of The Right Thing in order to make it to your first value inflection
point, not just whatever it takes to score a couple of dozen freebie users
(although every user is awesome!) Without some idea of where you are in your
startup process and what your next value point is, you can't make those kinds
of decisions. They're weighted choices.

Look at it this way: assume that you hacked something together doing almost
the "wrong" thing every time and ended up 2 months later with a couple of
hundred freebie users. Perhaps a dozen or so are even paying for your code,
say ten bucks a year. But guess what? Your income won't support the amount of
maintenance you have to do on your sloppy code, and customer growth is
inadequate to pay for anybody to help you out. Somehow the bills have to be
paid, and instead of sales and growth you're spending a large portion of your
time as "database garbage man"

So you're stuck with crappy code and not enough resources to maintain the
customer base you currently have. It's called "getting upside down" in a
project. Not a happy spot to be in. (And it can happen without the lean code
problem)

The best scenario is where you make the conscious decision to do a few quality
items _even at the expense of customer growth for a few days or a week or two_
and then consciously make the decision _not_ to do a bunch of other things
because the hit is too much.

It's complicated.

~~~
omouse
It's not that complicated. Quality takes time.

As Greenspun said once, if we want software engineers to be treated with
respect, we need to force employers to respect a certain minimum standard of
quality.

Regarding your example...by releasing something badly designed/programmed into
the wild and taking money from customers, you are committing fraud. It needs
to be seen this way otherwise we will never be forced to take the time to
create quality software.

~~~
DanielBMarkham
So if I wrote a really buggy program but you could still use it to make a
million bucks in the next week, I would be committing fraud?

Could somebody please defraud me like that? Please?

~~~
omouse
You're being too literal. What I'm saying is that, at the moment, there is no
harmful consequence for buggy programs. Buggy programs that people buy means
$$ from the initial sale of the product and your company continues to live and
you get paid in $$ to maintain and fix the buggy program.

As long as the software can run, we're rewarded for creating it, no matter its
quality.

In our own minds, we have to treat that as fraud and need to feel awful about
it.

~~~
DanielBMarkham
What I'm saying is that people judge the value of the program on just that --
it's value to them. The quality of construction can certainly affect perceived
value but a perfectly-constructed program which provides no value is useless.
No, it's worse than useless, because it's an example of craftsmanship-over-
function.

All the quality in the world isn't going to make a pointless program worth
anything. But a crappily-written program that provides lots of value is a
great thing no matter how horribly it was hacked together.

