

We don't write tests. There just isn't time for luxuries. - byosko
http://jamesgolick.com/2007/8/28/we-dont-write-tests-there-just-isnt-time-for-luxuries

======
breck
I'm a chronic manual tester. But I want to start getting into automated tests.

What are the most important things to test? What are some good intros to pick
up the basics on writing automated tests?

~~~
pius
I personally think that test/behavior driven development is the way to go, so
my advice will be geared in that direction. With that in mind, the "reference"
text on it is probably "Test Driven Development: By Example" by Kent Beck.
I've only skimmed it, myself, but it seems solid and Kent Beck is a smart guy.

The cool thing about testing (and TDD in particular) is that even introducing
a bare minimum of that sort of discipline into your work consistent will
quickly pay pretty big dividends. A quick way to get started is to read this
Wikipedia article: <http://en.wikipedia.org/wiki/Test-driven_development>.
Focus on understanding the TDD cycle. Mocks and stubs are useful to know, but
you can safely consider them "advanced" and skip them if they confuse you
initially.

Then, pick a test harness for your platform, say RSpec
(<http://rspec.rubyforge.org>), if you're a Ruby person. Read the
documentation and search for blogs and tutorial articles on the framework
you've chosen. Then just start designing a simple module using it!

As you practice, you'll find it useful to read up on Mocks and Stubs and dig
deeper into the philosophy and best practices. Dave Astels has some great
writings out there.

~~~
plinkplonk
I personally found Kent Beck's book very tedious and boring. In it , simple
classes are bludgeoned into submission through the over use of tests.

From Ganssle's article at
<http://www.ganssle.com/articles/deconstructing%20xp.htm>

"XP's motivating guru, Kent Beck, also has a new book out. Test Driven
Development (Addison Wesley, 2003, ISBN 0-321-14653-0) focuses on XP's testing
practice.

I bought the book because I'm fascinated with testing. It's usually done as an
afterthought, and rarely hits the hard conditions. Unit tests are notoriously
poor. The programmer runs his new function through the simplest of situations,
totally ignoring timing issues or boundary conditions. So of course the
function passes, only to inflict the team with agony when integrated into the
entire system. XP's idea of writing the tests in parallel with the code is
quite brilliant; only then do we see all possible conditions that can occur.

Beck argues for building tests first and deriving the code from these tests.
It's sort of like building a comprehensive final exam and then designing a
class from that. An intriguing idea.

But X what if the test is wrong? Test driven development (TDD) then guarantees
the code will be wrong as well.

Worse, TDD calls for building even the smallest project by implementing the
minimal functionality needed to do anything. Need a function to compute the
Fibonacci series? Create the simplest possible test first X in this case,
check to see that fibonnaci(0) is 0. Then write a function that passes that
test. Try fibonnaci(1); that breaks the test, so recode both the test and the
function. Iterate till correct.

The book shows how to build a factorial program, which results in 91 lines of
code and 89 of test... after 125 compilations! Klaxons sound, common sense
alarms trip.

Agile proponents love the interactive and fast action of this sort of method.
But programming isn't about playing Doom and Quake. If you're looking for an
adrenaline rush try bungee jumping.

125 compilations for a trivial bit of code is not fast. Dynamic? You bet. But
not fast.

"

Test _Driven_ Development is not quite the same as having unit tests or
automated tests, which I believe the poster asked for. The jury is still very
much out on how useful TDD (and XP for that matter) really is.

Many talented programmers believe XP/TDD is essentially hype. . This is not to
say Pius is wrong, just to add a note of caution, be skeptical of the XP hype
machine.

"Try it and decide for yourself" is probably the best advice possible

~~~
pius
I totally agree with your last statement.

------
geebee
"Everybody Tests" is a good point. Some devs throw away their tests, and other
devs keep the tests around and run them periodically, but devs who write code
iteratively (which seems like pretty much all of them) write something that
could be easily turned into a unit test.

If you don't like TDD (and I have to admit I'm not a fan), you can still
produce tests while you write code (you probably already are). Whenever you
write a bit of code, you probably do something to check whether it is
producing the expected outcome - even if it is as simple as writing a little
script to generate some output that you visually inspect as it flashes by on
the screen. If it looks good, you keep developing. If it fails, you figure out
why.

Just store your these mini-testing scripts in a unit test, and verify the
expectation/output in a short statement. If you use a framework, you'll have
some methods for this available to you - if you don't, no biggie, just write
something that honks loud and gives you a message diagnosing the failure.

It's not going to satisfy the TDD crowd, and honestly your coverage won't be
as good as if you had done TDD, but there you go - you just got yourself some
unit tests, and you didn't have to disrupt the mental flow that you prefer to
TDD.

------
euccastro
I wrote some opinions on this in other thread:

<http://news.ycombinator.com/item?id=46526>

Unless your software will kill people if it fails or something, my advice is:
observe what proportion of your development time you spend debugging. If it's
one half or more, unit testing would probably benefit you. If it's
significantly less than 50%, then you're probably better off riding bareback,
because writing and updating those tests will take at least half of your time.

The problem I see with the "you'll test anyway, you may just as well automate
it" argument is that unit tests are no substitute for testing with real users.

------
bsaunder
"the break even point for an automated tester is when they've run their tests
twenty times"...

I find that (at least in early development), when I'm in the zone I refactor
several times a day (differnent parts of the system of course). If my break
even point is twenty times, I'd never get there. Every refactoring would
virtually obliterate all of the previously automated tests.

------
niels
I've recently begun using automated tests of basically all my code. Both
frontend and backend, when I'm doing web stuff. I usually write the code to a
minimum first, and then implement tests. It's a huge relief to use these tests
when I refactor and refine code. I definitely think automated testing is the
way to go in most cases.

------
imsteve
Sometimes it just does slow you down overall.

Remind me again, how many web startups have failed because of a bug? How many?
Hm.

~~~
bct
Are you implying that the number is very small or very large?

~~~
imsteve
SMALL. Are you implying that it's not?

Or maybe you're with the aaron crowd in believing that it has to be absolutely
perfect or it's useless. Heck, even that only could really apply to the design
and architecture of an app.

Perfection is rarely necessary.

edit: And what's with so many downvotes? I thought yc was more mature.

~~~
sbraford
How many teams fail to execute and lose precious time to market debugging
things that they could've caught earlier?

I am not a 100% TDD fanboy either, but it's not something to dismiss
wholeheartedly.

~~~
imsteve
That's why I said sometimes. If you read the article, it claims that this is
never the case. I presented my disagreement with this.

~~~
sbraford
Sorry, I think you're right. There are cases when it can slow you down
overall.

Adding tests up front, from my experience, adds about 40% overhead to a
project. (time spent actually writing the tests) You may get some of this time
back later, as the author claims, as the automated test suite is run.

I've also been on projects where the code & entire test suite was re-written &
scrapped after 6 months of initial development, pretty much making the whole
test up front thing of little value.

~~~
imsteve
To be fair, it seems that everyone other than I took this in the context of
more traditional software project, not an ill-specified one man startup
project.

I was referring to the later.

~~~
ardit33
ok, so now I get it. TTD is better suited for those large companies where
people don't know what the f are doing, so they need to test everything (just
making sure), and when speed doesn't matter, as big companies everything goes
slow.

The same enviroment and thinking that brought us java, and made it a
clusterfuck of "abstractions", and "design principles".

I'd rather just code ad design at will, and be able to refactor, change, or
throw away code without being held hostage of these tests which will break at
some point, not because your code is not right, but because they were design
to test something according to a thinking, and when that something changes,
all those tests are obsolete.

~~~
jamesgolick
I think the problem that you might be facing is more related to the quality of
your tests than testing itself. If your tests are written correctly, you
should never have to throw them away when you refactor.

In fact, that's one of the major benefits of testing: a refactoring safety
net!

~~~
pius
Right. How can you refactor code that has no tests? Basically, you've got no
clue if your new refactored code works because all the precious time you've
spent using the old code went towards ephemeral tests that weren't written
down. That makes refactoring a _lot_ more risky than it needs to be.

~~~
pius
What I meant to say is "How can you _effectively_ refactor code that has no
tests?"

Refactoring is inevitable. I find that on projects where I've written tests,
refactoring goes a _lot_ better.

------
edw519
I have always believed that there is an inverse correlation between the amount
of precise attention devoted in design/development and the amount of time
spent in any debugger. Nothing here changes that thinking.

------
brintoul
I dislike writing more code for my tests than the "actual" code. I do not eat
the "test driven development" drivel hook-line-and-sinker. Sorry, call me an
unenlightened developer.

~~~
ardit33
I am in the same boat. Maybe TTD was invented for people certain kind of
people that like to "plan" stuff well in advance? Maybe my brain is not wired
for TTD, but I tried to like it, and I couldn't.

I can see how TTD is important in critical software (avionics, banking,
trading) etc... but for the average web 2.0 speed of execution should be
paramount, and I think TTD just slows you down. You end up writting x2 the
code, plus everytime you have to change something, you have to do it twice.

In mobile client development, if found TTD pretty useless, unless you are
doing mission critical things, where speed of writing a software is less
important than the accuracy of it. (imagine phones crashing, not a good
think).

~~~
pius
Just a quick note on "mission critical." I've always found that to be an
interesting term. I'd argue that software components critical to your business
are mission critical, practically by definition. It doesn't matter if I'm an
old-school avionics company or a Web 2.0 social bookmarking site . . . my
software is mission critical to my business.

~~~
dfranke
Crashes are a lot worse for business if you're an avionics company than if
you're a social bookmarking company.

~~~
BrandonM
I second that. If you have something like a display bug or some strange
inconsistency in a website, having that bug there is actually a chance for you
to show how responsive you can be to user bug requests. That is, having and
quickly fixing a bug could turn out to be a good thing. That is not the case
if you are NASA or your local nuclear power plant.

~~~
edw519
"having that bug there is actually a chance for you to show how responsive you
can be to user bug"

and how sloppy you were up until now

