
Why testers? - johns
http://www.joelonsoftware.com/items/2010/01/26.html
======
msluyter
Eh, it seems fairly unobjectionable if a bit overly simplistic to me. I did
testing for about 7 years prior to dev, and my take is that it a) depends on
the size of the team and b) a fully fleshed out QA team needs a variety of
skills and fills several roles:

On the one hand, you need 1) grinders who can do the sort of boring, script
driven sorts of regression testing that are essential but difficult to
automate.[1] You also need 2) people with a good eye for usability and with a
good overview of the system who can provide the sort of feedback Joel
mentions. A certain amount of creativity and exploratory skills helps with ad
hoc testing and devising test cases. "I wonder what happens if I push this
button and then _that_ one."

Then, 3) someone who can read code and who understand lower level
(OS/database) stuff can come up with some interesting test cases that a black
box tester may miss. As in, "what happens to temp files/tables that are
created by the app? are they ever deleted?" And finally, 4) if you can find
them, you need someone who _can_ program in order to build test tools and
automated frameworks (that's mostly what I did). The goal of the latter being
improving the efficiency of the entire team. Finding all of these wrapped in a
single person is almost impossible, as #1 tends to be incompatible with the
rest.

[1] Obviously, if you can automate these in a way that's cost effective,
that's preferable, but that's not the typical case.

------
mrshoe
The dog analogy is a bit off. Programmers are not like dogs. We can remember
mistakes we made a year ago and still learn from them. You want a short
feedback cycle, not because it's the only way your programmers will learn
anything, but because you can only improve so much each cycle. The only way to
improve the necessary amount is to go through a lot of cycles, preferably in
the shortest possible time.

Feedback from testers is great, but I've found a bigger reason why I love our
QA team. For whatever reason, it's hard to implement something and test it at
the same time. It's just hard to make my brain think in both modes at once.
It's also hard to test something that I've implemented, because my brain is
biased by the thought processes it used during implementation. Trying to think
in both modes (implementation and testing) means I'll probably end up doing
neither effectively. If you have a great QA team, you don't have to think in
both modes. It's great.

That hints at his point about letting incompetent programmers do QA. I
completely agree with him. A tester thinks differently than a programmer. Hire
great testers. _So many_ companies make that mistake.

I'm glad to see Joel take a public stand on automated tests, too. Automated
tests have their place and their value, but his criticisms of them are
completely valid. No amount of automated testing can obviate the need for good
human QA. For _some applications_ , automated testing can be such a pain to
get right and maintain, that it's often more work than it's worth.

~~~
billswift
The quicker the feedback the better for anyone - read any fairly recent (last
several decades) book on study skills and learning. It is largely the same
principle as having trouble reading your own code that you wrote a few weeks
ago.

------
run4yourlives
To be honest, I think this is one of the few areas where Joel is off the mark.

In my experience, the "tester" is a useless position that usually only exists
because of a deficiency in one of two areas.

The first reason you need testers is that your programmers can't - for
whatever reason - turn out code that works. Either they don't sufficiently
understand the problem scope, they're overworked, or they're just lazy and
don't bother running through enough of their own quality controls before
releasing changes.

The second area is that you aren't getting the right feedback from customers.
Joel uses an example in his article:

 _With long-cycle shrinkwrap software, it can take a year or more to hear
feedback from customers._

The problem is the cycle is too long. Fix that.

I find mostly this is due to more basic things, like separating client service
from programming (hello feedback!), allowing salesmen to mollycoddle their
clients, or not even bothering to ask or listen.

The "tester" is the response to the existence of either one of both of the
symptom above. By their very existence, they are a sign that you are either
working in a company that has placed itself in a less than ideal market
(selling enterprise software, for example), is poorly led, or has leadership
that doesn't listen.

All this thinking hearkens back to computer engineering as building a house;
and analogy so flawed that even PG saw fit to write a book with a more apt
one.

This has been my experience and I have yet to see a situation where it isn't
true.

~~~
pixelbath
To me, your experience sounds an awful lot like not having much experience in
a production environment.

In my experience, context-switching between programming and testing gets quite
cumbersome with difficult problems, and can lead to overlooking smaller
issues. If I'm busy testing every single edge case I can think of, who's doing
the development? With separate testers, you get the benefit of concurrency
(the software can continue being tested while issues from the previous version
are fixed), and subsequently more software developed.

"The problem is the cycle is too long. Fix that."

How so? What if I'm developing a new product that has a full specification and
design document, and is expected to meet customer acceptance goals? I don't
see a deficiency in the development cycle if I need someone else to help test
my software in a manner that I simply don't have time for.

Does your assertion mean that any software that contains bugs is caused by
lack of programmer skill? I, too, would like to learn the secret to writing
completely issue-free code if you're willing to share it.

Now, please don't take what I say to mean programmers shouldn't test their own
creations. If you hand something that crashes 100% of the time to a tester,
you're being lazy. The purpose of testers goes far beyond simply checking to
see if it's "code that works."

~~~
run4yourlives
_To me, your experience sounds an awful lot like not having much experience in
a production environment._

Actually on the contrary, it comes from a growing understanding that the
'production environment' is in fact the problem!

 _In my experience, context-switching between programming and testing gets
quite cumbersome with difficult problems, and can lead to overlooking smaller
issues._

Whether or not a programmer or a designated tester is actually doing the work
has no bearing on over-testing. In fact I'd argue that this is _more_ likely
to occur if you use a tester.

* If I'm busy testing every single edge case I can think of, who's doing the development?*

Nobody! That's the crux of the matter right there:

1\. Testing sucks, so if someone else is doing it then it's better for you,
and, more importantly

2\. the pressure to continue to develop new features at speeds that are
unsuistainable all but ensure that you'll eventually find yourself with so
much technical debt that it will kill you.

 _What if I'm developing a new product that has a full specification and
design document, and is expected to meet customer acceptance goals?_

I'd say run away from this waterfall design process as fast as possible, to be
frank.

 _I don't see a deficiency in the development cycle if I need someone else to
help test my software in a manner that I simply don't have time for._

I do. I see a development cycle where you haven't been allocated enough time!

 _Does your assertion mean that any software that contains bugs is caused by
lack of programmer skill?_

No, because in order for a bug to be a bug it must not only exist in the code
but also in the feature set. (i.e. it must be found) The fact of the matter is
that nobody - not you, not your testers, and certainly not your customers
knows exactly how the software will be used until it is actually being used.

------
rudd
I think more interesting than this actual article is Fog Creek's "QA and Test
Specialist" job listing linked at the end:
<http://www.fogcreek.com/Jobs/QA.html>

There are a bunch of small errors on the page that are the kind of things
they'd expect a tester to catch and require fixing for. For example, "you've"
is written with a backtick ("you`ve"), "valuable" is spelled with the letter L
as a numeral one, "morale" is spelled "moral," etc. I wonder if in the phone
interview they're going to bring these up. If they didn't notice the errors,
they're probably not the right hire!

~~~
felideon
It probably depends on the type of company, but if there is a compliance/legal
department, they're the ones looking for typos and stuff and even the
marketing department. Specially when it's content (as opposed to a WinForms
dialog box, for example) I don't think proofreading is part a tester's job,
but correct me if I'm wrong.

Of course, the ones you mention are pretty blatant, so I would assume that it
would benefit him to mention them since it shows great attention to detail and
quality.

------
nathanb
I think it's worth pointing out that with many companies, testers do as much
(if not more!) programming as programmers do. I know in our company we have a
metric that tracks what percentage of tests are automated, and guess who
writes the automated tests? Testers. Granted it's a different _kind_ of
programming since it's Perl rather than C, but our testers crank out quite a
bit of code.

------
wheaties
I'd hate to be his HR person. The deluge of resumes is going to be
overwhelming. Although, I don't agree with all that he says I applaud him for
pointing out something companies often times miss: testers should test other
people's code, not their own code.

~~~
gridspy
If you want to get the top .1%, you need to evaluate 1000 people.

~~~
barrkel
Probably a fair amount more than that, as there will be a group of people who
drift from one job interview to the next, not getting hired at any strong rate
anywhere.

