
How not to teach people to debug - sliced_lime
http://www.entertainingcode.com/archives/black-box-black-box-testing/
======
lincolnq
I've seen these systems and I think they can be good for teaching programming.

Basically, they force the student to think about the whole problem at once,
design a complete solution, and implement it without making mistakes. If you
screw up, you'll have a very tough time finding the mistakes, since the system
doesn't tell you where the mistake might be. It prevents you from repeatedly
patching your system rather than redesigning it, which is a big problem I see
in a lot of programmers.

It's not good for the absolute beginners, but for people who know a little bit
how to think about algorithms and data structures, and are trying to improve
their code quality, it seems like a good exercise. I know I've had some
rewarding experiences debugging code by thinking about it; using a debugger,
while easier, seems generally less satisfying.

~~~
zokier
I don't see how these systems limit your ability to debug your code. You need
just write your own testcases, which is good exercise in itself, and debug
against those. If it passes your own testcases and still fails in the grading
system, it indicates that either your testcases are not good enough, or that
there is a bug in the system.

------
amalcon
The problem isn't just that the system doesn't give enough feedback. The
problem is that it also gives it at the wrong time.

In the real world, if your own testing misses a bug, you don't find out until
it's live and causes a problem. In most grading systems, if your own testing
misses a bug, you don't find out until the grader's (hopefully more
comprehensive) testing catches it and you lose points.

In this system, if your testing misses a bug, you're immediately told that a
bug exists, and given no other information -- but there's no actual penalty
for doing this. In the real world, you're told what the incorrect behavior
_is_ , but there's a penalty in that this incorrect behavior has at least
inconvenienced someone.

------
mquander
My easy (if extremely unpopular) fix:

You can only submit your code for a given assignment a maximum of N times,
where N is appropriate to the assignment difficulty and the level of the
class. Then you can't pursue a haphazard "try this? nope, didn't work"
strategy, and you have to reason out your program correctly on your own
without appealing to the black box.

Alternately, if you wish to be less draconian, simply deduct points for every
incorrect submission.

~~~
zokier
The places I have seen this kind of system in use, they work just like that.
Limited number of submissions and flexible pointing system. Those systems gave
points according to how many tests you passed, and there was minimum score to
pass. The other system I have seen, also scored by coding style, ie.
commenting and correct indentation etc. You could see style points and
functionality points separately and there was separate minimum points (usually
quite high in style).

And IMHO these are good tools. But they don't teach you debugging (at least
not directly). They teach that you need to test and debug your code _before_
you submit it, which imho is good practice anywhere, ie you should test and
debug your code before committing to SVN (or push your changes to main tree in
DVCS-talk).

Of course that should be told to students, and they should be taught proper
debugging/testing-methods. I mean these kind systems do not prohibit doing
your own tests and debugging those before submitting, now do they?

------
ShabbyDoo
This is how the Facebook puzzles work except that you only get a pass/fail
response. To help people ensure that they aren't failing due to some sort of
packaging problem, they have simple puzzles (one step up from hello world),
and it's not frowned upon to help someone with those.

What Facebook puzzlers end up doing is creating their own testcases and
validating that those testcases encompass all the interesting corner cases and
are complex enough to prove the efficiency of the algorithm in use.

Perhaps the hidden bit of learning from "The System" is that it forces
students to think through the entire scope of possible inputs for their
program and ensure they test against a representative set of cases.

------
mseebach
I have a similar experience from college. We'd get an empty Java project, a
"runner" and a set of simple test cases in the form of input/expected-output
files.

We were told that the teachers would run hundreds of test cases more on out
code, so we'd better think about it.

A very important thing to remember is that this course was a programming
course (concurrent, I think), NOT a software development course. Many of these
"I had to do this stupid thing in school that I'll never need in the real
world"-rants miss those differences.

