I totally agree, but even just the basics can be a great addition. What I find odd is even just simple things such as pre or post conditions don't appear to be used widely, let alone cool features built into languages such as Eiffel or D.
Testing private methods? Wrong. Don't do this. They'll be covered by your public API, if they aren't, delete them. If you need to test code that is private, you've got a missing abstraction. Pull that code out into a separate class, make the method public.
Testing you get a null reference when you send in null to a method? So what? You're testing Java/C#/etc... if you do this. Test for behaviour.
Need to use tear down methods? You're not unit testing anymore.
At least the article never recommended testing accessors... Again, please don't.
I just had a quick look through my unit tests for place where I test a private method.
I have a method which looks through an array for a pair of adjacent variables, with a particular property (they have p < V1 > V2 < n for variables V1 and V2, relating to the previous variable and next variable, and where < is a more complicated operator).
Anyway, this gets used lots of times by my algorithm. However, I added some unit tests because I wanted to make sure that the obvious corner cases (beginning and end of array, not occurring) get checked correctly. Hopefully these cases are hit by my input data, but it is hard to construct input data that has particular properties.
I could pull this out into another class, but the method is actually fairly small and not used anywhere else, so that seems like a bit of a waste really.
On checking you get a null reference when you pass null into a method. I make this part of the API of some of my functions (in my case C++, so null pointer actually). I want to check that actually happens, as opposed to some nasty crash. Isn't that checking for behaviour? What else should I do instead?
"Anyway, this gets used lots of times by my algorithm"
You've got a missing class. The fact you're so concerned with testing it highlights this.
The fact it's private I should be free to rename, add or remove parameters. If I did this to your private method, I'd break unit tests, despite the code being private.
For the null reference problem, it depends on your language. But the point remains the same. For public API's, e.g something you let the outside world consume, checking for null etc.. will be required. For internal code, don't bother. Go as high as you can in the stack and fix the root cause of the null reference in the first place.
I've got a missing class, for a 5 line function called from one place? I suppose I can see your point of view, but I'm not sure I agree.
On your second point, I do disagree. I like to view all functions, even internals one, as having a documented API, covering things like bad inputs. This means that all functions (in debugging/assertion mode at least) promise to sanity-check their inputs for null pointers, etc.
In C++ at least I find this a necessity, as a single wrong pointer can cause horrible memory corruption all over my code. The whole point of these assertions is to allow me to find the place in the stack where the null pointers are coming from without pulling all my hair out in the process!
That's not the right way to think about internal APIs. To make changes effectively you need to be willing to redefine those boundaries with a minimum of friction. If you make each function check its inputs then you won't be able to decompose your problem into small enough functions (a good function is usually <10 lines), and you'll mostly be duplicating your own work.
Null is probably the worst mistake in programming history; never use it internally, never call an internal function with it, and then you don't need to make your functions check for it. If you're really paranoid you can use a static analysis tool to enforce that you're doing it right.
Just Null might be a bit limited, however there are many other similar issues (some examples I can think of include functions which take a positive integer, or a non-empty container, or a sorted container, or two containers of equal size, or an integer which is smaller than the size of another argument, which is a container).
I used to be very slack on internal checks. As times goes by I have pushed more of these into the type system where appropriate, and as checks. While unit tests on your external API can be useful, it can be hard to ensure coverage of all the nasty corner cases of your internals, some of which are only hit one time a million, or hundred million.
I find having all these checks makes me more confident about changing my internal APIs. If anyone is calling a function inappropriately for it's new interface, I will quickly get an assertion flagged up, as opposed to code carrying on calling a function incorrectly.
Just so we aren't talking at cross purposes, I feel I should say most of my programming is algorithmic, particularly in AI. In this field it can be devilishly difficult to construct good unit tests and ways of hitting bits of algorithms -- I am currently working on an algorithm which has function which I believe is required but which I cannot construct test data for my external API which causes it be called. Always a little worrying!
>As times goes by I have pushed more of these into the type system where appropriate
Yeah, the type system is the appropriate place for most of these (including null even).
>I find having all these checks makes me more confident about changing my internal APIs. If anyone is calling a function inappropriately for it's new interface, I will quickly get an assertion flagged up, as opposed to code carrying on calling a function incorrectly.
I think we may be talking at cross purposes wrt "public". I don't mean "only ever test your external API". I do mean "only ever test public methods of a class, in the technical sense of public".
Of course a large project will end up with lower-level and higher-level layers, and eventually you do need to define more rigid boundaries between the two to avoid everything becoming an unmaintainable mess, and testing at these boundaries is entirely appropriate. But a single class should belong to one layer or another.
>I am currently working on an algorithm which has function which I believe is required but which I cannot construct test data for my external API which causes it be called. Always a little worrying!
More than a little. My advice (not that you need it) would be to try and encode the constraint that leads to it not being called in the type system, then push it up through the layers.
If they are truly independent it would not matter what language they were written in.
By imposing a constraint of not the same, that imposes a dependency.
Exactly. Refactoring is the process of changing code without changing behaviour. If that means you change some tests so be it. There is no rule to say that you can't do this, but many devs seem to fear this.
That's right Colin. I completely agree, however if you break unit tests during a refactor, yet there is no functional change - bin the tests as long as you have end to end tests higher up. I should add that any new code you add should have unit tests around. E.g the new code added as per the refactor if needs be.
With good unit tests you can indeed refactor without fear, but we should not take the stance of never breaking any unit tests. Providing there is no functional change, break away! Such stance relies on end to end tests/pairing/some other process to ensure you are not breaking valid behaviour though.
Would love to know the best way people handle this.