It seems like there are at least two kinds of software people: the clever and the careful. I'm oversimplifying of course but it seems that some people really enjoy making things in new, clever ways. They like new programming languages, they like doing stuff in far fewer lines of code than it would normally take, they love clever techniques, new algorithms and so on. In contrast, the careful people just really want to make sure it works and don't really care as much about using some neat trick.
I think all good programmers have aspects of both. I think I tend to skew more toward the clever side but I think I have a healthy dose of careful. Another programmer I know is super paranoid about bugs and is much more focused on being careful. Perhaps that is because she spent a lot more time in a real software company which ships code to big customers where as I spend a lot more time on research style projects.
So I think I have a relatively good sense of how to find the clever programmers. But my company is currently searching for someone more on the careful end. How does one find the careful programmers? How does one find the kind of person how says "My system never goes down. It's tested 10-ways from Sunday and is bulletproof"?
ask to see some large-scale code samples. if they can't provide any (which is understandable, as these are often the property of former employers; and a lot of people who lead real lives don't have a lot of spare time to work on open source projects), just see how they actually do on an actual, real, end-to-end project.
it doesn't even have to be terribly deep or complicated. but the main thing is, it should be VERY non-sexy. the unsexier, the better.
because that's what distinguishes the (supposedly) clever ones from the careful ones: they just don't have the patience or willingness to "sully" themselves for the sake of, you know, that "other 90%". particularly for something that just isn't all that cool, and that they can't get bragging rights for.
so you'll find that their stuff works, umm, "mostly." except for this or that (frequently and naturally occurring) corner case. and for all the extra manual housekeeping it requires. oh and that it also just tends to break. a lot. (to which they'll always be able to say, "oh, I'll just fix that"... for as long as they're around, that is.)
and for the fact that you'll just be able to see that everything's done in a slapdash, one-off way that's hard to read or generalize.
and I DON'T mean in the pragmatic, "worse is better" sense; but in the fricken' careless, "I'm too good for this gruntwork" sense.
ultimately the real distinction isn't clever v. careful; the smarter programmers, in my experience, are always, without exception, also the very careful ones.
it's more a distinction between those who talk -- and like to get credit for projects that are really collective efforts -- and those who simply do (and take pride and satisfaction in simply doing it very, very well.)