The Internet is trying to give the gift of free traffic and The Onion is saying: "no thanks." Most sites have to pay for traffic, and wouldn't be trowing it away.
Way to play it like Big Media. What's next? A pay wall?
As a web analyst, I can tell you that in general old, deprecated content does not get very many visits except from spiders. I would not at all be surprised if the marginal ad revenue is break-even compared to the extra server load.
Note also that they're not just throwing the traffic away. It's a decent 404. It's not the best I've seen, there's room for improvements, but it's directing people straight to the archive so they can look for the story they were linked to.
Spidering and organic activity are bundled together in this 66% number. It would be interesting to know what percentage of this was people who wanted to see an Onion page.
The spiders and organic visitors should have been 301 redirected to the correct location. Spiders learn the 301 redirects, and some of their articles may even benefited from better search rankings.
These hits will never (and should never!) translate into even a single real user in my case.
I don't think you can fix this with code, much less your usual tactic of adding heuristics to moderate/voteweight/ban.
I, for one, won't try to second-guess your decisions. The Onion has been around for a long (internet) time, and I suspect most of the Web101 suggestions have been considered.
Besides, you guys have left me in stitches too many times for me to be particularly critical. Thanks for existing!