This actually calls for advances in analytics packages. If you could specify the threshold on a per page (or regular expression for complex requests), and have the system track and notify you of threshold exceptions, this wouldn't be too difficult to manage with some nice sliders.
Sounds like a good problem for an analytics startup to tackle.
Sounds like a good problem for an analytics startup to tackle.