Some small public APIs are backed by large enough implementations that it pays off to be able to test implementation details. Sure, it might be "poorly factored" code that should have a bigger API and smaller guts, but that's not always an something you can change. Also, writing tests for internal behavior before refactoring can give you a good blueprint for how the refactored code should behave--being able to read the tests to specify unclear behavior is, while far from enjoyable in some cases, better than nothing.
You're right that there are some pretty silly test suites that break encapsulation for a coverage number without actually testing anything useful, though.
I don't think an absolute "all tests must behave thus" rule (e.g. coverage requirements, "only test public functionality", "refactor the instant something isn't easily testable") is useful. Explain the benefits of each path, and make sure the decision of what compromise to make--and in any project more than a one-developer hobby, you will have to compromise here eventually--is in the hands of people with the experience and common sense to make the right one.