Over a long enough time, every API will be called in every possible way. If an API misbehaves, some code somewhere will begin to rely on the specific kind of misbehavior. This is just the reality of publishing API's, so you deal with it and fix bugs as they are discovered.
If you make your entire class definition public in a large project, you will eventually have code poking at the internals of your class. I've seen this countless times, it's some kind of inevitable natural thing.
You know everything about how your code works, but I don't. A well-defined API with clear access rules gives me a fair chance to avoid using your class in ways you didn't intend.
In small-enough teams, or with small-enough projects, it's possible for everyone to understand (mostly) everything, but there's some magic number of concurrent developers where that privilege ends.