That has not been my experience, and my direction sense is nil. The address system also has not harmed the Japanese distribution industry, which is quite possibly the most efficient in the world.
(These days all the routing is computerized, but prior to that all routing made use of the fact that Japanese addresses get progressively more specific. You used that to sort the parcel at every location so that it got to a distribution center closer to the destination, at which point one of your carriers who had worked in that neighborhood for years would get it directly to the proper door.)
I don’t know how many postmen share each zip code. In Copenhagen the western district has been sub-divided into a lot of zip codes so it could be that each is its own route, but they only did it for the western district, the other districts still only have a single zip code — I suspect that they did it to simplify sorting but found that people are more likely to make mistakes (when you have a dozen zip codes for the colloquially same district).
(These days all the routing is computerized, but prior to that all routing made use of the fact that Japanese addresses get progressively more specific. You used that to sort the parcel at every location so that it got to a distribution center closer to the destination, at which point one of your carriers who had worked in that neighborhood for years would get it directly to the proper door.)