Here is Richard Feynman talking about the same problem:
While in Kyoto I tried to learn Japanese with a vengeance. I worked much harder at it, and got to a point where I could go around in taxis and do things. I took lessons from a Japanese man every day for an hour. One day he was teaching me the word for "see." "All right," he said. "You want to say, 'May I see your garden?' What do you say?" I made up a sentence with the word that I had just learned. "No, no!" he said. "When you say to someone, 'Would you like to see my garden? you use the first 'see.' But when you want to see someone else's garden, you must use another 'see,' which is more polite." "Would you like to glance at my lousy garden?" is essentially what you're saying in the first case, but when you want to look at the other fella's garden, you have to say something like, "May I observe your gorgeous garden?" So there's two different words you have to use. Then he gave me another one: "You go to a temple, and you want to look at the gardens..." I made up a sentence, this time with the polite "see." "No, no!" he said. "In the temple, the gardens are much more elegant. So you have to say something that would be equivalent to 'May I hang my eyes on your most exquisite gardens?" Three or four different words for one idea, because when I'm doing it, it's miserable; when you're doing it, it's elegant.
Japanese language is very tightly integrated with Japanese society, to the extent that I would cheekily argue it doesn't even count as speaking Japanese outside of the context of a Japanese social setting or structure.
To elaborate on your point regarding the proliferation of vocabulary from honorific synonyms, consider how much of the grammatical gamut is wrapped up in identifying the speaker's relationship with the listener. Who is the in-group? Who is the out-group? Is the speaker a man or woman? What is the relative social position of the speaker and listener? Is the utterance written or spoken?
Japanese is a language that identifies the answers to all of these questions with explicit grammar markers, while omitting number, case, conjugation, and employs only two tenses (realis and irrealis).
Learning Japanese in a classroom outside of Japan, or heaven forfend, through individual study alone, is an awkward and disconnected experience, since there is scarce opportunity to practice the major components of Japanese grammar in a real way.
Of course we all operated in Japanese in a professional setting daily, so knew the "correct" way to speak, but also had the ability to just drop all that when we wanted (which Japanese people basically never do). Totally different way to use the language, and if any Japanese people happened to overhear, it kind of blew their minds.
I found it very interesting to unpick all the different levels of linguistic, cultural, and social meanings and conventions in the language.
1. It is told in a sensationalist way that either is or is close to being orientalism. I believe Feynman was known for his story-telling skills (sometimes with some hyperbole for effect), and I think this case is no different.
2. That said, for someone doing a crash course in Japanese, I can see how this can come across as being confusing... but it's not terribly complicated for a smart person like Feynman. Maybe the teacher wasn't particularly skilled...
3. That said, if Japanese is learned in context over some period of time, using the wrong verb form in these contexts just intuitively sounds wrong to the point that it makes me (and others) reflexively wince.
4. That said, the deeper levels of keigo (not the minor stuff mentioned by Feynman) can actually be challenging to learn, even for Japanese people. This is mainly because the content is unfamiliar since the contexts are not experienced regularly (or ever)... until they are, then the keigo becomes natural.
5. That said, someone like Feynman (and in fact most westerners) would get a pass for using any form of "see" if it is in Japanese. Most Japanese I know have low expectations (rightly or wrongly) for foreigners speaking Japanese, so using the wrong verb form typically doesn't even raise an eyebrow.
The simple elementary school stuff isn't hard. But the highschool and college-level stuff? Ugh. Thousands of characters with multiple pronunciations each, sometimes with very tiny differences between them.
Picking the right pronunciation on the other hand...ends up being totally reliant on if I know the vocab word or not.
Each verb has different forms for formal/informal/very formal, but they're highly regular. For example "iku" is "go" and "ikimasu" is "go (polite)". "wakaru" is "understand" and "wakarimasu" is "understand (polite)". It's not like you have to learn four forms of each verb, you learn the basic one and then you know the rest. The ultra-formal versions, which are more rarely used, are also regular - they involve the infinitive of the verb plus a conjugated auxiliary verb IIRC, like saying "presume to go", "presume to understand" etc.
見る -> 見ます -> ご覧になる -> 拝見する
That is a lot different than just learning four different conjugations for 見る - it becomes an entirely different word!