Regarding syntax, what I find most annoying is the lack of a standard way to do common things across different functions. For example, to ignore missing values in a dataset (denoted by NA), there are as many variants are there are functions: na.rm, na.omit, na.action etc. So there's different syntax for doing the same thing in different parts of the language.
na.rm is a argument name, na.omit a function, na.action an attribute. They are not the same thing.
Anyway, you're right with respect to the lack of standard way. One thing I personally find most annoying is function naming. Some functions use the dot convention (do.this), some use camel case (doThis), some use underscores (do_this) etc. And what is most annyoing: this is even true for novel functions that were just introduced in recent releases.