Reckon you could give us some idea of the other side? What are the alternative explanations proposed? The passage here (albeit translated) seems pretty unequivocal.
It also doesn't seem that unlikely. I know a fair few adults who mutter to themselves under their breath while reading, never having managed to kick the habit of reading aloud.
The argument for the Ambrose case is that the surprise isn't him reading silently, it's him reading silently while he has guests. (and elsewhere in the text Augustine mentions himself reading silently by himself in a way that suggests it's entirely normal)
It also doesn't seem that unlikely. I know a fair few adults who mutter to themselves under their breath while reading, never having managed to kick the habit of reading aloud.