Robert Leighton: "Feynman has a peculiar property, which is that at the time he's explaining something, it appears very clear and transparent--you can see how everything fits, and you go away feeling very good about it, as if, "Well, there's a lot of loose ends there that I want to follow up on; but boy, wasn't that great!" And about two hours later, like what they say about Chinese food, it's all gone and you're hungry again. And you don't remember what happened.[0]
I have read his QED book many times and I experience the same clarity when reading as you mention. Then later I lose that and am able to re-read it with the same enthusiasm I had the first time. So that property does have its advantages. (grin)
BTW, I have this same experience with many other science books for laypeople. It is very possible that I do understand them while reading but then I am unable to remember the complex thoughts.
[0]: Feynman's Tips on Physics: Reflections, Advice, Insights, Practice - https://books.google.com/books?id=FZ4fAQAAQBAJ&lpg=PT32&ots=...
edit: wrong name.