...Then he basically asserts that we can extrapolate the near-term availability of tons more compute power from Moore’s Law, which is where he lost me.
We’re already running into the limits of physical law in trying to move semiconductor fabrication to smaller and smaller processes, and there are very real and interesting challenges to be overcome before, I think, we can resume anything close to the exponential growth we’ve enjoyed over the last 40 years.
This guy may well think a lot about these difficulties, but not mentioning them at all made his argument sound incredibly naïve to me.
That's not what he's asserting. Even with Moore's law dead, OpenAI claims there is significant room with ASICs, analog computing, and simply throwing more money at the problem. There is a ton of low-hanging fruit in Non-Von Neumann architectures. We should expect it to be plucked, as we have huge use case which is potentially limitlessly profitable.
That is irrelevant. You just scale horizontally, more and more data-centers. Sure, it will not be free like in the past.