Abandoning speculation entirely is too much. If you just go to in order with a reasonable pipeline length nobody has yet figured a way to smuggle out speculative state on something like an ARM A53. That means giving up performance but only a factor of 4 or so. Giving up speculation entirely would mean an order of magnitude larger performance loss.