It amazes me that everyone so fetishizes Transformer architectures that they cannot imagine alternative--when the alternative is obvious.