On: From multi-head to latent attention: The evolution of attention mechanisms

[zackangelo]

Not quite a frontier model but definitely built by a frontier lab: Grok 2 was recently open sourced and I believe it uses a fairly standard MHA architecture with MoE.