I was not here arguing in favor of explicitly tiered memory. The implied answer to your original question, "why not have a pin to cache functionality?" is that it's effectively the same as having another OS managed memory layer, which is bad since it complicates the architecture. I'll take some cache misses over having to manage it explicitly.
Not only that, if you had enough cache to fit everything then there wouldn't be cache misses, and if you didn't, cache misses are pretty unavoidable.
It's like the existing APIs for pining things in memory so they can't get paged out. They have very specific uses and normal programs generally don't use them and shouldn't.
