"The Memory Police", by Yoko Ogawa [1]. An island where things get disappeared by the Memory Police - physically from the world and from people's minds. A few people are immune to forgetting, so the Memory Police is on the case.
It's not really similar,since qntm writes more in the vein of SCPs - paranormal occurences while The Memory Police is more sinister in an an ethical and political way, a bit like state mandated Alzheimer.
[1] - https://www.goodreads.com/en/book/show/37004370-the-memory-p...