SanDisk still makes the Sansa Clip which satisfies your requirements and takes microSD cards. I have a previous generation of the same, it's a very nice device that does (mostly) one thing and does it well.
And of course, as far as bands releasing albums on SD cards with artwork... They tried that with something called MQS SD. Needless to say, it didn't go so well.
Very close to what I'm looking for. I was thinking pretty hard about it a few months ago, and I did considered that exact one. Ideally, I'd like no screen (small LCD at most); push buttons (SanDisk looks like touch, but the description doesn't clarify); SD card; standard replaceable battery (this is the hardest feature to find); ogg and flac support.
https://www.sandisk.com/home/mp3-players/clip-jam
And of course, as far as bands releasing albums on SD cards with artwork... They tried that with something called MQS SD. Needless to say, it didn't go so well.
https://www.youtube.com/watch?v=GbGDPFVjvVU