I have a similar desire, I'm considering trying out AWS' RDS Aurora Serverless v2. (ugh, that's a mouthful) Full Postgres/MySQL but extremely quick scale up and down so as long as you have small use, it shouldn't cost a ton.
I'm also super interested in Cloudflare D1, looking to get my hands on it and try it out.
Check out PlanetScale [0] (MySQL). It's free for hobby projects and then a flat $30/mo (+usage if you go over their very high limits). You can run your dev/qa/prod environment all on one "database" using their branches and you also get zero-downtime schema changes which is pretty sweet. I've been very happy with them so far.
I migrated from RDS Aurora Serverless V1 after being frustrated with the high prices (V2 is even worse) and the annoyance of sleeping DBs (for my dev/qa environments, I couldn't afford $45x3/mo). Now I get all 3 for only $30/mo and safe/easy schema migrations. It's been a joy to work with and their support has been very responsive. This week is a "launch week" for them so you'll probably see a few posts about it this week and I encourage you check them out if you are looking for scalable managed MySQL DBs.
V2 is a garbage fire and that was before I knew it didn't scale to zero.
It doubles the cost of V1 and they have the gall to say:
> most Aurora Serverless customers can lower costs by moving to v2 due to a lower starting capacity of 0.5 ACU (vs. 1 ACU in v1)
Oh, why thank you AWS, how kind of you. I can go to 0.5? That doesn't matter at all, the price is the save as 1 ACU on V1.
D1 is interesting but if you want Serverless MySQL then check out PlanetScale [0]. It's free for hobby projects (it will turn off if you go 1 week with no activity) and then it's a flat $30/mo and you can have 2 production branches and 5 development branches. I've been very happy with them so far. The usage-based pricing is a little scary (I know I don't think in rows read/written) but the limits are super high and so far my usage has been barely a blip.
I'm also super interested in Cloudflare D1, looking to get my hands on it and try it out.