This is super useful, thanks for posting. I hate AWS complexity-as-a-service, just give me a damn baremetal machine and let me run and tune my own services. I love having full visibility into raw disk/cpu performance and even benching my hardware.
So many useful comparisons here, Bunny.net vs. CloudFlare, and the fact that you got this far without even using fail2ban!
Questions (not necessarily for OP, but for anyone)
- Give us an idea of the maximum load this server is doing in terms of requests per second?
- Anyone choosing AMD Epyc over Intel to avoid spectre/meltdown mitigation slowdowns?
- Any use of postfix or other SMTP forwarding via Postmark or another email provider?
- What is your Postgres configuration like for your heaviest loads? max_connections? Just wondering how Postgres scales across cores.
- Request rate is pretty low on average and peaks at around 15 requests/second. That's nothing really and it would probably take 100x or more request rate before I saw any kind of CPU bottlenecking or similar; my upload would probably bottleneck first. The biggest resource usage on the server comes from some expensive database queries made by some of my applications.
- I'd definitely be down to use that kind of CPU but it wasn't available at the price point I wanted. Most of my stuff isn't CPU-bound anyway.
- I used to self-host my own mailserver and webmail, but to be honest it was a disaster. Even after spending tons of time configuring reverse DNS, DMARC, SKIN, SPF, and the other list of arcane and specialized stuff you need, I still had tons of issues with mail I sent going to spam. I gave up and pay Google to host my mail for my own domains now.
- I really haven't done much tuning. MySQL/MariaDB is the DB that gets the most load and its settings are most default. I have put a lot of effort into profiling my apps' query patterns and making sure proper indexes and schemas are in place, though.
So many useful comparisons here, Bunny.net vs. CloudFlare, and the fact that you got this far without even using fail2ban!
Questions (not necessarily for OP, but for anyone)
- Give us an idea of the maximum load this server is doing in terms of requests per second?
- Anyone choosing AMD Epyc over Intel to avoid spectre/meltdown mitigation slowdowns?
- Any use of postfix or other SMTP forwarding via Postmark or another email provider?
- What is your Postgres configuration like for your heaviest loads? max_connections? Just wondering how Postgres scales across cores.