I'm not sure resource scaling is what's needed here. Just good ol' caching. Shameless plug: I built Cachoid[0] for caching as a service (leveraging Varnish mainly) to help with situations like this.
I understand that this is a shameless plug but I've met teams where the first response to any performance problem is caching
If you're reading this and have performance issues look at caches as a last resort, first figure out why things are slow
I'm assuming a traditional stack here but use mysql jet profiler, or SQL equivalent first, then your languages profiler xhprof?
If your queries are slow or impossible make sure your tables are normalised and indexed, make sure opcache is on and xdebug off
Mysql out of the box is very conservative with its resources, the limits can often be raised and the web server can often be configured to be faster, make sure you're sending the correct http cache headers
There's so many things to consider with performance Please don't denormalise your data slap it in redis/ use varnish and call it a day
No cache is free, eventually you have to pay the pied piper when you do make sure you can afford to have never used the cache at all
Some of my hardest bugs have come from out of sync caches & trying to build performant data architecture around systems that had data stuffed into a redis black hole aged for years..
I dunno, I'd say solve your customer pain first, then mess around with your profiler. If throwing up a cache solves your immediate problem, by all means throw up a cache.
Just make sure it's your first step, not your last step.
Lots of good common sense advice and practical experience there. I've had my fair share as a devops/sysadmin working on crazy systems (more worse than better haha). So it's unsurprising how common sense isn't always put into practice.
I do believe caching to be the proverbial Swiss Army Knife of performance. Some of its tools can be used properly and others as band-aid / stop gap. Even band-aids serve a useful purpose. But I can definitely see how a system can accrue technical debt when over leveraging caching.
But to believe in prevalent, perfectly built systems is to deny the status quo of the landscape. When you democratize development like WordPress has done with its plugins and Node has done with npm, as popular examples, you definitely have it coming.
Also if you're doing key scans in redis, you're doing it wrong, it's possible to work around with indexes you have to manually maintain, but it's not worth it
Maybe state that it is a Varnish configurator. I initially thought it was a new caching proxy, thinking "Why would anyone need that when they have Varnish?".
My point is that a caching server is a resource like any other. Adding additional backend capacity, and adding additional caching capacity are both "adding resources"/"scaling up".
Doing smart things to do more with the same resources, do the same with fewer resources, or by replacing expensive resources with cheaper resources, is optimization, not scaling.
I totally agree. Scaling with docker containers is not a silver bullet for a LOT of problems. Optimizing your boring/old tech stack can much more often a better investment than betting or (even worse) switching on docker as your foundation.
