The goal of testing is to prevent defects from surfacing in production. So track every defect that surfaces in production, so that you can watch that go to zero over time.
Whenever a defect comes up in production, edit the test plan such that you would have caught that defect. Now you won't be bitten by that class of defect in production again.
If you keep updating the test plan in this way you will see a dramatic drop in defects released to production. Once you've done this for a while, you will probably discover that your biggest source of defects released into production have to do with how different your test environment is from your production environment. So you will then start attacking that issue by setting up a proper staging environment, where the staging environment mirrors production as closely as practical.
Then you will start to discover that your biggest source of defects released into production becomes other things, such as little problems with your release methodology, which you can then address.
But the key concept here is: document what your test plan is, and continuously improve it. It's important to note that you must actually follow the documented procedure for this to work. If you write a document so big that you won't actually do it, you're doing it wrong, make a smaller document. If you feel like you only need to do 2 minutes worth of resting, document what you will do during those 2 minutes. You can start with an empty test plan and that will work, as long as you continuously improve your test plan. The same goes for the procedures that you use to deploy. Always follow the same procedure exactly as documented, because you will need to improve that procedure.
I have followed these procedures at a number of companies and in a variety of environments, and seen it turn chaotic messes around many times.
Once you have this process down solid, you can automate some or all of it. But the important thing is the overall set of processes around testing and deploying software, and the process for improving those processes. How much of it is automatic versus manual matters a lot less.
As for resources, I'd recommend books on continuous improvement. Because as you get better at testing, you'll discover that General process improvement is what you really need in order to cover the range of things that cause defects in production.