Yes, you would also audit the quality system for your suppliers to confirm they are sufficiently controlling for upstream changes. In theory you can have all your ducks in a row.
"In theory" is doing a lot of heavy lifting there. ;)
Depending on the product and quantity, you can factor your purchase price level times 2-10 for every level of sub- and sub-sub-supplier you want to have audited to your "wacky spec" - which may even still sound kinda reasonable, until you realize your attack surface is basically fractal to the n-th degree. The amount of process steps and auxiliaries used in manufacturing is absolutely staggering.
Edit: I need to add this depends a lot on the sector. There's useful certificates for a lot of industries, if you choose to believe them.