What you need is a super-deep immortal vault that isn't anywhere near any known fault lines, and who's ingress point is sufficiently above ground, and can be accessed safely if under water, and supplies its own electricity for thousands of years.
Everything outside of the facility will have to be electrically neural up to extreme voltages (due to the possibility of plasma storms arising that make the worst thunderstorms on Earth look like a nice day outside).
I kid, but the more secure a vault is the more irretrievable it is, which only makes it useful to aliens, not us.
The unanimous response was "not much". Mostly anomalously concentrated resources. So... if they are right, even though we have no reason to believe there was a developed civilization before us, it's not that crazy to merely entertain the possibility.