In theory you could store it both offsite (using a hidden Micro SD card) and on your phone simultaneously, with a special key combination, e.g. pressing Vol Up + Vol Down + Power causing the phone to be rebooted, and the key erased from RAM... So in that case you would need to get the offsite copy, from GPS coordinates you have memorized, using a GPS receiver that is known not to leave any location information in it's EEPROM / Flash.
Of course all this info has to be double checked to see if it actually works, and forensic tools run against the phone to be really really sure the key's not being written to Flash in any way, or remains in RAM after a reboot.
Sure, knock yourself out if that's your thing. But what I meant was basically "power user" level devices, like what systemd-cryptenroll lists. So fido2, pkcs11 (smartcards and tokens) and tpm2. If you want to bury any of them in the woods, I guess you can.
Of course all this info has to be double checked to see if it actually works, and forensic tools run against the phone to be really really sure the key's not being written to Flash in any way, or remains in RAM after a reboot.