I think it's important to distinguish between "deleting" and "securely erasing" here. The former often provides only a layer of obscurity, while the latter takes expertise to perform reliably.
Ideally the wallet's key deletion functionality would include ensuring the private data doesn't remain on disk (and warn if the media makes this impossible), but I think this is more or less impossible; a secure erase facility really needs to be implemented at the OS level, since it requires knowledge of the workings of the filesystems in use as well as its interactions with the physical media. And it gets worse; in the case of any solid state devices that perform their own write balancing, even the operating system can't know what data has actually been lost.
Of course, if someone has your (unencrypted) hard drive digging around for old ZCash key data is probably low on the list of privacy-compromising information available to them anyway.