Imagine a web server that didn't wipe plaintext passwords or encryption keys from its memory after finishing with them. If the web server was remotely exploitable then it could be possible to obtain the contents of the memory of that process remotely, thus possibly leaking passwords or other sensitive information of other people that have connected to that web server at some point in the past.
> how can you be sure that someone didn't access it before your memset() call?
True, but there's a difference between having a very small window of opportunity where the data could be obtained via a remote exploit, and leaving the window wide open for possibly endless period of time.
Obviously you shouldn't have any remote exploits in the code in the first place, but it's good practice for secure programming to keep the sensitive information in memory for as short a period as possible just in case there is something that you aren't aware of.