The problem really is that it is impossible to verify that the content someone uploads came from their mind and not a computer program. And at some point probably all content is at least influenced by AI. The real issue is also not that I used chatgpt to look up a synonym or asked a question before writing an article, the problem is when I copy paste the content and claim I wrote it.
Ignoring the privacy and security issues for a moment, how would having a digital ID prove that the blog post I put on my site came only out of my own mind and I didn't use an LLM for it?