Its really easy to test if an LLM is actually self aware when you have your own harness. Simply take a conversation with one LLM and continue that conversation with a different LLM and then ask the new LLM about what it wrote you earlier. Mostly the result will be that it will tell you about what the other LLM wrote as if it was what it wrote and it will not deny that it didn't write any of it. I say mostly as these they're nondeterministic and do go off the rails from time to time. The only things LLMs are aware of is what's in their context and the middleware + harness completely controls what is the context before generation begins.
There is a huge difference between being self aware and being able to effectively generate language relating to self awareness.
There is a huge difference between being self aware and being able to effectively generate language relating to self awareness.