Teaching modifies the learner. Prompting doesn't modify the model. It provides additional context that influences a single inference. A person who has learned something can apply it years later without being reminded. An LLM generally cannot unless the knowledge is incorporated into the model itself or provided again.
sleeping helps learning, but people can learn without sleep. You can teach something to a person and they can usually do it immediately in some capacity. If you prevent a person from sleeping they will have reduced capacity to remember things. But it will most certainly not be zero.
I don't think it's generally thought of an advantage that LLMs are stateless. In deployment it's nice, it would be much better if the model could continually learn.
I'm not claiming LLMs are not useful, they most certainly are.