Object permanence and a communications channel is enough for this. Give children (who get along with each other) a pile of sticks and leave them alone for half an hour, and there's half a chance their game will ignore the sticks. Most children wouldn't want to have their play mediated by the computer in the way you describe, because the ergonomics are so poor.
I'm reminded of that guy who bought an AI enabled toy for his daughter and got increasingly exasperated as she kept turning it off and treating it as a normal toy.
That thread has a lot of good observations in it. I was probably wrong in framing the problem as "ergonomics".
> Dr. Michelle (@MichelleSaidel): I think because it takes away control from the child. Play is how children work through emotions, impulses and conflicts and well as try out new behaviors. I would think if would be super irritating to have the toy shape and control your play- like a totally dominating playmate!
> Alex Volkov (Thursd/AI) (@altryne): It did feel dominating! she wanted to make it clothes, and it was like, "meanwhile, here's another thing we can do" lacking context of what she's already doing
> The Short Straw (@short_straw): The real question you should ask yourself is why you felt compelled to turn it back on each time she turned it off.
> Angelo Angelli JD (@AngelliAngelo): Kids are pretty decent bullshit detectors and a lot of AI is bullshit.
> Foxhercules (@Foxena): […] I would like to point out again that the only things I sent this child were articulated 3d prints. beyond being able to move their arms, legs and tails, these things were made out of extruded plastic and are not exactly marvels of engineering. […] My takeaway from this is that, this is what children need. they don't need fancy with tons of bells and whistles with play on any sort of rails. And there's not a thing that AI can do to replace a Child's imagination NOR SHOULD IT.
The majority of American children have an active Roblox account. Those who don't are likely to play Minecraft or Fortnite. Play mediated by the computer in this way is already one of the most popular forms of play. Kids are going to go absolutely nuts for this and if you think otherwise, you really need to talk to some children.
[1] Not meant as an insult. Working professionals don't have time for this stuff.