A little worried how young children watching these videos may develop inaccurate impressions of physics in nature.
For instance, that ladybug looks pretty natural, but there's a little glitch in there that an unwitting observer, who's never seen a ladybug move before, may mistake as being normal. And maybe it is! And maybe it isn't?
The sailing ship - are those water movements correct?
The sinking of the elephant into snow - how deep is too deep? Should there be snow on the elephant or would it have melted from body heat? Should some of the snow fall off during movement or is it maybe packed down too tightly already?
There's no way to know because they aren't actual recordings, and if you don't know that, and this tech improves leaps and bounds (as we know it will), it will eventually become published and will be taken at face value by many.
> For instance, that ladybug looks pretty natural, but there's a little glitch in there that an unwitting observer, who's never seen a ladybug move before, may mistake as being normal. And maybe it is! And maybe it isn't?
Well, none of the existing animation movies follow exact laws of physics.
Animation doesn't follow exact laws of physics, but the specific ways they don't follow physics have very deliberate intent behind them. There's a pretty clear difference between the coyote running off a cliff and taking 2 seconds to start falling, and a character awkwardly floating over the ground because an AI model got confused.
>but the specific ways they don't follow physics have very deliberate intent behind them.
That is only true for well crafted things. There's plenty of stuff that's just wrong for no reason beyond ease of creation or lack of care about the output.
Although, plenty of kids have tied a blanket around their necks and jumped off some furniture or a low roof, right? Breaking a leg or twisting an ankle in their attempt to imitate their favorite animated superhero.
Take the example to the extreme: In 10 years, I prompt my photo album app "Generate photorealistic video of my mother playing with a ladybug".
The juxtaposition of something that looks extremely real (your mother) and something that never happened (ladybug) is something that's hard for the mind to reconcile.
The presence of a real thing inadvertently and subconsciously gives confidence to the fake thing also being real.
I think this hooks in quite well to the existing dialogue about movies in particular. Take an action movie. It looks real but is entirely fabricated.
It is indeed something that society has to shift to deal with.
Personally, I'm not sure that it's the photoreal aspect that poses the biggest challenge. I think that we are mentally prepared to handle that as long as it's not out of control (malicious deep-fakes used to personally target and harass people, etc.) I think the biggest challenge has already been identified, namely, passing off fake media as being real. If we know something is fake, we can put a mental filter in place, like a movie. If there is no way to know what is real and what is fake, then our perception reality itself starts to break down. That would be a major new shift, and certainly not one that I think would be positive.
I'm still waiting on the future waves of PTSD from hyper realistic horror games. I can't think of a worse thing to do then hand a kid a VR headset (or game system) and have them play a game that is designed to activate every single fight or flight nerve in the body on a level that is almost indistinguishable from reality. 20 years ago that would have been the plot to a torture porn flick.
Even worse than that is when people get USED to it and no longer have a natural aversion to horrific scenes taking place in the real world.
This AI stuff accelerates that process of illusion but in every possible direction at once.
As much as people don't want to believe it, by beholding we are indeed changed.
Distinguishing reality from fiction is useful, but it doesn’t shape our desires or define our values. As a culture, we’ve grown colder and more detached. Think of the first Dracula film—audiences were so shaken by a simple eerie face that some reportedly lost control in the theater. Compare that visceral reaction to the apathy we feel toward far more shocking imagery today.
If media didn’t profoundly affect us, how could exposure therapy rewire fears? Why would billions be spent on advertising if it didn’t work? Why would propaganda or education exist if ideas couldn’t be planted and nurtured through storytelling?
Is there any meaningful difference between a sermon from the pulpit and a feature film in the theater? Both are designed to influence, persuade, and reshape our worldview.
As Alan Moore aptly put it:
"Art is, like magic, the science of manipulating symbols, words, or images to achieve changes in consciousness."
In my opinion the old adage holds true, you are what you eat. And we will soon be eating unimaginable mountains of artificial content cooked up by dream engines tuned to our every desire and whim.
> Distinguishing reality from fiction is useful, but it doesn’t shape our desires or define our values. As a culture, we’ve grown colder and more detached. Think of the first Dracula film—audiences were so shaken by a simple eerie face that some reportedly lost control in the theater. Compare that visceral reaction to the apathy we feel toward far more shocking imagery today.
Huh? The first half of this contradicts the second. We haven't "grown colder and more detached", we've adapted to the fact that images are no longer reliable indicators of reality. What we do and don't value in the real world hasn't changed.
> And we will soon be eating unimaginable mountains of artificial content cooked up by dream engines tuned to our every desire and whim.
Always has been. Multi-channel TV was already that, and attracted the same kind of doomerism.
I looked at the Sora videos and all the subject "weights" and "heft" are off. And in the same way that Anna Taylor-Joy's jump in the The Gorge at the end of the new movie trailer looked not much better than years-ago Spiderman swinging on a rope.
Feels like you're looking for a strawman argument, and may have found one.
I would retort that animation and real-life-looking video do different things to our psyche. As an uneducated wanna-be intellectual, I would lean toward thinking real-looking objects more directly influence our perception of life than animations.
Animation can look real though, e.g sci-fi vfx. But maybe you’re concerned about how prolific it may be? I could see that. Personally I think it’ll be fine. It’s just that disruptive tools create uncertainty. Or maybe I’m overcompensating to avoid being the “old man yelling at cloud” dude.
There’s also a huge difference in what people, even children, expect when sitting down to watch a movie versus seeing a clip of some funny cat/seal hybrid playing football while I’m looking for the Bluey episode we left off on. My daughter is almost five and cautiously asks “is that real?” about a lot of things now. It definitely makes me work harder when trying to explain the things that don’t look real but actually are; one could definitely feel like it takes some of the magic away from moments. I feel alright in my ability to handle it, it’s my responsibility to try, but it isn’t as simple as the Looney Tunes argument or, I believe, dramatic effects in movies and TV.
Yet, in a movie setting it's clear something is a special effect or alike which is not the case for GenAI. Massive underestimation of the potential impact in this thread, scary.
Maybe. Or maybe some people massively underestimate our ability to cope with fiction and new media types.
I am sure that there were people decrying radio for all these same reasons (“how will the children know that the voices aren’t people in the same room?”)
Not a bad point, those representations have, in some cases, caused widespread misunderstandings among people who learn about those concepts from movies... and this is all while simultaneously knowing "it's just a movie".
Yes but a movie is a movie whereas these AI-generated videos will likely be used to replace stock footage in other (documentary, promotional, etc.) contexts
If the producer wants to publish bad physics, they get bad physics.
If the producer wants to publish good physics, they get good physics.
It doesn't matter if it is AI, CGI, live action, stop motion, pen-and-ink animation, or anything else.
The output is whatever the production team wants it to be, just as has been the case for as long as we've had cinema (or advertising or documentaries or TikToks or whatevers).
You don't have full control over AI-generated images though, or not to the same extent producers have with CGI.
There's a video on sora.com at the very bottom, with tennis players on the roof, notice how one player just walks "through" the net. I don't think you can fix this other than by just cutting the video earlier.
There's already techniques for controlling AI generated images. There's ControlNet for Stable Diffusion and there are already techniques to take existing footage and style-morphing it with AI. For larger budget productions I would anticipate video production tooling to arise where directors and animators have fine grained influence and control over the wireframes within a 3D scene to directly prevent or fix issues like clipping, volumetric changes, visual consistency, text generation, gravity, etc. Or even just them recording and producing their video in a lower budget format and then having it re-rendered with AI to set the style or mood but adhering to scene layout, perspective, timing, cuts, etc. Not just for mitigating AI errors but also just for controlling their vision of the final product.
Or they could simply brute force it by clipping the scene at the problem point and have it try, try again with another re-render iteration from that point until it's no longer problematic. Or just do the bulk of the work with AI and do video inpainting for small areas to fix or reserve the human CGI artists for fixing unmitigatable problems that crop up if they're fixable without full re-rendering (whichever probably ends up less expensive).
Plus with what we've recently seen with world models that have been released in the last week or so, AI will soon get better at having a full and accurate representation of the world it creates and future generations of this technology beyond what Sora is doing simply won't make these mistakes.
People don't watch The Matrix expecting a documentary on how we all got plugged in. If someone generated the referenced ladybug movie for use in a science classroom, that's a problem.
And it's already harmful in some cases. E.g. people drag people out of a crashed car because they think it's going to explode, sometimes seriously injuring them.
Did you see the movie Battleship? Or a good percent of recent and not so recent action movies, at least Matrix could be argued that it was about a virtual reality.
Between omnipresent cgi in movies and tv, animation, and video game physics (all of which are human-coded approximations of real physics, often intentionally distorted for various reasons), that ship has long since sailed.
> The sinking of the elephant into snow - how deep is too deep? Should there be snow on the elephant or would it have melted from body heat? Should some of the snow fall off during movement or is it maybe packed down too tightly already?
Should there be an elephant in the snow? The layers of possible confusion, and subtle incorrect understandings go much deeper.
With the same reasoning, do reindeer actually fly and pull a sleigh carrying a 200-pound man along with tons of gifts? I believe you're underestimating human intelligence and our ability to apply logic and reasoning.
Or just inaccurate impressions of the physical world.
My young kids and I happened to see a video of some very cute baby seals jumping onto a boat. It was not immediately clear it was AI-generated, but after a few runs I noticed it was a bit too good to be true. The kids would never have known otherwise.
YouTube Shorts are full of AI animal videos with distorted proportions, living in the wrong habitat, and so on. They popped up on my son’s account and I hate them for the reasons you outline. They aren’t cartoonish enough explain away, nor realistic enough to be educational.
> A little worried how young children watching these videos may develop inaccurate impressions of physics in nature.
I'm less concerned with physics for children--assuming they get enough time outdoors--and more about adulthood biases and media-literacy.
In particular, a turbocharged version of a problem we already have: People grow up watching movies and become subconsciously taught that flaws of the creation pipeline (e.g. lens flare, depth of field) are signs of "realism" in a general sense.
That manifests in things such as video-games where your human character somehow sees the world with crappy video-cameras for eyes. (Excepting a cyberpunk context, where that would actually make sense.)
Yes, entertainment spreads lots of myths. But bad physics from AI movies is only a tiny part of the problem. This is similar to worries about the misconceptions people might get from playing too many video games, reading too many novels, watching too much TV, or participating too much in social media.
It helps somewhat that people are fairly aware that entertainment is fake and usually don’t take it too seriously.
Me too. While I'm generally optimistic about generative art, at this point the models still have this dreamlike quality; things look OK at first glance, but you often get the feeling something is off. Because it is. Texture, geometry, lights, shadows, effects of gravity, etc. are more or less inconsistent.
I do worry that, as we get exposed more and more to such art, we'll become less sensitive to this feeling, which effectively means we'll become less calibrated to actual reality. I worry this will screw with people's "system 1" intuitions long-term (but then I can't say exactly how; I guess we'll find out soon enough).
What is physics besides next token/frame prediction? I'm not sure these videos deserve the label "inaccurate" as who's to judge what way of generating next tokens/frames is better? Even if you you judge the "physical" world to be "better", I think it's much more harmful to teach young children to be skeptical of AI as their futures will depend on integrating them in their lives. Also, with enough data, such models will not only match, but probably exceed "real-physics" models in quality, fidelity, and speed.
i wouldnt expect young children to learn how to walk by watching people walk on a screen, regardless of if its a real person walking, or an ai animation.
the real world gives way more stimulus
watching the animations might help them play video games, but i again imagine that the feedback is what will do the real job.
even for the real ladybug video, who says the behaviour on screen is similar to what a typical ladybug does? if its on video, the ladybug was probably doing something weird amd unexpected
Sure this is problematic for society although I'm not concerned about what you are mentioning. I remember as a kid noticing how in looney tunes wile e coyote could run off the cliff a few steps and thinking maybe there's a way to do that. Or kids arguing about whether it was possible to perform a sonic boom like in street fighter. Or jumping off the playground with an umbrella etc
Don't be, physics laws miss interpretation are very quick to correct with a reality check. I'm more worried for kids that have to learn how the world works trough a screen. Just let them play outside and interact with other kids and nature. Let them fall and cry, and scratch and itch, it will make them stronger and healthier adults.
I think it's unnecessary to worry about obviously bad stuff in nascent and rapidly developing technology. The people who spent most time with it (the developers) are aware of the obviously bad stuff and will work to improve it.
Young generation that will grow up with this tools will have completely different approach to anything virtual. Remember how prople though that camera stole part of their soul when they see themselves copied on picture?
Video games and movies have existed for a long time. I think children today will end up being more discerning than us because they will grow up sifting through AI generated content.
That could be nice. If you think that rabbits crawl like on the sora.com homepage, but then you see one hopping in real life, you might have more of a sense of wonder about the world.
AI physics isn't worth worrying about compared to other inaccurate things kids see in movies. It doesn't seem to hurt them.
If you really want something to worry about, consider that movies regularly show pint-sized women successfully drop kicking men significantly bigger than themselves in ways that look highly plausible but aren't. It's not AI but it violates basic laws of biology and physics anyway. Teaching girls they can physically fight off several men at once when they aren't strong enough to do that seems like it could have pretty dangerous consequences, but in practice it doesn't seem to cause problems. People realize pretty quick that movie physics isn't real.
I know this sounds judgmental, but this reminds me of the idiom “touch grass”. Children should be outdoors observing real life and not be consuming AI slop. You are not overthinking this, this will most likely be bad for children and everyone in the long run.
For instance, that ladybug looks pretty natural, but there's a little glitch in there that an unwitting observer, who's never seen a ladybug move before, may mistake as being normal. And maybe it is! And maybe it isn't?
The sailing ship - are those water movements correct?
The sinking of the elephant into snow - how deep is too deep? Should there be snow on the elephant or would it have melted from body heat? Should some of the snow fall off during movement or is it maybe packed down too tightly already?
There's no way to know because they aren't actual recordings, and if you don't know that, and this tech improves leaps and bounds (as we know it will), it will eventually become published and will be taken at face value by many.
Hopefully I'm just overthinking it.