> Even video taken at 300K fps by the Slow Mo Guys was slowed down even further and looks even smoother than the original.
This suggests that the video accompanying that quote shows 300k fps footage, but that can't be right.
If you film at 300k fps and play back at a (generous) 60 fps, that is a slow-down factor of 5000. The footage of the tennis racket and the bursting balloon is slowed down, but not 5000x. Perhaps 50x, perhaps even 500x (doubtful) but definitely not 5000x.
Say the pool jump or the racket swing each takes only 0.1s real time. At 5000x that would mean 500s, or about 8 minutes. Clearly, the slow footage isn't _that_ slow.
This being said, the Nvidia results do look very impressive, I'm not trying to take away from that.
The tennis racket jello shot is 2500 FPS and then played at 30 FPS and interpolated to 120 FPS. The 300k number is just pure screw up in the article. The actual paper states it is 300k frames total generated not frames per second.
There is still a lot of room for improvement in video. I can see a day when video is seen for what it is... A collection of sensor readings. Seen this way, differences in sensitivity between imaging units can be used to gain extra information about the true color of objects. Multiple readings from the same object can be used to re-create the scene in great detail.
Imagine a future where you could zoom in and rotate around an out of focus object seen at the beginning of a video because there were a few closeup views of it 10 minutes later...
TVs already have this, it's referred to as motion smoothing and people hate it. TVs with high refresh rates create interpolated frames to smooth the video when the source material is at a lower frame rate. It works for some situations like sports, but in other situations it's very unpopular -like movies, but it's been a standard feature in many tvs for a long time.
This is really cool, but are there any uses for this outside of slow-mo shots in showbiz? Certainly interpolated frames lack the fidelity to be of scientific use.
Not quite fair comparisons with the slow mo guys vids.
Take a 1/4 cut of the original frames, generate your slow mo from that, and let us compare with the same rates.
The virtual slow mo of slow mo was neat, but it felt more like self flattery than scientific.
Any methodology for getting more information from less information is going to involve guess work at some point, surely? This guess work just happens to be helped along by having the results of (I assume) millions or more separate videos where they already had the higher speed footage.
This is also not extrapolation, rather quite the opposite, it's interpolation.
But that's what our brains do too. We might think we're seeing the world as it is, but much of our smooth and consistent view of the world has a lot to do with our brains patching things up to make it look that way.
The thing is, that it does raise some questions - I can see things like fake super-resolution and fake slow motion and other alternative realities generated by machine learning easily fooling humans. Once things become sufficiently advanced, who's to guarantee will we always know what the true source is? Could future training be done on data that itself was generated via machine learning. That brings up a load of other questions. Interesting times ahead...
Don't our brains also extrapolate? I am sure our eyes feed the brain at a limited frequency, and the brain combines and extrapolates the images to create the illusion of smooth streams.
How high is that frequency is yet unknown:
"Tests with Air force pilots have shown, that they could identify the plane on a flashed picture that was flashed only for 1/220th of a second.That is identifying. So it's pretty safe to say, that recognizing, that SOME light was there is possible with 1/300th of a second." http://www.100fps.com/how_many_frames_can_humans_see.htm
This suggests that the video accompanying that quote shows 300k fps footage, but that can't be right.
If you film at 300k fps and play back at a (generous) 60 fps, that is a slow-down factor of 5000. The footage of the tennis racket and the bursting balloon is slowed down, but not 5000x. Perhaps 50x, perhaps even 500x (doubtful) but definitely not 5000x.
Say the pool jump or the racket swing each takes only 0.1s real time. At 5000x that would mean 500s, or about 8 minutes. Clearly, the slow footage isn't _that_ slow.
This being said, the Nvidia results do look very impressive, I'm not trying to take away from that.