This work is a deep learning extension of our previous average image project: https://m.youtube.com/watch?v=1QgL_aPPCpM. See the New Yorker article for details:
I guess the deep learning might be a better way to blend millions of images for creating new visual content.
A common technique concept artists use for matte painting is to take existing images and blend them in to their creation, so this would almost be an evolution of that methodology.
For example, it's not hard to imagine future organic sculpting packages (e.g. ZBrush) having this type of tech integrated. Perhaps in-game character sculpting systems as well.
Appears to be fundamentally 2D, but the interpolation between orientations gives it a sort of meta 3D aspect.
However - What knowledge or tools would help me in best affecting the work that the neural net then produces? As in, effect the "style" that the network applies?
Work like Deep style transfer, or Prisma can try to transfer the style of one painting to an existing user photo. But you cannot use it as painting tool for creating new stuff.
There's got to be a way, although it might be incestuous. Use Deep style transfer and/or Prisma to massively increase the body of work, by transforming other work into that style, and then using that as training data for this...? Then I guess the artistry is in filtering those images, but that's a lot of images...
OOOOOHHHH WAIT. Remember how there's that dude who gets shown surveillance images from the middle east, and a computer watches his brain for the faster-than-thought responses to there being things in those images? That same trick MIGHT work for artistic sensibilities, but the response might not be identifiable enough.
As you said, one can also apply other filters like Prisma.
Markov models are best suited to situations where an observation k-steps in the past gives exponentially less information about the present (decaying according to something like λ^k for 0 <= λ < 1).
Intuitively, the amount of context imparted by a word or phrase decays somewhat more slowly.
That is, if I know the previous five words, I can make a good prediction about the next one, and likely the next one, and slightly less likely the one after that, whereas in a Markovian setting my confidence in my predictions should decay much more quickly.
So in answer to the grandparent, such a thing should be reasonably straightforward to build if it doesn't exist already, and it may offer improvements over a similar model based on Markov chains.
2. Why is this? Lin & Tegmark offer details in the paper, but it comes from the fact that the singular values of the transition matrix are all less than or equal to one (an aperiodic & ergodic transition matrix has only one singular value equal to one), and so the other singular vectors fall away exponentially quickly, with the exponent's base being their corresponding singular value.
and the short film they made, using that script: https://www.youtube.com/watch?v=LY7x2Ihqjmc
(disclosure: i work for github on events/AV)
How often have i looked for shoes or clothing items that have "a stripe of white around the soles, black for the body with some dark red decals" with this i could basically enter this as some kind of visual search query.
This is Python + OpenCV, but I'm under the impression Adobe is a pretty serious C++ shop and has their own graphics libraries (I'm mainly aware of boost::GIL and their STL)
This work was supported, in part, by funding from Adobe, eBay and Intel, as well as a hardware grant from NVIDIA. J.-Y. Zhu is supported by Facebook Graduate Fellowship.
- Lack of close up detail as expected from generative networks. Looks like someone has used the Photoshop clone tool.
- Low resolution results as seems to be common with GANs.
On the other hand, in the recent years, we see dramatic improvement of image quality from these generative models. Overall I think this is a promising and exciting direction.