

Deep Dreams with Caffe - miket
https://github.com/google/deepdream/blob/master/dream.ipynb

======
karpathy
One thing people might not realize (I'm not sure how obvious it is) is that
these renders depend strongly on the statistics of the training data used for
the ConvNet. In particular you're seeing a lot of dog faces because there is a
large number of dog classes in the ImageNet dataset (several hundred classes
out of 1000 are dogs), so the ConvNet allocates a lot of its capacity to
worrying about their fine-grained features.

In particular, if you train ConvNets on other data you will get very different
hallucinations. It might be interesting to train (or even fine-tune) the
networks on different data and see how the results vary. For example,
different medical datasets, or datasets made entirely of faces (e.g. Faces in
the Wild data), galaxies, etc.

It's also possible to take Image Captioning models and use the same idea to
hallucinate images that are very likely for some specific sentence. There are
a lot of fun ideas to play with.

~~~
amelius
So how much computational effort would it take to train with a different set
of images, to reach the same level of training as this existing data?

Would it be possible on a simple commercial computer?

~~~
MoOmer
OpenCL support is coming, but it's not as performing as CUDA support yet for
Caffe.

Grab a couple of video cards and have fun!

~~~
magixx
Might finally be putting some bitcoin GPUs to use.

------
zan2434
[http://deepdreams.zainshah.net](http://deepdreams.zainshah.net) spun up a
simple web server so you can try your own! Please be gentle :)

~~~
bemmu
I tried to run the same image in it a few times.
[http://imgur.com/5Hsuoiy](http://imgur.com/5Hsuoiy)

Since you have this all set up, can you make some feedback loop animations for
example with zooming? Or apply this to each frame of a movie? For example
something famous like Charlie Bit My Finger. Hopefully using the deeper more
horrifying setting.

~~~
quantombone
Here is a zooming example. I definitely noticed that it makes people's eyes
look evil. Maybe it's hallucinating animal eyes on top of human eyes...

[http://imgur.com/kThM9vf](http://imgur.com/kThM9vf)

~~~
tim333
That's a bit freaky

------
Liquix
The visuals generated by the neural network remind me of visuals experienced
under the influence of psilocybin or LSD. I wonder if I am making an unjust
leap or if there is a similar organic process (searching for familiar
patterns) taking place in the mind? Fascinating, thanks for sharing.

~~~
sabalaba
No hypothesis is unjust! It could also be related some of the experiences
people have in sensory deprivation tanks. Your brain attempting to see
structure in noise and hallucinates. One hypothesis would be that on LSD, and
other psychoactive substances, this feedback loop is somehow enhanced. There
might be a few doctorates to be earned in testing these hypotheses.

------
hellbanner
"Be careful running the code above, it can bring you into very strange
realms!"

Reminds me of Charlie Stross's new novel,

"A brief recap: magic is the name given to the practice of manipulating the
ultrastructure of reality by carrying out mathematical operations. We live in
a multiverse, and certain operators trigger echoes in the Platonic realm of
mathematical truth, echoes which can be amplified and fed back into our (and
other) realities. Computers, being machines for executing mathematical
operations at very high speed, are useful to us as occult engines. Likewise,
some of us have the ability to carry out magical operations in our own heads,
albeit at terrible cost."

[http://www.tor.com/2015/06/30/excerpt-the-annihilation-
score...](http://www.tor.com/2015/06/30/excerpt-the-annihilation-score-
charles-stross/)

~~~
thirdtruck
You might also like Shadowfist
([http://shadowfist.com](http://shadowfist.com)), a card game that used to
have the Purists, a playable faction powered by esoteric, math-centric magic.

------
malkia
Here are some images I've done -
[https://www.facebook.com/media/set/?set=a.720197931442169.10...](https://www.facebook.com/media/set/?set=a.720197931442169.1073741832.100003559066571&type=1&l=5513aa040e)

~~~
MrBuddyCasino
Those Teletubbies are perfect. Best I've seen yet.

~~~
fla
At the very bottom of the uncanny valley

------
cing
Great, I got the dependencies installed on OSX and I'm already monsterifying a
head shot for LinkedIn. Now, to find a way to get this working in real time
with a webcam...

~~~
sciencerobot
I'm stuck at compiling Caffe :\

~~~
mrbrowning
I had a lot of difficulty getting it compiled too, but this helped a lot:

[http://installing-caffe-the-right-way.wikidot.com](http://installing-caffe-
the-right-way.wikidot.com)

That guide is mostly correct, assuming your reply here means that you're also
using OS X. It should get you all the way to a working Caffe install. The one
thing it doesn't get right is that your PYTHON_INCLUDE and PYTHON_LIB
variables should _both_ point to the relevant folders from your Homebrew
Python install (I had no luck attempting to compile pycaffe against either
Anaconda or system Python, both would just segfault when I imported the
module). In my case, that was (assuming you've already installed numpy with
Homebrew pip):

PYTHON_INCLUDE :=
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/include/python2.7
\
/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/numpy/core/include

PYTHON_LIB is exactly as it is in the example Makefile.config on that page,
except adjusted for version number if you've installed Python via Homebrew
since 2.7.10 was released.

------
benanne
We sort of reverse-engineered this last week and set up a stream with live
interactive "hallucinations":
[http://www.twitch.tv/317070](http://www.twitch.tv/317070)

You can suggest what objects the network should dream about (combinations of
two are also possible).

Our code will be published on GitHub later today!

~~~
majora2007
I'm very excited to see the code. I read your blog post earlier this week and
am very intrigued.

------
pierrec
Amazing that it easily runs on consumer hardware, this dispels suspicions that
a Google cluster was necessary for these results.

I'm wondering if it's possible to use this with a model that was trained on a
database without labels, just pictures. Is such a thing even possible? For
this particular application, labeling and categories are ultimately
superfluous, but are they required in order to get there?

------
nicklovescode
Can someone please create a SASS interface to play with it? Would love to send
this to family/friends who can't easily spin up the code.

~~~
zan2434
[http://deepdreams.zainshah.net/?q=1](http://deepdreams.zainshah.net/?q=1)

------
spot
A simpler version of this idea (making an image A out of matching pieces of a
set of images B) was implemented in the early 90s and released as open source:
[http://draves.org/fuse/](http://draves.org/fuse/)

------
rsp1984
I always wonder why sometimes the system finds faces and other elements in
essentially untextured / homogeneous parts of images. Wouldn't there be some
sort of "data term" in the energy functional that would suppress these results
and/or move them to other parts of the image?

Perhaps this is working entirely differently and I'm thinking too much in the
classical computer vision realm. Would love some explanation though.

~~~
wodenokoto
I imagine the chance of an input that would result in zero confidence in all
output nodes is damn near zero.

There will basically always be an output nose with the highest confidence, no
matter how low.

------
rayalez
This is really cool. I wonder what it would look like applied to video.

Also I didn't know that github displays .ipynb, that's pretty awesome.

~~~
jrabek
This should be combined with the oculus with a camera on the front.

~~~
lawlessone
Simulated lsd?

------
johnwatson11218
Does anyone know if this technique can be used to slurp up a database and
produce "typical" records for populating a test database? This is a problem
that I struggled with a few years ago and still haven't found a good automated
solution.

~~~
sova
Could you refine your question? This is a post about image processing via
neural network. Do you mean take an existing database, learn via neural
network, and populate a fresh one with "learned" attributes?

~~~
johnwatson11218
yes that sounds correct. I'm thinking of something where I take an existing db
and train a NN on it, then populate a test db with things like "typical
account", "typical delinquent account", etc. This db could then be used for
automated testing. I have seen approaches like Factory Girl in rails but the
new rows are just incrementing fields somehow. Another approach would be to
model a column statistically then generate random values that conform to that
model. I'm thinking of something that is so general it can find and model
relationships in a db. For example it should be able to see that most people
have 2 or 3 credit cards on file and generate test data that is like that.
This may not be a problem for NN but the idea of running the networks
backwards and "imagining" things it has learned seems like a good fit. I have
played around with markov chains trained on a first + last name that could
generate made up names but that is as far as I got with it.

------
taliesinb
The dogs, eyes, and Dali-like bird-dogs are really cool. I've seen some
insects, too, but not very often.

Are there any other flavors of hallucination? Why all the dogs? I suppose
ImageNet has a lot of dog varieties in its category list.

------
malkia
A Trip To The Moon - [http://imgur.com/a/EkAkv](http://imgur.com/a/EkAkv)

------
sova
So awesomely trippy, love it.

------
llSourcell
ugh so annoying to compile can someone make this easier

~~~
grrowl
[http://ryankennedy.io/running-the-deep-dream/](http://ryankennedy.io/running-
the-deep-dream/) installs a Docker container, couldn't be easier.

------
Nordavind
But... How do you do this?

------
armab
Such things is the reason why I like scientific-friendly Python community.

