
Show HN: Paint yourself in the style of classic art, in your browser - reiinakano
https://github.com/reiinakano/fast-style-transfer-deeplearnjs/
======
logane
Cool work! We made this demo using the same style transfer network in
TensorFire: [https://tenso.rs/demos/fast-neural-
style/](https://tenso.rs/demos/fast-neural-style/)! Your size slider was a
nice touch :)

~~~
reiinakano
You're lengstrom from Github! I must let you know your models are exactly what
I used! I don't have a beefy GPU so I was forced to use what you already had,
haha.

For everyone reading, these guys coded up the neural network I used, all I did
was port it to a different library. :)

Eagerly waiting for the Tensorfire API!

------
michrassena
When I see the style transfers, I'm reminded of the early days of Photoshop,
and whatever else was a commonplace photo editing application of the mid-90s
(like Fractal Painter). It was fun to play with the idea of oil painting, or
watercolor, or to simulate paper textures. How is style transfer an
improvement? Undoubtedly, it's more sophisticated and requires more
computational resources.

Can it be used for artistic purposes, or is it only skin deep like those old
filters? It looks to me as if this is pure imitation, your images of a night
with friends at the street cafe, or stacking bails of hay can look like a Van
Gogh when you upload it to Instagram.

Am I missing something?

~~~
robbrown451
Of course, computers can't be creative, so it is just imitation. What it
creates is not novel. It's just doing what the programmer told it to do. Blah
blah blah.

In my opinion, it is one step -- and an impressive step -- in the direction of
computers being able to actually be creative. The areas where humans can do
something that computers can't is shrinking. You can dismiss it if you want,
but at what point will you acknowledge that what the computer is doing isn't
simply imitation? I guess you can keep moving the goalposts, but to me, this
is pretty impressive.

~~~
SimbaOnSteroids
This assumes creativity is anything more than copying from multiple sources.

~~~
musage
If it's not more, how come there is anything to copy from?

~~~
reiinakano
Copying from nature with tiny little variations I guess?

~~~
musage
Little compared to what? They're infinitely big compared to no variations.
Ignoring that part because it's unclear is like throwing out the baby and
keeping the bathwater to me.

------
hnlmorg
I don't know if it's my machine or whether my expectations are too high, but
every generated image looks like the same blurry version of the original but
with a different colour pallet. I certainly don't get anything as beautiful as
the output examples.

This happens regardless of the source content I use (any of the examples or
even my own uploaded files) nor any of the reference styles.

User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/61.0.3163.100 Safari/537.36

(Chromium running on ArchLinux AMD64)

~~~
hnlmorg
Looks like I'm getting the same results with Firefox on the same host PC as
well.

Screenshot showing the same input values used as the one in your example:

[https://armyofcrabs.com/rand/Screenshot_20171002_170417.png](https://armyofcrabs.com/rand/Screenshot_20171002_170417.png)

~~~
reiinakano
These definitely look wrong. Unfortunately, it's super hard to debug since I
can't reproduce. :(

My only advice I guess is to try it on a different machine when you have one
available. Sorry, if this is a problem with the library I used, I can't really
do anything to fix it. :/

~~~
hnlmorg
Not sure if this helps but I've tried the project you based this off[1] and
that produces images correctly (or at least that look more like my
expectations).

(I say "based", I'm not sure just how much of their code - if any - you used.
But you did compliment them earlier in this discussion)

[1] [https://tenso.rs/demos/fast-neural-style/](https://tenso.rs/demos/fast-
neural-style/)

~~~
reiinakano
Didn't really use any of their code. I reimplemented their model in a new
library called deeplearn.JS.

If the issue is indeed with the library, there's little I can do.

I will still try to find what the issue is but can't make promises. Sorry for
the letdown!

------
reiinakano
Demo website: [https://reiinakano.github.io/fast-style-transfer-
deeplearnjs](https://reiinakano.github.io/fast-style-transfer-deeplearnjs)

Deeplearn.JS library doesn't work on mobile yet, sorry to folks on the go!

~~~
Beltiras
Is the aspect ratio important? I'm choosing a roughly square image and it only
calculates half an image.

~~~
reiinakano
Aspect ratio shouldn't matter, the output should still be the same size as the
Content image. Mind sharing the pic?

~~~
Beltiras
[https://imgur.com/a/acanW](https://imgur.com/a/acanW)

~~~
reiinakano
I was able to stylize it fine. Maybe refresh and try again with a smaller
size?

------
gwbas1c
Might want to try some basic HTTP caching settings for the model. This way the
browser doesn't need to re-download everything every time.

This can probably be configured in the server instead of in the application.

~~~
wrigby
I think most CDNs will add reasonable cache-control headers as well - IIRC
Cloudflare does this with sane defaults out of the box.

~~~
reiinakano
Stupid question: Is CloudFlare free for CDN purposes? Right now I'm serving
everything from Github Pages. I don't expect there to be much traffic after
this initial HN spike dies down, so if it isn't free, it won't be worth it for
me to pay for it.

~~~
wrigby
Yep[1]! I used it for my girlfriend's small business site (Wordpress) and was
very happy with it.

I think CSS, JS, and images are cached - HTML is not.

1: [https://www.cloudflare.com/plans/](https://www.cloudflare.com/plans/)

------
earlybike
tl;dr: Impressive Prisma clone with six filters but running all calculations
locally in the browser. Compared to Prisma it feels slightly slower or as
slow, one filter almost froze my MBP 4@2.7.

~~~
reiinakano
It is currently VERY slow since there are a lot of non-optimized (non-GPU)
calls being made. Once those are implemented in deeplearn.JS, things should
run much faster. In fact, one of the creators of deeplearn.JS says it might be
feasible to do this realtime in a webcam (although not at 60FPS).

This was a super quick PoC demo I hacked together in a week on a very new
library. :)

------
davedx
Weirdly, something about the generated images (colour range?) breaks Twitter's
profile photo update. I actually took screenshots of the original taken with
my webcam and the deep learning style transferred version, the former worked
for my Twitter profile photo, and the latter resulted in an error. I tried
different resolutions too, and it always broke.

Shame, I really like the effect!

~~~
reiinakano
You took screenshots and it still broke? That's.. very odd, haha. I will look
into it and see if it outputs out-of-range colors.

EDIT: My bad, there are indeed out of range colors! I will update here when
fixed.

------
chias
The demo site doesn't appear to work on my end -- it gets a 404 attempting to
load "bundle.js"

~~~
supermdguy
Yeah, same for me. I added an issue: [https://github.com/reiinakano/fast-
style-transfer-deeplearnj...](https://github.com/reiinakano/fast-style-
transfer-deeplearnjs/issues/5).

~~~
chias
It appears to be working again (for me)

~~~
supermdguy
Yep, it's working now.

------
TekMol
Cool. I'm getting some great results. Would be awesome if you could train your
own model in the browser too.

By the way: How does a NN learn from a single image? I understand how a NN can
learn to classify images. But how can it learn a style from a single image?

~~~
reiinakano
Fast Neural Style Transfer works like this: Each style image is actually
trained on a neural network outside of the browser (using Python on
Tensorflow) for around 6+ hours with a GPU.

The output is a neural network that can take any content image and outputs it
in the style it was trained on.

The browser does this: 1\. Downloads the model (6.6MB~) for the particular
style. 2\. Does one pass through the content image and outputs the styled
image.

No training is done in the browser whatsoever.

~~~
TekMol
Yes, I am aware of that.

But I would like to know how the training works. How does a NN learn the style
of an image?

~~~
SamBam
I think you can consider one single image as a large corpus of data, in the
same was that you could train a simple Markov model from a large paragraph of
text. It doesn't matter to the model whether the original text was 500 words
in one piece of text, or five 100-word texts.

It's mostly looking at local regions and seeing what guesses it can make about
the next region given one region. (If I understand correctly).

------
wowtip
Is it using the same tech as android/ios app Deep Art Effects?

[https://www.deeparteffects.com/](https://www.deeparteffects.com/)

Results look very similar

~~~
reiinakano
Yes, most likely.

At most they probably tweaked things a little bit for some minor improvements,
but the underlying idea and algorithm is very likely the same.

Can't confirm this, though. :)

------
rsoto
Hey, it looks very cool. However, it's hanging my browser. Haven't looked at
the source, but maybe this should be implemented with web workers to increase
perceived performance.

Anyway, great work. Congrats!

~~~
reiinakano
Things are currently quite inefficient as there are some non-optimized (non-
WebGL) calls being made. DeeplearnJS is a new library but once optimized
functions are available, I'll update the site. :)

------
sabeltann
Uncaught ReferenceError: dialogPolyfill is not defined at
HTMLDocument.<anonymous> ((index):22672) at (index):24

~~~
reiinakano
This doesn't happen when I visit the site on Chrome on Linux. Do you mind
sharing your system + browser?

~~~
davecardwell
Same with Chrome/63.0.3223.8 on Intel Mac OS X 10_11_6

It’s the call to dialogPolyfill.registerDialog(dialog);

~~~
reiinakano
Hi, thanks for the report, but I'm not really a JS developer and have no idea
how to work around this. Would you happen to know if there are any fixes I
should look into?

------
Belphemur
Am I the only one that get "the stuff of nightmares" kind of results ?

Ubuntu 17.04 FF 57 & Chrome 59 with face and Udnie:

[https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano....](https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano.github.io)

~~~
reiinakano
To everyone facing this issue, please leave a comment here with your browser
and system specs:

[https://github.com/reiinakano/fast-style-transfer-
deeplearnj...](https://github.com/reiinakano/fast-style-transfer-
deeplearnjs/issues/7)

------
lerie82
I don't know if it's loading or what but it's just blank on my phone's chrome
browser.

~~~
reiinakano
Doesn't support mobile :) Please use desktop (Chrome suggested)

------
mercurysmessage
The chat program on Line, Rina, does this as well if you send a picture of a
person to her.

------
everdev
What size images can it generate? I've used other libraries that can't go past
800x600.

~~~
reiinakano
Theoretically, for this algorithm, there is no size limit.

Practically, if you select something too high, your browser will die for lack
of memory or you will grow old waiting for it to finish. My crappy 4GB RAM
laptop can only handle around 300 x 500. Try setting the size to maximum, and
if your computer can handle it, I'd be glad to guide you in using bigger
images.

------
fiatjaf
How do I input an image? There's no option to do that on Firefox 58.

~~~
reiinakano
"Upload from file" option not appearing?

~~~
fiatjaf
No controls at all were appearing. Now, from a different computer --- but
still at Firefox 58 -- everything is working fine (maybe you've fixed it in
the meantime, maybe not).

------
reacweb
Error: Failed to link vertex and fragment shaders.

I am on windows 7 using firefox 55.0.3

~~~
reiinakano
Do you mind trying Chrome? Deeplearn.JS is a new library and there are bound
to be weird bugs like this I have no hope of debugging. I believe the most
supported version is desktop Chrome.

------
samstave
Mobile not supported - please post some screens of results..

~~~
reiinakano
You can see one result in the repository README! :)

------
code-is-code
Looks like it does not work on mobile chrome

~~~
reiinakano
DeeplearnJS does not work on mobile at all unfortunately. It's a new library,
they'll get there. :)

~~~
code-is-code
That makes sense. An error-message would have been nice.

------
make3
you should likely put a warning message for mobile users to let them know the
website doesn't work on mobile

------
laythea
I get "Error: Unable to create WebGLTexture." every time I try and run a
sample. I got a 780ti, so its no graphics hardware problem...?

~~~
TheRealPomax
you probably want to file that on the issue tracker, not here.

~~~
laythea
Why not here? - if someone is going to put something "in my face" that does
not work (for me), I do not feel it inappropriate to respond in the same
forum.

------
sergiotapia
Demo doesn't load anythin

~~~
reiinakano
Fixed now. Sorry!

