
Style2Paints: AI colorization of images - setra
https://github.com/lllyasviel/style2paints
======
bencollier49
I was thinking of trying it on something less Anime, and then noticed that
another poster didn't like the original poster's subject-matter, so I ran it
on this:

[https://imgur.com/a/z00jG](https://imgur.com/a/z00jG)

Not bad? I guess?

~~~
Mithaldu
Hm, i wouldn't say that's usefully different from anime? The failure mode for
e.g. pencil sketches is honestly kind of impressive.
[https://i.imgur.com/vdceynO.png](https://i.imgur.com/vdceynO.png)
[https://i.imgur.com/2lvrpwE.png](https://i.imgur.com/2lvrpwE.png)

On the other hand:
[https://i.imgur.com/QkFDOVs.png](https://i.imgur.com/QkFDOVs.png)

Edit: Also, i missed this earlier, but this video shows nicely how the AI can
be guided with color hints:
[https://www.bilibili.com/video/av14443094/](https://www.bilibili.com/video/av14443094/)

------
myrryr
This is amazing. Do you think it would be possible to keep the coloring
constant across an animation?

If so, you will certainly have a market for this. The coloring work in
animation take a lot of time and is very expensive.

~~~
julius
Interesting. It would be amazing if you could find the time, to elaborate,
share some insights into this industry.

Specifically: What are 2-3 major companies that would profit? How does the
current process work; do they make an entire uncolored scene and then color
it? What do I have to google, to find example videos of uncolored animation?
Can you ballpark how much it costs to color a 30min TV-Episode?

~~~
anon203597
> What do I have to google, to find example videos of uncolored animation?

key animation, genga (原画)

uncolored:
[https://sakugabooru.com/post/show/23064](https://sakugabooru.com/post/show/23064)

finished:
[https://sakugabooru.com/post/show/23126](https://sakugabooru.com/post/show/23126)

> How does the current process work;

This depends a lot on the production process used. These days it's anything
between hand drawn frames with digital coloring to completely digital
processes using 2D vector graphics or 3D models.

------
CardenB
Title is a little misleading. This is AI colorization of drawings, not all
images.

~~~
IanCal
If you want to be pedantic about it, it does work do colourisation of all
images, it's just not as good as it is with sketches.

[https://imgur.com/a/jXIhx](https://imgur.com/a/jXIhx)

------
IanCal
Surprisingly good results!

------
diyseguy
This would be so great as some kind of Gimp filter plugin

~~~
diyseguy
[https://imgur.com/JtBfX26](https://imgur.com/JtBfX26)

~~~
diyseguy
[https://imgur.com/Ea6JOdv](https://imgur.com/Ea6JOdv)

[https://imgur.com/a/KJfMs](https://imgur.com/a/KJfMs)

------
artur_makly
is there any direct way to contact the developer(s) ? i cant read chinese

~~~
gwern
[https://www.reddit.com/r/MachineLearning/comments/72l4oi/pfi...](https://www.reddit.com/r/MachineLearning/comments/72l4oi/pfinally_managed_to_paint_on_anime_sketch_with/)

------
Jack000
any more info on the architecture? Looks like it might be a cgan.

------
PrunJuice
If it's a shame that the brightest minds of our generation are working on UIs
and advertising, what does that make this? (:

~~~
awesomepantsm
This is the result of the brightest minds working on advertising.

Who do you think made TensorFlow? Hint: it was Google.

~~~
mistercow
I think there's a legitimate point to be made that it's a shame that the best
way we've found to coordinate our resources to advance science and technology
is capitalism. It's not like capitalism doesn't work, but if you envision a
sort of "coordination oracle" which divides up resources ideally, and which
everyone magically trusts, it seems pretty obvious that we would be advancing
much more quickly.

But I also agree that there's not much point in dwelling on that point. So
far, finding ways to leverage capitalism into solving problems has tended to
be a lot more fruitful than looking for a replacement.

~~~
Jack000
Capitalism = greedy optimization with a Bayesian assumption. It tends to get
stuck in local maxima and does weird things when the inputs are biased, but
otherwise works ok.

There are almost certainly better ways to search the solution space, but most
serious attempts attempts thus far have been non-sensical or impractical.

~~~
Houshalter
No one objects that capitalism isn't a good optimization processes. Just that
it's optimizing the wrong utility function. At one point something like 50% of
STEM majors at MIT went to work on wall street. And the ones that don't go
there go to work making more efficient targeted advertisements or addictive
smartphone games or other unethical shit.

Our population has a limited number of "smart people". I don't think any alien
looking in from the outside, would think we are allocating them even remotely
efficiently. It would be pretty difficult to design a system that does worse
than ours on this aspect.

The solution isn't necessarily communism, but perhaps a hybrid. E.g. the
government funds things capitalism has no incentive to optimize for, like
scientific research.

~~~
KGIII
They may be smart, but they are still humans. You can't ethically assign them
to career paths based on some vision of social good. That goes against the
whole free agency thing and I'm pretty sure we've decided it's a horrible way
to treat people.

That's well over the line into dictator territory. The results never match the
predictions in the brochure. That's pretty much a sure fire way to get a
tribunal convened at the Hague or a not-friendly visit from a SEAL team.

~~~
Houshalter
Capitalism already assigns people career paths. How is that any more ethical?
Not many kids grow up saying "I want to work on Wall Street" or "I want to
make better targeted advertising." They go into those careers because they
offer piles of money, and the jobs they might want to do don't. We could fix
that.

~~~
KGIII
No, it doesn't assign them, they choose them. Many people don't decide on a
career path based on the greatest salary potential, but on things they enjoy,
problems that interest them, and places they want to be.

We are humans, not unthinking machines.

~~~
Houshalter
I think empirically people do choose jobs because of money. Otherwise why
would Wall Street firms pay so well to attract talent? Why would people go to
work there if they didn't care about money?

~~~
KGIII
That's a very small number of people, not all of whom are well paid. The work
is intellectually stimulating and has the ability to have far-reaching
impacts.

I think you're confusing empirically for being significant numerically. I will
offer a counter for your Wall Street example, look at how many go into
teaching.

~~~
Houshalter
>That's a very small number of people, not all of whom are well paid.

By 2006 about a quarter of MITs entire graduating class went to Wall Street.
Some firms were offering $400K salaries to junior level positions. They
weren't just taking huge numbers of people, but the ones they were taking were
the best of the best. Other companies at the time had a terrible time
recruiting anyone because they couldn't compete with that kind of money.

The problem with this is I don't think these firms are actually doing anything
productive for society. The amount of work, money, and resources that go into
getting a nanosecond edge for high frequency trading is just obscene. And all
of the stuff they develop must be kept top secret and under NDAs. So even if
they invent something cool by chance, they can't share it with the world or
let anyone else benefit from it.

~~~
KGIII
Yes, there are many more smart people than just those st MIT. I hold a Ph.D.
(Applied Mathematics) from MIT. I've never even been to Wall Street, even as a
tourist. I worked in traffic modeling, because the domain had interesting
problems to solve and the opportunity to leave a mark was there.

Wall St. doesn't even employ that many people, in total. I'd also be curious
as to how many remained working there.

Now, to remind us of the subject, we're discussing how it is immoral to
determine career paths for other people. To be clear, I have no issues with
providing incentives to get people to select a career. I'm good with that.
However, demanding or forcing is right out.

People incentivized are still exercising free will when they choose to do so.
Capitalism isn't determining their career paths. You could offer $75/hour to
people to eat horse poop and you're still not going to get a lot of takers.
People choose their careers for many reasons other than expected salary.

If they chose with salary as the criteria, we'd have no teachers. We'd have no
researchers. We'd have no field scientists. Nobody would work for the
government. We'd have few artists, few musicians, few authors.

People choose their career for many reasons, salary sometimes isn't even in
the top three reasons. Of course, some people do so for monetary reasons. I'm
not sure they are actually the most suited for the job.

The next time you go to get emergency medical treatment, do you want someone
who is there for the paycheck, or do you want someone who is there because
they want to save lives?

Now, your final paragraph... You don't think they are doing anything
productive for society. I'm not sure why that's a problem. We are not the
arbiters. There are very few jobs that someone won't point to and say the
people performing those jobs aren't productive. Usually, they are Ayn Rand
fans but I'll assume you're not one.

They create wealth. They create a system that enables transfer of wealth. They
enable companies to be partially owned by the public. They increase the
incomes of those who employ them. They enable people to retire comfortably.
They enable employees by getting them greater value for their share of the
company.

In short, they do lots of things that are considered productive. If I made you
$10,000 in profit per day, you'd probably think I was pretty productive. I
know the reverse is true.

Anyhow, your last sentence kind of draws a more complete picture for me.
First, you speak of insisting people do certain kinds of work which you decide
the priority of. That struck me as misguided, but I was willing to see it
through. Now, you're lamenting that you're not being given the output of
someone else's investment.

You can't ethically own humans. Insisting they do only certain tasks and then
wanting the output of their labor is, well, treating humans as if the are your
property. That is not okay, at least according to my morals.

If I'm misreading you, feel free to explain. However, it looks like you
initially wanted to make people, smart people, do certain kinds of work.
You've since added that you think paying them above average is like forcing
them, that they are equal. You're now asserting that their labor output
shouldn't belong to them.

Err... That sounds remarkably like slavery.

------
mrmondo
May I suggest that using creepy sketches of young girls may not be the best
example if you want people to take your software seriously?

~~~
imaginenore
What's creepy about them?

~~~
amelius
I suppose the images look creepy because of the combination of young age with
poses and looks that are not usually associated with girls of such young age.

~~~
anon203597
Yes, let's apply cultural standards universally, surely this will make the
world a better place. At least professional artists know how to avoid
offending anyone's tastes.

[https://commons.wikimedia.org/wiki/File:William-
Adolphe_Boug...](https://commons.wikimedia.org/wiki/File:William-
Adolphe_Bouguereau_\(1825-1905\)_-_Admiration_\(1897\).jpg)

~~~
amelius
Perhaps the painting you referenced was meant to evoke disgust, or feelings of
guilty pleasure. Imho, both are not really appropriate on a project page
especially if used excessively and without reason.

At least the child in your painting looks naturally shy, given the situation.
This in contrast to the cartoons, where the child seems to be taking the lead
in seduction. Why ascribe feelings to children which they don't have?

