> when doing paintovers on copyrighted images (VERY common)
What are you talking about? I've been doing drawing and digital painting as a hobby for a long time and tracing is absolutely not "VERY common". I don't know anybody who has ever done this.
> fan art where they paint trademarked characters (also VERY common)
This is true in the sense that many artists do it (besides confusing trademark law and copyright law: the character designs are copyright-protected, trademarks protect brand names and logos). However, it is not fair use (as far as I'm aware at least, I'm not a lawyer). A rightholder can request for fanart to be removed and the artist would have to remove it. Rightsholders almost never do, because fanart doesn't hurt them.
There's also more examples of it reproducing copyright-protected images, I pulled the "bloodborne box art" prompt from this article: https://arxiv.org/pdf/2212.03860.pdf
But I agree with you that reproducing images is very much not the intention of Stable Diffusion, and it's already very rare. The way I see it, the cases of Stable Diffusion reproducing images too closely is just a gotcha for establishing a court case.
Paintover does not have to mean actual 'tracing', a LOT of artists use photos as direct references and paint over them in a separate layer, keeping the composition, poses, colors very close to the original while still changing details and style enought to make it transformative enough to be considered a 'new work'.
Here are two examples of artist Sam Yang using two still frames from the tv show Squid Game and painting over those, the results which he then sells as prints:
That said, you could even get away with less transformation and still have it be considered original work, take Andy Warhol's 'Orange Marilyn' and 'Portrait of Mao', those are inked and flat color changes over photographs.
First of all, those are only two works in a very large body of works of an artist that seems to work almost entirely from imagination, which already counters the claim that this is a very common way of working, since even this artist would almost never work like that. Secondly, putting strangely much effort into a comment on Hacker News, I actually looked up the source frame of one of these: https://youtu.be/K6hOvyz65jM?t=236
It's definitely based on the frame but it's not a paint-over as you claim. I know this because there are too many mistakes with regards to proportion:
- Extending the slant roof in the background, it intersects with the left figure at around the height of the nose, but in the painting it intersects with the middle of her neck.
- Similarly the line of the fence on the left is at the height of her hairline, but in the painting it is at the height of the middle of the head, and also more slanted than in the frame.
- On the right side, the white part of the pillar is similarly too low compared to the figure.
- The pole in the background has a lot of things off with regards to size, thickness, or location too.
Essentially, everything is a bit off with regards to location, size and distance. It doesn't really make sense to paint over something and then still do everything differently from the base layer, so it was probably just drawn from reference the normal way -- probably having the picture on another screen and drawing it again from scratch, rather than directly painting over the frame.
I agree with regards to Warhol but that doesn't really establish it as very common amongst painters.
>that seems to work almost entirely from imagination
I very much doubt that.
>Secondly, putting strangely much effort into a comment on Hacker News
Note sure what you are implying here, could you elaborate ? The reason I know about these images is because they've been posted, alongside many other similar examples, in discussions regarding AI art.
>I know this because there are too many mistakes with regards to proportion:
Have you ever used programs like Photoshop, Krita et al ? You can start painting directly over a photo, and then easily transform the proportions of all components in the image, and since you draw them in layers, they can be done without affecting eachother.
I have no doubt that he started painting these over the reference photos, and then used the 'warp tool' in his painting program of choice to alter the proportions, a very common technique.
And this is PERFECTLY FINE, the resulting artwork is transformative enough to be considered a new work of art, which is true for practically every piece of art I've seen generated by Stable Diffusion, the only one I've seen that I'm doubtful about is the 'bloodborne box art' one, which is THE example that is always brought up as it such an outlier.
You can see his actual workflow on his YouTube channel. He shows his painting process there but doesn't show his sketching process, but I hope that you believe that people are able to draw from imagination at least.
> Note sure what you are implying here, could you elaborate?
I just meant I was probably putting to much effort into an online discussion.
> I have no doubt that he started painting these over the reference photos, and then used the 'warp tool' in his painting program of choice to alter the proportions, a very common technique.
It's simply not a common technique at all. I'm not sure why you're making these statements because it feels like your knowledge of how illustrators work is extremely limited. I've heard of people photobashing -- which is when artists combine photo manipulation and digital painting to more easily produce realistic artworks. It's got mixed opinions about it and many consider it cheating but within the field of concept art it's common because it's quick and easy. However, there's huge amounts of people who can just draw and paint from sight or imagination. There's the hyperrealists who often act as a human photocopier, but artists who do stylized art of any kind are just people who can draw from imagination. I'm not sure why that's something you "very much doubt" to be quite honest. Just looking on YouTube for things like art timelapses, you can find huge amounts of people who draw entirely from imagination. Take Kim Jung Gi as a somewhat well known example. That guy was famous amongst illustrators for drawing complicated scenes directly in pen without any sketches. But there's really plenty of people that can do these things.
You seem to be under the impression that the average artist uses every shortcut available to get a good result, but that is simply not true. Most artists I know refuse to do anything like photobashing because they consider it cheating and because it isn't how they want to work, nevermind directly drawing on top of things. Drawing from sight isn't uncommon as a way to study art, so in case you're wondering why Sam Yang would be able to reproduce the frame so closely, it's because that's how artists study painting.
> Have you ever used programs like Photoshop, Krita et al
Yes, very often. The thing is: Just because it's possible does not mean it actually happens.
What are you talking about? I've been doing drawing and digital painting as a hobby for a long time and tracing is absolutely not "VERY common". I don't know anybody who has ever done this.
> fan art where they paint trademarked characters (also VERY common)
This is true in the sense that many artists do it (besides confusing trademark law and copyright law: the character designs are copyright-protected, trademarks protect brand names and logos). However, it is not fair use (as far as I'm aware at least, I'm not a lawyer). A rightholder can request for fanart to be removed and the artist would have to remove it. Rightsholders almost never do, because fanart doesn't hurt them.
There's also more examples of it reproducing copyright-protected images, I pulled the "bloodborne box art" prompt from this article: https://arxiv.org/pdf/2212.03860.pdf But I agree with you that reproducing images is very much not the intention of Stable Diffusion, and it's already very rare. The way I see it, the cases of Stable Diffusion reproducing images too closely is just a gotcha for establishing a court case.