This is undoubtedly cool, but I worry quite a lot about people who aren't educated about ML interpreting these "historical renditions" as a "ground truth forensic". Rendering fossilized animals and coloring black and white photographs is also cool, but when you start to render the bone structure and "caucasian-ness" of statues, it starts to veer into some uncomfortable racial and ethnographic territory.
What data and models were used to synthesize these predictions?
You mention black and white colorization and that was my first thought upon seeing this. The statues are already super “photo-realistic” so I’m guessing this was a pretty similar process.
On a completely different note, I am not sure what you’re getting at with the racial stuff. Like what are you suggesting? That perhaps some of the emperors weren’t totally Caucasian but rendering them as such from the best available evidence is somehow racist? In my humble opinion, trying to find racial undertones in every tangential thing is a distraction from the real conversations that we should be having. And there’s a real possibility of alienating people that are otherwise sympathetic to the cause.
> On a completely different note, I am not sure what you’re getting at with the racial stuff
What I am saying is that this sort of work can lead to people believing these images as true, i.e. "this is what Roman Emperors looked like". If you ask an ML system to solve a problem for which the inputs cannot possibly determine the answer (i.e. the color of a fossilized animal or the skin color from a person's statue), it will give you a highly biased answer.
I fear that some laypeople think of ML as "magic software that can do anything" without considering whether a problem is non-ambiguous given its inputs (e.g. something like object classification is less ambiguous than this problem).
I'm not saying "burn it to the ground" - I wanted to learn more about what data and model was used to generate these images so that I might understand better where potential sources of racial / ethnographic bias might come from.
> trying to find racial undertones in every tangential thing
If you read my comments history you'll see that I'm no critical race theorist, nor do I discuss everything in a racial context.
Out of curiosity, what "cause" are you talking about?
I agree. I think the problem very much starts with the sculptures themselves. It seems rather likely that the sculptors would have had an incentive to "improve" some of the not so attractive features of those very powerful men.
A realistic ML model could theoretically compensate for that by reintroducing a bit of variability based on the variability we see in today's population of that region.
On the attractiveness point, the author mentions that in the case there were multiple visual sources, the one which would likely be considered less attractive was chosen. (Under the assumption that it was probably more accurate since as you stated, there was likely incentive for flattery.)
It certainly hasn't remained exactly the same, but there could be quite a bit of similarity left:
It is generally agreed that the invasions that followed for centuries after the fall of the Roman Empire did not significantly alter the local gene pool, because of the relatively small number of Germanics, or other migrants, compared to the large population of what constituted Roman Italy.
In 2008, Dutch geneticists determined that Italy is one of the last two remaining genetic islands in Europe, the other being Finland. This is due in part to the presence of the Alpine mountain chain which, over the centuries, has prevented large migration flows aimed at colonizing the Italian lands[1]
The migrants comprised war bands or tribes of 10,000 to 20,000 people,[5] but in the course of 100 years they numbered not more than 750,000 in total,[citation needed] compared to an average 40 million population of the Roman Empire at that time.[2]
The same can be said of the original advanced technology used to render a facsimile of the Roman Emperors, and in fact .. many stone sculptors have lost their prestige for the sake of this very same argument.
Since culture is a lie which persists only in the telling, this IS what the Roman Emperors look like, now. You shouldn't resist that, because there is no truth on this issue, anyway.
Is there anything in the HN guidelines about willfully dismissing or obscuring the truth? This comment makes me feel like there should be.
Saying "we'll never know anyway so stop asking" is abhorrent to me, and I imagine many on this forum. Not to mention it aims to end a discussion it doesn't understand.
Well, the sculpture is a fiction, and this AI generated image is a fiction - what's wrong with acknowledging that these fictions are all we have? Life is a collection of these facsimiles.
Acknowledging that is fine. Saying that makes further conversation irrelevant moves us in the wrong direction as a society.
Money is a fiction, history is a fiction, most scientific concepts eventually become fiction at a low enough level. And look at the world we've built on top of those fictions. It may not be perfect but it is undeniably meaningful.
You'll never know with certainty, but you can certainly know more than you do now. But you won't if you don't pursue the truth, and it becomes a self fulfilling prophecy. As I said in another post, maybe it's possible to sequence DNA of the emperor or a collection of contemporary subjects, determine an approximate ethnic makeup, and revise the images based on that.
Or, we can just assume we are already correct because we'll never know anyway.
I don't think the parent comment was saying "stop asking" merely that "the answer is unobtainable, so any discussion about it is going to be grounded on that"
> trying to find racial undertones in every tangential thing
In this case it isn't tangential but central, since the point is extrapolating physical appearance from incomplete data. It matters if the results predict inaccurate racial features because plenty of people unfortunately care about kind of thing.
And remember that laypeople read "researchers used machine learning" as "super smart people used advanced science to get this result so it's probably true."
I feel like the current climate of society shows that people who have such a simplistic view of science don't tend towards "smart people said this, so it must be true"....
It is, but the criticism is that it's easy to misunderstand and/or skip over the "incomplete" part, and get a wrong conclusion that the results are more accurate than they actually are.
My understanding of the original point is that ML models are trained on a data set. If you train the model on a data set that is only made up of white faces, the results will be whiter than if you train it on a data set that is made up solely of black faces. They said they'd like to know the makeup of the data set that the model is trained on to more fully understand how the results were produced.
> That perhaps some of the emperors weren’t totally Caucasian but rendering them as such from the best available evidence is somehow racist?
These days, sadly, yes. The definition of racism has been so diluted in recent years that, as you implied, I fear it's often a distraction from actual racial injustice. Anyone ever hear of the boy who cried wolf?
I don’t think gp was implying rendering as such to be racist, but was more concerned about people taking these renders as fact and using them to justify a prejudiced world view.
If someone is prone to accepting the idea that caucasians are superior to others they may use something as seemingly innocuous as this neat project to justify their positions and the lack of disclaimers towards the inaccuracies of certain aspects of these renderings may do a small harm societally.
On its own it’d be fine but in conjunction with other symbols and narratives it can become part of a larger, more sinister narrative.
The "racial stuff" seems relevant here. Sure, some people try and insert race into everything, but in this case it's a relevant, and interesting aspect of the problem domain.
My understanding is that the racial identity of the ancient Romans are actually a lot more ambiguous then most people realize, and the way the author attempts to reconcile that ambiguity has a huge impact on the image output here. And it's an interesting problem, Yann LeCun has discussed a bit about how to account for racial diversity via ML techniques (it's not just about the training data set!).
From what I recall, the reason for this racial ambiguity is because ancient Rome (and the entire Mediterranean region) was a cosmopolitan hotbed where many different ethnicities were mixing together all the time. On top of that, they didn't have the same kind of racial constructs we had, and so it's actually unclear what the races of various ancient Romans actually were.
The way I like to explain it to people is that culture, ancestry, and language are 3 separate (but related) things. Knowing one doesn't necessarily tell you anything about the other two. In this case, it would tell you that the genetics are not sufficient to describe the categories of Roman ethnicity without additional justification that in this one particular case, they are.
Colors are can be subjective and somewhat politicized even when there's a written description. It's kind of like how every culture tends to color Jesus to be more similar to the local people, even though everyone knows he's from the middle east.
I might call someone blond with blue eyes, but someone else might describe the same person hair with light brown with green eyes. If we had third person choose which colors represented the descriptions, they might pick completely different colors based on what they thought blond or light brown meant.
Caligula looks to be on point based on the restoration of his original colors in that article.
To your second point, that reminded me that Japan's traffic signals use a different color "green" (looks more blueish) than the rest of the world and they have a common, named color that doesn't really map to blue or green in the rest of the world: https://en.wikipedia.org/wiki/Ao_(color)
These images were rendered with a "white" bias. Facial features is one thing, but skin color is clearly not conveyed by statues, yet a clear choice was made here.
Is that not what 'blonde' generally is in adults? An image search for 'blonde man' returns men I'd consider 'blonde' which hair which seems objectively light brown (except those with obviously bleached highlights.)
The color of Augustus here[0] is definitely blonde for me, bordering on platinum, while what I meant is that it might as well have been darker, i.e. [1] which I would call "light chestnut".
It doesn't matter. Non-white people without steppe ancestry are pretty much uniformly dark-haired and don't have names akin to "subflavum" for traditional hair colors, because light yellow/gold hair just isn't a thing, regardless of what exactly they conceive of as "yellow/gold".
You can't just claim this without suggesting what's actually wrong. What is wrong, and where are the good sources from which we can make a comparison? If you're just guessing then you're no better than this ML model.
https://voshart.com/BIO the artist appears to be pretty socially conscious based on his other work.
Your comment frustrates me greatly because you are correct that people may internalize ML-generated images, and that's a real risk. ML already propagates racial bias within our prison system, as I'm sure you're aware. But Voshart's model isn't making decisions about employment or imprisonment, no, he's literally making art.
Art needs to be allowed uncomfortable racial and ethnographic territory. If not art, what do you think should occupy that space? Art is supposed to be the domain where we get to try out all these ideas and give the public exposure to what things like ML-generated images can be.
But mostly, I hope when the author reads your comment, he doesn't feel discouraged that doing anything in public might risk his career. Or that someone thinking about publishing their own art doesn't look at it and go "hmm, too risky maybe, I better not". The chilling effects that comments like yours have is much more damaging than any racial bias you might be trying to raise awareness about.
I remember when the conscious thing to do was to uplift the art of the unseen people in our world. I wish we could go back to that.
Parent comment seemed like a discussion, not an attack. He started with, this is really cool. What is art for if not to provoke emotion and discussion?
It gets into confusing territory when we talk about who is being "silenced" in a simple conversation where folks disagree, but I think the best outcome is that we sell to understand each other's points first, and then share our own. I hope I've demonstrated that here.
This is absolutely true. It seems like the author did extensive research looking for information about physical features of each emperor to try to produce an accurate rendition. If you look in the links at the right of the page, you can find pretty good documentation on the sources used. I think the link below would actually be a better one for HN.
>Artistic interpretations are, by their nature, more art than science but I’ve made an effort to cross-reference their appearance (hair, eyes, ethnicity etc.) to historical texts and coinage.
I have to agree with the grandparent, this seems like not very strong an underpinning for a project that clearly has the potential to be very controversial. There's no mention of accounting for the artist's potential implicit bias, the drift of ethnically-associated physiological features over the ~2000 years between the subjects and the data set that the process is based on, etc.
For a long time now, I've felt that the best practice for this sort of presentation isn't necessarily a static image, but an interactive experience that allows viewers to explore a range of reasonable outputs.
I agree with the grandparent as well to a degree. The statement "I have created photoreal portraits of Roman Emperors" is tenuous at best. I'm not intending to suggest that the creator of this work didn't spend a significant amount of time and effort or that the end result isn't very impressive. But I do think a disclaimer is warranted in that there's no guarantee that the resulting images actually look like the emperors. I also agree that some type of interactive visualization would be great. But I think these images are very cool as well :)
What does it mean to “render the bone structure of statues” ? A statue is already a three dimensional rendering. Besides Caucasian bone structure is supposed to include all of Europe, the Middle East and North Africa which is pretty much the bounds of the Roman Empire. I don’t remember any black or Asian Roman emperors so I have no idea who you think is being excluded here.
Not to mention that "bone structure" is not a fixed thing. The last 100 years or so, western skulls have seen a reduction in the size of jaw bones and sinuses, smaller mouths, narrower palates, etc. Apparently because we spend less time chewing on tough foods.
I commented about this above, but the race of various Roman emperors is not as clear as we think due to the cosmopolitan nature of the region, and different (lack of?) racial constructs. I believe Septimus Severus, for example, might have been black.
I have not in my Roman Emperor write-up itself been as overt. I thought it might be a bit too obvious. My writeups make it clear when no information about skin tone is available.
I did include such a clause in a forensic facial reconstruction experiment: saying: "My gut instinct is that the results are simply easier to look at, that each step towards realism is a step away from ground-truth skull measurements." because it was an experiment where there are likely living relatives.
Thanks Daniel for your response! I hope my GP comment didn't come across as hostile or discouraging to future artists who want to explore this kind of art. I want to be clear that I'm not offended by your work.
My discomfort stems more from my belief that a non-technical / layperson audience is much more likely to conflate "each step towards realism" with a "step towards ground-truth", the exact opposite to what you suggested. The writeups make clear when information is not available, but the artistic medium itself (a rendered still image showing only one possibility) does not showcase this.
> In response to Cocci’s findings, Voshart removed all mentions of the site and revised several portraits to better reflect their subjects’ probable complexions
You graciously made corrections for plausibly Nazi historical propaganda. However, not every artist/historian may elect to do this, out of scholarly pride or a personal politics or just laziness. The artist subsequently wields tremendous power over the images that show up on Google Image search, and their decisions may unintentionally contaminate future historical research by way of inducing subtle prejudices.
We live in an age of misinformation - and I think the information that must be protected most carefully is information of the past, specifically around people. DeepFakes of living celebrities are fairly easy to verify and disprove, but as soon as you go back ~5 decades or so, some people start to challenge whether certain events (Holocaust, Tiananmen Square Massacre) even actually happened.
Some ideas on how one might mitigate this:
- Provenance technology that shows the editing history (potentially including a MD5 of the dataset) of how an image was synthesized.
- A simple, unobtrusive watermark on the center of the image indicating that it is but one of N possibilities, where N gives a rough estimate of the entropy over the distribution. Sort of like the recycling triangle numbers.
- GIF form showing multiple possibilities given the data you have.
I think there is a grand misunderstanding around my use of machine learning. Your first two suggestions have an assumption that the images weren't heavily modified before going into Artbreeder which was usually the case. or modified in a circular manner, Artbreeder>Photoshop>Artbreeder>Photoshop any kind of MD5 is lost. I could show a progres of each but it is absolutely enormous. Potentially hundreds of decision trees.
As for GAN bias: once in Artbreeder the skin tone is entirely my choosing. If there is a bias in the depiction it is my own bias. There might be a slight bias towards a white, brow-haired female but is is more of a gravity that can be avoided with a bit of skill. There is also a challenge in hair that is -for lack of a better explanation- hair made with human intervention (braids, tall wigs, cornrows, dreads etc).
AS for your last suggestion: I've already been thinking of making videos that show a transition of skin ton of lightest plausible to darkest plausible. Tiberius for example I have a very compelling version as a pale-skinned white dude or a dark-skinned person who looks indian/pakistani. Done in paintover/colorization in Photoshop --directly on the bust- where proportions match.
In the second version of the poster I actually challenged myself to do at least do a quick depiction of each emperor with dark skin. Version 1 my approach was to assuming an average based on place of birth. This time I experimented with assume-darkest-features possible simply to test my assumptions and question my biases. Although quick and not robust, I think it was of value.
Your comment sounds a bit like obscurantism ? The classic "You shouldn't do this, it's bad because you're biased. Besides, it's impossible to not be biased, so don't bother". Even though there are documented descriptions of the physical aspect of these people.
I think these things are mostly for interest generation articles and possible exhibitions at museums. Artists have been sculpin these things since before the digital age, and while doing it with ML is cool and all, the results aren’t really more historically useful than what the sculptors did. But hey, if it generates interest I don’t see anything wrong with it either.
I wouldn’t be too worried about whether or not people view them as historical renditions. Most layman’s history “renditions” are mostly somewhat made up.
> What data and models were used to synthesize these predictions?
You have to ask the creator of ArtBreeder (https://www.artbreeder.com/about) which apparently uses BigGAN, StyleGAN, etc., trained on different datasets to offer these options:
That being said, there is a heavy manual touch (based on "Photoshop and historical references"). See for instance how different Septimus can appear compared to the webpage:
After your comment I was expecting to see all pale blonde pictures, but there was quite a variety in skin tone, hair color, etc. I remember reading that the Roman empire was diverse because of their history of conquering, but the extent of the diversity here really surprised me.
...Or were you saying that this is more diverse than is to be expected?
This is essentially the same issue with things like artists conception of dinosaurs. Lots of people probably have strong mental images of what color a T-Rex is, despite scientific evidence being very scant. Maybe "artist's conception" is a good term to put next to these sort of ML-generated images?
Same technological challenges, but the rendering of past historical figures' ethnographic features has much higher social stakes and is a considerably more politically sensitive issue than how a T-Rex is rendered.
What data and models were used to synthesize these predictions?