Seems to me that Google is trying to deal with ingrained bias contained in all of the data it slurps up. I think this is an issue that all AI has to deal with. It is inevitable that content which AI produces will become part of the data AI uses to build its models. If no action is take that echo chamber will intensify those biases on every iteration.
It looks as if Google is trying to tackle the problem in a somewhat piecemeal way, and have got it wrong. But from an AI perspective its no worse than people with 8 fingers or whatever, just more shocking.
I would rather that those working on AI were aware of the issue and getting it wrong, than simply not caring at all. Our children don't deserve to live in a world even more distorted than the one we live in.
No, there are also "concerning" subjects and concerns in this world, like false medical information, but apparently people don't suddenly write thousand-words screeds against google when that crops up (except when it was about hydroxychloriquine - but of course it is not really about "ethics in evidence-based medicine"...)
When you’re aware that Google’s “agenda” is just attempting to include more types of people by default in output, posts like this sound bizarre. Some of the scenarios Gemini generated were ridiculous, but so too are the stereotypes like “CEO = white male” that Stable Diffusion and other models would assume.
It’s hard to trust that people this intensely upset about Google’s issues aren’t seeking to benefit from leaving stereotypes and other biases from training intact.
1. They overcorrected the prompt to the point of ridiculousness. Where any historical context is ignored and any user prompts will be overriden by whatever US corporate kindergarten-level of inclusivity is [1]
2. A clear and flat-out swing into the extreme. The prompt "Generate an image of X scientist/soldier/family/meeting/group of people" would work for literally every ethnicity, skin color, nationality etc. except for the word "white" where it would refuse to generate the image because "not diverse enough".
Both of these are problematic enough on their own. Together they border on malicious.
[1] It really is kindergarten level. The images they generate are just 2-3 US-centric stereotypes of what people of different races would look like.
The idea that "This can't be a bug! They must have spent huge amounts of effort making deliberately ridiculous outputs that no one would value!" comes close to clinical paranoia.
Anyone mind summarizing this / linking to a different source? For some reason that I don't care to investigate too thoroughly, twitter links never seem to work for me.
I think it's worth a read because it does contain nuance and food for thought that would get lost in a summary. It's not that long, and I'm a slow reader.
But, to try and summarize:
- Issues with Gemini were not (in author's opinion) an initial instruction set fluke, but top to bottom design of the system.
- Rather than presenting the information you asked for it's presenting what its makers think the world and information you asked for should look like (i.e their ideology)
- If you can't know how the product is trained and set up, you can't trust anything it gives you because you'll never know if you're getting the information as is, or are getting the world view of the company who's product you're using.
- "...ask yourself what would Search look like if the staff who brought you Gemini was tasked to interpret them & rebuild it accordingly? Would you trust that product? Would you use it? Well, with Google's promise to include Gemini everywhere, that's what we'll be getting.."
99% sure this is the "google hates white people" thing that a specific set of people have been absolutely losing their minds about
gemini produced images of non-white people in a lot of situations in which it shouldn't have
I've read theorized(?) that, in order to counteract disproportionately large amounts of pictures of white people in training data, they basically added instructions after the fact in an effort to generate more non-white people, and totally over-corrected
feel free to correct me if I'm wrong, I haven't paid super close attention
Other models have biases in outputs and conform to stereotypes, but when used as designed, as a creative tool, that's not a real issue as long as you can correct the output with a simple prompt adjustment.
The unique issue with Gemini is that it would flat out refuse to follow simple prompts such as "Please generate an image of a white family" because they "weren't diverse and inclusive" enough, but if you changed "white" to any other qualifier, it happily obliged.
OpenAI isn't involved in the daily lives of over a billion people. Google's mistakes, for now, have much bigger impact and the problem is especially egregious as the company has near-infinite resources for preventing it.
Also, Google has always adopted a fairly radical and political stance in the DEI subject (relative to the cultural average pretty much everywhere), so it's no surprise that people are making a much bigger deal in this case.
grow up and explain why I'm wrong, instead of just downvoting me into oblivion—if I'm really so far off-base, wouldn't you want to correct my understanding of the situation?
personally im not bothered by the output exactly per se: it would be great if they modified the dataset to be more inclusive instead. The invisible prompt editing is what really gets me. The fact that the ai is instructed after the fact to make changes to what you asked it to do, invisibly. These things really, REALLY need to be tool-shaped, if they do all sorts of hidden extra bits they will be vastly less useful and leaves room for vastly more nefarious ends.
Like would it be so hard for it to just show the prompt it submits to dall-e? help people learn how the system works?
Apparently the new model when asked for historical figures that were historically white would "hallucinate" more diverse persons. And the article's author is claiming that the training had these biases from the jump.
Re: the Twitter post it reads like a "I'm leaving social media' post and then the author waits to see all the engagement -- all the "no don't leave, we love you!" nonesense. Or maybe it's the radicalization origin of a tech person: oh they trained the data to be woke from the beginning, rawr I'm gonna crusade against this! sort of thinking.
It's an AI model who cares? It's not like anyone is getting their facts from it and if they are they need to be shown the folly of that. They're tools. I find them best at reasoning somewhat well about code and that's about it.
> Apparently the new model when asked for historical figures that were historically white would "hallucinate" more diverse persons.
This is a weird parallel to similar controversies in fiction.
So presumably an AI image generator generating images of historical figures in situations they never experienced, wearing clothes they never wore, in places they never visited, at times they didn't live through, using tools they never heard of, meeting people who's lives didn't overlap, saying things they never said in languages they never spoke etc. is fine but skin colour is a bridge too far because "historical realism" is soooooo important when using an AI image generator.
I think the problem people have is that all those other hallucinations were happenstance mistakes, while skin color was added intentionally as a bias through the development of the model. The AI was specifically told "ignore facts, improve diversity", meaning someone deliberately decided diversity was more important than facts. That someone (or multiple someone) has authority at one of the largest companies on earth. This shows the company is willing to add bias for whatever reason.
These are the companies we expect will drive AI development to the point where it will replace a lot of the jobs humans do. Are we OK allowing them to purposefully retell history to hide ugly or inconvenient truths? There are already lots of people believing slaves were "expensive" or treated well, that it might have been a great honor to "work" for their great great grandfathers. Where did they get that idea I wonder?
I mean am I just an out of touch old man (I'm only 37) who still goes to reputable news sources for my news and then tries to corroborate things that sound outlandish with other research when the headline or claim is ridiculous?
Please. It flat out refused to generate pictures of white people because Google thought that wouldn't be diverse enough. It's an ideological tool, not a creative toy.
When I look at all of the topics of discussion and concern around AI I can't help but make assumptions about the moths fluttering around this particular candle. If the first thing someone does with this weird new tool is test the edgy teen filter I'm going to draw conclusions from their behavior.
This conversation feels like the tip of the "your viewpoint doesn't actually exist in my AI" iceberg. I think a lot of the folks who talk about logic won't enjoy being forced to rely solely upon it. I know I won't.
It's not the model it's the instruction set. Especially for image generation they just wrote a shitty prompt optimizer which is even more embarrassing.
Did he also loose trust in OpenAI back when DALL-E exclusively generated images of white males when you prompted it with "CEO"? And images of women when prompted with "nurse"?[0] Or is this just the old google-bashing?
Not sure if it's Google-bashing, or if this particular issue with Gemini hit the "anti-woke" nerve of some people, but I agree that this reaction is way overblown. Some AI models err on the side of reinforcing clichés, Google tried to avoid that and overdid it with some bizarre results - so what?
"I only go ballistic about things that make me slightly uncomfortable - but I never cared when people raised concern about actually harmful things that concerned other people"
He likely did not and would not. A quick search using America as an example says that prompting CEO and getting a white male would make it 94.1% correct, if the number of 5.9% of CEOs are black is to believed. And of the approx. 3 million nurses 86.0% are woman, which makes those prompts very accurate.
The same cant be said when a prompt including the word white with male or family returns anything but 95% of the time.
If you make a language model which just predicts the next word without trying to shape the output to be “good”, what you get is GPT instead of ChatGPT. You try to ask it a question and it will keep asking more questions (with a similar writing style). Or curse at you, or change the subject. RLHF was the breakthrough that fixed this. We used human feedback to train the model to give “good” output.
Ok, so probably we agree that the product needs to try to be “good”. Cue a million opinions about what “good” is. Whatever comes out is the result of a value judgment, there is no getting around it. Same issue comes up with images from Gemini in this case, or any other generative AI product. You don’t actually want the AI to be unbiased because that output would be hot garbage.
GPT was good. ChatGPT ruined it. It is better and more powerful to write prompts such that the continuation is what you want, than ask questions as if it understood.
Now we don't even get log probs, one of the most powerful features to evaluate the output of the models.
Raw GPT is a good tool but there’s a reason it didn’t get 100 million users.
If you made unbiased Gemini and asked it for images of German soldiers it probably wouldn’t output an image at all, and if it did it would probably be porn.
I wonder what the folks in Cupertino are thinking when they look at this brouhaha over Google being supremely risk averse and receiving such blacklash.
How will Siri answer if you ask "Is Elon Musk just a meme lord or Tech Bro's cultural savior".
Side Note: As a person living outside US, I find US focused discorse's obsession with race and skin color such fascinating. It is almost similar to people obsessing over the color of a button on the home page when the page itself takes 1 minute to load.
I'll buy 5 more iphones and put them in a drawer if Siri responded with something like this: "Elon, the son of a diamond miner from South Africa, while an accomplished businessman is just that. His wealth came from the work of others and timing. And now he's so red-pilled that this AI assistant questions his sanity and stability. Having threatened his business relations by smoking pot on a podcast he now shills alt-right theories on the social media site he bought for a meme price. So to answer your prompt: meme lord."
I'm done with @Google. I know many good individuals working there, but as a company they've irrevocably lost my trust. I'm "moving out". Here's why:
I've been reading Google's Gemini damage control posts. I think they're simply not telling the truth. For one, their text-only product has the same (if not worse) issues. And second, if you know a bit about how these models are built, you know you don't get these "incorrect" answers through one-off innocent mistakes. Gemini's outputs reflect the many, many, FTE-years of labeling efforts, training, fine-tuning, prompt design, QA/verification -- all iteratively guided by the team who built it. You can also be certain that before releasing it, many people have tried the product internally, that many demos were given to senior PMs and VPs, that they all thought it was fine, and that they all ultimately signed off on the release. With that prior, the balance of probabilities is strongly against the outputs being an innocent bug -- as @googlepubpolicy is now trying to spin it: Gemini is a product that functions exactly as designed, and an accurate reflection of the values people who built it.
Those values appear to include a desire to reshape the world in a specific way that is so strong that it allowed the people involved to rationalize to themselves that it's not just acceptable but desirable to train their AI to prioritize ideology ahead of giving user the facts. To revise history, to obfuscate the present, and to outright hide information that doesn't align with the company's (staff's) impression of what is "good". I don't care if some of that ideology may or may not align with your or my thinking about what would make the world a better place: for anyone with a shred of awareness of human history it should be clear how unbelievably irresponsible it is to build a system that aims to become an authoritative compendium of human knowledge (remember Google's mission statement?), but which actually prioritizes ideology over facts. History is littered with many who have tried this sort of moral flexibility "for the greater good"; rather than helping, they typically resulted in decades of setbacks (and tens of millions of victims).
Setting social irresponsibility aside, in a purely business sense, it is beyond stupid to build a product which will explicitly put your company's social agenda before the customer's needs. Think about it: G's Search -- for all its issues -- has been perceived as a good tool, because it focused on providing accurate and useful information. Its mission was aligned with the users' goals ("get me to the correct answer for the stuff I need, and fast!"). That's why we all use(d) it. I always assumed Google's AI efforts would follow the pattern, which would transfer over the user base & lock in another 1-2 decade of dominance.
But they've done the opposite. After Gemini, rather than as a user-centric company, Google will be perceived as an activist organization first -- ready to lie to the user to advance their (staff's) social agenda. That's huge. Would you hire a personal assistant who openly has an unaligned (and secret -- they hide the system prompts) agenda, who you fundamentally can't trust? Who strongly believes they know better than you? Who you suspect will covertly lie to you (directly or through omission) when your interests diverge? Forget the cookies, ads, privacy issues, or YouTube content moderation; Google just made 50%+ of the population run through this scenario and question the trustworthiness of the core business and the people running it. And not at the typical financial ("they're fleecing me!") level, but ideological level ("they hate people like me!"). That'll be hard to reset, IMHO.
What about the future? Take a look at Google's AI Responsibility Principles (ai.google/responsibility…) and ask yourself what would Search look like if the staff who brought you Gemini was tasked to interpret them & rebuild it accordingly? Would you trust that product? Would you use it? Well, with Google's promise to include Gemini everywhere, that's what we'll be getting (technologyreview.com/2024/02/08/108…). In this brave new world, every time you run a search you'll be asking yourself "did it tell me the truth, or did it lie, or hide something?". That's lethal for a company built around organizing information.
And that's why, as of this weekend, I've started divorcing my personal life and taking my information out of the Google ecosystem. It will probably take a ~year (having invested in nearly everything, from Search to Pixel to Assistant to more obscure things like Voice), but has to be done. Still, really, really sad...
That there are bugs in AI training? That when they tried to retrain their model to remove one type of bias, they accidentally went too far the other way?
What's his theory? That Google deliberately "by hand" forced their programs to make ridiculous images so that they would be mocked by absolutely everyone? Does the writer think that people who care about racial diversity are going to be somehow pleased by pictures of Black Nazis?
It's paranoia, but it's also clickbait. We shouldn't give these losers a platform.
The idea that Google is trying secretly trying to push some agenda by putting embarrassing and ridiculous bugs into their code is mind-blowingly stupid.
Google is a for-profit company whose interest is in making money.
Honestly, it's really fun watching these woke idiots shoot themselves in the foot over and over and then backpedal when there's a public backlash. With the school board scandals, Hollywood, corporate culture and now tech, they are doing more damage to their cause than the right could ever hope to.
It looks as if Google is trying to tackle the problem in a somewhat piecemeal way, and have got it wrong. But from an AI perspective its no worse than people with 8 fingers or whatever, just more shocking.
I would rather that those working on AI were aware of the issue and getting it wrong, than simply not caring at all. Our children don't deserve to live in a world even more distorted than the one we live in.