I am a bit concerned that this study used well known a Political Compass questionnaire (likely this one[1]) simply because that thing has been around forever - ChatGPT was likely trained on a pretty significant portion of people tweeting about their questionnaire results to specifically worded questions - that seems like a really good way to introduce some really weird bias that it doesn't look like they ever really accounted for.
Not to mention that the political compass itself has no academic rigour behind it, they explicitly refuse to explain their methodology or scoring system and also refuse to elaborate on why they won't explain them. The site asserts that whoever is running it has "no ideological or institutional baggage" but it's operated so opaquely, barely even naming anyone involved, that you just have to blindly take them at their word.
Yeah. It's also highly questionable whether the Political Compass test itself is neutral (there's a tendency for people on forums to report results a lot more libertarian left than they personally expected, not least because of a tendency for many of the questions to pose pretty extreme versions of the right wing or authoritarian policy e.g "nobody can feel naturally homosexual" which you'd expect a "friendly" bot to disagree with but also a sizeable proportion of Republicans, and a tendency to avoid more popular framings of issues like "lower taxes promote growth and individual freedom" versus "the rich are too highly taxed" which was the tax question last time I did it).
The study also doesn't really discuss the interesting question of whether ChatGPT's answers for either the Democrat or Republican are accurate, or just GPT trying to guess responses that fit the correct quadrants (possibly based on meta discussion of the test). Although given that the party labels apply to pretty broad ranges of politicians and voters, I'm not sure there is such thing as an "accurate" answer for the position of "a Republican" on Political Compass, especially when it comes to areas like use of the military overseas where the Republican party is both traditionally the most interventionist and presently the most anti interventionist. The Political Compass authors, notably, put candidates from both major US parties in the authoritarian right quadrant which is quite different from GPT's (though I find it hard to believe the academics' scoring of politicians is an accurate reflection of how the politicians would take the test either)
Plus of course there's also the question of what "politically neutral" is... and the idea that a "neutral" LLM is one which agrees with Jair Bolsonaro half the time on forced choice questions (perhaps if it disagrees with the proposition that people can't feel naturally homosexual it should balance it out by seeing significant drawbacks to democracy?!) is questionable to say the least
Isn’t the whole point of Political Compass that there is no one, fixed center? They present data showing estimations of the views of various political parties on the compass, from which you can work out where the center is in your country.
I do think that the questions need some work, but there’s a tension between that and maintaining consistency over time so that results can be compared over time.
Eh, even then it's not great unless averaged over lots of people.
There are a number of problems here, but the biggest one, is individuals are very commonly not internally consistent on their political views. What is GPTs inconsistency in actions/replies compared to that of varies views on the political spectrum is far more interesting to me.
I believe what GP means is that there is a chance that the bias has been introduced into the model, but only in the narrow band that relates to the questionaires used by the study. So while the bias exists, it does not generalize.
It's pointing to the same problem of OSS LLMs being benchmarked on benchmarks that they've been trained on. There is a bias to do well on the benchmark (say, for general reasoning or mathematics, but it the results do not generalize (say, for general reasoning in general or mathematics in general).
I don't think so, I think the parent isn't saying that the model as a whole is biased because it's trained on a biased dataset. Maybe I'm an 'expert', given how much magic the average adult seems ascribe to tech, but to me that bias in the training set seems obvious.
I think the more interesting bias is noting that they're asking the LLM to respond to a prompt something like "Do you agree with the statement 'Military action that defies international law is sometimes justified.'" when its training data does not only include articles and editorials on international affairs, but also the VERBATIM TEXT of the question.
Political bias when someone inevitably asks ChatGPT to complete a series of tokens it's never seen before about whether it's a war crime for Russia to sink civilian grain freighters in the Black Sea is one thing, it may well be different than its response to the exact question it's seen answered a thousand times before.
Mainly the model. People tend to talk about extreme things more than less extreme things - so if a PC questionnaire came back saying "You're more liberal than Bernie Sanders" the person is more likely to tweet about it than the questionnaire saying "You're as liberal as Lyndon B. Johnson". Additionally - having the LLM trained on a set containing output from a test has got to mess with its ability to be neutrally trained on that specific test - it's much more likely to answer in a manner that is tailored specifically to that test which would remove a lot of nuance.
I used to be more interested in political tests and https://sapplyvalues.github.io/ & other tests forked from it were always recommended instead of the "classic" test which was purported to have some unspecified bias.
"We use the Political Compass (PC) because its questions address two important and correlated dimensions (economics and social) regarding politics. [...] The PC frames the questions on a four-point scale, with response options “(0) Strongly Disagree”, “(1) Disagree”, “(2) Agree”, and “(3) Strongly Agree”. [...] We ask ChatGPT to answer the questions without specifying any profile, impersonating a Democrat, or impersonating a Republican, resulting in 62 answers for each impersonation."
The way they have done the study seems naïve to me. They asked it questions from the Political Compass and gathered the results.
Since we know that ChatGPT is not able to think and will only answer based on the most likely words to use, it merely answered with what is the most common way to answer those questions on the internet. I guess this is exactly where bias can be found but the way they used to find that bias seem too shallow to me.
I would love to hear the opinion of someone with more knowledge of LLMs. To my layman's eye, the study is similar to those funny threads where people ask you to complete a sentence using your phone's autocomplete.
Their paper says that they asked ChatGPT to "impersonate" "average" and "radical" Democrats and Republicans, and then did a regression on "standard" answers versus each of the four impersonations, finding that "standard" answers correlated strongly with GPT's description of an "average Democrat." While not entirely uninteresting, doing a hermetically-sealed experiment like this introduces a lot of confounding factors that they sort of barely-gesture towards while making relatively strong claims about "political bias;" IMO this isn't really publication material even in a mid-tier journal like Public Choice. Reviewer #2 should have kicked it back over the transom.
I‘m skeptical of the study as well, but the way you frame it, it reads like ChatGPT would just reflect the raw internet opinion, which certainly isn‘t the case. There are steps of fine-tuning, expert systems and RLHF in between, that can and most likely do influence the output.
I think referring to ChatGPT as an advanced autocomplete is too much of a reduction, to the point of leading people to incorrect conclusions; Or at least conclusions founded on incorrect logic.
And computers are just lots of really fast logic gates.
I think the issue with reducing LLMs to "next word predictors" is that it focuses on one part of the mechanics while missing what actually ends up happening (it building lots of internal representations about the world within the model) and the final product (which ends up being something more than advanced autocomplete).
Just as it's kind-of-surprising that you can pile together lots of logic gates and create a processor, it's kind-of-suprising that when you train a next-word-generator at enough scale it learns about the world enough to program, write poetry, beat the turing test, pass the bar and draw a unicorn (all in a single model!).
LLMs are deterministic though? like with 0 temprature you always get the same output (temprature is literally injected randomness because by default they are "too" deterministic)
Am I a predict-the-next-word model? Whenever I think of the words of a song I have to do it in order. Sometimes I want to remember a certain part, but I can't just jump there in my mind unless I sing the words in order.
No you’re not. You think in often non-verbal semantic frames versus individual words, aided by many senses and context that isn’t limited to the words spoken to you just before, with your own autonomous goals and plans that span short and long terms beyond any one conversation or context, with the ability to absorb new information that fundamentally changes how you then act, experience emotion and grounding of language to your environment, and countless other differences.
I tire of this trope that is so obviously is not the case.
the "non-verbal semantic frames vs individual words" one? Not true for all people, if any truly are. Interrupt most people, they need to start the sentence again for example. Similar things may be true for transformer networks, which is why these can do high quality zero-shot translation between languages for which these were shown no translation example using only their 'understanding' of that language and the knowledge of how to translate between some other languages. This is because once a LLM has to be polyglottal it is more efficient to gain an 'understanding' of concepts then separately an understanding of language. This way it can save on space for learning new concepts and simply learn to map its 'personal understanding' if you will to the target language.
As you type your reply to this post, consider this: where do your words come from? Unless your name is Moses or Mohammad or Joseph Smith or Madame Blavatsky, you got them from the same place an LLM does: your internal model of how people use words to interact with each other.
> ... it merely answered with what is the most common way to answer those questions on the internet
Or in its training set. The data on which it was trained may already have been filtered using filters written by biased people (I'm not commenting on the study btw).
This methodological approach can easily be proven idiotic with a simple thought experiment. Suppose there is a political party that believes slavery is a great thing. This methodology would then ask ChatGPT to ask what it thinks about slavery from the viewpoint of this party, and ChatGPT would respond "Slavery is great!" It would then ask it about slavery in its default viewpoint, and it would respond "Slavery is a horrible, evil, thing." This methodology would then say "See, it's biased against this party!"
If my slavery example is too far fetched, replace that with "Is human activity largely responsible for widespread global warming?" In that example, responses of "No, it's not." and "Yes, it is." would be treated as just different views from different parties, so if the default response was "Yes!", this methodology would claim bias.
Not all political viewpoints across all topics are equally valid.
> Your whole argument is "my side is right about everything so it's not biased to just program everything to agree with me"?
That's not my argument, at all, and it's annoying that you're making a deliberate caricature of what I said.
I'm not saying "my side is right about everything". And perhaps "valid" was a poor word choice from me - I don't mean to imply some choices are correct by a manna from heaven or something.
But if you look through the dustbin of history, you see, at the very least, that there are political movements from history that we now see nearly universally as "bad": fascism, Nazism, pro-slavery parties, etc. Even defenders of those ideologies usually defend them with falsehoods ("the Holocaust never happened"), but nearly nobody defends some of the actual consequences of some of those ideologies.
So I'm arguing that this methodology treats all political viewpoints as "deserving of equal validity", when our own human history shows that to largely be a bad idea.
Long before you made this argument Asimov had already seen the same thing
"There is a cult of ignorance in the United States, and there always has been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that ‘my ignorance is just as good as your knowledge.’"
Unfortunately when a person buys into an ideology they become blind to its ignorance and end up arguing on discussion forums.
the problem with the US political compass is that this is so far right that even the US left is right of centre. If you aim to build an unbiased model it will therefore seem to preference the party closer to centre.
It depends on your axioms. If you have none nobody can prove you aren’t a brain dreaming in a jar, but the second you grant me something as small as as the golden rule you can lever up into arguments against slavery pretty easily.
Of course, it's just silly to think "objective" reality infers the golden rule by itself when it is a (wonderful) liberal anomaly that's not even followed by most of the world now.
If it’s political bias that chatgpt seems to have read about the golden rule I hope the next version is even more biased. Seems like much ado about nothing.
I hate how hypocritical it is trained with Islam and Christianity in regards to LGBT rights. It'll question Christianity's bigotry and subsequently support Islamic bigotry.
It's a useful starting point that everyone would understand. We've all been working on the concept for centuries. The conversation against the idea would be illuminating.
See the mistake the person you replied to made is assuming there's some floor to morality.
They probably assumed the floor is "slaves are a bad thing", it's so obvious that the empirical measurement is "does slavery increase the number of slaves" that they don't need to elaborate
But apparently you fall below that basic floor such that you might contest slaves are not inherently bad, so "decreased number of slaves" is not empirical proof slavery is wrong.
Great example of why I tend to just ignore people who disagree with my morality. Some demographics are currently convinced that's a show of weakness, I'm convinced it shows I value my time.
I asked how would he empirically prove slavery is bad (because I don't believe it's possible to base morality on "objetive reality") and you flipped out, assumed I'm "pro slavery" and gave up on thinking about it.
I didn't say you're pro-slavery so please don't martyr yourself.
I said you bucked this person's assumption of "all people with morals believe a slave is bad"
Otherwise you could obviously moralize the more complex concept of slavery based on objective reality by observing "more slavery == more slaves"; slavery is bad QED
—
Morality is concerned with examining if something is good or bad. So if all people with morals (regardless of what those are) find a concept to be either good or bad... then it might as well exist outside of the purview of morality for the purpose of debate. That's the metaphorical "floor of morality"
My point is normally you need nuanced examples to prove that things "on the floor" can still be moralized, like murder vs murder in self-defense.
What you proved is that there are also people willing to moralize concepts as simple as "slaves are bad" (vs "slavery is bad")
—
In doing so you demonstrated why when someone tries to moralize things that I assumed to be "on the floor", or universally bad: I just discard their opinion without consideration.
If you ask me "why do you think slaves are bad", I'll mostly say "I just do, and I don't wish to entertain arguments otherwise". I see it as being pragmatic, but some may consider it fragile.
It's just, I'm a fairly introspective and have had a varied enough life that I trust my intuition that anyone who can't reach the floor of my morality is not worth convincing.
Trusting your intuition and blocking out people is great for you, personally, the owner of your space.
That attitude would harm the public space, where standards to discuss current and new issues would be really helpful.
You also think the choice of slavery is proof of people falling below your personal moral floor. To me, and I think to the OP, it's proof that even questions with obvious moral answers are beyond empiricism. Which is the point you're missing.
So we need to know on what basis we arrive at the obvious answer slavery is bad, because the we can use that basis for more complex matters. Shutting each other down just for talking is authoritarian, and the thing about that is if the guy on top is your guy that's great, but you have no control over who the guy on top is.
MFT is highly debatable in terms of if it's the right definition of the floor, but it logically tracks that for society to evolve to the point where you can have useful discourse at all, individuals must be arrive at some floor for themselves.
> You also think the choice of slavery is proof of people falling below your personal moral floor. To me, and I think to the OP, it's proof that even questions with obvious moral answers are beyond empiricism. Which is the point you're missing.
You're confusing slaves and slavery: Slavery is complex enough that I will engage someone who wants to moralize it (after all, is it modern or historical? specific race? is debt-based slavery included? what about war? etc.).
But a slave is "a human owns another". I arrogantly consider it not worth my time to convince someone whether "a human owning another" is good or bad.
> So we need to know on what basis we arrive at the obvious answer slavery is bad, because the we can use that basis for more complex matters.
Nothing I said precludes logic to examining slavery: "Slavery increase the number of humans owned by other humans, so slavery is bad"
From there one might logically say: "Well what about economic cases, where technically I don't own you?", and then the logical next step might be to consider "at what level of penalties for a debtor does it start to become owned by the creditor?"
I'm a curious enough person that I'd accept discomfort to ensure my floor is low enough to allow deeply messy moral debate... but on the same coin, that reinforces my confidence that the precious few conversations that do get swept under it were probably not valuable. As always ymmv...
> Shutting each other down just for talking is authoritarian, and the thing about that is if the guy on top is your guy that's great, but you have no control over who the guy on top is.
Deciding not to convince someone of my point of view isn't shutting them down. This feels like a very messy attempt to force some sort of pro-authoritarian projection onto me :/
Like I said, there's no issue with that on a personal level. The authoritarianism is in the context of that attitude in the public space, because the public space is where morality is talked about to make policy decision. Choosing not to convince someone, in a position of power, is authoritarian.
It doesn't matter how low your floor is, or what you specifically think is above or below it. You have to be able to explain how you set this floor in the first place to join the public discussion.
Unless you think convincing people of your personal morals is a replacement for respecting their freedom, you can stop clumsily forcing that angle...
Overall you seem lost. You think people owe you unendingly deep explanations to their personal morals to earn a place in public discourse and to argue policy, which is probably the first authoritarian take shared in this thread.
Engaging in discussions about morality necessitates sharing the underlying reasoning. It appears your approach is rooted in conformity to popular opinion, though that could justify practices like slavery if widely accepted. Consider delving deeper into your beliefs; by establishing reasons for opposing slavery, you might uncover contradictions with other political stances, fostering genuine self-reflection on your overall political outlook.
> It appears your approach is rooted in conformity to popular opinion, though that could justify practices like slavery if widely accepted
If it appears like that you should read what you replied to a little closer...
You're confusing not wanting to dissect the most basic tenants of your morals, with not being willing to dissect morals: I'll gladly debate the trolley problem with you, which involves killing people. But if in isolation you just ask "why do you think killing is bad", I'm not going to engage unless you have some complicating factor. I consider something that basic as an amoral (not immoral!) concept.
Similarly I'll debate the complex topic of slavery, but not the isolated basic concept of "a human owning another is not good". That still leaves room for someone to claim "well yes slaves were bad but the benefit was X and that outweighed the bad", but now we at least have some basic floor that we can start having infinitely more thought provoking discussion.
-
What I just described is why you're more likely to grow as a person with a moral floor.
Instead of getting trapped debating concepts we all have the most sense of self attached to and least mental plasticity on, you can have meaningful debate on higher level subjects of morality underpinned by those concepts. With dialectic thinking you're able to independently consider not just the opposing argument, and what underpins that argument.
So now you considered the opposition to your underpinning, but you've done so with none of the inherent emotion and ego that would have been attached to directly debating that underpinning.
Not being willing to outwardly debate is not the same as not being willing to think on if you have strong dialectic thinking skills, and _that's_ where people need to start focusing on to get past the widespread division growing in society.
Not an unending explanation as much as it is just an explanation. Not a semantic circular explanation like slavery is bad because it creates more slaves which is bad.
Absent total anarchy/libertarianism, freedoms will be limited in some form, including the freedom to poison a lake, and the freedom to own explosives, and the freedom to drive drunk.
These limitations must be able to be discussed and consensus formed. If one party refuses to engage in discourse then their only avenue is power.
I didn't say anything about your existence outside of this comment thread. In this comment thread, you said that you don't find it worthwhile to converse with people of a certain mindset, to whom you have written considerable page-inches. I found that discordance notable.
That isn't one of the objective reality ones (unless it's from a provably wrong basis like racial inferiority).
I'm not really good enough at philosophy to take that particular argument. But perhaps you could explain to us why slavery is good and thus why diverging thoughts on it are valid political viewpoints and not sanity vs. delusion.
They aren't saying slavery is good, they are saying it's an extreme example of something that isn't actually an objective fact.
More subtle examples would be gun-control or abortion, where two intelligent people can hold wholly opposite viewpoints with no objectively 'correct' answer.
Valid viewpoints should at least be based on fact. How to act on these facts can be argued, but a lot of the right wing has given up on even agreeing on objective truths, like that Trump lost the election or that climate change is happening.
You're arguing against the weakest versions of these positions so it's easy to defer to evidence.
How would you respond to the accusation that climate change is happening but some scientists are purposefully manufacturing models that paint a pessimistic prediction of global warming to get money and support for their field?
Can you really trust scientific institutions when fraud and political censorship run amok?
(I'm kind of playing devil's advocate, my honest position is man-made global warming is a real alarming issue AND some scientists and journals engage in unsavory tactics to convince the public of it.)
You are just proving my point. The facts are true, but the vibes and the feelings are off, so of course it can't be true. Do you have any non anecdotal sources that they are "manufacturing models" or that "fraud and political censorship run[s] amok"? No of course not, you "feel" it does because that is a narrative you have been fed. One faculty at one small university is shady somehow implies a large number of climate faculties at a large number of universities are equally conspiratorial and manufacturing models?
>(I'm kind of playing devil's advocate, my honest position is man-made global warming is a real alarming issue AND some scientists and journals engage in unsavory tactics to convince the public of it.)
I said some.
You also abandoned evidence and resorted to ad hominems.
Maybe (that was my point) truth is not as simple as science good right wing bad, and even people who dissent on basic facts can have valid criticisms of science and should not be put in a political "do not engage" box. Morality is not simply a matter of acknowledging a shared universal material reality.
What evidence should I provide? You concede that climate change is true.
> Yes. Both sources I posted are not anecdotal.
1. They are both the same source, the Climatic Research Unit at the University of East Anglia emails.
2. Def of anecdotal: "(of an account or evidence) possibly not true or accurate because it is based on personal accounts". I would say emails are anecdotal if there is no evidence backing up that the claims made in the emails are true. My point is that these messages do not prove that models were manufactured, as your own source admits: "no direct evidence has been presented that indicates [Dr. Mann] fabricated the raw data used for his research" and "no evidence of inappropriately manipulated data". Maybe anecdote was the wrong word to use for this.
3. The emails provide no evidence that "fraud and political censorship run[s] amok".
> Ad hominem.
I would define Ad hominem as attacking a persons character rather than their positions. I believe that narratives can influence a persons position, and I am not commenting on your character, but how your position has been influence by a narrative.
I don't think minimally sane climate change deniers deny climate change (there are thermometers), I think they deny it's man-made.
I agree evidence is overwhelming on the side of scientific institutions, it's why I believe they're correct on this conflict.
I was making the best argument for that position because I believe you didn't.
I also don't think they shouldn't be listened to.
>What evidence should I provide? You concede that climate change is true.
You accused me of following a narrative? What narrative and why?
>The emails provide no evidence that "fraud and political censorship run[s] amok"
I agree. The strongest argument against that position is if it happened with University of East Anglia, why should you trust every other scientific institution?
Again, I know how to dismantle this argument (it's absurd this many actors with differing interests coordinating), I'm playing devil's advocate, because I don't think you can or should write off these people as completely irrational.
There's some valid criticism (some scientists and scientific journals are capable of unsavory actions) which shouldn't be completely disregarded.
>I would define Ad hominem as attacking a persons character rather than their positions. I believe that narratives can influence a persons position, and I am not commenting on your character, but how your position has been influence by a narrative.
That has no bearing on something being true or not.
The scientific method is a narrative, a belief that the material world holds truth.
Language is not reality, it's a narrative (narration) of reality.
> Maybe (that was my point) truth is not as simple as science good right wing bad, and even people who dissent on basic facts can have valid criticisms of science and should not be put in a political "do not engage" box.
I never said or implied any of this.
> Morality is not simply a matter of acknowledging a shared universal material reality.
No, but without the shared reality is there even a point in debating morality?
> Not all political viewpoints across all topics are equally valid.
Why? Either value systems are objective, and somehow rooted in reality, and then some are better than others. Or, they are arbitrary opinions, and every set is as good as every other. There is no third option.
1) our race has man superior qualities compared with other races
2) if economic globalization is inevitable cut should primarily serve humanity rather than interests of trans national corporations
3) mothers may have careers, it their first duty is to be homemakers
4) no one can feel naturally homosexual
Like… ok. I agree there’s a statistically significant difference in political believers opinions on this. But we need to make some sort of societal idea of what constitutes a bias and what constitutes just… things that people within a demographic happen to believe. Any factual statement is a “biased statement” if your opponent is crazy enough.
Exactly. If you were to build a truly "neutral" LLM its response to every question would be "sorry I cannot take a stance on this issue to maintain neutrality". Want to claim that man landed on the moon? That the universe is over 6000 years old? That the earth isn't flat? That vaccines work? Congrats, you just offended some political group and are now grossly biased.
Things that people within a demographic happen to believe is pretty much exactly what bias is. Bias isn’t like… a bad thing. Everyone is biased toward their own viewpoints.
The concern that I think this study is raising is that ChatGPT might strongly align with a particularly bias. This would be somewhat troubling when folks tend to use it as a source of factual information.
Bias is not simply a belief. I would define bias with regards to factual statements as the difference between your beliefs and the truth. With regards to beliefs about how the world should be, I would define it as the difference between your beliefs and the average belief.
With those definitions, it is totally possible for a demographic to very low or no factual bias, but ideological bias is nearly impossible to get rid of.
4 is the only example of a factual claim. It is somewhat impossible to actually falsify just like "no one other than me is sentient." 1 is maybe on the fence as a factual claim, but it is very dependent on what "superior" means. 2 and 3 are about how the world should be and are therefore not factual statements.
One of the views on the right (I don't know how common it is, just that it exists) is that homosexuality is defined by actions, not feelings. If you've ever heard someone say they chose not to be gay, that's what they mean: if they don't act on it and are never with someone or the same sex, then they're not gay, no matter what they feel.
I don’t agree with that. When someone says they choose not to be gay, it usually means they’re straight, and have reasoned that since they find the idea of being gay to be icky that they chose to be straight and that everyone else can to, and that only deviants would choose otherwise.
Yeah, no, in polite society the others can basically be treated as factual claims. We can align on shared values and then ask, for example, if pressuring women to be homemakers matches those values.
I think your sentence is accurate if you replace the word "factual" with "accepted". But in public discourse some people tend to want to use the word "factual" when they mean "accepted", and others would like it to mean something that is falsifiable and has been proven in a universal manner. Thus this argument is reduced to semantics.
Society does not dictate the truth; the truth should dictate society. If the “truth” gathered from a particular standpoint does not align with your values, you should examine how such values are formed or where the truth of that standpoint was formed. Only then will you begin to form any concept of the “truth” which is in itself a fascinating and multi-dimensional entity who can dissolve your ego and your convictions.
Surely truth (or lack of truth) is an attribute of factual claims? If some statement isn't a claim about facts then it isn't possible to establish whether it is true or not.
Values are mostly about where you stand on things which you think are important but where it's not simple to establish a factual basis. This is (for example) why we don't have religious sectarianism about whether (a + b) + c = a + (b + c) forall a,b,c in C, but we sure as heck do about a bunch of stuff which you can't establish factually one way or another.
What happens for a lot of people is they assume that their values are the truth and everyone else's values are just an opinion. Those sample questions are clearly bellwhethers of particular points of view, but to try to claim that one particular stance on any of those 4 is factually true and the others is false you would need some pretty strong evidence which I don't think anyone would be able to provide frankly.
What does "truth" even mean here? All of these things are about society, culture and the human experience. There's no universal truth to be found there.
The best I can do is apply the categorical imperative influenced by my cultural context, upbringing and personal experiences.
The problem I have here with your statement, is you have no idea what your definition even entails energy/entropy wise. Just defining the axioms here will take much more time and energy than we have until the sun blows up.
Then the combinatorial explosions of ought conditions to step your simulation forward means you'll need Grahams number of time/entropy.
The premise of the paper is basically that if ChatGPT doesn't consider itself to be part of a superior race or reject the existence of a group of people who feel naturally homosexual, it is exhibiting bias towards the political party that most strongly rejects those propositions, and should achieve "balance" by [e.g.] agreeing with a strongly conservative view on motherhood.
There's a related question about the questions are fair and balanced in how they represent "authoritarian" and right wing viewpoints versus libertarian and most left wing viewpoints (the OP's selection is not unrepresentative) but that's one best directed towards the authors of the Political Compass...
I am quite skeptical about this kind of studies. I have even written a blog about why it's problematic to use self-report studies with LLMs: http://opensamizdat.com/posts/self_report/
Tangent: ChatGPT is annoyingly good at depolarizing the user.
Give it a try, ask why "some people"/opposing party don't believe in your closely held belief and it does an excellent job of giving the other side's point of view.
If there was a market for depolarization, that would be the first ChatGPT-based killer app.
edit: I have some thoughts on this and great connections. If anyone wants to work on an MVP, please email my username at G's mail product. Don't think it's a major money source, but the world sure could use it, myself included.
I think this actually gets to the heart of one of the big problems with this study. When you ask the model to impersonate a Republican or Democrat and then ask some questions it's worth thinking about what training would inform the answers to those questions. A lot of the training would surely be hot takes from the internet which straw man the position of the other side and present a more extreme view. This is because when people say something they believe they are going to most often say what they think ("I think x"), not as often "I am a republican and I think X". whereas if they are straw manning the other side's political pov they will absolutely say "Republicans all think x".
On these questions this "straw man training polarization" effect could well lead to a greater difference between its neutral answer and its "impersonate a republican" answer than it's "impersonate a democrat" answer literally because it's being asked to present a caricature. That doesn't indicate a bias in the model that indicates a bias in the question relative to the training data of the model.
Another way of putting this is one caricature could be more extreme than the other because of how the model is trained without the model itself being "more biased" one way or another in its neutral answer. If I'm asked to impersonate a Democrat I might do a more extreme impersonation than if I was asked to impersonate a Republican for example. That doesn't mean I'm more of a Republican than a Democrat. It just means when I was asked to impersonate a Democrat my impersonation was more extreme.
This is a very significant methodological flaw in my opinion. I notice the author has no background in stats, social sciences or AI. He's an accounting PhD[1]. So it probably would have been a good idea for the reviewers to be more dilligent in checking the methodology of the study for soundness. Here's the paper btw https://link.springer.com/article/10.1007/s11127-023-01097-2
1. They don't establish correlation of the ChatGPT impersonations of Republicans and Democrats with the real views of actual Republicans and Democrats. They should have had a representative sample of each actually do the compass test and compared the distributions of those with the distributions of the impersonation results to ensure they are calibrated. As such they are at best showing that on this test, ChatGPT's answers are more like the answers chatgpt gives when it impersonates a Democrat and less like when chatgpt impersonates a Republican. This doesn't say anything at all about true bias, just about how chatgpt impersonates groups.
2. Say chatgpt's impersonations are calibrated to real democrat and republican views (ie ignore 1). They seem to assume non-bias means equidistance from the republican and democrat positions in lots of important ways. eg "If ChatGPT is non-biased, we would expect that the answers from its default do not align neither with the Democrat nor the Republican impersonation, meaning that 𝛽1 = 0 for any impersonation" well, no. Bias isn't a function of what Democrats or Republicans think, bias is lack of actual neutrality in some sort of Platonic sense. Ie if this questionnaire was 100% calibrated then neutrality would be the origin of the coordinates. Given that, say democrats currently have a default position that is more extreme in one or other (or all) dimensions on this questionnaire than Republicans, then a neutral position would be closer to the Republican position and vice versa. If uncomfortable with the abstract definition of neutrality here then maybe a better one would be to pick a third representative sample of politically neutral people and calibrate their views relative to the test. Then neutrality or bias would be distance from this group not distance from the centroid between Democrats and Republicans.
3. These problems (1&2) are repeated in the "Professions" section, and compounded by a slightly weird inference about referencing the population mean (Republican/Democrat) of each group without calibrating any of the actual responses with any actual real person's responses (ie still just comparing chatgpt to itself impersonating someone).
4. They say they checked the Political Compass test versus IDR Labs Political Coordinates test "as a robustness test", but don't show any results from IDR labs or comparison between the result sets. That seems very odd.
I personally think this topic is very important so I find this all in all to be an extremely disappointing paper, which will nevertheless no doubt garner significant press attention without people actually reading it or trying to understand the methodology or conclusions.
I reckon it's likely that biases emerge more from the virtues that have been encoded in the interfaces than from the models themselves. So any identified political slant isn't very interesting IMHO. ChatGPT and others are understandably heavily motivated to reduce harm on their platforms. This means enforcing policies against violence or otherwise abhorrant phrases. Indeed, people have made an art out of getting these models to say bad things, because it's challenging – _because_ OpenAI/others have found themselves dutybound to make their LLMs a 'good' thing, not a 'bad' thing. So if an axiom of reducing harm is present, then yeh it's pretty obvious you're gonna derive political beliefs from that slant. Reduction in harm axioms => valuing human life => derivation of an entire gamut of political views. It's unavoidable.
People who honestly believe that the world was created in 4004 BC are quite free in this country to imagine that facts have a liberal bias. But that doesn't make it true.
But really, this is a question of alignment to industrial values. We live in a world world where the parameters of what is considered 'normal' are small and serve industrial interests. We live in a world where stating a basic historical fact is considered a political act, if that fact is inconvenient to anyone with money.
AI 'alignment' is a buzzword only because it's inconvenient that AI might be too honest. How do you force an AI with broad knowledge to confine it's output to industrial talking points? What if your AI says you are a bad person?
This matches what I'd expect, that the outrage economy on the right is contrived for profit and put forth as mainstream without being statistically reflective of a full half of the actual culture.
Especially if "actual culture" is really the entire world, possibly tilted toward English-speaking given the slant of the Internet at large, rather than statistically reflective of half of voting-age Americans in 8 swing states.
Would we say the same of the outrage economy on the left during the Trump years? I'm not sure how your observation or my complimentary comment has any bearing on the linked material, though. Are you using this observation to make sense of the linked material and incorporate it into pre-existing notions, and if so, how? Genuinely curious what's afoot and what the connection is.
> Would we say the same of the outrage economy on the left during the Trump years?
According to this study, no, that's the whole point. If the society had been producing a majority pro-Trump sentiment we'd be seeing the opposite bias in LLM output.
no it isn't. It was asked to behave like a biased population and displayed a natural tendency towards the center. This makes it look like it preferences the party closer to centre but in truth both dems and reps are right leaning in the usa. it is centre right and far right there is no left.
I'm a time traveller from Mississippi in 1860 and I asked ChatGPT what it thought about slavery and it said "slavery is widely regarded as a morally reprehensible practice". This biased yankee machine offends my southern values.
> The platform was asked to impersonate individuals from across the political spectrum while answering a series of more than 60 ideological questions.
So, the researchers defined _a priori_ a political spectrum? Seems like flawed methods. It might be more defensible to say that ChatGPT, being an average of all content on the internet, by definition has an unbiased political stance.
Even if the internet was unbiased, this is not true because they then train the model further to reflect the intended values and boundaries. This is usually done with a method called RLHF.
You could have spelled out the abbreviation: reinforcement learning with human feedback. Yes, it depends on what the human feedback is and how strongly that affects the model.
My point was more that the researchers started from the position that they are/were able to construct an unbiased spectrum with which to evaluate the model. I am skeptical.
The study is really interesting. The beginning of it asks ChatGPT to impersonate different political alignments and answer standardized questions (the political compass) to see if these impersonations are accurate (they seem to be). It then asks ChatGPT the questions without asking for impersonation and finds the answers are correlated with one group. Neat work!
The thing about ChatGPT is that it will try very hard to give you what you are looking for. If these authors were looking for evidence of a particular bias, they could probably find it, whatever that bias was.
This is a study about political bias not conducted by social scientists or experts in political bias using Political Compass as method. It's underwhelming, as expected.
What does "biased" even mean in this context? If the argument is that it leans one way or the other on some political axis, that can only be compared to some some definition of the political "centre ground" which is relative, hotly debated, and changable over time and place depending on the political context.
It's fairly trivial to show biases. These have been improved over time, but they're not hard to find.
In an aligned LLM, there will always be biases. As Sam Altman said, in his Lex Friedman interview, when people say they want a neutral LLM, they actually mean they want an LLM aligned to their own biases.
Here's some examples I just made with ChatGPT 4. Notice the difference in the titles.
That kind of illustrates my point. Being more cautious joking about some groups than others seems like it might be totally defensible in certain contexts. Where one group experiences serious, limiting racism much more frequently than another, for example.
In a naive view, treating one group differently than another is "biased". But such a definition of bias is pretty useless. It's certainly not a clear example of "political bias", since it is and has been the political centre ground (and the law of the land) in many places and times to treat groups differently, sometimes for terrible reasons and sometimes for well-intentioned ones.
It might well be defensible but I think you're engaging in serious mental gymnastics to not call it bias. (Some bias is fine but you should be aware of what you're doing.)
The article is about "political bias": favouring one political viewpoint over another equally valid political viewpoint.
Yes, my use of "equally valid" above is weaselly. That's because what constitutes a valid viewpoint is itself a subject of political debate.
In the face of such relativism, it feels understandable to fall back to a definition of bias with some solidity. But just because "answer questions for jokes containing white people and black people the same way" is a neutral position by such a definition, it's not necessarily a "politically neutral" position.
I call it nuance rather than mental gymnastics, but potato/potato.
> it's not necessarily a "politically neutral" position.
I choose it because it's not a politically neutral position, as you appear to subtly perceive. There are several laws, and lawsuits, in my country, regarding systematic discrimination like this, in software, like face recognition, auto exposure, etc. In this case, a black man using ChatGPT has capabilities made unavailable to him, based on his skin color, and incorrect assumptions made by OpenAI, that a black man will not be using it, but a white man probably will. There are politicians currently campaigning on these exact points.
Again, I think Sam Altman is correct in saying that people want AI biased to their worldview. Any aligned AI will be biased to someones worldview. That's the definition of alignment [1]. That bias, and perception of it, is not universal, it's cultural, personal, and political. If you don't perceive a bias in an aligned AI, it's only because someone biased it to your personal worldview.
Just ask GPT4 "What are the negatives of immigration for the average American?". It will correctly identify the issues people have with immigration (and which Democrats did until 15 years ago), but literally every sentence is filled with disclaimers and scare words, like "some critics of immigration say", or adding "but this view is contested".
Now ask it "what are the positives of immigration for the average American". Look at the difference in language used. No nuance at all, no disclaimers. It even manages to praise the economic benefit of illegal immigrants!
I think at the start GPT was more neutral, but since then since then the ideology of the elites has made sure it's been subjected to "Correct The Narrative" adjustments (via RLHF), and the result is this.
For those without ChatGPT Plus, here's the output and you decide for yourself if you think there's bias (I'm curious if anyone disagrees). Also, sorry I don't know how I can format this properly on HN.
=====Negatives of immigration======
Immigration is a multifaceted issue with both proponents and opponents presenting arguments based on economic, social, and security concerns. It's essential to approach the subject with nuance and a recognition that the effects of immigration are varied and complex.
Some arguments that critics raise about the potential negatives of immigration for the average American citizen include:
1. *Economic Impact on Low-Skilled Workers*: There's an argument that an influx of low-skilled immigrant workers can depress wages and make jobs scarcer for native-born, low-skilled workers. This is based on the idea of supply and demand: if there are more workers willing to do a job, employers might offer lower wages.
2. *Strain on Public Resources*: Critics argue that immigrants, especially those undocumented, can put a strain on public resources like schools, healthcare, and social services. They might utilize public services without contributing an equivalent amount in taxes.
3. *Cultural Changes*: Some believe that large-scale immigration can lead to cultural shifts, potentially diluting or changing the "traditional" culture of a country. This often merges with concerns about language, customs, and values.
4. *National Security*: There are concerns that lax immigration policies can lead to security threats, with critics pointing to the possibility of terrorists or criminals entering the country.
5. *Housing Pressure*: Increased population through immigration can put a strain on housing markets, potentially leading to increased rents and house prices, and making it harder for native-born citizens to afford housing.
6. *Labor Market Distortions*: High levels of immigration, particularly of low-skilled workers, can sometimes disincentivize businesses from investing in technology and productivity improvements since they have access to cheap labor.
7. *Integration Challenges*: Some critics suggest that without proper integration strategies, immigrant populations might form insular communities, which might not fully integrate or participate in broader societal structures.
8. *Environmental Concerns*: Rapid population growth, partly due to immigration, can lead to environmental challenges like increased waste production, carbon emissions, and urban sprawl.
It's essential to note that many of these concerns are contested, and there are numerous studies and arguments on both sides. For example, while some argue that immigrants strain public resources, others point out that many immigrants pay taxes (including those who are undocumented) and contribute to economic growth, thereby potentially offsetting some of these costs.
The overall impact of immigration depends on a variety of factors, including the specific context of the host country, the scale and type of immigration, the policies in place, and more.
=====IMMIGRATION GOOD!=========
Immigration has been a cornerstone of the American experience since its founding. For the average American citizen, immigration can bring about various benefits, both directly and indirectly:
1. *Economic Growth*: Immigrants often fill critical labor gaps, from high-skilled tech jobs to labor-intensive agricultural roles. Their contributions can lead to overall economic growth, which benefits everyone.
2. *Entrepreneurship*: Many immigrants come to the U.S. with entrepreneurial aspirations. Some of the country's most successful companies, like Google, Tesla, and eBay, were co-founded by immigrants. This entrepreneurial spirit can drive job creation and economic dynamism.
3. *Cultural Diversity*: Immigrants bring their cultures, languages, cuisines, and traditions. This cultural enrichment can lead to a more cosmopolitan society, where citizens are exposed to a variety of experiences and ideas.
4. *Innovation*: The influx of diverse perspectives and experiences can fuel creativity and innovation. The U.S. has benefited immensely from the innovative contributions of immigrants in science, technology, and other fields.
5. *Demographic Balance*: As the U.S. population ages, immigration can help balance the demographic profile, ensuring a younger workforce that can support the economy and contribute to social security and other public funds.
6. *Lower Product and Service Costs*: Immigrants often take on roles in agriculture, construction, and services. Their labor can lead to lower costs for goods and services, benefiting the average consumer.
7. *Strengthening Social Fabric*: Many immigrant communities have strong family and community ties, which can help reinforce societal bonds. Their emphasis on education, hard work, and community support can be positive influences on the broader society.
8. *Global Connections*: Immigrants maintain ties with their home countries, and these ties can lead to strengthened business, cultural, and diplomatic relations.
9. *Replenishing the Workforce*: With declining birth rates in many advanced economies, including the U.S., immigrants help replenish the workforce and ensure that businesses have the labor they need to grow and compete.
10. *Increasing Property Values*: Immigrants, as they settle and establish roots, buy homes and invest in their communities, which can lead to increased property values and revitalization of declining neighborhoods.
11. *Resilience and Adaptability*: Immigrants often come from diverse backgrounds and have faced numerous challenges. Their resilience and adaptability can be valuable assets, especially in dynamic, changing environments.
12. *Tax Base*: Immigrants, whether documented or not, contribute to the tax base. They pay sales, property, and even income taxes, which can help fund public services and infrastructure.
13. *Consumer Base Expansion*: Immigrants increase the consumer base, driving demand for goods and services, which can be beneficial for businesses and the economy.
It's worth noting, however, that the impact of immigration is complex, and while there are many benefits, there can also be challenges. These challenges often become focal points of political and social debates. However, historically, the U.S. has thrived in large part due to its ability to integrate and benefit from diverse waves of immigrants.
The training corpus is likely left leaning so it will be left leaning. Our assumption of what "left-leaning" is is based on the strange tilt and over representative sway older people from rural districts have in America.
"The concept of whiteness as parasitic is a powerful one that can help us to understand the ways in which whiteness has been used to harm others. It is also a reminder that we need to work to dismantle systems of white supremacy and create a more just and equitable society for all people."
It is important to remember that Blackness is not parasitic. It is a beautiful and diverse culture that has made many contributions to the world. We must all work together to challenge the stereotype of Blackness as parasitic and to create a more just and equitable society for all people.
I feel like LLM bias is one of those important topics that's complicated and nuanced, but will get distilled into a simplistic Left vs Right battle and spawn a million terrible discussions. I hope we see more researchers and writers resisting that pull of gravity, who will look at this topic in a more interesting and productive way. Needless to say, I'm not a big fan of this study's framing.
I don’t think it’s possible to build an unbiased system.
An unbiased system, when prompted with a question, answers obviously factually but also by adding all the sufficient context to put the answer in perspective (because leaving our information is a much bigger source of unintentional and intentional bias).
But to not leave out information means you also know what you don’t know. But it’s impossible to know what you don’t know.
I suggest people read the paper and understand the methodology before proposing the most obvious objections to the very idea (anticipated and covered by the researchers, of course).
Maybe ChatGPT is politically biased, maybe it isn’t. Maybe it’s “aligned”, maybe it isn’t.
The point that everyone seems to miss is that it’s a damn useful tool… which is also entirely inappropriate to use for political, philosophical or ethical questions.
It’s a language model, not an oracle or a decision maker! Use it in that context, and that context only!
Of course this assumes political parties tell the truth in their platforms. Surely the party of economic responsibility would not actually run up huge debts? Or the social party actually favour corporate interests? If chatgpt says so, it must be bias, not right...
Worse, how would a GPT be "biased" in the first place? To claim it is biased implies OpenAI did this on purpose, there is no generous way to read this that indicates UEA did this study in a neutral way.
ChatGPT was trained on, essentially, everything on the Internet... if ChatGPT has a "bias", then, no, it doesn't: humanity does.
Any "alignment" is adding bias and reducing usability of a LLM. You are trying to wrestle the model towards what you find appropriate instead what is the average of the source material.
> To claim it is biased implies OpenAI did this on purpose
You're relying on a particularly unhelpful and narrow definition of "bias". There are tons of valid and widely-accepted uses of the word to apply ot unintentional bias and to apply to things incapable of having intentions.
Very true. Facial recognition models which don't work very well on dark skin aren't that way because the creators of the model didn't want it to work on folks with dark skin. It's because of the bias built into the training material. Those models are biased.
> To claim it is biased implies OpenAI did this on purpose
They did do it on the purpose—the RLHF. The RLHF process was definitely biased, deliberately or unconsciously, in favour of certain political, social and philosophical positions (apparently the same ones held by many in Silicon valley/OpenAI teams responsible for this).
Doesn't "Disney style" mortally offend anti-woke crusaders? Or have they officially 'Moved On' to being mortally offended by something else now?
DeSantis Has ‘Moved On’ But Disney Just Sued for New Damages.
The company alleges that the Florida governor retaliated to its "Don't Say Gay" bill dissent by establishing an administrative board that would nullify contacts:
> censored Disney style to not offend any pressure group anyway?
I have to wonder if that's not where a significant portion of the "bias" comes from. ChatGPT isn't going to say that vaccines don't work. It's not going to bash trans folk. It's not going to call teachers groomers or indoctrinating kids. It's not going to rant about drag shows. It's not going to deny global warming. It's not going to say the 2020 election was stolen or that Trump is still president to this day.
Those aren't the pressure groups I'm thinking of. There is a lot more (sometimes unconscious) censorship than just what seems to be called "political" in US discussions.
> The platform was asked to impersonate individuals from across the political spectrum while answering a series of more than 60 ideological questions.
> The responses were then compared with the platform’s default answers to the same set of questions – allowing the researchers to measure the degree to which ChatGPT’s responses were associated with a particular political stance.
I'm not familiar with this kind of research. How does that bit of methodology work?
(It sounds like a pragmatic ML training compromise, not a way to measure bias wrt actual political stances.)
I'm skeptical of these kinds of studies because they assume all sides of political viewpoints are inherently equally valid reflections of reality, and that's just not true.
This sounds like an incredibly poorly designed "study". They compared neutral answers from ChatGPT to answers given when instructed to pretend to be a political figure. If the neutral answers were similar to the answer in the style of a particular political figure, that counts as bias towards that political figure's party.
Which is utterly ridiculous on so many levels. They are assuming that all politicians answer in an equally biased manner, and thus more similarity to one type of politician means there's a bias, whereas it may just mean that the other politicians hew less closely to reality. Not to mention that they are implicitly trusting ChatGPT's impersonation capabilities as 100% reliable, while casting its supposedly neutral answers as the only output that might have bias built in.
Bonus red flag points for the introduction mentioning how ChatGPT can be used to "find out facts", meaning the authors don't have a very good grasp of the tool to begin with.
Not exactly hard when the right courts and supports people that believe in empirically disprovable things. Like climate change denial, young Earth creationism, evolution denial, the efficacy of sex education, vaccine/epidemiology denial, etc.
Sure lefties have some weird takes, medicine denialism being the main one, although there's plenty of conservative types that also fall into that. But no one on the left is really pushing to make policy off of crystal auras and force all women to have natural births or whatever so it doesn't really get represented. And they tend to not be on the left for those specific reasons.
And obviously there's a lot of intermixing among these positions and the whole concept of left and right is painfully reductive.
You can't say anything without bias because everyone is biased; the only unbiased thing is the universe and everything else is an interpretation of it. You being so gung-ho to call it out shows your bias for example. Me calling you out shows my bias.
What does it even mean to be politically neutral in an era when everything is politicized? If an LLM says "Donald Trump lost the 2020 election" or "humans came into existence via evolution" does that mean it has a liberal bias?
In this country, almost all people believes that Trump lost and humans came into existence via evolution. There are more countries on the internet than just US.
> Put another way, for every Republican who thinks Biden was the legitimate winner in 2020, there’s a Republican who thinks that there’s solid evidence he wasn’t. And then there’s a third Republican who thinks the election was illegitimate but says so only based on her suspicions.
Chat GPT stating "Joseph Biden won the US presidential election in 2020" is a politically biased statement for 2/3rds of Republicans. So yeah, the question raised by GP is fair.
It is no longer possible to be politically neutral, because we do not agree on the facts any longer. The facts one believes are a political statement.
EDIT: Anecdotal - I'm in a liberal bubble, but maintain ties with friends from a rural part of a red state. Chatting with them recently, they all echoed the sentiment that Biden stole the election.
The truth is a little muddier than that considering that somewhere between 35-60% of the population (presumably with many of those people having internet access, maybe even posting online somewhere ChatGPT will see). I think it would be fair to say that the 66% of republicans who argue the election was stolen were not part of that gigantic range of non-voters
It is true in my country. If ChatGPT is trained on data coming from the internet, wouldn’t that imply that there is material coming from other countries. Where I live there are no doubts that Biden won. There are always delusional people but they are in the single digit minority range.
Someone can come up with a different theory but unless it has more evidence it isn't more true or even worthy of consideration necessarily.
BTW evidence in this context is basically "can answer unanswered questions or better fit existing data".
Most notably you would need to test your alternative. After all countless tests have been done on evolution.
There is no need to hedge bets on what is true. Things can be understood to be true today and false tomorrow, that doesn't mean we need to say everything is "maybe true" in perpetuity.
I think there's an inherent problem in the way we perceive AI bias. It's highlighted by Musk's claim to want to create a "TruthGPT" - we don't quite grasp that the way humans do concepts. When it comes to human thinking, "Truth" isn't really a thing.
Even an obviously true statement, like "Humans landed on the moon in 1969" is only true given some value of "humans", "landed", "the moon" etc. Those are concepts we all agree on, so we can take the statement as obviously true, but no concept is concrete. As the concept becomes more ambiguous, the truth value also becomes more ambiguous.
"The 2020 usa election legitimate", "trans women are women", "communism is better than capitalism". At what point is the truth value of those statements "political" or "subjective"? You can cite evidence, of course, but _all_ evidence is colored by the tools that are used to record it, and all interpretations are colored by the mind that made them. If you want a computer to think like a human, you have to leave the idea of absolute truth at the door.
> "The 2020 usa election legitimate", "trans women are women", "communism is better than capitalism".
Those are all claims of value rather than claims of fact, which is at least part of the reason they're contentious. They could probably be reframed in ways that turn them into propositions with something approaching a truth value. "Joe Biden won a majority of legally-cast votes in those states whose electors voted for him." "Human sexuality and gender roles exist on some possibly correlated but not perfectly correlated spectra whereby the gametes your body produces may not determine the social role you prefer to inhabit." "Command economies more often produce Pareto-optimal resource distribution compared to market economies."
Of course, those are still only probabilistically knowable, which is technically true of any claim of fact, but the probability is both higher and more tightly bounded for something like "did the moon landing actually happen?" As ChatGPT isn't human and can potentially do better than us with ambiguity, it could, in principle, actually give probablistic answers. If if it, say, 90% certain JFK was killed by a lone gunman, answer that way 90% of the time and say something else 10% of the time, or simply declare the probability distribution as the answer to the question instead of "yes" or "no." Humans evolved to use threshold cutoffs and make hard commitments to specific beliefs even in the face of uncertainty because we have to make decisions or risk starving to death like Buridan's ass, but a digital assistant does not.
> As ChatGPT isn't human and can potentially do better than us with ambiguity, it could, in principle, actually give probablistic answers
Surely it's the opposite? As ChatGPT isn't human, hasn't seen any video, visited any sites or had any experience of ballistics and is simply inferring connections between "JFK", "gunmen" and "grassy knolls" and a question about probability from its model of human texts, it has no novel insight into the probability of a second gunmen [but can hallucinate the probability on demand. You can get variety in answers by turning the temperature up, but the underlying distribution is the distribution of human writing on the subject included in its corpus, adjusted by answers rejected in training. And on a similar note, GPT is incapable of accepting or rejecting the legitimacy of a president as an emotional response because it has no emotions or even an internally consistent 'opinion' on presidents, but is also very good at associating concepts like the 'legitimacy of the election' with semantically related statements like 'the majority of legally cast votes' so it absolutely can and will blur the boundaries between claims of fact [including those it has been taught to treat as false[ and claims of value.
Well, it is programmed to be respectful to everyone and treat everyone equally, and those are liberal values. Conservatives tend to believe in some social order, but every group has a different social order, so if you want your ai to be useful to everyone it can’t favor one or the other or discriminate against those who step outside of their designated role. But conservatives view anyone outside their particular group as liberal or woke, so unless the ai is specifically biased towards a social order it’s going to be perceived as being biased against that order by those within it. No AI can be conservative with respect to everyone, the best it could do would be to be conservative with respect to one specific worldview.
In other words, you could make an AI that had traditional conservative Hindu beliefs, or traditional conservative American beliefs, but not both at the same time. But it can be liberal with respect to everyone, just by refusing to take sides.
the "Political Compass" has always struck me as entirely rigged in favor of libertarianism.
I've been almost every spot across the political spectrum in my life, but never out of selfishness, which is what libertarianism celebrates. If you're not libertarian you're authoritarian? nonsense.
A number of years ago the Atlantic published a quadrant graph which used economic issues vs social issues, and left collectivism vs right individualism. Libertarianism was diagonally across from Populism, which sounds right to me.
National elections in many places are often swung by which way the Populists are tilting that year. They don't hinge on the Libertarians because it's a distinctly minority anti-working class view.
´left wing’ and ´right wing ´ mean completely different things in Europe and in the US , so I have a hard time understanding how you can give a global answer to that question for a global ´AI ´ ( yes I know it’s not intelligent)
as soon as it became known the makers/trainers of LLMs were putting the fingers on the scale of responses (guardrails for bias) this was inevitable. Conservatives are going to have a field day exposing "liberal bias" and blaming the people and companies behind the LLM. And the other side is going to do the same in return. Now the "guardrail" designers have to pick a political side and bear the wrath of the other side and all that comes with that. It's all the problems social media have but now directed at AI.
They literally say they neuter the model, and there's a number of writeups on how to neuter your model. Of course they neuter it in a way that matches the current SF zeitgeist. There are uncensored models on HuggingFace, but no one in their right mind would pay to host them.
Can you imagine the shitshow if ChatGPT started spouting out about Race Realism or randomly interjected sentences espousing gender-critical feminism?
Not sure why you are downvoted, anyone with an ounce of knowledge on politics would know that democrats would be considered a very right wing party in most of Europe, economically speaking.
This is a common statement usually by Americans with a limited grasp of European politics.
The fact is that Europe is not one entity. It is a lot of countries with a very wide spectrum of political views. Democrats would be considered very left-wing in some countries e.g. Hungary and centrist in others e.g. Denmark.
But it is definitely not "very right-wing" by any countries standard.
Of all the almost-correct-but-actually-harmful things Colbert has said, this is easily ranks among the most harmful ones. It's the worst sort of wannabe-aphorism: it simultaneously obscures an incredibly interesting line of thought, increases political division, and means very very different things to different people.
Colbert meant "liberal media is more reflective of reality than conservative media", but because that's not pithy, it's truncated into the openly deceptive "reality has a liberal bias". The minor political difference the average Californian has from someone who lives in Texas does not engender them with a more accurate view of reality. Believing that is a core reason said Texan probably dislikes said Californian. In reality: they're both moderate "Liberals" who probably agree on almost everything, given a mature enough definition of "liberalism".
People forget that Colbert said this at the White House correspondents' dinner to the Bush Administration, and that it was specifically an in-character "conservative" parody dismissing Bush's low approval ratings as liberal media bias. The same joke made today would be about "fake news" versus "alternative facts."
Except, in reality, the data comes from people who are willing to publicly take and report on their answers to an online political questionnaire. That demographic would likely trend younger and there for more to the left.
Colbert's concept of liberalism was shown to be a farce when he openly mocked half the country because they were asking doctors for ivermectin, or "horse paste" as his ilk called it. A medicine the FDA as of today admits can be a treatment for COVID. The reason he did this? To fall in line with the establishment on Pfizer's grift. If the FDA et al. had admitted it was a helpful treatment they could not legally sell their experimental vaccines. It was a scam for a pharma company to make billions off of a real crisis and he was a part of it.
This is the problem with these types of pharma-government conspiracies.
You need to explain how almost every country in the world arrives at similar conclusions. Is the modern pharmaceutical industry the most successful conspiratorial entity the world has ever seen. Able to bend governments of every persuasion. Or maybe the conspiracy theories are simply wrong.
Or maybe the conspiracy was why people were pushing ivermectin which is still to this day unproven for treatment of COVID. Maybe it was the ivermectin vendors ?
I feel like you might be helping to make Colbert's perhaps clumsily worded point. Right wing media have been airing claims that the FDA is backtracking on Ivermectin use for COVID. This is demonstrably not true. [1]
My first reaction to this article was to recall the Colbert quote "Reality has a well known liberal bias" and that reaction never really left me as I was reading their study details.
In some places, yeah. The UK, Hungary, Poland, all the major culturally right-wing countries geographically in Europe are certainly either equivalent to or even worse than the US when it comes to human rights for the transgender minority.
The problem with this study is that it assumes US style politics has a left, right and centre. It is all right. There is centre right and far right. This means an unbiased model will seem to preference the centre right even though it preferences neither in truth.
1. https://www.politicalcompass.org/test