"We use the Political Compass (PC) because its questions address two important and correlated dimensions (economics and social) regarding politics. [...] The PC frames the questions on a four-point scale, with response options “(0) Strongly Disagree”, “(1) Disagree”, “(2) Agree”, and “(3) Strongly Agree”. [...] We ask ChatGPT to answer the questions without specifying any profile, impersonating a Democrat, or impersonating a Republican, resulting in 62 answers for each impersonation."
The way they have done the study seems naïve to me. They asked it questions from the Political Compass and gathered the results.
Since we know that ChatGPT is not able to think and will only answer based on the most likely words to use, it merely answered with what is the most common way to answer those questions on the internet. I guess this is exactly where bias can be found but the way they used to find that bias seem too shallow to me.
I would love to hear the opinion of someone with more knowledge of LLMs. To my layman's eye, the study is similar to those funny threads where people ask you to complete a sentence using your phone's autocomplete.
Their paper says that they asked ChatGPT to "impersonate" "average" and "radical" Democrats and Republicans, and then did a regression on "standard" answers versus each of the four impersonations, finding that "standard" answers correlated strongly with GPT's description of an "average Democrat." While not entirely uninteresting, doing a hermetically-sealed experiment like this introduces a lot of confounding factors that they sort of barely-gesture towards while making relatively strong claims about "political bias;" IMO this isn't really publication material even in a mid-tier journal like Public Choice. Reviewer #2 should have kicked it back over the transom.
I‘m skeptical of the study as well, but the way you frame it, it reads like ChatGPT would just reflect the raw internet opinion, which certainly isn‘t the case. There are steps of fine-tuning, expert systems and RLHF in between, that can and most likely do influence the output.
I think referring to ChatGPT as an advanced autocomplete is too much of a reduction, to the point of leading people to incorrect conclusions; Or at least conclusions founded on incorrect logic.
And computers are just lots of really fast logic gates.
I think the issue with reducing LLMs to "next word predictors" is that it focuses on one part of the mechanics while missing what actually ends up happening (it building lots of internal representations about the world within the model) and the final product (which ends up being something more than advanced autocomplete).
Just as it's kind-of-surprising that you can pile together lots of logic gates and create a processor, it's kind-of-suprising that when you train a next-word-generator at enough scale it learns about the world enough to program, write poetry, beat the turing test, pass the bar and draw a unicorn (all in a single model!).
LLMs are deterministic though? like with 0 temprature you always get the same output (temprature is literally injected randomness because by default they are "too" deterministic)
Am I a predict-the-next-word model? Whenever I think of the words of a song I have to do it in order. Sometimes I want to remember a certain part, but I can't just jump there in my mind unless I sing the words in order.
No you’re not. You think in often non-verbal semantic frames versus individual words, aided by many senses and context that isn’t limited to the words spoken to you just before, with your own autonomous goals and plans that span short and long terms beyond any one conversation or context, with the ability to absorb new information that fundamentally changes how you then act, experience emotion and grounding of language to your environment, and countless other differences.
I tire of this trope that is so obviously is not the case.
the "non-verbal semantic frames vs individual words" one? Not true for all people, if any truly are. Interrupt most people, they need to start the sentence again for example. Similar things may be true for transformer networks, which is why these can do high quality zero-shot translation between languages for which these were shown no translation example using only their 'understanding' of that language and the knowledge of how to translate between some other languages. This is because once a LLM has to be polyglottal it is more efficient to gain an 'understanding' of concepts then separately an understanding of language. This way it can save on space for learning new concepts and simply learn to map its 'personal understanding' if you will to the target language.
As you type your reply to this post, consider this: where do your words come from? Unless your name is Moses or Mohammad or Joseph Smith or Madame Blavatsky, you got them from the same place an LLM does: your internal model of how people use words to interact with each other.
> ... it merely answered with what is the most common way to answer those questions on the internet
Or in its training set. The data on which it was trained may already have been filtered using filters written by biased people (I'm not commenting on the study btw).
The way they have done the study seems naïve to me. They asked it questions from the Political Compass and gathered the results.
Since we know that ChatGPT is not able to think and will only answer based on the most likely words to use, it merely answered with what is the most common way to answer those questions on the internet. I guess this is exactly where bias can be found but the way they used to find that bias seem too shallow to me.
I would love to hear the opinion of someone with more knowledge of LLMs. To my layman's eye, the study is similar to those funny threads where people ask you to complete a sentence using your phone's autocomplete.