Do you "demand money" when you do the job you were contracted to do? You make it sound like they're money grubbing and greedy for expecting payment for their services.
Whatever it may sound like, the illustrator didn't just do something nice and help the fellow human (client). If you say "but the illustrator needs the money to support himself", maybe OP also needs the money, maybe his grandparents are ill and have medical bills.
Yes? The illustrator could have decided not to pay for a prominent ad, and indeed they did, and they did not get a prominent ad. This is business, why are we acting like artists are some ineffable fairy dust? (I know why: politics.) If an ad brings more business, why not pay? It's not a friend, it's a business relationship.
That's true in the same sense as "There is nothing qualitatively different about erroneous vs non-erroneous output" for a dog vs. cat image classifier.
I think that’s a bad example, as classifiers tend to output a floating point number and you use some threshold/activation function to collapse the classifier into a particular state. In that sense there is nothing qualitatively different when the classifier outputs 0.85 vs 0.87.
If it makes a false claim, then it's an error. If it says the test was passed or a class was implemented but it was not, then it makes a false factual statement.
I'd say a hallucination (very misleading word) or confabulation or "making shit up" happens when an LLM uses factual / evidential language purely based on local statistical expectations of the text, instead of it drawing from actual evidence in its context pointing to it.
This is murkier in the case of general knowledge questions, like when and where was some famous person born. It may then be a spectrum from fully making something up based on how the name sounds, all the way to confidently retrieving it from its weights correctly. In between, we can get hallucinations. But newer models are taught to use Web Search when unsure, and it works pretty well, though not perfectly. I don't see any fundamental limit here. It's just not perfect. Trying to solve "the hallucination problem" is basically like saying "our dog vs. cat classifier is pretty good already with its 99% accuracy, now all we need to do is the tiny little task of eliminating the 1% error, and we will be golden". Like, no shit, there is some error yes. People are working to reduce it. It will never be absolutely 100%. It's not an insight to say we should remove hallucinations.
Intuitively, something about that version bothers me... I think it's because the choice of a clearer "erroneous" has dropped the fundamental framing problem from "hallucination": The false implication that "true" (anthropomorphic) sight/thought usually happens.
To fold that back in (and ham-it-up a bit) how about:
> There is nothing qualitatively different when Pet Classifier <ironic-quote>maliciously misreports</ironic-quote> your cat as a dog, compared to when it works <ironic-quote>honestly</ironic-quote>.
These models are outputting language that we can decode as factual claims and we can check those, and we can say it made a mistake / error or that it answered correctly. This doesn't require any squishy assertions to how it "feels" while doing it or anything like that. It's an externally observable thing.
I agree that hallucination isn't some kind of "different" operation than "normal". It's not like when a train derails and you can point to it. It just operates as normal and sometimes that yields correct factual outputs, sometimes not. You don't have to metaphysically ascribe any kind of intent to it.
I'm not sure how people conceptualize these things who weren't doing classical machine learning before all this. To me, "hallucination" is shorthand, and we know it's not like humans on drugs or something. It was used in the literature also for any kind of generative imputing of missing information from a learned prior. For example in image inpainting a GAN "hallucinates" the missing part of the image. This terminology was already used in the 2010s and probably earlier. Or in image colorization of grayscale photos, the model "hallucinates" the color information.
Then the word escaped into the mainstream and people have weird connotations about it.
> To me, "hallucination" is shorthand [...] Then the word escaped into the mainstream
One of my bugbears is when people abuse the term "Ponzi Scheme" to refer to literally anything the think is unsustainable. (As opposed to something that, at a minimum, requires someone telling factual-lies about assets.) Kind of like if folks started calling every kind of software error a "Buffer Overflow."
I think the colloquially Ponzi Scheme is a synonym for any pyramid scheme, where the people in the smaller upper layers rely on an ever growing sequence of lower layers entering to receive their payouts, and it works as long as there are newer people to bring in but at some point the recruitment pipeline is exhausted and the bottom layer ends up losing their entire investment and the higher you are in the pyramid the better you end up.
This is quite a specific sense, and isn't equal to just any kind of "unsustainable" system. Things can be unsustainable for reasons other than being a pyramid scheme relying on rapidly growing numbers of people being brought in by the ones already in the system.
(Going off topic, I know.) It's a less-egregious substitution, but Ponzi != Pyramid in some important ways. If they were the same, we wouldn't have needed to invent different terms. :)
1. Actual Fraud
* A Ponzi Scheme requires lying to people about what assets currently exist, or how much of the asset is reserved as "theirs".
* A Pyramid Scheme just asks people to collectively speculate (without guarantee) that tangible profits will arrive.
2. Layers
* A Ponzi Scheme can be one criminal and an indefinite number of unwitting victims. [0]
* A Pyramid scheme (like the structure) requires *growing interior layers* of participants that blur the line between victimizer and victim.
[0] Insofar as some Ponzi Scheme targets do not suffer losses, that's not because they were necessarily complicit, but because the criminal strategically chose to let some marks exit in order to maintain the illusion.
> 2. the point of doing mathematics is to be “productive” in the first place. This is silly. Many people get into mathematics because of the beauty of understanding, for example.
Are they independently wealthy? Or do they have a deal with their local supermarket that they can take food for free?
What economic system do you recommend instead? Marxist-Leninist communism? Anarchism? Feudalism? I guess none of these will be fitting for what's coming...
It was shown quite some time ago that training LLMs on programming tasks improves their logical reasoning skills also in other natural language domains. So I could see math also being a training gym for AI even if the final use case is not directly math-related. Having to solve math problems efficiently can build in skills that come handy in all kinds of more everyday tasks or science and engineering.
It’s also very useful signal that the reasoning trace is leading to solving open problems - you can be certain that you’re not landing somewhere inside the training data.
I think the entire thing here is that these solutions _are_ inherently interpolations of existing work in the field. That's the "super power" that LLMs have. To interpolate mass amounts of multi-dimensional data.
This depends on some handwavy use of the term "interpolation", not the mathematical definition. Mathematically interpolation usually means that the query point is in the convex hull of the data points, and that almost never happens in high-dimensional spaces.
See:
Learning in High Dimension Always Amounts to Extrapolation
Randall Balestriero, Jerome Pesenti, Yann LeCun
https://arxiv.org/abs/2110.09485
I guess you mean by "interpolation" that it's some kind of nonlinear combination of the training data, but that's an almost vacuous statement. Any input-output relationship has to be so by definition.
Or perhaps you mean that interpolation is when the test input comes from the same distribution as the training input (though this is not technically the meaning of "interpolation"). But this is also quite difficult to pin down.
AI capabilities keep getting bumpier and bumpier. There's no reason to believe it won't stall out on a front that it is already quite bad at (correctly formalizing in human language)
Because making AI better is orthogonal to making human lives better. We are quite literally racing towards artificial superintelligence without any serious plan or preparation for the societal impacts that will have.
Did people prepare for or ever fully foresaw the downstream impact of any other major technological revolution, such as farming or the industrial revolutions or the Internet?
Has there been any other major technological revolution that underwent exponential recursive self improvement?
If AI development stopped right now, we would see a technological revolution at a scale similar to the Internet. AI development has not slowed down, let alone stopped. This technology demands far more planning than we are allocating.
AI inherently has misaligned incentives (as do the companies developing them). I don't trust them to outline the consequences of something like this and to develop an appropriate plan, and neither should you. I believe in humans, but we need time.
I don't think that is the thing that makes AI better. In the metaphor the idea is that someone scored a goal and then people retroactively shift the goalpost to say it doesn't count as a goal.
That's an uphill battle. People will hand wave that the s curve has flattened, and think it will stay at current levels. They claimed this confidently year after year over the last few years.
Other people have claimed confidently year after year over the last few years that the latest model was AGI - or that the singularity would arrive in a few months anyway. All the battles are uphill when you’re at a local miminum, I guess.
AGI has no goalpost-shifting-resistant definition anyway. In my estimation, the current tech is AGI by how I understood the term when it was introduced (at a time when AI as a label got adopted for things like image classifiers and machine translators and people needed a label for something that can work across tasks with a pure natural language interface telling it what to do). (And no, sentience, consciousness, inner experience, some amorphous "real understanding" etc was not part of the definition) We are in the scifi future just got used to it. Or haven't realized it yet because human systems have inertia and we could some oracle philosophers stone that knows the answer to every question in a nanosecond and we'd still lag 10 years in how to fit it into existing work flows and getting wet ink signatures onto paper authorizing it's use in the bureaucracy.
This has already been the case in AI/ML and computer vision papers via flag planting papers. Have an idea, super quickly publish a hasty work based on it that doest actually work, methodological and eval issues, engineering terrible, slow, bad results etc. But it was the first so now your concurrent work that was much better evaluated, better implemented, etc is suddenly worthless and unpublishable.
reply