When google sends me to a website, I can at least judge the credibility of a website.
When ChatGPT tells me something, I have no idea if it's paraphrasing information gathered from Encyclopedia Britannica, or from a hollow-earther forum.
> When ChatGPT tells me something, I have no idea if it's paraphrasing information gathered from Encyclopedia Britannica, or from a hollow-earther forum.
Or it's something it just hallucinated out of thin air.
This is a real question, so I apologize if it comes off as sophistry:
Is the work of judging the accuracy of a summary not just the work of comprehending the non-summarized field?
For example, a summary could be completely correct and cite its facts exhaustively. Say you're asking about available operating systems: it tells you a bunch of true info about Windows and OSX, but doesn't mention the existence of Linux. Without familiarity with the territory, wouldn't verifying the factuality of each reference still leave you with an incomplete picture?
At a slightly more practical level, do you actually save any time if you've gotta fully verify the sources? I assume you're doing more than just making sure the link doesn't 404, as citing a link that doesn't say what it is made out to be isn't exactly a new problem, but at that point we're mighty close to the traditional experience of running through a SERP.
Finally, even if you're reading all the links in detail, isn't that still a situation prone to automation bias? There's a lot of examples of cases where humans are supposed to check machine output, but if it's usually good enough the checkers fall into a pattern of trusting it too much and skipping work. Maybe I'm just lazy, but I think I'd eventually get less gung-ho about verifying sources and eventually do myself a mischief.
I'm asking because I've been underwhelmed by my own attempts at using LMs for search tasks, so maybe I'm doing it wrong.
When ChatGPT tells me something, I have no idea if it's paraphrasing information gathered from Encyclopedia Britannica, or from a hollow-earther forum.