As someone else said, the problem is ChatGPT lies straight to your face, whereas at least Google's answers are based on structured data by someone. It's traceable, whereas I would not trust ChatGPT ever to tell me the correct temperature to cook a steak.
If it's citing all it's sources, it's not ChatGPT, it's a natural language interface around search, which Google has been doing for years including with voice...
"Get answers instead of being overwhelmed by options. Bing looks at search results across the web and summarizes responses to your specific questions and needs."
There's summarised cited searches and then there is the more ChatGPT option which is also available.
Presumably they have bolted on something that cites sources. This doesn't mean that the things ChatGPT says are true, just that you can check some sources that are plausibly relevant.
Traditional search practically forces you to read the sources directly. With ChatGPT, people will just assume its citation is correct due to laziness.
I've seen this with many human published articles on the Internet too. The litany of citations fool everyone into thinking that the article is credible, but when I actually read the citations, they don't support the article at all.
Specifically just the 400 best/recent ones. Google will never return more than 400 results for a query despite reporting ~80,000,000 results. If you click next enough it will run out after showing you ~38 pages of 10 results or almost 4 pages of 100 results.
This is the intended behavior. It kind of ruins serendipitious web surfing via search. I look through 4 pages of 100 results in less than a minute and the web as google search presents it just feels so shallow now.
Even this kind of project only helps so much. ChatGPT has absolutely no understanding of anything, it just regurgitates words and sentences in a way that sounds like it knows something. It's next sentence might be completely, dangerously wrong. You can't even claim that it lies to you. It needs to have an inkling of the truth to be able to lie in the first place. It's basically a well-trained parrot that has learned when to say the right combinations of phrases it's memorized at the right time in the right order to earn a treat. It has no idea what any given word means, let alone what it just said.
If someone built something that parsed sentences, stored facts, learned relationships between them, learned enough language processing to make connections between them in order to be able to say something new about the subject it didn't already know before-hand, then I'd be on that train. Right now I only see misery in our future if we just work on honing ChatGPT and it's ilk to better simulate BS.
I don’t know if you saw screenshots of the Prometheus model inside Bing. But it presents the sources with all its answers. It is no ChatGPT but a new model created for search.
The SEO content served up by Google search nowadays isn't much better, and from what I've seen I don't think a lot of it is being generated by someone either.
Perhaps a bad example, since ChatGPT consistently gets the steak temperature right (or at least gives the same values as Google). Internal temperature of about 130-135°F (54-57°C) for medium rare, etc.
It's just one data point but it may indicate the commenter is miscalibrated on what ChatGPT answers well and what it doesn't.
It reminds me of when Wikipedia was new and we were told repeatedly that we couldn't trust it (by teachers, articles, etc.). It didn't matter if we could point to studies that found Wikipedia had similar accuracy to other encyclopedias. They objected on the grounds that anyone could edit it and anything on it _could_ be wrong and there's no publisher or paid editors to ensure accuracy.
ChatGPT tends to do well on common questions, where the answer is plastered in hundreds of articles across the internet. The internal cooking temperature of a steak is a great example of this. There are many other types of questions it fails at.
A better example of where you shouldn't trust ChatGPT is asking it the distance between two cities. It'll get it right for a few common pairs of cities (eg London to Paris), but it'll give you a wrong answer for most less common pairs (eg London to Cairo).
The difference between ChatGPT and Wikipedia is that Wikipedia actually cites its sources.
Also ChatGPT is only knowledgeable about general things, but even there it makes errors. It's basically a very complex scraping algorithm, the more interesting part is the language generation; even then this stuff seems at least unethical if not illegal since it's using other people's work/research without citation.
In many cases Wikipedia's source links are dead. And nobody bothers to update them. Or they cite some random book that I have no way to get and read myself.
Who checks if that book actually exists and is not made up? Especially if it's not a highly politically charged topic.
And then there are circular citations. Someone posts unverified/false info on wikipedia. Then it gets cited by some blog or other media that wikipedia considers trustworthy. And that then gets added to wikipedia as a reliable source.
Agreed. I wouldn't use ChatGPT for this because Google is faster and more reliable. I just thought the example was a bit odd since ChatGPT gets it right, and it's a type of question it generally gets right.
The correct temperature to sous vide a steak depends on the desired level of doneness. The following temperatures are guidelines for cooking steaks to different levels of doneness:
It's important to note that cooking steaks at low temperatures for extended periods of time can result in a more tender and flavorful finished product, compared to cooking steaks at high temperatures for a shorter amount of time.
The problem is that if you don't already know the answer, it's hard to tell the difference between ChatGPT giving you the correct information and ChatGPT authoritatively spouting the wrong information.
Here's a chat I had today with ChatGPT:
Q: Who was the first US president with a last name starting with R?
A: Rutherford B. Hayes was the first US president with a last name starting with "R."
Q: no, a surname starting with R
A: Rutherford B. Hayes was the first U.S. President with a surname starting with "R."
Q: no, Rutherford is his first name
A: The first US president with a surname starting with "R" was Richard Nixon.
Q: no, Richard is his first name
A: The first U.S. President with a surname starting with "R" was James A. Garfield.
Here it's obvious ChatGPT is spouting nonsense, but in other cases it may not be.
Just for fun, I tried again today with a new chat:
Q: Who was the first US president whose surname begins with "R"?
A: The first US president whose surname begins with the letter "R" was John Quincy Adams.
Q: Adams does not begin with "R"
A: I apologize for the mistake. The first US president whose surname begins with the letter "R" was Rutherford B. Hayes.
Q: Hayes does not begin with "R"
A: My apologies for the mistake, the first US president whose surname begins with the letter "R" was James A. Garfield.
Q: Garfield does not begin with "R"
A: I apologize for the error. The first US president whose surname begins with the letter "R" was Richard Nixon.
Q: Nixon does not begin with "R"
A: I apologize for the mistake. The first US president whose surname begins with the letter "R" was George Washington.
Q: Washington does not begin with "R"
A: I apologize for the mistake. The first US president whose surname begins with the letter "R" was Franklin D. Roosevelt.
Q: Wasn't Theodore Roosevelt president before FDR?
A: Yes, you are correct. Theodore Roosevelt was the 26th President of the United States, serving from 1901 to 1909. He was the first US president whose surname begins with the letter "R." Thank you for pointing out the error in my previous answer.
So at least it's capable of parroting the correct answer after you deliver it. :-)
