99.99% of people would be unable to answer that question (without looking it up, I mean). Such hyper-specific queries for highly technical information from niche fields say very little about the model's overall performance at natural language tasks.
If you ask things like "Which of these animals doesn't live in Africa?" or "What is the most reactive chemical element?", ChatGPT's answers are almost always correct. And they are far more likely to be correct than the average (unaided) human's.
Update. This morning I asked ChatGPT what day today was. It answered correctly. I then asked how it could know that given that its training data ends in September 2021. It said it was based on the number of days since its training data ended. I pointed out it still had no way of knowing that number of days if it had no knowledge past September 2021. It kept apologizing and repeating the same story over and over.
99.99% of people would be unable to answer that question (without looking it up, I mean). Such hyper-specific queries for highly technical information from niche fields say very little about the model's overall performance at natural language tasks.
If you ask things like "Which of these animals doesn't live in Africa?" or "What is the most reactive chemical element?", ChatGPT's answers are almost always correct. And they are far more likely to be correct than the average (unaided) human's.