> I think it “understood” the question because it “knew” how to write the Python code to get the right answer.
That’s what makes me suspicious of LLMs, they might just be coincidentally or accidentally answering in a way that you agree with.
Don’t mean to nitpick or be pedantic. I just think the question was really poorly worded and might have a lot of room for confirmation bias in the results.
> List of US Presidents with their ages at inauguration
That’s what the python script had at the top. I guess I don’t know why you didn’t ask that in the first place.
Edit: you’re not the same person who originally posted the comment I responded to, and I think I came off a bit too harshly here in text, but don’t mean any offense.
It was a good idea to ask to see the code. It was much more to the point and clear what question the LLM perceived you asking of it.
The second example about buckets was interesting. I guess LLMs help with coding if you know enough of of the problem and what a reasonable answer looks like, but you don’t know what you don’t know. LLMs are useful because you can just ask why things may not work or don’t work in any given context or generally speaking or in a completely open ended way that is often hard to explain or articulate for non-experts, making troubleshooting difficult as you might not even know how to search for solutions.
You might appreciate this link if you’re not familiar with it:
I was demonstrating how bad that LLMs are at simple math.
If I just asked a list of ages in order, there was probably some training data for it to recite. By asking for it to reverse it, it was forcing the LLM to do math.
I also knew the answer was simple with Python.
On another note, with ChatGPT 4, you can ask it to verify its answers on the internet and to provide sources
You’re also scarface_74? Not that there’s anything wrong with sockpuppets on HN in the absence of vote manipulation or ban evasion that I know of, I just don’t know why you’d use one in this manner, hence my confusion. Karma management?
I saw a blue icon of some kind on the link you shared but didn’t click it.
No worries, that was somewhat ambiguous to me also, and confusing. I thought you might be a different person who had edited their comment after receiving downvotes. I mean, it’s reasonable to assume in most cases that different usernames are different people. Sorry to make you repeat yourself!
Maybe email hn@ycombinator.com to ask about your rate limits as I have encountered similar issues myself in the past and have found dang to be very helpful and informative in every way, even when the cause is valid and/or something I did wrong. #1 admin/mod on the internet imo
I tried asking the question more clearly
I think it “understood” the question because it “knew” how to write the Python code to get the right answer. It parsed the question as expected
The previous link doesn’t show the Python. This one does.
https://chat.openai.com/share/a5e21a97-7206-4392-893c-55c531...
LLMs are generally not good at math. But in my experience ChatGPT is good at creating Python code to solve math problems