Their smallest model outperforms GPT-4 on Code. I'm sceptical that it'll hold up... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

hackerlight on March 4, 2024 | parent | context | favorite | on: Claude 3 model family

Their smallest model outperforms GPT-4 on Code. I'm sceptical that it'll hold up to real world use though.

nopinsight on March 4, 2024 [–]

Just a note that the 67.0% HumanEval figure for GPT-4 is from its first release in March 2023. The actual performance of current ChatGPT-4 on similar problems might be better due to OpenAI's internal system prompts, possible fine-tuning, and other tricks.

Join us for AI Startup School this June 16-17 in San Francisco!
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact