On: I genuinely don't understand why some people are still bullish about LLMs

[senordevnyc]

It's always perplexing when people talk about LLMs as "it", as if there's only one model out there, and they're all equally accurate.

FWIW, here's 4o writing a selection sort: https://chatgpt.com/share/67e60f66-aacc-800c-9e1d-303982f54d...

[threeseed → senordevnyc]

I don't understand the point of that share. There are likely thousands of implementations of selection sort on the internet and so being able to recreate one isn't impressive in the slightest.

And all the models are identical in not being able to discern what is real or something it just made up.

[dvsfish → threeseed]

I guess the point is that op was pretty adamant that LLMs refuse to write selection sorts?

[wqaatwt → dvsfish]

No? I mean if they refused that would actually be a reasonably good outcome. The real problem is if they generally can write selection sorts but occasionally go haywire due to additional context and start hallucinating.

I mean asking a straightforward question like: https://chatgpt.com/share/67e60f66-aacc-800c-9e1d-303982f54d... is entirely pointless as a test

[senordevnyc → threeseed]

Because, to be blunt, I think this is total bullshit if you're using a decent model:

"I've even caught it literally writing the wrong algorithm when asked to implement a specific and well known algorithm. For example, asking it "write a selection sort" and watching it write a bubble sort instead. No amount of re-prompts pushes it to the right algorithm in those cases either, instead it'll regenerate the same wrong algorithm over and over."