> That truly sounds like a skill issue. This no-true-Scotsman angle is silly. I said documentation and tests, I don't know how you got "doctests" out of that. I said "documentation", and "tests". I didn't say "the simplest form of documentation", that is another straw man on your behalf.
What are you on about? Doctest is the simplest form of documentation and test. I.e. you don't have to write an in-depth test, you just need to understand what the function does. I expect even juniors can write a doctest that passes the compiler check. Not a good, not a passing one doctest, a COMPILING one. It's rate of writing a passing one was even worse.
> Wow, the arrogance. There is absolutely nothing to justify this assumption.
Ok, then. Prove what exactly hard problems did you give it?
I gave my examples, I noticed it fails at complex tasks like YAML parser in an unknown language.
I noticed when confronted with anything harder than writing pure boilerplate, it fails. E.g. it would fail 10% of the time.
> Unfortunately, this discourse has made it clear that you do need considerable practice
You can practice with a stochastic parrot all you want, it won't make it an Einstein. Programming is all about converting requirements to math, and LLMs aren't good at it. Do I need to link stuff like doing basic calculation and counting 'r' in the word 'strawberries'.
The best you can do is half the error rate, but that follows a power law. You need to double the energy to half the error rate. So unless you intend to boil the surface of the Earth to get it to be decent at programming, I don't think it's going to change anytime soon.
> You have demonstrated throughout this discussion that you aren't capable of assessing machine intelligence.
Pure ad hominem. You've demonstrated nothing outside your ""Trust me bro, it's not a bubble"" and ""You're wrong"". I'm using double double quotes so you don't assume I'm quoting you.
> You didn't even bother clarifying which LLM you used.
For YAML parser, I used Chat GPT-4o at my friend's place. For the rest of the tasks I used JetBrains AI assistant, which is a mix of Chat GPT-4, GPT-4o and GPT-3.
What are you on about? Doctest is the simplest form of documentation and test. I.e. you don't have to write an in-depth test, you just need to understand what the function does. I expect even juniors can write a doctest that passes the compiler check. Not a good, not a passing one doctest, a COMPILING one. It's rate of writing a passing one was even worse.
> Wow, the arrogance. There is absolutely nothing to justify this assumption.
Ok, then. Prove what exactly hard problems did you give it?
I gave my examples, I noticed it fails at complex tasks like YAML parser in an unknown language.
I noticed when confronted with anything harder than writing pure boilerplate, it fails. E.g. it would fail 10% of the time.
> Unfortunately, this discourse has made it clear that you do need considerable practice
You can practice with a stochastic parrot all you want, it won't make it an Einstein. Programming is all about converting requirements to math, and LLMs aren't good at it. Do I need to link stuff like doing basic calculation and counting 'r' in the word 'strawberries'.
The best you can do is half the error rate, but that follows a power law. You need to double the energy to half the error rate. So unless you intend to boil the surface of the Earth to get it to be decent at programming, I don't think it's going to change anytime soon.
> You have demonstrated throughout this discussion that you aren't capable of assessing machine intelligence.
Pure ad hominem. You've demonstrated nothing outside your ""Trust me bro, it's not a bubble"" and ""You're wrong"". I'm using double double quotes so you don't assume I'm quoting you.
> You didn't even bother clarifying which LLM you used.
For YAML parser, I used Chat GPT-4o at my friend's place. For the rest of the tasks I used JetBrains AI assistant, which is a mix of Chat GPT-4, GPT-4o and GPT-3.