typical Gell-Mann amnesia. I write a lot so I see this when I've used AI to generate writing.
If I ask it to generate something that I understand: "Wow this is the most surface level garbage anyone could create by reading one SEO-optimized article"
When it's about something I don't know anything about: "Wow it's impressive how far AI has come"
What I do find AI super useful for is finding stuff like metaphors.
"I want to find a metaphor between product-led vs. sales-led growth strategies for startups. Please give me a list of 10 i could use"
Those prompts are incredibly useful. I reckon there's some type of analogy with coding here.
I think the car analogy is interesting because I'd say 90+% of drivers don't really know how the engine works besides gasoline being burned to make the car move lol.
FWIW, you can often feed the generated output back into the AI and ask it to troubleshoot, like "when I ran this code, I got ____ error" or "it's missing the ____, how do I fix that" and it will usually apologize and try again. It takes a few tries. But it usually does with humans too.
Interesting. It sounds like there are two parts to this:
a) Product sense becoming more important i.e. talking to users and figuring out what they want and how to build it rather than the pure ability of using code to make the computer do something
b) Having the machine but no idea how to repair/augment it only gets you so far. Like you might be able to run a lawn care business but if your lawnmower breaks and you can't repair it your business is dead.
ChatGPT will allow you to start making progress 10x faster than before. It will help you be productive from the start without programming experience.
However as your project grows you will be forced to learn programming principles in order to progress. ChatGPT will help you learn these principles 10x faster.
My suggestion is don't think so binary about it. ChatGPT+GitHub copilot will make it way easier to code, and to learn how to code, and you will get stuck far less often (than you would have before LLMs) but you should still expect to have to learn a lot along the way.
LLMs are to programming what a power drill is to carpentry. Way easier to use and faster than a screw driver, but you still have to use it...
It is also about knowing when to mow the lawn. Generally, you shouldn't mow the lawn when it is too wet, as it is too heavy on your mower. You also want to avoid mowing it when it has been too sunny, as the grass will turn brown. That requires experience beyond buying a mower.
My experience with ChatGPT for coding has actually been the opposite: It was better at understanding and guessing underlying needs than the typical programmer I've worked with, who usually needed everything spelled out in technical bullet point requirements. Clients don't think like that (because they don't know how software works), but programmers need that (or need to come up with it on their own) to write the code. ChatGPT on the other hand seems pretty good at using its vast training data to guess at your intent and needs, as long as it's a relatively common use case.
I would say that's one of its great strengths (its ability to combine your prompt and its vast natural language data to give you an answer that effectively summarizes what your good options are). It's a better product owner than most programmers, in other words.
On the other hand, its coding ability isn't very impressive. It has the ability to generate code and it will run it and debug it if you ask it to, but it doesn't always do that by itself, and it's not great at doing "sanity tests" along the way to make sure each unit of code it's producing is actually doing what it should. For the most part it's doing the same sort of statistical modeling of language for coding as it does for natural languages, but code is a lot more precise and nuanced and can't just be a popularity contest of most statistical likely outcomes.
As a programmer, even a beginner one, you can usually spot these mistakes or at least ask it to explain or change some part of the generated code. However, for someone with zero coding experience, it might be harder to even spot these mistakes to begin with. You might be able to deploy the code but it might not actually do the thing you think it's doing, or maybe only do it under some circumstances but not others.
To be fair, though, the same things can happen when you work with human programmers. There's a lot of back and forth there too and trial and error. And many of us aren't great communicators, so ChatGPT has an advantage there. If you're patient with it, it might be a nice long term partner. But it's not quite at the "make me a _____ app and deploy it" one-prompt stage yet. Probably soon! I have no doubt one day soon it will replace developers like me, but for the time being, it works more effectively when paired with a human (and vice versa).
I think part of that is because their only intent is to tell the story in a way people understand. They're not trying to sound smart because there's no editor or boss that reviews their writing.
Their point is to make sure someone gets the story, not that they get promoted or something
That makes sense. It reminds me of Drunk History. The way a drunk person will just tell you a story without any complex fluff because sentences are expensive when you’re drunk.
If I ask it to generate something that I understand: "Wow this is the most surface level garbage anyone could create by reading one SEO-optimized article"
When it's about something I don't know anything about: "Wow it's impressive how far AI has come"
What I do find AI super useful for is finding stuff like metaphors.
"I want to find a metaphor between product-led vs. sales-led growth strategies for startups. Please give me a list of 10 i could use"
Those prompts are incredibly useful. I reckon there's some type of analogy with coding here.