Well before AI co-pilots something happened to the good old admins--they started to disappear only to be replaced by "AWS devops" (their job titles) who have never wired a network using routers, switches, and cables. I noticed that they started lacking basic networking knowledge and couldn't set up networking inside AWS. They just didn't know what a gateway, NAT, or subnet is.
Similar things are happening with AI co-pilots. We have an increasing number of people who "write code", but the number of people who can understand and review code is not increasing. There is also a problem of injecting ethics and politics into those tools, which can produce silly results. I asked Bard to write me a Python function to turn the US Constitution into a palindrome. Bard refused and gave ma a lecture on how the US Constitution is too important to be played with in such trivial fashion. I then asked it to produce code that turns the US national anthem into a palindrome, it refused again. So I asked it do the same but with the Russian national anthem and it spat out code without telling me off. I then asked it to generate code for simple tasks and it did an OK job, except the formatting and the fonts used were different every time, because it just lifted code from different webpages and recombined it like a massively hungover student waking up to realise he's supposed to hand in the assignment in one hour.
> Well before AI co-pilots something happened to the good old admins--they started to disappear only to be replaced by "AWS devops" (their job titles) who have never wired a network using routers, switches, and cables. I noticed that they started lacking basic networking knowledge and couldn't set up networking inside AWS. They just didn't know what a gateway, NAT, or subnet is.
I see the same arguments about higher level languages. All of the tech industry is about standing on the shoulders of giants. I do believe that lower level knowledge can help a great deal but using this argument I can say something like:
Kids these days don’t know basic assembly, they have no idea what an or/and/nor/xor gate is, they’ve never built a computer from components (no, not cpu/mb/ram/etc, I’m talking transistors and soldering).
Maybe LLMs are different but I don’t think they are, they are yet another tool that some people will abuse and some will use wisely. No different from an IDE or a higher-level language in my book.
I have next to zero idea how many things I use daily work (like my car for example), it doesn’t stop me from being able to drive where I need to go. I have configured lower level networking equipment in the past and I couldn’t be happier that I don’t have to do that drudgery anymore.
All I’m saying is this is a dangerous argument to make because someone can always one-up (one-down?) you. “Oh you had NAT? Luxury! In my day we didn’t even have a network, we had to….” (See also: Four Yorkshiremen [0])
I do not wholly agree with you. Just to pick on one thing in your reply...
> Kids these days don’t know basic assembly
It is a problem. Knowing assembly is crucial if we want to have more efficient, faster hardware. Someone has to write kernels and drivers. The efficiencies happening on the lower level of the stack make the upper layers of the stack faster, more stable. It is also important that we have a large pool of devs who know how to do it. Otherwise that knowledge will be taken in-house and shared under strict NDAs preventing devs working for competition and effectively killing all the efforts of the Open Source community to wrangle free access the computer out of the claws of corporations.
This is not a problem that AI has brought about first, but co-pilots are accelerating it.
Just to provide some context, I could not wait for 16-bit, 32-bit, and then 64-bit architectures to become popular. I was the happiest dev around when object oriented programming tools became available, same to Linux, web, all frameworks and even cloud computing. But AI as it is currently being sold is a shit tool. If it stays and doesn't disappear after the VC money runs out, we will have to adapt our ways of working and educational courses to teach devs to review work of others recompiled by AI (it can't write shit on its own). Not sure it is worth the effort, because every time a new framework is introduced the results generated by AI will be of low quality until the models get trained on exaples written by humans... What's the point of using AI in that case?
Agree here. As a 90's kid I dove into computers, Internet and programming head first. I went into the field out of passion, not money. I received an engineering degree.
If I was the same kid now, I'd get into a bootcamp or self teach and get a web dev job because the money is lucrative. Yeah sure, maybe i'll dabble in foundationa litems here and there, but most won't. It doesn't advance your career in web/software.
IMHO we will see a brain drain in computer/electronics engineering in 20 years when the pipeline of graduates dwindle and those with experience retire. Hope LLMs can innovate enough to make up, I guess.
> If I was the same kid now, I'd get into a bootcamp or self teach and get a web dev job because the money is lucrative. Yeah sure, maybe i'll dabble in foundationa litems here and there, but most won't. It doesn't advance your career in web/software.
That sure sounds like it has nothing to do with LLMs and everything to do with factors not all related to even computer science. “Capitalism” or maybe the more trendy “late-stage capitalism” is probably more to blame here than anything else.
Also I reject the idea that there are no kids today that get involved in tech/computers for the joy of learning/exploration.
I bristle whenever I see arguments about “kids these days didn’t learn exactly how I did so it must be wrong”. It reeks of “old man yells at cloud” (note, I was also a 90’s kid). It’s the same BS I heard as a kid about how computers would rot your brain or they were horrible for <insert stupid prediction that didn’t come true>.
I agree that it reeks of old men yelling at clouds but there is data driven precedent here: with the advent of smartphones, computer literacy in education has dropped like a rock. It isn’t just kids these days, it’s a genuine shift in computing across the entire populace that happens to really effect the funnel for programmers at the entrance to that funnel. Copilots are an extension of that development, not necessarily something wholly new.
I’ve heard this anecdotally from every educator I know at the high school and college level. CS students are increasingly entering without any computer proficiency whatsoever so it has to be built up from scratch.
Yep, despite all the talk of young people being "digital natives", installing apps from Android/iOS app store, taking pictures and chatting with friends doesn't teach you much that is valuable for office work.
PC gaming and specifically modding is what I can thank for much of my computer proficiency, but even that is easy these days with Steam and its workshops for many games.
Of course you’ve now effectively circled back to the early 1980s at some level when there was not a lot of expectation that college freshmen had touched a computer much less were proficient even if the wanted major in CS/EE.
I think the apt analogy here isn't "old men yelling at clouds" but the historical way that one certain previous generation become more adept than those before or after at working with a certain then-new technology -- thus becoming a uniquely skilled generation in that technology, moreso than their elders or their youngers, never to be repeated again.
I'm not talking about millenials/gen X with computers -- I'm talking about silent gen/(boomers/greatest gen?) with cars.
In both cases, a new technology dominated the world, but was new and brittle at first. The kids who saw it in their youth were fascinated, but because of its brittleness they had to become at least minor experts in at least minor troubleshooting and mainteanance (like tuning a carburetor or defragging a hard disk) just to access the coolness.
I mean, it really was a pain to be a car owner in 1960. A lot like being a computer owner in 1995. If you wanted to enjoy one, you were going to need to change a spark plug or registry value once in a while. And you had to be ready to recover from an engine overheating or a blue screen of death, because these were not rare events.
Then the future generations increasingly lost touch with those skills because the technology got smoothed out as it developed further.
So I think it's quite plausible that future generations will permanently have less interest in serious skills with computers since the same thing has happened with cars. There is a way smaller perecentage of people my age (millenial) who are "into" cars or have moderate familiarity with car repair than people in their 70s. So it seems plausible the same pattern could continue to play out with computers, rather than a "old man yells at cloud" illusion
> Knowing assembly is crucial if we want to have more efficient, faster hardware
this is a goal for maybe 2% of the industry. And those folks know their assembler or they don't get hired. I think we are producing skills in the proportion of its requirement in the industry.
I hate to dash your hopes of never seeing LLMs write code again but Llama3 70B is not terrible as a copilot and it’s small enough to run on comparatively cheap hardware or even consumer hardware when quantized plus it’s publicly available so from now on there will always be some form of copilot service or tool that will be in use, that ship has sailed. The good news is that it doesn’t look like we’ve maxed out the capabilities of GPTs yet so as more are trained and compute becomes cheaper and as we produce and accumulate more data they should keep getting better for a while at least.
> Not sure it is worth the effort, because every time a new framework is introduced the results generated by AI will be of low quality until the models get trained on exaples written by humans... What's the point of using AI in that case?
LLMs will not replace us, they just take some of the tedium out of writing code. Yes, they will have to be trained on new frameworks and languages but I don’t see that as a problem. People have to wait for SO questions and blog posts to be written about new frameworks and languages as it stands today.
I’m not sure I agree with your arguments about needing a lot of devs to understand assembly or the like. Or rather I disagree with your prediction of where that leads. We are effectively already there and it hasn’t led to your future. We have an amazing open source ecosystem and that billions of people rely on daily without knowing how every part of it works. Open source has only grown during that time.
I’ve never done the math myself for calculating the distance between 2 points on a globe. I just grab the Haversine formula and keep going. I don’t see that as a problem. Similarly I’ll reach for an open source library to solve a boring problem instead of doing it myself. I quite literally see no difference in that and using an LLM.
> Kids these days don’t know basic assembly, they have no idea what an or/and/nor/xor gate is, they’ve never built a computer from components (no, not cpu/mb/ram/etc, I’m talking transistors and soldering).
At each layer, though, we have a legible formal theory bridging one level to the next, and the abstractions tend to be so reliable the last thing most of us will consider is a hardware/assembler/compiler bug (though they do occasionally happen).
LLMs bridge layers illegibly and probabilistically. I can already see how they're helpful, but they do have some different characteristics that mean reliability and the connection to lower levels of the stack are an issue.
> LLMs bridge layers illegibly and probabilistically.
I’m not sure I agree but I want to make sure we aren’t talking past each other.
Using an LLM to write code does not have this problem IMHO.
Using an LLM as part of the code does have this problem.
I’m not writing off the usefulness of calling out to an LLM from code but it does bring “illegibly and probabilistically” into the mix whereas using an LLM to write code that’s human-reviewed/modified does not.
In fact, you should be unable to tell what I wrote vs what the LLM wrote if I’m doing right (IMHO). “Clever” code is always bad no matter who/what wrote it.
The only time I’ve been asked “did you use <insert LLM> to write this?” Was due to the speed at which I completed something or because it’s in a language that the person asking the question knew I didn’t know very well.
LLM's can be good at guessing the code you want, but I've seen them guess the comment or documentation correctly, then the implementation is the opposite of what you asked. It also generated tests which were for the incorrect case, with comments showing the opposite of what you wanted.
For somebody new at the language or new to programming in general, that difference can be imperceptible until it gets to a review.
I've also managed to spot people using an LLM to write stuff due to them doing things which won't pass linters or code tests.
The subtle differences between comment/code can absolutely be a problem, I’ve fallen prey to it as well but as long as you test/execute the code you’ll catch it if you didn’t catch it from reading over the code. I often reject anything overly clever or that I can’t reason out quickly when it comes to LLM generated code.
As for not passing linters or code tests that’s unacceptable behavior from an employee. If that happened more than a few times I’d be having a talk with the employee followed by showing them the door if they continued to do it. I’d behave the exact same way if no LLM was involved.
I do wish LLMs like Copilot could use the lint rules and/or tests to inform the code they generate but thankfully it’s normally a keystroke away for me to reform a block of code to match the style I prefer so it not the end of the world.
To use LLMs (or any tool) you have to be able to show you can use it correctly, safely, effectively, etc. If you can’t do that then you have no business being in a coding role IMHO.
Points for a great twisting of the metaphor I used, I quite like it.
I agree _someone_ has to have that knowledge, I reject they _everyone_ must have that knowledge.
I have zero desire to learn more about logic gates, they don’t interest me and while they enable me to do the work I do I don’t need to know about them to do the work. The same way that I couldn’t tell you how my car engines works (ok, at a high level I could, probably about the same level as I could talk about logic gates) but I’m able to drive from point A to point B without issue.
There are people who enjoy assembly, let them work on it. I’m not one of those people and I don’t have a problem with those people but I do have a problem with people who think I must know assembly to be an effective developer.
> I have zero desire to learn more about logic gates
Sure, but you know that they exist, which to me suggests that you learned about them somewhere and probably have at least some foundational understanding of how they work
The problem is when people can skip any of that broad foundation and go straight to specializing somewhere, a lot of general knowledge is lost
Part of learning the broad foundations is also learning that those options exist for a specialization path
Someone who may love working on hardware designs may wind up spending their life as a mediocre web developer because they took an AI assisted web bootcamp that skips learning important fundamentals
It's an engine, and can make all kinds of things move.
If you don't know it works, sure you can drive a car, but a lot of missed opportunities to leverage the same exact tech for other use cases disappear when you don't know fundamentals.
Interesting, the college I went to (2009) didn’t teach it at all. C++ was the lowest level they taught and they switched to Python after I left (as in they use it instead of c++ last I heard).
If I ever needed to write assembly (which probably means something has gone horribly wrong in my life) then I’m sure I could learn it on the fly, just like I’ve learned other things never taught in college on an as-needed basis.
This argument falls apart for me personally. I'm sure we could go back a decade or two and find someone saying something like:
It’s quite alarming to think that the most advanced language many new software engineers are familiar with is merely C or C++.
Imagine graduating without ever having programmed in assembly. It’s not that C or C++ aren’t capable languages, but we’re definitely in a strange era when the deeper understanding of machine interaction through lower-level programming is becoming a rarity.
I don't think that's quite the same, but in general I agree with that sentiment, even though you weren't serious about it.
Folks should know their basic C/C++ just like they should know enough x86 to make sense of what godbolt.org tells them. But oh, well, that's probably a lonely hill to die on. Having a rough idea of all these levels of abstraction work and interact is something I profit a lot in my line of work, but that's just a small niche and I get that (profiling and optimizing scientific, signal processing routines).
I consider assembly to be a basic skill like arithmetic. Most of us have a calculator on hand 24/7, but being able to look at a clock and say 'I've got 12 minutes' is useful.
I had to take an "operating systems" class in Java 15 years ago, luckily i already knew enough about C and assembly to know it's a waste of time. the professor thought it was more important to teach "fundamentals" than low-level programming.
i got a c in that class (the grade not the language) because i didn't know java and kept having difficulties with the way it forces you to shoehorn OOP into everything.
I don't remember seeing a class on it but it was a long time ago. I also don't remember it being required to get a degree. I just looked on my college's website and they offer 2 assembly classes it looks like, I'm not sure if they existed when I went there (I assume they did) and neither looks like it's required but I don't have the time or energy to dive back into that kafkaesque hellscape of figuring out what's required to graduate.
However, I didn't get a degree so maybe it was shoehorned into some high level course. I had a job writing software while I was in college and dropped out to do that full time once I felt I wasn't getting any value from continuing my "education". It's never been an issue in my career and I've done well by my own metrics at least.
Yeah my husbands college (I'm a SW eng, he wants to be one) teaches C++ as the base...and this is a "just" a state school in a Arkansas.
Also gotta correct myself, after asking him it looks like I was wrong. Assembly isn't a direct requirement but it's one of three electives you have to chose from.
You have to choose between either assembly, cybersecurity or data science and assembly is considered the easiest apparently.
I feel comparing knowing assembly to knowing about NAT/Subnets/Gateway is a false equivalency.
Outside of a small set of folks most devs need NOT know assembly due to compilers being very good at converting to assembly.
Not the same Subnet/Gateway/NAT. These are first class constructs in most cloud developments and has huge implications on security, performance and costs. Unless you have some kind of automated way that abstracts away these issues (akin to compilers) most devops folks would need to learn about these concepts while working in a cloud environment.
I think there may be a difference in kind here than with a lot of previous labor-saving abstractions.
To use an LLM, you still need to know the language you are using it to write. You don't need to know C or assembly as a Python/JS programmer today, but you do need to know JS to be an LLM-heavy JS programmer.
> You don't need to know C or assembly as a Python/JS programmer today,
I agree, not everyone does. Maybe we have progressed to the point we can all say those people are wrong and their arguments were always wrong, maybe not.
> but you do need to know JS to be an LLM-heavy JS programmer.
I also agree. That's my whole point, you do need to know the language to be able to effectively use an LLM.
My issue is with people who immediately reject LLMs as a net-bad because they do part of the job for you. I reeks of "well I had to do it the hard way so you do too" which I cannot abide and immediately flips the bozo bit on them for me.
Okay, but if you don't know what the JS is doing and why how are you ever going to architect stuff efficiently or think up more complex things in the abstract? I fail to see how you can just know HTML and CSS, that's like saying you have the alphabet down but somebody else can speak or write for you. HTML and CSS are just markup and style languages, it isn't really code.
CoPilot is just blundering along using what it has learned from elsewhere - which is not always correct.
I'm very impressed with the best models — but, because I remember how awful NLP used to be, "very impressed" still means I rate the best as being around the level of an intern/work placement student most of the time, and even at their best still only a junior.
It's great, if you're OK with that. I've used GPT-3.5 to make my own personal pay-as-you-go web interface for any LLM that's API-compatible with OpenAI despite not being a professional Web Developer.
It's fairly fragile because it's doing stupid things to get the "good enough" result.
Bu that's OK, because it's for me, I'm not selling it.
(As for the lesser models… I asked one for a single page web app version of Tetris; it started off lazy, then suddenly switched from writing a game in JavaScript into writing a machine learning script in python!)
> They just didn't know what a gateway, NAT, or subnet is.
If you ask me, they shouldn't have to, in the same way that you probably don't know how to diagnose a coax cable problem.
The major public clouds all have completely software-defined networks (SDNs), and concepts like subnets are basically emulated for the sake of legacy systems.
Why would we want subnets, like... at all? Why can't all systems just have an IP address and use point-to-point communications? That's literally what the underlying SDN does with the packets anyway! The subnets and routes you see don't map 1:1 to the underlying network. The packets are stuffed inside a VXLAN packet and routed between random hypervisors scattered randomly across multiple buildings. Just give up the pretence, give everything an system-assigned public-routable IPv6 address and be done with it. Better yet, auto-register everything in a private DNS zone and auto-generate internal use SSL certs too.
The amount of wiring we do manually is just insane, just so that Azure and Amazon can help greybeards pretend that they're operating a Cisco router and need to stand in line to file the paperwork required to get an SSL certificate.
> you probably don't know how to diagnose a coax cable problem
No, that was definitely part of the sysadmin job description ~25 years ago, both line discipline and physical medium problems (I had a TDR and a fiberglass fish at my desk for my first sysadmin job and was kind of bummed to find out that isn't remotely normal anymore)
Problem is that all those systems are proprietary to individual clouds.
Congratulations folks! We undid several decades of progress and went back to the 1960s and 1970s when software can only run on the specific mainframe for which it was written.
Which is why I don't write for the cloud. My personal stuff goes into a dedicated linux server setup with docker compose, and my work stuff is all hosted on windows servers.
I'd prefer switching those windows servers to Linux, but we're too integrated with Microsoft at this point.
It’d be easier to swap those Windows servers out for Linux than it is to migrate the typical cloud native application from one cloud to another.
The cloud is the most closed, most locked in, most rent extraction oriented computing paradigm since the pre-minicomputer mainframe era. It’s really a return to that era.
Computing tends to cyclically reinvent wheels. I suppose next we will reinvent minis, micros, LANs, etc.
I think you’ve stumbled on the solution there - *something* happened between the 70s and today to address that thing from the 1960s, and it was motivated by capital-seeking. There’s a high probability that it will happen again, and if it does not then it’s likely the market didn’t demand it. Or, the world is just one big corporation and we’re all screwed. One of those.
The problem with that idea is that it requires every router to know about every system on the network. Unless they abandon Ethernet and replace it with MPLS source routing, which maybe they should. You'd still have a maximum subnet size, because the hardware firewall that would still have to be in front of every server would have a limited capacity.
I've noticed that ChatGPT will speed up the monotonous part of writing code, but will often inject a few bugs into the code that I need to fix. In a way, ChatGPT has become my personal junior developer (which I only use for basic stuff) while I'm still dependent on my own expertise to review and fix the code, but at least it's still faster than using google and writing most of it myself. Juniors will always have bugs in their code, but when you bypass the step of writing the code and figuring it out for yourself, how do you effectively learn and become a better developer?
In a way, ChatGPT both replaces the need for junior developers and creates a situation where you'll end up with a developer shortage because not everyone can start right away as a mid-senior level.
This is definitely appealing to my curmudgeon heart. But I translate it into my corner of the world and the equivalent is everyone should know c. We don’t need everyone to know c.
Everything else being exactly equal would I like someone I work with to know c over not? Sure.
But would I take a world with many fewer programmers that all knew c over the one we have today—definitely not. It would be a significantly poorer world.
Your mentioning of AWS devops reminded me of a story my dad, a mechanical engineer, told me. He travels a lot and works with engineers all over the world. He was talking to me about how the new "wave" of engineers (mainly in Canada and America) that he works with there is a a complete lack of people that know how to do things like manually calibrating various parts by hand.
He's had to completely rethink how they approach their industrial designs and build machines and transition to much more electronic based things with fewer moving parts, and build things they can just adjust with computers and point-and-click GUIs, because people just don't know how these old things work from a physical reality perspective. My dad was very distraught by the caliber of engineers he's seeing in the "next generation".
But that's not the only thing.
In Canada, our aging population of tradespeople are retiring in droves and we have a dearth of younger people to replace them. It's so bad, that in my province they've completely redesigned the Grade 11-12 highschool curriculum to have "fast track" paths into trades where 80% of their time is spent in coop/apprenticeship (good thing!). We have a housing crisis and we have nobody that actually knows how to build houses in our country while the federal government imports 1 million people per year with nowhere for them to live. Most kids these days just want to be "influencers".
Spelling and grammar in young people is absolutely atrocious because they are totally dependent on autocorrect; and they can't type using physical keyboards because they only ever used a touchscreen keyboard on phones and tablets.
It's not just programming being destroyed by copilots. It's an overall dumbing down of our entire civilization.
I have the same issue with young software engineers. We used to install every Linux distro because it was fun. Junior are now scared of the command-line.
Another example that I see all the time: I recently had to review a pull request from a junior. 200 lines of very simple C++ doing almost nothing, you can't fail that. I had to write more than 50 suggestions and comments on very basic stuff that the guy should have learned when he started coding. Yes, they must learn the ropes, but it seems like the passion is not there. We used to be passionate about coding, and all I see is people who went to private college to make money but don't enjoy what they do.
I was a hobbyist programmer installing all sorts of Linux and BSD distros on my computers since the age of 10. My first real software job still had me go through a period where I had 50+ comments on some of my initial changes because life before work doesn't really prepare you for what you need to know on the job. If they aren't learning from your feedback and constantly making the same mistakes then I understand, but it's somewhat misguided to expect folks to simply know a lot early in their career.
Heck even folks coming in with experience, but from a company with a very different coding culture or simply a different primary language can result in the same situation.
Do you/they learn is the question. Not programming but a different role in tech but I’ve had a recent intern who did a very competent job and vacuumed up feedback and opportunities to learn. And I’ve had a longer tenure person who moved in from a different type of role at a different company who… did not. And just couldn’t appreciate how what they were creating was a net time sink for everyone else. They didn’t last.
Niche-ness isn’t a hard metric, but the argument is the same for “this is what happens when a profession hits a critical mass of people just in it for the money and not at all for the enjoyment of the profession.”
lol, i'm reinstalling a linux distro right now just because I want to try a new one
I work with seniors that are scared of the command line. I can somewhat understand why Microsoft might be trying to turn everything into AI Copilots because when I'm trying to explain to someone how to revert changes in a file with `git reset/checkout` they practically recoil in terror at the suggestion of using the terminal. They are married to their git guis and have no idea how it works. Best AI-ify as much as possible to keep people in their ecosystems.
But are they any less productive? My experience is no. I was a command line aficionado when I entered the workforce, but I saw several people get the same results as me in a purely GUI environment. I see the same today. They get shit done and that is what is required. I feel like older devs romanticize certain parts of their workflow too much.
They're not necessarily dumber (maybe in the sense that they would be less able to survive on a desert island), but rather the same neural circuitry is being repurposed for other things. If anything the environment is dumber and they're just better at efficiently navigating it than we are.
When I need to explain to other so-called seniors how to revert changes to a file with git checkout and they recoil in terror in using the command line, then yes - I believe they are actually getting dumber.
No. Most kids strive to follow whoever "mentors" and teaches them.
What is sad is that this job is left to influencers. Not the kid's fault if our education system, parenting and attention economy are completely f'd up.
> There is also a problem of injecting ethics and politics into those tools, which can produce silly results.
Well, they can, but the "no filter" option does not necessarily generate satisfactory results either. I find it hard to blame Google for not wanting to put their brand on an AI providing a ranking list of races by intelligence or whatever kind of offensive thing people are going to try to make it produce.
We are talking about a code generator. Dev tools ought to be neutral when it comes to politics and ethics otherwise you get a smart hammer that will refuse to nail Jesus to a cross. That's not how tools work.
Safety devices put into tools to prevent misuse or accidents are also not that unusual. Feel free to change my example to something like "a code snippet that will use the 23andMe API to prevent users with insufficient European ancestry from registering an account" if you want a more code-specific example of reputational risk from an unfiltered AI.
> and it did an OK job, except the formatting and the fonts used were different every time, because it just lifted code from different webpages and recombined it like a massively hungover student
Tangent, but it used different fonts? I can’t imagine how this is possible given how LLMs work under the hood (sampling likely tokens which represent Unicode character sequences - it doesn’t operate on rich text so afaik has no way of seeing or changing font information). Is there something weird happing with the frontend breaking the code/not-code formatting they do (as a cosmetic thing rendering the output)?
As constitution - changing one but not another could be a matter of cultural sensitivity as well (although I bet Bard isn’t that smart).
As a Polish person, I don’t mind you changing the constitution, but for example with our national anthemn we are way more attached to changing how it sounds than Americans are to theirs. Ditto where our flag and our symbol can be used.
As for Bard - did you try arguing with it that constitution guarantees you the freedom to play with it? I managed to convince gpt to do some weird stuff by arguing for cultural sensitivity.
What Bard couldn't understand was that what I actually expected it to do was to write (well, steal, these things are simply giving answers that look like plausible answers by recombining other people's work) a function that takes a long piece of text and turns it into a palindrome. It cannot deal with ambiguity and imprecision, but will refuse how to acknowledge that fact and will keep generating random answers like a public schoolboy who insists on winning an argument despite not knowing a thing about what it is he's arguing for or against. When I asked Bard a more precise question and asked for a function that takes a long piece of text and turns it into a palindrome it gave me a different implementation of it every time I asked the same question behaving like a kid who has no clue, but will give a random answer and watch the teacher to see if the teacher accepts it.
I will not argue with Bard or any other tool of that sort. I have better things to do. They are supposed to be good at translation, so I thought ok, let's try giving it a piece of my own writing in English and ask it to translate it into French (a gendered language). It could not infer from the text that the internal monologue that the protagonist was a female, so it translated it all using male voice. I asked it to switch gender to female voice but it stopped doing so after translating about two pages of the text and even on those two pages it kept switching to male voice. Then, after translating three pages it replied "I don't know how to do it" when I asked it translate another page, even though I used the same prompt I was using to ask it to translate previous pages. These are toys not tools. They are shit at their job and a waste of time and electricity.
Since you are Polish, I was told by my Polish colleagues that onet.pl, a large Polish news site uses AI to auto-translate articles from foreign media sources and that all of those translations are done in the male voice, even when the original text clearly describes what a woman did or said. My colleagues say it's been going on for a while and nobody bothers to fix these translations. If true, the future is here, it is shit, and AI is good at one thing that VCs love to fund, i.e. extraction of value through destruction of value.
I agree. The state of level of “basic” knowledge in developers are dropping.
The consequences are that developers can tackle basic tasks which are supported by the frameworks they use, but once something is not supported or straightforward they don’t know what to do and get completely stuck.
From society’s point of view, the usefulness and value of the task force decreases and important problems are not solved or aren’t efficiently solved.
They don't know what they don't know. (Devs not knowing where to start writing a piece of code that works on actual bytes and not strings or numbers is a classic example of this problem.) Sometimes they are good devs in their space and mean well, but they spend their working lives on a different level of abstraction and in a different problem domain. The danger with co-pilots is that devs using them will even not know what to ask for, because they will not know what it is they need to write. There is value in going through docs, writing your own code, making mistakes, seeing your code fail, fixing it. That's how you learn to spot mistakes made by AI, but I don't think that having a coding puppy and spending time fixing its mistakes is conducive to writing better code or improving developer productivity.
> We have an increasing number of people who "write code", but the number of people who can understand and review code is not increasing.
Something like this was happening with coding bootcamps as well. Lots of people who can code what they were directly taught but don’t have the basics to go outside of that.
Yes I agree. But good companies are aware of this fenomenon and these so called chatgpt programmers are considered juniors. There is always place for you as an experienced developer who understands memory management, io performance, stack executions, step debuggers, compile optimizers and much more.
> because it just lifted code from different webpages and recombined it like a massively hungover student waking up to realise he's supposed to hand in the assignment in one hour.
That’s CTO level coding right there, gotta tell bard about my great app idea.
> I asked Bard to write me a Python function to turn the US Constitution into a palindrome. Bard refused and gave ma a lecture on how the US Constitution is too important to be played with in such trivial fashion.
Jesus, they have coded pearl-clutching and finger-wagging lectures in my computers. You know, those machines that used to be brutally efficient at doing what you asked of them.
> those machines that used to be brutally efficient at doing what you asked of them.
In a way, that's a natural thing to happen. Computer systems are weird in that we have an extremely good model of how they work at a certain level: "processor will (almost) always process assembly exactly as it should, etc."
When computers first came around, this "computer does exactly as you say" thing was weird to people because there weren't really any such things before computers. People would make small mistakes in their code and wonder why the computer can't "just do the obvious correct thing like any sensible person".
However, as multiple complex software systems interact and build on top of each other, this ability to "understand everything" at one of these "perfect model" levels (e.g assembly, logic gates, state machines...) becomes useless. To work with them, fuzzy models (like those we use for other humans or animals) are more appropriate. Complex software systems will routinely not do the same thing when given the same command. They will change over time, break in incomprehensible und unpredictable ways, and, as the newest addition, argue with you instead of doing what you say. That development makes total sense. At least those systems can now "take a hint".
I'm all for moving to higher-level skills, when I was an adjunct professor I usually didn't put a huge emphasis on syntax unless I thought it was important to explain a concept, and I very rarely took off points for syntax.
That said, I don't know that I would allow AI for learning to code in the class. I do not consider "prompt engineering" to be a substitute for programming, and it's very easy to get AI to write some code and copypaste it, not understand anything it said, and be done.
Last year, when I was administering a test, I said that they were allowed to use any IDE they wanted, but they could not use any AI assistance. I am pretty convinced one student just copypasted my test questions into ChatGPT or BingAI and then copied the answers. I didn't have a way to prove it so I had to give him the benefit of the doubt and grade it assuming he did it honestly.
Before someone says "YOU SHOULD TEACH THEM TO USE THE AI TOOLS LOL THAT'S THE FUTURE!!!", stop. That's not true, not yet at least. Just because I can call an Uber doesn't mean I get to say I know how to drive. Just because I can go to McDonalds doesn't mean I get to say I know how to cook. I was teaching them how to write Java and Python, the goal of the class was for them to learn Java and Python, it wasn't learning how to copypaste into OpenAI.
You can't learn to play the guitar by reading books about it, or watching YouTube videos, or reading sheet music, or getting "personalized AI tips".
You can only learn to play the guitar by picking it up and spending a lot of time mucking about with it. There's a bit more to it, but this is really the core: you need to play and get that muscle memory and "feeling" for it. You need to "rewire your brain".
Coding, or any other skill for that matter, is no different. The only way to learn to code is to actually write code. Even if you fully understand anything ChatGPT gives you (which most students probably don't), that's no substitute for actually writing the code yourself.
I don't see how it's not hugely harmful for the development of students and junior programmers to use AI. Even if these tools were perfect (which they're not), you need to develop these basic skills of learning to read the code. You need to make mistakes and end up with crummy unmaintainable code, and realize what you did wrong. Etc. etc.
Even for senior programmers I'm not so convinced AI tools are actually all that beneficial in the long run.
Only when AI systems will be able to fully understand entire systems and full context and can completely replace human programmers will that change. I'd estimate that's at least 50 years off, if not (significantly) longer. Anything before that: you need someone who fully understands the code and context, in depth.
Having a vague, conceptual understanding of a topic is often confused with expertise. Also known as knowing enough to be dangerous.
What worries me is that AI code is getting complicated and harder to correct. I recently noticed that after my IDE generated some code I felt a palpable sense of dread.
I often have to go over it in detail because it makes subtle, hard to catch errors. It pulls me out of my flow. I am starting to dislike that effect and I am certain huge swaths of the (new) coding population will lack the skill and motivation to correct these things (as various earlier comments here show).
This same issue existed with Stack Overflow, but AI further lowers the barrier to be dangerous by throwing a brick on the gas pedal.
I’ve also found it makes otherwise knowledgeable and experienced people think they can do stuff without learning it. I’ve had several people on my team tell me they think Copilot will help them get up to speed and help out with some of the stuff I’m working on. So far none of them have done anything and I’m not sure how Copilot explaining a block of code is going to do anything for them. It’s already a very easy syntax to read, and they all already have coding experience in other languages. It’s a tool to let them think helping will be easy, so they volunteer, then do nothing because it isn’t the reality of the situation.
Coding, or any other skill for that matter, is no different.
Back in the early days of computing people wrote software by working it out on paper, encoding it on to punch cards, and then giving that program (deck of cards) to an operator who loaded them and ran the code. If it didn't work properly you would get back a print out of the 'debug' which amounted to a memory dump. You'd then patch your punch cards based on working out where you'd screwed up, and try again.
That was the computer software industry for about a decade before time-sharing, VDUs, etc.
People absolutely did learn to code by reading books and nothing much else. Access to computers was so restricted (because time to use a computer was shared between lots of people) there weren't any other options.
Heck, I learned a lot of early web stuff like TCP, HTML, etc reading books at my parents house when I was at home from uni and didn't have my computer with me, and that was the late 90s. Of course you can learn coding by studying the theory without practicing. It's just a lot less fun.
That is actively engaging your brain, just at a very slow pace with a very slow feedback loop.
Dijksta's "I had everything worked out before I wrote the code, because the computer didn't exist yet" is the same. It's like working mathematics out: that's not "just reading", it's actively engaging. Actively writing. Completely different from the passive consumption of AI output.
> Of course you can learn coding by studying the theory without practising. It's just a lot less fun.
No you can't. When you first started an editor or IDE you wrote some vague code-shaped junk that probably wasn't even syntactically correct, and didn't learn programming until you practised. Of course you need to learn some some theory from a book or instructor, but that's not the same as actually learning something.
Yep Knuth learned by reading an IBM manual with source code while sitting on a beach during summer vacation. Decades later systems hackers learned by reading illicit copies of Lions' Commentary on Unix w/Source
Exactly, it's the same with autonomous driving or any other activity (even art!) that has always required a human. While it can "look cool" and like it "understands the context" on the surface, when diving deeper it will _always_ misunderstand the nuance, because it doesn't have all of the lived human experience and reasoning.
Being even 80% (or 90 or even 95) there isn't enough - something will always be missed because it's only able to "reason" probabilistically within a narrow area not far away from the training data.
This is very well put. I think the next point of discussion is what fraction of "software developers" "need" to know how to code in the deep way you describe.
I don’t think the guitar analogy is a great one. You have to learn by picking it up because no expert plays the guitar using personalised AI. But there are expert programmers who use lots of AI tools, so AI is more of an important part of learning to code.
Doesn't matter what (some) experts use; it's about learning fundamentals. You can only become fluent by writing and reading code, at times struggling to do so. This is the only way to learn any serious skill in life, from programming to music to chess to woodworking to cooking to physics to ... anything.
I'd say that especially because assistants are not good enough yet, it makes sense to allow students to use them. LLMs will produce output that is messed up, students will spend time figuring out how to fix that and learn the language grammar this way.
I recently learned Swift & Metal this way - knowing Python and other languages, I kept pasting my pseudo-code, asking GPT to convert it into Metal/Swift, and then iterating on that. Took me a week or two to get the language without reaching for any sort of manual/tutorial. Speaking from experience - it would take me 2-4x as much time to learn the same thing otherwise.
If GPT was any better, I'd have no need to fix the code, and I wouldn't learn as much.
I want to emphasize that I'm not really a luddite here. I pay for ChatGPT (well, Kagi Professional), I use it daily, and I do think it's a very valuable learning tool. I even encouraged my students to use it outside of class if they needed help with any of the theory.
I use ChatGPT all the time for learning new stuff, but a) I already know how to program well enough to teach a class on it, and b) it's a supplement to doing stuff on my own, without AI help. I don't feel I learn that much from just copying and pasting.
Totally agree with your process though; correcting and arguing with ChatGPT is an insanely good way to learn stuff. It's really helped me get better at TLA+ and Isabelle proofs.
You can upgrade any family member to Kagi Ultimate plan for $15/mo and it will give you unlimited access to GPT-4, Claude 3 Opus, Gemini 1.5 Pro and other state of the models.
Yeah, they don't advertise it very well. It was actually a pretty good value add for me; I was already paying $10 for Kagi search, and $20 for OpenAI, so the $25 for both actually saved me $5/month.
I think you can upgrade to Ultimate at a per-account level for another $15/month.
EDIT: Accidentally wrote "pro" when I meant Ultimate.
> I recently learned Swift & Metal this way - knowing Python and other languages, I kept pasting my pseudo-code, asking GPT to convert it into Metal/Swift, and then iterating on that.
Significant here is that you knew other languages already. I've had the same experience with GitHub Copilot, but I'm cautious about recommending it to new learners who don't yet know the fundamentals.
All indications I've seen indicate that it's easy for people who already know the foundations of programming to use AI tools to learn new tech, and that can actually be the most effective way for them, but that it doesn't work nearly as well for people who don't have knowledge of other languages and frameworks to lean on.
> and that can actually be the most effective way for them
A lot of it is just a general lack of learning material for that demographic. For most languages, you get to choose between an introductory text that assumes you've never even heard of a "pointer" or an advanced text that is meant as a reference for an experienced developer of the language.
There's not really a lot of stuff out there to teach someone who already knows another language, who just wants to know the syntax and common idioms in that language. For example, a C programmer can easily pick up Python, learn some syntax, and go nuts -- but they just won't know about stuff like list comprehensions or dataclasses unless someone points it out. They'll write Python code that reads like C, not like Python.
It's similar with maths - calculators are invaluable, but being actually good at advanced maths just NEEDS you to grind through basic calculations so you adopt those basic "pattern recognition" skills on how numbers work, can be broken down and swappe out.
Yup. I gave a lab homework to an SE class of 42. After reading through at least 12 LLM generated, similar looking, and wrong code submissions, I've cancelled the whole lab assignment. And students complained in class evaluations about it.
An alternative is to keep the lab assignment but grade them on a final exam. Then it is in their interest to do the lab assignment by themselves rather than with an LLM.
>LLMs will produce output that is messed up, students will spend time figuring out how to fix that and learn the language grammar this way.
I’ve never met anyone who prefers debugging over writing fresh code. Did LLMs just automate away the fun part and leave us with the drudgery? That sounds like a horrible way to learn. All work and no play…
It also would mainly teach how to get good at spotting those little mistakes. But without context from writing code, it seems like it would be harder to pickup on what doesn’t look right. They’d also miss out on learning how to take a problem and break it down so it can be done with code. That’s a foundational skill that takes time and effort to build, which is being farmed out to the LLM.
With super difficult algorithmic code, for me, it works best to just start writing and then fix, than to get stuck not knowing how to write.
Algorithmic code aside, I’m learning cuda now, and I just told gpt to write me some code and I debugged it then (with gpt’s assistance as well). It took me an hour to produce something that I expected it would take 2-3 days otherwise.
As for picking up what doesn’t look right - if it doesn’t work, you can pinpoint place where sth is messed up using a debugger/prints, and then you learn how the code works on the way.
I remember that when I was learning to program 30 years ago was the same - I rewrote pong game from a magazine, then kept changing and messing up things until I learned how it should work.
It’s like having infinite stack overflow examples (and explanations) to sample from and iterate on. It’s unlikely any one of them will solve a complex problem you’re working on but by reading and experimenting with enough of them eventually you grok it.
I think it would be quite mentally difficult to be successful at writing device drivers (any hardware device, many different fields) if a person had the slightest reservation about debugging. I've got past a lot of blockers by creative trial and error and making it work was the satisfaction driver. I actually enjoy the making things work part. I have had employees who felt the same, one of my jobs was telling management to fuck off while the person worked on the bug, for days without result if need be[1]. The "just writing code" part is how one actually gets to the interesting part where I earn my salary.
I have no idea how an LLM could help with that process.
[1] I successfully found a bug in a CFD solver (not a device driver) in the early days of templated g++ only after 2 weeks of fairly grueling half time work. Missing ampersand in an argument list. Believe it or not I was very happy and doubled down on the code. However! I once fucked up a job situation where my MPI-IO driver failed because I had a very subtle linux RAID hardware bug that I could not find after weeks of effort. I found it later, too late. That truly sucked. I really don't know how LLMs could possibly help with any of this.
In 1993 or so g++ couldn't do it, but I suspect that all c++ compilers today would. Then why would I need anything else? However the point is that when dealing with proprietary hardware devices you occasionally get a situation where the incantations as "documented" should work, but they don't, and the usual software process diagnostics are silent about it. Some domain specific experienced creativity is required to coax a response to begin finding the illness. Yes you can pay for support and escalate but small shop management sometimes is a bit hesitant to pay for that.
I am very curious how an LLM is supposed to be trained on situations whose context does not exist on the open internet.
Perhaps only after an assignment where prompts and answers are shown, and most of the answers look plausible but have a bunch of subtle mistakes, and students need to determine which ones are wrong and why.
I recently learned javascript this way - knowing English and other languages (Thai, Mandarin), I kept pasting my pseudo-code, asking GPT to convert it into javascript, and then iterating on that. Took me a week or two to get the language without reaching for any sort of manual/tutorial. Speaking without experience - it would take me 2-4x as much time to learn the same thing otherwise.
:) My story is not entirely true, but close. My point being llms are learning language and logic (mostly English currently). Programming languages are just languages with logic (hopefully).
And if you think the ability to shorten meaning of a complex idea is exclusively the purview of code, think of a word like "tacky" or "verisimilitude"- complex ideas expressed in a shorter format, often with intended context with significant impact on the operations in the sentence around them.
Maybe I’m dumb but I’ve never found a single textbook, manual or tutorial that taught me everything I need to know about a subject. I always have to make a circuit of multiple sources and examples before I understand fully.
I guess what you really learned is that the Apple-specific ecosystem technologies are so crappy, you always use a middleware to defeat them. Either Unity, React Native, or in this case, a 170b parameter super advanced LLM.
There's just no getting through to those Apple guys to stop making their own thing and being a huge PITA.
I don't do Swift or JS/React professionally (well, tiny bit, but I'm not a "React professional"). However, I've made non-toy Swift/SwiftUI* (~7k loc) and React (~4k loc) project.
Imho Swift/SwiftUI is way more of a joy to work with. In particular, SwiftUI is much easier to visually parse whereas JSX is noisy and IMO hideous. I also ended up needing to bring in way more third-party dependencies for JS/React, something I hate doing if avoidable**.
* It is my understanding that many iOS professionals prefer UIKit bc SwiftUI is still missing stuff. I only ended up needing to fall back to it once or twice.
** ClojureScript/Reagent do solve most of my issues with JS/React (by essentially replacing them). Clojure is well-designed with a great standard library (so less need for dependencies or implementing stuff that should be in a standard library). Hiccup is way preferable to JSX.
The assistants might be good enough to produce homework level or test level answers to contrived questions. But then utterly fall over in a real world codebase.
I have yet to find an AI tool or combination of AI tools that can hold my hand and give me correct solutions for my coding problems.
I have worked exclusively with React/Next, Node, and Mongo for years until I took over a legacy code base in October.
I had to teach myself Python/Flask, Vue, and Laravel without any kind of documentation or guidance from the previous developers or anyone else in my company.
AI has been able to help me get a bit further down the road each time I run into an issue. But AI hasn't been able to hand me perfect solutions for any of my problems. It helps me better understand the existing code base, but thus far none of the answer provided have been 100% accurate.
Real world tasks don't fit in a chat window, and using AI is barely an improvement over not using it. Just from one of my latest LLM coding adventures: it gets versions wrong, imports wrong, messes up versioning when learning documentations, resulting in many wasted hours. It's ok to fill in boilerplate, not ok to help you do things you couldn't do without AI.
If anything, it's like cooking with kids, not like going to McDonalds. I'd give extra points if they can solve the task with LLM in time. The hardest parts in programming is finding subtle bugs and reviewing code written by others - two tasks that LLMs can't help us with.
If they can easily solve the tasks with LLMs then it is a legitimate question to ask if you should be teaching that skill. Only common ones can be solved that way though. Why don't you give them a bugged code to fix, that way LLM inspiration is not going to work, if you check first to make sure LLMs can't fix the bug?
I don't think I agree with your last paragraph. ChatGPT is getting better and better, I have reason to think it won't continue to keep incrementally improving. As such, I think allowing students to use it to answer coding questions is going to make it so they don't actually understand anything.
I said in sibling thread that I'd be fine enough with having a class like "Software Engineering Using AI" or something, but when the class is specifically about learning Object Oriented programming and Java and Python, I do not think having heavy use of ChatGPT is a good idea.
Also, not all the questions were pure coding, I had some more conceptual questions on there, and ChatGPT is really good at answering those.
If you throw the same question at it 15 different ways, it can eventually give you ideas for optimizations that you probably wouldn't have thought of otherwise. It knows parts of APIs that I've never used. ByteBuffer#getLong, ByteBuffer#duplicate, StringBuilder#deleteCharAt, RandomGenerator#nextLong(long)
> I do not consider "prompt engineering" to be a substitute for programming
It's also not nearly mature enough for learning it be a good ROI in a degree program. Community college or adult education class? Maybe. Bootcamp track? Sure, I guess. Novelty elective for a few easy credits and some fun? Totally.
But if I'm a student in the midst of a prolonged, expensive program spanning years, learning how to coax results out of today's new generative AI tooling is not preparing me very well at all for what I can expect when I try to enter the workforce 2 or 4 or 8 more years. The tools and the ways to interface them, "Prompt Engineering" or whatever else it's called will inevitably evolve dramatically between now and then. So why am I learning it while I'm still deep in my academic bubbles? And what are prospective employers getting from a degree that focused heavily on some now-defunct and dated techniques? My degree is supposed to mean that I've learned foundational material and am ready to be productive on something, but what that mean when too much of what I've learned is outdated?
If there were a class called "Utilizing AI for software engineering" or something, even at the university level, that wouldn't bother me.
What bothers me is that people have told me that I should just allow AI for literally everything because it's the future and you should be teaching them the future or something, but I think that's kind of dumb. In those classes, the goal was for them to leave having some competence in with Python, Java, and Object Oriented programming, and I firmly do not believe you can get an understanding of that just by copying and pasting from ChatGPT, and I think even Copilot might hinder the process a bit.
To be clear, I love ChatGPT, I use it every day, it's a very valuable tool that people probably should learn how to use, I just don't feel that it's a substitute for actually learning new stuff.
> What bothers me is that people have told me that I should just allow AI for literally everything because it's the future and you should be teaching them the future or something,
I wonder what "people" told you that. My personal experience is that such advice usually comes from people who understand neither AI nor what I teach. Most of them are university administrators of sorts, and AI is a problem for them more than for me.
Introductory classes teach skills that can be performed reasonably well by AI. Those skills are the foundation you need to build higher level skills. Just like kids need to know how to read to be functional in society and in their later classes, despite screen readers doing an excellent job.
When I teach a foundations class this is my focus, and I don't fool myself or my students into thinking that they will be using those skills directly, but I try to convey the idea that the skills pervade through much of what they will later learn and do.
However that means that I cannot force students to learn. They can cheat, and it's easy, and I prefer spending my efforts on helping the learners than catching the cheaters.
The university administrators, however, are in the business of selling diplomas, which are only worth what the lowest common deminator is worth. So cheaters are a big problem for them. Typically for such people, they just bury their heads in the sand and prefer to claim that teaching students to use chatGPT (that lowest common denominator) is where the value is.
I'm not sure if you're implying that I'm making shit up by putting "people" in quotes, but here's at least a little evidence. [1] [2] [3]
Otherwise it's been with in-person conversations and I didn't record them, there's a spectrum to how completely they suggest I allow AI.
Everything else you said I more or less agree with. Obviously if someone wants to cheat they're going to do it, but I feel that until we restructure the job market to not take GPAs as seriously (which I think I'd probably be onboard with), we should at least have cursory efforts to try and minimize cheating. I'm not saying we have to have Fort Knox, just basic deterrence.
I'm not an adjunct anymore, partly because I took stuff way too personally and it was kind of depressing me, partly because it was time consuming without much pay, but largely because I realized that most universities are kind of a racket (particularly the textbook corporations are a special kind of evil).
I wasn't implying anything. Because your use of the word "people" left the context very vague, I was just cautiously trying to not speak in your name when discussing the people who are pushing me to teach students how to use chatgpt.
With respect to deterring teaching I totally agree that we should go for it. There are ways to mitigate the value of cheating and ways to promote the value of learning, both of which are deterrents. Personally I like having lots of small tasks that build and follow on one another. If the student is working and trying to learn it makes sense and we get to reinforce the high level skills that matter. If the student is cheating it should become increasingly harder to keep a consistent story.
However if we turn this into a cop and robbers game that's what we're going to get.
As for the focus on GPA I think that the tide is turning. Employers need to find an alternative that doesn't eat their time.
And yes universities are rackets and aren't good value employers. They don't even offer job security anymore.
If it’s an introductory course I can see how using generated code would be harmful but I’m guessing the students can’t since they don’t have the experience needed to make a distinction between good and bad code but perhaps that’s the thing you could teach them about LLMs by having a class where they are given an “easy” problem that you might normally assign that an LLM can solve and then also a hard problem that that it will fail to do properly and let the students see the difference for themselves when they have it generate code for both. That may provide valuable insight.
I agree. I attempted to use Copilot more this week for a couple projects at work I had to get done quickly. I hadn’t been using it at all for the past few months, but when a co-worker asked for help getting it working it reminded me to take another look. These particular things seemed better suited for Copilot than some of the other work I had.
I found that having a solid foundation was critical for knowing if something would work, and even just to write a prompt that was semi-decent.
At one point I asked for what I wanted, but it kept only doing half of it and I could tell just by glancing at the code it was wrong. I then had to get very, very specific about what I wanted. It eventually gave a correct answer, but it was the long annoying option I was trying to avoid, so it didn’t change my end result, and I don’t even think it saved me any typing due to all the prompts to get there. It just gave me some level of confirmation that there wasn’t an obvious better way that I’d be able to find quickly.
Knowing syntax is a side-effect of having written a minimum number of LoC, so grading syntax is in effect rewarding experience. Similar to how a large vocabulary is a side-effect of having read a lot.
To me, exams are taken in halls, written on paper, proctored, under deadline. Points may be deducted for syntax mistakes or unclarity as the examiner wishes.
Separately, home work is graded in ways that already makes cheating pointless; usually for 1) ambition/difficulty in the chosen problem 2) clarity in presentation and proofs, argued in person to TAs.
LLM should have no bearing on any part of education (CS or otherwise) unless the school was already a mess.
I dunno I wasn't all that familiar with Python (I'd done a Django project a decade ago) and I picked Python for a recent project writing some control code for some hardware connected to a Linux box and I have learned quite a bit about Python by doing it.
Perhaps the lesson is about how to read and evaluate code and how to test code. If students get good at how and when to spot errors and how to construct test scenarios that ensure the code is doing what it should then perhaps that will lead to even higher quality code than if they were learning, producing bugs, and then learning how to evaluate the code and test what they had written.
> it's very easy to get AI to write some code and copypaste it, not understand anything it said, and be done.
This is why I think that the old way of having tutorials instead of tests is vastly superior. When you are in a tutorial group (typically five or fewer students) and your tutor asks you to explain something to the other members of the group you can't hide behind an AI, a textbook, or even your own notes. Your lack of preparedness and understanding is made abundantly clear.
> Last year, when I was administering a test, I said that they were allowed to use any IDE they wanted, but they could not use any AI assistance. I am pretty convinced one student just copypasted my test questions into ChatGPT or BingAI and then copied the answers. I didn't have a way to prove it so I had to give him the benefit of the doubt and grade it assuming he did it honestly.
If I were a teacher, I would have asked the popular bullshit generators to generate solutions for the test questions, and if a student’s solutions were very similar, I’d have them do some 1:1 live-coding to prove their innocence.
I did actually try and get ChatGPT and BingAI and Google Bard to generate it. I didn’t know about Claude at the time.
The issue is that the thing he submitted was correct, so I was going completely off “vibes”; I never got it dead-to-rights with AI generating a one-for-one match. It got pretty similar, and ChatGPT text does have kind of a recognizable style to it, but I didn’t feel comfortable reporting a student for cheating and risking them getting expelled if I wasn’t 100% sure.
I might have tried to get him to do a one on one coding session, but this was the final exam and literally three hours after it I had to fly to the UK for an unrelated trip for three weeks. Grades were due in one week, so I didn’t really have a means of testing him.
> Just because I can call an Uber doesn't mean I get to say I know how to drive. Just because I can go to McDonalds doesn't mean I get to say I know how to cook.
I hear what you are saying but where does it end? Is Hello Fresh cooking? Is going to grocery store cooking or do you need to buy it direct from a farmer? Do you need to grow the food yourself?
Is renting a car “driving”? Is leasing a car “driving”? If you can’t fix a car and understand how it works are you really driving?
Yes, most of those questions are ridiculous but they sound the same to me as some complaints about using LLMs. Those complaints sound very similar to backlash against higher-level languages.
Python? You’re not a real developer unless you use C. C? You’re not a real developer unless you use assembly. Assembly? Must be nice, unless you’re writing 0’s and 1’s you can’t call yourself a developer. 0’s and 1’s? Let me get out my soldering gun and show you what it takes to be a real developer….
So many strawmen. Following a recipe, even with the ingredients coming from a meal kit, is still cooking. If you’re behind the wheel, you’re driving, no matter who owns the car or if you can perform any maintenance.
A C developer working in notepad.exe is a real developer, as is a Python developer working in PyCharm and using the standard IDE features to improve their productivity. Someone blindly copy-pasting output from a LLM is not a developer.
Good, I’m glad you grasped the point of my comment. I was talking about the absurdity of people “gatekeeping” programming. Those arguments are just as silly as people saying using an LLM (in any capacity) is wrong and means you aren’t programming anymore.
Yes, blinding pasting code from an LLM does not a developer make. However that’s not what I suggested. I believe LLMs can be useful but you need to understand what it’s generating. The same way that SO is useful as long as you understand the code you are reusing (ideally modifying and reusing instead of a straight copy/paste).
> I believe LLMs can be useful but you need to understand what it’s generating
That was never in dispute and not disagreed with in what I wrote. I didn’t say “using an LLM is wrong in any capacity” nor is that implied by anything I wrote. I use ChatGPT daily, I even told students they should use it if they needed help with understanding concepts outside of class”. I didn’t “gatekeep” programming, I just said they couldn’t use AI during an exam.
> I believe LLMs can be useful but you need to understand what it’s generating.
Yeah, if only we had some way of EXAMining if the students understand what they were generating. Like, crazy idea, maybe we could have some kind of crazy test where they aren’t allowed to use ChatGPT or Copilot to make sure they understand the concepts first before we let them have a hand-holding world.
I never said you said that, I said some people. I never said you should or shouldn’t use LLMs on exams, I have no idea why that’s being brought into this conversation.
I can only assume you’ve lost the thread and/or think you’re replying to someone else. This will be my last reply.
Your first post was in response to me claiming that going to McDonalds wasn’t cooking or getting an Uber isn’t the same as knowing how to drive. Specifically, they you were responding to was an anecdote about a student cheating on an exam by using ChatGPT, at least it appeared that way to me. It’s not weird to think your response was in regards to that.
Maybe I misread your intent there, I just got a vibe that you were defending the use of LLMs during exams since that was the thing you were responding to. Apologies if I misread.
> I am pretty convinced one student just copypasted my test questions into ChatGPT or BingAI and then copied the answers. I didn't have a way to prove it so I had to give him the benefit of the doubt and grade it assuming he did it honestly.
I wonder if a more AI-resistant approach is to provide poor code examples and to ask students to improve it and specifically explain their motivations.
They can’t just rely on code output from an LLM. They need to understand what flaws exist and connect the flaw to the fix, which requires (for now, I think) higher-level comprehension.
I would honestly think it doesn't matter. Students, like professionals, should be given the world and all of it's resources to generate a solution. If I can just punch the question into AI and get a valid answer then maybe the question should be modified.
If you always defer to a computer then you never learn to disagree with the computer, you become just as much of an automaton as it.
I love LLMs for coding but only because I have the experience to instantly evaluate every suggestion and discard a double-digit percentage of them. If an inexperienced programmer uses Copilot they aren’t going to have the confidence to disagree and it will stunt their development.
Does that really parse? There's plenty of things that have definite answers that are a Google search away, but we still expect things be able to do.
Did we just stop teaching kids basic spelling because spell check was built into MS Word? No, of course not, because even though spelling has been a more-or-less solved problem for people who already know how to read for the last thirty years, having kids learn to spell helps with their actual comprehension of a subject.
Also, if they do not learn the fundamental concepts then they will be able to differentiate a good solution from a bad solution. There's a lot of really shitty code that does technically accomplish the goal that it sets out for, and until you learn the fundamentals that fact won't be clear.
Do you also think that primary school students should not be taught, say, multiplication? Because they can, of course, use a calculator. No need to ask them what 3 times 4 is.
Early 2023, when everyone started using chat GPT for coding, I thought it would be a big boost because it enabled us to quickly employ a large number of people in a project regardless of language or framework.
An year into the project I am forced to revise my opinion. When browsing my code-base I often stumble in abstruse niche solutions for problems that should not have existed. It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.
Having an oracle that knows all answers is useless if you don't know what to ask.
> It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy.
Isnt this what code reviews are for? I catch a decent amount of code that looks AI generated. Typically, some very foreign pattern or syntax that this engineers never used nor is common in the codebase. Or something weirdly obtuse that could be refactored and shows a lack of understanding.
Normally I ask something like, "Interesting approach! Is there a reason to do it this way over (mention a similar pattern in our codebase)?" or if it's egregious, I might ask, "Can you explain this to me?".
This feel similar to early career engineers copy pasting stack overflow code. Now its just faster and easier for them to do. It's still fairly easy to spot though.
You would. And if the juniors don't really _write_ code in the languages you use, if they don't make mistakes and do research themselves, it'll take them a lot longer to learn them sufficiently to be able to do those reviews.
Listening to people talk in Japanese, I picked up enough to have some idea what they're talking about. I can't speak the language beyond some canned phrases. I certainly wouldn't claim to know Japanese. And I definitely wouldn't get a job writing books in Japanese, armed with Google Translate.
For programming, my solution to this right now is lots of pair programming. Really gives you a good idea of where somebody is at, what they still need to learn, and lots of teaching opportunities. I just hired a junior and we spend about 8 hours a week pairing.
Yeah. Are we assuming 100% of the people working on this project are incompetent? I thought OP was talking about new people joining a project producing low quality code that was likely generated by AI.
Code reviews, done by existing more senior members of the project/team, should prevent this. If it isnt then the project has more issues than AI generated code.
There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down.
I would agree with there being no substitute for doing something correctly the first place, but I would argue that in this case the "first place" is hiring/training better so your employees don't try to throw unrefined AI shit at the walls, but instead take the output of AI and hone it before creating a PR.
If you have an engineering culture that doesn't emphasize thorough code review (at least of juniors, leads and architects emergency-pushing is a different story) that's a problem. In addition to catching bugs, that's a major vector for passing on knowledge.
I agree with you, no doubt there. But still, technology often offers the path of least resistance. So even if you want to hire better people, what of all the people who will grow up training themselves with Copilot? Yes, you can filter them out but it will be harder and harder to find good people. And then, companies will want to go after the bottom line: what if they can get a bigger and more complex product out there with AI assistance but with code that is more unreliable in the long term?
I tell you, it's spiralling out of control. Besides, even if you're doing the hiring, you may not have control if there is a profit motive. Us technical people cannot control the race to the bottom line, especially over the period of decades.
A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.
Code review isn’t inspection, that would be testing - whether automatic or manual.
Code review is the feedback mechanism that allows quality control to improve the process, teaching the engineers how to do it right in the first place.
Sure. If its generating 85% code that passes a code review and 15% of code that doesnt... I'll take those percentages. That is certainly higher than the average early career engineers pr.
If you're trying to do things efficiently you can't afford not doing code reviews. Only well-funded organizations can afford to write the same code twice after the first attempt has to be thrown out.
I work at a FANMG company and see the value of code reviews for th code we're working on. But, I shipped 20 commercial products before that with no code review, just play testing (games). Our velocity without code review was easily 2x what it is with. For those projects I think that would still be true whereas I'd never suggest getting rid of code review at my current job.
The point being I think it depends on the project, it's size, audience, lifetime, etc whether or not code review is a win. my gut says it usually is a win but not always
At the one company I was where we did code reviews, they took at most one hour per day. The reviewer would go through the changes, ask for explanations when they didn't understand something, suggest small modifications to fit the codebase's style/structure/vibe, and move on.
They definitely didn't take 50% of our time. Half my time wasn't even spent coding, it was spent thinking about problems (and it was a workplace with an unusually high coding-over-thinking ratio, because of its good practices).
> At the one company I was where we did code reviews, they took at most one hour per day.
> They definitely didn't take 50% of our time.
Imo introducing code reviews might very well decrease the velocity by 50% even if the actual time spent on code reviews is much less.
I see at least two reasons for this: 1) increased need for synchronization/communication 2) increased subjective frictions and mental overhead (if only for task switching).
Code reviews are not special in this respect, similar effects can be caused by any changes to the process.
Thorough code reviews are expensive. When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing.
Less time may be required if you have good rapport and long work relationship with the person, but I would say halving productivity sounds about right if we are talking mandatory reviews.
Using an LLM to generate code means having to do the same. The longer the autocompleted text, the more careful you have to be vetting it. Lower predictability, compared to a person and especially one you know well, means that time requirement would not go down too much.
> When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing
Agree 100%, even when LLMs aren’t involved.
Certainly not all code reviews are like this—in many cases the approach is fundamentally sound, but you might have some suggestions to improve clarity or performance.
But where I work it’s not all that rare to open up a PR and find that the person has gone full speed in the wrong direction. In those cases you’re losing at least a morning to do a lot of the same leg work they did to find a solution for the problem, to then get them set off again down a better path.
Now in the ideal case they understood the problem they needed to solve and perhaps just didn’t have the knowledge or experience to know why their initial solution wasn’t good; in such a scenario the second attempt is usually much quicker and much higher quality. But introduce LLMs into the mix and you can make no guarantees about what, if anything, was understood. That’s where things really start to go off the rails imo.
> in many cases the approach is fundamentally sound
Sure, but I mean you still have to determine whether it is sound or not in the first place; i.e., you must know how to solve the problem. You can only evaluate it as right or not if you have something to compare against, after all.
If you have worked with a person for a while, though, I can see that you can spot various patterns (and ultimately whether the approach is sane) faster thanks to established rapport. Not so with LLMs, as you probably agree.
I mean, it's something we did once per day, usually towards the end of the day. This was pre-Covid, so our manager was in the room next to ours and we could just pop by his office to tell him we were ready for today's review.
It really doesn't need to be a massive amount of task-switching. And the benefits were obvious.
It probably also depends on the people that you're working with. I can easily imagine the velocity plummeting when the person reviewing your code loves to nitpick and bicker and ask for endless further changes with every next round of alterations that you do.
Doubly so if they want to do synchronous calls to go over everything and essentially create visible work out of code that already worked and was good enough in the first place.
I'm not saying that there aren't times like that for everyone occasionally, but there are people for whom that behaviour is a consistent pattern.
OMG this. Nitpicking code reviews are the worst. I've actually instituted a rule that code must be formatted with an opinionated formatter and linted, and whatever the linter/formatter decide is right, and no formatting/linting comments are allowed on code review to avoid this sort of bullshit review comment.
Code review matters when you have a product that has an indeterminate lifespan. When new people come on, they need to learn the codebase and code review both helps ensure a codebase that is easy for new devs to understand (because hard to understand code doesn't survive review) and also it provides an onboarding experience as new devs can be coached during code review.
"One and done" products like games where you're mostly fixing bugs and releasing new content without major overhauls to the underlying code probably only need code reviews for code that is complex/clever where bugs are likely to be hiding, beyond that QA is fine.
Not just thrown out, but also the customer relations when the bugs affect customers, the investigation to find the cause of the bugs, the migrations to fix any bad/corrupted data because of the bugs. Code review is a far cheaper way to catch errors, bad code, footguns and places where bugs might creep in than letting them into production and having them impact customers. Code reviews also help keep code tidier and cleaner which makes future understanding and changes easier. Small companies can’t afford to omit code reviews.
Can you elaborate on some scenarios you are describing? The only organization I can imagine that can't do code reviews would be organizations of 1. I don't even see how performing code reviews could ever be a net negative in terms of money nor overall productivity.
Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
usually 'organization' excludes projects consisting of just one person, and if there are two people working on a project that involves writing code, the cheapest amount of that code for both of them to be familiar with is never zero, though it is often less than 100% of all the code they write. so i don't think that 'not all organizations can afford code reviews' is true, interpreted in the usual way. maybe you can be more specific about the scenario you're thinking of?
As mentioned in the other comments, code reviews are typically not done in the game industry, for example, or other write-once, never read scenarios. Very early stage startups typically don’t have formal code reviews either.
while that is true, and some of it is rational, other times it's just another stupid way people waste money and fuck up their projects. because even in a startup or a game there is code that people waste time debugging alone, or waiting for the one guy who knows it to get back from vacation, or out of bed
I recently was stumbled upon a code change done by a colleague who just followed the suggestion as is from Copilot, where it recommend `func (string, string) (error, struct)` whereas everywhere in the code we use `func (string, string) (struct, error)`
When I asked him what prompted him to do that, he said copilot suggest it so I just followed. I wonder if you could hijack copilot's results and inject malicious code as many end users does not understand lot of the niche code it generates sometimes, you could manipulate them to add the malicious code to the org's codebase.
The insidious thing is that quite possibly, it "feels" there should be a couple of bugs like that because all the codebases it was trained on had a few.
It might even take the context of the typos in your code comments, and conclude "yeah, this easy to miss subtle error feels right about here".
That is a problem but thankfully there is a lot of attention on training with highly curated high quality data right now because it is a known problem. Buggy code is still valuable training data if you use it as part of a question and evaluate the response against a corrected version of the code when training the model to perform a task like bug fixing.
It's definitely possible to inject malicious code that humans don't spot, there was a whole competition dedicated to humans doing this in C well before LLMs: https://en.wikipedia.org/wiki/Underhanded_C_Contest
Now I'm wondering, can you put in a comment which the LLM will pay attention to such that it generates subtle back-doors? And can this comment be such that humans looking at the code don't realise this behaviour will be due to the comment?
Almost two decades ago, I saw a junior colleague (this was Java) try to add "static" to almost all of one specific class' methods and members, i.e. making them global instead of attached to each class instance . Obviously this completely broke the software, albeit it did build. When questioned during review, my colleague just shrugged and said "Because Eclipse suggested to do so".
Apparently, if you tried to access a class member without specifying a class instance, one of Eclipse's "auto-fix-it" suggestions was to make all members of that class static, and he just followed that suggestion blindly.
> A year into the project I am forced to revise my opinion. When browsing my code-base I often stumble in abstruse niche solutions for problems that should not have existed.
This is a widespread problem regardless of AI. Hence the myriad Stack Overflow users who are frustrated after asking insane questions and getting pushback, who then dig their heels in after being told the entire approach they’re using to solve a problem is bonkers and they’re going to run into endless problems continuing down the path their on.
Not that people aren’t on too fine a hair trigger for that kind of response. But the sensitivity of that reaction is a learned defense mechanism for the sheer volume of it.
> Hence the myriad Stack Overflow users who are frustrated after asking insane questions and getting pushback, who then dig their heels in after being told the entire approach they’re using to solve a problem is bonkers and they’re going to run into endless problems continuing down the path their on.
The problem is, SO can't tell someone who asks an insane question from someone who asks the same question but has constraints that make it sane. *
So in time, sane people during unusual stuff stop asking questions and you're left with homework.
* For example, "we can't afford to refactor the whole codebase because some architecture astronaut on SO says so" is a constraint.
Or another nice one is "this is not and will never be a project that will handle google-like volumes of data".
> The problem is, SO can't tell someone who asks an insane question from someone who asks the same question but has constraints that make it sane.
Stack Overflow is not there to help you solve your use-case. It's there to create a body of knowledge that everyone can refer to. You need to spell out your specific reasons so that the question and answers become useful to others.
A lot of the friction on Stack Overflow comes from people thinking it's a free help website rather than an attempt to create a collaborative knowledgebase.
I think a lot of SO's problems comes from it trying to pretend it was building a knowledge base when it was really only ever a question and answer site.
It was nice to use early in its life when every answer was fresh. These days, the stale knowledge outweighs the fresh and the top answer is usually no longer correct.
If SO wants to be a knowledgebase, it needs to to be redesigned as it's current structure is not well suited to it.
How ‘bout remembering a million tokens? I’m not feeling too confident about that. Basically my only moat, if there is one, is that I’m able to rely on higher level cognition which LLMs don’t yet have, rather than just on associative memory alone.
I might, actually. Think of where electric cars were six years ago — 2018. Not much has changed. Or, at least, there are still fundamental problems to be solved.
In the same way I can imagine that by 2030 LLMs will still have memory problems and hallucinations. Although I’m sure by then we’ll have something better than pure LLMs.
I've heard claims that context without forgetfulness has already been reached 2 months ago, but as I'm not a domain expert I don't trust that I can differentiate breakthroughs from marketing BS, and I definitely can't differentiate either of those from a Clever Hans: https://arstechnica.com/information-technology/2024/03/claud...
I work in this field, so here's a comment with higher signal-to-noise ratio than you'll commonly find on HN when it comes to LLMs: notice how the demo use cases for very long context stuff deal almost universally with point retrieval, and never demonstrate a high degree of in-context learning. That is not coincidental. The ability to retrieve stuff is pretty great and superhuman already. The ability to reason about it or combine it in nontrivial ways leaves a lot to be desired still - for that you have to train (or at least fine tune) the underlying model. Which IMO is great, because it neatly plugs the gaps in human capability.
So do I, and worse. Look, all I’m saying is I’m thankful for this crutch that helps me deal with the limitations of my associative memory, so as long as it can’t think and can’t replace me entirely
bacopa and lions mane was night and day for my limited memory. But, the obvious, writing simpler code, keeping a notebook while working, etc, and really spending time breaking down problems into much smaller scopes, while simultaneously keeping copious notes on where I am in a given process helped immensely with dealing with peanut brained memory. sure, it's not quick, but my work is usually very readable and understandable for the future reader. I'm not convinced that a tool to help me overcome that memory barrier would actually help me write better code, maybe just write worse code faster. Of course, that's probably the corporate goal though.
Lions mane didn't seem to do much for me. My memory is actually not that bad, though certainly I have seen people with _much_ better memory than mine. I still maintain contact with some of them I met in college, and then later in the various companies. It's just that I deal with so much information and so many streams of it, even writing it down would be a massive chore.
I would pay for a pre-packaged system which could _locally_ and _privately_ make sense of all the emails, PDFs, slack messages, web pages I saw, other documents shared with me, code, etc etc, and make it all easily queryable with natural language, with references back to the sources. Sooner or later someone will make something like that.
The context heads of a LLM are more analogous to the sort of processing that goes on in, e.g., Brocca's Area of your brain as opposed to working memory. You can't have anything analogous to working memory as long as LLMs are operating on a strict feed forward basis[1]. And the fact that LLMs can talk so fluently without anything like a human working memory (yet) is a bit terrifying.
[1] Technically LLMs do have a forget that last toke and go back so I can try again operation so this is only 99% true.
I think humans have better general recall whilst lacking any kind of precision. After reading an entire book, I definitely can’t replicate much (if any) of the precise wording of it, but given a reasonably improbable sentence I can probably tell with certainty that it didn’t appear. LLMs are probably much more prone to believing they’ve read things that aren’t there and don’t even pass a basic sanity check, no matter how long the context window.
>> Having an oracle that knows all answers is useless if you don't know what to ask.
That is a great point. The issue of not asking the right questions has been around as far as I can remember but I guess it wasn't seen as the bottleneck because people were so focused on solving problems by any means possible that they never had to think about solving problems in a simple way. We're still very far from that though and in some ways we have taken steps back. I hope AI will help to shift human focus towards code architecture because that's something that has been severely neglected. Most complex projects I've seen are severely over-engineered... They are complex but they should not have grown to hundreds of thousands of lines of code; had people asked the right questions, focused on the right problems and chosen the right trade-offs, they would have been under 10K lines and way more efficient, interoperable and reliable.
I should note though, that my experience with coding with AI is that it often makes mistakes for complex algorithms, or it implements them in an inefficient way and I almost always have to change them. I get a lot of benefit from asking questions about APIs and to verify my assumptions or if I need a suggestion about possible approaches to do something.
How do you know it was "AI-fuelled"? And what makes it a "frenzy"?
People have been committing terrible code to projects for decades now, long before AI.
The solution is a code review process that works, and accountability if experienced employees are approving commits without properly reviewing them.
AI shouldn't have anything to do with it. Bad code shouldn't be passing review period, no matter if it was AI-assisted or not. And if your org doesn't do code review, then that's the actual problem.
> and accountability if experienced employees are approving commits without properly reviewing them.
You’re putting the entire responsibility on senior employees. So we need much more of them. In fact, we don’t need juniors, because we can generate all possible code combinations. After all, it’s the responsibility of the seniors to select which one is correct.
It’s like how hiring was made crap by the “One-click apply” on LinkedIn and all other platforms. Sure it’s easy for thousands of people to apply. Fact is, we offer quite a good job with high salary, and were looking for 5 people. We’ve spent a full year selecting them, because we’ve receive hundreds of irrelevant applications, probably some AI-generated.
It’s no use to flood a filter with crap, hoping that the filter will do better work because it has a lot of input.
This is almost a special case of the very similar (and common) argument that ‘AI-generated misinformation isn’t bad; we’ve always had misinformation’.
The answer is also the same.
Volume. AI makes it trivially easy to generate vast amounts of it that don’t betray their lack of coherence easily. As with much AI content, it creates arbitrary amounts of work for humans to have to sift through in order to know it’s right. And it gives confidence to those who don’t know very much to then start polluting the informationsphere with endless amounts of codswallop.
> Having an oracle that knows all answers is useless if you don't know what to ask.
Honestly I find this to be the biggest advantage to using a coding LLM. It's like a more interactive debugging duck. By the time I've described my problem in sufficient detail for the LLM to generate a useful answer, I've solved it.
I’ve had a project I’ve been doing for ~6 months learning python through Copilot+ChatGPT. It feels like any other project that accrues technical debt, but that debt is just a lot weirder. It was rough at first, but refactoring has gotten a lot better recently with bigger context sizes.
Good one, however, like Google making memory worse, copilot seems to erode knowledge of programming quite quickly. Last week I was with a colleague at my house; when I hired him during covid he knew these things just fine, but after not having to type in a list comprehension for around 2 years now, when my Starlink was down for a bit, he couldn’t for the life of him create one; he asked Claude opus on his phone instead.
For real. I have the same experience. I'm 19, freelance developer writing Python among other things. Except I never learned Python syntax, and maybe never will.
For a senior-level professional? Yes, it does matter.
I wouldn't be comfortable with an accountant who couldn't do practical arithmetic in their head, or a surveyor who didn't have a fluent grasp of trigonometry and the ratios or function values for common angles.
Of course -- I don't care about the people working for them as a bookkeeper or assistent. They can be button monkeys as much as their boss lets them, but I also wouldn't expect those folk to reach very high in their career. Not everyone's going to, and a disinterest in and lack of technical fluency is a darn good predictor.
Your teacher was giving good advice about building skills and internalizing knowledge, because those are what contribute to mastery of a craft; maybe you were just being too pedantic or cynical to hear it?
I think this advice is backwards, as is the problem. I’d trust an accountant’s mind or a builder’s eye if I knew that they are acting from experience. But you don’t wanna trust someone who believes in-their-head is superior or reliable and makes you good.
The core idea is no, calculating in your head doesn’t turn you into a pro. Being a pro makes calculating in your head safe enough to rely on until you get to your desk and double-check.
That said, I was an integrator half my life and seen some accountants, big and small. Everyone used a calculator, it’s right next to their mouse and there’s another one elsewhere. And no one ever tried to talk about numbers afk, except for ballparks. Most of the times they suggest to go and query a db together. I find it very professional.
I just trust my accountant will use excel and what they are doing isn't rocket science.
I have no interest interacting with someone who is going to get into how great the slide ruler is and how kids these days need to learn the slide ruler. When I was younger I thought this type of person was highly admirable but now older and wiser I see how full of shit they are.
>> If you need Copilot to code in Python, have you really learned Python?
> Does it really matter though? It sounds awfully like when a school teacher said you're not going to have an calculator in your pocket.
Yes. A younger me had teachers say that to me, and that younger me thought they were wrong.
But it turns out they were right, and younger me was wrong. Calculator dependence created a ceiling built from "educational debt" that many years later severely limited my mathematical ability.
The problem is focusing too much on getting the "answer," and losing sight of building the abilities that allow you to get there with your own mind. Eventually the need to manipulate your automated crutch turns into a bottleneck.
It’s also kind of annoying to intuitively get to the correct answer and have everyone claim they can’t distinguish between that and you using a calculator.
I wonder if that would be an issue if nobody had any.
Isn't it obvious? If you can't do arithmetic without a calculator, it makes it hard to do algebraic manipulation. (Un)luckly a calculator that could do that arrived in my lap just in time. Then you get to calculus, and it was the same story. I eked by with a lot of furious typing, but the mental workload of that made it untenable towards the end and I wasn't really gaining much anyway because of that. It would have been far better if I'd just hadn't been allowed to use a calculator from the start.
It didn't affect my ability to reason and prove things, just as long those things don't strongly require the knowledge and skills I should have gotten from the calculator shaped gap in my education. I lack a lot of the background knowledge and/or familiarity and comfort with many skills that I should have.
I think we're using the term “mathematical ability” in different ways. For me, it's an ability to prove a theorem, not to solve a differential equation.
Yes, it does matter. While there are plenty of situations where it's foolish to forbid calculators, that's not universally true.
Forbidding calculators and requiring students to do mental math for absolutely everything is unnecessary. But requiring students to solve integrals by hand when they're learning about integrals? Entirely reasonable.
If your goal when teaching coding is to teach the mechanical process of writing code: sure, go ahead and use LLMs for that process. But if your goal is to develop a deeper understanding of how to code, then LLMs can very easily obscure that. The end goal is not always just the answer.
> But requiring students to solve integrals by hand when they're learning about integrals? Entirely reasonable.
I remember having to write code on a piece of paper in university.
Curiously, this lead to a lot of other students not really learning or caring about indentations and attempting to make code formatted in a generally readable way once they actually got access to IDEs and such.
They'd write it much like you would regular text on a piece of paper and would get absolutely stumped with unbalanced brackets whenever some nested structure would be needed.
Not only that, but somehow a lot of them didn't quite get around to using the various refactoring features of the tools, since they treated them like editors for writing text.
Did it help me memorize the basic constructs of the languages in question? Absolutely not, at least in the long term, since nowadays I have to jump between a lot of different languages and none of the syntax quite sticks, rather a more high level approach to problem solving.
LLMs just make that easier for me, letting me offload more work around language constructs and boilerplate, albeit requiring occasional intervention.
AI is not replacing the same function as a calculator does when solving a math problem.
Using AI to “help” learn to program is replacing the person’s effort thinking through the problems fully and is ultimately going to stunt their learning.
A calculator is fine so long as you understand what the operations actually represent. Remembering the multiplication table is not important, but understanding what multiplication actually does is.
The problem with AI-assisted coding is that, applied uncritically, it circumvents this understanding. And without it, you won't even be able to judge whether the code that CoPilot etc spit out is correct for the task.
yeah, at the moment anyway, because if you don't know what the syntax means, you can't tell when the code means something you don't want. that means you can't tell which copilot suggestions to accept and which ones to reject, and you can't debug your code; asking gpt-4 what's wrong with it only goes so far
imagine if the calculator in your pocket gave you subtly wrong answers most of the time
This is my new capability- I am not a coder or a programmer and I can get things built, in my own time, at my own speed, solo.
Would it be better code if someone with 3 years of university and 5 years of coding practice did it? Yes, very probably, but the gap seems to be narrowing. Humorously I don't know enough about good code to tell you whether what I build with llms is good code. Sometime I build a function that feels magical, other times it seems like a fragile mess. But I don't know.
Do I know "javascript" or "python" or the theory of good coding practice? No, not currently. But I am building things. Things that I have personal, very specific requirements for.
Where I don't have to liaise or berate someone else. Where I don't have to pay someone else. Where I don't share the recognition (if there is any ever) of the thing, I and only I have produced- (with chatGPT, Gemini and most recently llama3).
Folks have been feeling superior for 70 years and earning a good living because they spoke the intermediary language of compute engines. What makes them actually special NOW is computer science, the theory- the languages, we have very cheap (and in the case of open source local models free) translators for those now. And they can teach you some computer science as well, but that is still time and practice.
I'm the muggle. The blunt. And I'm loving this glowy lantern, this psi-rig.
One of the lessons that one learns as a programmer is to be able to write code that one can later read back and understand. This includes code written by others as well as code written by oneself.
When it comes to production quality code that should capture complex and/or business-critical functionality, you do want an experienced person to have architected the solution and to have written the code and for that code to have been tested and reviewed.
The risk right now is of many IT companies trying to win bids by throwing inexperienced devs at complex problems and committing to lower prices and timelines by procuring a USD 20 per month Github Co-Pilot subscription.
You individually may enjoy being able to put together solutions as a non-programmer. Good for you. I myself recently used ChatGPT to understand how to write a web-app using Rust and I was able to get things working with some trial and error so I understand your feeling of liberation and of accomplishment.
Many of us on this discussion thread work in teams and on projects where the code is written for professional reasons and for business outcomes. The discussion is therefore focused on the reliability of and the readability of AI-assisted coding.
hmm.
I ended up with a few 750+ line chunks of js, beyond the ability of chatgpt to parse back at me. So my go-to technique now is to break it into smaller chunks and make them files in a folder structure, rather than existing inside a single *.js
So readability is an issue for me- even more so because I rely on ChatGPT to parse it back- sometimes I understand the code, but usually I need the llm to confirm my understanding.
I'm not sure if this scales for teams. My work has sourcegraph, which should assist with codebases. So far it hasn't been particularly useful- I can use it to find specific vulnerable libraries, keys in code etc, but that is just search.
What I really need is things like "show me the complete chain of code for this particular user activity in the app and highlight tokens used in authentication" ... - something senior engineers struggle to pull from our hundreds of services and huge pile of code. And so far sourcegraph and lightstep are incapable of doing that job. Maybe with better RAG or infinite context length or some other improvement there will be that tool. But currently the combined output of 1000's of engineers over years almost un-navigable.
Some of that code might be crisp, some of it is definitely of llm-like quality (in a bad way)- I know this because I hear people's explanation of said code and how they misremembered it's function during post mortems. Folks copy and pasting outdated example code from the wiki etc. ie making things they don't understand. I presume that used to happen from stackoverflow too. Engineers moving to llm won't make too much difference IMO.
I agree, your points are valid, but I see "prompt engineering" as democratization of the ability to code. Previously this was all out of reach for me, behind a wall of memorization of language and syntax that I touched in the Pascal era and never crossed.
12 hours to build my first node.js app that did something in exactly the way I had wanted for 30 years. (including installing git and vscode on windows- see, now I am truly one to be reviled)
The problem I currently have with AI-generated code, as an experienced programmer who can read and understand the output, isn’t that the code quality is bad, but that it’s often buggy and wrong. I recently recommended in my company that if copilot is allowed to be used, the developers using it must thoroughly understand every line of code it writes before accepting it, because the risk of error is too high.
Copilot may work for simple scripts, but even for that where it mostly for things right, in my experience it still introduced subtle bugs and incorrect results more often than not.
I've been coding Python for 15 years, but I probably couldn't code in Python now without Copilot, or a lot of reference docs. There's so much meaningless trivia and idioms I've purposefully pushed out of my mind so I can focus on other things. If copilot has my back, why do I need to remember that crap?
Learning to be a good programmer is as much about learning how to avoid technical dept as it is about learning to use a programming language. It may take a while until an AI assistant will be able to help with that.
>Having an oracle that knows all answers is useless if you don't know what to ask.
This sentence summarizes the issue with the current AI debacle, along with the whole "just copy/pase code from stackoverflow and earn top bucks" meme that was going around in 2010s.
You're not gonna be a valuable dev if you're just write wrong code faster. Not only does chatgpt/copilot give haphazard code half of the time, it approaches seemingly random syntax and format. Even if LLMs are polished, you're gonna need stand software engineering knowledge to know what's right and wrong.
Coding just keeps getting grosser and weirder decade over decade as new layers of abstraction and complexity get piled on. Just about everyone who does this shit for a living eventually hits a break point where the headassery du jour becomes too much to turn a blind eye to.
LLMs are no excuse for bad code reviews or developers who don’t know what it’s spitting out.
In my code reviews the person who wrote the code needs to explain to me what they changed and why. If they can’t then we are going to have a problem. If you don’t understand the code that an LLM spits out you don’t use it, it’s that simple. If you use it and can’t explain it, well… we are going to have to have some discussions and if it keeps happening you’re going to need to find other employment.
The exact same thing has been happening for pretty much the entire time we’ve had internet. Stack Overflow being the primary example now but there were plenty of other resources before SO. People have always been able to copy/paste code they don’t understand and shove it into a codebase. LLMs make that easier, no doubt, but the core issue has always been there and we, as an industry, have had decades to come up with defenses to this. Code review being the best tool in our toolbox IMHO.
> You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to.
> Alas, that does not remotely resemble how people are pitching this technology.
It is exactly what happened to you: it wrote bullshit. Plausible bullshit but bullshit nonetheless.
It’s also just not as good at the task anymore. It frequently gets lazy and gives you an outline with a bunch of vague pseudocode. Compare to when GPT-4 was slower at producing output, but all of that output was solid, detailed work. Some of the magic that made you say “wow” feels like it’s been enshittified out of it.
I sometimes try the free chatgpt when I run into a problem and it's just hilarious how terrible it is. Loves to go around in circles with the same made up solution that has no basis in reality, using functions in libraries that would be great if they actually existed.
I noticed that like since a week ago? Output faster, but not impressive. Now I just skip to stack overflow or docs. The output is also giving error a lot more, as if the libraries on which the example is based off was old. Sometimes it's really trivial task just to save time, and it's just not of any help. Still helpful when you want to start something new, it just doesn't scale that well.
Yes, rather poor but people can always post new answers and votes sort the answers. It might not work all that well but there is a mechanism for improvement and to keep things up to date.
Language models can copy the top answers from SO, ingest docs and specs etc. And then the information is never updated? Or are they going to train it from scratch? On what? Outdated github saved games?
It is difficult to get a man to understand something when his salary depends on his not understanding it. ~ U. Sinclair
A significant problem is the subconscious defense mechanism or bias that compels us to conclude that AI has various shortcomings, asserting the ongoing need for status quo.
The capabilities of GPT-3.x in early 2023 pale in comparison to today's AI, and it will continue to evolve and improve.
I’m surprised to see this comment was downvoted heavily. The quote is very popular around here, to the point where there is an Ask HN if it’s the most quoted quote:
> Students can spend more time on optimization, ethical issues, and the user-friendliness of a system rather than focusing on the syntax of the code
And when they land a job in the industry, where exactly are the optimization and ethics when transforming XML into JSON and back again?
I might be in the wrong company, but I see most developers do extremely mundane tasks. At university, they have learned to code Dijkstra's algorithm blindfolded, but they never get to implement it again, because it is already available in a library.
The real added value of a developer is to transform human requirements into code. Transforming the human requirements into prompts, which are then automatically transformed into code is a big win, but I wonder if it will change things at a fundamental level. We are already way too smart for the jobs we do.
90% of my job is spent redesigning and optimizing code that “just transforms XML into JSON and back again.” The problem is when you don’t have a good mental model for how the code works and how it will evolve over time you make bad design decisions. An algorithm that’s fast for 99% of inputs can be accidentally quadratic for 1% of inputs and lead to major tail latency (group membership in AuthZ systems is a great example) which leads to months of wasted human effort over years because of the bad design decision (and now nobody is at the company who knows how it works anymore). Does this sound familiar to anyone else?
The point of learning Dijkstra’s algorithm isn’t to be able to reproduce it, the point is that you know what’s available to you to solve a problem and what areas of the algorithm you can tweak for your use case.
If you don’t have a good mental model of how the code works and how you write a maintainable solution then you’re not able to analyze the tradeoffs of the high level competing requirements like AppSec vs UX vs Maintenance vs Engineering Effort that is at the heart of what a software engineer does.
And this doesn’t change with AI either, the code that comes out of LLMs is often poorly designed but you can’t tell that unless you have worked on different types of systems to know why we might use immutable data here but shared memory in this situation.
A dev who coded Dijkstra will catch up with any new framework in no time. A dev who never went into algorithms will not even know how to program some things and where to find information.
Having worked with both kinds - I'll take a dev with experience in algorithms and no knowledge of a framework we need to use over an opposite. Especially in times of GPT, where gpt will allow you to switch to a new framework in no time, but struggle with anything besides the boilerplate code structure.
Curious. I see a lot of whining about the rapid changes in frontend development on HN. Are these all just people who failed to learn Dijkstra's algorithm?
I've been programming for 25 years. Thanks to LLM assistance I hope to never have to hand-write code that converts XML to JSON ever again.
(Last night I ended up writing that kind of code because GPT-4 didn't quite get it right for me, but 90% of my code like that has been LLM-created in the past 6 months and I look forward to that creeping closer to 100%)
I think us folks that have been coding for 20+ years can properly utilize AI coding assistants because we can take the code it spits out and usually understand it pretty quickly and debug it if needed. It's just a skill that comes from writing code for so long.
I have seen many jr level programmers struggle with AI coding assistants because they ask a question, get some code, paste it in, realize it doesn't work, but don't know why or the right questions to ask to properly debug it and get it working.
Nothing wrong with SO or even using an AI assistant as a jr. You should use any tool that helps you, but I still believe nothing beats practice and doing some manual coding to really learn and understand the concepts.
I still think one should reach out to a book first, then goes on to whatever method is best (asking a senior/mentor, SO, ChatGPT,...). It gives you more information and the material for better questions.
You can do this, and sometimes it works, but more often than not it does not. it's basically the blind leading the blind. With the current state of LLM hallucinations, it's very easy to be given bad or outright wrong answers, and if you don't know any better (which as a jr you probably don't) you'll just end up running in circles.
I think this will def get better over time though.
damn right, I tried using GPT-4 for to make a text editor enhanced with CodeMirror 6, but the model kept confusing with version 5, and the thing is, they moved stuff around and deleted some modules, so it was a total mess
when you refactor the whole framework you should change its name to keep docs straight, a mere version change is not strong enough for LLMs
> Thanks to LLM assistance I hope to never have to hand-write code that converts XML to JSON ever again.
There is that. Although, if you need, say, to convert .fbx to Collada, there won't be enough examples out there for an LLM to infer how to do it. There's progress; most tools that wrangle meshes now speak glTF, so at least you only need O(N) converters, not O(N^2).
There are a number of programming in the large problems that tend not to be taught. A large collection of problems revolve around having state in one place that you want somewhere else. Sometimes you can just pack up the whole state and send it. That works if it's not too big and doesn't change too often. Perhaps the data is huge. Or somebody isn't allowed full access to it. Or it's changing rapidly. Or you need to keep up with changes with low latency. Or you must have guaranteed eventual consistency. Those problems come up all the time in areas from finance to game dev.
Inefficient solutions to such problems lead to bloated browsers, slow web pages, and inefficient back ends.
An example is XMLRPC.
Efficient solutions to those problems are complex, hard to modify, obscure, and hard to debug. An example is the internal representation of Microsoft Word .doc documents. Much of the web is trying to ship state from somewhere to somewhere else, in rather bulky ways.
This can have business implications. In the metaverse area, where I've been doing some things, Improbable tried to deal with such synchronization problems in a very general way. They blew through about US$400 million in venture capital, and produced a working system that needs so much server capacity that five indy games went bust, and their big demo, Otherside, is so expensive to run that they only turn it on for demos a few times a year.
This may also be why Meta blew their metaverse business so badly and spent so much money.
Doing this by special casing code results in an obscure, hard to modify system with race conditions and consistentency problems. That's the classic MMO approach, and is used by Second Life/Open Simulator.
A general understanding of such problems is useful, and is teachable. But what do you read for that? Not books on "how to do thing X in framework Y".
I've been programming for about as long but I've yet to see a case where searching on Google (and often opening Stack overflow) wasn't quicker than ChatGPT. More often than not if I'm looking something up I'll get a bunch of answers on SO and quickly skim through them to get what I was looking for. This is way quicker than prompting over and over again hoping to get an answer.
But after programming for so long the types of things I'm searching are basically API references. I know exactly where I'm going I don't need help getting there.
You have to learn how to most effectively use ChatGPT etc. If you use it the same way you use Google you'll probably get better results from Google.
Here's an example from yesterday where I knocked together a tool for converting Atom XML to JSON using XSLT in a couple of minutes on my phone, just out of curiosity to see what it would look like: https://chat.openai.com/share/4bc7922b-b1c2-411c-aef9-b41b0f...
You (and i) are more of a raccoon picking though a dumpster looking for working scraps of code that you can glue together.
Let's be candid, we all have package management Stockholm syndrome to some degree... the reality is that our laziness keeps us going back for more.
The performance gain comes at the cost of not knowing how some large percentage of your stack works or if it is going to be the next XZ.
Im not going to stop using packages, because that would suck, but I'm not so rosy eyed about them any longer. There are lots of problems in ecosystems and we're bad about addressing them.
Vernor Vinge very accurately predicted how programming will develop 25 years ago.
"Pham Nuwen spent years learning to program/explore. Programming went back to the beginning of time. It was a little like the midden out back of his father’s castle. Where the creek had worn that away, ten meters down, there were the crumpled hulks of machines — flying machines, the peasants said — from the great days of Canberra’s original colonial era. But the castle midden was clean and fresh compared to what lay within the Reprise’s local net. There were programs here that had been written five thousand years ago, before Humankind ever left Earth. The wonder of it—the horror of it, Sura said — was that unlike the useless wrecks of Canberra’s past, these programs still worked! And via a million million circuitous threads of inheritance, many of the oldest programs still ran in the bowels of the Qeng Ho system. Take the Traders’ method of timekeeping. The frame corrections were incredibly complex — and down at the very bottom of it was a little program that ran a counter. Second by second, the Qeng Ho counted from the instant that a human had first set foot on Old Earth’s moon. But if you looked at it still more closely… the starting instant was actually about fifteen million seconds later, the 0 - second of one of Humankind’s first computer operating systems. So behind all the top-level interfaces was layer under layer of support. Some of that software had been designed for wildly different situations. Every so often, the inconsistencies caused fatal accidents. Despite the romance of spaceflight, the most common accidents were simply caused by ancient, misused programs finally getting their revenge.
“We should rewrite it all,” said Pham. “It’s been done,” said Sura, not looking up. She was preparing to go off-Watch, and had spent the last four days trying to root a problem out of the coldsleep automation. “It’s been tried,” corrected Bret, just back from the freezers. “But even the top levels of fleet system code are enormous. You and a thousand of your friends would have to work for a century or so to reproduce it.” Trinli grinned evilly. “And guess what — even if you did, by the time you finished, you’d have your own set of inconsistencies. And you still wouldn’t be consistent with all the applications that might be needed now and then.” Sura gave up on her debugging for the moment. “The word for all this is ‘mature programming environment.’ Basically, when hardware performance has been pushed to its final limit, and programmmers have had several centuries to code, you reach a point where there is far more signicant code than can be rationalized. The best you can do is understand the overall layering, and know how to search for the oddball tool that may come in handy ...”"
Lots of different ecosystems. Mamba + docker are great though. Pip is ok. Apt works, but we were using the same thing in 2006. If these different bits didn't exist, and have ChatGPT telling me that they exist then most work wouldn't get done...so there's that.
>(Last night I ended up writing that kind of code because GPT-4 didn't quite get it right for me, but 90% of my code like that has been LLM-created in the past 6 months and I look forward to that creeping closer to 100%)
I have bad news for your long term employability if that's the case.
I find that my code is 25% LLM-able on a good day with minor editing.
On a bad day it is 10% LLM-able with me having to edit every line to get them to fit together.
I have to admit I've written a gpt4 api shell utility which transforms text based on descriptions like "split string on '-' after a '~' but only if it's in a the second csv column and select the third".
I feel dirty using it - send your data to the cloud and use a ~peta flop- but it's faster than trying to remember how to do it in awk for under 1000 lines.
i really don't think simon motherfucking willison needs to worry about his long-term employability, unless either he suffers a disabling injury, or llms get better than humans at everything, or somehow people stop wanting software
XSLT is the perfect example of a technology that I might consider using today if I can get an LLM to do all of the work for me - see also jq and AppleScript which I use frequently now after avoiding them both for years.
I think you missed the point. People attempt to solve problems with LLMs that are problems only to them because they are unaware of existing solutions that have solved the these “problems” a long time ago. If you don’t know something exists, you can’t ask an LLM about it.
I played around with it a bit and it looks like you have to do some pretty gnarly things to get JSON string escaping to work in XSLT output - at least for the XSLT 1.0 version supported by most web browsers: https://chat.openai.com/share/4bc7922b-b1c2-411c-aef9-b41b0f...
It's a great tool when you can actually read and understand the code AI is feeding you. When you don't really know how the language works or don't know programming in general, you end up getting stuck trying to guess why this valid piece of code (that looks like it should do what you asked the LLM for), is not working properly.
I can only imagine that given current trends, this period of "LLM writes boilerplate/broken code and SWE builds on it/fixes it" will be incredibly fleeting.
This is actually what sucks about interviewing. My work is mostly working with managers/executives to deliver them a solution, I provide tremendous value to the business. I am full stack dev and can write a complete solution with tests, CI/CD pipelines and setup infrastructure.
I once interviewed and was asked what is the difference between thread and a task. I provided a 'dumb' correct answer because I did not remember the text book definition, as I have been out of school for several years. The interviewer ignored everything I've said and told me "Actually, it's <insert textbook defintion>".
It almost seems that computer science teaches you too much of the theory and not enough of "get shit done".
I really enjoyed computer science and graduated with honors, but writing business code is kind of meh and boring. I know people who dropped out of CS due to having a hard time with algorithms and more advanced courses. I feel like they should have stayed and would do fine doing boring work. However, nobody tells you this, there seems to be a lot of gatekeeping in tech.
> where exactly are the optimization and ethics when transforming XML into JSON and back again?
What's are the XML and JSON being used for?
If you're working for a company, you almost certainly aren't working for the "XML to JSON, library salesperson" company, you're probably connecting different API endpoints. What do those endpoints do? Do they do ethical things?
>And when they land a job in the industry, where exactly are the optimization and ethics when transforming XML into JSON and back again?
The programmers who are mostly only capable of doing this kind of grudge work will lose their jobs and need to transition and adapt to a new form of grudge work.
And yes, as you point out, that's a lot if not most of them.
I must admit that my post was not very clear (reflecting the current state of my thoughts about this matter), but what I meant to say is exactly the opposite of your suggestion.
Some people believe that AI will take our jobs, but my point is that our work is already practically trivial, yet for some odd reason we are still in short supply.
As much as some of us have become JIRA slaves, our future generations will probably become AI slaves. True creative programming is beholden to a very select group of people, simply because there is so little demand for new things.
I have multiple friends who are doctors, and they will all tell you that 95% of their work is just trivial grudge work that almost anyone can do. Most cases that come to them are easily treatable and don't really require much of any expertise to diagnose or manage.
Nevertheless, doctors in many areas are in short supply and it's incredibly hard to become a doctor. Why? Because for the 5% of hard cases when you absolutely need an expert or else there are serious consequences, and it's very hard in advance to predict if someone's case is the 95% of trivial grudge work, or that 5% of critical work.
Software development is the same, the vast majority of the work is trivial grudge work and there's a lot of that grudge work, enough of it to keep millions of people employed. But there's still that 5% of the time when you need to be an expert, when you need to know data structures and algorithms and good architecture and have solid knowledge of your domain. The problem with software development is that most people just take that 95% for granted, get complacent and think that the 95% of trivial work is the job and that they don't need to be prepared for the remaining 5%.
My opinion, and that's all it is, is that in the future there won't be a need for that 95% of work, it will be automated and all that's left will be the remaining 5%. The people who became complacent and came to believe that their job is the 95% of trivial work will find themselves in a very difficult position.
I total agree with you here. I was going to post almost the same thing.
"The problem with software development is that most people just take that 95% for granted, get complacent and think that the 95% of trivial work is the job and that they don't need to be prepared for the remaining 5%."
I have found that many developers just aren't capable of the remaining 5% and companies that hire need you for the 95% hope you can handle the rest.
I'm currently a consultant for a pretty large company and I deal with their back-end software systems. Most of my job is mundane and easy. However, there have been on-fire issues where if I wasn't good at my job, the company would have lost hundreds of thousands of dollars.
"Nevertheless, doctors in many areas are in short supply and it's incredibly hard to become a doctor."
At least in my country both of these things are caused by doctor's unions lobbying to limit student intake to the point where lots of perfectly capable people simply can't get admitted. It has got to the point where many go study abroad in other EU countries.
Anyway, even if everyone in the workforce was capable of the 5% hard work, companies could still fire lots of people, enough to give us universal basic income or third world war. Everyone in the workforce will be affected at that point.
When I was in school and search was becoming a thing, the teacher said something that resonated with me for all these years.
"It's going to be less and less about knowing, and becoming more and more about knowing where to find the answer".
That held true for search and that holds true with AI coding assistant. From knowing how to use google search, to knowing how to prompt to get a reasonable clue.
We've just added a new meta to the problem solving puzzle: is the computer giving me the right answer or is it confidently throwing me a red herring.
We've gone from knowing the code we run, to barely understanding how the compiler does it, to having no clue what is happening...
Soon enough someone will invent an OS where everything is done by speech or thoughts; and typing on keyboard will become a thing of the past, like hand writing... Programming will be done on the fly by the user's request and the computer will be compiling and executing it with a high degree of confidence. This is a future that is more likely to happen before the thinking machines really happen.
> That held true for search and that holds true with AI coding assistant.
100% - this was not obvious to me until I was working at a startup and delegated a task to some others. They tried for close to an hour to find a solution to a problem and came up empty handed. I had a go and instantly found the answer.
What was the difference? It was our ability to put into Google the right terms to find the answer.
Already I find myself doing most coding via AI, with my work being high level skills (choosing how to organise code, knowing when to refactor and when the AI has gone too far).
The productivity increase is crazy.
The future quite likely will look like the minority report interface.
I've observed another meta as well: IDEs/language servers.
I was writing a Wordpress plugin and by default in PHP all function definitions are global, e.g. is_single that WP defines is global. Naturally to avoid conflicts you would use a namespace, but since most of the examples I saw used prefixed function names, I decided to try that as well.
In my opinion, coding just feels much clearer when you prefix all your names, like my_library_my_func. When I see my_library_, I just immediately know that is my code. If you use different prefixes for different modules, you can tell from a glance which modules a piece of code interacts with. In C++, you can tell a random word is a field by the simple fact everyone prefixes them with m_. I've seen people using prefixes for pointers as well.
This is the same concept you have when designing an UI. Everyone agrees a problem with hamburger menus is that you don't know what's inside the burger until you click on it. If you want people to be aware something exists, you need to make it appear on screen, explicitly.
Instead, languages have evolved toward the very opposite. We ran away from self-documentating variable names and prefixes, and toward namespaces, "const" and other keywords, and "OOP" that feels less object-orientated and more press-the-dot-key-to-get-a-list-of-methods orientated. We depend on docstrings that show in the IDE, and then on auto-generated documentation from those docstrings. Typescript is probably the worst case of this, because it gives you a lot of tools to make the dot work the way you want, you'll spend way too much tinkering with the dot instead of just writing code that actually runs.
Thanks to this, when you look at code, you have to wait until your language server figures out what you're looking at for you. If the IDE doesn't work, it's effectively impossible to code anything in many cases. But the variable names are shorter so people think it's cleaner and better, like removing features from an UI to make it cleaner.
A language where every definition is global is the worst starting point imaginable. Of course, solutions to that will remain severely lacking. Prefixed variables as you propose were abandoned for good reason.
Perhaps give Rust or Python, maybe Go a spin, and see how they do namespaces, imports and such. I find that mostly sane and readable, as they retain the full chain of custody (you can always trace where a symbol comes from by its qualification or import statement, unlike in less sane ecosystems such as C/C++/C#/Java, and apparently the insane version PHP uses).
Modern C# is a lovely language overall but I think OP talked about the module system where the more explicit approach of Python is arguably better.
When dealing with C#, if you’re browsing code un anything less than a IDE it can be really hard to find where a symbol comes from. And that is if the author was nice enough to use one folder per namespace, otherwise it becomes a nightmare.
In case of external references, most of the time, you do not need to browse the source files - that's something you reach for if inline documentation and type system are insufficient. Arguably much more of an issue in dynamically typed languages, yes.
Although even source browsing is now easy - you just press F12, and packages come with sourcelink integrated automatically (previously you had to rely on IDE to resolve symbols or decompile it from IL (readable but not perfect, Rider did and still does it really well)).
Navigating the solution itself does not differ significantly from Rust and its crate system. People usually structure the code by projects and folders, sometimes aggregating multiple types in a file (because you don't need a separate file per a couple of 4 line class declarations) sometimes not but I generally find the structure way more browsable than e.g. what a typical TypeScript project looks like. The regular symbol navigation rules apply - F12, Shift+F12 and friends.
In Rust, Python and Go you can trace every symbol back to its source by just reading source code. `cat file` would suffice.
In C#, that’s impossible by design, as you say. C# imports work like wildcard imports in Rust and Python, which are wildly frowned upon for very good reason (except for specific cases like preludes in Rust).
I’m impressed by your teacher’s foresight. That’s _exactly_ how I see it. I don’t memorize things, I just know _how_ to find them when I need them. You can store a much larger “index” in your head than raw knowledge, which is what I strive for.
The problem is that going from assembly to advanced compilers still produced the same output every time with variations based on you selecting them from a limited pool of options.
With LLMs, statistics play a much more important role so you're really not in the same realm anymore. It's like going from digital to analog, things work differently at a core level.
I still believe that warhammer 40K is the most accurate depiction of advanced tech. We will all just become tech priests praying to the machine spirit to get the computer to do what we want.
I worry for the next generation. I loved math until college. I was in a hurry to get gen-ed and general engineering classes out of the way so I signed up for an accelerated math class that was one of if not the very first pilot program to teach calculus with Mathematica. It also collapsed three semesters into two.
I learned first hand why teachers want you to master the fundamentals before reaching for a calculator. Most of the time if I couldn't figure out how to solve the problem, I would reverse engineer the steps from the solution. But I wasn't really learning how to think about the problems for myself.
I did abysmally on every single test. I got a B- in the class only because the homework was weighted ridiculously high and I had a >95% on that portion of my grade.
How to not recreate that failure mode will probably be this generation's big crisis.
Similar story here with high school calc. My TI-89 had a better CAS than the TI-84. It's been 16 years, and I often find myself wishing I knew how to do basic integrals.
I get by, but I often feel a bit handicapped vs my peers. Like building a house on a shaky foundation.
After using these tools, I feel like they are only good at producing code that is already simple to write if you know what you're doing. Basically if your problem fits neatly into something that would have appeared in documentation or directly in a Stack Overflow post that the training data vacuumed up, you might get something usable.
For anything even slightly non-trivial, I always spend more time fighting the prompt than I would have if I just sat down, read the docs, and thought a little about the problem.
And this doesn't even get into things like style, readability, etc.
Maybe this will change with later incarnations of these tools, but a cursory understanding of how these tools work make me suspicious. How can something that, from first principles, generates statistically random output, produce reliably functioning code for large non-trivial tasks?
That's a very very different requirement then producing something like a poem or summarizing meeting transcripts, where a little randomness is expected and nearly all the training data is in the right format of regular human dialogue?
There's no intelligence in these tools, just regurgitation. The exact opposite of what you need for solving software engineering problems.
I agree they're best at simple things and things that are talked about on the internet a lot (they're pretty great at parsers for example).
But... a significant amount of code is simple. Even in complex projects there's still a lot of trivial code.
And even when they produce something wrong, it's usually still faster than starting from scratch. At least for smallish bits of code. Though admittedly it can be a bit more frustrating on occasion because you're doing something more like code review and who doesn't hate code review?
totally agree. I use it for boilerplate functions and restapi templates. It saves me a lot of time for these specific things but it's been wrong many times for very library parameters.
For example, let’s say you wanted to write code in a language you did not know. Then these simple tasks can be accelerated because you’re not blocked by syntax errors.
Technology always has a weakening effect. AI copilots will make us lazier and produce more buggy code. It can increase productivity for certain boring tasks (generate a bunch of empty classes, etc.). It can increase productivity for early on in projects. Otherwise it's just a crutch.
My sister had her first Python class in college. She used an LLM successfully for all her class exercises during the year.
But during her final year project, she got stuck. The task got too hard for her and the LLM. The learning curve was broken.
Like a game designer that gives the player increasingly powerful items, educators should preserve that curve, or it becomes impossible to learn anything.
I mean, that sounds like a form of code review, which is very standard in the industry. Is it the group aspect or recording a video that would have driven you away? I’m just a little confused.
I think there is a bit of a false delta fallacy being floated here. Universities / colleges never focused on the syntax. It was always about problem solving. I think what they are saying is they will continue what they have always been doing while pretending that the past is different than it actually was.
Exactly. 25 years ago in university none of my professors or assistant professors bothered with syntax, that was left to the compilers or interpreters of the language we were supposed to do our assignments in....
We had very different professors. 15 years ago I was writing C programs on paper and being graded on my syntax. In my first database class I lost 1pt on every question because I didn’t end my, otherwise perfect, queries with a “;”.
Both are just 1 of many examples of why I found college, for computer programming, to be nearly a complete waste of time. I’m not saying I didn’t learn anything but I learned way less than I did on my own and most of what they taught was horribly outdated and didn’t take into account how you’d actually be writing code in the real world.
It’s like the teachers that said “you won’t be walking around with a calculator in your pocket always”. That’s not me saying math isn’t important, it is, but it’s the same flawed argument that makes you memorize syntax as if your IDE won’t auto-complete the boilerplate for you or pretends you don’t have all of the internet at your fingerprints to look up things. It would have been much better for the professors to spend their time teaching how best to manage a project or take business requirements and turn them a spec or pseudo code. That’s a skill I actually use every day and it hasn’t changed much whereas the language I use and how I write code has changed quite a bit.
I absolutely loved my university education in computer engineering. It was so great to learn about the math (lots!) and physics underpinning how computers / transistors actually work. The first year was common engineering so even had to take a couple of unrelated classes like thermodynamics and vector mechanics (with law and philosophy as options). Comp sci classes were really good as well: compiler design, databases, algos and data structures, AI, cryptogrpahy, etc. The utility of this knowledge is probably debatable but I enjoyed it very much and I think it helps at least a little bit.
I'm sure it depends heavily on where you go to school or maybe I'm just not well suited to the style of teaching I recieved. Personally I did not enjoy math once we hit integrals in calculus and I hated physics. I understand they are both very important but they have had no bearing on my professional life. There is value in learning something not directly applicable to your day job but when I was in college I saw it primarily as a means to an end and I'm sure I'm not alone in that. If I were to go back now just to learn for the sake of learning I'd view it differently.
Then again, I currently see college as a massive scam for most people with the cost where it is right now and colleges doing a shit job of preparing people for the real world so I wouldn't set foot in one as things stand.
Integrals in calculous is pretty much the beginning!
Fair take in any case. Colleges today seem far more politicized (and expensive) than back in my day. The whole space seems ripe for disruption as a lot of people just want to learn without all of the associated baggage.
> Integrals in calculous is pretty much the beginning!
And maybe I would have found more interesting, and more importantly: applicable, concepts has I gone further. I loved math in Elementary/Middle/High School, it just got a lot more abstract for me in college and I've alway struggled with "Ok, how will I actually use this?".
> Fair take in any case. Colleges today seem far more politicized (and expensive) than back in my day. The whole space seems ripe for disruption as a lot of people just want to learn without all of the associated baggage.
Couldn't agree more. College today can still do at least 2 things decently well (if extremely overpriced) which are to expose you to different viewpoints, idea, and concepts as well as to give you a lot more freedom but with some "training wheels". You're often in a dorm and on a meal plan so you have somewhere to live and something to eat but you now set your own schedule and are able to make decisions that were often made for you (at least in my case). I'm grateful for that experience and the ability to make stupid mistakes without terrible consequences. I also learned a lot about the world outside my hometown that was critical to my development as person.
I'm close to going off on a, well worn (for me), tangent now about how we need to teach meal planning, financial planning, etc to kids but yes, I think college is ripe for disruption. I value higher learning and believe in it but what colleges have become is very gross.
Yeah, when I have given coding exams, I make it pretty clear that they're allowed to use IntelliJ or PyCharm or Eclipse or any IDE (within some degree of reason, we've argued about LLMs in a sister thread that we don't need to get into again :) ).
I think in real life, syntax errors are harmless; they don't compile/run so they don't really get deployed. I think it doesn't really hurt understanding to have little red underlines if you forgot a semicolon, and if you don't really understand the algorithm you're going for, the smart autocomplete by hitting the `.` isn't going to help you do anything other than avoid typos.
I wish my professors had been more like you, or at least approached it with similar reasoning.
Writing c++ boilerplate for a function on paper and losing points for leaving off a ";" both made my blood boil and heavily shaped my views on how programming is taught in college (or at least the one I went to). I also just didn't care much for c++ since I was much more interested in web development (PHP/JS at the time). I used to write all my c++ programs in PHP then once I got them working I'd convert them to c++ to submit to the teacher.
I have no issues going lower-level if needed, I've written Perl (which I consider under python/PHP personally, not sure what others think) when it makes sense for the task (log parsing) and written a tiny bit of C here and there. Lower-level languages just require much more mental overhead for me whereas I can move much faster in a higher-level language and shorten my "Write code"-"See result" cycle which is important for me personally.
Yeah, I almost never took off points for syntax, and never during exams. I would take off two points for homework assignments if it didn't compile simply because I'd have to go and manually fix it so it'd compile so I could check the behavior, which was annoying, but I felt that was a little unfair during exams because those are on a much shorter timespan and are more stressful.
I think some professors are a bit sociopathic. Fundamentally a computer science class should be teaching computer science concepts. Anyone can learn the syntax of a language pretty quickly. As a teenager I used to think I was super smart because I would "learn" a new programming language every week because I'd more or less just pick up the syntax differences between the new language and C++, and so I felt like because I could write a loop in the new language I "knew" it. It's much harder to learn and understand the concepts, and despite being a software engineer for 13 years I don't pretend to understand all of them (sort of a Dunning Kruger thing I guess?).
Professors should know this. The value-add of college should be more (or at least different) than you can pick up from an O'Reilly book you buy at Barnes and Noble for $30.
> Anyone can learn the syntax of a language pretty quickly.
Agreed, I can become "dangerous" in a new language quickly but actually being good in a language takes practice and time.
> The value-add of college should be more (or at least different) than you can pick up from an O'Reilly book you buy at Barnes and Noble for $30.
And to be fair it is more and different, just not 1,000-10,000 times better (when the cost is). There is a ton of value in having a good teacher, unfortunately we pay most teachers at all levels peanuts but expect them all to be rock stars. I seriously considered teaching at one point in my life but couldn't stomach the crap they have to put up with for so little. The teaching/mentoring I've done in a professional capacity has always been incredibly rewarding.
Yep, there's a reason that I am no longer an adjunct professor. There were parts I liked about it, but it was a lot of work and even more of a time sink, for not much pay. It was immensely satisfying to teach students who wanted to be there and wanted to learn, it was less satisfying when students wouldn't show up to any classes start begging for more time to take the exam, or when students would cheat on tests.
I do recommend doing the adjunct thing for one semester if you're able to, it actually is something that I think has really helped me in a lot of ways (increased empathy for people struggling with math/CS concepts, slowing down my talking speed when speaking publicly, etc.). You might like it, you might hate it, you might not be sure what you feel about it (that's my case), but the worst case scenario is you don't do it next semester.
The real benefit of ChatGPT in coding is that it significantly lowers the threshhold for regular people to start learning coding.
A.I. rarely gives a complete solution and even less so the bigger and more complex the code becomes.
The truth is that in the end you will have to learn coding 'for real' because A.I. will only help with a partial solution.
It's terrific for newbies because it makes it so easy to start coding right away and get results with simple and small tasks.
Exactly, AI support in IDEs useful for more than just producing ready made solutions. In fact, it isn't really good at that because of context size limitations (most non trivial sized code bases are simply too large). And it will make mistakes, omit stuff, etc.
But, I've used it to review, debug, document, test, and explain code. I occasionally use it to stub out a solution but usually it then requires manual tuning. Any modestly complex algorithmic stuff it usually makes lots of mistakes. But it is able to spot uncovered edge cases, or suggest ways to test algorithms.
I think more of it as an artificial pair programmer pointing out my mistakes, and helping me be more productive. It's also good at figuring out syntax issues. Often when dealing with languages I'm less proficient in the main battle is simply figuring out how to do X in a language. If you are trying to learn things like Rust (which is hard to learn), AI is your best friend. Because it can explain you your code, your mistakes, and suggest alternate ways of doing things. All you need to do is ask. Remembering to do that is the hard part.
I tried using the pycharm copilot and that lasted a couple days. I think the UI side of picking between copilot and regular autocomplete needs to be ironed out a bit better, but I think those are short term problems. What actually stopped me was the suggestions for comments. When the grey text appeared as I was typing I realized how much it was guiding what I should think about and how hard it was to express what I wanted. I would try to type my thoughts but be derailed as I saw an alternate wording appear like an unwanted whisper that is just ahead of your thoughts. I think the same was happening in code to a large extent as well but the comments really hammered it home.
The simple code is just that, simple. Taking care of the simple code may free you to think about the bigger things, but the copilot grey text whisper in my face as I am typing crushes my bigger thinking and removes all the gains from taking away the small stuff. I don't actually need to be faster at the small stuff. I need to think clearer about the big things.
I will check back on this in a bit but for now my reaction is decidedly 'meh'.
I haven't used the new assistants that much yet, but my impression so far is that automatic suggestions is a bad idea (at least for me). I'd rather trigger it explicitly. That way one can afford a bit more compute per suggestion as well. I think you would get a feel for when the suggestions will be useful and when they won't.
One feature I think could be useful is to ask it to correct very basic errors in a section of code, but this need to be well-integrated in the editor to be worth it. This could also be automatic (the detection) - like an slightly more intelligent jetbrains inspection.
I've also tried some "whole program" synthesis with mixed results. It usually can get something up and running, but almost every time it has had subtle bugs in it - fixing these (with or without the AI) often got me close or higher to what it would have taken me to do it from scratch.
The advantage(?) is I might never have bothered beginning from scratch - getting the first rough draft get you immediately into iteration mode.
As an example - I asked GPT-4 to write a python program reading in a json file conforming to a jsonchema[1] and transforming this into a GPX file.
It ended up generating code trying to get `properties` keys from the json *file*. So it clearly did not understand the schema. Maybe jsonschema is to obscure?
[1] (I provided the schema and a link to the schema standard (though I'm not sure it actually read the link))
I really like the concept of triggering it explicitly. I hope something like that becomes part of the standard UI for these things. Getting the UI right is a major part of how useful these will be.
Interesting. Writing comments is a great use-case of LLMs for me. It gets it right about 50-75% of the time which means way less typing for me. I’ll let the LLM explain what the code is doing and then I’ll add in context specific to this codebase/my company.
If it gets it wrong I just keep typing until it guesses the rest or I finish writing the comment.
One of my first coworkers, my senior at that time, told me that software development is 50% debugging, 40% reading and thinking and only 10% coding. Optimizing those 10% is not that important.
I think a lot of people who have abstract problem solving skills don’t want to make tic tac toe games and to-do lists. They just want to get on and code, building what they want. LLMs are great for throwing you in the deep end.
If the code doesn’t work and you’re in a complete mess about debugging it - that’s great. It’ll force you to think through the WHY and HOW far more carefully than writing some stupid Hello World thing or doing another tutorial. You end up reading the documents and wrestling with ideas.
Best approach I find is to throw yourself at a LLM, then backfill your knowledge with books and resources, and speak to experienced people. That way you craft your own learning experience. Especially valuable if you just aren’t surrounded by software engineers - you’re forced to learn.
You try to build somethign complex, encounter a concept you don't understand fully, go to the literature and learn it because you need to know it, it unlocks the next step of building. Obviously you need to start the very basics with Hello World - i.e. if you've never coded before, you should grasp the very basics of how code works and how languages work. However after that, there's no need to spend hundreds of hours doing tutorials. Building a thing is always better because the problems you encounter are real.
Ai copilots are taking raw non measured code and treating as all equal to each other....its about like taking a few answers from A students and a whole lot of answers from D students and will only produce a sh*t-stained result.
Its emphatically why I do not use any AI in my VSCode IDE...as you cannot trust the damn evaluation where each raw input is treated as the same every other input...
We devs use the same damn reasoning for developing a special set of lint rules as those pertain to the exp of coding and what things we want intended in our code constructs according best practices...
In short words, fluency is practice in both mistakes and corrections....there emphatically is no other way towards code fluency!
Imo there's a difference between criticizing industry use and learning. If you haven't learned to code, but just to tell an LLM what you want, you will not be a successful software engineer.
There's no difference here in my mind between cheating and the models. If someone holds your hand for all your homework in school, you're not going to learn, whether that's a person or a language model.
So if AI advances in 4 years to the point where we're all deprecated, then fine, a student learning it got lucky. But otherwise, you're setting yourself up for failure if you lean too heavily on LLMs.
I am not arguing they have plenty of shortcomings rather than the very defensive attitude of many devs around here.
I'll speculate bluntly on the reason: the overemphasis many have put on the interview skillse is the reason why many feel threatened in the first place. Because that "skillset" is among the most replaceable with AI advances.
Speaking as someone who is defensive about it, it’s because most people pontificating about the AI takeover of programming aren’t actually in the industry or they’re the ones making the tools and have a vested interest in it actually happening.
IDK, it seems to me that the people who genuinely feel threatened by AI, are the ones that are impressed by it and/or fear it will only get better. Whereas those pointing out the shortcomings, and/or know it will always have its shortcomings, don't see any reason to feel threatened.
I really wonder about the programmers who feel threatened by LLMs. The vast majority of my daily work in software engineering does not remotely boil down to "write code for a discrete isolated task". It's all about solving difficult problems in a complex, interconnected system. Feature requests which require previous assumptions to be thrown out and new architectural decisions to be made.
Can current LLMs help with that? Almost never. What about in 5-10 years? Probably a little more, but I'm extremely skeptical that they'll ever reach beyond the tip of the iceberg.
I can only think it stems from fear of the unknown, leading people to believe in the worst-case scenarios.
But the way I look at it is: no matter how good AI gets, for AI to do something, someone must train it, instruct it, or ask it to do something for them, and then verify the results. All of which takes work from people. I mean even the mere instruction of saying "hey AI, do my taxes for me" takes work from me as I had to muster the very instruction, and there's no getting around this. So there will always be software developers regardless of however good AI gets. We may stop calling such people software developers, but I don't see why that matters.
Now, as whether we need less software developers because of efficiency gains? Unlikely to happen, because jobs are created from demand, and humans capacity for demanding things is endless, so the capacity for jobs is endless. This is the reason why the Jevons paradox happens. But in the case that we do in fact need less software developers, it still doesn't matter, because all it does is free up the time to do another job, just like a dishwasher does...
I'm not in big tech and I am absolutely worried because middle management and the c suite think my job will be redundant soon, even though I can see it won't, and thus they'll lay me or my peers off before shtf and ofc they fave no consequences for that decision.
May be tech managers are better, but seeing the recent run of layoffs, it seems they're not really that much better. The problem fundamentally is your boss doesn't know how to code and will be so impressed that he can get code to write fizzbuzz, so why does he need you anymore?
I don't feel threatened as a senior software developer. Given the kind of output I've seen from the current generation of these tools, their thoughtless use at scale will, if anything, create more demand for people who have enough understanding of how things work to clean up the inevitable mess that they will produce.
I am, however, extremely concerned as a user, because the senior management in many places will push for them anyway, with the resulting drop in software quality across the board.
I'm not threatened, juniors are because they won't learn. Also I do spend hours reviewing their fake code and trying to list all the bugs inside it instead of blindly accepting it. It's not my job to do this, it's dangerous and a waste of time.
> Genuinely seems many feel threatened by these tools that they need to emphasize the shortcomings.
I think you're projecting a little bit.
You're not skilled or experienced enough in the field to understand that programming sometimes goes beyond simply gluing together snippets of simple, generalized code until you have something that sorta works, and thus believe AI to be more than capable of replacing you, and you don't understand why other, more experienced people don't feel the same way.
I'm waiting for "Google Translate is changing the way languages are taught."
"Students don't need to learn grammar and syntax and really any vocabulary. AI assistance lets them understand higher level German concepts without needing to know how to conjugate for gender and number or remember if a noun uses der, die, or das."
You see a lot of examples of things that seem absurd when applied to non-technical topics, but as soon as you apply them to computers all skepticism goes out the window. If I told you I have a person who speaks perfect English but frequently lies and outright makes things up, you'd say that's a politician and not to be trusted. But if I say I have a computer with those same properties, suddenly it's the most amazing thing in the world, and we should implicitly trust it to make our decisions and educate our children.
> If I told you I have a person who speaks perfect English but frequently lies and outright makes things up, you'd say that's a politician and not to be trusted. But if I say I have a computer with those same properties, suddenly it's the most amazing thing in the world, and we should implicitly trust it to make our decisions and educate our children.
Because a computer is a tool and a politician deems you a tool.
Anyway machine learning will change the way languages are taught. It did change how people learn games like Chess and Go. Having a conversation (with voice-to-text) with a bot in your second/third language will be an important part of language acquisition. It's honestly hard to imagine how it's not the case for me.
I am learning German, and I will read something that I don’t quite understand, make my best guess at what it means, and run it through Google translate. Then I compare the results and it helps me see where I went astray. Similarly, I write something that I think is proper German, and again compare the results to what Google translate says it should be. It’s sort of like having a native speaker sitting next to you and correcting you (except you have to come up with the explanation of the correct answer on your own, which is where knowledge of gender, conjugation, and declension comes in handy).
University language schools had many discussions about how google translate and friends effected how, and what, they should teach — now they are having more talks and discussions about how ChatGPT effects things.
Are you maybe just in tech circles, not academic language circles?
I feel like you're missing the point. If you're having a machine translate it for you, you aren't learning German. Similarly, if you're having a machine translate an English-language request into Python, you aren't learning Python.
To say that, as the article says, "Professors are shifting away from syntax and emphasizing higher-level skills" essentially means "Professors have stopped teaching programing languages and have started teaching how to turn natural language problems into AI prompts."
Higher-level problem solving skills are important, but I am skeptical of eschewing a lower-level foundation in favor of only teaching higher-level problem solving skills.
Thanks for this, really helps ground the current situation for me.
But as another commenter said, the analogy isn't 1:1. The example would be "Students don't need to learn German, they can just use Google translate for their time in Germany (and be hated for it by the locals)"
Ah, but the analogy isn't about using German. It's about learning German. If I signed up for a German class and what they taught me was how to use Google translate, I'd be pretty angry.
>you'd say that's a politician and not to be trusted
And yet, it seems that kind of politician is common and seemingly rules the world at this point. I'll let you decide what that means for the future of humanity.
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Kernighan's Law
With an LLM it is possible to write code that's too "clever", thus you won't be able to debug it. And in many cases neither can the LLM.
> Good luck debugging code you aren't skilled enough to write.
We already have this problem, even without the LLM-generated code. I see younger developers pull out all sorts of tiny libraries because a) they think it will save them time, b) they are insecure about the correctness of their code, and c) they often have no idea where to start when problems start to happen (solving problem is usually writing a github ticket or blindly trying another library).
I think LLMs will exacerbate the problem. These days I rarely see engineering spirit in figuring out how things work, willing to understand deeply OS or compiler internals, or even trying some crazy ideas, like "let's write a JIT to speed up our DSL".
> Good luck debugging code you aren't skilled enough to write
I mean, at the end of the day I am not a coder, therefore any code I debug I'm probably not good enough to write, yet I've found numerous bugs in both open source and proprietary code with suggestions on how to fix it.
"We should be making AI a copilot—not the autopilot—for learning."
Doesn't work. A friend has been a teacher, at a not cheap private school, for going on 25+ years now. What she sees at almost every level is an avalanche of LLM "cheating". Even when there is extreme upfront emphasis on "this course is to help you learn, not just do whatever to find the 'correct' answers as quickly and easily as possible" the majority of students do the latter. She guesses that at least half are pasting in answers without even reviewing them. There are of course the few students that are there out of passion for learning, but those are the exceptions. This doesn't just apply to CS.
There are cultural and political ramifications that make enforcing this difficult (read rich parents that want those good grades). So she struggles along trying to swim against the current of LLM answers.
Quoting her directly, from about a month ago, "If they cannot type something into their phones and get an instant answer they're not interested."
I saw a post on reddit a while ago that described this terrible situation:
...a kid was tasked with researching Peru "in the news." After 10 minutes I went to check on them and they had nothing done. They told me there was nothing about Peru in the news and showed me their Google search results. They had googled "in the news" only. They didn't include the word Peru in their search. Not only did they not use the technology correctly but they didn't ask for help or say anything after their one attempt to get information failed. They just sat there.
A bit like I personally think teaching programming in a typed language is a good idea so that the kids internalize that conceptually before they jump to a lang with auto casting.
Same with the tools...I think there is value in doing it the hard way initially.
I was listening to the Clean Code audiobook, and the author made the case that every programming paradigm introduced restrictions into what the programmer was allowed to do; i.e., operated at a higher level of abstraction. The author then claimed that since the existing paradigms all came out in quick succession without any new paradigms in decades, and there was no power left to strip, no new paradigms would be likely to emerge.
And here we are in 2024 watching the emergence of natural language programming, where we merely describe the program's specification in natural language and have the computer generate the rest. There's your new paradigm.
> And here we are in 2024 watching the emergence of natural language programming, where we merely describe the program's specification in natural language and have the computer generate the rest. There's your new paradigm.
Is it? When you take procedural, object-oriented, logic, and functional programming, they did not only changes how we code, they also change how we think about the solution. We had new entities and we could describe the relations between them to describe an algorithm. Natural languages have been there forever, but the thing it lacks is formality. Every term describe a very large class of objects and specifying which one you're referring too is arduous. The kind of specification needed to describe problem solving steps is what created programming languages in the first place.
Programming languages are not some arcane art that we need to suffer through. They're the best way to specify to a machine how it should work to do a task. Maybe AI can be the new machine, but its capabilities so far are not that impactful IMO.
Many programs already start as natural language specifications by product managers. Consider a simple search engine. The spec could simply be "take a query and return a paginated list of results ranked by personalized relevancy".
That simple request is enough to generate a mountain of code. The real problem starts when you want to iterate.
> The spec could simply be "take a query and return a paginated list of results ranked by personalized relevancy".
The real problem is specifying what any of the quoted terms means (nouns and verbs). What is a query? What is a result? How many items in a page?... And then you recurse from whatever definitions and descriptions they gave you until what you have is terms from your favorite programming languages. Somewhere in the middle, you land into whitepapers and research materials. And sometimes the result is only good if you have PBs of data and lots of compute powers.
Having lots of code is not where you want to start unless you like to manage complexity from the get go and have clear understanding of the domain space.
Most engineers are perfectly capable of understanding this specification without an explanation of each term. We can deal with ambiguity. So can ML models; that's what they're for. When we are not sure, we can ask for clarification or look it up. So can today's LLMs...
I think natural language programming is a lot like dealing with a contractor.
In his latest book ("Functional Design ..."), Martin argues that functional programming today is what object oriented programming was in the 80/90s. Hopefully, this is correct. Before everyone learned common design patterns, people did lots of random confusing stuff.
No, LLM is not a paradigm for programming, but it might be a new paradigm for autocomplete, refactoring, translating, and so on.
I think this trend will cut both ways. I have seen engineers, including senior-level, who think their job is just to write excellent code. This breed is doomed.
I have also seen juniors with amazing ideas and ability to iterate, who get that the job is delivering value to customers, who are held back by their lack of expertise in the craft. This breed will flourish.
The old maxim “90% perspiration 10% inspiration” may shift as automation provides more leverage. This could be great for younger, less-entrenched, more-flexible students.
I am not convinced. I am teaching a numerical techniques course in python. I have been thinking of how to use the LLMs for the course but haven't found the way that doesn't short-circuit learning.
I.e. the analogy that I came to mind is, when you are learning multiplication table, the calculator is not useful, it only useful when you already know the table and need to multiply large numbers. Similarly, when learning basic coding techniques having an LLM is not useful as well.
I agree with the emphasis on high-level thinking and problem decomposition in current curriculum. The real craft of writing a program is abstracting components to interface with each other. AI tools suck with unclear instructions but excel at low-level implementations. The curriculum at my university is already designed to teach us the design and problem solving solving skills, and oftentimes I found those skills quite complementary to what AI tools can do.
It’s important to know exactly why the code works once it does, you could avoid it but it will catch up to you because that code may have a bug in some exceptional cases
Having great taste will be the most valuable skill. Artists like Picaso trained their eye, then he painted his masterpieces. You don't have to play an instrument now to make music on computer. Elvis had very basic guitar playing skills yet played amazingly beautiful songs that moved millions of people.
Being able to pick out what Ai generates with taste is what will count.
Something else less related that I find really useful about these chatbots is their ability to take some abstract explanation of some problem/attempts to tackle it, and it'll often spit out a name for it, and explain that this sort of problem is already well studied. That can be really difficult with other more traditional tools.
I feel like this could be a big deal in science in general. People of different fields often do unnecessary extra work (as in reinventing the wheel) because they're not aware that there's, say, 10 years of research on it.
To take an extreme example, just think of the poor fella doing medical research who rediscovered integration[0].
> I feel like this could be a big deal in science in general. People of different fields often do unnecessary extra work (as in reinventing the wheel) because they're not aware that there's, say, 10 years of research on it.
Maybe an introductory search in other fields is what required? Or heading to the library? Even in software engineering, we see people reinventing the wheel because they can't be bothered to read a computer science book.
On the one hand, this is something I also hope LLMs will be able to do.
On the other hand, it seems to be something LLMs are particularly bad at. There are many similar but subtly different concepts. People use the same words for different concepts and different words for the same concepts. Every paper invents new terminology. And there is often very little written material available.
> People use the same words for different concepts and different words for the same concepts.
Left unchecked, it will be amplified by LLMs. Detecting and ignoring the poisoned data requires way more "intelligence" than the current approaches can achieve.
Not sure if it'll exactly be bad, some naming schemes are just bad anyways. Old man yells at a cloud using words incorrectly...
These tools to me serve about two functions: writing regular expressions (and even then I'll tell it it's wrong once or twice before attempting to check the output), and saving me keystrokes for code I already know I'll write but is easier to describe. In both cases I already know what code snippets I want mentally, and it saves me minutes at most.
I'm glad I grew up in a time before computer science degrees were common. I learned to write Algol 60 (1969), Basic (1970), Fortran (1974), Z80 (1978), 6502 (1981), all simply as tools to solve a problem. Were we ahead of time then in emphasizing high level thinking?
IMO most proposed changes to teaching using AI are misguided. My philosophy regarding AI copilots and teaching is: as calculators are for those who already know how to compute, so are AI copilots for those who can already develop software. They are tools to leverage when you know the craft, to accelerate. They are not there to do what you don't understand.
What does it multiply? I'm genuinely curious because besides insecure examples of a new framework I (might) want to use, I haven't seen a great use-case.
My reaction has at best been "oh look how cute it is", like watching a kitten try to walk, when I have tried various LLMs out there.
I made a comment about this above, but here's an example:
I'm doing a complete rewrite of a C# project with Rider and Copilot integrates to it directly. This allows me to:
- Have the previous project open and ask questions about it from the LLM, like "Where is the DbContext initialised" or "How does FooModel connect to the database" and it'll give me the direct answer and links to the .cs files where it found the information.
- In the new version I can paste a bunch of SQL CREATE statements to Copilot and it can create the models for them automatically. It knows how to do foreign keys both in [annotations] and using the fluent model based on the statements I gave it.
- I can ask questions like "how can this be done more efficiently" and it gives decent suggestions. Like I learned about the C#11 'required' keyword[0] by having it be suggested to me :)
- I can make it do the tedious code changes for me like "write me the primary key generators for entity framework for classes FooModel, BarModel and BazModel" and I can just click "insert at cursor".
- It's also pretty well integrated to projects, so the code suggestions it gives are usually pretty well on point. Like I can just start typing DbSet and it'll automatically give me each of the models in the project (linked via a shared project) one by one as completions 100% correctly. Takes me 5 seconds to add 5 DbSets.
Working with an LMM takes some getting used to, just like with pair coding in general.
Your "partner" doesn't know what you think, so you need to communicate what you're doing efficiently and in a way they can understand. "Prompt engineering" is a stupid term, but it's still valid. There is a certain way you need to communicate with LLMs so that they understand what you want and give you a concise answer.
I use it like I would a Junior coder, give it the simple tasks I know 100% how to do myself, but can't be bothered because of the tedium.
My experience with GitHub copilot is that I have to pay double attention to not let in anything stupid. The best are suggestions for what should be printed in logs.
For me it’s amazing over the course of 20 years how junior engineers keep finding new ways to commit shit code. It used to be called Javascript now it’s this.
I think I agree with your point (I would pick a different example, though)
LLM programming removes the need to write tedious code, which means the author has fewer opportunities to ask themselves if there are easier ways to model the problem they're solving. We would never have gotten language features like Objects, Iterators, first-class functions, and exceptions if all of the boilerplate to emulate those features could be autogenerated so easily.
Going forward, there will be less impetus for programmers to pick better languages and abstractions for their work, since the boilerplate cost of picking the wrong one is subsidized by the llm.
So I think the issue isn't so much that people won't know the syntax rules and bizzare-semantics trivia, but that people won't realize that the syntax rules could be different and that the whole point is for them to make your life easier.
What I think he means is that when you program, you take terms that has specific meanings (in whatever abstract machine the compiler/interpreter embodies) and arrange them according to the rules in order to describe the steps of a solution so that the machine can solve it automatically. The syntax are not that actually important, you can compose new terms from basic terms and create a language that way. You abstract that way, expanding or reducing the vocabulary you have to work with, finding some specific relations between the term that facilitates describing (paradigms). Syntax is important because the machine only understand those specific terms arranged in those specific ways. It just that the set of problem it can solve is very large, the solutions more so, and your program is just only one element. Solving the problem is not programming, restricting it to what the machine can do is. And for that syntax is what you got, because that's what the machine accept.
The correct analogy here is grammar, not spelling.
Someone who hasn't mastered the written grammar of the language in which they wish to write a novel, is unlikely to produce anything someone else would want to read.
Prerequisite, yes, but it is not the thing itself. Just because you can spell doesn't mean you can write a novel, or every Spelling Bee winner should become an award winning novelist. The meaning of the words, the structure of the novel, characterizations, etc. all matter as well, and are ultimately much more important than just spelling things correctly.
Defining programming as just syntax is nonsense. The SHOUTED IS (like we're on AOL again for some reason) places outsized emphasis on syntax. Either that it is "just syntax" as randomdata wrote or the majority of programming as the SHOUTED IS could also imply, makes no sense. Just creating a syntactically valid program is insufficient for creating a correct or useful program, just as correctly spelling a string of words is not enough to create a novel.
“This program, print the numbers from 1 to 10 in python”
In order to run my simple program just copy paste it into your LLM, of choice and execute it
Notice that the program does what it supposed to do it is deterministic unless you use something like gpt2 and it solves the problem that I aim to solve
Programming most definitely isn't syntax. When I visualize software solutions in my head, not a single bit of code is up there. Syntax is just a textual representation of concepts.
> When I visualize software solutions in my head, not a single bit of code is up there.
If you envision a mechanical solution without anything like a mill or lathe up there, do you consider yourself to be machining?
> Syntax is just a textual representation of concepts.
Mechanical parts are just the metal (or whatever material) representation of concepts. Does programming not refer to processes around taking those concepts and turning them into something 'tangible' of a certain variety?
It seems to me that programming is, in fact, just syntax (within some loose interpretation). As you call attention to, syntax doesn't mean much in a vacuum, but the bigger picture that includes thinking about concepts usually gets the "engineering" moniker.
I can explain a bubble sort to anyone, but if they want to program one they’ll need a programming language.
What distinguishes programming languages from: random text, spoken languages, or even other programming languages? Syntax.
I hate that you define a function in python with: def func(): ; in clojure with: (defn func []; and Haskell with: func = ; just as much now as anyone starting to explore an interest in building with technology.
I hate that to participate with the internet you need to know 3 programming languages: html, JavaScript, and css. (While completely ignoring the backend.)
But it is that way because different people have different ideas of how to move these 0s and 1s around.
Now perhaps one of these overly optimistic proselytizers of the llm teachers might say, “exactly, and we can use this tool to abstract that madness away.”
Fine, but it’s something different than programming.
I say that without any condescension of either party. What’s it matter if the desired goals are reached? I mean, except that these tools are a silicon fire created by stealing the worlds creative output, and owned by unethical sociopaths who are trying to sell it back to us.
But when discussing education, this distinction is important to ensure programming is still taught in parallel to people who are only taught to use an llm.
Have we reached the apex of programming? Do all we need is to just stochastically complete tokens of preexisting code?
Or do we still have work to do in this field?
Who’s going to do the programming if everyone in the discipline learned only how to avoid the programming?
> I hate that you define a function in python with: def func(): ; in clojure with: (defn func []; and Haskell with: func = ; just as much now as anyone starting to explore an interest in building with technology.
That's just tradeoffs. Perfomance of C vs easiness of python, flexibility of lisp vs (rigidity?) of Java.
> I hate that to participate with the internet you need to know 3 programming languages:
You only need one (see Gemini protocol), CSS is for getting it pretty and Javascript is for some interactivity. We can probably concoct something in lisp to have only one language (jk). I think it's better to teach HTML first with style and script tags and attributes.
> Now perhaps one of these overly optimistic proselytizers of the llm teachers might say, “exactly, and we can use this tool to abstract that madness away.”
There's no madness. There's only engineering and business tradeoffs. You learn the paradigms, you learn how they map to the Von Neumann/Turing|Lambda Calculus|... machines and you're set. Now what's left is learning the libraries for existing solutions and interactions with other systems.
I agree that programming is not "using llm". Programming can be llm assisted (utility varies), just like it can be IDE assissted, snippet assisted, books and SO assisted.
I don't believe we've reached the Apex of programming. Society evolves and we will have new problems to solve. We still have not finished solving existing one.
I’m not sure why you’re being downvoted. Among all CS students, when they are weak, the #1 problem I observe is a poor ability to “see” code down to the character. They literally do not notice mistakes. I’ve seen their ChatGPT histories, their questions are too vague. It’s two sides of the same coin.
Absolutely atrocious idea, but not really novel. The idea that you can go through shortcuts and remove rigorous training in CS has been happening all over higher ed, it's why every new generation of CS graduates to be frank feels more clueless than the last one. I include myself in this to some extent, my university education about 10 years ago already felt like a glorified bootcamp and I had to learn a lot myself.
But I do thank at least one prof I had for having us take tests on paper and deduct one point for each syntax error, it's probably why I pay at least some attention when I write code.
Saying programmers should not focus on syntax and this "it's just like, concepts bro" is such a pothead mentality, it's like saying you can be a great novelist if you need to use auto correct on every third word because spelling doesn't matter. Find me one great novelists who does not have an excellent grasp on language because you have spelling assistants. If you don't have precision, a grasp on the mechanics and memory and the fundamentals down you'll be worse than useless in any discipline.
by the time current freshmen graduate, even AI-first CS education will be outdated. There will be no new jobs left for humans, only the old incumbents clinging to their tenure as all knowledge work is automated
Well before AI co-pilots something happened to the good old admins--they started to disappear only to be replaced by "AWS devops" (their job titles) who have never wired a network using routers, switches, and cables. I noticed that they started lacking basic networking knowledge and couldn't set up networking inside AWS. They just didn't know what a gateway, NAT, or subnet is.
Similar things are happening with AI co-pilots. We have an increasing number of people who "write code", but the number of people who can understand and review code is not increasing. There is also a problem of injecting ethics and politics into those tools, which can produce silly results. I asked Bard to write me a Python function to turn the US Constitution into a palindrome. Bard refused and gave ma a lecture on how the US Constitution is too important to be played with in such trivial fashion. I then asked it to produce code that turns the US national anthem into a palindrome, it refused again. So I asked it do the same but with the Russian national anthem and it spat out code without telling me off. I then asked it to generate code for simple tasks and it did an OK job, except the formatting and the fonts used were different every time, because it just lifted code from different webpages and recombined it like a massively hungover student waking up to realise he's supposed to hand in the assignment in one hour.