Hacker News new | past | comments | ask | show | jobs | submit login
ChatGPT has entered the classroom: how LLMs could transform education (nature.com)
43 points by rntn on Nov 15, 2023 | hide | past | favorite | 56 comments



I generally find LLMs to be useful, but I think these kinds of proclamations are a bit overblown. Many students do not want the kind of introspective, solo learning experience that a chatbot on a laptop provides. They want a social experience or a physically active experience, not one that requires them to sit quietly in a chair for 8 hours a day. And honestly, a classroom of students staring at screens talking to AI bots sounds like a dystopia to me.

All of the best educational experiences I've had in high school and college came from a passionate teacher that conveyed excitement to the students. Or (especially in college-level philosophy courses) they were small discussion groups that went on for hours. Neither of these things is going to be replicated by talking to a chatbot and most of these applications are just further extensions of a (in my opinion) broken industrial educational model.

That aside, I do think LLMs could be useful as a supplementary tutor, especially when your teacher isn't up to par.


And yet, individual tutoring provides a two-sigma improvement in outcomes compared to a mass lecture. Besides, language generation can be just as helpful in group settings. It just becomes "let's see what the robot says and talk about it" instead of everyone talking to the robot on their own.


Huge assumption that talking to a chatbot = individual tutoring. People keep making this analogy and I don't buy it at all. Having a physical human being in front of you is a lot different than talking to your computer.


I absolutely buy this but also think we should figure out what works about the human tutor and try to get it into a free piece of software so that the 99% of kids without individual tutors can get a similar benefit.


I'm not sure it's something that can be replicated by software. Human beings like human beings. Physicality isn't just an add-on, it's fundamental to who we are. A person sitting next to you explaining something is much, much different than someone or some thing teaching you via a device.

An AI tutor that passes the Turing Test and communicates via speech is roughly comparable to an anonymous human teacher teaching via Zoom. There's no physicality and no continuity to the relationship.

To use an example: my high school history teacher was an old ex-Marine with a buzz cut and big square glasses. I enjoyed his class because he was a cool guy who seemed to have seen the world, and I was interested in what he had to say about wars and historical events.

With an AI, no matter how realistic it is, there is zero of this context. Sure, it can pretend to be a grizzled ex-marine. But who cares what it thinks? A teenager is going to find it lame and boring, not equivalent to an interesting individual tutor.


> no physicality

So it requires embodiment? Rodney Brooks might agree. You're saying it like it can't be done, but I'm hearing it just requires hardware.

> no continuity

Feels like a very tractable software problem.


It's the class sizes. When I was growing up we would have close to 50 kids in class.


Well, then just have the class communicate with the chatbot in study groups, taking turns to type, if "social" is what you're after. The way class time is currently used reflects the need to keep that grade level engaged with appropriate activities, and only as you get to the older ages does it converge on that kind of solo self-study. But it has to, eventually. You don't do research without figuring out how to do it. The flaws in leaning on LLMs are just another form of "precision vs accuracy" - GPT is always precise and not always accurate, just in the verbal domain instead of the numeric one. But we do have many tools like that in the numeric domain. The limitations can be learned.

If the LLM gives a solid three-star "fast food" education, that is actually considerably better than letting it all fall on the shoulders of the babysitters that currently serve in many classrooms.


> not one that requires them to sit quietly in a chair for 8 hours a day

Sounds like the public school I went to... Sit in your assigned chair, don't talk, little physical activity. Prison for kids.

I'll take the AI tutor that can be tailored to each kid and in any environment.


> And honestly, a classroom of students staring at screens talking to AI bots sounds like a dystopia to me.

Would it be less dystopian if it were replaced with a hologram of a historical figure / scientist speaking as though an actual primary source on the matter but with the same LLM behind it?


Not really, because I don't think kids are going to find a piece of software to be a compelling teacher. Having a physical presence is important for physical beings like humans. My guess is, they'd just ignore or make fun of a hologram. The "appeal" of a historical figure doing the teaching is also not really compelling to anyone that isn't already interested in history.


Your teachers must have been much better than mine. A well produced YouTube video is better than any in-person school lecture I've received. Combine that with 1-on-1 next-generation LLM fine-tuned to be a helpful tutor and it will make the best possible education today, readily accessible to a lot more kids.


Same-same. The problem with the ChatGPT-tutor future is not "having to read".

Also, I believe that having tens of thousands of teachers with their unions and quirks has its issues, but is a much more resilient system at the nation level than a centralized tutor system. (The centralized tutor is too easy to game, bias, get seeded with native ads, etc.)


As someone who learned by listening to teachers and otherwise skipping all the readings I find that not compelling


Yeah, I'm the exact opposite of you there.

I would just never listen to whatever happened in class. My strategy was to be best friends with the top-of-the-class student, and spend the classes copying her notes from the previous class. Listening was so boring and slow and sleep-inducing.

But my point is that if the plan is:

> We will sit your kids to chat with a black-box AI bot for the duration of their school term, and the bot will teach them about life

Then the problem is not "why don't we give them Airpods to listen to the black-box AI chatbot, instead of reading what the black-box AI chatbot is typing"


I get the idea, but the dystopian part is the students being continually isolated in their educational time with their only contact being an impersonal computer program (or something that fakes being personal, but is not actually free to be personal).

The computer dressing up in human form doesn't really change it.


I always get very skeptical with putting more technology in the classrooms (at least here in the US). The primary problem is funding and too many kids for a single teacher. Educational innovation comes in with a bang and out with a whimper when the study turns out to be flawed, often quite severely.

I'm sure LLMs can augment learning in some settings, esp in higher ed, but putting more computer time for kids learning basics (I mean K-8 mostly) I hope is handled more carefully than things like Quizlet...


One of my high school teachers had a quote she was fond of repeating:

"Education requires three things: a teacher, a student, and a stump. Unwilling to change the first, unable to change the second, educators spend all their money on the third."


> I'm sure LLMs can augment learning in some settings, esp in higher ed

LLMs are great explainers of known theory. They can adapt to the level of the user and illustrate with concrete examples. They have infinite patience, there is no pressure.


> LLMs are great explainers of known theory.

When they don't lie. When they don't make up citations that don't exist.

I think GPT-4 is moderately useful for software development, and I use it daily, but I have to check even very minor things, and having ~25 years of programming experience means I can do that intuitively. A kid doesn't have that.


> When they don't lie. When they don't make up citations that don't exist.

I think this will be close to 0 of a problem when explaining to a HS biology student something like, what a mitochondria does or what mitosis is. It only fails (increasingly rarely) when you push the boundaries of human knowledge.


That assumes that the asker doesn't then ask follow-up questions that do probe beyond the could've-been-a-Wikipedia-page starting point. And I don't know about you, but how `structlog` should be configured had better not be "the boundaries of human knowledge". As I experienced yesterday evening, GPT-4 doesn't even do that right without human assistance.

LLMs are useful tools. They are dramatically less useful if you can't detect bullshit. By definition, a student is poorly equipped to do so, and moreso if allowed to atrophy in the area of research capacity because they can just ask an LLM. (A pocket calculator is a good analogy--life carries much less friction when you can efficiently do arithmetic and basic algebra in your head, because the affordances of dragging out a calculator to know how to minimize the change you get back are bad.)


GPT4 with no tuning produces less bullshit than 99% of teachers.


I don't think it can. I have some experience in teaching and one of the worst issues I initially had was that you can't believe how messed the knowledge can be in heads of the kids. They literally mess everything – topics, words, culture, personal experience. Everything. To teach kids effectively you have to have a lot of background knowledge – from their family background to the info from teacher in previous class and you have to adapt constantly.

People who are talking about automation of education are grown up people who think that all people think structurally and just lack the specific knowledge. Kids actually don't do it.


Honestly, I think the primary problem in the US is parents completely offloading the responsibility of educating their kids onto the teacher. You need to have the appropriate push and environment at home for a successful education. You would never see that in South or East Asia.


This is a very economically driven situation in the US. Upper middles load their kids up non stop with educational stuff outside of the schools, but the lower middle on down can't afford this (it's really expensive) and school often functions primarily as child care.


I think you’re discounting relatively cheap fixes like mindset and pushing your kids to succeed at home. There’s plenty you can do besides throwing money at a problem. A lot of first generation Indian and Chinese immigrants exclusively rely on those methods, and this is usually after a day of working long hours at intensive jobs.


One nice thing with LLMs is that there are no "stupid questions". You can throw it whatever is on your mind and get some insights.


You're right; but it should be noted that that is a design choice not an inherent property of the technology.

Currently, the design goal isn't to make LLMs feel "lifelike." It is likely that lifelike LLMs will be released in the future, which could result in sarcastic or dismissive replies to poor questions or missing information.


Backwards. LLMs will naturally adapt lifelike patterns of their training data and are at least easy to prompt into behaving as such.

We tune them to specifically not do that.


I expect public school systems to place limits on LLMs. "Don't ask about that." "You can't talk about that topic at school." And so on.


As this technology becomes cheaper and more specialized, this is really exciting. Imagine being able to ask your textbook specific questions about what you're trying to understand.


Youtube was for many years the best place to find educational material. Sadly every year is harder to search for with all those "you may also like", "people who searched this also search this other irrelevant thing", "shorts".


I'm a PhD student, mostly off on my own doing dissertation research and writing at this point, and typically getting one hour per week one-on-one with my advisor to ask questions and get guidance. So I'm already in the habit of reading and experimenting as much as I can to figure things out for myself, leaving only the toughest questions for those meetings with my advisor.

Since he turned me on to using ChatGPT, however, I've found that I have fewer and fewer of those questions. ChatGPT seems to be able to answer most of them for me, instantly instead of waiting until next Monday afternoon, and probably better than my advisor could answer them off the top of his head.

With that said, most kids are not mature PhD students who have already "learned how to learn", know what they need to learn about, know when and how to verify things by consulting other sources, and have an intrinsic motivation to attain a deep understanding of things.


I think the elephant in the room is that if LLM's improve at the same pace as diffusors. Then there won't be much to educate towards. What does a decade of agentic multimodal model progress look like?

https://rocketdash.io/midjourney-and-the-progress-of-ai-in-2...


I disagree, I think very little of what is taught in school would be made unnecessary by LLMs.

We teach maths, but calculators didn't stop this. Even if LLMs can take a less structured question and call a calculator, it just means maths moves more high level, and to understand the goals students would still need a foundational understanding anyway.

We teach language for communication, and we'll still need to communicate. LLMs can write emails for us, but only if we can communicate what we want written. We need to be able to understand the output, decide what we want to change about it, and voice that back to the LLM. They're a time saver but don't really remove the necessity of the skills development in some sense.

We teach languages for communication with others. Translation apps haven't stopped this. Being able to speak to someone is still worth a lot. If this does get solved in the future it'll be by UX improvements not AI improvements.

We teach creative skills, but we know most students don't go into creative jobs. These skills are taught for pleasure, for cultural reasons, and to create well-rounded individuals.

We teach history, geography, and science, not so that students can regurgitate facts (even if that is what ultimately gets tested) but because we as a society believe it is important to understand and learn from what came before us and how the world around us works, and be able to ask the right questions about the world.

LLMs, if they continue to improve, might change how we assess students, and they might change the focus somewhat, just like the introduction of the ubiquitous home PC or the internet did, but I don't think they're going to eliminate most of what is taught in school.


I think the fundamental question to be answered is "Will humans be needed to move society forward in the future?"

People stand behind "Sure, AIs can write emails, but they won't know what to write emails about". But to me that doesn't make sense. The holy grail of AI, what we are racing towards at a breakneck pace, what every company really wants, is the AI that knows what to write the email about...

OpenAI, Meta, Google, homebrew. None of them are trying to build a calculator. They are all trying to build a calculator user. It's a very big difference and their progress has been nothing short of remarkable and unexpectedly fast.


Even then, I don't think that changes many or any of the arguments. Perhaps you could drop the maths, but people still need to speak to each other, will probably still write to each other, will probably still write to persuade or describe even if it's just for pleasure. People will still want to know how the world works, we should definitely still learn from our history. As I said, the focus on why we're learning may change, and the testing may change, but I don't think that just because there are no jobs that require a human to write an email means we won't teach kids to write.


Are we looking to create AI agents? Citizens, not slaves?


Early education especially is not something that AI can handle, because it isn't just about learning facts or processes. Take small motor skill acquisition, for example: you have kids play with play-dough, finger-paint, fold origami, play musical instruments, and learn various handwriting styles like block and cursive and calligraphic styles, all to teach kids how to move their hands in complex ways to achieve what their brains want. AI isn't going to do that for kids, and tablets seem to have made motor-skills acquisition worse since the range of movements needed for a tablet is much smaller than that needed for 3d manipulation of objects in the real world.

I don't think AI is going to excel in language education, either: I get bored chatting with a bot very quickly. Language education is all about keeping students engaged and communicating about tasks they actually care about. I don't see chatbots like ChatGPT doing that.


> We teach... we teach... we teach...

Reading this, not sure about anyone else's experience but very little written here reflects how any of those subjects were actually approached and you know just the assumption that because you have something called a "teacher" in a class called "x" means that the "teacher" actually taught "x", because yeah from my experience a lot of it was just about regurgitating facts, no creativity was never really taught and was often punished, actual speaking and actual communication was never really taught only judged, and no way did "learn from what came before us and how the world around us works" ever occur within the walls of any classroom I was unfortunate enough to sit within.

Wouldn't ever risk my kids future by throwing them into that system, honestly feel extremely lucky it didn't completely screw mine up.


I tried to get it across but perhaps the point didn't land – I'm not talking about any given classroom or the practical aspects of it. Curriculums have all sorts of issues, teaching is increasingly a job of meeting quotas, school funding is problematic all over the world, and examination sucks.

However, we're still choosing as a society to send kids to school. The fact that school systems suck in so much of the world doesn't stop it from being the right idea in theory, and if you ask almost anyone on the street "should kids learn to read" or "should kids be taught about their history" you'll almost unanimously get answers of absolutely. (We can debate what "their history" means, but most people agree something should be taught, they just disagree on what)


Is it true that calculators did not reduce math education on tasks that the calculator automated? I find that dubious.


I didn't suggest it reduced education on tasks that the calculator automated, I just don't think it reduced the desire of society in general to educate kids in maths. We now just have a blended approach of learning enough of the basics to understand why a calculator is valuable, and in some cases more advanced maths that students are able to tackle because of the availability of calculators. My point is that we decided as a society that we couldn't stop teaching kids maths just because calculators exist.


This really could be a plot for "Idiocracy 2", showing humanity hundreds of years in future, where all education was given up in favour of LLMs and a bunch of undergrads/startups.


I'm not sure how this connects. Diffusion models are great for certain tasks, not for others. They have sort of a niche use in audio and are the SOTA for image generation, but there is a lot of education (almost all) that doesn't apply to those domains.

When we're talking about multimodal ML systems, the core is generally a transformer-based LLM with no diffusion component.


I think the point is that diffusion models improved at a very fast rate, how do we think things will look if LLMs (in whatever form) see the same rate of improvements?


Ah, ok, I read that wrong. You're right. Thank you.


Related, Sal Khan of Khan Academy, "How AI could save (not destroy) education" https://www.ted.com/talks/sal_khan_how_ai_could_save_not_des...


For some jobs, why bother with classrooms? Throw people straight into a job and have the LLM teach them as they go. It’s a return to apprenticeship but your overseer is an AI. Cheap and very scalable.


Why not hire an agentic multimodal instead? When they are developed.


How accurate is it compared to the average teacher?

In other words, what's the teacher error rate vs the AI tutor error rate? And what happens when these rates achieve parity?


Error rate is not as relevant as patience, flexibility, and availability to answer questions


Perhaps, but isn't it better to refrain until someone at least tries to answer the (grandparent) question?


No?


> And what happens when these rates achieve parity?

Is there data to suggest that teachers are lowering their error rate enough to eventually reach parity? In particular out-pacing the technology which continues to lower its error rate.

I don't think error rate is a useful metric, otherwise we'd already be replacing teachers. There is much more to teaching than being hyper-accurate.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: