> "Nearly everyone recognizes that MOOCs are by and large a failure with ~2% completion rates, but they make us feel good because now it’s the students’ fault not they’re not learning, not the school’s"
I understand the context the author is approaching this from - but it discounts the idea that MOOCs can be used the same way we use books.
I've utilized about a dozen different MOOCs and I've _never_ completed a single one of them. Instead I learned the concepts I needed to, maybe completed some test material if it was freely available, and moved on.
Some of those MOOCs were gateways into my career as a developer - so they were absolutely vital to me. This again seems to be an issue with measurement/metrics - I don't believe completion rates are worth that much in MOOCs, just like they aren't worth that much in technical books.
Not knocking your own choices, but its more the backlash to the 2013 "MOOCs are the greatest thing ever! KhanAcademy/EdX/Coursera! Wooo!" Personally, I love them and would love to work with those companies. In fact, when I started teaching, I used many of the aspects I saw MOOCs using in my traditional classrooms.
However, there is a truth to the statement. Almost no one finishes a MOOC (some never even log in to take MOOCs they register for), and predictive models can determine whether someone will complete the MOOC as early as the 2nd week (if not sooner). Some people do grab what they want and leave, but that is not the case for everyone. If we then look to justify why bother, its a little hard to warrant hosting costs, etc. if you aren't seeing anything come from it. If I ran a MOOC with a 95% fail rate, how can I label it successful.
We are unable to do meaningful research on MOOC strategy effectiveness because we are too busy trying to understand why people never log in in the first place.
> some never even log in to take MOOCs they register for
I've done this many times. Why? On some platforms (like Coursera) you retain access to the content of a course you have registered for, but that content isn't available for people who didn't register once the class is over. If I think I might ever want to view that content, my best bet is to register (there's no cost) and then have that content available when I want it.
I've also done this on platforms like Udacity where that's not the issue, because by registering I can create a curated list of courses I'm interested in possibly looking at in the future.
In my experience, the big failings of MOOCs is that they try to copy university classes, and university classes just aren't that good a lot of the time. Many people don't find lectures to be useful at all, but the vast majority of MOOCs focus on them. After the fifth time I'm stuck waiting for the teacher to finish their long personal anecdote or humorous story I usually give up on the MOOC and go looking for a good textbook.
I don't think I've ever seen a MOOC that had written content anywhere close to the quality of Dive Into Python or Learn C the Hard Way.
Also, MOOC efforts for community building are (at least were) focused on a particular class rather than a particular subject. This doesn't lend itself well to long-term community building, especially when people are going through these courses and very different paces.
There seems to be lots of discussion about how MOOCs can be improved, but not much effort into actually trying different approaches.
> the big failings of MOOCs is that they try to copy university classes, and university classes just aren't that good a lot of the time
We are still trying to understand what makes for better "learning". If I practice the piano everyday, I will no doubt have a better ability to play, but I may not understand music theory. As a shameless plug, I am trying to add practice to be a core part of learning Computer Science and a link to my research platform is in my profile.
This is the same separate of vocational schools and coding bootcamps to graduate schools. The traditional 4-year school sits in the middle ground between application and theory and is probably why we have such difficulty deciding "success".
> to finish their long personal anecdote or humorous story I usually give up on the MOOC and go looking for a good textbook
This is ultimately student preference. Some students like a more affable teacher, some don't want a social-able instructor (so long as its not interfering). I will say that your post is confusing as you say you don't want instructor anecdotes but then MOOCs should be doing more for larger-scope community building.
I will argue larger-scope community building is difficult and MOOCs struggle with it; I would say because of its availability. It is harder to build community when the individuals come from such diverse backgrounds. Many factors can make it difficult to build bounds (social, cultural, temporal, etc.). Furthermore, you cannot control why the student wants to take a MOOC. They may not want to give that additional effort to community building for a subject they do not perceive as having higher priority to other life decisions.
I would disagree with "actually trying different approaches". This is being done and academic journals are trying to study them (the deadline for the Journal for Education Data Mining just passed). There are also non-academic approaches like Duolingo and Khan Academy, and they are trying different techniques as well.
Ultimately, there is no panacea for learning. We know engagement can lead to learning and therefore if we can maintain engagement, we might be able to teach. Community-building can be that engagement. Self motivation can be. At that point we need to research these things more to know for sure.
Sure. But the last time I checked, almost all MOOCs were lecture focused. Like I said, I've yet to find any with a text component as good as Dive Into Python or Learn C the Hard Way or other free online textbooks. Perhaps there's are a few out there, but every time I've browsed MOOCs I've found just about everyone to be lecture focused. Out of the dozens I've looked at (on multiple platforms) I can only think of one that wasn't, and that was eventually removed.
Plenty of people use the internet to learn plenty of things. I think it's time to consider how much of the "failure" of MOOCs is due to the failure of online learning, and how much of it is due to the fact that university classes aren't a great way to teach people things (at least for a large chunk of the population).
> I will say that your post is confusing as you say you don't want instructor anecdotes but then MOOCs should be doing more for larger-scope community building.
Instructor anecdotes were an example of why I don't like lectures. With a book you can scan over content that is superfluous and get to the information you need. You can re-read or take slowly the parts you have trouble with, and quickly skip over the parts you already know. All of this is much, much harder to do with lectures.
I'm confused as to why you think professor anecdotes are related to large-scope community building? They're quite different things, and if people view them as serving the same purpose then there's even more of a misunderstanding when it comes to education than I had previously believed.
As for the difficulty of community building, I think you should broaden your horizons a bit. For instance, if you're a bit late on a Coursera course the forums are pretty much dead, and you're better off discussing things or asking questions on another site. Likewise with Udacity - the course might still be active, but everyone that has completed the course has moved on and won't see your message. This isn't because of a difficulty in community building, but because of a conscious decision that most MOOCs take to segregate their forums by class.
(I will ask if you could define "community building" a little more. You did not in your original post, and it seems my assumption was incorrect. Before trying to address better, I'd like to have a more concrete definition)
While I cannot speak for all MOOCS, I have not had the same experience. MIT's CS 6.00x course on EdX was work heavy, Udacity's Web Development course was work heavy, and even Coursera's Design of Everyday Things had participation components. In the only (barely a) MOOC that was video only, it was predominantly a "follow along with me" coding process.
For instructor lectures, I give more positive responses than negative about my anecdotes. At the end of the day, I'm human and I'm trying to enjoy my job (which students can tell if you don't). Development of tutoring systems is still in research phases as we identify knowledge components for different subjects. A history course operates different than a computer science course.
I will say EdX has produced research on instructor lecture videos. Users prefer 3-5 videos and so instructors should design courses like that. Instructors that simply upload a classroom lecture are not appropriately transitioning their material for online use. This satisfies your being able to flip from topic to topic.
I will end the Dunning-Kruger effect suggests that students are not the best at assessing what they know and that a professional instructor has a better idea of what "knowing" something means. This again gets back to my discussion on vocational vs. graduate school and again, humans are flawed, imperfect beings. While an intelligent tutoring system can alleviate this issue, humans are still building them for the foreseeable future.
> Users prefer 3-5 videos and so instructors should design courses like that. Instructors that simply upload a classroom lecture are not appropriately transitioning their material for online use. This satisfies your being able to flip from topic to topic.
In my personal experience, it really doesn't. Breaking something into segments can make things a bit easier, but it still leads to a lot of wasteful time if there's a minute and a half of useful information within an 8 minutes block. You can't scan through it the way you can with a book, and it's much more difficult to review a difficult piece of information you just received (it's easy to re-read a sentence slowly, whereas rewind a video to the beginning of the last sentence is more cumbersome and you're going to be watching it at the same speed).
Text also tends to be much more succinct, where as lectures are often repetitive and meandering.
I think the assumption that multiple video segments solves the problem is instructive. Might not a large part of the problem be instructors saying "We've done X, which has solved Y issue" instead of saying "We've done X, let's look at whether or not it has solved Y issue"? I appreciate the fact that people are attempting to solve these issues, but if there's no differentiation between attempting to solve something and successfully solving something than it shouldn't be a surprise
> I will end the Dunning-Kruger effect suggests that students are not the best at assessing what they know and that a professional instructor has a better idea of what "knowing" something means.
This is a pretty big assumption, and one that I don't think is accurate (based on personal experience, and my experience talking with both students and professors). The best way I've found to test one's ability is to actually apply it to a task, where it usually becomes quickly clear to the individual where the holes in their understanding are.
Both are important. There are multiple questions to be asked.
For extremely self motivated people, MOOCS are a win, no doubt. Again, MOOCs are definitely better than no MOOCs.
But the promise was that MOOCs can just replace college to a large degree, because the content you get in college will be online. In that instance the question becomes "do MOOCs work for most people?" For what we'll call the "average" person, studies have shown that just isn't the case. Most people don't learn well from MOOCs, unfortunately.
Therefore what we're learning is that a non-trivial amount of what makes a college education successful is some combination of the following things that MOOCs don't have: external pressure, scheduled courses, due dates, a community of learners, a physical campus, etc.
As such, I view part of the next step to figure out which of those aspects MOOCs are missing that are vital to the mix in order to help average people learn the things they need to know. Traditional education just says all of them are necessary, but I'm not convinced that's the case.
Do most people learn well from traditional methods though?
If dropping out of a MOOC involved a big financial hit, social humiliation, and the loss of a great deal of personal freedom, the completion rates would skyrocket. Completely different incentives.
The reasoning that traditional education is successful because it has high completion rates because of external pressures is a bit circular, isn't it?
It also works the other way. It's socially/professionally acceptable to stop working for 3/4 years to go to university, somewhat less so to stop working to take a bunch of MOOCs. Therefore, most people taking MOOCs are also working full time/studying/looking for a job, and they're also likely to be older and have more life commitments than most students. So MOOCs are naturally going to take a back seat to everything else going on in a person's life. They also don't carry the same recognition as having a degree.
MOOCs also come with a number of unique advantages compared to traditional universities:
- They widen access to education to people that aren't able to attend university
- People can take courses they know they might struggle with without fear of flunking them and harming their academic record
- Mature students don't need to make life sacrifices in order to take a MOOC. They don't have the dilemma of whether to up roots and move to attend a prestigious university
Perhaps. The question is which of those things really move the needle; can you have some, not all? For example, our completion rates are 90%+ - better than universities, but there is no social humiliation, loss of personal freedom, or financial hit.
What is the typical demographic of your cohorts? How do you compare to college programs that have similar cohort demographics?
I think all educational outcomes should be measured with respect to the answers to these two questions.
K12 schools in extremely wealthy areas can compare themselves to like peers, but comparing themselves to high-poverty areas is a useless comparison. Neither institution would do well under the other's constraints.
Could you cite the studies that say "MOOCs don't work for the average person"? While it is certainly an intuitive statement, I find it hard to empirically verify because the population of people who take MOOCs are very different from those who attend colleges.
For example, those who take MOOCs are probably working full / part-time, trying to pick up some knowledge on the side. They will obviously have less time for learning than full-time students.
While "average person" is not the term I'd use, the question you're asking is a bit loaded. What measure is used to say a MOOC "works"? If we are saying completion, then high drop out rates are an issue. If we say high interactivity among those that don't drop out, this is where the current field of research in MOOCs is at. While I do not study it, colleagues in my lab are.
My point is that quantifying the effectiveness of a MOOC is an ill-defined domain. Instead, we acknowledge the attrition rate and do analysis on those that pursue the course. Current research looks at system interactions, the social networks of MOOC forums, etc. to identify student behaviors that show higher "gains", be it course completion or grade.
If a student leaves a MOOC, there is no way to identify why they leave. Likewise, unless asked, it is hard to identify why a student joins a MOOC. Students enrolled in a MOOC can come from a variety of culture, geographical, sociological, motivation, etc. backgrounds. As such, assuming traditional student behaviors is ill advised. However, in the process of learning, access to material is not enough to learning the material and additional effort is needed by the student to build the necessary mental models in their head. If a student drops out of a MOOC, it is hard for the runners/analysts of the MOOC to appropriately say the student learned the material.
As austenallred mentions, current education research is looking at the motivation of learning, as well as what constitutes motivation (and learning for that matter).
>some never even log in to take MOOCs they register fo
So what? How is this at all relevant?
If we look at how many people have walked into a bookstore or library, have browsed through a book but never bothered to actually buy and read it, would we say books are a failure? Would you say books have a 95% fail rate? This is such a pointless train of thought.
Why did you decide that MOOCs should be held to the same standard as libraries or bookstores?
MOOCs are marketed as courses, and sometimes as an alternative form of higher education. Furthermore, I doubt the people who go to the trouble of designing curricula for MOOCs expect students to drop out partway through. Comparing MOOCs to college courses seems like a better match to the image the MOOC companies themselves have promoted. And if we make that comparison, completion definitely matters.
The fundamental difference is that MOOCs are recorded rather than performed live. This makes both the distribution and access patterns more similar to books than to college courses.
Of course any particular MOOC can be made more or less like a normal class by putting the content behind a hefty paywall, enforcing strict due dates, and adding perverse performance incentives.
> Some people do grab what they want and leave, but that is not the case for everyone.
Are there metrics that have measured this? What percentage of users find a MOOC useful even if it wasn't completed?
What I was trying to get at is that by looking at completion rates we might not be looking at the whole picture. Especially if the question we're revolving around is whether the course is worth the creation/hosting costs.
> some never even log in to take MOOCs they register for
[...]
> we are too busy trying to understand why people never log in in the first place
So you have at least 4 classes of 'user':
- did not register
- registered but never logged in again
- logged in at least once
- completed course
It seems to me that there's a different way to categorize the users in measuring course effectiveness:
- did not find the course useful/helpful
- did find the course useful/helpful
So if you are able to craft a course that has magically high engagement, but people don't find it particularly useful - you've still failed.
Maybe the value that course-makers hope to deliver through completion of the course simply isn't high enough?
If a user completes 70% of a course but doesn't see the need for the rest of it, then stopping at that point is the logical move. What sets college/university apart in this is degree which has a very high social impact. The fact remains that MOOC achievements are still widely looked down upon as somehow lesser than their college equivalent.
This whole discourse has encouraged me to revisit those courses I found most effective and complete them. At the very least I can help those courses stats and provide some further narrative of their utility.
A major issue with using "useful/helpful-ness" as our metric for success is quantifying the term. A self-report Likert scale is susceptible to users just rating 1 or 7, or saying its useful
even when it was not (https://en.wikipedia.org/wiki/Self-report_study#Disadvantage..., not the greatest citation, but I am short for time).
The issue then looks at what is "success" in a MOOC. If the goal is to just have videos online, I'll just watch YouTube (as I have my own lecture series there). However, observing is considered one of the lower level of learning and things like Bloom's taxonomy point out there needs to be some type of interaction for better learning gains. These interactions require the student to have a more active role in the learning process. If they are not interacting with the system (logging in, watching videos, completing exercises, etc.), then they are not taking this needed active role. This is where "drop out" begins to be quantified and where then we can measure what worked, what didn't.
To address you example, if the user stops at 70%, researchers will ask "why?" From there, analysis of student behaviors, effectiveness of interface/instruction/material, etc. will arise. If the ability to study these things is confounded by the fact that the vast majority quit before completion, it makes it harder to answer the "Why" and "How do we fix it" questions.
Again, what we quantify as "successful" is still up for debate; but if a student drops out, it becomes harder to tell if they learned from the course and if it was student or material that drove that decision.
I suspect something like Udacity (on which classes rae not free and there are more structures 'degree' programs) has far higher completion rates, since it's not reasonable to just impulsively sign up and then stop. That is certainly true for something like Georgia Tech's OMCS. I still think the hype was justified in that stuff like OMCS is legitimately the future of education.
Given the nearly non-existent barrier to entry, I'm not sure that completion/engagement percentages mean much. MOOC content can be useful for help learning certain types of things. But so can books, YouTube, etc. Useful resources for (mostly) continuing professional education has to be seen as a very large step back from changing the future of education for underserved populations.
Why would you complete a MOOC "cover to cover" when they offer no accreditation? People just pick up what they need and move on. If they were offering a complete program and degrees, millions would complete them. MOOCs are amazing, but a simple VOD system can't replace the education system on it's own.
I can also confirm I've had this experience. I learned a lot from MOOCs I never completed. That's why I like Khan Academy's model for educational content - aligned in paths, and with some gamification around completing, but ultimately made to be searchable, and relatively easy to pick up in the middle. And that's how I've used it as well - I probably have only a module or two completed, but I've watched many videos just trying to get a better grasp on one concept that wasn't well enough explained in a book I was using, or just to refresh my recollection.
The biggest, most idiotic, and most frustrating problem IMO is that most MOOCs are inflexible in their schedule. This model is perfect for self-paced learning and very few serve that.
Plus, there are instructors and materials that aren't great, and are more useful for the basic introduction to concepts in the first few lessons, which can then enable self-learning that's more effective than the poorly taught meat & potatoes of the course.
(I've finished probably 3-4 MOOCs and didn't finish about the same amount.)
My personal thought is its the same reason people quit gyms , fail at New Year's Resolutions, or even quit Free-to-Play games. It is difficult to create and maintain habit.
Maybe the problem is in measuring them as sort of an all-or-nothing experience, like nothing happened if you didn't see it all the way through. Perhaps we should be judging progress on the basis of individual modules: did you master the material in this one? Or equivalently, the courses should be smaller and better linked with each other: "this course requires you to have passed one of these other courses...".
I think what stops us from using that model is that each course might define its terms differently and assume slightly different pre-requisites such that it's infeasible to teach just one small thing in a way that it can be directly applied to someone else's minicourse.
Exactly. I completed 5 out of ~50 courses I have ever enrolled in. One of the courses I did not complete was instrumental in getting me into graduate school in a field unrelated to my formal background. Focusing on the completion rates misses the point.
Similar experiences here. I get what what I need from a couple of videos and never look back.
I wish that lecturers would move to YouTube and structure each lecture as its own instead of having the burden to fit them into a course. For example, if I don’t have the appropriate prerequisite knowledge to understand what I’m currently watching, I should be able to search for and learn it quickly instead of having to browse through intermediate lectures.
I understand the context the author is approaching this from - but it discounts the idea that MOOCs can be used the same way we use books.
I've utilized about a dozen different MOOCs and I've _never_ completed a single one of them. Instead I learned the concepts I needed to, maybe completed some test material if it was freely available, and moved on.
Some of those MOOCs were gateways into my career as a developer - so they were absolutely vital to me. This again seems to be an issue with measurement/metrics - I don't believe completion rates are worth that much in MOOCs, just like they aren't worth that much in technical books.