> Some people do grab what they want and leave, but that is not the case for everyone.
Are there metrics that have measured this? What percentage of users find a MOOC useful even if it wasn't completed?
What I was trying to get at is that by looking at completion rates we might not be looking at the whole picture. Especially if the question we're revolving around is whether the course is worth the creation/hosting costs.
> some never even log in to take MOOCs they register for
[...]
> we are too busy trying to understand why people never log in in the first place
So you have at least 4 classes of 'user':
- did not register
- registered but never logged in again
- logged in at least once
- completed course
It seems to me that there's a different way to categorize the users in measuring course effectiveness:
- did not find the course useful/helpful
- did find the course useful/helpful
So if you are able to craft a course that has magically high engagement, but people don't find it particularly useful - you've still failed.
Maybe the value that course-makers hope to deliver through completion of the course simply isn't high enough?
If a user completes 70% of a course but doesn't see the need for the rest of it, then stopping at that point is the logical move. What sets college/university apart in this is degree which has a very high social impact. The fact remains that MOOC achievements are still widely looked down upon as somehow lesser than their college equivalent.
This whole discourse has encouraged me to revisit those courses I found most effective and complete them. At the very least I can help those courses stats and provide some further narrative of their utility.
A major issue with using "useful/helpful-ness" as our metric for success is quantifying the term. A self-report Likert scale is susceptible to users just rating 1 or 7, or saying its useful
even when it was not (https://en.wikipedia.org/wiki/Self-report_study#Disadvantage..., not the greatest citation, but I am short for time).
The issue then looks at what is "success" in a MOOC. If the goal is to just have videos online, I'll just watch YouTube (as I have my own lecture series there). However, observing is considered one of the lower level of learning and things like Bloom's taxonomy point out there needs to be some type of interaction for better learning gains. These interactions require the student to have a more active role in the learning process. If they are not interacting with the system (logging in, watching videos, completing exercises, etc.), then they are not taking this needed active role. This is where "drop out" begins to be quantified and where then we can measure what worked, what didn't.
To address you example, if the user stops at 70%, researchers will ask "why?" From there, analysis of student behaviors, effectiveness of interface/instruction/material, etc. will arise. If the ability to study these things is confounded by the fact that the vast majority quit before completion, it makes it harder to answer the "Why" and "How do we fix it" questions.
Again, what we quantify as "successful" is still up for debate; but if a student drops out, it becomes harder to tell if they learned from the course and if it was student or material that drove that decision.
Are there metrics that have measured this? What percentage of users find a MOOC useful even if it wasn't completed?
What I was trying to get at is that by looking at completion rates we might not be looking at the whole picture. Especially if the question we're revolving around is whether the course is worth the creation/hosting costs.
> some never even log in to take MOOCs they register for
[...]
> we are too busy trying to understand why people never log in in the first place
So you have at least 4 classes of 'user':
- did not register
- registered but never logged in again
- logged in at least once
- completed course
It seems to me that there's a different way to categorize the users in measuring course effectiveness:
- did not find the course useful/helpful
- did find the course useful/helpful
So if you are able to craft a course that has magically high engagement, but people don't find it particularly useful - you've still failed.
Maybe the value that course-makers hope to deliver through completion of the course simply isn't high enough?
If a user completes 70% of a course but doesn't see the need for the rest of it, then stopping at that point is the logical move. What sets college/university apart in this is degree which has a very high social impact. The fact remains that MOOC achievements are still widely looked down upon as somehow lesser than their college equivalent.
This whole discourse has encouraged me to revisit those courses I found most effective and complete them. At the very least I can help those courses stats and provide some further narrative of their utility.