
Online Education's Dirty Secret - Awful Retention - pkrein
http://rein.pk/online-educations-dirty-secret-awful-retention/
======
pamelafox
(Coursera engineer here) It's great to see thoughtful critiques like this.

Thankfully, we have a data analytics team now (3 out of our 17 engineers), and
they are studying our retention statistics and the factors that affect it.
They're also running A/B experiments to see what increases it and getting some
interesting results.

We do see some big advantages of the timed model for learning, particularly in
classes with peer-to-peer grading and evaluations, but there's obviously a big
desire for the self-study mode, which is enabled for a few of our classes
currently.

We're also introducing things like Signature Track, which some students sign
up for just to encourage themselves to make it to the end (and it seems to
work for many of them).

We'll keep experimenting to see what makes students both happy and successful.
:-)

~~~
jahmohn
Hi, Pamela. I am a long-time user of Coursera and I love the site.

I have feedback for you as regards retention of students.

The number one thing is due dates and late penalties. If I sign up for a
course and work through the first few weeks and get everything done, and then
I miss a deadline because life happens, suddenly I am no longer able to get a
100% in the class. I do not mind getting an A- in a class if it happens
naturally because the material is difficult, but losing my A+ or A because I
am 12 hours late on an assignment ruins the experience for me. I usually drop
a class when this happens and wait for it to come around again.

A counter-example to prove the rule: Robert Sedgewick's Algorithms course. The
last time he gave it, he had late penalties and I only had time to get through
the first two weeks of the class without getting behind. This time around,
there were no late penalties. Being enormously busy with other things, I was
not able to keep up with the class as it went along. However, because I lost
no credit for doing things late, I was able to complete the entire class in
the final 10 days of course and I got an A. I have no problem staying up all
night to finish interesting programming assignments and quizzes, but I need to
be allowed to do this on my schedule.

I strongly encourage you to encourage your professors to do away with late
penalties. It should be up to us as students to determine what our work
schedule is. I know for some courses, those that use peer-evaluation, this is
impossible, but many more could take this route than currently do.

I agree that Signature Track does have the effect of encouraging one to keep
up with the class.

~~~
olalonde
Genuine question: why do you care so much about your grade?

~~~
VLM
Both sides are pretending its a "for credit" class. So both draconian late
penalties and great concern over the grade are appropriate.

------
henrik_w
Taking a Coursera course is _hard work_. I've taken 3 courses so for (one on
databases and two on algorithms) over the past year and a half. This is about
as much as I can manage (I work full time and have a family), even though I
would like to take more (so many interesting to choose from). But the main
reason for this limit is the amount of work it takes, not how the course is
presented, or how well written the mails are.

For me, the pace of the course is actually a plus. If there were no deadline,
I simply would not get around to doing it. I remember when I first found all
the MIT courses on-line (several years ago). I really wanted to take some of
them, but because there was no schedule and no deadline, I never got around to
taking any of them. It's only with Coursera that I have actually taken (and
finished) any.

I've written about my experience of all three courses on my blog. The latest
was Algorithms: Design and Analysis, part 2
[http://henrikwarne.com/2013/02/18/coursera-algorithms-
course...](http://henrikwarne.com/2013/02/18/coursera-algorithms-course-
part2/)

------
nbouscal
I don't see how something can be a "dirty secret" when everyone already knows
about it. This is not news, and I think the author is well aware of that and
just wanted an eye-grabbing headline.

As for courses being too fast: they're college courses. That's a core part of
their value proposition. If they were slower, or did not follow a rigid
schedule at all (like the "better" examples presented in the article), they
would be a fundamentally different product. The author simply doesn't
understand the concept behind MOOCs, and probably isn't their intended
audience. He would be better served watching lectures on Youtube (I don't mean
that sarcastically, there are fantastic courses available there).

Synchronous learning is not, as another poster claims, an anachronism. It
simply isn't necessary or valuable for everyone. For some, myself included, it
is a significant benefit. That is the market that Coursera and edX are
targeting, and one shouldn't criticize a company simply because one isn't a
member of their target market.

~~~
lawnchair_larry
Or should you not criticize dissenters just because it happens to fit you?
There could be a better way for more people, regardless of your personal
experience.

~~~
nbouscal
There is a substantive difference between the service offered by iTunes U,
Youtube, and countless others, and the service offered by Coursera and edX.
This difference corresponds to a difference in learning styles (synchronous v.
asynchronous, as it has been phrased in this thread), and it doesn't make
sense for any one product to target two diametrically opposed learning styles.
What does make sense is the approach that Coursera has actually already been
taking: offer both as separate products. You can take classes on schedule, or
you can take them off schedule for no certificate. Complaining about the
synchronous model because you are an asynchronous learner simply doesn't make
any sense.

~~~
lawnchair_larry
Nothing that you've listed is an asynchronous course. Currently, the choice is
synchronous course or asynchronous video playlist. Coursera can likely fill
that gap with a single database flag. It makes plenty of sense to suggest that
they consider doing so.

You're mistaken, actually - in most cases, coursera does not offer taking
classes off schedule for no certificate. That's exactly what everyone is
asking for.

~~~
colin_jack
I'm not sure everyone is asking for that. I think it would seriously water
down the courses in terms of discussions in the forums, sharing online (study
room, hangouts), in terms of the quizzes and assignments.

The only thing I'd like is the more intense 6 week courses split over say 10
weeks. Other than that I think they are taking the right approach.

------
manifold
My problem was mainly the availability of the materials. Codecademy and
duolingo give you access to as much as you require from the start and you can
go through as quickly as you like. The university driven sites limit access to
so much per week (though I'm not sure how courses will operate the second
time) and demand you stick to their schedule, though granted this may be due
to their need to peer-review the more demanding assignments.

Unfortunately my free time isn't available in nice predetermined six-week
chunks, but even if I am able to catch up three weeks or more in a weekend the
courses gave a very negative vibe about continuing to progress as soon as you
miss a single one of their deadlines (i.e.- "you missed our deadline for this
multiple-choice computer marked test, so your effort no longer counts"). I've
'failed' several coursera sessions in the fourth or fifth week for that
reason.

Timetabling seems a very traditional educational view, and it contrasted
sharply with codecademy and sites like duolingo where I spent Jan and Feb
learning the basics of new languages - computer and human. I finished the
courses I took because I did them at my own pace.

------
viveutvivas
The way that Udacity does it is perfect for me -- work through at your own
pace. Coursera has a couple of self-paced classes, I believe, and I wish that
more of the courses had that option.

I think the majority of MOOC course developers want to run their online
courses as closely to their university versions as possible, since that's what
they're used to working with. It also provides a handy way for them to go off-
duty, in a sense, if the course has a finite end date. Using their current
pacing structure (which is incredibly difficult, as the OP points out, for
people that aren't full-time students) allows the teachers to do something
other than devote themselves solely to the course, assuming they don't want to
just post an archive and leave it alone -- which would meet a lot of people's
needs, but misses the whole teacher-student interaction, which is pretty much
missing from MOOCs anyway. If they want to provide an environment that's like
a classroom, with students interacting with the instructors and with each
other, you kind of need everybody at the same pace. It would be nice for us if
they could slow that pace down, but that would probably increase the workload
for the instructors.

We're still early in this game. I'm glad that so many professors have been
willing to invest the time into developing the courses, and I understand why
they are currently set up to be conveniently structured for _them_. I think
we'll start to see some improvements if/when the money appears in the MOOC
game. Once it's no longer basically charity work for the instructors, there
will probably be more efforts to work around student schedules.

~~~
goostavos
I agree completely on the Udacity model allowing you to work through at your
own pace, as well as keeping the course open After it finishes. The strict
scheduling thing with other sites drives me mad!

In addition to having a normal job, I'm a traditional university student, so
any of the online classes I take are simply out of interest in the subject. I
dip into the class when I have time, almost like a leisure activity. I just
don't currently have the time to fit in more strict course work on top of my
already over loaded class schedule and work week.

------
spikels
Everyone needs to understand that the low completion rates are not as bad as
they at first seem. The way to try out these classes for free is to sign up.
There is no obligation at all: completely voluntary, no cost, no time
commitment and it will not show up on permanent record.

These completion rates are actually more like conversion rates for free
trials. As many of us know these are almost always quite low. How many people
actually ever read books they "Look Inside" on Amazon? How many people finish
long articles on the web? Now many people sign up for paying accounts after
free trials of your new website? Not very many.

They are not the same as dropout rates in high schools or universities despite
what some online education haters say[1]. Like any disruptive technology the
existing players are threatened so you need to pay attention to the source of
the criticism.

Instead completion rates are a metric that can be used to improve the class
such as the suggestions here. The developers of online classes should and are
using A/B testing to improve completion rates. However like almost all
conversion rates they will likely remain low.

[1] [http://mitsloan.mit.edu/shared/ods/documents/High-Costs-
of-F...](http://mitsloan.mit.edu/shared/ods/documents/High-Costs-of-Free-
Online-Education.pdf&PubID=5082)

------
UnoriginalGuy
That's because online education tries to simulate traditional education.

Back awhile ago free online lecture videos (e.g. Berkeley) were simply
uploaded to a page and anyone could go watch them without signing up for a
class or otherwise jumping through hoops.

This resulted in me watching a few random lectures about subjects I know
little or nothing about. Like econ classes, engineering, science, or similar.

This was good because it was short bursts of information without any hassle,
commitment, and similar.

These days everyone has locked their content behind sign up/registration
pages, and you're expected to commit to a tradition semester of a class in
order to get a worthless certificate of completion.

They also dictate the speed at which you learn. No more learning at your own
pace, no, you have to do one a week for as many weeks as it goes on for like
it was a traditional class.

And why? Why indeed. Why is online education simulating University education
when it actually has no relationship to it? You aren't getting a University
qualification by completing a "class," there is no reason why a "class" has to
be X number of weeks or you should have to complete the rereq's in order to
take it.

Places like Khan Academy have got this right. I can go to Khan Academy right
now, click on a video series, watch a few bites of information and then stop
when I'm ready/bored. Coursera's class system is just pointless, stressful,
and annoying.

If Coursera was transferable then it would absolutely make sense to do. But
that doesn't look likely and while that isn't the case they're just making
access to learning material more difficult with no obvious benefit to anyone.

~~~
shawabawa3
> I can go to Khan Academy right now, click on a video series, watch a few
> bites of information and then stop when I'm ready/bored.

You can. But you wont.

The sad fact is that deadline motivate people. Without having deadlines and
structure coursera would have gone nowhere.

~~~
lawnchair_larry
This claim is baseless. Please reserve it for after various MOOCs have tried
different models and the data supports it.

It's possible and even likely that the schedule contributes to motivation for
many, but my experience has been the opposite and I'm probably not unique.

~~~
summerdown2
You are not unique. I much prefer studying at my own pace, too.

I found class timescales so demotivating I dropped all the Coursera courses in
favour of the Udacity ones as soon as it became obvious other things in my
life would keep getting in the way.

------
DennisP
I actually often "never really intended to seriously take the class in the
first place." Coursera doesn't let you view the materials unless you sign up
for the class, so I sign up for every class I might be interested in browsing
a little. Then I can view the materials at leisure, even long after the class
is finished.

On the other hand, I mostly completed the first algorithms course and Model
Thinking, and while Daphne's class kicked my ass the first time, I'm already
reviewing to tackle it again and try to finish it this time.

It is true though that the time commitment is horrendous, especially for
Daphne's class, which estimates 15-20 hours per week! If I fall behind, I may
finish slower than their schedule and miss the homeworks and final exam, in
which case I'll officially be an incomplete but I'll still have gotten through
all the material.

In short, retention is an irrelevant metric for online classes, especially for
Coursera which has such a strong incentive to sign up "unseriously." My
suggestion would be to keep accepting and grading homeworks and exams at any
time, and give extra props on the certs for completing on time with the other
students.

------
Jormundir
You should title this article "How to cheapen the online education
experience". You may be right, that edX and Coursera require a big commitment,
but that's what learning requires. Do you want to play games for an hour, or
do you want to advance your skills and understanding?

(I've completed the Codeacademy Javascript track, and a full Coursera class
(livin' in the 5%, wooo!)).

You say Codeacademy got it right, but I think you're dead wrong. As other
articles today have pointed out, at best you're going to learn syntax on
codeacademy, and the most basic programming principles, but you'll be miles
away from being a capable programmer. The jump from the online editor with
tiny little exercises to setting up your own environment and programming your
own project is huge. Many of their lessons give misleading or outright
incorrect information, not to mention teaching you bad practices.

What this article says is the huge barrier to entry, is not a barrier to entry
to online education, but a barrier to entry of learning. What the author
proposes is solutions to hide the barrier of learning, by gamifying the
platforms and making them more "fun" with a disregard for the depth of
effectiveness, and the consequences of pursuing such techniques. (Obviously
I'm delving into opinion land, but I strongly believe gamification cheapens
and platform and doesn't produce the deep engagement necessary for learning.
This is my assessment from my experiences, I'm not going to tell someone their
wrong for feeling differently about this. But I think if you wanted to create
a deeper experience from gamifying education, it would take a tremendous
effort, far different from the typical "Wow you completed 5 exercises! ZOMG
KEEP GOING!" joke of rewards that other websites have.

I don't want to rail on the online education effort either, but this is plain
cheap thinking. Coursera and EdX are doing an excellent job, but they
certainly have many problems to surmount. They've definitely solved the
problem of higher education accessibility, now the problems they have are in
the effectiveness of education, and on administration and grading that gets
the f#$% out of the way of learning. But making it "easier to learn"? I call
utter bulls$%t.

~~~
asafira
I think I agree with most of your points, but disagree with your disapproval
of "easier to learn". The presentation of material really makes a gigantic
difference on how well you learn the material (if at all). While I agree that
coursera and edX shouldn't hasten towards "gamification" of any sort, I think
there is plenty of work in making the classes of better quality, and I think
that _will_ make it easier to learn.

I was taking a course on coursera where I thought the professor was talking
too quickly --- so I actually slowed it down to 75% the pace in the video.
Awesome! I was happy to be able to do that.

Otherwise though, I don't think the course was presented that well. For one,
lectures were shitty recordings of slides; I was looking at nothing but text
and static images for entire lectures (never seeing the professor's face), and
they were blurry, hard to read, and just hard to understand. Of _course_ that
makes it harder to learn! I have to focus on figuring out what the hell is
written there! And there was no (free) textbook for the course.

Anyway, I think the point I am just trying to make is that while the author of
this article might be suggesting to Coursera to take steps in questionable
directions, I DO think a lot of those courses could be better polished, and
that would make it easier to learn from them.

~~~
Jormundir
Very agreed. There's certainly much these online courses can do in the lecture
quality department alone. (Maybe they just need more quality control instead
of a rush to fill their sites with content). I think you can do a lot to make
the learning process much smoother and more streamlined. My opposition to the
suggestions in the article is on the grounds that the author is searching for
a shortcut to the learning process (or maybe just wants a really addicting,
very shallow introduction to the topics), which is exactly opposite of what
these sites are aiming to achieve.

I'm all for making these platforms more fun, engaging and rewarding. The more
the better. But my main point is that the focus, the most important aspect of
education is the process of learning and that often requires prolonged focus
(to follow along and venture all the way down the rabbit hole of a subject),
deep engagement (so your brain can make all of the connections, tie all of the
strings together), and interest. The authors suggestions don't really
reinforce the process of learning (as I know), they may even weaken it by
creating distractions, giving rewards for too little work (obviously
subjective, but being subjective you're going to help some people sometimes,
and hurt some people sometimes) and getting students to focus on the rewards
rather than their understanding, and rather than building implicit positive
feelings about their accomplishments.

------
nnq
Why do people think _retention_ is a good thing?! I think _retention_ is
_actually bad_ even for classical education.

The lower the retention rate, the more the likelihood that they are following
lots of courses at the same time or multitasking with something else. This
means that _people are "exploring" more!_ I think that "exploratory learning"
has always been stifled by classical organized education and is one of the
reasons why I hate most academic environments (yeah, they're cool if you're in
one of the top 10% unis or in a "privileged" position", but not for the rest
of cases...). I believe that, after a certain level of baseline knowledge, in
any field, it's actually _more important that someone learns "what they
want/need/have inclination for/find more interesting" than that they "learn
more"!_ Maybe more time exploring and less time actually uploading things to
your mind is better (not "productive", just "better", and I said "fuck
productivity" a lot lately because I found that it just doesn't lead to better
anything). All the new ways of doing education make exploration easier (as in
you can explore a lot without really "wasting" that much time), and we should
take advantage of this!

------
darrellsilver
Pacing is incredibly important to motivation. The thing is, there's no single
answer. Even in traditional, full-time, in person classrooms where everyone
puts in the same class hours, natural ability, different skills at the
beginning and better attention mean that students will always learn at
different speeds.

The ideal model is one of personalized education. What's kind of shocking is
that online education so far has basically ignored this altogether. There's
been zero investment in ways to continually pace and encourage students to
learn at a speed that is both challenging and within reason.

This isn't too surprising: it's a hard problem, and it's much easier to have
users paying a subscription fee forever (for those services that are paid) and
blame themselves when they fall behind.

At Thinkful (<http://www.thinkful.com/>) we see evidence that our learning
model, which pairs experts and students together much more like you'd expect
from tutoring, sees 10x the retention and completion as other online course
options.

~~~
lawnchair_larry
This is not only a shameless plug but is actually a load of crap.

~~~
darrellsilver
Please elaborate. Unless "load of crap" fully articulates your argument. If
so, thanks for the feedback.

------
KC8ZKF
My mother used to tell me that my eyes were bigger than my stomach when I took
food more food than I could eat. I have the same problem with MOOCs.

The future me always has more time, is always more dedicated to learning, is
always more focused. So I sign up, and when the class starts I'm no longer the
better, stronger, faster, future, me, I'm just me.

Still I manage to finish some, and learn a little from the ones that I drop,
so I don't see it as a problem.

------
droithomme
Another week, another article complaining that teaching a class which 5-10
thousand people successfully complete, and which cost less than $10 per
successful student to provide, is an abysmal failure.

Meanwhile, the schools that the authors hold up as more successful examples of
education have 20-500 successful graduates per session offered, nearly all of
whom are idle young wealthy white people living in rich western countries, and
who pay up to several thousand dollars per session each for this privilege.

------
jeffdavis
The article seems to imply that, in an ideal world, all of these student would
stay throughout the course. But I don't think it's clear that the churn is
bad.

There is some lower bound to the amount of time that it takes to learn
something (absent some educational revolution). Many people aren't even
willing to pay that lower bound simply because they have other priorities in
life. But they do like to sample (which is good), and potentially finish out a
course if nothing else gets in the way.

Online courses are great because they lower the cost of sampling, so we
shouldn't be surprised that there are more samplers, and fewer people
finishing out the courses. If 1000 people complete an online course, then
that's great, even if 99000 people signed up and disappeared a week later.

That being said, I'm pleased to see the specific criticisms offered in the
article, and I hope they lead to a better balance for more people.

------
waterlesscloud
The author says courses shouldn't have deadlines, but then talks about
retention that's measured as of the deadlines.

I signed up for the Hinton neural network class when it was almost over, and
completed it well after it ended.

Was I "retained"? Did I not count as completing the class?

Just because there's deadlines doesn't mean you have to finish it by the
deadline.

------
kvb
Thankfully the article is much better than its title. Low retention is not a
problem _per se_ \- since there's no cost to signing up for (and subsequently
dropping) these online courses, it's to be expected that retention won't be
great. However, I agree that retention could probably be improved among the
marginal subscribers by using some of the techniques covered in the post.

------
tunesmith
For the past year, I've only been seriously interested in three Coursera
courses - Scala, Ng's Machine Learning, and Probabilistic Graphical Networks.

All three overlap at the same time!! And they don't give assurances of when
they will be offered again, or even if. Argh. I've started Scala but doubt
I'll be able to continue if it I attempt PGN again. This happened to me last
fall and I ended up completing none of the three.

Plus, it is apparently "known"... somewhere... that certain courses are easier
if you take other courses first... but good luck finding that information when
you want to refer to it.

~~~
waterlesscloud
I agree the scheduling makes it difficult. I'm doing the NLP class right now,
and scheduled to start the PGM class in a couple of weeks. I'm worried about
the overlap, but I'll make it work out.

But I also wanted to do the Scala course. I didn't start it because I know I'd
not be able to keep up with all three when they overlapped for a few weeks.
It's a shame, but I had to prioritize.

------
andrewem
My wife also wanted to take that Introductory Human Physiology course. She's a
freelance medical translator, so improving her knowledge of that subject
matter will have a direct positive impact on her career. She's not among the
people who Daphne Koller said "never really intended to seriously take the
class in the first place" - she bought the suggested books ahead of time,
marked it all down on her calendar, and planned out time to work on it.

What she found was that the course was way more difficult than the description
suggested, and proved to be well above what she could understand with her
current background. She watched the first video and found that it was going to
take way more time than the course description predicted, because she would
have to try to fill in a tremendous amount of background knowledge that the
teachers presumed. On the plus side, from the course forums she found about
resources (videos, etc.) that were more at her current level, and has been
working through these.

The comments made elsewhere about the synchronous model for these courses is
probably right for her case - she had to wait a number of weeks in order to
find out it wasn't the right class for her. This is what you expect when
you're in college, at which point you either drop the class and maybe find a
different one to take, or tough it out in a class that's not as interesting or
useful as you'd hoped. I hope online courses can do better than this.

~~~
eah13
> What she found was that the course was way more difficult than the
> description suggested, and proved to be well above what she could understand
> with her current background. She watched the first video and found that it
> was going to take way more time than the course description predicted,
> because she would have to try to fill in a tremendous amount of background
> knowledge that the teachers presumed.

It's quite possible that this is a function of having so little open content
out there right now. I can imagine a future where the background your wife
needed would be easy to get from the platform itself, so that the profs'
demanding teaching could serve audiences both with and without that
background. Right now many of these courses must feel like skyhooks but once
they're embedded in a web of knowledge (an earlier commenter called these
'nodes') they'll be more accessible i think.

There's still a huge problem of collecting and sequencing these nodes and I
think many students will need direct contact with instructors of some sort to
do this.

~~~
andrewem
I agree with your points, and thanks for putting them so clearly. The
traditional solution for this is to have each university course specify its
prerequisite courses, and for students to talk to the professor before the
class starts if they're not sure it's appropriate. The latter happened to my
friends and I much more during freshman year, when we weren't sure how our
high school classes would prepare us, though it didn't always work perfectly -
I knew lots of people who re-took calculus or a low-level foreign language
class after having had it in high school, and found it way too easy.

Maybe in 5 or 10 years there will be enough online courses available that it
would be clear that the Duke physiology class is harder, and some other class
would be a good prerequisite to it. Or maybe there will be smaller modules
that would fill in gaps for students who were close to ready for the class, or
maybe it'll all be in smaller modules that you'll assemble into a coherent
whole based on your current knowledge.

~~~
eah13
An additional possibility would be measuring the background knowledge students
have and are comfortable with so it can be better matched up with each course.

The thing is, all of these speculations bear empirical confirmation. We have
to see what works best in the wild and there will be a coevolution of student
behavior, teaching strategies, and companies in the space.

------
denzil_correa
One of the main problems the author has with Coursera or EdX is that it is
"too fast". To address this problem, I urge the author to try Udacity [0]. You
can set learning at your own pace.

That said, not all material on Coursera is "amazing". Some of the classes have
very high completion rate like _Functional Programming with Scala_ has a 19.2%
completion rate [1]. Similarly, the class taken by Andrew Ng on Machine
Learning is fantastic. However, many of my peers had bad reviews on Daphne
Koller's Probabilistic Graphical Models class. Last year, I myself registered
for one of these MOOC's and found that half of the course was good while the
other half was quite bad - both halves had different professors.

At some point of time, universities would have to realize that great
researchers do not make great teachers. Some excel in both - researching &
teaching while some in just one of those two fields.

PS - Other problems on reengagement do stand though.

[0] <https://www.udacity.com/> [1]
<http://www.katyjordan.com/MOOCproject.html>

~~~
DennisP
Koller's class is difficult, but it's a graduate-level CS class with almost
all the same material as the class at Stanford. I thought she taught it well.
It was rough, and I didn't make it through the whole thing the first time, but
I'm back to try again.

~~~
denzil_correa

        Koller's class is difficult, but it's a graduate-level CS class
    

A graduate level class is not a guilt-free pass to make it "difficult". A
course no matter what level, should be engaging and challenging. It need not
be difficult to be an engaging course.

------
michaelfeathers
_We could come up with a bunch of excuses about why online education will
always have worse retention, or we could figure out ways to fix it._

Or we could ask whether it is a problem. I'm not sure that it is.

I don't think the numbers are surprising or unnatural at all. The thing to
compare them against is not traditional education but rather the patterns of
all people who take the autodidactic route.. every person who has ever picked
up a book and not finished it. Even then, the comparison isn't perfect.

In fact, we could argue that low retention numbers are actually a good thing
for non-credit courses. It means that the content is challenging and focused,
and that people are able to discover early whether it is or isn't for them or
cherry-pick the content that they need and move on.

If there are usability problems, fix them. But if participant retention got
dramatically higher, I'd worry.

------
nolite
really great points in this article. I hope that Coursera sees this and really
takes it seriously. I love what they're trying to do, and have signed up for
about 30 classes myself that I totally intended to study, and just couldn't
finish a single one. And its true, artificially imposing the traditional
college course model just does NOT make sense in the online context. Why are
there deadline for problem sets, and test schedules? Fix that, and it'll
change everything

------
fcorr
I'm taking two Coursera courses at the moment - Calculus One and Pattern-
Oriented Software Architectures for Concurrent and Networked Software. I'm
very pleased with the material and presentation in both, but I don't think I
could handle more than two such (relatively difficult) courses at any one
time.

Codecademy and Udacity allow you to freely dip in and out of lectures and
assessments, whereas Coursera courses demand that you deliver
quizzes/assignments every week. It's hard to recommend one approach over the
other -- with weekly deadlines I find I have more of an incentive to engage in
the lectures and course materials, but the added failure conditions can push
students to abandon the course.

------
johnwatson11218
One of the things to that I found confusing with the coursera classes was that
each one seems to have its own set of rules for dealing with homework
assignments, late homework, and partial credit. At one point I was signed up
for two classes at the same time and I assumed they had the same policy for
when homework was due. It wasn't a big deal but there were several times I
would have a free hour or so and it was a task to figure out the best way to
use that time. I think it is a positive thing that the courses are trying out
a lot of different approaches to see what works best but this reminded me of
the parts of academia that I didn't really care for.

------
steveinflow
I don't think low retention is a secret and I don't think it's a bad thing- it
shows that there's no barrier to showing up but there is a barrier to
completion. That's what we want- equal opportunity, without giving completion
away unearned.

For my part I sign up for more classes than I could possibly complete.
Sometimes I just want to take a peak at the material to see if it grabs me.
And sometimes I just want to get familiar with a subject without going all-in,
picking and choosing the lectures that are most relevant to me.

People shouldn't feel like they're hurting Coursera's/EdX's/Udacity's stats by
dipping their toes.

------
jurassic
There will continue to be low retention until there is an incentive to finish.
Without the credentialing piece, I don't see a problem with low retention. I
think it's awesome that people dip in and out as their needs and interests
change. In every class I ever took at MIT/Caltech there were at least a couple
lectures I would have been happy to skip. MOOCs enable that.

I hope there will be more short and tightly focused online courses in the
future instead of monolithic traditional-length courses we have today. I think
that'd go a long way to improving the apparent retention rate.

------
graycat
I believe on-line learning can work.

Why? Three reasons:

First, a 'dirty little secret' of the US software industry is how much of the
learning from the beginning of electronic computing to the present has been
just from individuals teaching themselves from books, e.g., K&R on C, Lippman
on C++, Ullman on database, Sedgewick on algorithms, and on-line, e.g.,
Microsoft's MSDN site, StackOverflow, etc., essentially independently without
courses, lectures, problem sessions, credits, homework, tests, etc.

Or since just K&R, ..., StackOverflow, etc. have been responsible for so much
learning so far, then 'the bar is low' and on-line courses should do even
better.

Second, in my experience in technical subjects, pure and applied math,
mathematical physics, some topics in electronic engineering, e.g., surrounding
the fast Fourier transform (FFT), and software, with a class or not, nearly
all the learning (in my case) took place from study, alone, in a quiet room,
from good materials just on paper. My 'educated guess' is that on-line
learning can't replace such learning but can help stimulate more of it.

Third, for a researcher in applied math and software, and also likely some
other fields, one of the main 'work items' is to take recent books and papers
and work through them much as working through advanced course materials in
such subjects.

Back to my case, in the fields I worked hardest on and did the best in, math,
physics, and software, starting in the ninth grade, through my Ph.D., and in
my career to the present, I did nearly all the work with relatively little
contribution from teachers. E.g., in plane geometry my teacher was the most
offensive person I've ever known in education, and I slept in her class and
refused to admit doing her assigned homework. Instead, I worked every non-
trivial problem in the book including the more advanced supplementary problems
in the back where she never made any assignments. Then, after working all
those problems, no wonder, on the state test in the subject, I did fine: I
came in second best in the class; the guy who beat me also beat me by a few
points on the Math SAT -- we were 1, 2 in the school. Net, my approach to
learning plane geometry worked fine.

I never took freshman calculus. The college I started at wanted me to take
some math that really was just a review of what I'd covered in four years of
math in high school. So, a girl in the class let me know when the tests were,
and I showed for those. The teacher said I was the best math student he'd ever
had. Meanwhile, I got a good calculus book and started in, worked hard, and
did well. For my sophomore year I went to a much better college and started on
their sophomore calculus and did fine. Yup, never took freshman calculus.

When I went to graduate school, I took a problem with me and had an intuitive
solution. My first year had some good courses (one was just terrific, from a
star student of E. Cinlar now long at Princeton) and gave me what I needed to
turn my intuitive solution into a solid math solution; I did that in my first
summer, independently; and that was the research for my Ph.D.

I continued that way in my career: E.g., in a software house working for the
US DoD, I saw a problem in a specification, got Blackman and Tukey, 'The
Measurement of Power Spectra', and read it carefully enough to see what was
wrong with the specification and how to fix it. Right: without courses,
lectures, problem sessions, ....

As far as I can tell, nearly all the technical content on HN, StackOverflow
and other Internet fora is from people who taught themselves in similar ways
with little or nothing in courses, lectures, problem sessions, .... And that's
part of what researchers have to do and is just part of getting tenure as a
research professor.

So, since so much work is being done by essentially independent study now,
just by not making things worse on-line courses should be able to look
successful.

But I see some problems with the on-line materials I tried:

(1) The video quality just sucked. I couldn't read the board. That meant I
couldn't copy what was on the board and study it. Bummer.

(2) The sound quality was not good enough.

(3) The course materials, e.g., on paper or in PDF files, were from not good
enough down to just missing.

(4) Sadly the quality of the course content was too low; apparently the main
reason was the desire to make the course more 'appropriate', that is,
'easier', for 'the common man in the street'. But, omitting material 'waters
down' the course content and, really, for a good student, requires that they
fill in the gaps for themselves -- bummer.

E.g., I looked at the course by Stanford professor Ng on 'Machine Learning'.
What I saw were weaknesses (1)-(4) above. For more, (A) a lot that he was
doing was maximum likelihood estimation but with far too little explanation
and justification; so, I would have had to have run off and studied maximum
likelihood estimation on my own. So, again I was on my own to do some
independent work, trusting Professor Ng that somehow maximum likelihood
estimation was better than it has long seemed in the statistics community. (B)
He mentioned the 'maximization' to be done via following gradients, and that
is an overly simplistic and not very promising approach to maximization -- the
standard, first problem is that spend nearly all the computing time moving in
directions nearly orthogonal to the direction really should be moving in.

So, from (A) and (B), I concluded that for a good course I would have had to
have taken his lectures just as topics to be investigated, gone to good
materials elsewhere, collected good details, and written my own text. That's
his job as a professor, not my job as a student. His field, 'machine
learning', didn't look worth that much work for me now. I've done some serious
work in several cases of applied math that could be called 'machine learning'
as much as his material, and I'm left without much respect for his material.

I looked at the course 'Probabilistic Graphical Models' taught by Daphne
Koller. Since I very much liked a course by a star student of E. Cinlar, maybe
I should like Kollar's course. Sorry, I didn't -- the quality looked too low.
Better quality from Stanford? Sure, K. Chung, H. Royden, D. Luenberger (his
'Optimization by Vector Space Methods' is a beautifully, even elegantly, done
one mile long applied math dessert buffet), D. Knuth.

For courses in 'how to code', that is, introductory material in software,
gotta be kidding! 'Coding' alone is easy; it's just, pick a language, learn
the basic syntax, and write if-the-else, do-while, call-return, allocate-free,
etc. It's easy but doesn't take one very far. So, don't get very far with
thousands of Web pages of documentation at MSDN on .NET, ASP.NET, ADO.NET,
administration of SQL Server, IIS, Windows Server, etc. or the equivalent in
the Linux world.

I have two broad conclusions:

First, traditionally in academic material in technical subjects, the author
and the student 'reached' to each other, and they connected at a well written
textbook. So, the author went far enough toward the student to prepare a good
text -- and the best texts are terrific. And the student reached far enough
toward the author to make do with little or no more than a good text. It's how
I learned plane geometry, freshman calculus, theoretical, applied, and
numerical linear algebra, everything I learned about statistics, most of what
I learned about advanced calculus, signal processing and the FFT, stochastic
optimal control, artificial intelligence, ..., and everything I learned about
software. E.g., it's heavily what worked for me.

The on-line community will have to face the fact of this 'reaching'. In
effect, on-line learning requires the professor to do more work of a kind that
promises not to be well rewarded by tenure and promotion committees. So, net,
so far a student should still reach mostly for one of the best texts, on paper
or PDF.

Second, the situation will 'settle out': The professors will come to
understand what minimum quality is needed, and the students will realize that
the learning is not just a spectator sport, is not like watching a movie, and
still requires nearly all the traditional work from a book or PDF file. Then
the courses will get better; the students just looking to watch a movie won't
sign up; and the course completion rates will increase.

~~~
pkrein
I really like your idea of "reaching" here. I think that in the MOOC model,
the role of the MOOC is to help the professor extend their reach towards the
students. I think the MOOC should bring the content marketing & designe
experts to the table with the professor and help put together the front-end of
the class.

Part of that means helping the professor craft every email to the class to
maximize engagement. In fact, the professor probably shouldn't even be doing
_any_ of that administrative stuff. The MOOC should handle all of that,
including the skills required to "extend the reach towards the students" as
you aptly put it.

I think Coursera and EdX can do _much_ better at using their design and
content marketing talent to make these classes more accessible and successful.

~~~
eah13
> I think Coursera and EdX can do much better at using their design and
> content marketing talent to make these classes more accessible and
> successful.

There's an interesting balance here. Professors I've talked to bemoan the lack
of 'support' from Coursera (usually the universities provide a pot of money
for profs to use themselves). But at the same time they would balk at some
marketing guru coming in an 'spinning' their course. Profs are used to large
amounts of freedom in their courses and aren't used to incorporating feedback
from the outside into their plans.

It's a tough line to walk I think: freedom vs accessibility.

~~~
pkrein
oh i don't mean a marketing person who "spins" the course. but rather somebody
who understands what messages to deliver to which students when. for example,
if I haven't done any of the assignments in 3 weeks, telling me about admin
for the upcoming exam is worthless. a designer and a content specialist
understand this deeply and can help make the professor's communications more
effective. "spinning" them would be worse...

~~~
eah13
Oh, definitely. I think a TA role like this that would help customize
delivery, answer questions, and motivate would be great. I'm not sure its
happening.

But regardless, I think many profs would feel like marketing help of any kind
would be more like spinning content to make it more attractive to students,
and worry about the integrity of what they're teaching.

------
tsiki
I agree with the sentiment, but I wonder if having a low barrier of entry is
really the solution, or does having a low barrier of entry average out to a
low level of initial commitment. It'd be interesting to see if an online
course for which you pay for would be better at retention. From a business
point of view, they could even offer to give you the money back if you finish
the course, or use some other retention trick, like have people calling you to
create some social pressure if you seem to be falling behind.

~~~
Paul_D_Santana
Hah, I was about to post the same idea, about paying for a course and getting
the money back on completion. Maybe even redistribute the funds from prior
students who hadn't finished the class to students who complete the class and
go back to help others. Get paid for answering questions, kind of like a
university TA.

Another idea I had was that students who completed a course, and achieved a
certain level of proficiency, would get access to a special forum. This forum
could be shared by potential employers.

------
tokenadult
I've bookmarked this interesting analysis to add the link to my personal
website. I especially like the comparisons among differing brands of online
courses.

Before I read the fine article (which I recommend you read too), I was going
to react just to the headline. And that reaction is still there. The reason we
(correctly) criticize lack of retention in online education is that online
education makes apparent metrics that show the lack of retention. But I would
suggest, for learned discussion by all the participants here, that most forms
of classroom learning, and especially compulsory secondary schooling in most
of the developed world, suffers from a very bad lack of engagement on the part
of the students. (Some of the comments posted here before this comment tell
stories about that.) A high school student may attend classes often enough to
avoid being expelled and gain a diploma, but the student may operate in
"regurgitate and forget" mode the whole time he is in high school, not really
remembering anything he supposedly was taught. The one kind of deep learning
many high school students engage in is learning how to pretend to be studying.
If online education develops better demonstrations of learner acquisition of
knowledge and skill than mere seat time in a classroom, it may be a powerful
force for exposing the dirty secret of classroom education.

------
zallarak
The title of this article is very exaggerated and needlessly so. The only
reason you see awful retention numbers is because it's so easy to sign up.
Even if I'm only partially interesting in a class, I'll sign up. I often sign
up for 4 or 5 classes that I'm interested in and will finish 1 or 2 per term,
just so I don't miss the chance to sign up later/have access to course
materials.

If _you_ have issues with the class, that's a personal problem and you simply
need to work on your willpower.

------
hojoff79
I think the main issue is a matter of motiviation for the classes. And I do
not mean people being lazy, but the original impetus behind signing up for
these classes.

For the Coursera classes I did sign up for, I did it because I am interested
in the subject area, know almost nothing currently and wanted to gain some
functional understanding. I would say of the classes I signed up for, and
probably 50% of those I have seen all the lectures. But then I go off and
incorporate into side projects to test out / make sure I understand the
materials. I have never turned in an assignment, (almost) never posted on a
message board and never opened a final exam. Why should I? I am going straight
to the application of the content into my projects, I've already achieved my
goal of understanding and expanding my capabilities.

What reasons would someone have for wanting official completion of the course?
On most courses, you do not gain any officially recognized credit, the
certificate has limited (if any) commercial marketability for someone... so
again, why would I want to go through the hassle of officially completing the
course?

Two takeaways from these thoughts are the following: 1) How many people like
myself have utilized the content of the course but are not seen as retained
due to the fact they do not partake in the academic completion side? 2) What
potential motivators can exist in the future to drive completion of the
academic side of the coures? (college credit is the first one that comes to
mind, but still does not help motivating many users who are above that age).
What other drivers could come into being to incentivize?

------
kvasan
Things work differently for different minds, I find that Courseras model of
commitment works better for me. I tried 2 Udacity classes which are self-
paced, I actually did quite a lot of the modules but somehow I moved on. On
Coursera you have to live with the class and at least try to make all
assigments and such on time. I think one thing that works good for me is the
feeling of having a special oportuinty and "if i dont do this I will be left
behind in the new age". Dont get me wrong I would love to be the kind of
person I thought I was; searching the web, finding places and things most
people would not assume were there and then putting them together. But I need
the feeling of being on a "quest" and building up a illusion of "if I make
this, things will change". Latter rambling is maybe more general, my point is
some people (me) just cant handle the notion of almost force-feed overflow of
accesible information and thinking that everybody doing it, so why bother (if
the retention in the article is true that should be instrumentaly good for my
crazy notion). Sorry for the crazy rant, if only the mind lacked biases. Maybe
I should atleast try those mentioned in the article instead of being
overpowered by strange thoughts.

------
johnminter
The hard truth is that the student gets out of a class what they put into it.
I have finished 3 Coursera classes, two of which gave certificates. All three
were well worth my time, but I worked my tail off to complete the challenging
assignments. I found the support staff (TAs) helpful in the forums. Life does
get in the way some times but all three coures had at least a week to finish
an assignment, typically two and the option of applying late days on quizzes.

The biggest thing I noticed were the clueless forum posts by students who
obviously had not read much of the provided material. Two of the classes had
material similar to ESR's "How to ask questions the smart way" and this was
largely ignored. I think we are seeing the fall out of the distraction of
modern society. Sadly, tl;dr doesn't work with complex technical material.

That said, it was a privilege to interact with motivated fellow students.
Their input in the forums helped eliminate misconceptions.

By the way, the statistics for our Data Analysis class were similar to those
from the OP - from the instructor: "There were approximately 102,000 students
enrolled in the course, about 51,000 watched videos, 20,000 did quizzes, and
5,500 did/graded the data analysis assignments."

------
eah13
tl;dr: What's at stake here is the proper roles of ed tech in education. MOOCs
are media, only one part of the ecosystem that is developing here, and I think
that finding a central role for great educators is key.

Coursera and other MOOCs are media properties that monetize viewership
(thorugh certificates rather than ads). As media properties, michaelochurch
and others are right that the synchronous model isn't really going to work in
the long term.

I think ed tech should be about connecting teachers with the content they need
to teach (and I'd include self-educators amongst 'teachers' here). Students
need synchronous access to teachers, but those teachers don't necessarily have
to have created that content just now. This suggests that online education
might look more like a 1:100 teacher-student ratio rather than 1:100k.
Nothing's more motivating and engaging than a human being.

Personally, my best experiences have come from two sources: self-study from
great materials and guided study from great teachers. This suggests that a
role for ed tech might be connecting great teacher with great content rather
than broadcasting teacher-as-content. Personalization, engagement, motivation,
etc all come for free if we empower great teachers rather than seek to replace
them.

It's misguided to think that some professor from a great school can best
deliver the content to all audiences. Imagine great open content accessible to
every teacher for use in their own classrooms. With an inverted model of
teaching, the lets them push content out to the students for self-study and
then have synchronous classes where they, the teacher, can motivate and
customize the delivery of tough concepts for their particular group of
students.

~~~
silverlake
We are following this model with LearningLine (www.learninglineapp.com). We
offer an optional "class" format where an instructor is available to answer
questions and review homework. With most eLearning, when students hit a topic
they don't understand, they have to figure it out themselves. Some will quit
when they hit this wall. Our instructors are there to help students keep
going. No more web searches for help, just send a question and an instructor
will answer and even get on Skype to explain in person.

~~~
eah13
Interesting. I see you have a lot of courses on offer and are 'powered by
DevelopMentor'. Are you affiliated with them or serving as a portal/add on to
their existing content?

~~~
silverlake
DevelopMentor built LearningLine. LearningLine is a general purpose online
learning system, sort of like Udemy. DevelopMentor produces content for
developer training. We have a bunch of partners developing additional training
material. LearningLine is actually free for others to add their content into
it. Our focus is on getting high completion rates. In fact our instructors are
paid when students complete a task, not when students sign-up.

~~~
eah13
> In fact our instructors are paid when students complete a task, not when
> students sign-up.

Interesting model. I like the thought of trying to align student and
instructor motivations. Getting paid for engagement is a great way to do that.

I wonder, though, if it puts pressure on the difficulty of the tasks? Can
instructors game this system (if they wanted to, as I'm sure most don't) by
the types of tasks they assign, thereby potentially impacting the quality of
your content?

~~~
silverlake
Sorry, that whole bit about paying instructors is just for DevelopMentor's
paid content. We review all our content and instructors to ensure high
quality. We are obsessed with this problem of retention and completion.

Our partners can put whatever they want in our system. So far our partners are
providing free classes to train their customers on new products and features.
If a customer learns all the features in your product, it becomes stickier
(hopefully).

------
mani27
Even if 5% people(say 5000 people) complete the course, whats wrong in that ?
5000 is still a big number. Many Prof's wouldn't have taught that many
students all their life. I have myself completed 7 courses from edX,udacity
and coursera. Each of these have a different approach. Udacity for beginers
while edX and coursera have some advance courses and I found these courses a
great supplement to my course work at my school.

~~~
jonathanjaeger
True, that's a nice number. But you have to keep in mind many of these
startups, while doing something noble, still need to make a big business
(often they're VC-backed). Increasing retention rates is essential for
business and for making online education viable. I'm not sure if 5% retention
makes the model work or if you need >10%.

~~~
queensnake
Also, their mindshare: "We're changing the world, 100K under-served people
signed up!" vs, 5K who finish and most have at least a Master's already anyway
:)

------
walrus
I'm more likely to finish a few chapters of a textbook than a series of
videos:

1\. Videos are usually too long. There isn't a natural break to stop and try
things out. (Udacity does a good job with this one.)

2\. Information density isn't constant. With text, this isn't an issue: I
naturally adjust my reading speed if I hit a sparse or dense area. With video,
I have to either live with it or constantly re-adjust the video speed.

------
zabramow
I'm a big fan of Codeacademy myself. But, as someone who doesn't come from a
technical background, I feel like I'm often completing many of their lessons,
but not internalizing the information or actually learning how to code.

I don't plan on giving up, but I'm curious if others have had that experience
or if I'm the only person who's figured out how to use Codeacademy without
actually learning to code.

------
te
MOOC providers could enhance engagement and retention by providing audio-only
content in addition to or even instead of video-based content. I've got a
couple hours per DAY where I crave quality audio content but am physically
unable to watch videos at that time. In contrast, I've only got a couple hours
per WEEK that I can watch videos.

~~~
eah13
This is an interesting approach. I think if they paid more attention to where
and when exactly people would engage with content they could increase
retention. I also have more time for audio than video and would love lectures
of some sort delivered to my mobile. This could create a sort of immersion
experience when combined with video and exercises.

------
lessnonymous
I find most of the comments on this post fascinating. Probably partly due to
the audience, and partly due to the slant the author took on the topic.

His bait is retention .. and that's something HN readers understand from a
business POV. But retention isn't his issue. Retention is a problem for
Coursera et al.

His problem (and mine) is that we (personally) found it hard to stay the
course. And Coursera don't help us.

I've a full time job, two side projects, a small business and a family. I'm
about to start the Gamification course and I probably wont finish it because I
don't have time. Not because I don't want to, but because the course expects
all students to spend 4-6 hours per week studying. Those weeks would be rare
for me.

How hard would it be to offer a 2 hour-per-week version? Or whatever
commitment I'm able to make? Then, when the inevitable happens, let me hit the
'pause' button.

------
mark_l_watson
That was a well thought out article, a good read, and I don't agree with most
of it. I think Cousera is Amazing.

The only thing I don't particularly like is the inflexibility on turning in
assignments. I completed five classes but did not pass one because something
happened work wise that caused me to turn it several assignments late and that
really knocked down my score. Great class though.

I have also started four other classes that I dropped after 1-3 weeks. I am
still glad I was exposed to some of the material for those classes, so still a
positive experience.

Accepting the figure that 95% of students drop classes on average, I would
like to know if most of those people in general felt their time had been
worthwhile spent. That is the important question.

------
jimbobimbo
I did one Coursera course, where I'd finish Q&A tests. But the course I'm
taking now requires me to do some video/audio assignment, share it with peers
and provide peer critique. Sorry, but I have a life, so I'm there just for the
content.

------
auctiontheory
Coursera courses are supposed to be college courses. College courses are a lot
of work, and you have to get your reading and assignments done on time. This
should not come as a surprise.

CodeAcademy (also wonderful) is a completely different animal. It is not
comparable to Coursera.

What you're calling a bug (aka "dirty secret") of Coursera is actually a huge
feature: that we can all, whatever our goals or our budget, sign up and check
it out. Quite unlike an class at MIT or Duke, which is only available to a
pre-selected few.

In short, the MOOCs, and specifically Coursera, are awesome. We are so lucky
to have this opportunity. Quit yer bellyachin'.

------
vdp
A relevant graph: <http://www.katyjordan.com/MOOCproject.html>

IMO for a class with tens of thousands people signed up, 5% retention rate is
not really that bad.

------
tansey
I saw a talk from Andrew Ng a couple of weeks ago where he was asked about the
low retention rate. He countered that 40% of people who submit the first
homework actually complete the class. I would say that is pretty great
retention rate.

Personally, I've signed up for dozens of Coursera courses but I've only
completed one (Game Theory). It's not necessarily that the other courses were
lacking the ability to keep me enrolled, but rather I never planned to try at
them. I sign up to most classes to see the videos and get surface-level
understanding of a subject.

------
timedoctor
I think a major problem with retention is that the course is not a recognised
qualification.

The next step for online education is get a real qualification that improves
your job prospects through places like Coursera.

Also I think there are a bunch of other hacks such as collaborative motivation
(getting a group of members to commit to each other to complete the course and
automatically reporting to each other when they have completed each section
where everyone in the group has collective responsibility to help each person
in the group to complete the course.

------
sefk
I don't necessarily think this is a bad thing.

For Stanford's database class that just finished we had 64 thousand register,
but only 20,000 actually do some work, and then close to 5k actually take the
final or get a statement of accomplishment.

My blog response: <http://sef.kloninger.com/2013/03/online-ed-retention/>

I'm the engineering manager on Class2Go, Stanford's open-source MOOC platform.
Check it out: <http://class2go.stanford.edu/>

~~~
vixen99
I was one of those students (great course!) and you're right. So, folk sign up
on the spur of the moment and then drop out! Big deal. Where’s the dirty
secret? Was there any cost to signing up on the part of the student or the
instructors? Not at all. Why then probe further into what would be a vast
range of personal reasons for not completing the course?

We have to ask ‘Why do people make cost-free momentary decisions and then fail
to act on them?’. Do we think this an important question? I suggest most of us
have much better things to do with our time than agonize over this.

------
jiggy2011
Does it matter if not everyone watches right until the end and completes the
final coursework?

I want to know as much as I can about different subjects but I want to immerse
myself in them to different levels.

Sometimes I am happy just to stick a few videos on in the background while I
do something else, so I get the gist of what something is about and know what
to focus on later if I need more information.

I don't necessarily care if I could pass an exam or not.

------
hkon
An anecdote, I followed this course this year,
<https://www.coursera.org/course/growtogreatness> and the number of student
increased every week.

The guy presenting, Ed Hess, was very skilled and interesting and the material
was very relevant. I also enjoyed the discussion of various homework on the
course forum.

------
futhey
Since the barrier to entry for enrollment is zero, compared to traditional
education, directly comparing them is rather unfair.

------
riggins
I'm curious about enrollment trends.

Having taken online courses from the start my sense is that
enrollment/participation is slowing. The bulletin boards of some classes are
pretty dead on the first run.

Its not crazy to think that the people most likely to take the classes would
participate right away and finding the next batch of students will be harder.

------
demian
"Awful" is kind of relative.

We are talking about a new model, low "retention" seems to be normal for free
clases wich have absolutely no barrier of entry.

I have taken 5 courses, only finished 2 "on time". For the other 3 I
downloaded the material, got some exercises on sites like MIT OCW, and
finished them at my own pace.

------
sopooneo
At a fitness center, personal trainers serve the role not just of instructing
their clients, but of renewing motivation. That task is absolutely dependent
of physical proximity. I feel like this dynamic is being completely ignored in
most discussions of online education.

------
gfodor
Learning is hard. Self-study is very hard. Incentives matter.

The main incentive to power through a Coursera course is personal enrichment.
This is not the reason most people go to college.

------
donniezazen
Online education has no similar incentive as formal education has. And one of
the most important aspect of online education for me is to learn what I find
interesting.

------
jcmontalbano
This article exposed me to a novel usage of "churn".

<http://en.wikipedia.org/wiki/Churn_rate>

------
laughfactory
Just to throw my hat (opinion) into the ring...

I've tried pretty much all of the "MOOC's" out there, and so far the only two
I've stuck with were Codecademy and Udacity. Why? Precisely for the reasons
that are mentioned at the article which started this discussion. The format of
Edx and Coursera is just all wrong for most people who DO want to learn
something but have a busy life that is not conducive to a traditional college
class.

Besides which, all too often, Coursera and Edx just throw a college class
online (lectures, course notes, homeworks, quizzes, and tests) and call it a
day. Then they have arbitrary deadlines--which I acknowledge might help some
people but the first time I got a deadline email I decided that it wasn't for
me...no matter how interested I was in the class topic. I'm already married
and a father who is working part-time while doing a Master's program in
Computer Science. So I don't have huge chunks of time available for
traditional course work (in addition to my Master's coursework). I have little
bits of time every day which I use effectively to progress through the
Codecademy and Udacity coursework. Slowly but steadily I'm getting through
both and making substantial progress.

But, since Coursera and Edx require huge blocks of time committed every week,
and a tight timeline to complete the course, I just don't find it's workable.
Which is sad since they seem to offer some tremendous material.

In short, I think that Coursera and Edx (and probably colleges in general)
should learn a thing or two from the highly modularized approach of Udacity
and Codecademy.

I find that my learning retention and interest remains much higher with both
Udacity and Codecademy because they present a little material (little bite-
size chunks) which slowly build into more and more challenging material and
intersperse "challenges" or "quizzes" throughout--rather than the traditional
approach which--by necessity--was far less responsive.

The fact is that online learning doesn't just reach potentially numerous
people that wouldn't have otherwise been exposed to such educational
opportunities at such a low price (currently free), it potentially allows for
a radical rethinking of the traditional education model.

I think that it is sheer ignorance to say, as some have in the comments I've
read (and as--apparently--Coursera maintains), that those who don't finish a
course just aren't interested enough or motivated enough. This is ridiculous.
The very same (allegedly "unmotivated" or "undisciplined" or "uninterested")
people may be successfully working their way through Codecademy or Udacity.
And yes, it may take them a while, but eventually they'll complete a
Codecademy "track" or a Udacity course. And they'll do it one bite-size chunk
at a time...and sometimes, say on a weekend, they'll plow through a whole
bunch of bite-sized chunks.

When Coursera and Edx adopt a more bite-sized format which isn't oriented
around arbitrary deadlines, I'll check them out again. Until then I'll stick
with Codeacademy and Udacity.

Bottom-line: the standard model is broken. Lectures and homeworks and quizzes
and tests are poor ways to educate. For many years they were the best we could
do within the technological constraints we faced. But now we can do better.
Heck, I'd encourage everyone working the bleeding edge of online education to
think "outside the box" and try new approaches. While the best new approaches
I've seen have been those utilized by Udacity and Codecademy, that's not to
say that there aren't even better methods--methods better at retaining
learners (keeping learners engaged), AND methods which excel at increasing
retention of material.

------
newobj
Seems a bit early to optimize for retention. Or to have dirty secrets. Or real
suffer almost any kind of analysis at all.

------
mikecane
I've not tried these. Do they all require you to sit at a desktop PC or are
they tablet-friendly or tablet-customized?

------
Turing_Machine
Another article that equates "MOOC" with "online education".

Sigh.

------
tsumnia
Retention rates are low in everything, not just online classes. Ultimately,
its how the student reacts to the plateau. Anyone in any area of teaching, I'd
highly recommend reading "Mastery" by George Leonard. In the book, he talks
about 3 of learning types that can stifle progress: the Dabbler, the
Obsessive, and the Hacker (not to be confused with the IT world's rough
definition).

* The Dabbler will pick up tons of things, "overjoyed" about the newness that they are learning. Eventually, they hit their first plateau and the 'newness' becomes boring and they'll seek out another activity to experience the same 'newness' high. Maybe they'll come back to it later and see if they can progress again.

* The Obsessive will seek out perfection on this new activity, pouring over every YouTube video or article to figure out how to be better. When they plateau, they'll try harder; hoping for the same results. If the return isn't there, the Obsessive will look for another activity.

* The Hacker will actually ride out the plateau potentially indefinitely. In some cases, this will create progress eventually, but in others, it can become the cap of their learning capability (hence, 'a hack'). They just don't put the extra effort needed to progress further.

While all these types are negative, two-thirds drop the activity for something
new, something they might be a natural at. From a business perspective, a
monthly subscription favors Hacker-type customers; where a 'per course'
pricing scheme favors Dabblers.

Steering back to online courses, I think the issue isn't 100% the site's
fault; but that certain students will join for the excitement, but fan out
eventually (Leonard goes on later to discuss how to better approach each of
these types as the Teacher). I've signed up to all of the sites in the
article, along with Udacity (where I'm currently taking the Web Dev course).
For Udacity and edX, I've completed the courses I've signed up for; however at
Duolingo, I stopped after the Basics 1 in Spanish. I want to learn Spanish,
but the drive and time isn't there for me.

Jumping back to 'Mastery', Leonard talks about 5 keys that can help promote
mastery and can be used to retain students: Instruction, Practice, Surrender,
Intentionality, and The Edge.

* Instruction is obvious: have great teachers(not just through credentials, but are generally good AT teaching), something these sites offer tenfold.

* Practice is where I felt edX's course fell through. It was a lot of lecture, but the Finger Exercises were more the tradition 'regurgitation' of knowledge from traditional classes. Duolingo and Udacity (at least the Web course) seem to have this concept taken care of. Duolingo offers different ways to practice and at least with Web Development, you can to see the fruits of your labor with each class.

* Surrender is more for the student than the instructor, but talks about accepting you aren't a Master as something and so sometimes you need to take two steps back to take one step forward (in Aikido, we refer to this a 'Kihon Waza', or practicing the basics). So many people have the mindset of "If you're not doing it this way, you're doing it the wrong way", that this can stifle learning. As an instructor, thinking up new and creative ways to practice the basics can help alleviate this.

* Intentionality talks about visualizing the action, and while focused more on physical activities, could be utilized online. Give the students a view at the end results/expected return on their time. Sure, its dangling the proverbial 'carrot on a stick' in front of them, but it can help them sees where the fruits of their labor. If you can mix this with some of the Practice key, you could get better results (IMO).

* The Edge tip-toes the line between endless practice and pushing the envelope (though Leonard says it isn't). For a new student, this can be a little harder, considering they are still learning; but creation of 'outside the box' milestone/projects could benefit from this. The student's amassed this bit of knowledge; make them use it in a less than cookie cutter way. This would be a great undertaking by online courses, due to how do you 'personally' grade someone's submission, especially overtime? Maybe setting it up in a 'there is no right/wrong answer' way, namely because if a student takes a risk, they shouldn't be penalized for it.

Like I said, I'd recommend the book. I teach community college currently, and
while most this is anecdotal opinion, I feel like it's helped me maintain
engaged students.

------
tkahn6
I usually sign up for a bunch of classes just so I can access the material at
a later point when I have time or when I'm interested. I'm pretty sure that in
most cases Coursera locks you out of old material unless you were enrolled. I
don't follow along with the class.

Synchronous learning, IMO, is an anachronism. We have video recording
technology, practically infinite bandwidth, and near speed of light
communication. The idea of synchronous learning, where you have five thousand
teachers teaching the same Calculus I class every year, is predicated on a
world where these technologies don't exist.

As an aside, for this reason I think the collegiate model is going to
completely change in the near future as people start realizing how absurd it
is. The real value professors provide is having someone to answer questions
(and in a lot of cases, professors don't even take questions during lecture).
And even that functionality can and is (in my experience) mostly replaced by
other students in the class or students who have already taken the class.

Put the material up, and let communities form around it. I don't need the
professor to answer my questions, all I need is the relevant forum (in the
original sense) to ask.

~~~
michaelt

      I usually sign up for a bunch of classes just so I can 
      access the material at a later point when I have time or 
      when I'm interested.
    

How many of those classes have you done the work for?

Personally I find the synchronous model helps me complete the course, as it
removes the temptation to put things off.

~~~
scott_s
I think the synchronous model makes it easier to have small communities of
students working through the same thing at the same time, helping each other.

~~~
notimetorelax
On top of that it makes work of TA easier. Everyone works more or less on the
same problem, be it programming assignment, quiz, etc. If there's a new
question TA most probably remember what was in the last lectures and what are
the common issues students are having this week.

------
michaelochurch
The issue with Coursera is the synchronous model. The material's good and the
selection's excellent, and they deserve a lot of props for solving such a
critical problem. College life is sanitized. Real World interjections are rare
enough, at that age, to be handled case-by-case when it comes to extensions
and such. Adult life is messy and complicated-- sick parents become more
common, job demands fluctuate-- and synchronous education is just brittle. I
didn't find Coursera courses to be "too fast". The paces were fine, so long as
I didn't have fires in the rest of life.

It's not a "dirty secret", though. One should know and expect that. If
anything, low retention is a good thing insofar as it means that the courses
are demanding and people who aren't dedicated trickle out.

I feel like we have three problems to solve in online education. I'm sure
there are plenty more, but 4 stand out right now:

(1) Hidden node discovery. You're 23. You just learned Python. You want to be
a Data Scientist in 3 years. How do you get there? "Data Scientist job" is one
node, and there are a bunch of prerequisites to those (hidden nodes) and
prerequisites to those. How do you navigate this network? The 23-year-old
programmer doesn't know where those hidden nodes are. In other words, the
Google for Learning and Development.

(2) Forward learning. Recommendations. Things that would be interesting to a
person that she doesn't know she wants to know, because she doesn't know that
it exists. Since there's a lot of investment here (you're not just buying a
book and possibly reading it, but anticipating putting 50+ cognitively intense
hours into a course) it would be nice if the service gave indications as to
_why_ it was making those rec's.

(3) Interactivity and (buzzword warning) "gameification". When you haul out an
800-word machine learning textbook, you often have to go for a long time
(hours) without the "kick". There's a flat array of 50 exercises of which 25
are easy, and 25 are really hard and will take a long time (they might be
worth doing, but they aren't quick) and it's hard to pick which ones to focus
on. The programming exercises in texts often don't get your creative juices
flowing. Not a lot of people can get through long spells without feedback, and
the skill of "making your own feedback" seems to be losing ground in our
distracted culture.

(4) "Social". Online study groups. This is Big and I don't know how it's going
to evolve. How do we keep quality control in place and make sure that our
automated expert discovery mechanisms work?

~~~
btilly
_...in our ADD culture._

As someone who has personal interaction with the disorder (I'm not officially
diagnosed, but probably have it, my son has it, several siblings, nieces and
nephews as well), I hate this phrase. Because people use it to mean the exact
opposite of what the actual disorder is.

People with ADD or ADHD (different names for pretty much the same thing) do
not lack focus. To the contrary, a primary characteristic is extreme focus.
Instead we lack the ability to focus on what we are told to focus on.

Society labels this "distractibility" because the person does not focus on
what those around would want them to focus on (following instructions, getting
dressed, doing homework, etc) but instead goes back to what interests them
(bugs, dinosaurs, math, etc). But in terms of ability to get fascinated by
something, then follow through on complex tasks, people with ADD or ADHD are
significantly more capable than the general public.

~~~
jcheng
That's an interesting and appealing description, but sounds pretty different
than what Wikipedia has to say about ADD.
[http://en.m.wikipedia.org/wiki/Attention_deficit_disorder#se...](http://en.m.wikipedia.org/wiki/Attention_deficit_disorder#section_2)

~~~
btilly
It sounds different, but I am simply giving more detail about what happens
inside of the disorder. Any decent book written in the last decade will
confirm my description.

People with ADHD or ADD don't have executive control of what they will focus
on. Inability to focus on what you're told to focus on is noticed in most
environments as not listening to instructions, inability to stay on assigned
task, poor attention to detail if you do stay on assigned task, etc.
Everything in that link you posted is covered. In most environments your
excellent focus on things you're NOT "supposed" to be doing is not likely to
be noticed. Or if it is noticed, it likely increases frustration over the
inability to pay attention to assigned tasks.

Speaking personally, I was very confused when screening for ADHD was
recommended for my son. He seemed to have excellent focus - at 3 he would
often get involved in a detailed task for half an hour or more, at 5 he would
sit quietly through a full length documentary, then demonstrate how much he
had learned for days afterwards. It was not until my sister explained from her
experience that this was, in fact, characteristic that I became less resistant
to having him screened.

I was also resistant to medicating him because I'd absorbed all of the usual
biases about how overmedicating leads to drug abuse. However after being
pointed to the statistics for ADHD, medication became a no-brainer. Children
with ADHD do not become addicted to their medication, and have the same risks
of drug abuse as the general population. By contrast ADHD people who are not
treated are at massively increased risk for all kinds of drug abuse, including
both prescription drug abuse and cocaine. (Cocaine is an interesting one since
it actually is an effective ADHD medication! Though inconvenient, illegal, and
subject to abuse.)

Therefore popular wisdom is exactly backwards - appropriate medication for
ADHD reduces the odds of drug abuse later in life.

~~~
btilly
Responding to tripleeggg's dead comment.

I have a child with a diagnosis that significantly increases the risk of his
failing to graduate, using drugs, becoming a convicted felon, suffering
accidental death, and being unable to hold down a regular job. There is a safe
medication that addresses all of those risks, with reasonably moderate side
effects.

If some day you have a child in the same situation, you'll be free to try
whatever unproven experimental course of action you want. But I see no point
in taking such risks with my child's future.

