
Thrun pivots Udacity toward vocational education - Amadou
http://cis471.blogspot.com/2013/11/thrun-pivots-udacity-toward-vocational.html
======
glesica
The key quote from this article is right here:

"I think Thrun's elite background led him down a garden path. Any San Jose
State professor who had taught an introduction to statistics could have told
him that many (most?) of his students would not have basic arithmetic skills
and would "hate math." They are not Stanford students."

Thrun is an elite academic. He has most likely never even attended a
university that had to teach remedial math courses, let alone taken or taught
one. Thus, by extension, he has likely never had to deal with students who had
anything but excellent preparation for studying the material he was teaching.

I sat through most of Thrun's AI lectures and I felt that he and Norvig were
questionable teachers. They are clearly brilliant men, and their excitement
was contagious. They would be tremendous guest-lecturers in any CS course (the
particle filter lectures were Thrun's best because he was so excited, but he
spent most of his time gushing about how cool the ideas were). But as far as
teaching a course to non-Stanford students without the help of an army of TAs,
they were uninspired.

Compare Thrun's experience with the sentiments in this article about Miami
University in Ohio: [http://www.propublica.org/article/on-country-club-
campuses-a...](http://www.propublica.org/article/on-country-club-campuses-a-
public-university-ex-president-shares-his-second)

The common vein is that education is fairly easy when the people you are
educating are well-prepared and generally hail from the upper half of the
socioeconomic spectrum.

Teaching poor kids, and kids who, for other reasons do not have adequate
preparation is really something of a Sisyphean task. The problem is not in our
schools, and it can't be fixed by fiddling with the curriculum or delivery
method, it can only be truly fixed by fixing the underlying social problems,
but that usually means talking about sticky issues like racism, sexism, ethno-
centrism, and capitalism itself.

~~~
yummyfajitas
I'm sure I'll be downmodded for pointing this out, but your post also suggests
why these disparities persist even after college.

Via a sisyphean effort, colleges manage to graduate a group of unintelligent
and unmotivated students. But after graduation those students are still
unintelligent and unmotivated, they simply had knowledge of calculus forced
into their heads for long enough to pass an exam.

As an employer, I need employees who will show up and start building shit.
Walmart and McDonald's have time for heroic efforts in extracting value from
employees, I don't. This is part of the reason why github is such a great
hiring tool - pushing good code all by yourself is a strong indicator that you
are intelligent and motivated. Graduating college, not such a good indicator.

Interestingly, if colleges didn't engage in heroic efforts to get their
students to graduate, college would be a much better hiring filter.

~~~
mattmanser
I personally find this comment to be incredibly condescending. It smacks of
misplaced elitism.

It comes across as though you are a bad manager who simply doesn't know how to
nurture your new employees and you are trying to blame them.

If you don't want to nurture young people straight out of college, most of
whom have simply been told exactly what to do at each stage, don't hire them.
Hire experienced professionals.

That's why they cost more.

~~~
yummyfajitas
I'm not trying to blame anyone and I have nothing against inexperienced
people. I'm quite happy with the people I've managed in the past, most of whom
have been fairly inexperienced. They were generally better employees than I
was a manager [1].

There is a difference between nurturing and handholding. Nurturing is opening
the door for them, handholding is pushing them through it.

A concrete example. I had a young college kid put in charge of a bunch of
mechanical turks, most of whom were considerably older and more experienced
than her. Her first few days of the job were rough - the older girls thought
they deserved the job and gave her pushback and I was also unintentionally
undermining her. Nurturing her involved asking what she needed, redirecting
all work queries to her, and building a report to let her change the business
process (she switched from time tracking to goal tracking).

Nurturing was simply telling her underlings "ask little boss lady" when they
came to me and then going back to coding. That's exactly what she asked me to
do. Handholding would have involved "hey everyone, I need you all to be nice
to little boss lady and if you aren't I'm going to yell at you."

(Incidentally, if anyone needs a great manager in Mumbai, let me know. I think
she's on the market.)

[1] I failed to protect the people below me from the people above me and I
can't blame that on anyone else.

------
hkmurakami
This isn't surprising, really. MOOCs disproportionately benefit those who
already have the means, the background, and most importantly the drive to
learn.

The greatest challenge of education is to nurture the desire for learning in
students and for students to be in a situation in life (e.g. situation at home
- income, safety, etc) in order to even have a chance at becoming motivated.

MOOCs address the problem of access. But access wasn't the main problem to
begin with for the vast majority of students whom people like Thurn was trying
to reach.

~~~
foobarqux
It's worse than that: MOOCs don't increase employability even to someone who
has the "means, the background, and most importantly the drive to learn."

It's like reading a book: No one else give a shit.

I would add that for most jobs knowing more than the basics from a course, and
often not even that, isn't useful.

~~~
wrongc0ntinent
Books got me a few jobs.

~~~
graeme
Indeed. I got a job offer merely on the basis of having worked through much of
K & R.

------
Eliezer
I fiercely applaud Thrun's update on the uncomfortable evidence he gathered,
and wish him the best in his pivot.

(It still seems to me that MOOC's might be done right with enough effort, like
spending $100K per hour of nationwide-reproduced instruction the same we do as
with TV, but I have not been keeping up with the literature here and perhaps I
don't know how pessimistic I should be.)

~~~
seiji
A 40 minute TV show (scripted, non-reality) easily costs $1-3M to make.

I tried to produce some online learning videos recently. Doing it myself took
10 hours of post-production work to create 1 _minute_ of final content. After
making 10 minutes of content (which took over a month), I had to stop for my
own sanity. [Actually, this post reminded me to finally upload them. They're
currently uploading to
[http://www.youtube.com/user/learndofun](http://www.youtube.com/user/learndofun)
if you're curious.]

For some educational material (especially intro content), you want well
produced, easy to follow material. The higher up you go, you can just throw a
camera in a room and people will be happy with things like
[http://videolectures.net](http://videolectures.net).

~~~
hershel
Aren't khan academy videos pretty cheap to produce, but quite good( except for
some pedagogical errors khan is doing because he isn't a teacher, but that's
easily solvable) ?

~~~
mathgladiator
> except for some pedagogical errors khan is doing

care to explain as I'm confused?

~~~
psbp
Khan's lectures are a bit extemporaneous in an attempt to be more
conversational. Because of this, he often makes errors or explains topics
sloppily.

It's not so egregious that it ruins the content, and it seems to fit his
approach of fun, approachable lectures that will lead the student to explore
the topic further with classmates, instructors, other internet resources.

~~~
TheLegace
Also, I wasn't to fond of using black background and bright colored squigly
writing for learning calculus. I have ADHD and learning math and calculus has
been such a huge challenge. But I found work done by patrickjmt[1] to be
amazing. It's clear neat and concise, and was able to get through Single and
Multivariable thanks to him. My girlfriend and my sister really loved the way
he explained calculus as well.

[http://patrickjmt.com/](http://patrickjmt.com/)

------
AnotherDesigner
Isn't this just them trying to monetize online classes? I've read a few other
articles that covered this topic and it seemed he was pivoting away from
"education should be free" to wanting paid offerings. I know he has, along
with others, complained about the limited number of people completing courses
or passing them. I just feel like that's an excuse. I looked over a number
over courses but didn't complete them because they were boring, lacked
necessary prerequisites or had a poor user experience. I think it's a little
early for giving up and I hope something takes free online education seriously
enough to solve the problems. I'll also mention that not a single person I
know outside of the tech industry has even heard of this stuff. You have to at
least try a little before you quit, right?

~~~
Brakenshire
When you learn for the sake of learning, rather than certification, there's
little point in crossing all the t's and dotting all the i's, in entirely
finishing a course. You take what you need from it, and move on. Lifelong
learning precisely means that you mix education in more closely with the
application of the information you have learnt. When I use these resources, I
jump around from course to course because I am actively engaged in my
education, trying to get exactly what I need to move on to the next step.

If MOOCs want people to follow the material point to point, from start to end,
they will need to provide a level of certification which gives people a
meaningful incentive to do so.

------
g9yuayon
Did anyone take Thrun's basic stats course? Is it the same one that Udacity
offered to the public? I'm bothered by this paragraph: "I think Thrun's elite
background led him down a garden path. Any San Jose State professor who had
taught an introduction to statistics could have told him that many (most?) of
his students would not have basic arithmetic skills and would "hate math."
They are not Stanford students."

San Jose State is a reputable university with many graduates doing well in the
valley. The statement that "many of his students would not have basic
arithmetic skills" sounds shocking.

~~~
vellum
At SJSU, 33% of the incoming freshmen need remedial math and 49% need remedial
English. It's like that in most of the Cal State schools.

[http://www.calstate.edu/pa/clips2007/march/14march/unprep.sh...](http://www.calstate.edu/pa/clips2007/march/14march/unprep.shtml)

~~~
jrs99
at every school in the world, there are kids that need help with math. Even at
MIT there will be some kids that have been memorizing formulas... that's
simply because that's how they've been taught for their entire life. Then they
hit a wall and they're really in trouble.

Thrun can't teach kids that have, for a decade, been learning how to memorize
crap. How many math teachers in high school and middle school can prove the
pythagorean theorem? That should be the survey that we worry about.

Thrun should be making courses for fifth graders also if he wants to teach 20
year olds intro to stat.

~~~
g9yuayon
Then the education system has a serious problem. Kids respond to incentives.
Memorizing formulas would work only if students can get by homework and exams
with rote memorization. We probably should learn from
Russia/India/Korea/China, where it's almost impossible for students to pass
STEM exams with mere memorization. Not that their exams are hard. It's just
that their educators take effort to make sure the problems in their exams
encourage intuitive understanding, original thinking, and creative problem
solving. Ironically, US universities do exactly that in their classrooms. I
wonder why high schools can't just do the same.

~~~
CGudapati
Umm.. India? I am from India and rote memorization of math formulas is very
evident. I can not speak for other countries but you can pass all the exams
till high school using rote using rote memorization and get 100 % too. Only
exams like JEE and to an extent AIEEE actually test your ability to apply
concepts and that's why they have the smallest selectivity ratio.

------
kenster07
I do not see this as a problem with Thrun, but as a problem with the current
educational system. If MOOC's carried a fraction of the "academic credibility"
that college degrees did, Udacity would be a home run already. This will
require a cultural shift, and companies like Udacity are sowing the seeds.

As Warren Buffet has said, the courses he took at Wharton barely differed in
quality from the ones he took at Nebraska. Our culture is so obsessed with
shallow accolades that it is slow to recognize a humongous opportunity right
underneath its very nose.

There is absolutely no reason that MOOC's cannot teach many college-level
courses as good as or better than their real life counterparts. Unfortunately,
the motivation of students largely stems from the aforementioned shallow
accolades.

As a filtering mechanism for professional skills, I think the current academic
system has some flaws which MOOC addresses nicely, perhaps not as a complete
replacement, but a powerful complement. But the reality is, academia should be
used to filter academics, and in many industries, a degree represents nothing
more than 4 years that could have been spent learning the actual profession
and adding value to the economy.

~~~
jimhefferon
> If MOOC's carried a fraction of the "academic credibility" that college
> degrees did, Udacity would be a home run already.

It is unclear to me that any educational system that passes such a small
percentage of students in Elem Stats would ever have a hope of gaining
academic credibility. That's the playing field.

What an instructor at SJSU would do that Thrun is not going to do is lots of
office hours, question-answering, and mentoring/being there. That's what it
seems to require.

Personally I think it took a great deal of courage for Thrun to recognize that
this is a game he is unlikely to win.

~~~
fnordfnordfnord
>It is unclear to me that any educational system that passes such a small
percentage of students in Elem Stats would ever have a hope of gaining
academic credibility.

Do you think that if they inflated their pass-rates that you'd find them
credible?

There is too much focus on pass-rates[1]. I shouldn't have to point out to a
person who implies having taken statistics that it is silly to compare
selective institutions' pass-rates to the pass-rates of open-enrollment
institutions. Add the benefits of "free" and "I don't have to leave my
home/workplace/etc." as is the case with MOOCs, and you have the potential for
greatly expanded enrollment. Why would you think that large enrollment growth
would generate better pass rates?

[1] I'm an instructor at a two-year college / vocational college where some
people seem to never shut up about pass rates. The State also seems to think
that pass-rates == quality, and has set about incentive-izing programs with
"good" pass-rates. Even worse, one local (large) employer insists upon "B" or
above average for anyone it hires.

>Personally I think it took a great deal of courage for Thrun to recognize
that this is a game he is unlikely to win.

Maybe he's not in it for the money, but to make genuine improvement.

~~~
jimhefferon
> I'm an instructor at a two-year college / vocational college where some
> people seem to never shut up about pass rates.

Yes, those people comprise the playing field.

------
Amadou
The article makes it sound like he gave up real quick. That he wrote class
materials at a level above where the actual students were and instead of
adjusting to their level for the next semester, he just threw up his hands in
frustration.

Is there more to the story?

~~~
kkowalczyk
Yes. Read the original article ([http://www.fastcompany.com/3021473/udacity-
sebastian-thrun-u...](http://www.fastcompany.com/3021473/udacity-sebastian-
thrun-uphill-climb)) this is based on. It's much more informative.

~~~
Amadou
Thanks, I saw the link to it and didn't realize it was current, I thought it
was just background on Udacity before this pivot. I probably should have
submitted that link instead.

------
kriro
As an academic I feel no schadenfreude at all. I feel somewhat sad.

This model will work eventually. It's mostly a matter of how the courses are
structured and taught. I don't think it'll replace traditional universities
but it should challenge the price structures and general lazy-bureaucracy
attitude of many of them.

If you look at programming in particular I feel very good about all these
online courses. It's easier than ever to teach yourself programming which is a
net+ for society. I don't understand how more accessible education for
everyone is ever a bad thing.

I don't think it's just education that's broken though. HR is also broken
because they mostly look for the signature on the dotted line (oh look someone
else has done the vetting for us) and tend to value certification over actual
skill. Long term, I hope it'll be enough to list coursera etc. courses, books
you read and a github link on your resume. IT should strive to be a leader in
this shift because it's generally an industry where self-learners can do
extremly well and have traditionally shown it's possible.

~~~
trendoid
>Long term, I hope it'll be enough to list coursera etc. courses, books you
read and a github link on your resume.

It sounds nice but problem with this is that you get so many people who join
courses just to show off certificates and end up learning almost nothing. If
you include books too, there will be popular posts like "books you should read
and have on your CV!". This totally spoils the idea of reading and learning
for its own sake which is what education should be all about.

------
dinkumthinkum
Well, all this hype was so overblown and the anti-university jibber jabber has
been at fever pitch for some time now. It's nice to see practical thinking
rather than breathless "down with universities because Internet"
pronouncements.

~~~
psbp
Because the university approach is working perfectly. No humongous debt or
diminishing returns issues at all.

..The current university system is just as untenable. It's just established.

------
Pitarou
The "secret sauce" of the MOOC is the on-line community of motivated learners.
Can they make this work in industrial training...?

------
ziedaniel1
It's a shame Udacity hasn't worked as well as Thrun hoped. I'm convinced that
the failure of existing online courses has been primarily a problem of
motivation, rather than an inherent pedagogical problem in online coursework.
It's very hard to motivate yourself to consistently work on coursework if
there are no deadlines and dropping the course just means you keep going with
it sometime later. Still, I can't really explain why the SJSU courses failed
as well; it sounds like students had actual deadlines, and were "required" to
complete the courses to the same degree they're required to complete their
regular courses. It's possible that the online tools to ask for help didn't
work as well as in-person interaction, or that sitting in front of a computer
doing things at random times is less motivating than actually attending
lectures in person at regular times. There's one thing I'm certain of,
however: it's a huge waste to have universities pay professors (who are
selected for their research ability, not teaching skills) to repeatedly
lecture the same material to small groups of people. A hybrid model might work
best. Lectures could still be online (but possibly at fixed times to prevent
procrastination), but they would be supplemented by in-person recitations,
where students would meet in small groups with instructors and have the
opportunity to ask questions or review particular topics, with the instructor
as well as with peers. This model is still far more expensive and less
accessible than online courses hoped to be, but it might actually work.

------
bachback
I took the web dev course by the founder of Hipmunk and it was absolutely
great!

This was going from "best thing ever" to "lousy idea" in light speed. That is
the downside of big ideas if they haven't been validated early. Not only
investment of time and resource, but psychological energy. This is really the
story of all change. It it unlikely, extremely hard and often at the wrong
time.

------
_delirium
This is an interesting shift, but feels less promising than the initial
rhetoric. Vocational education in STEM fields delivered by for-profit
institutions is already pretty well-trodden ground. ITT Tech, for example, is
a large incumbent in this sector. Why should someone go with Udacity instead?

~~~
sitkack
Price, access. But mostly price. One could start Udacity while in middle
school and be bad ass before leaving high school. I doubt ITT Tech offers
that. Highly motivated students will excel and be able to test out of classes
they don't need to waste their time with.

------
EGreg
How is this an inverted classroom if the MOOCs don't have a component where
students come in after each class and take a test showing they've mastered the
lesson's material? If they don't pass then they have to get remedial
correction 1 on 1. Just like in a regular class.

That's what motivates them and pushes them to work. After all habits are hard
to form and break. Wanting to learn is not enough, putting in a regular effort
to do so is a habit that needs to be coached. That's one of the main jobs of
educators for under-motivated students (read: most no grad students). And
sorely missing from Udacity.

CONCLUSION: Udacity's "experiment" blatantly omitted a crucial hidden
variable.

------
krisoft
Pity. The only MOOC I have ever finished, and truly enjoyed, was Thrun's
"Artificial Intelligence for Robotics". It was magnificent to get an easy to
follow introduction into a deep academic field by someone who is so
demonstrably accomplished.

I know that I'm just being selfish here, but I was hoping that Udacity will
continue on that path. I would love to learn celestial mechanics from
Belbruno, or neural networks from Hinton, or operation system design from
Linus in the accesible Udacity style. It seems that won't happen any time
soon, or ever.

~~~
cheeko1234
Why is it so difficult to have the motivation to finish MOOC? That's the main
problem with online education, and we need to figure out what is causing so
many people to quit halfway.

~~~
summerdown2
Another thing may be that for many people half-way is good enough. I took the
Udacity cryptography course and really enjoyed it. I never took the exam
because I didn't see the point. Presumably I'd have been marked down as
"failed." But I think that's only because their model for learning still
includes "we're the ones who certify you." I'd like to see a model that says
"here's the knowledge - go as far as you want."

------
riffraff
I would be very curious to see numbers for udacity competitors.

I know I gave up every time I tried to do something on udacity while I
completed most of the classes I took on coursera, and all the original
stanford ones.

I blame this on the interaction between class style and myself being terrible
at self management without deadlines.

I mean, I took the original Thrun AI class with a coursera-like model and I
did it fine, while I don't know if I got to the 3rd lecture of Thrun's Stats
101.

~~~
vellum
There's an interactive chart here:

[http://www.katyjordan.com/MOOCproject.html](http://www.katyjordan.com/MOOCproject.html)

------
walshemj
The problem with this pivot that proper vocational education also requires a
fair amount of mathematics and even more tricky for an online course a lot of
hands on work.

I started on the professional apprentice track back in the day and our college
had two entire blocks filed with machine tools and labs.

------
psbp
I really wish Udacity would clarify their approach going forward. Will their
classes become solely vocational, or will they continue to develop
conventional courses? It's not either/or.

------
EGreg
Thrun should look at the Khan academy.

That said, Udacity can serve a different purpose. And after all, the computer
courses there are vocational...

------
robomartin
Par for the course. Pioneers, as they say, are the ones with arrows on their
backs. The "I told you so" crowd are cowards who would never stick their necks
out to try anything new. That, or they have ulterior motives such as job
security or income protection.

The truth is online education is here to stay and it will get better with
time. It will, eventually, replace big chunks out of traditional education.
Probably not all.

My son is currently going through MIT's 6.00.1x Intro to Computer Science and
Programming on edX. He is 14 and in High School. Watching him progress from
mechanically typing conditional statements to having his mind opened to
computational thinking has been an amazing experience. Yes, this course is
pushing him around and challenging him in big ways. It isn't easy.

From what I've seen, there are only two ways one could succeed with these
kinds of courses: self motivation or external support (or both). In our case
he has all of my support. I am actually taking the class alongside him so I
can see what and how they are teaching in order to help him out.

Motivation is a huge factor. He is a member of the local FRC robotics team and
was involved in FLL before that.

We work on every problem set in a collaborative manner, with me guiding rather
than providing solutions as well as simply being there to expand on topics
that are not covered to a great depth (pointers comes to mind). Lately I've
been doing a lot more watching than guiding as he has definitely begun to
think like a programmer and is solving most every problem without external
help.

For me it's been an interesting review of topics I have not touched in years.
Recursion, for example, is something I haven't touched in quite some time as I
have not run into problems and systems that could justify the resources
required when using these techniques. Playing with recursion in an academic
setting and helping my son learn the concept was lots of fun.

I can absolutely see that a course such as 6.00.1x would be impossible to
complete for a kid without the support of a parent. Not sure if that parent
has to have domain knowledge or not. I can't be a good judge this because I
obviously do and all of our conversations have taken advantage of this.

I can also see the difficulties in entering into some of these courses without
the necessary preparation. Students who went through school by mechanically
doing math without really understanding math tend to not do well on higher
level courses regardless of whether these courses are online or in person.

There's also the case of the working engineer who might need to brush-up on
skills before attempting a class. Using myself as an example, I have not used
statistics in any formal way in a long time. If I wanted to take an online ML
class I'd have to spend an amount of time reviewing statistics and probably a
couple of other areas in Calculus.

In this sense this is where, perhaps, MOOC's do it wrong. Conventional live
courses go through a qualification phase in order to ensure that the "herd"
has reasonably uniform and adequate capabilities. The beauty of MOOC's is that
anyone can jump in. And that's absolutely fantastic. What might be lacking is
a departure from a linear model of teaching. Why can't I enroll in that ML
class and, when and as required, take off in a branch and review statistics to
then come back and "merge" into the main thread of the class. Perhaps this
non-linear approach is what is missing.

All MOOC's are pretty much online versions of some kind of a traditional live
class. Lectures, problems, homework, tests. All presented in a linear
timescale and on a similarly linear schedule. A learning system that is truly
after the acquisition of knowledge must work differently. It must take a
highly interactive approach in which the teaching system is flexible enough
to, effectively, deliver curriculum that is customized to the needs of each
and every person.

This is a challenge. We have to be glad there are people like Thurn who are
willing to stick their necks out, try, fail and try again. The critics are
usually people who will never compromise their station in life to try and
drive progress. They don't want arrows in their backs. Far easier to shoot
them at pioneers, eventually you hit one or two of them and for a brief moment
in time you might actually sound like you know what you are talking about.
Reality, however, is quite different.

What I would tell Thurn is: Don't give up. Don't exit the segment. Try to
figure out how to change the approach and make it work.

