

Khan Academy Statistics videos are not good - ColinWright
http://learnandteachstatistics.wordpress.com/2012/07/30/khan-not-good/

======
alukima
I've seen more criticism of Khan Academy recently than praise. As someone who
wasn't able to afford college and is working their way through MIT open
courseware I find these videos helpful.

Sometimes I just can't wrap my head around one little concept and I research
it until I do. Khan Academy and similar sites have been essential to me
getting through multiple courses. Are these videos as thorough as a course
taught by an experienced professor? Nope. But I don't think that's the point
of them. It's a place where people like me can learn the foundations before
moving on.

Before online courses existed people in my situation had little to nothing.
Sometimes I bought used, out of date text books but when I got stuck on
something that was the end. (This was in early 2000s).

Often I see people attempting to compare sites like KA to college courses.
That's not really a fair comparison as I highly doubt that anyone with the
option of college is choosing to watch KA videos instead. They are for people
who don't have that option.

Because of sites Codecademy, Learn Code the Hard Way, Khan Academy, ect., I
went from working in tech support and living under the poverty line to junior
web developer. I will never claim that I'm as good as someone who attended
professionally taught courses but I now have the ability to continue to learn
and get better.

~~~
anon1385
(This post isn't intended to be personal so I hope I don't cause any offence).

I often see people talking about how great they personally found a MOOC
course, but that self-assessment is largely useless. To put it bluntly: you
don't know what aspects of a topic you don't understand, were not told about,
or whether the information you received was at all accurate, because you lack
the expertise to determine that.

For example, people thought they had learned a lot from the Udacity Statistics
101 course and it received glowing reviews from participants. But
unfortunately they did not learn about some of the most important basic ideas
in statistics, and some of the things they did learn were just wrong:

[http://www.angrymath.com/2012/09/udacity-
statistics-101.html](http://www.angrymath.com/2012/09/udacity-
statistics-101.html)

 _the course is amazingly, shockingly awful. It is poorly structured; it
evidences an almost complete lack of planning for the lectures; it routinely
fails to properly define or use standard terms or notation; it necessitates
occasional massive gaps where “magic” happens; and it results in nonstandard
computations that would not be accepted in normal statistical work._

 _Astoundingly, the Udacity Introduction to Statistics course manages to go
almost its entire length without ever mentioning or making any distinction
between the population and sample in a study. I say I 'm “astounded” because
in my classes (and any one I've surveyed or looked at), this is the key idea
in introductory inferential statistics. It's the very first thing that is
mentioned in my class (or the book), and it's the very last thing on the last
day, too. It's the entire reason why inferential statistics is necessary in
the first place. In fact, the very word “statistics” means measures for one
(sample) and not the other (population) – but you'll never learn that from
this class._

~~~
UK-Al05
"I often see people talking about how great they personally found a MOOC
course, but that self-assessment is largely useless. To put it bluntly: you
don't know what aspects of a topic you don't understand, were not told about,
or whether the information you received was at all accurate, because you lack
the expertise to determine that."

And? This also applies to to normal educational environments, not just MOOCS.
Why do people assume that courses delivered on premises by an average teacher
is going to be better than a MOOC delivered by world class guy. I'd wager that
on average on premises course has more gaps/problems/mistakes than a MOOC.
These on-premise courses will never be exposed to criticism, mistakes never
seen. Moocs are exposed to every educational 'expert' there is, so lots of
criticism pops up. That's a good thing. Normal courses don't have so many
'experts' looking at them.

~~~
anon1385
>Why do people assume that courses delivered on premises by an average teacher
is going to be better than a MOOC delivered by world class guy.

Assume? I just gave you an example of a shit course delivered by a 'world
class guy'. I've done plenty of real life classes in statistics and none of
the teachers/professors were startlingly brilliant communicators but at least
they were able get over the basic principles of the subject. They also didn't
waste my time telling me how great they were.

~~~
UK-Al05
You guys _may_ be delivering good stuff, but there are a lot who are not.
Remember how many community colleges/for-profits/universities/high-schools
there are in the world. The majority of the teachers recruited were probably
averagely talented. These courses will never be exposed to criticism, but
moocs are. That's a good thing.

------
ig1
Someone whose maths teaching videos aren't very popular complains that hers
videos are better than the videos which are popular.

Khan academy isn't popular because he has money from Bill Gates, his videos
were hugely popular on youtube long before that, for the simple reason that
they satisfied user demand.

She needs to ask herself why there's more demand for people who want to learn
about "mean, mode and median" than there is demand for learning about "levels
of measurement and sampling methods". I'm guessing most kids and adults will
never have to design randomized trials and data collection, but they will come
across averages in every day life.

From her description she mentions how she spends a lot of time thinking about
theory of teaching and rewriting and editing to correct mistake and edit out
hesitations, what was noticeably missing was any discussion of applying user
analytics, A/B testing, etc. to better understand how her users were
interacting with her videos, when they were losing attention span, etc. One of
the reason that Khan Academy is successful is they take a very metrics driven
approach to iteration, something that should be very familiar to any startup
founder.

Sure Khan's videos are flawed but people want to watch them. It doesn't matter
how much better pedagogically Dr Nic's videos are if people don't want to
watch them.

~~~
chrischen
Satisfying short-term demand doesn't mean it's always the best option. For
example, it might be in the best interest of a nation or people to plan for
the long term and optimize for future demands.

~~~
ig1
Let's assume for arguments sakes that Dr Nic's videos results in better
outcomes ten years down the line but no-one wants to watch them. What's the
solution ? - you can't force people to watch Dr Nic's videos, and even if you
could the outcome is likely to be significantly worse because you were forcing
them.

~~~
chrischen
Kids are forced to go to school. Are you suggesting they should not be forced?

------
zainny
Is this a joke? So the examples that she gives as to why KA is terrible are:

(1) He used a pie chart where perhaps a bar or column chart would have been
more suitable.

(2) She's "pretty sure" he got the explanation of confidence intervals wrong,
but was "so confused by the end" she can't be sure.

(3) The p-value video was accurate but dull.

And all this plus some other _very_ minor complaints (" _he mispronounces the
adjectival use of “arithmetic”, which is a bit embarrassing_ " \- Seriously??)
is enough to justify a massive tirade that makes accusations Sal is lazy,
doesn't care, is incompetent, etc? I'm certainly more than willing to hear
valid arguments KA is not perfect, but this is just rubbish from someone who
frankly seems bitter nobody cares about her videos.

~~~
ZeroGravitas
Bar charts are _always_ preferable to pie charts. But the video was about
"reading pie charts", so this is perhaps the one occasion when using a pie
chart can actually be justified.

~~~
duskwuff
But if you're going to give instructions on reading a pie chart, the pie chart
that you give as an example should at least make some sense in that format.
Time-series data represented in a pie chart is abnormal.

------
ahsanhilal
It is really hard to substitute good teaching for kids K-12 with a video of
someone drawing things on a board and explaining it. I work with kids all the
time, and I see implementation of Khan Academy in US schools not really
working. The only kids it seems to work for are those are personally motivated
and are driven hard by parents/guardians to learn.

The MOOC model though does a lot to solve the access to education problem. If
there are kids in under-developed countries who are driven to learn, and are
able to access to MOOC videos then the potential impact is huge. I have a
really good friend of mine implementing this strategy in low-income schools in
Pakistan, and the results seem quite mixed. The kids who are driven will
learn, whereas, the kids who need more attention and guidance will need a lot
more teacher attention. Unfortunately, this seems to be the majority of kids.
Teachers in the US also report similar results. Though for society in general
it seems a net positive.

The other part of the problem is to engage kids to learn more. Engaging kids
to learn is a much harder problem to solve, and possibly the most valuable
one. Anyone in the industry who can provide a content network that truly
engages and educates kids will gain the most.

On teaching: different people absorb and understand any educational material
in different ways. The best teachers have strategies built-up, through
experience, to make different kids understand the same concepts. The real
technology breakthrough could be when we can pre-empt the strategy required to
teach a concept to a particular kid, automate the solution and engage the kid
by providing loads of cool ways to play around with the material through
immersive experiences.

~~~
psbp
Khan describes his approach as "flipping the classroom" where kids would watch
Khan's lectures at home or during lecture time at school, and then have a
teacher who implemented activities and clarified any confusion with the
material. I think it's a much preferable approach than one where kids are
herded through concepts and tests.

~~~
bluecalm
And not only kids. I remember from my univ days (comp sci) we have this
algorithm and data structures course. Typical lecture (2 hours) went like
this:

-introduce some completely new (for most of the audience) algorithm or a data structure

-start doing advanced analysis of its complexity in various cases (or in case of data structure costs of various operations)

The problem with that is that when you encounter something new you usually
need some time to get familiar with it, to play around with it for a while to
get intuitive grasp of what that thing is and what properties it has. Without
it it's very difficult to follow any advanced analysis because you probably
don't even fully understand how the thing works for various inputs/doing
various operations. As it was, 75% of the lecture was too difficult to follow
for most of the audience which showed (most people zoned out and just
mindlessly taking notes).

It would be much better if home assignment before the lecture was to read some
theory and maybe try to implement the algorithm/data structure to see what
problems you encounter and then you would come to the lecture with specific
questions to answer and understanding what may constitute a hard part. This
flipping approach makes a lot of sense to me. It's similar to learning by
doing: you first get some very basic understanding and then try to do things
and learn along. Traditional approach on the other hand is similar to reading
several 500p+ books about OOP and Java and not writing a single line of code
along the way.

------
harrytuttle
Agree myself. I think Khan Academy is very overrated as a whole.

If someone did a paper version that doesn't stutter through problems like
Salman does and actually has some pre-planned structure and peer review, it'd
be better.

Oh wait I just described the status quo before...

~~~
sanderjd
Your first paragraph may be right, but I think the rest is wrong.

Paper education materials have a recurring distribution cost, whereas digital
materials have a larger up-front but mostly one-time cost. Investment in
_both_ digital education materials and widespread access to computers is what
is revolutionary, not the specific lessons.

There is always some sort of blurb in an article like this that says something
dismissive like "such a such a video which was proved bad by such and such
people was switched out for a better one after the criticism". That's the
point! One video file on some server somewhere is overwritten and _everybody_
immediately has better materials than they did before. No new workbooks, no
new editions of textbooks, no printing press or shipping involvement
whatsoever, it's just _fixed_.

I'm sure a lot of these educators agree with that premise and would love to
start a competing platform or work for Khan Academy itself on improving its
material. Perhaps the real criticism is that Khan Academy is a major hype
suck, making competitors hard to launch, and that it's apparent success has
made it over-confident and closed to wider collaboration with educators.

~~~
harrytuttle
I'm not comparing paper to interactive media.

I'd be happy with a regularly updated ebook, worksheets, Q&A stuff that I can
do offline i.e. print it and sit in the garden/park for a bit with.

My main problem is that you can't consume video at your own rate, just the
rate the author defines you should be consuming it, which in this case is
"jumpy".

I've had the blessing of a good lecturer or two in the past. Salman is not one
of them nor is anywhere near them.

~~~
ZeroGravitas
I personally dislike video and prefer reading myself, I would however like to
point out that video (compared with someone standing at the front of a room
and lecturing to you) gives you much more control over pacing. You can pause,
rewind, fast-forward and even, on Youtube and HTML5 video (though dependant on
your browser), watch at various speeds faster and slower than real-time. (Khan
apparently switched from tutoring his cousins face-to-face precisely because
they said the preferred him on Youtube, which he'd originally done as a
"second-best" offer when he wasn't available, as it put them back in control.)

Complaints from people who make videos and put them online (like the lady in
the article) get far more respect from me than people who vaguely claim that
you can't replace face-to-face lecturing, and that all face-to-face lecturing
is of a high-quality. Doubly so since, I personally would prefer a well
written article/book to either.

This particular lady's videos reminded me of suffering through bad
powerpoints, that could be much better done as written material. All they seem
to add is someone reading to me, and not in a particularly pleasant tone
(think how much criticism she'd get from someone who does voiceovers
professionally! These amateurs who don't know anything trying to do stuff
themselves, not even realising what a fool they make of themselves!) At least
Khan works through problems in the traditional blackboard manner, which seems
slightly harder to translate to simple ebooks.

------
bayesianhorse
I don't believe watching Khan Academy videos as someone who knows the stuff
well enough to teach is a fair game.

The value and merit of Khan Academy is that it does the job better for a huge
amount of people, than anything before. Self-studying text-book devouring
students aren't really included.

Though, maybe, there will be text versions of the video... Maybe a wiki?

------
stiff
I am hardly a big fan of Khan Academy, but not only is there hardly any
precise or logically sound criticism in this article, the video presented as
exemplary is horrible, just unbelievingly dull, the level of enthusiasm is
about that of a voice synthesizer. Together with the strange aversion of
mathematics for the sake of supposed "pedagogy", I find it rather
unconvincing, to put it mildly...

~~~
nly
Couldn't agree more. The video shown reminds me of the educational videos they
used to show on the BBC in the early hours of the morning in the 90s so
teachers could record them for classes... just mind numbing. I think we've
learnt now that it's ok for education to be engaging.

------
atmosx
I studied chemistry and physiology (anatomy) using Khan's video and I can say
that it's not overrated at all, it's very useful way of explaining and it
really helps you understand _at least basic_ concepts.

Of course to learn _anything_ you have to be __motivated __, either as an
adult or kid. Otherwise you 're better of doing it wrong.

------
halo
I feel Khan Academy and current MOOCs need to be treated like alchemy or
Freud.

It's people creating half-baked initial versions of a potentially
revolutionary idea, even if it may not live up to the hype and never quite
deliver on the promise of turning base metals into gold.

~~~
dnautics
I don't know about freud, but alchemy did eventually turn into chemistry. You
have to start somewhere.

~~~
kybernetikos
I think that's the point he's making: these are early days and what we've got
at the moment is likely to be partly or wholly wrong, but it's necessary and
pointing in a good direction.

~~~
c2prods
I agree. At least we should give them credits for making things change.

------
nickik
I will not comments on learning statistics, since I not a expert.

But what kind of bothers me about these kind of blogpost that the are all
really negative, as if Khan Academy was bad because some videos are bad. Even
from videos that are not perfect about a subject people can still learn.

Im all for blogpost that start a discussion about improving some parts of Khan
Academy, but the tone in this blogpost kind of bothers me.

------
tokenadult
Teaching statistics is harder than most people give it credit for being. One
problem in teaching statistics is that many people teach statistics after not
studying much statistics, as discussed in the article "Advice to Mathematics
Teachers on Evaluating Introductory Statistics Textbooks,"

[http://statland.org/MyPapers/MAAFIXED.PDF](http://statland.org/MyPapers/MAAFIXED.PDF)

which leads to generations of teachers and learners who think they know more
about statistics than they actually do.

Another thing that is hard to teach about statistics is valid inferences from
what kind of data were gathered in the first place, as discussed in the
article "The Introductory Statistics Course: A Ptolemaic Curriculum?"

[http://escholarship.org/uc/item/6hb3k0nz](http://escholarship.org/uc/item/6hb3k0nz)

It is easy to find examples of statistics courses and particular statistics
lessons that are lousy. The harder task, as discussed by the authors I cite
above, is building better statistics curricula for deeper understanding of the
subject.

From the article kindly submitted here:

"I don’t like the Khan Academy videos about statistics. But I can see why some
people do. Some are okay, though some are very bad. I’m rather sorry they
exist though, as they perpetuate the idea of statistics as mathematics."

That's a very legitimate and cogent complaint, one worth taking to heart at
Khan Academy.

I'd like to hear from HN readers who have taken up the article author's
challenge of comparing videos about such topics as confidence intervals, which
is a tricky concept I have to write about a lot for my work.

------
hedgew
Why not: "Here's how Khan Academy Statistics videos could be improved"?

------
bachback
the Khan format doesn't work for everything and the spectrum is probably much
too wide, but the appeal is exactly that it is one guy explaining writing.
that way the students experience the thought process, which is what most
teachers forget about (if they even know and care about the subject matter)

by the way, those videos on the site are terrible and the exact reason why
this kind of "thing" does not and never will work.

------
daemonk
If you are getting your education from only one source of information, then
you are doing it wrong. Khan academy is just one source out of many on the
web. I personally find their videos to be pretty superficial in content, but a
good starting point nevertheless. If you really want to learn something in
depth, read from as many sources as you can until the picture is clear.

------
Ashuu
I don't know why people point fingers at Khan Academy! I mean he is doing
something good, something that no one thought of doing. Quoting from article-
"Sal Khan made little YouTube videos to teach a family member maths. Other
people watched them and found them useful. Bill Gates discovered them and
threw money at them. Now there are heaps of videos, with some back up
exercises, and some people think this is the best thing to happen to maths
(and other) education" I think people are jealous because Salman Khan did
something which other people might also do but they didn't! Even if they did,
money might be the main motive where for Khan Academy, its the free & better
education. Update: I was reading comments and found that lady saying
"Unfortunately they take a lot of effort to make when you do it properly, and
I don’t have the backing of Bill Gates." This is ridiculous.

------
keithpeter
_" To start with the list of topics under the statistics heading showed a
strong mathematics influence. This may reflect the state of the curriculum in
the United States, but in no way reflects current understanding of how
statistics is best understood."_

Anyone producing teaching materials will tend to use the existing school or
university syllabi as a guide. Search YouTube for a mathematical topic and
include 'GCSE' in the search term to see a pretty good mapping of the range of
teaching styles in UK/England and Wales (Scotland has a different system).

I think anyone _here_ thinking of authoring materials for use by school age
students should break the topics down as small as they can (high
'granularity'). That will allow teachers to include them in different
programmes, and to shop around for different approaches.

------
SeanLuke
> I’m rather sorry they exist though, as they perpetuate the idea of
> statistics as mathematics.

Wait, what?

~~~
tokenadult
See the articles linked from my comment elsewhere in this thread.

[https://news.ycombinator.com/item?id=6302719](https://news.ycombinator.com/item?id=6302719)

------
anaphor
"I couldn’t find anything about variation, levels of measurement and sampling
methods"

I'm not sure I understand how these are not all mathematical concepts. Anyone
care to explain?

------
nudetayne
Khan Academy isn't good and never will be. It fills the gap for new college
students who didn't pay attention in class and never learned how to read a
textbook. True classroom education comes from sharing experience, not
regurgitating material and doing practice problems.

~~~
eruditely
Of course it is you who knows where "true classroom education" comes from. I
can think of many actual examples of classroom education that happened without
sharing experience, although maybe not "true classroom education".

~~~
nudetayne
What else is classroom education if you're not sharing experience? You can
study on your own, so putting stuff on the board from a book doesn't count.
You can do practice problems on your own, so going through examples from a
book doesn't count. What else is left? The only thing to be gained from a
classroom is experience that's unique to classmates and the instructor.

~~~
kibibu
> What else is left?

Many people (including me) learn _much_ better from listening to a person talk
than from reading textbooks, even with no avenue to communicate back to that
person.

~~~
nudetayne
You likely don't know how to read textbooks correctly. It's a legitimate
problem in school systems around the world - most kids are taught from an
early age to do exactly what the teacher says and that's how you learn. Thus,
they never learn anything on their own.

~~~
kibibu
The idea that different people learn in different ways isn't one I just came
up with. See the brief survey on Wikipedia here:
[http://en.wikipedia.org/wiki/Learning_styles](http://en.wikipedia.org/wiki/Learning_styles)

I find the idea that my preference for listening over reading textbooks
implies I don't know how to read them is somewhat bemusing.

~~~
nudetayne
I doubt you could come up with anything since you can only learn from other
people, apparently. "Learning styles" is just another way of pronouncing
"lazy".

