
How well does Khan Academy teach? - ColinWright
http://www.washingtonpost.com/blogs/answer-sheet/post/how-well-does-khan-academy-teach/2012/07/25/gJQA9bWEAX_blog.html
======
ColinWright
I spend a lot of my time in outreach, enhancement, and enrichment activities,
and I've now met several people who have taken to watching lots of the Khan
videos. Some of them are doing spectacularly well, possibly partly because of
them, although correlation versus causation, _etc._

However, I have had some seriously worrying conversations with people who have
some deep, deep misconceptions, and who believe they have been vindicated by
some parts of some of the videos. Again, correlation versus causation, _etc._
, but some of the things these people believe they've learned from the KA, and
believe must be right because they (think) they've seen it on KA really worry
me.

I agree with Sal when he says:

    
    
        We believe that we are in the early days of
        what we are and feedback will only make that
        better.  I agree with you that no organization
        should be upheld as a magic bullet for education
        woes.
    

However, part of that feedback will rightly be negative, and some of the
fanboi-isms I see worry me almost as much as some of the misconceptions I see.

It's a brilliant body of work, certainly ground-breaking, and potentially
revolutionary. But it has its problems, and denying that doesn't help.

~~~
jcromartie
> some of the things these people believe they've learned from the KA, and
> believe must be right because they (think) they've seen it on KA really
> worry me

What kind of things?

Are you suggesting that flesh-and-blood teachers or brick-and-mortar schools
are less likely to spread misinformation or leave students with
misconceptions? I can tell you a hundred false things I was taught by "real"
school, and some of them are probably just things I _think_ I was taught.

~~~
sp332
_Are you suggesting that flesh-and-blood teachers or brick-and-mortar schools
are less likely to spread misinformation or leave students with
misconceptions?_

Possibly, because when you leave school you can realize your misconceptions
and learn from other people, as well as pointing out other people's
misconceptions. But if everyone uses Khan Academy, everyone will end up with
the same misconceptions.

~~~
jjoonathan
Except the self-fixing process you claim to be characteristic of physical
institutional learning only happens when there is a significant number of
people in the class that are A) In a position to think confidently and
critically about the material B) willing to invest time and effort (often lots
of both) in finding the correct conclusion and C) in contact with you about
their insights. I have found the confluence of those factors to happen
exceedingly rarely in physical classes and significantly less rarely in hybrid
physical/online classes. This isn't surprising because lectures place severe
limitations on two-way communication that online environments don't. With a
traditional class structure (lecture/discussion section) I suspect at least
3/4 of questions go unasked and at least 3/4 of the received answers never get
shared beyond the immediate group of individuals doing the asking (in
comparison to an online forum, where the answers are googleable or at the very
least spread to the entire class).

Also, online learning wins in the limiting case where courses have been
iterated and feedback / FAQs taken into account. In a brick and mortar
institution, "institutional memory" is effectively lost as professors retire
or shift interests (if it ever existed in the first place) and their
replacements introduce their own slew of idiosyncrasies, poor methodology, and
misconceptions. In contrast, an online lecture never suffers this degradation.
Sure, new lectures may replace old lectures (presumably at different sites),
but _both will be available for a time_ and during that time, students will
pick up on inconsistencies. That's not true of brick-and-mortar lectures,
where the only source by which a student can verify the material they are
presented is the textbook or its alternatives. Combined with the tendency of
courses to discourage reading the textbook (it's almost always a less point-
efficient use of time to read textbook chapters than to do homework, practice
problems, and go to office hours, which provide information better matched to
the material that will be tested on), the status quo is a recipe for failure.
My experiences confirm that this recipe for failure produces failure (in the
pedagogical sense).

The third factor that favors online learning with regard to misconceptions are
its twin killer features: the pause button and the rewind button. These reduce
self-inflicted misconceptions tenfold.

So, sp322, I think the opposite of what you stated is true. I would welcome a
conflicting opinion, because usually when I go on rants similar to the above
three paragraphs all I get are nodding heads.

------
Caerus
I'm really starting to believe that these "critiques" of Khan Academy are just
teachers and professors terrified of becoming irrelevant.

The question shouldn't be "How well does Khan Academy teach" (well, maybe
someday). The important question right now is "Is Khan Academy a better
learning tool than most high school and college teachers?". In my experience,
the answer overwhelmingly is "yes".

I did have some truly fantastic teachers/professors on the way to a STEM
degree, but they were rare. Most were terrible to mediocre, even at a semi-
prestigious university. Khan Academy has greatly reinforced my understanding
of some subjects, and it's been exceedingly easy to learn some new subjects.

So yes, it doesn't stack up to the top tier teachers and probably never will.
But that doesn't matter, because there aren't all that many fantastic teachers
and Khan Academy is better than the rest.

~~~
crusso
_I'm really starting to believe that these "critiques" of Khan Academy are
just teachers and professors terrified of becoming irrelevant._

I try not to ascribe unstated motives to people, but I really wonder at the
smell of pedantry and elitism in an article like that.

Khan offers a free educational service that is very popular. That is a Good
Thing(tm)... PERIOD.

People are logging in to his site to learn things. Khan has created a whole
model for learning online that has some people enthusiastic about learning
more than they could by sitting in class -- yet the criticism from this
article is that the Khan videos aren't ideal in using the latest buzz-wordy
techniques approved by academia?

Maybe the authors of the articles/papers could interact with Sal Khan and
offer help. Or if the authors really felt strongly about their qualifications
vs Khan's, maybe they could offer free online teaching videos that were more
"correct".

This type of headline sensationalism designed to take down someone doing a
good thing is just sad.

~~~
jshowa
> Khan offers a free educational service that is very popular. That is a Good
> Thing(tm)... PERIOD.

Yes, because when something is popular and free, its got to be good. _cough_
religion anyone _cough_.

If that is considered to be the measure of quality these days, our society is
doomed. Education is an important thing, ignoring it and treating this one man
as some "messiah" of education when he clearly introduces wrong and obtrusive
concepts for the sake of "getting it out to the masses" is not education.

~~~
crusso
Besides learning about his project online, I don't have any use of or interest
in Khan Academy. There's no religion on my part, just a little disgust that a
for-profit company would try to smear KA over minutia.

It would be like if Khan Academy was giving out free meals to homeless people
and McDonalds fabricated reports on how the food was unhealthy and used their
influence to get the Washington Post to carry the smear and add to it.

 _wrong and obtrusive concepts_

Now who is showing a religious level of bias?

I'm a pragmatist. If something is free that I need, I use it. If something is
for-pay that I need and the price is justifiable, I use it. My commercial
software architectures often mix proprietary and open source solutions and I
don't make excuses for either business model.

One thing about free, you can always get a refund on your investment and you
have to really suspect the motives and the character of someone who attacks
"free". Not real surprising that the attacker in this case was a for-profit
competitor, was it?

~~~
jshowa
I don't know if we are talking about the same article, I'm talking about this
one "How well does Khan Academy teach?" by the Michigan University math prof
who outlines, in detail, basic concepts like adding decimals and
multiplication that are taught at Khan Academy in a way that goes against
research.

There is no religious bias, its a difference between a ineffective and wrong
way vs. an effective right way. The professor performed no smear against Khan
Academy, but the fact he only saw two videos that cause confusion in basic
math concepts, it should make one skeptical, especially when someone like you
justifies it on the basis of popularity, pragmatism and "minutia".

It seems like you read one article from the Washington Post that wasn't
written very well and from that one article, deduce that WP is on some smear
campaign. Absurdity of the highest order.

------
yummyfajitas
The main criticism of the article is a bit silly:

 _Khan will put the video out there and see how people react to it. He
perceives this to be a better approach than incorporating results of quality
research projects into his instructional decisions. In the age of No Child
Left Behind and its mandate for “scientifically based research” as the
foundation for classroom instruction, this seems lazy._

Yes, Khan is doing his own scientific research, on the theory that he does a
better job than the educational establishment. This might be arrogant (given
the quality of educational research I've seen, I don't think it is), but it
isn't lazy.

As for the claims that Khan does a worse job than other teachers, no data is
provided to back this up. Just vague claims that if he did something
different, he might get better results.

~~~
smacktoward
But just "putting the video out there and seeing how people react to it" isn't
"scientific research." Science involves testable hypotheses and controlled
experiments. Just throwing things against the wall and seeing which ones stick
is something different.

~~~
PotatoEngineer
"Putting the video out there" is an experiment. It's not _rigorously_
controlled, but it's about as controlled as a website is likely to get. So
they put something out there, see how well it works, and maybe tweak it.

That's the very essence of scientific experiments. You don't always need a
"hypothesis" to do science. (Or, if you like, the hypothesis is "This video
will teach students to do X.")

------
latch
This article is a critique of the traditional aspect of teaching (having a
teacher lecture students). This is obviously important, so it's good to have
professional educators provide feedback.

However, I thought the aspects that was supposed to be "revolutionary" was
detailed tracking and analysis of students and that feedback loop (either back
into the system, to the student themselves or to teachers). They seem to
completely ignore this.

Khan Academy isn't disruptive because of Salman Khan's lessons. It's
disruptive because of the underlying platform, which any teacher (some
possibly better than Salman) or any student could take advantage of.

~~~
mgurlitz
I think one of the most groundbreaking ideas they have is "flipping the
classroom" [1], where students watch Sal's videos at home before class for
homework, then in class they work on problems with the teacher and other
students. That's where the ability for teachers to track and target
deficiencies really becomes powerful.

[1]: [http://www.usatoday.com/life/people/story/2012-05-30/sal-
kha...](http://www.usatoday.com/life/people/story/2012-05-30/sal-khan-profile-
khan-academy/55270348/1)

~~~
binxbolling
I have seen this firsthand in Massachusetts and it was very powerful. The kids
were all clustered around iPads or Macbooks, helping each other, doing
problems, and reviewing what they learned for homework. The teacher was
rotating around the room, helping stuck kids, encouraging others to do peer-
to-peer coaching, etc. It was so absurdly different than how I "did school,"
yet so logical. The teacher felt she could now really differentiate her
instruction, and the kids were impressively engaged. Gone are the days of 40
minute lectures and 10 minutes of practice— you know, where the kids who got
the lecture blaze through worksheets and the rest sit dumbfounded and then
take all their problems & questions home for the evening.

All that to say: yes, a dude sketching some problems on a tablet is not the
"revolutionary" part (if there is one at all) of Khan Academy.

------
mtgx
One of the issues I have with Khan Academy is that they are using exactly the
same style of teaching as in schools...just ported online. Is that really the
best way to go about online education?

I find the AcademicEarth.org method of simply filming classes even worse. Yes,
they do allow for what you'd normally call "quality education" to be
accessible by anyone online, so that's a pretty big step, but I get this
feeling that true disruptive online education is supposed to be something more
radical...to make things 10x better for learning, not just replicate the
offline learning experience.

~~~
Jach
I wouldn't call it exactly the same... The biggest difference that frequently
gets glossed over is the Coaching interface. In normal schools it's hard to
get one-on-one tutoring with someone who can actually help, as well as with
someone who knows exactly what you're struggling with. I'm sure most people
who've done tutor work have encountered students who can't even _describe_
what it is they're having problems with, they're that far gone.

What improvements do you hope to see? Making it easier to find short-term
tutors would be on my list, throw up some IRC channels that tutors can monitor
and get notified of students seeking help. I think one-on-one teaching is the
most optimal.

As for how easy it is to learn, as long as we aren't hardwiring people's
brains with information as seen in _The Matrix_ learning is going to still
take work. (Not necessarily hard work in every instance, but work.) I'd like
to see more ways to interact and make the knowledge part of one's own
experiences rather than as received truth (something Polanyi got mostly right
in his Personal Knowledge book). Of course I'm instantly suspicious of any
technique or product that tries to oversell its ability to teach concepts
faster with better retention, and even more suspicious when it tries to sell
itself as "it's so fun the user won't even realize they are learning!" That
said, I can see benefits of various systems over both doing nothing and the
status quo. Tools like Anki for instance work well, if used. The advice some
programming texts give to manually type the programs also has obvious
benefits, basically a subset of techniques that train the mind to follow and
notice common actions. (There was an app linked on HN a few weeks ago that
replicates the motions of solving algebra problems without actually doing any
algebra, I can see how "eliminating two like things from both sides" feeling
"okay" may help some people a little.) Khan has mentioned that he has offline
students watch lecture videos as homework and do problems or answer questions
during classroom time, the opposite of the traditional school, I can see its
benefits over the traditional method too for classes where information is
typically transferred via lecture.

What things like KA, AcademicEarth, and Wikipedia all have in common though is
that they change the utility function of learning. That is, when they're not
being used as crutches to get through the traditional schools. Students can
learn what they want, when they want, stress-free. The classical model is
"pass the tests with scores above this threshold or die." (Not literally of
course.) It encourages cramming and forgetting. Cumulative testing doesn't
help because the course only lasts maximally 9 months (with a big break in the
middle), and at higher levels only 3-4 months. Sometimes it's even just one
term rather than a full semester. It's not enough. Add in repetition of the
_wrong things_ (it's always the _most basic_ things that get repeated
endlessly when they're the easiest things to remember!), poor communication
between professors of what they actually cover, poor communication between
student and professor of what the student actually understands, fixed
deadlines, anxiety over money (whether living expenses or paying $1500 for a
dumb class), anxiety over spending X hours of your life for a useless
class/assignment, _retaking_ the entirety of a class instead of focusing on
what you're missing specifically, being forced to take classes one has no
interest or skill in, being exposed to one resource, _et cetera_ , it's
amazing the classical system works at all.

------
bhntr3
This article reeks of an academic defending his turf to me. Basically, it all
comes down to the author accusing Sal Khan of willfully ignoring research.
This is based on a quote that I believe is intentionally misinterpreted. Khan
says "you put stuff out there and you see how people react to it". This is
basic web testing stuff. That doesn't mean he's ignoring research. Like we
don't willfully ignore design best practices with the assumption that A/B
testing will sort out the UX. That would be stupid.

I like Khan Academy's videos. But if you watch Khan's TED talk, it's clear
neither he nor bill gates sees khan academy as operating outside the education
system. His lectures are meant as a complement to quality instruction in
classroom by a teacher. Criticizing the Khan academy ecosystem as an
independent and parallel instruction system is silly.

Khan is a great lecturer. He's better than every lecturer I've had. He may not
be the best total educator. I wrote a blog post about this a while back:
[http://cyclicreality.com/post/26719449459/why-khan-
academy-i...](http://cyclicreality.com/post/26719449459/why-khan-academy-is-
great)

The hype may be too much. Maybe it's not revolutionary. But getting a teacher
who can explain and engage recorded and available free online is a great idea
even if it's not the be all and end all of education.

------
macspoofing
Oh for God's sake. Khan Academy is great complementary source. It doesn't
replace schools and teachers. If you're a student and you devote extra time to
understanding material you learned in class by watching Khan's videos, you're
already on the right track, and you're going to do well because clearly you
care. If you gain some incorrect understanding of a concept because the video
either omitted some fact, or got it wrong, you'll get it wrong on the test,
and you'll learn why. If there's a discrepancy between one of Khan's videos
and what your teacher taught you, that's a great opportunity to ask your
teacher to explain it.

I don't see the downside to such a great resource.

//

Also, there are many many terrible teachers out there. I had terrible teachers
at the elementary and high school level. Teachers who were either incompetent
or did not care. Khan Academy is not the problem.

------
cmwright
I find the notion that these videos are ineffective baffling. When I found
these videos during my _university_ calculus course, they literally saved my
grade due to the extremely poor quality of my professor (and TAs). I find Sal
to be incredibly effective, no nonsense teacher who at this point has probably
taught me 5 or 6 different subjects with efficiency that no professor, nor TA
has provided.

As a recent graduate: Thank You, Sal! Keep up the good work.

~~~
Jtsummers
The critique wasn't that they are innefective, it's that they contain poor
pedagogy. Their specific examples are from arithmetic videos, where the
presentation and notation is inconsistent or the examples/problems were
insufficient and could easily lead to misunderstanding the material. Your
experience is with the calculus videos. It's possible the criticism doesn't
apply to them. Or because you were a more advanced student of mathematics and
were being provided feedback from your course any poor use of notation or poor
problem sets wouldn't have had the same, potentially negative, impact on you.

------
delinka
tl;dr, with my own interpretation of the article: "although we are not
students in today's world, we presume to know and understand the mind of
today's student and can tell you from our own learned perspective that Khan
just doesn't have what it takes to teach."

I'm certain I read nothing more than snobbery in this article. It reads like
"we didn't think of this free online disruptive academy first" or "how dare
you do something differently from and outside of Official Academia."

Do you know who the _best_ teachers are? I mean the absolute best, the ones
who actually _get_ what the student is struggling with? Other students-- the
ones who _just_ got through that particular lesson, the ones that _just_ had
the exact same struggle. The students' ability to teach like this declines
with experience with the material because it all starts to become second-hand
and they forget all the little things they struggled with early on. When an
instructor takes note of students teaching students and the hows and whys of
that interaction and remembers to cover the same territory in the next class,
he becomes the best non-student teacher he can.

My mother is my perfect anecdotal example of this. In her 50s, she's getting a
degree. While taking calculus, she also worked in the tutoring lab. After
understanding new material, she'd be an absolute godsend for students for
about two weeks and then her helpfulness would fall off. (You could also
attribute this to all these classes being mostly in sync with at most two
weeks of lag - that being the case, she would be less exposed to the
reteaching of the material ... and forget those little details.)

~~~
Jtsummers
I think you took something different from the article than I did. They provide
a critique, strongly worded at times, but not totally bashing Khan Academy or
the general approach. They specify their concerns, and even offer suggestions
on ways it can be improved (specifically, examining existing pedagogy and
coordinating with some of the top math and science teachers to generate better
content).

One of the authors is a current teacher, working with real students. He gets
feedback from them either directly (by questions, comments, curses) or
indirectly (homework, tests, quizzes) to gauge his effectiveness as an
educator. Khan (until they added problems to the site) didn't have this
feedback.

Their criticism boils down to a few main points.

1\. He uses an inconsistent presentation of algebra and arithmetic work and
notation, which could lead to students becoming confused when he suddenly
changes it.

2\. He uses a poor selection of examples, without offering more or better
examples students may develop a mathematical toolset based on a flawed
understanding, which will lead to problems further down the line.

3\. The problems available to students are insufficient to provide proper
mathematical practice and ensure that some of the common misunderstandings are
shown to be wrong.

~~~
crusso
So an article on the Washington Post site based upon a for-profit founder's
nit-picking of Khan's materials is representative of the right way to go about
improving what Khan himself says is an effort to supplement other educational
methods?

There's a difference between constructive criticism that's effectively
administered vs a publicly-rendered smear attempt that tries to hide behind
pedantry.

~~~
Jtsummers
My comment was in reply to delinka's TL;DR. And you seem to be using a similar
one. I don't know that it's "representative of the right way" to put
criticisms like this in such a public forum, but it's not exactly new either.
The first article is (is that the for-profit founder you're referring to?)
definitely not constructive criticism. This article is in response to Khan's
rebuttal, and at least offers a claim of more substantial research into Khan
Academy's offerings before they provide their criticism.

------
codegeek
I have looked at a few Khan videos and here is my 2 cents. The reason
he/videos is effective is not because he is teaching a great content (I mean
algebra is algebra). What he excels at is keeping _you_ engaged and interested
throughout. This makes tremendous difference to anyone who wants to learn.
There are lot of people who know great things but can they teach it to others?
Now that is an art that Khan has mastered.

------
tokenadult
Colin once again submits an article from the United States popular press, here
a co-authored op-ed piece, about mathematics education reform in the United
States, one of my main topics of personal research for more than a decade. The
regular opinion column to which this guest piece was submitted, the "Answer
Sheet" edited by Valerie Strauss, is basically propaganda for the current
school system, and has been caught here on HN before stretching facts beyond
all recognition to make political points.

<https://news.ycombinator.com/item?id=3327847>

Michael Paul Goldenberg has such a lengthy online trail of writings about
mathematics education that as I entered his name in Google, autocomplete
finished the search query

[https://www.google.com/search?q=michael+paul+goldenberg+math...](https://www.google.com/search?q=michael+paul+goldenberg+mathematics)

and I was led to some of his more recent writings. (I used to interact with
him quite regularly online in specialized email lists about mathematics
education reform, about a decade ago.) He, um, definitely has a point of view
in his approach to education reform. It's all about the providers for
altogether too many people who look at education results and school practices
in the United States.

The response to the usual excuses for United States school performance by
another observer,

[http://www.nationalreview.com/agenda/255997/are-tino-
sananda...](http://www.nationalreview.com/agenda/255997/are-tino-sanandaji-
and-nea-same-page-us-educational-performance-reihan-salam)

who points out how often critiques of schools in the United States are
responded to by excuses that shift blame from providers of "education" to the
learners in their care:

"Consider that Americans tend to have more disposable income than citizens of
other advanced market democracies, at least some of which can be devoted to
supplemental instruction. After all, parents have fairly strong incentives to
secure educational advantages for their children. This suggests that our
schools are performing very poorly indeed.

"Don’t believe the hype."

is closer to reality than many of the critiques of outside-the-box approaches
to mathematics education in the United States.

That said, I have been up-front here on HN in suggesting ways that Khan
Academy can improve, for example by building more online practice that is
truly problems rather than exercises (379 days ago),

<http://news.ycombinator.com/item?id=2760663>

"Just for friendly advice to the Khan Academy exercise developers, I'll repost
my FAQ about the distinction between "exercises" and "problems" in mathematics
education. It would be great to see more problems on the Khan Academy site."

and the Khan Academy developers have been listening, and I have had
interesting off-forum email interaction with them as they attempt to improve
the instructional model at Khan Academy.

To date, I recommend to my own children and to my clients in my own
supplemental mathematics education program that they also turn to ALEKS

<http://www.aleks.com/>

(Yet another edit. About the time I posted this, someone else asked below
another comment,

 _So who is making the site that will deliver more personalized instruction?
Where is the research that site will use, telling all about which kinds of
personalization are proven and how much effect they will have?_

and ALEKS is an answer to those questions in large part. Browse around the
ALEKS site to see its links to its research base.)

and to Art of Problem Solving

<http://www.artofproblemsolving.com/>

for more online mathematics instruction resources, and I also share specific
links to specialized sites on particular topics with clients and with my
children. Besides that, I fill my house with books about mathematics, and
circulate other books about mathematics frequently from various local
libraries.

I also recommend that all my students use the American Mathematics Competition

<http://amc.maa.org/>

materials and other mathematical contest materials as a reality check on how
well they are learning mathematics.

In general, I think mathematics is much too important a subject to be single-
sourced from any source. Especially, mathematics is much too important to be
left to the United States public school system in its current condition.

I was just rereading The Teaching Gap: Best Ideas from the World's Teachers
for Improving Education in the Classroom (1999) the other day. It reminded me
of facts I had already learned from other sources, including living overseas
for two three-year stays in east Asia.

"Readers who are parents will know that there are differences among American
teachers; they might even have fought to move their child from one teacher's
class into another teacher's class. Our point is that these differences, which
appear so large within our culture, are dwarfed by the gap in general methods
of teaching that exist across cultures. We are not talking about gaps in
teachers' competence but about a gap in teaching methods." p. x

"When we watched a lesson from another country, we suddenly saw something
different. Now we were struck by the similarity among the U.S. lessons and by
how different they were from the other country's lesson. When we watched a
Japanese lesson, for example, we noticed that the teacher presents a problem
to the students without first demonstrating how to solve the problem. We
realized that U.S. teachers almost never do this, and now we saw that a
feature we hardly noticed before is perhaps one of the most important features
of U.S. lessons--that the teacher almost always demonstrates a procedure for
solving problems before assigning them to students. This is the value of
cross-cultural comparisons. They allow us to detect the underlying
commonalities that define particular systems of teaching, commonalities that
otherwise hide in the background." p. 77

Plenty of authors, including some who should be better known and mentioned
more often by the co-authors of the article Colin kindly submitted here, have
had plenty of thoughtful things to say about ways in which United States
mathematical education could improve.

In February 2012, Annie Keeghan wrote a blog post, "Afraid of Your Child's
Math Textbook? You Should Be,"

[http://open.salon.com/blog/annie_keeghan/2012/02/17/afraid_o...](http://open.salon.com/blog/annie_keeghan/2012/02/17/afraid_of_your_childs_math_textbook_you_should_be)

in which she described the current process publishers follow in the United
States to produce new mathematics textbook. Low bids for writing, rushed
deadlines, and no one with a strong mathematical background reviewing the
books results in school textbooks that are not useful for learning
mathematics. Moreover, although all new textbook series in the United States
are likely to claim that they "expose" students to the Common Core standards,
they are not usually designed carefully to develop mathematical understanding
according to any set of standards.

In a January 2012 lecture,

<http://math.berkeley.edu/~wu/Lisbon2010_4.pdf>

Professor Hung-hsi Wu of UC Berkeley points out a problem of fraction addition
from the federal National Assessment of Educational Progress (NAEP) survey
project. On page 39 of his presentation handout (numbered in the .PDF of his
lecture notes as page 38), he shows the fraction addition problem

12/13 + 7/8

for which eighth grade students were not even required to give a numerically
exact answer, but only an estimate of the correct answer to the nearest
natural number from five answer choices. Even at that, very few students chose
the correct answer.

Patricia Clark Kenschaft, professor of mathematics at Montclair State
University in New Jersey, reported in her article "Racial Equity Requires
Teaching Elementary School Teachers More Mathematics" in the Notices of the
American Mathematical Society

<http://www.ams.org/notices/200502/fea-kenschaft.pdf>

about elementary teachers' knowledge of mathematics in New Jersey:

"The teachers are eager and able to learn. I vividly remember one summer class
when I taught why the multiplication algorithm works for two-digit numbers
using base ten blocks. I have no difficulty doing this with third graders, but
this particular class was all elementary school teachers. At the end of the
half hour, one third-grade teacher raised her hand. 'Why wasn’t I told this
secret before?' she demanded. It was one of those rare speechless moments for
Pat Kenschaft. In the quiet that ensued, the teacher stood up.

"'Did you know this secret before?' she asked the person nearest her. She
shook her head. 'Did you know this secret before?' the inquirer persisted,
walking around the class. 'Did you know this secret before?' she kept asking.
Everyone shook her or his head. She whirled around and looked at me with fury
in her eyes. 'Why wasn’t I taught this before? I’ve been teaching third grade
for thirty years. If I had been taught this thirty years ago, I could have
been such a better teacher!!!'"

A discussion of the Common Core Standards in Mathematics, "The Common Core
Math StandardsAre they a step forward or backward?"

<http://educationnext.org/the-common-core-math-standards/>

gets into further details of how mathematicians look at the general school
curriculum in the United States. It is not the worst curriculum possible, and
survivors of the system often have access to outside resources to supplement
school lessons, but the public school instruction in mathematics in the United
States still shows plenty of room for improvement.

After edit: I was asked in a reply what I think about the essay "Lockhart's
Lament." I think it is an interesting read, but less practical for reforming
mathematics education than I had hoped. (I say the same in general about
articles by Keith Devlin, the mathematician who popularized Lockhart's
Lament.) To reform education, it is important to be relentlessly empirical,
and look again and again and again at the best practices of the highest-
achieving countries. That's why I prefer several of the links I submitted to
Lockhart's interesting essay as policy guidance for United States parents,
taxpayers, and learners.

Another edit: HN user danso just kindly posted

<http://news.ycombinator.com/item?id=4301758>

a link to a response by Sal Khan in the same Washington Post op-ed column
about education. Direct link is

[http://www.washingtonpost.com/blogs/answer-sheet/post/sal-
kh...](http://www.washingtonpost.com/blogs/answer-sheet/post/sal-khan-
responds-to-critic/2012/07/25/gJQA83rW9W_blog.html)

~~~
crusso
_and the Khan Academy developers have been listening, and I have had
interesting off-forum email interaction_

And THERE is the difference between genuinely-given constructive criticism and
the questionably-motivated smearing that those folks working with the
Washington Post are attempting to deliver.

Thank you very much for the links. I've already sent some of them on to my
wife so that she and I can research them a bit more as instructional aids for
our children.

~~~
Trianglemancsd
As one of those who worked on one of the Washington Post pieces, I would
invite you to come by my blog, <a
href="[http://christopherdanielson.wordpress.com>Overthinking](http://christopherdanielson.wordpress.com>Overthinking)
My Teaching</a>, where I think you will find my motives to quite pure-better
understanding of student learning of mathematics.

In fact it was very important to me that the piece I co-wrote
<strong>not</strong> be a smear piece. I can't speak for the comments I did
not write. But a careful read of the post itself should reveal a claim backed
up by evidence. The claim is that Sal Khan is lacking some important knowledge
for teaching. That's not a smear. I would gladly engage in a discussion that
pointed to evidence that he does know some important things about how people
learn.

~~~
crusso
I looked at your blog. From my perspective, you folks are down in the weeds.
Khan isn't using the optimal technique for teaching decimals? Who cares? The
problem isn't that kids aren't learning optimal methods.

The problem with education in this country is that we funnel most all of our
public education dollars into a system and related organizations that are only
tangentially motivated to educate our children. Watch how the public school
systems and teachers unions react to vouchers and charter schools and you'll
see how much they care about educating kids vs protecting their turf and power
base.

I've noted that those within the education system or closely aligned with it
absolutely hated "Waiting for Superman". They nitpick it and try to blow up
minor faults with it in a similar way to how you're going after Khan. To those
of us outside of that power structure, that documentary showed how the
establishment is utterly opposed to actually solving problems in education in
this country if those solutions occur outside of that establishment.

I view your articles vs Khan to be in a similar vein. What Khan has done is
he's added a solution to our society for our education problems that operates
outside of the current public education power structure. Rather than build on
what he's done and improve on it through working with Khan or doing something
better -- you folks have taken the route of trying to publicly take him down
in the media.

------
pmelendez
From the article:

... "“I think frankly, the best way to do it is you put stuff out there and
you see how people react to it; and we have exercises on our site too, so we
see whether they’re able to see how they react to it anecdotally.”

Khan will put the video out there and see how people react to it. He perceives
this to be a better approach than incorporating results of quality research
projects into his instructional decisions. In the age of No Child Left Behind
and its mandate for “scientifically based research” as the foundation for
classroom instruction, this seems lazy." ...

I was agreed with the author until I read those lines. In my opinion they are
disingenuous at the very least.

Every teacher, even those ones with a high knowledge on PCK, have to adapt
their content every year. They try with an explanation on a topic and see how
students react and based on that improve the content for the next year.

That's exactly what Khan said he is doing, publish and see how people react to
those videos. It might be harder for him to measure the content in comparison
with an on site teacher. But saying that "Sal Khan is on record as dismissing
this research" seems an overstatement to me in the best case.

------
cmcewen
This criticism does seem to be more harsh than is necessary, but I don't see
what the problem is in seeking the advice from those with more experience
teaching. Khan has created something amazing, but that doesn't mean he is
infallible nor that his videos should stay as they are today.

If you read the second to last paragraph, they don't suggest that Khan Academy
shuts down, or that he stops teaching. They literally just say, "Hey, it'd be
nice if he asked some really great teachers to look over his examples before
he uses them to make sure he doesn't confuse people." I'm not an expert, but
sounds like it couldn't hurt.

------
cvursache
It's probably a good sign that Khan Academy is getting criticized by some
members of mainstream media lately. It shows the growing importance of a great
disruptive organization.

~~~
rimantas
Or it may be just that it is worth criticizing? I personally found Khan
Academy lessons of very poor quality.

------
wtvanhest
I’m a huge fan of the Kahn academy videos and in order for it to get even
better it needs people to criticize the curriculum so I am a big fan of this
article.

I view Kahn Academy as the start to a new type of education that is in its
very early stages. Kahn is a smart man who will probably take the criticism
and use it to be better his instruction and add supplemental videos etc. I
would be shocked if he didn’t add in a 3rd comparing decimals video after this
criticism.

Keep up the great work, I know I’m watching.

------
novalis
"Comparing decimals. Decimal fractions (decimals for short — the numbers to
the right of the decimal point) are a notoriously challenging topic in the
elementary math curriculum."

Just to point out this is the most ass backwards way to talk about decimal
fractions I have ever witnessed, but nothing would stop said Valerie Strauss
on pedagogical insufficiency spotting... "decimals for short — the numbers to
the right of the decimal point" ?!? Whisky Tango Foxtrot

Edit: downvoted is it because you would also believe decimal fractions are
decimals and those are the numbers to the right of the decimal point per
article explanation... because being a silent ignorant with a karma finger
really fits you.

~~~
Jtsummers
The author isn't Valerie Strauss.

"Decimals" is (in the US at least) a common way of referring to decimal
fractions (that is, numbers represented like: 12.345) vs the whole number or
integer representation that the students would already be familiar with (like
12) or other fractional notations like 6 7/10.

~~~
novalis
I am under the impression the author of the article is Valerie Strauss and she
is presenting a small piece critique by Christopher Danielson and Michael Paul
Goldenberg at the top of her article. If not, there is a problem with the
placement of her name at the end of it.

As for the abbreviation presented, it is just terrible and is used in article
while accusing others of lack of Pedagogical Content Knowledge.

~~~
Jtsummers
So you believe she speaks about herself using "We" instead of "I"?

Her contribution is the first 2 paragraphs, the rest is the critique from
Christopher Danielson and Michael Paul Goldenberg.

Edit: And please explain, how is the abbreviation terrible, because I've
clearly missed something.

~~~
novalis
"We" may well be used instead of "I" and at the end it reads "By Valerie
Strauss". The attribution is placed in a random way all through the article
and that threw me.

The abbreviation is terrible because in the article it is used in reference to
"the numbers to the right of the decimal point"; "decimals for short".

~~~
Jtsummers
To the first point: The first and second paragraphs are italicized, clearly
distinct from the rest. This is intended to show a distinction in voice
(authorship). The content of the first two paragraphs are nothing but framing
for what follows. You're either being excessively pedantic, or have elected
not to read the article to have so much difficulty in recognizing authorship
of the 2 parts.

To the second, I still don't see what makes that terrible. It's saying,
decimals is short for decimal fractions, and provides a definition for decimal
fractions.

~~~
novalis
Article starts with: "How well does Khan Academy teach? By Valerie Strauss",
then "By Christopher Danielson and Michael Paul Goldenberg" on top of their
contribution. At the end of the article there is a : "By Valerie Strauss". So
attribution is displayed twice before text blocks and once after text blocks,
if you don't see the failure in that, I am so sorry. I don't know how pedantic
it is to confuse voices when they are attributed in two different ways when it
is clear that the second way is the most common, but I will leave it to you to
quantify it, not that I care for it. But do keep up the winding, you will be
on to something, sometime soon.

Second: "Comparing decimals. Decimal fractions (decimals for short — the
numbers to the right of the decimal point) are a notoriously challenging topic
in the elementary math curriculum."

For the last time, decimals are NOT the numbers to the right of the decimal
point like stated. "decimals" is being used for a subpar abbreviation of
decimal fractions AND the left side numbers to the decimal point are a part of
it, unlike what said authors state. Someone with pedagogical concerns should
be awake when stating stuff like this and if you don't see the mess that is,
that is your own problem. I have stated this already.

------
philwise
The article misses the point of Khan Academy. The point is that the world only
ever has to find _one good example_ of how to teach something.

If someone comes up with a theory on a better way to teach something, cool.
A/B test it and keep the winner.

------
anon_d
Fuck that noise. If you don't like it, do it better; all this whining and nit-
picking is without value. Khan Academy is innovative and game changing, but
you can't innovate by insisting on perfection.

~~~
gizmo686
No, but after you inovation comes incremental improvement. If Khan Academy
does not do this (and I have no idea weather or not they are), then another
group will take their highly inovative idea and do it better.

------
jdietrich
Across the developed world libraries are closing, or being forced to reinvent
themselves. They're doing this because their original purpose is being done
better by new technology. It is simply no longer rational to have a big
building to house information printed on slices of dead tree. Libraries and
librarians do lots of important and useful things other than simply lend
books, but they're having to figure out how to reinvent themselves now that
the core service is obsolete.

The vast majority of contact hours in the vast majority of educational
institutions consist of chalk-and-talk - someone writing things on a board and
talking. The available evidence shows that there is no clear benefit to doing
this in the flesh over delivering the same approach through video, and that
there might be significant benefits to providing it in a modular format that
can be paused and rewound. Sal might not be the greatest math teacher on
earth, but soon enough, somewhere on the internet will be lessons by 99th
percentile teachers on every imaginable topic.

If your best argument is "We can do what the internet does, only marginally
better", then you're in deep trouble - we've seen how well that has played out
for any number of people. The economies of scale are too great, the rate of
iteration too rapid. You're just not going to beat the internet at supplying
data. Educators and schools have to work out what they are uniquely equipped
to do, or face the same inevitable obsolescence that is befalling libraries
and travel agents and local newspapers and record stores and myriad other
businesses.

------
danso
It should be noted that Kaplan, Inc. is owned by the Washington Post Co. and
is, I believe, the WaPo Co.'s largest source of revenue:
<http://en.wikipedia.org/wiki/Kaplan,_Inc>.

(I'm of the belief that it's possible for a large news company to give equal
commentary share to competing interests, but I don't follow the WaPo's
education coverage enough to be able to make a judgment in this case)

------
nhp
It seems to me that most of the unpleasantness in Khan Academy criticisms stem
from the fact that, most of these comment concern other peoples' comments on
Khan Academy. Revolutionary/Messiah v/s borg/wrath-of-khan/emperor-without-
clothes and so on.

This is sad because the actual message, which is valid in some instances and
informative in all instances, gets lost in the noise. This is more so, given
that people at Khan Academy are responding positively to specific concerns on
specific topics.

And unlike for-profit companies, all his work, from the most boring video to
the best, are available to be criticized. We only see a few clippings from,
most likely, the very best videos from for-profit companies.

I hope the real critics and the real skeptics, i.e., those who would like to
make Khan Academy a success, will continue to bring their specific concerns to
the table; and just not all the distracting noise.

(Now, it is strange that the authors while pointing out the exaggerated praise
for Khan Academy, fail to point out the abuse that Khan had to endure. Even
stranger is that some commentators dismiss the abuse as "satire".)

------
tbatterii
Where is the video that represents the "right" way of comparing decimals? I'd
be genuinely curious to compare it with the khan video. And if that ever
happens, then Khan Academy is a net win in my opinion.

I would think it would be easy for the "nit-picking experts" to produce that
and it probably would have taken less time than the formal analysis they did.
Probably cheaper too.

edit: wording

------
delwin
Any critique that world-class teachers give Khan Academy would apply to most
K-12 math teachers as well. I think people are worried because Khan Academy
can spread much further (and do much more damage) than a conventional teacher,
who is fired if standardized testing scores drop below an arbitrary amount.

BUT the same effect is seen on Wikipedia, which has been reviewed and
corrected more than thousands of other encyclopedias, simply because it's so
open and accessible. Khan Academy was originally run by one guy, but they're
starting to open up and partner with other educational entities. They aren't
going to go unchecked. By no means is Khan Academy hurting the education
system, unless you look at things from an economic or political standpoint, in
which case it's potentially disruptive. But that would be cruel, wouldn't it?

------
drcube
"Our view is that content knowledge alone is inadequate for quality
instruction."

I cannot disagree more. The best teachers I had were college professors who
were all experts in their field and never took a class on pedagogy or
education. But they had the autonomy to change their teaching style based on
what was most effective for them and their students.

Our grade school teachers would be a lot better if they were experts in their
domain of knowledge and given more leeway to teach according to their
strengths and their students' needs. As it is, public school teachers are
pretty much automatons whose actions are dictated down to the minute by school
boards.

~~~
com
My best teachers were also content experts.

My worst teachers by a long shot were also content experts, holding, in one
case, the research chair of an academic department in a renowned research
university.

Couldn't teach to save his life and was lucky that the tutors put in a lot of
extra hours so that around 50% of the enrolled students passed his fairly easy
course (it was a revelation that after I had failed, and had to re-sit during
the summer, I asked an academic in a related field to coach me based on a
text, and his explanations were lucid, to the point, had an expository style
that engaged and that he actually wanted to address the material at hand).

An intelligent person with some domain knowledge, good teaching materials and
firm grasp on how students learn beats someone with all the domain knowledge
in the world who is a hopeless teacher, in my experience.

------
MTChirps
It's kind of an interesting question. Is it better to educate masses with an
inferior product that breeds misconception, or fewer people with a better
product that limits misconception?

~~~
smacktoward
If it breeds misconception, can you really call it education?

~~~
MTChirps
Perhaps the word "breeds" is strong. I would argue that even though Khan's
lessons are very imperfect, the accessibility offered may be a worthy trade-
off.

If knowing math at a superficial level opens doors for other learning, it's
hard to argue that it's NOT a worthy trade-off.

We as math teachers strive for perfection, but the truth is that having a deep
understanding of what an equals sign is and how it should function isn't
important for most people in their lives. Sure those in math careers will want
that depth (and they'll get it, since Khan alone can't prepare them for that
career), but everyone else will be fine whether they use the equals sign
properly or not.

In a lot of ways it's like prescriptive vs. descriptive linguistics. Are we
going to be the equivalent of a grammar nazi and come down hard on everyone we
see that uses improper notation? Or can we be satisfied if there is some
understanding there, even if it's imperfect or communicated poorly?

------
johnnyg
One snob article vs. a huge body of innovative work.

Game, set, match to Mr. Khan.

------
Tloewald
The thing I find most bizarre about Khan Academy (which, in principal, I love)
is the whacky focus on Khan. Why not crowd source the tutorials?

In my opinion Stack Overflow is a better model for this stuff. Imagine
something like Stack Overflow focused on video tutorials covering concepts
that got up voted. (heck, maybe it exists.) why can't I upload my take on
"slope" (or whatever) rather than just post Q&A on Khan's?

------
spinchange
Can anyone weigh in on their differing arguments regarding the definition of
slope? I am not a maths expert by any means, but am genuinely curious who is
correct here. Or is this a matter of one being technically correct versus the
other being correct in practice?

------
vph
Khan Academy is social-network type of learning, where you learn from the
community. This is good. But learning from expert educators is also essential
and perhaps irreplaceable. There are a lot of short-sighted people, who are
marveled by current technologies that allow people interact remotely and
immediately jump to the conclusion that we have an answer to learning, while
dismissing the entire tradition, culture and knowledge of and about learning
that has been established for centuries. Now, if Khan wants to make a quick
buck, that's fine. But he's serious about teaching and learning, he should
realize that Khan Academy and its approach is only complementary (at extremely
great at that) and is not a replacement of the traditional ways of teaching
and learning.

------
yread
Wow, such a high quality discussion on HN especially compared to the one under
the article. And on a topic that's only tangentially related to startups and
tech...

------
BlackNapoleon
So because its not perfect, lets drag it through the mud.

This is ONE MAN.

------
voxx
170,000,000 lessons.

"We contend that"

No. You cannot contend against 170 million.

