
What Students Do (And Don't Do) in Khan Academy - ColinWright
http://blog.mrmeyer.com/2014/what-students-do-and-dont-do-in-khan-academy/
======
Patrick_Devine
I've noticed with most critiques of KA that they set up a straw man of what KA
is trying to accomplish, and then they attack that, almost always without
giving a better alternative.

The fact is KA is a great tool for learning many subjects, and will only
continue to improve. The biggest take aways should be:

\- spaced repetition of subject matter is extremely useful for long term
memorization \- bite sized, repeatable lessons allow students to digest more
and "rewind" over parts of lectures they may have missed \- "gameification"
and small rewards are useful for getting students to keep pursuing a subject
\- students shouldn't move on to more advanced subjects until they've mastered
a particular foundational skill

No teaching tool or method is going to be perfect, but I think KA gives
parents, educators and most importantly _students_ a really useful tool for
self-paced learning that traditional methods can't easily offer.

~~~
IndianAstronaut
It should be noted that there are huge vested interests who want to tear down
Khan academy and similar sites.

~~~
zo1
Which "huge vested interests" are you referring to specifically? I can
understand that if the Khan academy were to _really_ take off, it'd shake
quite a lot of educational institutions. Everything from universities, state-
funded schools, teachers unions, the works. And I don't think a lot of them
would like that, as I it would appear to the individuals running/manning them
that their jobs are on the line.

The reason I say state-funded schools wouldn't like this is because they work
on a false premise. Supposedly that education is not free/cheap enough, and
that government funding is required to make it work for the masses. You take
away that, and public/state schools become glorified daycare so that the
parents can stay productive instead of having to babysit their offspring.

One addition: I've just setup a recurring donation to the Khan academy. One
small step at proving that society can function without state-coercion.

------
iandanforth
Please for the love of maths do not "align" with the common core standards. It
would be the saddest, most disheartening thing imaginable* for Khan Academy to
become a feeder to the American standardized testing system.

The author writes: "If one of Khan Academy’s goals is to prepare students for
success in Common Core mathematics, they’re emphasizing the wrong set of
skills."

I would like nothing better than the academy to publicly state this is NOT a
goal of theirs. Being good at math and passing math tests are orthogonal*
concepts. In my experience teaching to the test is the fastest way to kill the
love of any subject because, at the end of the day, the only answer you can
give to the question "Why is this important? or "Why should we do it this
way?" is "Because the test says so." It's _death_ to independent thought.

I don't know who his advisor is but please kill this dissertation before it
gets out into the world and he uses it for self promotion on Good Morning
America.

*intentional hyperbole

~~~
yummyfajitas
Consider a well designed test of an important curriculum. This means the
topics are important for $REASONS, and the test is highly correlated with
knowledge of the topics. If a student asks "why is this important", you can
reply "$REASONS". And you can "teach to the test" secure in the knowledge that
you are teaching useful materials.

Now, if you believe the curriculum is poorly designed or the test is poorly
correlated with it, I encourage you to make that argument. But explaining why
"Know relative sizes of measurement units within one system of units including
km, m, cm..." should not be part of the curriculum is a lot harder than making
emotional pleas about "love of the subject".

(I've skimmed parts of the common core on a few occasions. It looks pretty
good to me.
[http://www.corestandards.org/Math/Content/4/MD/](http://www.corestandards.org/Math/Content/4/MD/)
)

~~~
VLM
I can't speak for OP, although I basically agree with you and OP
simultaneously.

However, look at it as an optimization problem. CC testing standards and
format may be ideal testing tools for 49 minutes 5th period after lunch 5 days
a week in a 40 person classroom, performed in perfect lock step by all 40
students, only failure mode is repeating the entire class or developing a
hatred of the topic, aka standard public school experience.

However, what does "alignment" WRT KA mean when the educational experience has
completely changed? Its analog, you can watch a video 50 times will you get
it, jump back and forth, all that. I'd propose the "best" testing technique
for that environment probably looks a lot like KA and not so much CC because
you need instant feedback and simple questions. Its not quite as good as
personal 1 on 1 tutoring, but its better than sitting in class, more
importantly the experience is inherently different than sitting in class...

The proposal in the linked article seems to be that KA epic fails, solely in
"alignment", because it doesn't test like CC, which is trivial to test and
prove and analyze statistically. Factually looks like a good study. Maybe it
fails the relevance test because "alignment" is fuzzy to me (maybe it has a
special jargon meaning among CC teachers?) I think MAYBE OP and I are
concerned that KA aligns pretty well to a decent curriculum (CC) WRT topics
and order of topics and presentation standards, and I hope KA doesn't ruin the
KA assessment portion to better match CC assessments solely to get a checkbox
somewhere.

Maybe we can abstract out KA from the discussion. I'm tolerably good at math
and have tutored people. If you define alignment with CC as slavishly
following their assessment technique at all times even though I'm 1 on 1 in an
ongoing human conversation, such that everything I speak must statistically
match the CC assessment percentages in style, tone, and technique, that's
ridiculous. If you defined alignment as I'm talking about sets of linear
equations with infinite solution set the same general time, order, and way the
CC talks about it, thats OK, even if I assess a tutored student the "wrong
way" by asking the wrong type of questions (which may very well be the right
question for that kid at that moment while also being on average the wrong way
to teach all American Kids in a 40 person classroom)

Its seems reasonably non controversial that CC curriculum and assessment are
local maxima on society wide averages for teaching math. It seems reasonably
non controversial that KAs assessment tools seem to be a local maxima for
computerized delivery of help to kids that need extra help. I (and possibly
OP) don't think it reasonable to trash talk KA because the local maxima of
effectiveness for society as an average whole in a classroom setting doesn't
necessarily match KAs local maxima of effectiveness for "kids who need help at
home". The danger is screwing up KA enough to make it useless for the kids who
need it after school, just to achieve a marketing checkmark.

~~~
yummyfajitas
So from what little I've seen, KA is reasonably well aligned with common core
already. Most decent curriculums do. Also, I think you are misunderstanding
what CC is. CC is just a list of topics students should know - there is no "CC
testing standards and format".

If KA doesn't align with CC, it's because there is some particular topic that
KA forgot about. E.g. maybe common core requires students to _" Solve problems
involving addition and subtraction of fractions by using information presented
in line plots"_, but Khan forgot about that one.

I also don't think it would be harmful if KA made sure it covered the basic
topics of AIEEE or the IIT-JEE either (important Indian standardized exams). A
lot of people want to pass those exams and it would be a marginal amount of
extra work for KA to help them.

~~~
kennytilton
"there is no "CC testing standards and format"."

I see two. One is called PARC, one is called Smarter Balanced. And they want
to use them to close schools and fire teachers, so look for everyone to teach
to whatever sample tests they can get their hands on.

~~~
jblad
PARC and SBAC are indeed tests intended to measure students against the Common
Core State Standards. They are not from, or a part of, the CCSSI. People
definitely do want to misuse these tests to punish schools and teachers, but
none of this invalidates the Common Core. And while people will teach to the
sample tests, that's on people who want to use these to destroy schools, not
on the Common Core itself.

~~~
kennytilton
Ah, but if you check out the Common Core page they go on and on about student
performance being comparable between states, which is why the two standardized
tests in effect are inseparable from CCSS. btw, I know what you meant by the
rhetorical "people...do not want", but the problem is precisely that
"accountability" is the other dog whistle never far from any CCSS discussion.

I am afraid your healthy view of CCSS is not widely shared, and the discussion
would be much better if CC was offered purely as PD. Teachers love anything
that works.

------
Spooky23
Reading this article reinforced a perception I walked away with at an event
where parents were complaining about common core related topics.

The professional educator dialect of English is not very approachable for the
layman. To many people it sounds like somebody throwing out $10 words to make
the arguments sound more intelligent.

~~~
jwmerrill
> The professional educator dialect of English is not very approachable for
> the layman.

I think Dan's blog is largely pitched at other professional educators. All
professions develop their own technical vocabulary.

As an analogy, I recently enjoyed an article about the implementation of the
v8 garbage collector by Jay Conrod: [http://jayconrod.com/posts/55/a-tour-
of-v8-garbage-collectio...](http://jayconrod.com/posts/55/a-tour-
of-v8-garbage-collection)

Here's a sample paragraph:

"Distinguishing pointers and data on the heap is the first problem any garbage
collector needs to solve. The GC needs to follow pointers in order to discover
live objects. Most garbage collection algorithms can migrate objects from one
part of memory to another (to reduce fragmentation and increase locality), so
we also need to be able to rewrite pointers without disturbing plain old
data."

I think the article is an excellent piece of technical writing, but it would
also be reasonable to say "the professional compiler engineer dialect of
English is not very approachable to the layman."

That said, good communicators need to be able to change their presentation
based on the audience, and it's too bad that the people at the meeting you
mention couldn't find a common vocabulary.

If you want to see something by the author of this post that is pitched at the
layman, he made a very nice TedX presentation a few years ago:

[http://www.ted.com/talks/dan_meyer_math_curriculum_makeover?...](http://www.ted.com/talks/dan_meyer_math_curriculum_makeover?language=en#t-202117)

~~~
VLM
One sophistry problem is we will "ALL" theoretically take and pass Algebra, so
the teacher does not have the luxury of specialized jargon, the teacher MUST
be able to present it to the entire general public. A general education
teacher is expected to present effectively to the general public as the core
of their job, its not a nice to have or wouldn't it be interesting if it were
possible.

On the other hand, only maybe 1% of the population has to suffer thru garbage
collector theory in CS classes, and of that 1%, maybe only 0.0001% ever
implement a GC "in the real world" and can understand that post. There just
aren't that many GC programmers and there are a lot of humans in the general
public. In that case jargon is perfectly acceptable, even expected. Also the
core of a programmers job is rarely making public presentations.

~~~
jwmerrill
I didn't mean that teachers get a pass on opaque vocabulary when they're
teaching students. I meant that, just like every other profession, they have a
technical vocabulary that they use to discuss their own profession with other
teachers.

------
codingdave
Khan Academy isn't perfect. But it improves every year. I like it for what it
is, and have high hopes for what it might become.

~~~
gravity13
Can Khan Academy ever become something other than Khan Academy?

That's a profound question for me.

------
tokenadult
The author writes, in the interesting blog post submitted here, "We should
wonder why Khan Academy emphasizes this particular work. I have no inside
knowledge of Khan Academy’s operations or vision." Actually, here on Hacker
News we have had public discussions blessed with direct participation by Khan
Academy developers, so I think I have some sense of what Khan Academy intends
to do as it continues to develop online lessons. I've even had some off-site
email interaction with Khan Academy developers after threads here in which I
have "met" them while we discuss our common interest in mathematics education.
I certainly hope that Khan Academy staff will join the discussion of this
post.

The author also writes, "I find it more likely that Khan Academy’s exercise
set draws an accurate map of the strengths and weaknesses of education
technology in 2014. Khan Academy asks students to solve and calculate so
frequently, not because those are the mathematical actions mathematicians and
math teachers value most, but because those problems are easy to assign with a
computer in 2014." I think this statement is correct. The Khan Academy
developers have devoted a lot of thought to how to computerize problems rather
than exercises, and they are still working on building new problem types for
the mathematics lessons. I'm actually quite impressed with some of the problem
types that show up in more advanced mathematics lessons on Khan Academy that
go beyond the eighth grade level implied by current United States curriculum
standards. But the author's point is well taken that the Common Core, and good
mathematics curricula around the world, expect reasoning and explanation from
pupils in mathematics lessons already at elementary school age. That currently
is indeed difficult to automate.

The author's comments on development of Smarter Balanced Assessment (SBC) as a
new platform of online mathematics instruction is of course especially
interesting. I will be curious to see how this program continues to develop,
and I would be glad to hear from developers of that program in this thread
too.

(Disclosure: I am a teacher of voluntary-participation prealgebra mathematics
classes to advanced pupils of third-, fourth-, and fifth-grade age, using the
Art of Problem Solving prealgebra textbook by Rusczyk, Patrick, and Boppana as
a core text with many supplementary materials from around the world. I do not
view Khan Academy as a competitor to my classes, but rather as a helpful
supplement for many mathematics learners around the world. My daughter, who is
of a younger age than eighth-grade age, is currently doing many of the same
Khan Academy lessons as those just reviewed by the blog post author. I
recommend Khan Academy to my friends, and it will probably stay in the mix for
the mathematics studies of my three minor children for quite a few years yet.)

~~~
spicyj
(I work at Khan Academy.)

> _The Khan Academy developers have devoted a lot of thought to hope to
> computerize problems rather than exercises, and they are still working on
> building new problem types for the mathematics lessons. I 'm actually quite
> impressed with some of the problem types that show up in more advanced
> mathematics lessons on Khan Academy that go beyond the eighth grade level
> implied by current United States curriculum standards._

Thank you.

> _Common Core, and good mathematics curricular around the world, expect
> reasoning and explanation from pupils in mathematics lessons already at
> elementary school age. That currently is indeed difficult to automate._

This is definitely true. By no means is Khan Academy a complete solution for
mathematics instruction, nor is it likely to become one in the next few years.
Our site should ideally be used alongside other materials that test reasoning
and explanation in a non-automated way, as well as projects that allow
students to apply their knowledge. We have early experiments that take a much
more interactive approach to teaching certain concepts and encourage more
experimentation, but we're probably years away from developing them into a
shipping product.

> _The author 's comments on development of Smarter Balanced Assessment (SBC)
> as a new platform of online mathematics instruction is of course especially
> interesting. I will be curious to see how this program continues to develop,
> and I would be glad to hear from developers of that program in this thread
> too._

If I'm not mistaken, many of the SBAC problems (especially free-response ones)
are hand-graded, which can work if you charge for each test taken, but can't
work for us when we have millions of problems done every day, all graded
automatically. That said, we're exploring strategies like peer grading in
other subject areas (particularly our programming curriculum) and would be
excited to bring them to math as well.

~~~
zanny
> that test reasoning and explanation in a non-automated way

> early experiments that take a much more interactive approach

> we're exploring strategies like peer grading

Are you guys working on any genetic / learning algorithm implementations of AI
sufficient enough to evaluate responses to arbitrary prompts? It sounds both
like something that should be feasible with modern tech, especially when time
to response can be large (a student can wait hours for an answer to a complex
essay response) and the scale can be arbitrary (as many server clusters as
"its worth" yet also feasible (we can see similar scale literary processing in
other spaces).

I can imagine such an algorithmic implementation where you take some common
query given to students across hundreds of schools, with all the teacher
graded responses, and maybe after a few years it can get within an accuracy
confidence threshold to start grading the prompt, and after another few years
providing critique on that grade. That would be the most revolutionary
educational tech since the Internet itself, even if it were pigeonholed in
several years studying the same single problem for now. Sounds awesome either
way.

You can even use that peer grading tech to build up a library of problem /
solutions to build your causal relations table from. I just hope you guys have
_someone_ in a back room working on this, because to me it seems as real world
usable computational linguistics happen more often, there arise a ton of
useful applications even in the minimum case.

~~~
aroberge
False positives or false negatives in this type of fuzzy correction could be
very detrimental to students. Khan Academy is doing an excellent job at what
it is currently aiming to do. Doing less well by trying to do more would
decrease its value.

------
tjl
While I like the idea of Khan Academy, there are problems with some of the
contents of the lessons. For example, the lesson on Hooke's Law definitely has
issues:

[http://www.leancrew.com/all-this/2012/12/khan/](http://www.leancrew.com/all-
this/2012/12/khan/)

------
mkramlich
I visited KA for the first time in a while. It used to drop me into a table of
links to videos on various academic topics. Very simple, useful, fast, and the
Obviously Right Way (TM) to do it. At least in the default case, and for the
casual or first-time visitor.

However just now it required me to signup/login. Signing up asked for PII,
which is intrusive and strictly unnecessary for letting me see educational
videos. I gave a fake email, along with the other fields. It then complained
it couldn't send an email to it. When I pressed their button to fix the email
address, it then popped up a window for me to enter it, however it was auto-
forcing it down into an obviously stupid position such that it was impossible
to hit its Submit button. You can enter a new address but not hit Submit
because too close to the screen bottom's edge.

I love the orig vision, and the core content (Sal's lectures and his style)
but looks like they're now falling down at the simple stuff along the edges of
the problem space. Failing over inside the unnecessarily added complexity
(GUI/JS shenanigans, control fetish, privacy intrusion, etc), rather than in
the essential inescapable parts (text, images, embedded canned video, links,
curricula design, effective communication, etc.)

------
mathu7
I think all these sites (treehouse, code academy etc) are going to be hit with
class action lawsuits for implying that their app/whatever will make you
proficient at coding, and even land you a job. There are no shortcuts to
becoming good at coding. It takes much longer and is much harder than these
sites make it seem.

~~~
mathu7
yay down-votes. I'm not gonna delete the comment. I'll just make a new account

~~~
21echoes
in case you missed why your comment didn't add to the conversation: Khan
Academy and the article are both not about coding or learning to code. This
article is about math.

~~~
jimmaswell
Khan Academy did start to include a programming portion at some point.

~~~
21echoes
yes, Khan Academy has lessons on a huge number of topics. no, nothing in the
posted article has anything to do with "implying that [Khan Academy] will make
you proficient at coding"

