

Khan Academy and the Effectiveness of Science Videos - tokenadult
https://fnoschese.wordpress.com/2011/03/17/khan-academy-and-the-effectiveness-of-science-videos/

======
shii
I have experienced this first-hand. Let me give you a little anecdote:

I took Chemistry Honors as a sophomore in high school. Now a sophomore in
college/senior in high school, I'm again taking Chem since it's required as
part of my gen-ed. I basically never come to the packed lecture hall as I see
it as a waste of my time to listen to a repeat of a class I took 2 years ago.

I print off class lectures and view the khan vids and it's pretty easy. Yet
this hasn't really translated over to my performance in tests. I think a
culprit may be that I gloss over bits when I feel this all has a "I know this
stuff" feel.

Compare this to my biology honors course as a freshman in high school as well
as again as a freshman in college/junior in high school again. I didn't know
about khan academy then, and took notes in class for things that I wasn't sure
about (it was all still essentially review --props to my great HS freshman Bio
teachers). I also studied outside if class and did practice problems during my
free hour right after class. Aced the class and more effort was put towards
the learning. Any misconceptions/confusion was resolved with either a quick
google search or question at the tutoring center for homework help at college.

I plan to try and see if the physics course I'll be taking soon will produce
different results, since I haven't taken a physics class yet ever, so it'll be
pretty much all new. I plan on first studying with mainly Khan Academy only
and seeing the results, then with other more "traditional" methods. I'm mainly
testing to see whether the newness factor has any effect on my ability to
acquire and retain the content just from videos.

------
Udo
It's interesting that people are not really paying attention when the facts
presented differ from their own mental models - I suspect this also happens a
lot in school and university settings. I remember the only useful lessons were
those that actively engaged the learners (and not in some phony interview
style either, but real exchange with the teacher).

When watching almost any tutorial video, I find myself zoning out a lot. Often
times I even open up the browser and read something completely different while
the videos is still running.

A few weeks ago I had a discussion with a friend who challenged me to convince
him his adamant assertion that videos are the inevitable future of _all_ media
is wrong. At the time I just felt instinctively video was never going to
replace the written word and 2D graphical presentations, but I couldn't make a
convincing argument. It only now occurs to me there might actually be
scientific data in support of my suspicion and that instructions on video may
have an information-to-brain bandwidth similar to a very boring teacher after
all.

~~~
planckscnst
Unfortunately, most college lectures are even worse than videos. It is very
difficult to not zone-out or become distracted in a poorly-lit room with
uncomfortable chairs. And if you miss something, you are much less likely to
ask a professor to repeat it than you are to jump to a previous location in a
video.

I think the lectures need to be replaced by the videos as a home assignment.
Classroom time should be split into two activities - intelligent discussion on
the subject of the video, and actual problem solving with other classmates,
with the teacher immediately accessible for any questions.

~~~
yequalsx
I teach at mathematics at a community college. I have videos for each section
of the book and created problem sets for each section of the book. In essence,
I've created my own book.

My intention was to invert the role of homework and classwork. That is, that
homework would now consist of watching the lecture. Classwork would focus on
solving problems. The outcome? Disaster.

I tracked usage of the videos. Around half the class didn't watch a given
lecture. Around 1/3 consistently watched the lectures. The effect of this was
that over time only around 1/3 of the class was able (and willing) to engage
in problem solving and discussions about the material.

I had 2/3 of the students hopelessly lost by the tenth week. How does this
compare to the traditional format? When I give the standard lecture/homework
assignment type class around 2/3 will pass. I double the number of people
passing but I strongly suspect that they end up failing in the next course.
(It is quite easy to pass beginning math classes these days. ) My anecdotal
experience is that while I get more people to pass by giving lectures each day
I end up with a group of people who know the material less well.

My experience tells me that around 1/4 of the students will learn the material
from my video lectures because they are motivated and willing to engage me
when they are lost. Around 1/4 will stumble through the course and learn
something regardless of the format it is given in. The rest are mostly
hopeless. They lack motivation, drive, and the ability to work intellectually.

What you describe does work with motivated students and the right teacher. I
am not the right teacher for this though. Without motivated students I don't
think it works. That's the essence of the problem. Too many students don't
want to learn.

In a traditional class a student can hide their ignorance. They don't raise
their hand and don't participate. The facade of being studious and
understanding the material can be maintained. When you assign lecture watching
as the homework and then spend class time solving problems it becomes evident
who is not keeping up. It can't be hidden. This causes negative reactions in
my experience.

Here's a great lecture by Eric Mazur. It's long but well worth watching. It's
called, "Confessions of a Converted Lecturer".

<http://www.youtube.com/watch?v=WwslBPj8GgI>

~~~
planckscnst
I wonder if shorter videos would help. I know when I did video lectures, it
helped immensely to speed them up anywhere from 1.2x to 2x, depending on the
lecturer and the audio quality of the video.

Additionally, watching the video could be an assignment that counts towards a
grade. Sure they could sign in, turn on the video, and walk away, but this at
least provides additional motivation to do the first two steps, which may be
all you need to overcome the activation energy for many students.

Additionally, threats of failure help - I had a course that was considered the
filter - about 50% failed every semester. I think I was motivated by that
statistic because it ended up being just about the easiest class I had. The
motivation was less about not wanting to fail and more about showing
excellence where common people fail.

~~~
yequalsx
My videos tend to be under 20 minutes long. I try to keep them focused and to
the point. No fluff. I agree that generally, shorter is better.

Motivating them to watch is a problem for me. I'm philosophically opposed to
giving points for something a well trained monkey could do. I can't bring
myself to give points merely for pressing play on a video. I'm slowly coming
around though.

------
tres
Essentially this is a critique of lectures and traditional teaching models
which unfortunately uses Khan Academy as a pretext for discussing how lectures
don't work very well.

The tiny feedback loop that Khan Academy quizzes provide just renders the
whole point moot. And that's just one additional aspect of Khan Academy.

It's unfortunate because it's not as if the author doesn't understand that
Khan Academy is much more than just videos -- it's stated at the start of his
video. But all that is literally swept aside at the start of the video as it
focuses on the same old stuff that we've known for all too long -- teaching
via lecture doesn't work very well.

Anyone can put a video up on YouTube, but what makes Khan Academy
fundamentally different is not just the quality of the lectures, but the
structure of Khan Academy and supporting materials that go along with those
lectures.

The irony is that the author seems to be reinforcing their own idea of video
education by ignoring how Khan Academy doesn't fit within their preconceived
concepts.

~~~
aik
2 points:

1\. Having quizzes post-video is still different than having a different
teaching style. In terms of learning outcomes, the two styles could
potentially be equal, however in terms of student
motivation/engagement/interest, they are different.

2\. I'm not disagreeing with you necessarily here, but I thought the quizzes
are only for some math at the moment, and nothing else? This would mean that
at the moment his argument applies to everything except some math concepts?

~~~
tres
I should have said that I found the technique interesting and insightful.
Learning doesn't happen without engagement -- that's the key point. I just
thought it was a bit disingenuous to ignore what Khan Academy is beyond video
lectures & "worry" about it being bad for students.

Re: Point 2. Yes, I just looked over the Khan Academy knowledge map and it
does look like I was wrong; however, I'd still posit that there is more to
Khan Academy than just videos (I've talked about the "Coach" elsewhere in this
thread). But I can see the point the author is trying to make and I'm not
asserting that the method is wrong, but that he's glossing over everything
outside the video content of Khan Academy.

That said, I think the idea is a really good way to give students a better way
to engage with the subject. It adds another way for a student to engage, which
can never be a bad thing.

~~~
aik
Yeah I agree that he did gloss over some of the other Khan features.

About the method the video talks about -- I bought some audio-only lessons for
learning French once by Micheal Thomas that (for me anyway) confirms this
method. On the audio, the teacher had 2 students present, and the lessons
consisted of Michael teaching these two students. The teacher would often ask
questions and test their knowledge, and the students often would make
mistakes. Naturally he would always correct them. I found it incredibly
engaging and effective for myself.

------
slysf
The article glosses over the most important part of Khan Academy: pairing the
clear concise videos with practice/assessment. This is what engages the
student "on a deep enough level" and reinforces the lesson. The only thing
missing from the traditional classroom learning cycle is the possibility of
something being explained in a different way if the student is still having
difficulty.

~~~
shii
I think YouTube replies would be great in this case, by others (and even Sal)
using the same approach as Sal with Wacom and 10-15 min explanation of common
misconceptions and corrections.

~~~
rubergly
Sometimes I've wondered if the vision of Khan Academy executed to its fullest
would be to have a Stack Overflow-like up-/down-voting system for user-
submitted videos. Khan Academy is amazing (note: I'll be interning with them
this summer, and I'm SUPER excited), but it's never going to be accessible for
_everyone_ as long as it's just one teacher. I think Khan is a great teacher,
but there are always going to be at least a couple people who are confused by
wording in an unforeseen way. If everyone posts videos, and the best teachers'
videos get pushed up to the top, then it's bound to round out the experience
and help people who might not like Khan's personal style. This idea sounds
good in theory (kind of), but it would never work in practice since videos are
much more time-consuming to make and watch, and so could not easily be
digested and voted on by the masses. Sure, there's the potential that if you
open up the teaching to the masses that some really excellent material could
rise to the top, but it would also be drowned out in the noise of countless
sub-par teachers. With Khan producing all the videos, everything is guaranteed
to be high quality.

Sidebar aside, the site does have a place for users to ask/answer questions
below each video. What I realized while briefly mentioning Stack Overflow
above is that it would be _amazing_ if that kind of rapid-answer community
could be built up around these questions.

~~~
yequalsx
I've wanted to create such a site for many years now. My focus is on community
college level mathematics. My goal is to make it topic based.

My college's administration is a bit conservative when it comes to new ideas.
I think you are wrong about not having enough videos though. More and more
college faculty are creating their own videos. What is lacking is a central
storage location for these videos. I think people would be willing to upload
their videos and make them publicly available.

I could, over time, what evolves is a book. In math this would work because at
the beginning levels the knowledge is static. How to factor trinomials is not
ever going to change. I think a Stackoverflow site would work for lecture
videos.

------
dvse
Regarding "pseudoteaching" mentioned in the article - every great teacher is
to some degree a con artist. They know that in the end it is up to the student
to engage with the material - the only thing they can do as a teacher is to
communicate their excitement about the subject and provide a point of view
that would make the material easier to absorb, at least to some. Their goal is
never to reach everyone yet everyone leaving their lectures feels like they
have been made a party to some exciting secret.

~~~
pjscott
I had a remarkably effective teacher who said as much, quite emphatically. I
don't think most of the students actually believed him, writing it off as
bluster. It was true, though, and this idea permeated his whole approach to
teaching. It worked, too.

Years later, I still find this hilarious.

------
billpaetzke
Khan Academy and other video teachers need to have lots of problems for each
lesson for the students to solve. We learn best by doing.

For example, I remember my Calc II professor said that when he was in college,
he'd do every single problem in the book for that lesson--even though only
half of the problems were assigned (the ones w/o answers in the back). And
look where he ended up: as an expert in calculus.

Also, I'm going through Stanford's CS193p (iOS dev) on iTunesU right now in my
free time after work. Probably 80% of my learning happens by completing the
assignments.

~~~
archgoon
>Also, I'm going through Stanford's CS193p (iOS dev) on iTunesU right now in
my free time after work. Probably 80% of my learning happens by completing the
assignments.

Are these the assignments you used? Or are the ones on iTunesU different?

[http://www.stanford.edu/class/cs193p/cgi-
bin/drupal/download...](http://www.stanford.edu/class/cs193p/cgi-
bin/drupal/downloads-2010-fall)

~~~
billpaetzke
That page has the assignments, lecture notes, and source code from the lecture
demos. The lecture videos are hosted on iTunesU.

------
makeramen
I think if the students were told what they got _wrong_ on the first exam,
they would have easily done a lot better. Like some of the other commenters
stated, constant assessment is key. Once people start to figure out that some
of their most common conceptions are misconceptions, they'll start to question
everything, and then the learning really begins.

------
dpatru
The biggest flaw in tnis is that the OP criticizes khanacademy videos by
showing that vastly inferior videos don't work. Khan videos are good b/c they
make it easy to think: there are no distractions, just a blackboard and an
intelligent voice "in your head" guiding you through the problem domain. The
OP, instead of using a khan video or even a khan-like video, tries to teach
physics using a video shot outdoors featuring the face of a talking girl with
a guy juggling in the background. He might as well give his subjects a physics
book and turn them loose in a sports bar to study while following ncaa
basketball. Neither method is a good one for learning physics. What's sad is
that thd OP is not just some crank on the internet, but apparently a newly
minted PhD in education who will likely end up teaching teachers or running a
school system.

------
krosaen
Illuminating. Seems that the academy's recent focus on mastery and testing is
of utmost importance, at least that way students who watch the videos but do
not absorb the material will realize they don't actually "get it" when they
can't answer the questions. The key will be in devising problem sets that
truly require mastery and not just plug and chug or rote memorization.

------
kmfrk
One of my worst experiences in college is to have a lecturer run through some
gnarly theory, after which you get the impression that you understand it.

Then he goes "whoops, I made an error just now", and everything has to be
reassessed.

------
citizenkeys
Interesting video. The theory the article demonstrates through its video is
that people don't pay attention unless you begin with the popular
misconception. Otherwise, people automatically believe they know what you're
going to say.

~~~
muhuk
I agree that it would help people with those misconceptions learn. But I think
it would be a waste of time to listen to all possible wrongs if you are eager
to learn and adequately knowledgeable about a particular subject. Maybe
entertaining for the first few times, but inefficient in the long run.

