Working for the Khan Academy, I've seen the link to this video a bunch of times. Naturally, I was really curious about the voracity of the claims, so I did some research. It's actually worth reading the thesis as it's pretty interesting stuff. It's here: http://www.physics.usyd.edu.au/pdfs/research/super/PhD(Mulle...
The study was, as best I can tell after reading through much of it (a lot not relevant to this discussion), quite limited. The largest number of students considered for any of the experiments: 370. All from one school. Most of the other experiments included far fewer students (fewer than 100).
Chapter 9 (additional experiments in chapter 10), and the section on Participants and Design has most of the relevant information. I was particularly fascinated by the decision to remove students from the final result for "watching more than one multimedia treatment." I mean I understand why it's important to control for this, but part of the point of KA is to provide additional treatments of information.
I am not saying that this in any way invalidates the research, but I also don't think it's fair to say that this is somehow a surefire tested solution that is being handed over. In addition, because the research hasn't been repeated outside of the original setting -- at least I couldn't find it, please correct me here if I'm wrong -- it makes the claim that "because introducing misconceptions was effective in this study means it must be effective in all science videos" a pretty serious stretch.
I'm all for research. We're doing a lot of it right now with our pilot students/classrooms and existing users. But just because something has been research-tested doesn't mean it's automatically the right thing to implement for KA or its students.
The study was, as best I can tell after reading through much of it (a lot not relevant to this discussion), quite limited. The largest number of students considered for any of the experiments: 370. All from one school. Most of the other experiments included far fewer students (fewer than 100).
Chapter 9 (additional experiments in chapter 10), and the section on Participants and Design has most of the relevant information. I was particularly fascinated by the decision to remove students from the final result for "watching more than one multimedia treatment." I mean I understand why it's important to control for this, but part of the point of KA is to provide additional treatments of information.
I am not saying that this in any way invalidates the research, but I also don't think it's fair to say that this is somehow a surefire tested solution that is being handed over. In addition, because the research hasn't been repeated outside of the original setting -- at least I couldn't find it, please correct me here if I'm wrong -- it makes the claim that "because introducing misconceptions was effective in this study means it must be effective in all science videos" a pretty serious stretch.
I'm all for research. We're doing a lot of it right now with our pilot students/classrooms and existing users. But just because something has been research-tested doesn't mean it's automatically the right thing to implement for KA or its students.