Hacker News new | past | comments | ask | show | jobs | submit login

I wish the authors — or any authors — studied which courses actually deliver results as opposed to what they ought to deliver.

For instance, immediate feedback ought to deliver learning. So they looked at which courses give immediate feedback.

But it would be even better if somone looked at which learners actually learned. You took the course now build xyz. Or pass a test or something.

There might be a difference in the sorts of “immediate feedback” that are effective. It’s an indirect proxy of what you are after.

Author of the paper being discussed here. I'd love to study what students actually learn too! As we note in the paper, the analytical approach is a limited, indirect proxy.

Unfortunately, as others have noted, doing this is intractably hard. We've reached out to dozens of coding tutorial companies, but none want to share their visitors' contact information (understandably). We've tried contacting students on our campus and others to find people who've used tutorials, but few have. If you have ideas about how to contact tutorial users, please share!

The even harder problem is measuring learning. There are essentially no reliable, valid measures of any knowledge of programming (exams in courses are mostly unreliable, invalid measures). It's something we're working on in my lab, but it will take years, as it has in math and physics education.

Andy Ko Associate Professor University of Washington http://faculty.washington.edu/ajko

I love that the author of the paper responded to this. Thank you HN, and Andy.

I suspected it was hard or else, I assume, you would have done it. But I thought it was hard because getting everyone to take a standardized test before and after the courses would be hard, not because even the contact information was not available!

Also, what do you mean exams are unreliable? Is there something you can point me to. I did not know that was the case.

Indeed it would, but it's probably really difficult to research this, as you need a large population, and there will be plenty of external factors acting on those populations. You'd ideally want to use students with the same (or similar) age, and similar ability levels. Where do you find a suitably large number of those from? Then you'd need to remove any external influences - the effect of a good or bad teacher, for example.

That's not to say it can't be done, but is a much larger and more expensive piece of research.

Excellent idea. However, I think the main reason they don't review by that aspect is because there simply aren't any of those types of courses. The courses coming out of the traditional universities (on edx or coursera) seem to be very watered down versions of real university courses. For the free online, independently created courses they seem to be catering to the "immediate stimulation" crowd rather than the small group of people who want truly comprehensive online courses.

I agree though that project-based learning where the student takes an active role in the creation and development of projects is crucial. If you think about it, when the student enters the real world/work force, everything they do will be focused around projects of various size.

Courses are usually tied to revenue. (Some) post-secondary institutions can run like small countries. Other schools are hopefully starting to take the above approach and stay that way.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact