
A Year of Spaced Repetition Software in the Classroom - eurg
http://lesswrong.com/r/discussion/lw/mfm/a_year_of_spaced_repetition_software_in_the/
======
1123581321
I used to be very excited about spaced repetition, but haven't taken it as
seriously in several years. Yes, it is superior to more ordinary methods of
quizzing vocabulary, and I would still use it if I had to pass a
vocabulary/fact test on a subject I didn't care about.

As a method of developing real competency in a subject or practice, though, it
offers none of the benefits of immersion as it keeps the student just on the
edge of full engagement (and if we add immersion, we lose the time-savings
that spaced repetition offers.) It introduces a fragility into the student's
schedule that make lapses in 'discipline' more likely (not everyone has a
teacher and regular subject periods to keep them on task.) For anything beyond
basic key-value learning, it requires too many contortions and restatements of
the subject -- to fit the format -- to be feasible for many students/subjects.
The packaged decks available online help but usually don't meet the needs of
the specific course or subject concentration the student requires. We can
introduce variability to space how to solve a problem instead of a 'what'
answer, but then we need to consider muscle memory, developing intuition and
other things that the spaced repetition model does not address.

~~~
grovulent
I disagree with this in part. I'm using spaced repetition to learn maths - and
without it I just wouldn't progress at all. My natural aptitude isn't great.

Can you say a little bit more about what you mean by immersion? Do you mean -
say, having a teacher who can show you a problem from multiple angles,
allowing you to triangulate your understanding by asking numerous questions...
etc? Things like this?

I do agree that in terms of developing immediate understanding - having access
to this environment allows you to obtain understanding quicker. And I agree
that it doesn't cohere well with spaced repetition learning.

However, the problematic thing with learning with this way is that unless you
go on to use it daily, you lose almost all of it. So all you manage to do is
pass tests throughout your life - never developing a full suite of knowledge
that you can deploy. What an enormous waste of time!

Given that the energy needed to learn/recall an individual card drops
exponentially over time, over the long term this is a vastly more efficient
process for retaining information.

The only question is whether it can be used to learn complex and abstract
ideas. I believe it can.

Here's how I typically progress with learning a difficult, abstract
mathematical concept or problem.

First pass - I'll rote memorise a solution.

Then after a couple of weeks - I'll typically forget various parts of the
solution. More often than not these will be sub-problems that I don't
completely understand. So I'll create new cards that will provide training on
these smaller sub-tasks. In the meantime, I'll rote learn the solution to the
larger problem again.

What I find is that no matter how many times I rote learn a complex solution,
eventually I typically forget it. But the smaller, easier concepts stick.
Eventually my rote knowledge gets swapped out progressively by the smaller
units of understanding and I end up being able to work through the solution
without remembering it as a whole at all.

This is a frustratingly slow process in the beginning - particularly if you
are starting at the very beginning of a new field (as I am with maths). But in
my experience it's worth it. For the first time in my life I feel I'm actually
really learning a topic in a way that I'll truly own.

~~~
voltagex_
Do you have an example of some cards you're using? Was there an existing set
or did you develop them yourself?

~~~
grovulent
I developed them myself.

Mostly this doesn't add much overhead to the learning process. As I'm working
through a math's text book (in ebook or pdf form) I use the windows 7 snipping
tool to create two images - one of the question and solution. And then I just
insert these into my card program. This takes about 15-30 seconds at the most.

I use mnemosyne as my repetition software which allows me to insert images -
and I can also annotate my cards with latex. I have a number of latex macros
set up on my keyboard so that I've become pretty quick at inputting latex.

I continue to annotate my cards as I progress. For a complex problem, what
starts out as a simple cut and paste from a text book grows into a voluminous
set of notes covering every aspect of the problem that I've had trouble with
on different repetitions of the problem. These notes become what I use to
create the smaller cards.

For abstract and complex material - there is no substitute for creating your
own cards imo. You need to read through a text book anyway to ensure that
there is no important context that you've missed. And it isn't much overhead
to create your base set of cards, cutting and pasting as you go. But your gaps
in understanding are your own - and only you can identify those and fill them.

------
pella
classics:

Sivers: "Memorizing a programming language using spaced repetition software" (
2013)

[https://sivers.org/srs](https://sivers.org/srs) ( HN:
[https://news.ycombinator.com/item?id=5015183](https://news.ycombinator.com/item?id=5015183)
)

\---

Janki Method: Using spaced repetition systems to learn and retain technical
knowledge.

[http://www.jackkinsella.ie/2011/12/05/janki-
method.html](http://www.jackkinsella.ie/2011/12/05/janki-method.html)

[https://www.oxbridgenotes.com/articles/janki_method_refined](https://www.oxbridgenotes.com/articles/janki_method_refined)

\---

Gwern: SPACED REPETITION :
[http://www.gwern.net/Spaced%20repetition](http://www.gwern.net/Spaced%20repetition)

~~~
pcote
At the last programming job I was fired from, I was deemed to be a slow and
all around poor performer. About a year ago, I started doing my own version of
Jack Kinsella's Janki method. As a result, I've leveled up to what some call a
"full stack" web engineer.

SRS is a fun way to learn or else I wouldn't have kept at it so long. I would
even go as far as saying it's made my personal projects a little bit easier
for me to execute on. However.....

What I don't know is how applicable this is in the "real world". SRS systems
don't translate well in the "Learn-in-5-days-or-die" approaches to IT
training. As for benefits for work-related development and maintenance, I
don't have a clue one way or the other.

~~~
asimilator
I'm surprised that memory or memorization would be the key (for some people?
for yourself?) to learn a programming language.

I've studied a number of different subjects in my life, from foreign languages
to computer science to math, etc. I clearly see the benefit of SRS and
necessity of memorization in foreign languages (vocab is so critical, and
there's really no way to avoid straight memorization of it), but I don't see
how you could effectively encode your knowledge of a programming language on
to flash cards. And I'm not sure why you'd want to when you can easily look up
things in the documentation or on Stackoverflow.

~~~
pcote
Like with anything, there's a time to practice a craft and a time to perform
it. All SRS does is optimize those practice sessions. It's still up to me to
go beyond that and work the projects.

Stackoverflow and documentation serve to complement my Anki deck. I learn what
I need and apply it to my project. After that, Anki cards get created.

Ultimately, I just see "stuff that you can google" as supplements to memory
rather than replacements for it.

~~~
asimilator
> All SRS does is optimize those practice sessions.

I think I'm struggling to see how flash cards would be an effective practice
session. Why not just code something? Start a hobby project, contribute to
something open source, etc. You'll use everything you'd rote memorize in
context (and naturally start remembering it).

~~~
pcote
Example from something I added today....

\-------------------------------------------------- Front: Setup a flask view
that requires basic authentication. Use the httpauth extension.

Back: How did it go? \--------------------------------------------------

I have a lot of cards like this where I put the onus on myself to code up my
own samplers.

My hobby is one where I'm playing around with Magic the Gathering data in JSON
format. Over the past week, I designed a database, wrote SQLAlchemy scripts to
populate it, and built up some RESTful web service views over it in Flask.
It's been a lot of fun.

There'a a natural harmony where my performance on Anki reviews boosts my
project work and vice versa. If there's a "key thing" to it all, it's that I
have a project flow that works for me and makes me happy with the whole
process.

~~~
barefootford
Yep, exactly. The majority of the cards I do are either a problem where I
convert a sentence from English to X, or it's create a function that takes in
these parameters and returns this. Most cards should be challenges, not just
quick one word answers.

------
cJ0th
I overestimated the effect of spaced repetition. For six months, I used Anki
every day to recap Chinese characters. At the end of the term I was perfectly
prepared for the test. Afterwards, I didn't study my flash cards again. Only a
few weeks later I found out that I've forgotten most characters or at least
some of their details. I expected that after six months of daily practice
(about 20 minutes per day) I would have stored at least some characters in my
long term memory.

~~~
hiq
Anki is great to _help_ learning languages, however I think you have to make
some special effort to actually use the words (in full sentences) when you
learn them if you want to be able to reuse them in the future. One-to-one
flash cards only helps for basic information retention. For example I find
kanas much easier to learn than kanjis, as the former translate perfectly as
plain syllables in western languages, while the latter can be far more subtle.

Also after six months, if you're successful in your reviews, Anki should not
expect you to review your (old) cards before several weeks, so it's strange
that you forgot most of them for your test if it occurred in that range. It
can also depends on the type of test I guess, how was it?

~~~
mjklin
> The kanjis are far more subtle

In other words, they have to be memorized. No connection between writing and
pronunciation.

~~~
maehwasu
This is factually incorrect for the Chinese readings of kanji/hanja/hanzi.

------
roflmyeggo
As someone who has dabbled in both biology/chemistry and now computer science
academically, my thoughts on flash cards and spaced repetition software has
changed over the years.

On one side you have the biology/chemistry guys who tend to use a lot of
flashcards, and have a reputation for "just memorizing" facts. Then on the
other end you have the physics/computer science guys who often refer to
themselves as "lazy" for lack of memorizing and have a reputation for only
focusing on conceptual ideas.

It seems to me that the people who succeed utilize a mixture of both
approaches ("memorization" and "conceptual understanding"). I'm not sure when
these two approaches suddenly became mutually exclusive.

As an example, although there were some classes in Biology that required a
heavy amount of memorization, without the flash cards it would have been very
hard for me to make the conceptual connections during lectures, while reading,
etc.

Knowledge and understanding is kind of like a spider web, the more you have,
the more you are able to grasp.

Purely focusing on conceptualizing the topics at hand has never worked out as
well as I had hoped.

~~~
tfigment
The only class I ever used flash cards was in Organic Chemistry. I do not
think they helped long term memory at all as I recall next to nothing from
that. If anything the act of creating the cards helped more than anything. I
think this might be where the joining of the two sort of happens as you
indicate.

To further that point, during exams some classes allowed 1 sheet of paper with
anything you can put on it. I created some super dense cheat sheets in my time
and I am fairly sure that I never referenced them but once or twice in 4 years
of school. But again the active of creating them and sorting out important and
unimportant information was key in solidifying knowledge for the long term. If
someone handed me those cards or sheet they would have been useless to me in
the long run is at least how I see it.

The other thing I learned in school was anything I crammed for I knew I was
not going to learn. It may have been enough to get through the next couple of
days but it was not going to stick. I took some summer school courses and
found it to be a waste as it was too much like cramming as it was so
condensed.

I guess I did well in school by using something like SRS which was allowing
time between study sessions for information to settle and allow subsequent
review after time. There is probably more than one reason for university
courses to be on alternating days as it probably facilitates this kind of
thinking. The only thing is that SRS seems to be focused on much shorter time
periods than a day or 2.

~~~
chucksmash
What you say about the act of making the cards themselves rings true to me.

When I was preparing for the GRE in 2010 I first wrote down all the words and
definitions from both the Princeton Review and Kaplan GRE vocab books on a
legal pad. That didn't work as well as I'd hoped for quizzing (it's a pain to
try not to read 90% of the page) so I wrote a really basic data entry script
in Python that I used to input the words/defs and then a second one that would
generate random questions based off of them (definition + 4 words as MC).

By the time I was done with typing them all in and debugging my little quiz
script, I'd say I already had active recall on >90% of them. Having that
working quizzing tool at the end to practice recognition was just icing on the
cake - creating it in the first place was more useful to me.

------
hiq
For those interested in memory and learning and who want some hints on why SRS
work, Make It Stick[1] is probably a must-read.

[1] [http://makeitstick.net/](http://makeitstick.net/)

~~~
roflmyeggo
Great book. Interleaving contexts, spaced repetition, etc. - all great
concepts.

The most important takeaway for me was accepting that the tougher something is
to remember, the greater the benefit will be when we actually remember it.

------
qznc
Sounds like an interesting startup idea: SRS for classrooms

Probably not a millionaire-in-5-years or one-man-lifestyle business. More of a
improve-the-world kind.

------
slaypni
Last year, I have developed a JS library SM.js
([https://github.com/slaypni/SM-15](https://github.com/slaypni/SM-15)) which
is based on the algorithm used in the latest SuperMemo.

I have also developed a web service
([http://slaypni.github.io/flashward/](http://slaypni.github.io/flashward/))
for memorizing Foreign words using the library. Through the experience of
using it by myself, I felt that spaced repetition helped me to remember many
words for learning English.

I am feeling that most (free) SRS are using relatively simple algorithms (for
example, Anki is based on SM-2 which is developed in the late1980s). Although
I am not sure SM-15 is better enough, there could be some room to improve it.
It is nice if the library was helpful for some developers who create new SRS.

~~~
wahnfrieden
Newer isn't necessarily better with SuperMemo algorithms. They introduced
tremendous complexity without strong evidence of increased effectiveness, last
I looked into it - it seems to create more corner cases to manage, if
anything.

I'd be interested in materials I may be missing.

------
patrickmclaren
A couple months ago I put together a spaced repetition app
([https://www.memoread.io/](https://www.memoread.io/)) aimed at less tech
savvy students. At the time it didn't meet much success, although I may
revisit it in the future.

------
cthalupa
I've been using SRS to learn 日本語 for some time now. Prior to this, my ability
to remember kanji was basically nil. I could spend tons of time writing it
over and over again, and nearly immediately forget how to write it, and
shortly after, forget how to read it.

I switched to using Anki, and the reading at least became quite cemented - I
still struggle with writing at times, but it's a significant improvement over
what it was previously.

Daily study is hugely important - if I go out and have a crazy weekend, I hate
myself Monday, and my study session becomes 3 or 4 times longer than it would
otherwise.

~~~
MichaelGG
Have you looked at "Remembering the Kanji"? I was skeptical when reading
"Remebering the Kana", but halfway through it I found that despite the
silliness, it worked. I tried the Kanji book for a bit and was fairly
surprised how well it was working.

~~~
cthalupa
I'm not a huge fan of the Heisig method. I started off with it, but I realized
that a lot of the time, the key word I was learning for the kanji had
absolutely nothing to do with any meaning or reading for it.

What I currently do is learn new kanji via Kanji in Context, and then create
cards for them in my anki deck as I have learned them.

~~~
brokencup
You might want to look into "Kanji ABC" by Foerster & Tamura. It was developed
some years after Heisig and avoids using nonsensical keywords. I think their
approach is better than Heisig if you already have familiarity with a decent
number of kanji. The book is out of print so it may be a little tough to get a
hold of (I found mine online at Powell's used bookstore). You can see an
online version at work[1], but it isn't that useful on its own without the
book.

[1] [http://www.kanjiabc.net/](http://www.kanjiabc.net/)

------
bane
There's two major parts to learning a language: recognition and production.
SRS can be quite good as a tool to aid in recognition, but I've found it to be
lousy for production.

This is a classic problem in language pedagogy.

Some old comment of mine from almost a year ago
[https://news.ycombinator.com/item?id=8510165](https://news.ycombinator.com/item?id=8510165)

[https://news.ycombinator.com/item?id=8251569](https://news.ycombinator.com/item?id=8251569)

[https://news.ycombinator.com/item?id=8253351](https://news.ycombinator.com/item?id=8253351)

I think "completing the circuit" is among the most important aspect of
educational theory. Recognize and produce needs to happen in _any_ subject,
and a good student recognizes that SRS is but one tool for recognition, but
others are needed to complete that, and production is an entirely different
side of the coin.

~~~
sxyuan
> Where my bright students might have been used to high Bs and low As on
> tests, they were now breaking my scales. You could see it in the multiple
> choice, but it was most obvious in their writing: they were skillfully
> working in terminology at an unprecedented rate, and making way more
> attempts to use new vocabulary—attempts that were, for the most part,
> successful.

> Given the seemingly objective nature of Anki it might seem counterintuitive
> that the benefits would be more obvious in writing than in multiple choice,
> but it actually makes sense when I consider that even without SRS these
> students probably would have known the terms and the vocab well enough to
> get multiple choice questions right, but might have lacked the confidence to
> use them on their own initiative. Anki gave them that extra confidence.

Not that you're wrong, but the author also points out here that SRS can help
production by reinforcing the student's knowledge and boosting their
confidence in what they've learned when it comes to using it.

