I used to be very excited about spaced repetition, but haven't taken it as seriously in several years. Yes, it is superior to more ordinary methods of quizzing vocabulary, and I would still use it if I had to pass a vocabulary/fact test on a subject I didn't care about.
As a method of developing real competency in a subject or practice, though, it offers none of the benefits of immersion as it keeps the student just on the edge of full engagement (and if we add immersion, we lose the time-savings that spaced repetition offers.) It introduces a fragility into the student's schedule that make lapses in 'discipline' more likely (not everyone has a teacher and regular subject periods to keep them on task.) For anything beyond basic key-value learning, it requires too many contortions and restatements of the subject -- to fit the format -- to be feasible for many students/subjects. The packaged decks available online help but usually don't meet the needs of the specific course or subject concentration the student requires. We can introduce variability to space how to solve a problem instead of a 'what' answer, but then we need to consider muscle memory, developing intuition and other things that the spaced repetition model does not address.
This is why SRS is a tool but not a complete solution. In medical school, there is such an enormous amount of material to commit to memory (specifically for board exams) that a lot of students find SRS very helpful. It is possible (I did this for a little bit in the beginning) to rely on it too much and therefore not be able to make the connections needed, be it on an exam or in real life. There's an art to making good flashcards- knowing which context to test a fact in, how to give enough context but not too much, etc. Also, there is little point to memorizing cards without understanding the material first. If I struggle with a card, I know I need to review the material, develop a mnemonic, etc.
I disagree with this in part. I'm using spaced repetition to learn maths - and without it I just wouldn't progress at all. My natural aptitude isn't great.
Can you say a little bit more about what you mean by immersion? Do you mean - say, having a teacher who can show you a problem from multiple angles, allowing you to triangulate your understanding by asking numerous questions... etc? Things like this?
I do agree that in terms of developing immediate understanding - having access to this environment allows you to obtain understanding quicker. And I agree that it doesn't cohere well with spaced repetition learning.
However, the problematic thing with learning with this way is that unless you go on to use it daily, you lose almost all of it. So all you manage to do is pass tests throughout your life - never developing a full suite of knowledge that you can deploy. What an enormous waste of time!
Given that the energy needed to learn/recall an individual card drops exponentially over time, over the long term this is a vastly more efficient process for retaining information.
The only question is whether it can be used to learn complex and abstract ideas. I believe it can.
Here's how I typically progress with learning a difficult, abstract mathematical concept or problem.
First pass - I'll rote memorise a solution.
Then after a couple of weeks - I'll typically forget various parts of the solution. More often than not these will be sub-problems that I don't completely understand. So I'll create new cards that will provide training on these smaller sub-tasks. In the meantime, I'll rote learn the solution to the larger problem again.
What I find is that no matter how many times I rote learn a complex solution, eventually I typically forget it. But the smaller, easier concepts stick. Eventually my rote knowledge gets swapped out progressively by the smaller units of understanding and I end up being able to work through the solution without remembering it as a whole at all.
This is a frustratingly slow process in the beginning - particularly if you are starting at the very beginning of a new field (as I am with maths). But in my experience it's worth it. For the first time in my life I feel I'm actually really learning a topic in a way that I'll truly own.
Mostly this doesn't add much overhead to the learning process. As I'm working through a math's text book (in ebook or pdf form) I use the windows 7 snipping tool to create two images - one of the question and solution. And then I just insert these into my card program. This takes about 15-30 seconds at the most.
I use mnemosyne as my repetition software which allows me to insert images - and I can also annotate my cards with latex. I have a number of latex macros set up on my keyboard so that I've become pretty quick at inputting latex.
I continue to annotate my cards as I progress. For a complex problem, what starts out as a simple cut and paste from a text book grows into a voluminous set of notes covering every aspect of the problem that I've had trouble with on different repetitions of the problem. These notes become what I use to create the smaller cards.
For abstract and complex material - there is no substitute for creating your own cards imo. You need to read through a text book anyway to ensure that there is no important context that you've missed. And it isn't much overhead to create your base set of cards, cutting and pasting as you go. But your gaps in understanding are your own - and only you can identify those and fill them.
I've found SRS (with Supermemo) is indispensable for learning vocabulary of languages in which immersion is not possible or too impractical e.g. ancient Greek/Latin.
As you say, it requires discipline; but if something is important you'll find the time, if not you'll find an excuse.
At the last programming job I was fired from, I was deemed to be a slow and all around poor performer. About a year ago, I started doing my own version of Jack Kinsella's Janki method. As a result, I've leveled up to what some call a "full stack" web engineer.
SRS is a fun way to learn or else I wouldn't have kept at it so long. I would even go as far as saying it's made my personal projects a little bit easier for me to execute on. However.....
What I don't know is how applicable this is in the "real world". SRS systems don't translate well in the "Learn-in-5-days-or-die" approaches to IT training. As for benefits for work-related development and maintenance, I don't have a clue one way or the other.
I'm surprised that memory or memorization would be the key (for some people? for yourself?) to learn a programming language.
I've studied a number of different subjects in my life, from foreign languages to computer science to math, etc. I clearly see the benefit of SRS and necessity of memorization in foreign languages (vocab is so critical, and there's really no way to avoid straight memorization of it), but I don't see how you could effectively encode your knowledge of a programming language on to flash cards. And I'm not sure why you'd want to when you can easily look up things in the documentation or on Stackoverflow.
> And I'm not sure why you'd want to when you can easily look up things in the documentation or on Stackoverflow.
Why would you memorise foreign vocab when you could just look up what you want to say in a dictionary or phrasebook?
It's a silly analogy, but there's some merit in the comparison. Searching documentation takes time - perhaps not that long, but enough that if you had to do it constantly you'd become frustrated very quickly. If you look at some code you wrote recently, it's almost certain that there's a bunch of standard library function calls in there that you didn't have to think twice about writing. The same goes for basic programming patterns. The more you expand your repertoire of memorised functions and patterns, the easier it becomes to go from design to code.
Right, very much so, because a live conversation using vocab is very much different than anything you'd do while programming.
> The more you expand your repertoire of memorised functions and patterns, the easier it becomes to go from design to code.
Yeah sure, but the criticality is much less, isn't it? Like it's not the rote memorization of function calls, or standard libraries that makes a good programmer. Design patterns, idiomatic language structures (i.e. how do I write truly "Pythonic" code), etc are all way more important than, say, what does library X provide or what are the args to function Y.
Flash cards don't address the important parts of learning a programming language. And if you focus on the important parts, generally you'll get enough experience in the language that the things you'd want to memorize naturally just stick.
I don't think it's a particularly a silly analogy. You're going to be able to work better if you know what you're reaching for rather than googling around.
Idioms and design patterns are also very well suited to flash cards.
I think very few programmers used memorization techniques to learn a new language. When learning your first language (or one very different from those you know), understanding the ideas is much harder than remembering the facts, and when learning similar languages you start by coding something you know using syntax cheat sheets and Google, and by the time it works you remember the basics.
If I had to switch between perl, python, ruby, php, lua, javascript all the time, I would consider Anki to keep the rosetta stone data in memory, but not otherwise.
Like with anything, there's a time to practice a craft and a time to perform it. All SRS does is optimize those practice sessions. It's still up to me to go beyond that and work the projects.
Stackoverflow and documentation serve to complement my Anki deck. I learn what I need and apply it to my project. After that, Anki cards get created.
Ultimately, I just see "stuff that you can google" as supplements to memory rather than replacements for it.
> All SRS does is optimize those practice sessions.
I think I'm struggling to see how flash cards would be an effective practice session. Why not just code something? Start a hobby project, contribute to something open source, etc. You'll use everything you'd rote memorize in context (and naturally start remembering it).
--------------------------------------------------
Front:
Setup a flask view that requires basic authentication. Use the httpauth extension.
Back:
How did it go?
--------------------------------------------------
I have a lot of cards like this where I put the onus on myself to code up my own samplers.
My hobby is one where I'm playing around with Magic the Gathering data in JSON format. Over the past week, I designed a database, wrote SQLAlchemy scripts to populate it, and built up some RESTful web service views over it in Flask. It's been a lot of fun.
There'a a natural harmony where my performance on Anki reviews boosts my project work and vice versa. If there's a "key thing" to it all, it's that I have a project flow that works for me and makes me happy with the whole process.
Yep, exactly. The majority of the cards I do are either a problem where I convert a sentence from English to X, or it's create a function that takes in these parameters and returns this. Most cards should be challenges, not just quick one word answers.
Because "just coding something" won't cover every method/function idiom you've ever learned. Creating a card with a corresponding challenge can do that. If you just make a Ruby script to practice programming you're allowing yourself to forget another language or framework knowledge.
Memorization is definitely useful for programmers. It means you will spend less time browsing API documentation looking up existing functions applicable to your particular problem. If you program a lot in a particular language or API, a little bit of rote memorization saves a lot of time in the long run.
Spaced repetition relies on reviewing items before you forget them, and works best with active recall, not simply reading answers. Reading the docs whenever you forget is much less effective.
"...I don't see how you could effectively encode your knowledge of
a programming language on to flash cards."
I've occasionally wondered if an Anki deck composed of Clojure idioms would be valuable. It would be dependent on paredit conventions, teaching "shapes" along with symbols.
I have a Clojure Anki deck of function-name -> the output of (doc function).
The reason I started it was because I just kept forgetting the names of functions. I'd know exactly what I wanted the code to do, but then I would find myself not remembering the which word to invoke to do it. The flash cards have helped a lot there.
I'm not sure how you would fit an idiom into a flash card, but try it out and see how it goes.
I overestimated the effect of spaced repetition. For six months, I used Anki every day to recap Chinese characters. At the end of the term I was perfectly prepared for the test. Afterwards, I didn't study my flash cards again. Only a few weeks later I found out that I've forgotten most characters or at least some of their details. I expected that after six months of daily practice (about 20 minutes per day) I would have stored at least some characters in my long term memory.
I began my study of Chinese in 1975. I got to be good enough in reading Chinese to set the ceiling of scores obtained in an experimental administration of a test for Chinese language for speakers of English. (The test was not further developed by the test developers, but I saw the summary of norming administration scores.) I worked for quite a few years as a Chinese-English interpreter for government-sponsored people-to-people visits to the United States. I was just reading Chinese-language newspapers via Google News just before bopping over here to Hacker News.
Simply put, the biggest single mistake learners of Chinese make in learning Chinese characters is learning them only in isolation and not also by reading connected Chinese-language text in which Chinese characters appear in "compounds" (meaningful Chinese words). The full argument for doing this is developed by the late John DeFrancis (a brilliant language teacher and textbook author) in the front matter of his book Beginning Chinese Reader,[1] which is still a very worthwhile book decades after it was first published. DeFrancis built in a lot of spaced repetition in the book lessons, back in the days before "spaced repetition" was a trendy term, but he crucially also makes sure the lesson texts illustrate the various ways that Chinese characters enter into compounds to form high-frequency words in the modern Chinese vocabulary. Try it if you'd like to more Chinese in the most efficient way possible.
P.S. for a more extensive discussion of how to learn natural human languages effectively, see a comment of mine from about two years ago that was very popular with other readers of HN:
I'm really glad to see such an encouraging and useful comment on HN.
I'm right in the middle of the John DeFrancis series (just started Intermediate Chinese Reader volume 2). These books seem to be almost forgotten and I haven't found any modern equivalent, so I was starting to feel slightly lonely in my quest for Chinese reading fluency :) I started them after trying
* regular flashcards: not very effective for me, as all the accumulated knowledge would disappear very quickly after I stopped practicing
* "modern" graded readers: they're nice, but far less structured than the DeFrancis' series, and they often stop at a beginner/intermediate level.
What's truly unique about the DeFrancis' series compared to other graded readers is that the texts are adapted to the vocabulary lists, rather than the other way around. Therefore characters and compounds can be introduced logically, in a coherent manner: for example each new character comes with a cluster of words that use this character, making the whole much easier to remember. The downside is that the texts often feel contrived and boring, but if you're serious about learning to read it feels much more effective and gratifying on the long term. The approach used by most other readers is to start from a simple and enjoyable story, and then adding a glossary containing the more advanced words in the story. But although the story itself is more natural and enjoyable for the reader, the vocabulary itself is much harder to remember.
It would be awesome if someone was willing to put in all the hard work it took to craft the DeFrancis' series, and write something similar with a more modern and less academia-focused content.
I used Pimsleur to learn Japanese. They use a kind of SRS where words don't get quizzed directly, but embedded in small phrases. Using words in context made me more sure of myself and I was immediately able to use them in conversations or understand them in videos. With each phrase more than one concept could be refreshed. I think it is essential to put small phrases in order to develop the vocabulary.
Also, for absolute beginners this method has the advantage of forming a solid understanding of the core of the language (those 1000 words that see over 50% usage in regular conversation) and also learn accent really well - cards being in audio format. It feels as if I were a small child learning his native language.
Anki is great to help learning languages, however I think you have to make some special effort to actually use the words (in full sentences) when you learn them if you want to be able to reuse them in the future. One-to-one flash cards only helps for basic information retention. For example I find kanas much easier to learn than kanjis, as the former translate perfectly as plain syllables in western languages, while the latter can be far more subtle.
Also after six months, if you're successful in your reviews, Anki should not expect you to review your (old) cards before several weeks, so it's strange that you forgot most of them for your test if it occurred in that range. It can also depends on the type of test I guess, how was it?
> Also after six months, if you're successful in your reviews, Anki should not expect you to review your (old) cards before several weeks, so it's strange that you forgot most of them for your test if it occurred in that range.
The test went well. It was some weeks after the test that I began forgetting.
> however I think you have to make some special effort to actually use the words (in full sentences) when you learn them if you want to be able to reuse them in the future.
yeah, I absolutely agree. I think context is key. Nothing beats being forced to use a language.
"Anki is great to help learning languages, however I think you have to make some special effort to actually use the words (in full sentences) when you learn them if you want to be able to reuse them in the future. One-to-one flash cards only helps for basic information retention."
Try a different Anki deck, maybe. I've been studying Polish, and a good portion of my cards are phrases or full sentences.
Yeah, the "downside" of spaced repetition is that you can't stop and expect it to stick (unless you have other means of keeping your memory fresh, such as actually using a language you've been learning). I have some subjects in my stack of cards (10,014 and counting!) that I'm not really so interested in anymore, but I keep them around because I'm paranoid that in a few years I'll want to pick them up again and will have to start from scratch.
It's not really so much a downside as an unrealistic expectation, like expecting to keep weight off after you've stopped dieting.
What method did you use to initially learn the characters? If you just used Anki, I could see your result as making sense.
I used Heisig's Remembering the Hanzi to learn the characters and then would practice them in a space repetition system. A few years later I can still write most of them. His method has you create a story and make mental image associations for each character. Eventually you forget the stories over time, but the character usually sticks.
> What method did you use to initially learn the characters?
We were introduced to the characters in class and subsequently learned how to write them. So after being introduced to a character I repeated it on paper for a couple of lines. I repeated this exercise from time to time. I also read the stories of the text book out loud and listened to the recording of the texts. But as I've pointed out in my other post, you have to have context to really learn a language. Just the fact that I used several types of learning doesn't mean that they weren't kind of shallow.
The technique you've delineated above certainly sounds like a better approach.
I find that in my language learning, when I have a question about a vocabulary word or a grammar point, I have to go back in my memory to the social situation in which I first heard it. Whether on TV or in conversation. I personally don't get the same hit to my memory from flash cards, though it may just be me.
> I expected that after six months of daily practice (about 20 minutes per day) I would have stored at least some characters in my long term memory.
Yeah, it doesn't work that way with SRS. If you're not actively using what you learned with your SRS (like say using Chinese on a daily basis) of course you're going to forget it! It's the same with bodybuilding: You have to constantly train your muscles and eat enough calories or they will shrink back to what they were before. It's not like you can gym, get as big as you want and then think if you stop you'll retain whatever you built :)
As someone who has dabbled in both biology/chemistry and now computer science academically, my thoughts on flash cards and spaced repetition software has changed over the years.
On one side you have the biology/chemistry guys who tend to use a lot of flashcards, and have a reputation for "just memorizing" facts. Then on the other end you have the physics/computer science guys who often refer to themselves as "lazy" for lack of memorizing and have a reputation for only focusing on conceptual ideas.
It seems to me that the people who succeed utilize a mixture of both approaches ("memorization" and "conceptual understanding"). I'm not sure when these two approaches suddenly became mutually exclusive.
As an example, although there were some classes in Biology that required a heavy amount of memorization, without the flash cards it would have been very hard for me to make the conceptual connections during lectures, while reading, etc.
Knowledge and understanding is kind of like a spider web, the more you have, the more you are able to grasp.
Purely focusing on conceptualizing the topics at hand has never worked out as well as I had hoped.
The only class I ever used flash cards was in Organic Chemistry. I do not think they helped long term memory at all as I recall next to nothing from that. If anything the act of creating the cards helped more than anything. I think this might be where the joining of the two sort of happens as you indicate.
To further that point, during exams some classes allowed 1 sheet of paper with anything you can put on it. I created some super dense cheat sheets in my time and I am fairly sure that I never referenced them but once or twice in 4 years of school. But again the active of creating them and sorting out important and unimportant information was key in solidifying knowledge for the long term. If someone handed me those cards or sheet they would have been useless to me in the long run is at least how I see it.
The other thing I learned in school was anything I crammed for I knew I was not going to learn. It may have been enough to get through the next couple of days but it was not going to stick. I took some summer school courses and found it to be a waste as it was too much like cramming as it was so condensed.
I guess I did well in school by using something like SRS which was allowing time between study sessions for information to settle and allow subsequent review after time. There is probably more than one reason for university courses to be on alternating days as it probably facilitates this kind of thinking. The only thing is that SRS seems to be focused on much shorter time periods than a day or 2.
What you say about the act of making the cards themselves rings true to me.
When I was preparing for the GRE in 2010 I first wrote down all the words and definitions from both the Princeton Review and Kaplan GRE vocab books on a legal pad. That didn't work as well as I'd hoped for quizzing (it's a pain to try not to read 90% of the page) so I wrote a really basic data entry script in Python that I used to input the words/defs and then a second one that would generate random questions based off of them (definition + 4 words as MC).
By the time I was done with typing them all in and debugging my little quiz script, I'd say I already had active recall on >90% of them. Having that working quizzing tool at the end to practice recognition was just icing on the cake - creating it in the first place was more useful to me.
Great book. Interleaving contexts, spaced repetition, etc. - all great concepts.
The most important takeaway for me was accepting that the tougher something is to remember, the greater the benefit will be when we actually remember it.
There's two major parts to learning a language: recognition and production. SRS can be quite good as a tool to aid in recognition, but I've found it to be lousy for production.
I think "completing the circuit" is among the most important aspect of educational theory. Recognize and produce needs to happen in any subject, and a good student recognizes that SRS is but one tool for recognition, but others are needed to complete that, and production is an entirely different side of the coin.
> Where my bright students might have been used to high Bs and low As on tests, they were now breaking my scales. You could see it in the multiple choice, but it was most obvious in their writing: they were skillfully working in terminology at an unprecedented rate, and making way more attempts to use new vocabulary—attempts that were, for the most part, successful.
> Given the seemingly objective nature of Anki it might seem counterintuitive that the benefits would be more obvious in writing than in multiple choice, but it actually makes sense when I consider that even without SRS these students probably would have known the terms and the vocab well enough to get multiple choice questions right, but might have lacked the confidence to use them on their own initiative. Anki gave them that extra confidence.
Not that you're wrong, but the author also points out here that SRS can help production by reinforcing the student's knowledge and boosting their confidence in what they've learned when it comes to using it.
Last year, I have developed a JS library SM.js (https://github.com/slaypni/SM-15) which is based on the algorithm used in the latest SuperMemo.
I have also developed a web service (http://slaypni.github.io/flashward/) for memorizing Foreign words using the library. Through the experience of using it by myself, I felt that spaced repetition helped me to remember many words for learning English.
I am feeling that most (free) SRS are using relatively simple algorithms (for example, Anki is based on SM-2 which is developed in the late1980s). Although I am not sure SM-15 is better enough, there could be some room to improve it. It is nice if the library was helpful for some developers who create new SRS.
Newer isn't necessarily better with SuperMemo algorithms. They introduced tremendous complexity without strong evidence of increased effectiveness, last I looked into it - it seems to create more corner cases to manage, if anything.
A couple months ago I put together a spaced repetition app (https://www.memoread.io/) aimed at less tech savvy students. At the time it didn't meet much success, although I may revisit it in the future.
I've been using SRS to learn 日本語 for some time now. Prior to this, my ability to remember kanji was basically nil. I could spend tons of time writing it over and over again, and nearly immediately forget how to write it, and shortly after, forget how to read it.
I switched to using Anki, and the reading at least became quite cemented - I still struggle with writing at times, but it's a significant improvement over what it was previously.
Daily study is hugely important - if I go out and have a crazy weekend, I hate myself Monday, and my study session becomes 3 or 4 times longer than it would otherwise.
Have you looked at "Remembering the Kanji"? I was skeptical when reading "Remebering the Kana", but halfway through it I found that despite the silliness, it worked. I tried the Kanji book for a bit and was fairly surprised how well it was working.
I'm not a huge fan of the Heisig method. I started off with it, but I realized that a lot of the time, the key word I was learning for the kanji had absolutely nothing to do with any meaning or reading for it.
What I currently do is learn new kanji via Kanji in Context, and then create cards for them in my anki deck as I have learned them.
You might want to look into "Kanji ABC" by Foerster & Tamura. It was developed some years after Heisig and avoids using nonsensical keywords. I think their approach is better than Heisig if you already have familiarity with a decent number of kanji. The book is out of print so it may be a little tough to get a hold of (I found mine online at Powell's used bookstore). You can see an online version at work[1], but it isn't that useful on its own without the book.
Recommendation: "The Kodansha Kanji Learner's Course" by Andrew Scott Conning is a modern version of RTK. Same concept, but better organization, more kanji, and a more natural layout and learning order.
As a method of developing real competency in a subject or practice, though, it offers none of the benefits of immersion as it keeps the student just on the edge of full engagement (and if we add immersion, we lose the time-savings that spaced repetition offers.) It introduces a fragility into the student's schedule that make lapses in 'discipline' more likely (not everyone has a teacher and regular subject periods to keep them on task.) For anything beyond basic key-value learning, it requires too many contortions and restatements of the subject -- to fit the format -- to be feasible for many students/subjects. The packaged decks available online help but usually don't meet the needs of the specific course or subject concentration the student requires. We can introduce variability to space how to solve a problem instead of a 'what' answer, but then we need to consider muscle memory, developing intuition and other things that the spaced repetition model does not address.