I think it's doing all right though. Here are some of the decks I'm most proud of:
- Using lilypond snippets and a midi-to-wav generator, I wrote a perl script that generates a deck for every (non-accidental) note on a guitar fretboard. It shows the fret position with a question mark, and then when you identify the note, it plays the pitch and shows the note on a treble clef. I'll probably expand it to include accidentals soon.
- Also using lilypond, I wrote a jazz theory deck for major chords. Questions like, "What chord quality and inversion is this?" where it plays a chord, and answer would be m7b5 (half-dim) 2nd inversion. My ear training has improved a ton. And, what is the key scale for bII7/iii in B major? (Plays the chord audio). Answer: B melodic minor or E lydian/mixolydian, playing and showing the scale on treble clef. And an effect like gwern described has happened. I haven't actually memorized the answer to every card in the deck (since the perl script generated about 2500 of them). Instead, I've gotten faster of doing the actual music theory processing in my head to calculate/recognize the right answer almost instantaneously. End result is that it has improved my speed in reading lead sheets. (It has not, however, improved my melodic improvisation abilities. :-) )
The rest of my decks are basically based off of Coursera courses. Unfortunately, I haven't finished any of them yet. I routinely run into the problem where partway through the course, the amount of time I need to spend keeping up with the new cards limits my time available to watch new videos and do new assignments, so I don't finish the course. I am now taking the Scala course again, and I'm hopeful that since I'm fully reviewed up through week 4, I might be able to power through and finish it with a deck that fully reflects the course.
This also means that I am improperly creating a separate deck for each subject matter, when Anki doesn't let you scramble them. I should be putting them all in one deck, but I kinda don't want to before a deck is "complete". I'm also a couple of chapters into "Learn You A Haskell", with the rest of the book on pause.
Finally, I can echo that practicing helps a lot. For instance with the Haskell deck, I got up to fully reviewed, and each card was still kind of hard. Then I spent a couple of hours playing with the first Project Euler problem. After that, the entire deck was much easier and faster. So, retaining familiarity still is somewhat different than learning the concepts. If I have a fully reviewed Anki deck from Odersky's Scala course (on coursera), it won't mean I know Scala. But if I go start a Scala project, the Anki knowledge will help me a ton, and then afterwards, the experience from hacking on the project will make the Anki cards much easier, maybe even to the point that I could delete a bunch of them.
BTW, that's feature I wish I had from the deck - some kind of advice for when it might be appropriate to delete a card. I worry that I'm doing the equivalent of writing pointless unit tests sometimes.
For the people reading this that do jazz, it's based off of Randy Halberstadt's concepts of planets and harmony. Scale choices differ from the classic guideline of being simply chord based. For instance, dmi7 does not necessarily imply a d dorian scale. If it's in the key of C at that point, then you'd instead play C major bebop.
Unit tests never go away, though. Eventually a card will get scheduled for years out, and it takes a trivial amount of time to review, so it shouldn't be a problem. I'd imagine.
The Coursera decks I would absolutely release but I need to finish a course first, and I need to make sure they don't have assignment answers in them I guess.