Why I’m Skeptical About SRS for Conceptual Subjects

Spaced repetition software, such as Anki, works by making flashcards which will pop up to test you, just at the moment it estimates you’re most likely to forget. I was initially wary of the tool, but I quickly converted when learning Chinese. If you need to memorize tons of information, there are few tools more efficient.

Recently, a number of readers asked me about using it for conceptual subjects. Could you learn math or physics with Anki?

Here, I’m more doubtful. Let me explain.

What SRS is Good For

Spaced repetition software (SRS) works through flashcards. Put a Chinese character on one side and it’s meaning on the other. The software then encourages you to associate the character and meaning by testing you on it.

You can also memorize more complex information, by selectively omitting parts of a larger structure. So you could memorize a map by blanking out one of the spaces on the “question” card and having it filled in on the “answer” card.

In all cases, however, what’s being reinforced is a single link: prompt/response.

Sometimes prompt/response is most of what you need. Memorizing vocabulary doesn’t entirely reduce down to memorizing the translation, but it’s close enough. Especially when you need to memorize thousands of words, a slight loss of context and nuance is forgivable.

The Allure of Applying SRS to Math

Conceptual subjects—such as physics or math—don’t fit neatly into single links. You could try to memorize Newton’s second law of motion as F = ma. But that’s probably one of the least important parts. Knowing where it applies, manipulating the equation and the intuition behind it are also necessary.

The suggestion is, then, that you don’t use SRS to memorize “facts” about physics or math. Instead you use them to prompt questions. So a flashcard wouldn’t say “What is Newton’s second law?” but, “Solve this set of differential equations.”

But now the question is what do you solve? Do you solve the same question each time? That seems unfair. If I had a complicated question, I may not be learning to solve similar questions, but simply that the answer is “x = 7”. We want to understand concepts, not just that a particular instance of a problem had a particular answer.

We can escape this problem somewhat by randomizing the values. So instead of giving the same question, the variables are all modified within some range.

A Little Randomness May Not Be Enough

This creates a new problem: writing the software to make flashcards with randomly generated values. It’s not impossible, but it means you can no longer just create static content. There is software that does this, but it tends to be specialized, meaning it’s a lot harder for DIY learners who want to learn methods that will work for any subject. You need some kind of scripting language to procedurally generate problem sets.

Despite this improvement, the original problem still hasn’t been avoided. Now we might just be memorizing the solution procedure. Instead of “x = 7” it’s “subtract the top-left from the top-right and divide by two.”

Memorizing solution patterns is a heckuva lot better than memorizing answers. In fact, I suspect a lot of students get through math classes without ever going beyond memorizing solution patterns.

However, it’s usually not the end goal. What we really would like to see is a higher level of understanding. Recognizing that a certain type of problem involves conservation of energy or resonance. Even if we aren’t expecting ourselves to solve completely novel problems in each go, we should still have more flexibility than problems with identical surface features.

Avoiding this problem fully means either creating dozens of randomized problems to sample from, or a tool for generating randomized problems that would end up being quite complex.

In the end, I’m not sure such an approach would be substantially better than just setting up a regular time to do review questions. Such an approach lacks the timing features and card management of SRS, but it also avoids the problems of memorizing things that need to be understood.

What has been your experience? Have any of you tried using SRS to learn more complicated subjects which require conceptual knowledge? What worked and what didn’t?

Read This Next
How I'm Learning Spanish
  • Satvik Beri

    I sort of disagree with this. I don’t think SRS is good for *understanding* Math, but I think it’s a great way to maintain your understanding years down the road.

    I was a math major in college, and often found that when I reviewed material 2 years later I had forgotten significant chunks of it. It’s not like I had to relearn it from scratch, but I did have to spend hours refreshing my understanding.

    On the other hand, filling my SRS with some of the most common theorems and questions like “what are three interesting applications of the Riesz representation theorem?” or “Come up with a problem that’s best solved using induction on terms” has been a pretty good way to maintain my understanding with relatively low investment.

  • Anthony

    I agree with what has been written in this article. However, there is a way to use SRS that I believe eliminates most of these problems. Instead of using its flashcards to pose questions directly, the flashcards may be used to give instructions. In this way, its spacing algorithm may be used while retaining the flexibility of practice associated with non-SRS review. For example, one may give instructions to review a particular problem-set; in this case, the problems selected for review of the relevant concepts might be new.

    To increase the usefulness of spaced practice when focusing on questions that have been solved before, one could use a big-picture focus and explain the concepts involved in solving the problem while solving the problem, as if giving a tutorial.

  • Ebrahim Khalil

    Although I do not use Anki, I employ SRS a lot by going through what I learned with a consistent pattern.

    I would say the best way to go about using it for conceptual topics is to store what you understood or an explanation of it. Things that you need in order to start the procedure or some tricky situations that would be harder to practice for exclusively.

    This makes sure you’re being reinformed on a regular basis about what you understood so as to not have to do it all over again. I really like doing this because it opens the possibility of contrasting what you know as it would be so present and offer a possibility of deepening your understanding and make connections with other things.

  • Trent Fowler

    Jack Kinsella wrote about using Anki to learn to program:

    http://www.jackkinsella.ie/201

  • Duncan Smith

    Khan Academy is essentially SRS software for math (and some other subjects). As you get better at a problem type, you see that problem type less, and other problem types become available. It does tend to encourage memorizing solution patterns, but they’re getting better at creating questions that resist strict memorization. Also, sometimes you just want a blast of easier math review to get warmed up for learning harder topics.

  • Simon

    I agree with Satvik. I think that SRS is best for retaining knowledge that you already understand, rather than to teach that understanding.

    I think that SRS can be used for conceptual subjects. There is a page on the supermemo.com website that describes it to some degree:

    http://www.supermemo.com/engli

    I think that creating a flashcard of a formula and an answer would end up with you memorising the answer only, for example x=7. But if you create multiple flashcards showing the steps used to derive the answer it would be much more valuable. For example “What is the next step in solving this equation?”. I am extrapolating a little, as I don’t personally used SRS for mathematics.

    I do use it for pharmacology, biochemistry and medical information, which is both conceptual and declarative in nature. I have found it useful to try to make questions as abstract as possible.

  • Ryan Muller

    I’ve been thinking about this question for about five years. In short, you’re right to be skeptical, but I’d argue that a well done flashcard set could be a powerful part of learning any topic.

    First, any difficult topic has a language (or several) with which to communicate its concepts. There’s no reason to think such a language is any less SRS-able than a spoken language: there’s still a mountain of pure stimulus-response items to commit to memory.

    So what do we have to deal with other than “pure stimulus-response”. The KLI framework attempts to define exactly that (pdf: http://pact.cs.cmu.edu/pubs/Ko… ; my summary: http://wiki.learnstream.org/wi…. You already have the right idea: it’s about the variability of input and output. I would put it this way: everything is still stimulus-response. However the stimulus can be a _pattern_: we may not have a particular sign or symbol to describe it; the response can be a _procedure_: again, not a particular sign or symbol but a varied action taking into account the input pattern. I’d even take a leap and say this is the most general definition: even something like the letter “a” has incredibly varied visual and auditory pattern, it’s just a combination of our brain configuration and lots of repetition (naturally, it happens to be “spaced”) that has made it seem incredibly basic to you and me.

    So you’re right that showing a single example of the input pattern and asking for a single example of the output pattern, repeatedly, is not going to be the best way to learn. Asking “What’s better?” is essentially asking how exactly do we learn, and this is incredibly difficult. There are answers to particular examples that rely on numerous experiments and basically come down to long, complicated sequences of building off of things we know in weird metaphorical ways (Mind in Society and Origin of Concepts are good places to start if you want to dive deep into this world).

    I’ll just give one example that isn’t particularly “conceptual” but hopefully illustrates it. A flashcard service has a demo that shows pictures of big cats and asks for their species name. Two of these species were really similar (in the given pictures) and after a long time with spaced repetition I was still getting it wrong half the time. What would have been better? First and foremost, showing the damn pictures side-by-side so that I could compare the two and actually be able to *see* what’s different between them: oh! one has pointy ears and the other doesn’t. Can’t I just picture the one from before and do that? No. I don’t understand how photographic memory works for some people, but no. Maybe I could do that if I had a lot of pictures of one species: I’d start to generalize the “concept” of that species: these guys all have pointy ears, or whatever. The theme here is multiplicity: we’re naturally good at finding patterns and making analogies, but only when our visual mind is put to work in the right conditions. Another possibility is just to *point out* the distinguishing features. “These guys all have pointy ears, see?” Either way, now I can actually form the pointy ears -> species name connection, and–more importantly–it generalizes a little better when you see _any other picture of that species_.

    These and other techniques can accounted for when you build a card for SRS, but at this point the interesting thing is clearly not the spaced repetition system itself.

    We can jump to an extreme and say, if we broadly define an SRS, what’s the best possible outcome we could hope for? Well, let’s say the SRS can summon anything onto your computer screen and can do that in any sequence rather than the circa 1980 SM-2 algorithm or whatever. So an upper bound is that an SRS could do exactly the best possible thing you could have done sitting at a computer to learn something. Which is pretty good, but I would argue/guess is at best 2-3x faster for a good learner learning a well-defined topic for the first time. Why not more? Because very little educational research finds *huge* gains over a reasonable control case, competent computer users are good at finding decent resources, and competent learners are pretty good at avoiding things that are way too easy or hard to be helping you learn. Still that 2-3x can be significantly impactful.

  • Zafir

    Hey Scott, thanks great insight.

    I’ve just started medschool at UofT and have fallen in love with Anki for anatomy. Obviously, physiology is a bit tougher because it relies heavily on concepts.

    I’ve recently began tinkering with Anki though using one of your techniques. What if you have a separate deck called “Feynman Technique” and all it would do in timely manner is prompt a concept name/label. Then you would have to do a Feynman on the concept, teaching it to the best of your ability. Then depending on how you performed, you can select Hard, Good, or Easy.

    I feel like Feynmans are great learning tools, but I just keep forgetting to do them on learned concepts. Using Anki to remind you that “hey you might forget this soon, so try to teach it” can be an extremely effective tool, can it not?

    Looking forward to hearing your thoughts on this!

  • Nate

    For a year of uni, I wrote all my notes in question and answer format, qs on left, answers on right, and studied from this using active recall.

    I had tremendous ability to recall definitions and the like, but my understanding of subjects was poorer than usual. The big picture was missing.

    This year, I’m trying flow based after notes instead. Only a week in, will see how it goes 🙂

  • MH. Rahmani

    hey there.
    i use SRS for conceptual subjects too!
    here’s how:

    i always study for the “AHA!” point.
    if i dont “get” it, i just attack it from different directions: i chunk it, i look it up, i watch a video for it…
    until it get to the “AHA!” point, where everything makes sense all of a sudden.

    what i do next, is to make sure i dont forget WHY and HOW things are connected.
    and that turn into my question for the SRS input.
    for example, in the “F = ma” case, i just simply ask “why F = ma?”
    and then try to go through ALL the connections created in my brain for that formula to make sense.

    on the answer side? well i just put the minimum amount of info to check if i have visited all the nodes and connections.

    Here’s the broad picture:
    in my experience, even deep understanding of a subject can be very soon forgotten.
    SRS concept is to keep visiting and strengthening specific Nodes and connections in our brain.
    if it’s used in a specific way, it will work in that specific way.
    example:
    when i was studying Assembly, i mastered the boolean logic and operations and it all made sense to me. i knew the math and reasoning behind for example a NAND or XNOR operation and why it’s there.
    well, about a year later, when i had to study “Logical Design”, it all was gone! well not GONE, but when i saw the XNOR, i had to think back, write down on paper and again deduce the results and i was back on track again.

    essentially re visitting nodes and connections.

    SRS helps to strengthen those connections, so you would have an easier time using them when needed.

    BUT, it needs HONESTY (with yourself) and SMART DESIGN.

  • Nate

    Update: It’s been a year with flow based notes and the learn it once approach has been so valuable I have stopped with q and a notes. It saves so much time to actually learn the content the first time instead of delaying the learning for later while you take notes. It isn’t as comprehensive, but the learning/hour is easily doubled.

    The q and a notes were good though. I wish I could find a way to incorporate them again in a time efficient way.

  • Nate

    Update: It’s been a year with flow based notes and the learn it once approach has been so valuable I have stopped with q and a notes. It saves so much time to actually learn the content the first time instead of delaying the learning for later while you take notes. It isn’t as comprehensive, but the learning/hour is easily doubled.

    The q and a notes were good though. I wish I could find a way to incorporate them again in a time efficient way.

  • PUGEPUGEPUGE

    i think you should make q&a notes more like an essay type. for example instead of asking what, you aks “why is it like this?” or “how was it formulated?” or even just “explain etc.”. And then on the answer would be just info and evidences.
    One thing i learned from reading blogs like this is to never take notes that are a direct copy from the teacher. every note taken down should have been processed first by your brain by either summarizing it, using simpler words, or writing down/drawing metaphors.
    and then during your review you just answer the questions out loud and explain it to to yourself or to the air, you can also use pen and paper to draw better examples and diagrams

  • PUGEPUGEPUGE

    i think you should make q&a notes more like an essay type. for example instead of asking what, you aks “why is it like this?” or “how was it formulated?” or even just “explain etc.”. And then on the answer would be just info and evidences.
    One thing i learned from reading blogs like this is to never take notes that are a direct copy from the teacher. every note taken down should have been processed first by your brain by either summarizing it, using simpler words, or writing down/drawing metaphors.
    and then during your review you just answer the questions out loud and explain it to to yourself or to the air, you can also use pen and paper to draw better examples and diagrams

AS SEEN IN