I’ve shared some [1] reading [2] lists [3] based on my research for my upcoming book. But in addition to the 140 or so books I’ve read for the project, I’ve also read around 500 scientific papers. While most academic papers don’t make for lively reading, the best are fascinating.
Here I’d like to share a selection of some of the papers that had a significant influence on my thinking and might do the same for you too:
Side note: I’ve tried, when possible, to include links that follow to a PDF in case you want to read it yourself. Where that was unavailable, you’ll be on your own… cough Sci-Hub cough cough
1. Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching [4] by Paul Kirschner, John Sweller and Richard Clark
A major debate in educational research concerns the relative merits of direct instruction versus methods that rely on exploration, problem-solving or experiential learning, often called “constructivist [5]” teaching approaches.
Constructivism has an intuitive appeal. Students are often bored and disengaged in lectures. They fail to transfer what they learn to the real world. Real life involves active effort and problem-solving, whereas so much of traditional schooling seems to be regurgitation of memorized facts.
The authors here argue that those intuitions are misguided. The research favors strongly-guided forms of instruction. Methods in which skills are carefully explained and taught consistently outperform methods that rely on students discovering, inventing or creating their own solutions.
This paper attracted enough controversy that an entire book [6] was created with various experts weighing in on both sides. Ultimately, I think those favoring direct instruction made the better case, but the debate is almost certainly not over.
2. Conditions for Intuitive Expertise: A Failure to Disagree [7] by Daniel Kahneman and Gary Klein
How good are experts?
The two authors of this paper devoted their research careers to this question—and have come to very different conclusions.
Daniel Kahneman is famous for his work on heuristics and biases. His studies show how faulty our reasoning typically is, even in areas where we have extensive experience. Other research shows that experts often perform poorly, despite considerable confidence.1 [8]
Gary Klein has spent years studying top performers in naturalistic environments. His work with firefighters found that they often quickly make the right decision without stopping to weigh costs and benefits.
Despite their differences, the duo agree on much. Intuitive expertise is only possible when we are in favorable learning environments. There must be highly valid cues that indicate the nature of the situation, and we must have an opportunity to learn those cues through instruction and rapid feedback.
3. When and Where Do We Apply What We Learn? A Taxonomy for Far Transfer [9] by Susan Barnett and Stephen Ceci
Transfer is undoubtedly the most important issue in education. We’re engaging in transfer whenever we learn something in one setting and apply it in another. Thus, the utility of any school, book, course or training experience hinges on transfer.
Despite this, many more studies report failures of transfer than robust successes, and the causes of this have been endlessly debated.
Barnett and Ceci review some of the research while adding that what we call “transfer” can really be broken down into several different dimensions, such as:
- Transfer between domains (e.g., learning about exponential growth in a biology class and applying it to the stock market)
- Transfer between contexts (e.g., learning to multiply in school and then applying that at the grocery store)
- Transfer over time (e.g., learning something in 9th grade and applying it during your working life)
My opinion is that resolving questions about transfer is so difficult because they’re actually questions about how the mind works. Knowing how much transfer is possible hinges directly on how the mind represents skills and knowledge. Until a consensus theory emerges, transfer will continue to attract debate.
4. Self-Efficacy: Toward a Unifying Theory of Behavioral Change [10] by Albert Bandura
Classic theories of motivation focused on the relationship between outcome expectations and our willingness to act. Under these theories, we take actions we believe will be rewarded.
Bandura modified this by suggesting another variable: do we believe we can execute the action needed to get the result? If our self-efficacy is low, we may think that success is valuable but still fail to find the motivation to take action.
Bandura posited four contributors to self-efficacy, two weak and two strong:
- Bodily arousal (weak). Being agitated can undermine our confidence.
- Persuasion (weak). Being cheered on or told we can do it can modestly increase our self-efficacy.
- Vicarious experience (strong). Watching someone else succeed can convince us we can as well.
- Personal performance (strong). Succeeding at something is the most compelling evidence that we can perform the actions we need to.
5. The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring [11] by Benjamin Bloom
In this famous paper, Benjamin Bloom claims that students with one-on-one tutoring can perform two standard deviations better than an appropriate baseline.
This suggests that substantial learning gains are possible.
However, it’s impractical because the education system can hardly afford one teacher per student. Bloom’s challenge was to see if any pedagogical techniques could approach the gains seen with tutoring but could apply to a classroom with dozens of students.
Bloom believed he had found an answer in mastery learning. In this approach, students are given many interim tests. Those who fail to master the material are given new explanations, practice and an opportunity to try again. The idea is that catching difficulties early will prevent them from becoming ongoing problems.
While most meta-analyses of mastery learning put the effect size closer to 0.5 standard deviations [12], a far cry from the two sigma Bloom sought, this is still a relatively strong effect.
6. Blind Variation and Selective Retentions in Creative Thought as in Other Knowledge Processes [13] by Donald Campbell
Drawing on an analogy to biological evolution, Campbell argues that randomness is an overlooked variable in learning and creative thinking.
He argues that all knowledge generation proceeds by:
- A trial-and-error process to induce some knowledge about the world. As with evolution, we try things out and retain what works.
- Using previously-acquired knowledge instead of guessing. Once you uncover valid knowledge using the first process, you can apply it without the need for random guessing.
- Even applying previously-acquired knowledge, there is variation and randomness generating further possibilities.
This suggests that much of what we associate with inventive creativity is simply (a) acquiring the relevant set of knowledge associated with the field and then (b) trying out lots of things and keeping what works.
7. Antagonism Between Achievement and Enjoyment in ATI Studies [14] by Richard Clark
Aptitude-treatment interactions (ATI) are when the same learning technique elicits different effects in students with different prior abilities.
A common finding in ATI studies is that low-ability students learn much better with highly-structured approaches, whereas higher-ability students benefit from less-structured environments. One explanation seems to be that if you lack the necessary knowledge and skills, guidance ensures you learn the material correctly. Still, if you already have them, then challenging, open-ended environments give you needed practice applying what you know.
It would seem reasonable to expect students to opt for the method of learning that works best for them. Low-ability students would recognize their difficulties and seek structure; high-ability students would try more challenging assignments.
Except when Clark reviewed the ATI literature, he found the opposite. Students tend to prefer the method that works less well for them, often unaware that it hinders their achievement. One explanation is that learning is effortful, and we enjoy learning methods that seem to save us effort.
8. Long-Term Working Memory [15] by Anders Ericsson and Walter Kintsch
Few concepts are as central to the science of learning as working memory. Our mental bandwidth is limited, and we can only keep in mind a few things at a time.
Given that we can only hold a few things in memory at once, how do we perform complex tasks?
One theory is chunking [16]. Through experience, we learn to recognize whole patterns of information as a single unit, allowing us to keep more in our heads at once. Remembering a random seven-digit number is hard, but we easily recall our phone number.
Ericsson and Kintsch review evidence that suggests chunking is insufficient to explain expert performance. For example, experts seem strangely impervious to interruptions. When reading a story rife with distractor sentences, comprehension of the story itself remains largely intact. This starkly contrasts with typical memory experiments, where distractors can completely wipe out memory for a task.
Ericsson and Kintsch argue that as we gain skill in an activity, we get better at using our long-term memories as a form of working memory, effectively expanding our capacity for familiar tasks.
9. Does Learning to Read Improve Intelligence [17] by Stuart Ritchie, Timothy Bates and Robert Plomin
Keith Stanovich was among the first to propose that reading ability could bootstrap intelligence [18]. The logic of the hypothesis is compelling:
- Much of the world’s knowledge is available only through reading.
- Reading ability tends to be self-reinforcing, as good readers get more practice than poor ones.
- By reading more, people can learn more things and thus become smarter.
The study by Ritchie et al. explores this hypothesis further by examining how early reading ability impacts later intelligence. They studied identical twins to control for genetic differences in intelligence. The researchers found that twins with higher reading ability showed greater improvements in intelligence over time compared to their sibling.
10. Eliciting Self-Explanations Improves Understanding [19] by Michelene Chi, Nicholas de Leeuw, Mei-Hung Chiu, and Christian LaVancher
I’ve long been a fan of what I call the Feynman Technique: take a complicated concept or procedure, and explain it as if you were teaching it to someone else.
Thus, it was interesting to come across formal research on self-explanations! In this experiment, the researchers encouraged students to explain what they were learning. They found that engaging in explanation tended to increase students’ understanding of the material.
My preferred explanation for this effect is attention. When reading an explanation, you generally don’t have much motivation to test whether or not you understand it. In contrast, when generating an explanation, you get clear feedback about what you know and what you don’t. This feedback returns your attention to the source material or problem to work out what is missing, resulting in a richer understanding than if you had stuck to the reading alone.
Footnotes
- Examples of expert overconfidence includes Philip Tetlock’s Expert Political Judgement [20], where political experts did little better than chance, Paul Meehl’s well-replicated finding [21] that clinical judgement typically does worse than simple statistical rules, and the finding that active investors tend to fail to deliver superior returns.