Is Cog Sci Research Diverse Enough?
Jonathan Firth's Memory & Metacognition Updates #103
Last time, I compared research in cognitive science to the process of connecting dots. Even if we don’t have every possible dot (i.e., all the empirical studies we’d like), we can make certain assumptions based on theory.
And the fuller a picture we have based on research studies, the better those theories will be.
I also raised a common concern – too many research studies are carried out in conditions that don’t resemble the classroom. This makes their findings hard to generalise.
Blind spots
If research is systematically biased – missing out certain conditions or particular groups of learners – it can lead to a blind spot in our understanding.
We might be drawing flawed conclusions. Worse: on a metacognitive level, we might not realise that we are doing so!
If, for example, every study was conducted on adults, and every one used unrealistically simple learning materials, then it would be plausible that findings which seem robust and ‘evidence based’ might fail under other conditions.
Diversity
With this in mind, is cognitive science diverse enough? The obvious answer is no. Many studies are conducted on Western university undergraduates. Clearly we should do with more research on under-represented groups. This includes:
Studies conducted outside of Western countries
Studies on children, or on adults outside of formal education
Studies in realistic classroom contexts
Studies that don’t focus on neurotypical learners
However, the good news is that we can make some reasonable speculations about where and when phenomena such as desirable difficulties (see update #99) will generalise.
The default assumption will be that learning works in a roughly similar way across learners unless we have reason to think otherwise. And from what evidence has been established, a consistent picture has started to emerge.
What studies are there?
Off the top of my head, I can think of several studies that help to diversify the evidence. For example:
Spacing or interleaving effects have been found among older adults and pre-school children. Spacing has even been found in animals.
Spacing and retrieval have been tested with complex classroom activities from elementary school up.
While the large majority of classroom studies of retrieval practice have been carried out with Western participants, Agarwal et al (2021) review several that do not.
Following on from the point above, Agarwal and colleagues also note that “retrieval practice consistently improved learning across a range of ages, content areas, formats, etc”. This supports the idea that, like spacing, this reflects a basic and general feature of how memory works.
There are, after all, considerable commonalities in the basics of learning and memory across different groups. We are not all as different as we sometimes think!
Without clear evidence to the contrary, it’s fair to see evidence-based strategies from cognitive science research as ‘best bets’, regardless of context.
Moving forward
It would still be incredibly helpful to have a broader evidence base that includes more diverse groups of participants and more naturalistic contexts. This is something that researchers have repeatedly called for, and I do expect things to gradually improve in the coming years.
It is also valuable to support diversity among the research community itself. It’s worth checking out the list below, which features “early/mid-career experts, including women, BIPOC, and LGBTQ+ scientists” and “demonstrate[s] diversity beyond the traditional ivory tower”:
Cognitive scientists – retrievalpractice.org
Thanks for reading! Next week I’ll share a recent piece of writing on thinking skills and ‘thinking routines’ 😎
Jonathan
I hope you are enjoying Memory and Metacognition Updates. Subscribe for free to receive new posts and support my work – and consider sharing it with colleagues!! Thank you!
Please note that my slides, posts and similar materials are shared under a CC BY-NC 4.0 license. This means you can use or adapt them with attribution for non-commercial purposes. If you wish to use my materials for other purposes, feel free to get in touch.