Quite a lot of brain research uses the technique called fMRI—and now quite a lot of research shows that fMRI brain research fairly often leads to nonsense or bewilderment.
A New Study Turns Up Much Nothingness
A new study tries to sum up the situation: “What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis,” Maxwell L. Elliott, Annchen R. Knodt, David Ireland, Meriwether L. Morris, Richie Poulton, Sandhya Ramrakha, Maria L. Sison, Terrie E. Moffitt, Avshalom Caspi, and Ahmad R. Hariri, Psychological Science, epub 2020.
The authors, at Duke University, the University of Otago, and King’s College London, explain:
“Identifying brain biomarkers of disease risk is a growing priority in neuroscience…. Measuring brain activity using task functional MRI (fMRI) is a major focus of biomarker development; however, the reliability of task fMRI has not been systematically evaluated. We present converging evidence demonstrating poor reliability of task-fMRI measures….
Collectively, these findings demonstrate that common task-fMRI measures are not currently suitable for brain biomarker discovery or for individual-differences research.”
(Thanks to Edward Tufte for bringing this to our attention.)
Legacy of the Dead Salmon
The 2012 Ig Nobel Prize for neuroscience was awarded to Craig Bennett, Abigail Baird, Michael Miller, and George Wolford, for demonstrating that brain researchers, by using complicated instruments and simple statistics, can see meaningful brain activity anywhere — even in a dead salmon.
They documented their research, in the study “Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Multiple Comparisons Correction,” Craig M. Bennett, Abigail Baird, Michael B. Miller, and George L. Wolford, Journal of Serendipitous and Unexpected Results, vol. 1, no. 1, 2010, pp. 1-5. They first presented that work in poster form at a conference in 2009.
That paper drew outrage from many neuroscientists, as did the awarding of that Ig Nobel Prize. Many other neuroscientists were pleased, though. Among them was Bethany Brookshire, then blogging under the pen name “Scicurious”, for Scientific American. Brookshire’s report about that Ig Nobel Prize began:
IgNobel Prize in Neuroscience: The dead salmon study
I have to say that I am incredibly pleased that this study won the Ignobel. Not just because it’s a really fun study, but also because it really is one of those studies that makes you laugh, and then makes you THINK. And in the case of this study in particular, it has changed a lot about how we think about making corrections in fMRI, and may have actually really affected the way the data are published. And so, I present to you: the dead salmon study….
Building on the Foundation of the Dead Salmon
The new, 2020 “What Is the Test-Retest Reliability…” study is drawing favorable attention from many neurosciences. It builds on the knowledge found by the dead salmon study. Duke University issued a proud press release about it, though for whatever reason did not mention the dead salmon. The press release begins:
STUDIES OF BRAIN ACTIVITY AREN’T AS USEFUL AS SCIENTISTS THOUGHT
Duke researcher questions 15 years of his own work with a reexamination of functional MRI data
Beauty
There’s one thing most everyone agrees on: fMRI studies produce pretty pictures.