Researchers have been using MRI technology for two decades to identify how a person’s brain structure and function are linked to a range of mental health issues, from anxiety and depression to suicidal tendencies.
But a new study, published Wednesday in the scientific journal Nature, calls into question whether many of these studies are in fact producing valid results.
Many studies of this type, the authors of the paper found, tend to include fewer than two dozen participants, which is far short of the number needed to generate reliable results.
“Thousands of subjects are needed for a study,” said Scott Marek, a psychiatry researcher at the Washington University School of Medicine in St. Louis, and one of the study’s authors.
He described the finding as “a severe blow” to typical studies that use MRI to try to better understand mental health.
Studies that use magnetic resonance technology to record brain images often present conclusions preceded by a caveat about the small sample size.
But getting participants for this type of research can be a time-consuming process, and the work is expensive, with costs ranging from $600 to $2,000 an hour, according to Nico Dosenbach, a neurologist at the Washington University School of Medicine and another of the study authors.
The median number of participants in mental health-related studies employing brain MRI is about 23, he added.
But the study published in Nature showed that data from fewer than two dozen participants are often insufficient to provide reliable conclusions and in fact can yield “heavily skewed” results, Dosenbach said.
For their analysis, the researchers examined three of the largest studies that used brain MRI technology to reach conclusions about brain structure and mental health.
The three studies are still ongoing: the Human Connectome Project, with 1,200 participants; the Adolescent Brain Cognitive Development (ABCD) study, with 12,000 participants; and the UK Biobank study, with 35,700 participants.
The authors of the study published in Nature studied subsets of data from these three studies to determine whether assessments based on smaller numbers of participants were misleading or “reproducible,” meaning their results could be considered scientifically valid.
For example, the ABCD study assesses, among other things, whether the thickness of the brain’s gray matter may correlate with mental health and problem-solving ability.
The authors of the study published by Nature evaluated small subsets of data within the larger study and found that the subsets produced unreliable results compared to the results offered by the full ABCD dataset.
On the other hand, the authors found that when results are obtained from samples involving a few thousand participants, the conclusions tend to be similar to those found using the full dataset.
The authors performed millions of calculations using different sample sizes and the hundreds of brain regions explored in several of the large studies. And they repeatedly determined that subsets of data obtained from samples smaller than a few thousand participants did not yield results consistent with those of the full data set.
Marek said the study’s findings apply “completely” to fields other than mental health. Other fields, such as genomics and cancer research, have already struggled because of the limitation caused by small sample sizes, and have tried to correct their course, he pointed out.
“My guess is that this applies much more to population science than to any of these fields,” he said.
Translation by Paulo Migliacci
Chad-98Weaver, a distinguished author at NewsBulletin247, excels in the craft of article writing. With a keen eye for detail and a penchant for storytelling, Chad delivers informative and engaging content that resonates with readers across various subjects. His contributions are a testament to his dedication and expertise in the field of journalism.