Article claiming that lockdown does not prevent death by Covid is ‘unpublished’

A study that would have shown that staying at home did not help reduce the number of deaths from Covid-19 around the world has just been portrayed, that is, “unpublished” by the scientific journal in which the research was originally published. The measure serves to indicate that the work’s conclusions can no longer be considered reliable.

Signed by four researchers from UFRGS (Federal University of Rio Grande do Sul) and Unisinos (University of Vale do Rio dos Sinos, also in Rio Grande do Sul), the original study was published on March 5, 2021, in the specialized journal Scientific Reports, and quickly caught the attention of politicians and activists against measures that would restrict mobility during the pandemic. In a short time, the link containing the analysis reached almost 400 thousand hits, something very rare for the vast majority of scientific researches.

In the article’s retraction warning, however, Scientific Reports states: “Editors are no longer confident that the conclusions presented have an adequate basis.”

The work, which has as its first author the physician Ricardo Savaris, from the Department of Surgery at UFRGS, used mobility data recorded on cell phones, through the Google platform, to try to estimate, in different regions of the world, how people’s behavior was altered by the pandemic — usually because of government measures such as lockdowns.

These data were cross-checked with deaths by Covid-19 in the same areas. In the analysis, the team from Rio Grande do Sul compared pairs of regions – one with significant changes in mobility, supposedly indicating more time spent by people at home, and the other without. They used the difference between the rates of stay at home, on the one hand, and the difference between the number of deaths per million population, on the other, to estimate the effects of the restrictions.

They then came to the conclusion that, in 98% of the comparisons, there was no significant association between the two – that is, staying at home almost never seemed to affect the likelihood of an increase in deaths from Covid-19.

This result spread around the world, but criticisms of the work’s methodology and conclusions quickly surfaced. Scientific Reports soon published a note of concern (on March 11, less than a week after the original article was published) indicating that the research findings were being contested. And, shortly before the final retraction, two analyzes contrary to the conclusions of the Brazilian research were published by the same magazine.

One of them was coordinated by Gideon Meyerowitz-Katz, from the University of Wollongong, in Australia. He and his colleagues pointed out, among other things, the fact that the mobility data used in the original study is likely far from reproducing what happened in the population as a whole. This is because they can only be collected on Android phones where a specific (and optional) functionality that records the user’s movements was enabled.

. In addition, the study excluded all countries with fewer than 100 deaths from Covid-19, leaving out countries like New Zealand and Vietnam, which took quick and hard steps to contain the pandemic.

However, according to them, the biggest problem was the statistical methodology of the original survey. Meyerowitz-Katz and his colleagues decided to test it using computer-generated data, which clearly showed a very large effect of restricted mobility in decreasing mortality.

It was something that should have appeared in the statistical analysis using the Gaucho team’s methods, but it just didn’t happen. It would be a sign that the Brazilian study had not been properly designed from a mathematical point of view.

The other analysis, carried out by Brazilian economist Carlos Góes, a doctoral candidate at the University of California at San Diego, indicated the presence of even more serious problems in the original work. According to him, the method used – checking the difference between a country A, with mobility restrictions, and a country B, without restrictions, and the number of deaths between each of them – is nothing more than a kind of average between countries. Thus, the result will almost always indicate that there is no statistical association between staying at home and the situation of deaths in each region.

Both criticisms are cited directly in Scientific Reports’ disclaimer. The note, however, also says that Savaris and his colleagues do not agree with the measure. In a message sent to the blog Retraction Watch, which tracks cases of retraction of scientific articles, the UFRGS doctor criticized the “unpublished”.

“It was a bad decision by the editors. The retraction was based on simulated data, they changed the methodology and the reviewers [responsáveis por analisar o artigo original]”, declared Savaris. Wanted, by the leaf, he did not answer.

The press office of the Springer Nature group, responsible for publishing the Scientific Reports magazine, told leaf that the article retraction process followed the normal course of these cases. “In general, when someone expresses doubts about articles we publish, whether the original authors or other researchers and readers, we evaluate the case carefully, following an established process, consulting the authors and, when necessary, seeking advice from reviewers and other external experts. Such issues are often complex, which can take time for editors and authors to fully elucidate them,” says the official statement.