I scribbled the idea at the end if this post in June 2021 and decided to sit on it. It represented my general skepticism of the booming covid paper industry. Ok, ok, I’m promising you that I actually wrote this one year ago. Perhaps the very not relevant anymore Dogecoin reference will increase my credibility here?
Well, John Ioannidi and his team have a new article in PNAS, Massive covidization of research citations and the citation elite.
Abstract: Massive scientific productivity accompanied the COVID-19 pandemic. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021 and assessed the impact on scientist citation profiles. Using Scopus data until August 1, 2021, COVID-19 items accounted for 4% of papers published, 20% of citations received to papers published in 2020 to 2021, and >30% of citations received in 36 of the 174 disciplines of science (up to 79.3% in general and internal medicine). Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19; 110 scientists received ≥10,000 citations for COVID-19 work, but none received ≥10,000 citations for non–COVID-19 work published in 2020 to 2021. For many scientists, citations to their COVID-19 work already accounted for more than half of their total career citation count. Overall, these data show a strong covidization of research citations across science, with major impact on shaping the citation elite.
You see a massive citation bump of covid paper citations (purple dots).
From their conclusion:
COVID-19 papers published in 2020 received on average more than 8-fold the number of citations than non–COVID-19 papers and the difference exceeded 20-fold in General and Internal Medicine. Almost all of the top-100 most-cited reports published in 2020 across all science (not just biomedicine) were related to COVID-19, and the same applied to three quarters of the most-cited publications published in 2021. Many scientists received in a limited time high numbers of citations to their COVID-19 work and already have higher citations counts for COVID-19 alone than for all other scientific topics combined. Many authors who are highly cited for their COVID-19 work have had limited citation impact before the pandemic. COVID-19 is generating a new citation elite.
And:
Citation impact may not necessarily mean high quality or validity of the cited work. Many empirical evaluations of quality aspects of different segments of the COVID-19 scientific literature have consistently shown low quality (8–17). To our knowledge, there is no large-scale assessment of the correlation between quality scores (with all the difficulty of obtaining such scores) and citation impact of COVID-19 work specifically. However, other investigators have found that COVID-19 papers published in the most influential journals have weaker designs than non–COVID-19 papers in the same venues (14). Moreover, several extremely cited COVID-19 papers reflect topics that are debated or even refuted, such as editorials about the origin of the new coronavirus and early reports claiming effectiveness for interventions, such as hydroxychloroquine, that were not subsequently validated for major outcomes (e.g., mortality).
And:
As the total public funding for research is likely to change only gradually, and human productivity also has limits, it is plausible that persisting overemphasis on COVID-19 may reduce resources for other scientific work. This may have negative consequences on scientific progress, unless the imbalance in allocation is corrected promptly enough.
All considered, I think my much less thoughtful fretting below from last summer was pretty on the mark. Woohoo!
Wrote on 6/22/21, 7am. Drafting here. I’ll post at end of summer and see if I still agree.
Dumb hypothesis: most covid papers are academic dogecoin / gamestonk.
I was looking at google scholar today of prof's I really like and who do work on, let's keep this general, stratification.
They write awesome articles on substantive topics that get minor attention. cites: 20, 5, 17. Then along comes a 30-person slapped together covid paper on a totally unrelated topic. cites: 17million.
I saw this pattern 4-5 times without going out and looking for it.
My q: will papers like, "the networked facsimile of covid preparedness on reopening and the moderna vaccine" be of any substantive worth in 5 years? If you have a million cited toy paper totally outside your area...what value did you produce?
To me it's like: my car's repo'd but i have a dogecoin that cannot be turned back into money...i'm rich!