ArticleGigaScience2019
Over-optimization of academic publishing metrics: observing Goodhart's Law in action.
Article in GigaScience, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 72 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
72 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Advancing lung organoids toward clinical applications: a global perspective on research focus and future directions.Frontiers in medicine · 2025Pooled it
- Uses of Electronic Health Record Data to Measure the Clinical Learning Environment of Graduate Medical Education Trainees: A Systematic Review.Academic medicine : journal of the Association of American Medical Colleges · 2023Pooled it
- TOP 2025: An update to the Transparency and Openness Promotion Guidelines.Research integrity and peer review · 2026Article
- Institutional Research Stratification in Global Dental Schools: A 10-Year Bibliometric Comparison Across QS Ranking Tiers.International dental journal · 2026Article
- Fear of Missing Out in Academic Workers: A Theoretical Framework.Annals of the New York Academy of Sciences · 2026Article
- Local scientific relevance in the face of the tyranny of global metrics: the future of the diamond model and health evidence needs in the region.Revista peruana de medicina experimental y salud publica · 2026Article
- Beyond genuine collaboration: the rise of strategic co-authorship in contemporary academic publishing.Research integrity and peer review · 2026Article
- Understanding of Authorship Guidelines and the Frequency of Authorship Misuse: A Descriptive Cross-Sectional Study in the State of Qatar.Journal of empirical research on human research ethics : JERHRE · 2026Article
- Article
- The publish or perish, publish and perish, publish then perish, and now retract and perish cultures in academia.Naunyn-Schmiedeberg's archives of pharmacology · 2026Article
- Rosetta Statements: simplifying FAIR knowledge graph construction with a user-centred approach.Database : the journal of biological databases and curation · 2026Article
- Responsible research evaluation: integrating quality, leadership, and integrity in national systems. The case of Peru.Frontiers in research metrics and analytics · 2026Article
- Global Scientific Discourses on Maternal Nutrition: A Scientometric and Altmetric Exploration into Research Trends and Patterns.F1000Research · 2026Article
- Article
- Research literacy and its predictors among university students and graduates identified by machine learning and spatial analysis.Scientific reports · 2025Article
- Suggestions for extending the FAIR Principles based on a linguistic perspective on semantic interoperability.Scientific data · 2025Article
- Citation manipulation through citation mills and pre-print servers.Scientific reports · 2025Article
- Trends and Challenges in Plant Cryopreservation Research: A Meta-Analysis of Cryoprotective Agent Development and Research Focus.Plants (Basel, Switzerland) · 2025Review
- Article
- Network assortativity for a multidimensional evaluation of socio-economic territorial biases in university rankings.PloS one · 2025Article
12 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades. Moreover, recent studies indicate that these metrics have become targets and follow Goodhart's Law, according to which, "when a measure becomes a target, it ceases to be a good measure."
resultsIn this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening. In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists. Measures such as a journal's impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors. Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department.
conclusionsAcademic publishing has changed considerably; now we need to reconsider how we measure success.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.