Evidence map›Paper›PMID 27604469›Full record

ArticleGenome biology2016

An expanded evaluation of protein function prediction methods shows an improvement in accuracy.

Yuxiang Jiang, Tal Ronnen Oron, Wyatt T Clark, Asma R Bankapur, Daniel D'Andrea, Rosalba Lepore, Christopher S Funk, Indika Kahanda, Karin M Verspoor, Asa Ben-Hur and 137 more

Abstract read
In one paragraph

Article in Genome biology, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 196 papers.

0numbers the graph read from it
0cells of the map it votes in
196citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

196 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Article
  6. Advances in Protein Function Prediction from the Fifth CAFA Challenge.bioRxiv : the preprint server for biology · 2026
    Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Computational function prediction of bacteria and phage proteins.Microbiology and molecular biology reviews : MMBR · 2025
    Review
  19. Article
  20. Article

136 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

147 authors.

Yuxiang JiangDepartment of Computer Science and Informatics, Indiana University, Bloomington, IN, USA.
Tal Ronnen OronBuck Institute for Research on Aging, Novato, CA, USA.
Wyatt T ClarkDepartment of Molecular Biophysics and Biochemistry, Yale University, New Haven, CT, USA.
Asma R BankapurDepartment of Microbiology, Miami University, Oxford, OH, USA.
Daniel D'AndreaUniversity of Rome, La Sapienza, Rome, Italy.
Rosalba LeporeUniversity of Rome, La Sapienza, Rome, Italy.
Christopher S FunkComputational Bioscience Program, University of Colorado School of Medicine, Aurora, CO, USA.
Indika KahandaDepartment of Computer Science, Colorado State University, Fort Collins, CO, USA.
Karin M VerspoorDepartment of Computing and Information Systems, University of Melbourne, Parkville, Victoria, Australia.
Asa Ben-HurDepartment of Computer Science, Colorado State University, Fort Collins, CO, USA.
Da Chen Emily KooDepartment of Biology, New York University, New York, NY, USA.
Duncan Penfold-BrownSocial Media and Political Participation Lab, New York University, New York, NY, USA.
Dennis ShashaDepartment of Computer Science, New York University, New York, NY, USA.
Noah YoungsCY Data Science, New York, NY, USA.
Richard BonneauDepartment of Computer Science, New York University, New York, NY, USA.
Alexandra LinDepartment of Electrical Engineering and Computer Sciences, University of California Berkeley, Berkeley, CA, USA.
Sayed M E SahraeianDepartment of Plant and Microbial Biology, University of California Berkeley, Berkeley, CA, USA.
Pier Luigi MartelliBiocomputing Group, BiGeA, University of Bologna, Bologna, Italy.
Giuseppe ProfitiBiocomputing Group, BiGeA, University of Bologna, Bologna, Italy.
Rita CasadioBiocomputing Group, BiGeA, University of Bologna, Bologna, Italy.
Renzhi CaoComputer Science Department, University of Missouri, Columbia, MO, USA.
Zhaolong ZhongComputer Science Department, University of Missouri, Columbia, MO, USA.
Jianlin ChengComputer Science Department, University of Missouri, Columbia, MO, USA.
Adrian AltenhoffETH Zurich, Zurich, Switzerland.
Nives SkuncaETH Zurich, Zurich, Switzerland.
Christophe DessimozBioinformatics Group, Department of Computer Science, University College London, London, UK.
Tunca DoganEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Kai HakalaDepartment of Information Technology, University of Turku, Turku, Finland.
Suwisa KaewphanDepartment of Information Technology, University of Turku, Turku, Finland.
Farrokh MehryaryDepartment of Information Technology, University of Turku, Turku, Finland.
Tapio SalakoskiDepartment of Information Technology, University of Turku, Turku, Finland.
Filip GinterDepartment of Information Technology, University of Turku, Turku, Finland.
Hai FangUniversity of Bristol, Bristol, UK.
Ben SmithersUniversity of Bristol, Bristol, UK.
Matt OatesUniversity of Bristol, Bristol, UK.
Julian GoughUniversity of Bristol, Bristol, UK.
Petri TörönenInstitute of Biotechnology, University of Helsinki, Helsinki, Finland.
Patrik KoskinenInstitute of Biotechnology, University of Helsinki, Helsinki, Finland.
Liisa HolmInstitute of Biotechnology, University of Helsinki, Helsinki, Finland.
Ching-Tai ChenInstitute of Information Science, Academia Sinica, Taipei, Taiwan.
Wen-Lian HsuInstitute of Information Science, Academia Sinica, Taipei, Taiwan.
Kevin BrysonBioinformatics Group, Department of Computer Science, University College London, London, UK.
Domenico CozzettoBioinformatics Group, Department of Computer Science, University College London, London, UK.
Federico MinneciBioinformatics Group, Department of Computer Science, University College London, London, UK.
David T JonesBioinformatics Group, Department of Computer Science, University College London, London, UK.
Samuel ChapmanDepartment of Computational Science and Engineering, North Carolina A&T State University, Greensboro, NC, USA.
Dukka BkcDepartment of Computational Science and Engineering, North Carolina A&T State University, Greensboro, NC, USA.
Ishita K KhanDepartment of Computer Science, Purdue University, West Lafayette, IN, USA.
Daisuke KiharaDepartment of Computer Science, Purdue University, West Lafayette, IN, USA.
Dan OferDepartment of Biological Chemistry, Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.
Nadav RappoportDepartment of Biological Chemistry, Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.
Amos SternDepartment of Biological Chemistry, Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.
Elena Cibrian-UhalteEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Paul DennyCentre for Cardiovascular Genetics, Institute of Cardiovascular Science, University College London, London, UK.
Rebecca E FoulgerCentre for Cardiovascular Genetics, Institute of Cardiovascular Science, University College London, London, UK.
Reija HietaEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Duncan LeggeEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Ruth C LoveringCentre for Cardiovascular Genetics, Institute of Cardiovascular Science, University College London, London, UK.
Michele MagraneEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Anna N MelidoniCentre for Cardiovascular Genetics, Institute of Cardiovascular Science, University College London, London, UK.
Prudence Mutowo-MeullenetEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Klemens PichlerEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Aleksandra ShypitsynaEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Biao LiBuck Institute for Research on Aging, Novato, CA, USA.
Pooya ZakeriDepartment of Electrical Engineering, STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium.
Sarah ElShalDepartment of Electrical Engineering, STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium.
Léon-Charles TrancheventInserm UMR-S1052, CNRS UMR5286, Cancer Research Centre of Lyon, Lyon, France.
Sayoni DasInstitute of Structural and Molecular Biology, University College London, London, UK.
Natalie L DawsonInstitute of Structural and Molecular Biology, University College London, London, UK.
David LeeInstitute of Structural and Molecular Biology, University College London, London, UK.
Jonathan G LeesInstitute of Structural and Molecular Biology, University College London, London, UK.
Ian SillitoeInstitute of Structural and Molecular Biology, University College London, London, UK.
Prajwal BhatCerenode Inc., Boston, MA, USA.
Tamás NepuszMolde University College, Molde, Norway.
Alfonso E RomeroDepartment of Computer Science, Centre for Systems and Synthetic Biology, Royal Holloway University of London, Egham, UK.
Rajkumar SasidharanDepartment of Molecular, Cell and Developmental Biology, University of California at Los Angeles, Los Angeles, CA, USA.
Haixuan YangSchool of Mathematics, Statistics and Applied Mathematics, National University of Ireland, Galway, Ireland.
Alberto PaccanaroDepartment of Computer Science, Centre for Systems and Synthetic Biology, Royal Holloway University of London, Egham, UK.
Jesse GillisStanley Institute for Cognitive Genomics Cold Spring Harbor Laboratory, New York, NY, USA.
Adriana E Sedeño-CortésGraduate Program in Bioinformatics, University of British Columbia, Vancouver, Canada.
Paul PavlidisDepartment of Psychiatry and Michael Smith Laboratories, University of British Columbia, Vancouver, Canada.
Shou FengDepartment of Computer Science and Informatics, Indiana University, Bloomington, IN, USA.
Juan M CejuelaDepartment for Bioinformatics and Computational Biology-I12, Technische Universität München, Garching, Germany.
Tatyana GoldbergDepartment for Bioinformatics and Computational Biology-I12, Technische Universität München, Garching, Germany.
Tobias HampDepartment for Bioinformatics and Computational Biology-I12, Technische Universität München, Garching, Germany.
Lothar RichterDepartment for Bioinformatics and Computational Biology-I12, Technische Universität München, Garching, Germany.
Asaf SalamovDOE Joint Genome Institute, Walnut Creek, CA, USA.
Toni GabaldonBioinformatics and Genomics, Centre for Genomic Regulation, Barcelona, Spain.
Marina Marcet-HoubenBioinformatics and Genomics, Centre for Genomic Regulation, Barcelona, Spain.
Fran SupekUniversitat Pompeu Fabra, Barcelona, Spain.
Qingtian GongState Key Laboratory of Genetic Engineering, Collaborative Innovation Center of Genetics and Development, Department of Biostatistics and Computational Biology, School of Life Science, Fudan University, Shanghai, China.
Wei NingState Key Laboratory of Genetic Engineering, Collaborative Innovation Center of Genetics and Development, Department of Biostatistics and Computational Biology, School of Life Science, Fudan University, Shanghai, China.
Yuanpeng ZhouState Key Laboratory of Genetic Engineering, Collaborative Innovation Center of Genetics and Development, Department of Biostatistics and Computational Biology, School of Life Science, Fudan University, Shanghai, China.
Weidong TianState Key Laboratory of Genetic Engineering, Collaborative Innovation Center of Genetics and Development, Department of Biostatistics and Computational Biology, School of Life Science, Fudan University, Shanghai, China.
Marco FaldaDepartment of Molecular Medicine, University of Padua, Padua, Italy.
Paolo FontanaResearch and Innovation Center, Edmund Mach Foundation, San Michele all'Adige, Italy.
Enrico LavezzoDepartment of Molecular Medicine, University of Padua, Padua, Italy.
Stefano ToppoDepartment of Molecular Medicine, University of Padua, Padua, Italy.
Carlo FerrariDepartment of Information Engineering, University of Padua, Padova, Italy.
Manuel GiolloDepartment of Information Engineering, University of Padua, Padova, Italy.
Damiano PiovesanDepartment of Information Engineering, University of Padua, Padova, Italy.
Silvio C E TosattoDepartment of Information Engineering, University of Padua, Padova, Italy.
Angela Del PozoInstituto De Genetica Medica y Molecular, Hospital Universitario de La Paz, Madrid, Spain.
José M FernándezSpanish National Bioinformatics Institute, Spanish National Cancer Research Institute, Madrid, Spain.
Paolo MaiettaStructural and Computational Biology Programme, Spanish National Cancer Research Institute, Madrid, Spain.
Alfonso ValenciaStructural and Computational Biology Programme, Spanish National Cancer Research Institute, Madrid, Spain.
Michael L TressStructural and Computational Biology Programme, Spanish National Cancer Research Institute, Madrid, Spain.
Alfredo BensoControl and Computer Engineering Department, Politecnico di Torino, Torino, Italy.
Stefano Di CarloControl and Computer Engineering Department, Politecnico di Torino, Torino, Italy.
Gianfranco PolitanoControl and Computer Engineering Department, Politecnico di Torino, Torino, Italy.
Alessandro SavinoControl and Computer Engineering Department, Politecnico di Torino, Torino, Italy.
Hafeez Ur RehmanNational University of Computer & Emerging Sciences, Islamabad, Pakistan.
Matteo ReAnacleto Lab, Dipartimento di informatica, Università degli Studi di Milano, Milan, Italy.
Marco MesitiAnacleto Lab, Dipartimento di informatica, Università degli Studi di Milano, Milan, Italy.
Giorgio ValentiniAnacleto Lab, Dipartimento di informatica, Università degli Studi di Milano, Milan, Italy.
Joachim W BargstenApplied Bioinformatics, Bioscience, Wageningen University and Research Centre, Wageningen, Netherlands.
Aalt D J van DijkApplied Bioinformatics, Bioscience, Wageningen University and Research Centre, Wageningen, Netherlands.
Branislava GemovicCenter for Multidisciplinary Research, Institute of Nuclear Sciences Vinca, University of Belgrade, Belgrade, Serbia.
Sanja GlisicCenter for Multidisciplinary Research, Institute of Nuclear Sciences Vinca, University of Belgrade, Belgrade, Serbia.
Vladmir PerovicCenter for Multidisciplinary Research, Institute of Nuclear Sciences Vinca, University of Belgrade, Belgrade, Serbia.
Veljko VeljkovicCenter for Multidisciplinary Research, Institute of Nuclear Sciences Vinca, University of Belgrade, Belgrade, Serbia.
Nevena VeljkovicCenter for Multidisciplinary Research, Institute of Nuclear Sciences Vinca, University of Belgrade, Belgrade, Serbia.
Danillo C Almeida-E-SilvaDepartment of Computing and Mathematics FFCLRP-USP, University of Sao Paulo, Ribeirao Preto, Brazil.
Ricardo Z N VencioDepartment of Computing and Mathematics FFCLRP-USP, University of Sao Paulo, Ribeirao Preto, Brazil.
Malvika SharanInstitute for Molecular Infection Biology, University of Würzburg, Würzburg, Germany.
Jörg VogelInstitute for Molecular Infection Biology, University of Würzburg, Würzburg, Germany.
Lakesh KansakarDepartment of Computer and Information Sciences, Temple University, Philadelphia, PA, USA.
Shanshan ZhangDepartment of Computer and Information Sciences, Temple University, Philadelphia, PA, USA.
Slobodan VuceticDepartment of Computer and Information Sciences, Temple University, Philadelphia, PA, USA.
Zheng WangUniversity of Southern Mississippi, Hattiesburg, MS, USA.
Michael J E SternbergCentre for Integrative Systems Biology and Bioinformatics, Department of Life Sciences, Imperial College London, London, UK.
Mark N WassSchool of Biosciences, University of Kent, Canterbury, Kent, UK.
Rachael P HuntleyEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Maria J MartinEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Claire O'DonovanEuropean Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
Peter N RobinsonInstitut für Medizinische Genetik und Humangenetik, Charité - Universitätsmedizin Berlin, Berlin, Germany.
Yves MoreauDepartment of Electrical Engineering ESAT-SCD and IBBT-KU Leuven Future Health Department, Katholieke Universiteit Leuven, Leuven, Belgium.
Anna TramontanoUniversity of Rome, La Sapienza, Rome, Italy.
Patricia C BabbittCalifornia Institute for Quantitative Biosciences, University of California San Francisco, San Francisco, CA, USA.
Steven E BrennerDepartment of Plant and Microbial Biology, University of California Berkeley, Berkeley, CA, USA.
Michal LinialDepartment of Chemical Biology, The Hebrew University of Jerusalem, Jerusalem, Israel.
Christine A OrengoInstitute of Structural and Molecular Biology, University College London, London, UK.
Burkhard RostDepartment for Bioinformatics and Computational Biology-I12, Technische Universität München, Garching, Germany.
Casey S GreeneDepartment of Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, Philadelphia, PA, USA.
Sean D MooneyDepartment of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA, USA.
Iddo FriedbergDepartment of Microbiology, Miami University, Oxford, OH, USA. idoerg@iastate.edu.ORCID 0000-0002-1789-8000
Predrag RadivojacDepartment of Computer Science and Informatics, Indiana University, Bloomington, IN, USA. predrag@indiana.edu.

Funding

Transform Dissemination and Implementation Science in CTSA ProgramsUL1TR002319 · NCATS · UNIVERSITY OF WASHINGTON · PI John K. Amory · 2017 to 2026
$100.0M
Institute of Translational Health SciencesUL1TR000423 · NCATS · UNIVERSITY OF WASHINGTON · PI DISIS, MARY L. · 2012 to 2016
$49.4M
Laying the Foundation of Genomic EnzymologyR01GM060595 · NIGMS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI BABBITT, PATRICIA CLEMENT · 2000 to 2018
$5.4M
Integrated Prediction of Protein Struture at 1D, 2D and 3D LevelsR01GM093123 · NIGMS · UNIVERSITY OF MISSOURI-COLUMBIA · PI CHENG, JIANLIN · 2010 to 2023
$3.8M
Neuroinformatics for gene expression: networks, function and meta-analysisR01MH111099 · NIMH · UNIVERSITY OF BRITISH COLUMBIA · PI PAVLIDIS, PAUL · 2016 to 2025
$3.6M
Informatic Profiling of Clinically Relevant MutationR01LM009722 · NLM · UNIVERSITY OF WASHINGTON · PI MOONEY, SEAN DAVID · 2007 to 2014
$3.5M
Neuroinformatics for gene expression: networks, function and meta-analysisR01GM076990 · NIGMS · UNIVERSITY OF BRITISH COLUMBIA · PI BIROL, INANC, PAVLIDIS, PAUL · 2005 to 2014
$2.0M
Protein function prediction by statistical phylogenomicsR01GM071749 · NIGMS · UNIVERSITY OF CALIFORNIA BERKELEY · PI BRENNER, STEVEN E · 2007 to 2010
$1.4M
A computational framework for predicting the impact of mutations in autismR01MH105524 · NIMH · UNIVERSITY OF CALIFORNIA, SAN DIEGO · PI IAKOUCHEVA, LILIA M, RADIVOJAC, PREDRAG · 2014 to 2016
$1.4M
Identification of protein-metabolite interactome.R01GM097528 · NIGMS · PURDUE UNIVERSITY · PI KIHARA, DAISUKE · 2011 to 2014
$1.1M
Biotechnology and Biological Sciences Research Council BB/F00964X/1Biotechnology and Biological Sciences Research Council BB/F020481/1Biotechnology and Biological Sciences Research Council BB/G022569/1Biotechnology and Biological Sciences Research Council BB/G022771/1Biotechnology and Biological Sciences Research Council BB/K004131/1Biotechnology and Biological Sciences Research Council BB/L002817/1Biotechnology and Biological Sciences Research Council BB/L018241/1Biotechnology and Biological Sciences Research Council BB/L020505/1Biotechnology and Biological Sciences Research Council BB/M015009/1British Heart Foundation RG/13/5/30112NCATS NIH HHS UL1 TR000423NCATS NIH HHS UL1 TR002319NIGMS NIH HHS R01 GM060595NIGMS NIH HHS R01 GM071749NIGMS NIH HHS R01 GM076990NIGMS NIH HHS R01 GM093123NIGMS NIH HHS R01 GM097528NIMH NIH HHS R01 MH105524NIMH NIH HHS R01 MH111099NLM NIH HHS R01 LM009722Parkinson's UK G-1307
6 · The paper itself

Abstract

backgroundA major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging.

resultsWe conducted the second critical assessment of functional annotation (CAFA), a timed challenge to assess computational methods that automatically assign protein function. We evaluated 126 methods from 56 research groups for their ability to predict biological functions using Gene Ontology and gene-disease associations using Human Phenotype Ontology on a set of 3681 proteins from 18 species. CAFA2 featured expanded analysis compared with CAFA1, with regards to data set size, variety, and assessment metrics. To review progress in the field, the analysis compared the best methods from CAFA1 to those of CAFA2.

conclusionsThe top-performing methods in CAFA2 outperformed those from CAFA1. This increased accuracy can be attributed to a combination of the growing number of experimental annotations and improved methods for function prediction. The assessment also revealed that the definition of top-performing algorithms is ontology specific, that different performance metrics can be used to probe the nature of accurate predictions, and the relative diversity of predictions in the biological process and human phenotype ontologies. While there was methodological improvement between CAFA1 and CAFA2, the interpretation of results and usefulness of individual methods remain context-dependent.

Indexed as

Computational BiologySoftwareStructure-Activity RelationshipAlgorithmsDatabases, ProteinGene OntologyHumansMolecular Sequence AnnotationProteinsProteinsDisease gene prioritizationProtein function prediction

Identifiers

PMID27604469
PMCPMC5015320

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.