ArticlePLoS computational biology2018
Crowdsourcing image analysis for plant phenomics to generate ground truth data for machine learning.
Article in PLoS computational biology, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed, 73 citations in OpenAlex.
- Petal to the metal: The slow road to automating large-scale phenology labeling for herbarium specimens.Applications in plant sciences · 2026Article
- Harnessing artificial intelligence in plant breeding: innovations in digital phenotyping and breeding methodologies.TAG. Theoretical and applied genetics. Theoretische und angewandte Genetik · 2026Review
- ColabCuraTE: an easy-to-use, web-based pipeline for the manual curation of transposable elements.Mobile DNA · 2025Article
- The pipelines of deep learning-based plant image processing.Quantitative plant biology · 2025Review
- Accelerating domain-aware electron microscopy analysis using deep learning models with synthetic data and image-wide confidence scoring.npj computational materials · 2025Article
- Teaching transposon classification as a means to crowd source the curation of repeat annotation - a tardigrade perspective.Mobile DNA · 2024Article
- Deep learning to overcome Zernike phase-contrast nanoCT artifacts for automated micro-nano porosity segmentation in bone.Journal of synchrotron radiation · 2024Article
- Assessing the reliability of ChatGPT: a content analysis of self-generated and self-answered questions on clear aligners, TADs and digital imaging.Dental press journal of orthodontics · 2023Article
- Article
- The Challenge of Data Annotation in Deep Learning-A Case Study on Whole Plant Corn Silage.Sensors (Basel, Switzerland) · 2022Article
- Artificial Intelligence-Based Assistance System for Visual Inspection of X-ray Scatter Grids.Sensors (Basel, Switzerland) · 2022Article
- Improving Crowdsourcing-Based Image Classification Through Expanded Input Elicitation and Machine Learning.Frontiers in artificial intelligence · 2022Article
- Article
- Validation and tuning of in situ transcriptomics image processing workflows with crowdsourced annotations.PLoS computational biology · 2021Article
- Ten simple rules for designing analogue science games.PLoS computational biology · 2021Article
- A Review on Human-AI Interaction in Machine Learning and Insights for Medical Applications.International journal of environmental research and public health · 2021Review
- Maximizing human effort for analyzing scientific images: A case study using digitized herbarium sheets.Applications in plant sciences · 2020Article
- Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry.GigaScience · 2020Article
- What the radiologist should know about artificial intelligence - an ESR white paper.Insights into imaging · 2019Article
- Millimeter-Level Plant Disease Detection From Aerial PhotographsFrontiers in plant science · 2019Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors at 2 institutions in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The accuracy of machine learning tasks critically depends on high quality ground truth data. Therefore, in many cases, producing good ground truth data typically involves trained professionals; however, this can be costly in time, effort, and money. Here we explore the use of crowdsourcing to generate a large number of training data of good quality. We explore an image analysis task involving the segmentation of corn tassels from images taken in a field setting. We investigate the accuracy, speed and other quality metrics when this task is performed by students for academic credit, Amazon MTurk workers, and Master Amazon MTurk workers. We conclude that the Amazon MTurk and Master Mturk workers perform significantly better than the for-credit students, but with no significant difference between the two MTurk worker types. Furthermore, the quality of the segmentation produced by Amazon MTurk workers rivals that of an expert worker. We provide best practices to assess the quality of ground truth data, and to compare data quality produced by different sources. We conclude that properly managed crowdsourcing can be used to establish large volumes of viable ground truth data at a low cost and high quality, especially in the context of high throughput plant phenotyping. We also provide several metrics for assessing the quality of the generated datasets.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.