Evidence map›Paper›PMID 26072506›Full record

ArticleBioinformatics (Oxford, England)2015

Using collective expert judgements to evaluate quality measures of mass spectrometry images.

Andrew Palmer, Ekaterina Ovchinnikova, Mikael Thuné, Régis Lavigne, Blandine Guével, Andrey Dyatlov, Olga Vitek, Charles Pineau, Mats Borén, Theodore Alexandrov

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Bioinformatics (Oxford, England), 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers.

0numbers the graph read from it
0cells of the map it votes in
6citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

6 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Andrew PalmerEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Ekaterina OvchinnikovaEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Mikael ThunéEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Régis LavigneEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Blandine GuévelEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Andrey DyatlovEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Olga VitekEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Charles PineauEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Mats BorénEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA.
Theodore AlexandrovEuropean Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology, Karlsruhe, Germany, Denator, Uppsala, Sweden, Protim, Inserm U1085 - Irset, University of Rennes 1, Rennes, France, SCiLS GmbH, Bremen, Germany, College of Computer and Information Science, Northeastern University, Boston, MA, USA and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California San Diego, La Jolla, CA, USA European Molecular Biology Laboratory, Heidelberg, Germany, Center for Industrial Mathematics, University of Bremen, Bremen, Germany, High Performance Humanoid Technologies Lab, Institute for Anthropomatics, Karlsruhe Institute of Technolo

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

motivationImaging mass spectrometry (IMS) is a maturating technique of molecular imaging. Confidence in the reproducible quality of IMS data is essential for its integration into routine use. However, the predominant method for assessing quality is visual examination, a time consuming, unstandardized and non-scalable approach. So far, the problem of assessing the quality has only been marginally addressed and existing measures do not account for the spatial information of IMS data. Importantly, no approach exists for unbiased evaluation of potential quality measures.

resultsWe propose a novel approach for evaluating potential measures by creating a gold-standard set using collective expert judgements upon which we evaluated image-based measures. To produce a gold standard, we engaged 80 IMS experts, each to rate the relative quality between 52 pairs of ion images from MALDI-TOF IMS datasets of rat brain coronal sections. Experts' optional feedback on their expertise, the task and the survey showed that (i) they had diverse backgrounds and sufficient expertise, (ii) the task was properly understood, and (iii) the survey was comprehensible. A moderate inter-rater agreement was achieved with Krippendorff's alpha of 0.5. A gold-standard set of 634 pairs of images with accompanying ratings was constructed and showed a high agreement of 0.85. Eight families of potential measures with a range of parameters and statistical descriptors, giving 143 in total, were evaluated. Both signal-to-noise and spatial chaos-based measures performed highly with a correlation of 0.7 to 0.9 with the gold standard ratings. Moreover, we showed that a composite measure with the linear coefficients (trained on the gold standard with regularized least squares optimization and lasso) showed a strong linear correlation of 0.94 and an accuracy of 0.98 in predicting which image in a pair was of higher quality. AVAILABILITY AND IMPLEMENTATION: The anonymized data collected from the survey and the Matlab source code for data processing can be found at: https://github.com/alexandrovteam/IMS_quality.

Indexed as

Observer VariationAnimalsBrainHumansImage Processing, Computer-AssistedMiceMolecular ImagingPilot ProjectsQuality ControlRatsReference StandardsSpectrometry, Mass, Matrix-Assisted Laser Desorption-Ionization

Identifiers

PMID26072506
PMCPMC4765867

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.