Trial reportJournal of speech, language, and hearing research : JSLHR2023
Crowdsourced Perceptual Ratings of Voice Quality in People With Parkinson's Disease Before and After Intensive Voice and Articulation Therapies: Secondary Outcome of a Randomized Controlled Trial.
Trial report in Journal of speech, language, and hearing research : JSLHR, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 4 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
4 citing papers in PubMed, 11 citations in OpenAlex.
- Crowdsourcing and machine learning contests in Parkinson's disease research - when do they work?Journal of Parkinson's disease · 2026Review
- Does the Use of Crowdsourced Listeners Yield Different Speech Intelligibility Results Than In-Person Listeners for Typically Developing Children?Journal of speech, language, and hearing research : JSLHR · 2026Article
- Voice Outcomes of Clear Speech Cueing in Individuals With Parkinson's Disease.Journal of voice : official journal of the Voice Foundation · 2025Article
- On Acoustic Voice Quality Index measurement reliability in digital health applications: a narrative review and empirical evaluation of speech sample length requirements.Frontiers in digital health · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 1 institution in 1 country.
Funding
Abstract
purposeLimited research has examined the suitability of crowdsourced ratings to measure treatment effects in speakers with Parkinson's disease (PD), particularly for constructs such as voice quality. This study obtained measures of reliability and validity for crowdsourced listeners' ratings of voice quality in speech samples from a published study. We also investigated whether aggregated listener ratings would replicate the original study's findings of treatment effects based on the Acoustic Voice Quality Index (AVQI) measure.
methodThis study reports a secondary outcome measure of a randomized controlled trial with speakers with dysarthria associated with PD, including two active comparators (Lee Silverman Voice Treatment [LSVT LOUD] and LSVT ARTIC), an inactive comparator (untreated PD), and a healthy control group. Speech samples from three time points (pretreatment, posttreatment, and 6-month follow-up) were presented in random order for rating as "typical" or "atypical" with respect to voice quality. Untrained listeners were recruited through the Amazon Mechanical Turk crowdsourcing platform until each sample had at least 25 ratings.
resultsIntrarater reliability for tokens presented repeatedly was substantial (Cohen's κ = .65-.70), and interrater agreement significantly exceeded chance level. There was a significant correlation of moderate magnitude between the AVQI and the proportion of listeners classifying a given sample as "typical." Consistent with the original study, we found a significant interaction between group and time point, with the LSVT LOUD group alone showing significantly higher perceptually rated voice quality at posttreatment and follow-up relative to the pretreatment time point.
conclusionsThese results suggest that crowdsourcing can be a valid means to evaluate clinical speech samples, even for less familiar constructs such as voice quality. The findings also replicate the results of the study by Moya-Galé et al. (2022) and support their functional relevance by demonstrating that the effects of treatment measured acoustically in that study are perceptually apparent to everyday listeners.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.