ArticleBehavior research methods2017
Are all "research fields" equal? Rethinking practice for the use of data from crowdsourcing market places.
Article in Behavior research methods, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 13 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
13 citing papers in PubMed.
- Why you shouldn't trust data collected on MTurk.Behavior research methods · 2025Article
- Social cure model: testing the link between identity centrality and body appreciation in diverse sexual orientation and gender identity groups.International journal for equity in health · 2024Article
- Frustration and ennui among Amazon MTurk workers.Behavior research methods · 2023Article
- Concurrent, Convergent, and Discriminant Validity of theAssessment · 2023Article
- The fish can rot from the heart, not just the head: Exploring the detrimental impact of transgressions by leaders at multiple levels of an organization.The British journal of social psychology · 2023Article
- Recruitment and Retention in Remote Research: Learnings From a Large, Decentralized Real-world Study.JMIR formative research · 2022Article
- Does greater morning sickness predict carrying a girl? Analysis of nausea and vomiting during pregnancy from retrospective report.Archives of gynecology and obstetrics · 2021Article
- Servant Leadership and Creativity: A Study of the Sequential Mediating Roles of Psychological Safety and Employee Well-Being.Frontiers in psychology · 2021Article
- Mechanical Turk data collection in addiction research: utility, concerns and best practices.Addiction (Abingdon, England) · 2020Article
- Leader-Member Exchange Fosters Beneficial and Prevents Detrimental Workplace Behavior: Organizational Identification as the Linking Pin.Frontiers in psychology · 2020Article
- Conducting Web-Based Experiments for Numerical Cognition Research.Journal of cognition · 2019Article
- The Promise and Pitfalls of Using Crowdsourcing in Research Prioritization for Back Pain: Cross-Sectional Surveys.Journal of medical Internet research · 2017Article
- Conducting Online Behavioral Research Using Crowdsourcing Services in Japan.Frontiers in psychology · 2017Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
No grant is acknowledged in the PubMed record.
Abstract
New technologies like large-scale social media sites (e.g., Facebook and Twitter) and crowdsourcing services (e.g., Amazon Mechanical Turk, Crowdflower, Clickworker) are impacting social science research and providing many new and interesting avenues for research. The use of these new technologies for research has not been without challenges, and a recently published psychological study on Facebook has led to a widespread discussion of the ethics of conducting large-scale experiments online. Surprisingly little has been said about the ethics of conducting research using commercial crowdsourcing marketplaces. In this article, I focus on the question of which ethical questions are raised by data collection with crowdsourcing tools. I briefly draw on the implications of Internet research more generally, and then focus on the specific challenges that research with crowdsourcing tools faces. I identify fair pay and the related issue of respect for autonomy, as well as problems with the power dynamic between researcher and participant, which has implications for withdrawal without prejudice, as the major ethical challenges of crowdsourced data. Furthermore, I wish to draw attention to how we can develop a "best practice" for researchers using crowdsourcing tools.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.