Evidence map›Paper›PMID 37963198›Full record

ArticleHuman factors2024

Improving Social Bot Detection Through Aid and Training.

Ryan Kenny, Baruch Fischhoff, Alex Davis, Casey Canfield

Open access · hybridAbstract read
In one paragraph

Article in Human factors, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
0.6field-weighted citation impact, top 23% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed, 1 citations in OpenAlex.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors at 3 institutions in 1 country.

Ryan KennyUnited States Army, Fayetteville, NC, USA.
Baruch FischhoffCarnegie Mellon University, Pittsburgh, PA, USA.ORCID 0000-0002-3030-6874
Alex DavisCarnegie Mellon University, Pittsburgh, PA, USA.
Casey CanfieldMissouri University of Science and Technology, Rolla, MO, USA.ORCID 0000-0001-5325-3798
Carnegie Mellon University · USMissouri University of Science and Technology · USUnited States Army · US

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveWe test the effects of three aids on individuals' ability to detect social bots among Twitter personas: a bot indicator score, a training video, and a warning.

backgroundDetecting social bots can prevent online deception. We use a simulated social media task to evaluate three aids.

methodLay participants judged whether each of 60 Twitter personas was a human or social bot in a simulated online environment, using agreement between three machine learning algorithms to estimate the probability of each persona being a bot. Experiment 1 compared a control group and two intervention groups, one provided a bot indicator score for each tweet; the other provided a warning about social bots. Experiment 2 compared a control group and two intervention groups, one receiving the bot indicator scores and the other a training video, focused on heuristics for identifying social bots.

resultsThe bot indicator score intervention improved predictive performance and reduced overconfidence in both experiments. The training video was also effective, although somewhat less so. The warning had no effect. Participants rarely reported willingness to share content for a persona that they labeled as a bot, even when they agreed with it.

conclusionsInformative interventions improved social bot detection; warning alone did not. APPLICATION: We offer an experimental testbed and methodology that can be used to evaluate and refine interventions designed to reduce vulnerability to social bots. We show the value of two interventions that could be applied in many settings.

Indexed as

Social MediaAdultDeceptionFemaleHumansMachine LearningMaleYoung Adultdecision supportsignal detection theorysocial botssocial mediatraining

Identifiers

PMID37963198
PMCPMC11382440
OpenAlexW4388666658

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.