Evidence map›Paper›PMID 32748241›Full record

ArticleBehavior research methods2021

Framework for selecting and benchmarking mobile devices in psychophysiological research.

Ian R Kleckner, Mallory J Feldman, Matthew S Goodwin, Karen S Quigley

Open access · bronzeAbstract read
In one paragraph

Article in Behavior research methods, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 1 pooled it
2.5field-weighted citation impact, top 9% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 1 synthesis or guideline pooled it, 29 citations in OpenAlex.

  1. Guideline
  2. Trial
  3. Review
  4. Article
  5. Article
  6. Article
  7. Article
  8. Assessing Stress Level Scores Against Wearables-Driven Physiological Measurements.Stress and health : journal of the International Society for the Investigation of Stress · 2025
    Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Adaptive thresholding increases sensitivity to detect changes in the rate of skin conductance responses to psychologically arousing stimuli in both laboratory and ambulatory settings.International journal of psychophysiology : official journal of the International Organization of Psychophysiology · 2024
    Article
  15. Article
  16. Article
  17. Observational
  18. Article
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

4 authors at 3 institutions in 1 country.

Ian R KlecknerCancer Control Unit, Department of Surgery, Department of Neuroscience, University of Rochester Medical Center, 265 Crittenden Blvd, Box CU 420658, Rochester, NY, 14642, USA. Ian_Kleckner@URMC.Rochester.edu.
Mallory J FeldmanNortheastern University, Boston, MA, USA.
Matthew S GoodwinNortheastern University, Boston, MA, USA.
Karen S QuigleyNortheastern University, Boston, MA, USA.
Northeastern University · USEdith Nourse Rogers Memorial Veterans Hospital · USUniversity of Rochester Medical Center · US

Funding

URCC NCORP Research BaseUG1CA189961 · NCI · UNIVERSITY OF ROCHESTER · PI Michelle C Janelsins, KAREN M. MUSTIAN · 2014 to 2026
$67.7M
Affect regulation and Beta Amyloid: Maturational Factors in Aging and Age-Related PathologyR01MH113234 · NIMH · NORTHEASTERN UNIVERSITY · PI DICKERSON, BRADFORD C, ISAACOWITZ, DEREK M · 2017 to 2021
$4.2M
Fundamental subcortical mechanisms of affective processingU01CA193632 · NCI · NORTHEASTERN UNIVERSITY · PI BARRETT, LISA FELDMAN, SATPUTE, AJAY · 2016 to 2020
$3.0M
Clinical and Translational Cancer Control Research Training ProgramT32CA102618 · NCI · UNIVERSITY OF ROCHESTER · PI Michelle C Janelsins, Gary R Morrow · 2019 to 2026
$2.7M
The Effects of Exercise on Chemotherapy-Induced Peripheral Neuropathy, Inflammation, and Interoceptive Brain CircuitryK07CA221931 · NCI · UNIVERSITY OF ROCHESTER · PI KLECKNER, IAN ROBERT · 2018 to 2022
$795k
The Role Of Interoceptive Awareness In Affect And Related Mental FacultiesF32MH096533 · NIMH · NORTHEASTERN UNIVERSITY · PI KLECKNER, IAN ROBERT · 2012 to 2014
$151k
NCI NIH HHS K07 CA221931NCI NIH HHS T32 CA102618NCI NIH HHS U01 CA193632NCI NIH HHS UG1 CA189961NIMH NIH HHS F32 MH096533NIMH NIH HHS R01 MH113234
6 · The paper itself

Abstract

Commercially available consumer electronics in (smartwatches and wearable biosensors) are increasingly enabling acquisition of peripheral physiological and physical activity data inside and outside of laboratory settings. However, there is scant literature available for selecting and assessing the suitability of these novel devices for scientific use. To overcome this limitation, the current paper offers a framework to aid researchers in choosing and evaluating wearable technologies for use in empirical research. Our seven-step framework includes: (1) identifying signals of interest; (2) characterizing intended use cases; (3) identifying study-specific pragmatic needs; (4) selecting devices for evaluation; (5) establishing an assessment procedure; (6) performing qualitative and quantitative analyses on resulting data; and, if desired, (7) conducting power analyses to determine sample size needed to more rigorously compare performance across devices. We illustrate the application of the framework by comparing electrodermal, cardiovascular, and accelerometry data from a variety of commercial wireless sensors (Affectiva Q, Empatica E3, Empatica E4, Actiwave Cardio, Shimmer) relative to a well-validated, wired MindWare laboratory system. Our evaluations are performed in two studies (N = 10, N = 11) involving psychometrically sound, standardized tasks that include physical activity and affect induction. After applying our framework to this data, we conclude that only some commercially available consumer devices for physiological measurement are capable of wirelessly measuring peripheral physiological and physical activity data of sufficient quality for scientific use cases. Thus, the framework appears to be beneficial at suggesting steps for conducting more systematic, transparent, and rigorous evaluations of mobile physiological devices prior to deployment in studies.

Indexed as

BenchmarkingWearable Electronic DevicesAccelerometryComputers, HandheldExerciseHumansAccelerometryAffectAmbulatoryBenchmarkingElectrodermal activityHeart rateMonitoringPsychophysiologyStress

Identifiers

PMID32748241
PMCPMC7854837
OpenAlexW3047247256

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.