Evidence map›Paper›PMID 32509971›Full record

ArticleNPJ digital medicine2020

Design and testing of a mobile health application rating tool.

David M Levine, Zoe Co, Lisa P Newmark, Alissa R Groisser, A Jay Holmgren, Jennifer S Haas, David W Bates

Abstract read
In one paragraph

Article in NPJ digital medicine, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
35citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

35 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Observational
  3. Article
  4. Article
  5. Article
  6. Frameworks for assessing digital health technologies: a scoping review.International journal of technology assessment in health care · 2026
    Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Article
  15. The Internet, Apps, and the Anesthesiologist.Healthcare (Basel, Switzerland) · 2023
    Review
  16. Article
  17. Article
  18. Article
  19. Review
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

David M LevineDivision of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA USA.ORCID 0000-0002-4337-9896
Zoe CoDivision of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA USA.
Lisa P NewmarkDepartment of Clinical Quality and Analysis, Partners Healthcare System, Somerville, MA USA.
Alissa R GroisserDivision of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA USA.
A Jay HolmgrenHarvard Business School, Boston, MA USA.
Jennifer S HaasHarvard Medical School, Boston, MA USA.
David W BatesDivision of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA USA.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Mobile health applications ("apps") have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps' potentially important dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an app rating tool called THESIS that could promote informed app selection. We used a systematic process to select chronic disease apps with ≥4 stars and <4-stars and then rated them with THESIS to examine the tool's interrater reliability and internal consistency. We rated 211 apps, finding they performed fair overall (3.02 out of 5 [95% CI, 2.96-3.09]), but especially poorly for privacy/security (2.21 out of 5 [95% CI, 2.11-2.32]), interoperability (1.75 [95% CI, 1.59-1.91]), and availability in multiple languages (1.43 out of 5 [95% CI, 1.30-1.56]). Ratings using THESIS had fair interrater reliability (

Indexed as

DiagnosisHealth policyHealth servicesTherapeutics

Identifiers

PMID32509971
PMCPMC7242452

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.