Evidence map›Paper›PMID 37258851›Full record

ReviewNPJ digital medicine2023

Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework.

Jordan Silberman, Paul Wicks, Smit Patel, Siavash Sarlati, Siyeon Park, Igor O Korolev, Jenna R Carl, Jocelynn T Owusu, Vimal Mishra, Manpreet Kaur and 8 more

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in NPJ digital medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 23 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
23citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

23 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Article
  6. Article
  7. Article
  8. Article
  9. Frameworks for assessing digital health technologies: a scoping review.International journal of technology assessment in health care · 2026
    Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Review
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

18 authors.

Jordan SilbermanOffice of Medical Policy and Technology Assessment, Elevance Health, Palo Alto, CA, USA. jordan.silberman@elevancehealth.com.ORCID http://orcid.org/0000-0001-7965-3722
Paul WicksWicks Digital Health, Lichfield, UK.ORCID http://orcid.org/0000-0002-2293-9284
Smit PatelDigital Medicine Society, Boston, MA, USA.
Siavash SarlatiOffice of Medical Policy and Technology Assessment, Elevance Health, Palo Alto, CA, USA.
Siyeon ParkGeisinger Health System, Danville, PA, USA.
Igor O KorolevUConn Health, Farmington, CT, USA.
Jenna R CarlBig Health Inc., San Francisco, CA, USA.
Jocelynn T OwusuLyra Health, Burlingame, CA, USA.
Vimal MishraDepartment of Medicine and Health Administration, Virginia Commonwealth University, Richmond, VA, USA.
Manpreet KaurOffice of Medical Policy and Technology Assessment, Elevance Health, Palo Alto, CA, USA.
Vincent J WilleyHealthCore, Inc., Wilmington, DE, USA.
Madalina L SucalaAstraZeneca, Inc., NY, NY, USA.
Tim R CampelloneClick Therapeutics, New York, NY, USA.
Cindy GeogheganDigital Medicine Society, Boston, MA, USA.ORCID http://orcid.org/0000-0003-0992-3728
Isaac R Rodriguez-ChavezICON plc, Blue Bell, PA, USA.ORCID http://orcid.org/0000-0003-1787-8381
Benjamin VandendriesscheDepartment of Electrical, Computer and Systems Engineering, Case Western Reserve University, Cleveland, OH, USA.ORCID http://orcid.org/0000-0003-0672-0327
Evidence DEFINED Workgroup
Jennifer C GoldsackDigital Medicine Society, Boston, MA, USA.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Dozens of frameworks have been proposed to assess evidence for digital health interventions (DHIs), but existing frameworks may not facilitate DHI evidence reviews that meet the needs of stakeholder organizations including payers, health systems, trade organizations, and others. These organizations may benefit from a DHI assessment framework that is both rigorous and rapid. Here we propose a framework to assess Evidence in Digital health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED). Designed for real-world use, the Evidence DEFINED Quick Start Guide may help streamline DHI assessment. A checklist is provided summarizing high-priority evidence considerations in digital health. Evidence-to-recommendation guidelines are proposed, specifying degrees of adoption that may be appropriate for a range of evidence quality levels. Evidence DEFINED differs from prior frameworks in its inclusion of unique elements designed for rigor and speed. Rigor is increased by addressing three gaps in prior frameworks. First, prior frameworks are not adapted adequately to address evidence considerations that are unique to digital health. Second, prior frameworks do not specify evidence quality criteria requiring increased vigilance for DHIs in the current regulatory context. Third, extant frameworks rarely leverage established, robust methodologies that were developed for non-digital interventions. Speed is achieved in the Evidence DEFINED Framework through screening optimization and deprioritization of steps that may have limited value. The primary goals of Evidence DEFINED are to a) facilitate standardized, rapid, rigorous DHI evidence assessment in organizations and b) guide digital health solutions providers who wish to generate evidence that drives DHI adoption.

Identifiers

PMID37258851
PMCPMC10232404

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.