Evidence map›Paper›PMID 34249371›Full record

ArticleDigital health

The NICE Evidence Standards Framework for digital health and care technologies - Developing and maintaining an innovative evidence framework with global impact.

Harriet Unsworth, Bernice Dillon, Lucie Collinson, Helen Powell, Mark Salmon, Tosin Oladapo, Lynda Ayiku, Gary Shield, Joanne Holden, Neelam Patel and 5 more

Abstract read
In one paragraph

Article in Digital health. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 93 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
93citing papers in PubMed, 5 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

93 citing papers in PubMed, 5 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Trial
  7. Trial
  8. Trial
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Review
  16. Article
  17. Article
  18. Article
  19. Article
  20. Unraveling acceptance of healthcare innovations in neurorehabilitation: results from a discrete choice experiment.The European journal of health economics : HEPAC : health economics in prevention and care · 2026
    Article

33 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors.

Harriet UnsworthDepartment of Computer Science, University of Liverpool, UK.ORCID https://orcid.org/0000-0002-1427-6783
Bernice DillonCentre for Health Technology Evaluation, NICE, Manchester, UK.
Lucie CollinsonCentre for Health Technology Evaluation, NICE, Manchester, UK.
Helen PowellCentre for Health Technology Evaluation, NICE, Manchester, UK.
Mark SalmonDigital, Information and Technology Directorate, NICE, Manchester, UK.ORCID https://orcid.org/0000-0002-7909-3769
Tosin OladapoCentre for Health Technology Evaluation, NICE, Manchester, UK.ORCID https://orcid.org/0000-0003-4226-4349
Lynda AyikuDigital, Information and Technology Directorate, NICE, Manchester, UK.ORCID https://orcid.org/0000-0002-8055-3873
Gary ShieldHealth and Social Care Directorate, NICE, Manchester, UK.
Joanne HoldenCentre for Health Technology Evaluation, NICE, Manchester, UK.
Neelam PatelMedCity, London, UK.
Mark CampbellCentre for Health Technology Evaluation, NICE, Manchester, UK.
Felix GreavesScience, Evidence and Analytics Directorate, NICE, Manchester, UK.
Indra JoshiNHSx, London, UK.
John PowellNuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK.
Alexia TonnelDigital, Information and Technology Directorate, NICE, Manchester, UK.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveIn 2018, the UK National Institute for Health and Care Excellence (NICE), in partnership with Public Health England, NHS England, NHS Improvement and others, developed an evidence standards framework (ESF) for digital health and care technologies (DHTs). The ESF was designed to provide a standardised approach to guide developers and commissioners on the levels of evidence needed for the clinical and economic evaluation of DHTs by health and care systems.

methodsThe framework was developed using an agile and iterative methodology that included a literature review of existing initiatives and comparison of these against the requirements set by NHS England; iterative consultation with stakeholders through an expert working group and workshops; and questionnaire-based stakeholder input on a publicly available draft document.

resultsThe evidence standards framework has been well-received and to date the ESF has been viewed online over 55,000 times and downloaded over 19,000 times.

conclusionsIn April 2021 we published an update to the ESF. Here, we summarise the process through which the ESF was developed, reflect on its global impact to date, and describe NICE's ongoing work to maintain and improve the framework in the context for a fast moving, innovative field.

Indexed as

clinical evidencedigital healthgeneralHealth economicshealth technology assessment

Identifiers

PMID34249371
PMCPMC8236783

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.