ArticleJournal of medical Internet research2025
Developing Balanced Quality Indicators for Monitoring Virtual Care in Ambulatory Care Environments: Modified Delphi Panel Process.
Article in Journal of medical Internet research, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 4 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
4 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The quintuple aim as a framework for health system performance: a systematic review of empirical applications.BMC health services research · 2026Pooled it
- Developing a Core Outcome Set for the Evaluation of Remote Patient Monitoring Interventions Using the Sextuple Aim: Modified Delphi Study.Journal of medical Internet research · 2026Article
- Monitoring and evaluation of an artificial intelligence-enhanced wound care intervention in a rural health network: defining stakeholder expectations and shared priorities.Frontiers in digital health · 2026Article
- Nurse Roles in Virtual Care Within Team-Based Primary Care: Qualitative Findings From a Multiple Case Study in Two Canadian Jurisdictions.SAGE open nursingArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundWhile the volume of virtual visits increased with the COVID-19 pandemic, little is still known regarding the quality of virtual visits. Furthermore, there is limited guidance on how best to evaluate the quality of virtual care initiatives.
objectiveThe objective of this study was to curate a balanced set of quality indicators grounded in the Quintuple Aim quality framework and the National Academy of Medicine domains of quality, including sustainability, to monitor the quality of virtual care in ambulatory environments.
methodsPhase 1 involved curating a list of ambulatory virtual care quality indicators from published and grey literature, along with knowledge user feedback generated through a pre-Delphi survey; these indicators were mapped and categorized to the Quintuple Aim and National Academy of Medicine (NAM) Quality Domains. In phase 2, a 19-member pan-Canadian panel was convened as part of a 3-round modified Delphi panel process to rate and rank the quality indicators. The panel included clinicians, individuals with lived experience with the health care system, policy makers, academics, and administrators. Panelists rated indicators using the Agency for Healthcare Research and Quality measure attributes on a 9-point Likert scale in round 1, followed by a virtual Delphi panel meeting to discuss indicators before round 2 of re-rating, then a final round 3 of ranking which assessed the importance of indicators within each quality domain and subdomain. To address gaps in the literature, novel quality indicators of virtual care were identified and discussed in panel discussions, patient group consultations, and a pre-Delphi survey. For indicator advancement during the rating exercises, 75% or more of the panelists' responses in the top tertile (scores of 7-9) with a median composite score of 7 or greater was required.
resultsThere were 140 quality indicators identified in the literature which progressed through the 3 Delphi panel rounds. There was minimal attrition among Delphi panel members (17/19, 89% participated in all 3 rounds). After round 3, 25 indicators were included in the final scorecard. Aligned with the Quintuple Aim, there are 13 quality indicators on patient experience, 6 on provider experience, 2 on population health, 2 on health equity, and 2 on health system costs.
conclusionsA balanced set of 25 quality indicators of ambulatory virtual care was developed based on literature and consensus building from knowledge users across the health system. This curated set of indicators will support more comprehensive evaluations of virtual care. Organizations can use the set of indicators as part of a scorecard to compare across jurisdictions, identify priority areas, and ensure their virtual care initiatives are delivering high-quality care based on multiple domains of quality.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.