ArticleJournal of medical Internet research2021
Evaluation Framework for Successful Artificial Intelligence-Enabled Clinical Decision Support Systems: Mixed Methods Study.
Article in Journal of medical Internet research, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed, 1 synthesis or guideline pooled it.
- AI for IMPACTS Framework for Evaluating the Long-Term Real-World Impacts of AI-Powered Clinician Tools: Systematic Review and Narrative Synthesis.Journal of medical Internet research · 2025Pooled it
- Advancing Human-Centered AI in Clinical Decision Support: Sociocognitive Human-in-the-Loop Study in HIV Care.Journal of medical Internet research · 2026Article
- Evaluation Frameworks for Clinical AI Incorporating Validation Strategies, Real-World Applicability, and Ethical Principles: Scoping Review.Journal of medical Internet research · 2026Article
- Metrics Used for the Evaluation of Chatbots Providing Cancer Genetic Risk Assessment and Education: Systematic Review.JMIR AI · 2026Review
- Acceptability, Usability, and Perceived Quality of Care of a Digital Decision Support System in Tele-Primary Health Care in Sindh, Pakistan: Sequential Explanatory Mixed Methods Pilot Study.JMIR formative research · 2026Article
- Interorganizational Mechanisms for Developing and Implementing Clinical Decision Support Systems in Primary Care: Exploratory, Qualitative Case Study.Journal of medical Internet research · 2026Article
- Beyond performance metrics: evaluating clinical artificial intelligence (AI) post deployment.Trauma surgery & acute care open · 2026Article
- AI-based decision models for difficult airway assessment: from research innovation to clinical implementation-a narrative review.Frontiers in medicine · 2026Review
- Accelerating Digital Mental Health: The Society of Digital Psychiatry's Three-Pronged Road Map for Education, Digital Navigators, and AI.JMIR mental health · 2025Article
- Mindbench.ai: an actionable platform to evaluate the profile and performance of large language models in a mental healthcare context.NPP - digital psychiatry and neuroscience · 2025Article
- Prescription for Digital Evolution: Transformative Recommendations for Pharmacy Practice in the Digital Age.Journal of pharmacy practice · 2025Article
- Critical activities for successful implementation and adoption of AI in healthcare: towards a process framework for healthcare organizations.Frontiers in digital health · 2025Article
- Systematic review to understand users perspectives on AI-enabled decision aids to inform shared decision making.NPJ digital medicine · 2024Article
- Recent advances in artificial intelligent strategies for tissue engineering and regenerative medicine.Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging (ISSI) · 2024Review
- Evaluation of a Computer-Aided Clinical Decision Support System for Point-of-Care Use in Low-Resource Primary Care Settings: Acceptability Evaluation Study.JMIR human factors · 2024Article
- Machine Learning Model for Readmission Prediction of Patients With Heart Failure Based on Electronic Health Records: Protocol for a Quasi-Experimental Study for Impact Assessment.JMIR research protocols · 2024Article
- Evaluating a clinical decision support point of care instrument in low resource setting.BMC medical informatics and decision making · 2023Article
- Clinical research staff perceptions on a natural language processing-driven tool for eligibility prescreening: An iterative usability assessment.International journal of medical informatics · 2023Article
- Demystifying Supervised Learning in Healthcare 4.0: A New Reality of Transforming Diagnostic Medicine.Diagnostics (Basel, Switzerland) · 2022Review
- Conformity assessment of a computer vision-based posture analysis system for the screening of postural deformation.BMC musculoskeletal disorders · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundClinical decision support systems are designed to utilize medical data, knowledge, and analysis engines and to generate patient-specific assessments or recommendations to health professionals in order to assist decision making. Artificial intelligence-enabled clinical decision support systems aid the decision-making process through an intelligent component. Well-defined evaluation methods are essential to ensure the seamless integration and contribution of these systems to clinical practice.
objectiveThe purpose of this study was to develop and validate a measurement instrument and test the interrelationships of evaluation variables for an artificial intelligence-enabled clinical decision support system evaluation framework.
methodsAn artificial intelligence-enabled clinical decision support system evaluation framework consisting of 6 variables was developed. A Delphi process was conducted to develop the measurement instrument items. Cognitive interviews and pretesting were performed to refine the questions. Web-based survey response data were analyzed to remove irrelevant questions from the measurement instrument, to test dimensional structure, and to assess reliability and validity. The interrelationships of relevant variables were tested and verified using path analysis, and a 28-item measurement instrument was developed. Measurement instrument survey responses were collected from 156 respondents.
resultsThe Cronbach α of the measurement instrument was 0.963, and its content validity was 0.943. Values of average variance extracted ranged from 0.582 to 0.756, and values of the heterotrait-monotrait ratio ranged from 0.376 to 0.896. The final model had a good fit (χ
conclusionsUser acceptance is the central dimension of artificial intelligence-enabled clinical decision support system success. Acceptance was directly influenced by perceived ease of use, information quality, service quality, and perceived benefit. Acceptance was also indirectly influenced by system quality and information quality through perceived ease of use. User acceptance and perceived benefit were interrelated.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.