ArticleInternational journal for quality in health care : journal of the International Society for Quality in Health Care2020
Do performance indicators predict regulator ratings of healthcare providers? Cross-sectional study of acute hospitals in England.
Article in International journal for quality in health care : journal of the International Society for Quality in Health Care, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
5 citing papers in PubMed.
- A Comparison of Regulatory Maternity Unit Ratings With Clinical Outcomes and Practice Measures: An Observational Study Using Routinely Collected Data.BJOG : an international journal of obstetrics and gynaecology · 2025Observational
- Analysis of Medico-Legal Complaint Data: A Retrospective Study of Three Large Italian University Hospitals.Healthcare (Basel, Switzerland) · 2023Article
- An overview and visual analysis of research on government regulation in healthcare.Frontiers in public health · 2023Review
- The measurement and improvement of maternity service performance through inspection and rating: An observational study of maternity services in acute hospitals in England.Health policy (Amsterdam, Netherlands) · 2020Observational
- Using quality indicators to predict inspection ratings: cross-sectional study of general practices in England.The British journal of general practice : the journal of the Royal College of General Practitioners · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo determine whether a large set of care performance indicators ('Intelligent Monitoring (IM)') can be used to predict the Care Quality Commission's (CQC) acute hospital trust provider ratings.
designThe IM dataset and first-inspection ratings were used to build linear and ordered logistic regression models for the whole dataset (all trusts). This was repeated for subsets of the trusts, with these models then applied to predict the inspection ratings of the remaining trusts.
settingThe United Kingdom Department of Health and Social Care's Care Quality Commission is the regulator for all health and social care services in England. We consider their first-inspection cycle of acute hospital trusts (2013-2016).
participantsAll 156 English NHS acute hospital trusts. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): Percentage of correct predictions and weighted kappa.
resultsOnly 24% of the predicted overall ratings for the test sample were correct and the weighted kappa of 0.01 indicates very poor agreement between predicted and actual ratings. This lack of predictive power is also found for each of the rating domains.
conclusionWhile hospital inspections draw on a much wider set of information, the poor power of performance indicators to predict subsequent inspection ratings may call into question the validity of indicators, ratings or both. We conclude that a number of changes to the way performance indicators are collected and used could improve their predictive value, and suggest that assessing predictive power should be undertaken prospectively when the sets of indicators are being designed and selected by regulators.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.