External validity limitations of randomized trials cited in adult critical care clinical practice guidelines: a systematic review of guidelines published from 2015 to 2020.

Objective: Clinical practice guidelines (CPGs) support evidence-informed decision-making, but the randomized controlled trials (RCTs) cited as supporting evidence may have restricted eligibility criteria and limited external validity for clinically complex populations. This study aimed to describe exclusion criteria and underrepresented populations in RCTs cited by adult critical care CPGs published from October 2015 to October 2020. Design: Systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Data sources: PubMed and Scopus electronic databases were searched from October 9, 2015, to October 9, 2020. Eligibility criteria for selecting studies: A CPG was included if it fulfilled one of the following criteria: (1) the target population was adult critically ill patients; (2) critical care experts were involved in its development; or (3) all authors agreed that it was relevant to critical care medicine (CCM). CPGs were excluded if (1) the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was not used to develop the recommendations; (2) they were editorials, comments, meta-analyses, and reviews, duplicates, protocols, or irrelevant to CCM; (3) they were original studies that investigated the implementation of the CPGs; or (4) the topics specifically addressed critically ill child patients. Data extraction and analysis: Two independent reviewers extracted CPG characteristics, recommendations, RCT citations, and RCT exclusion criteria. Guideline quality was evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool. Descriptive statistics were used for analysis. Results: Thirty-three CPGs publications were included. The overall average score for each AGREE II domain was >70%. These CPGs contained 1,134 recommendations and cited 1,231 RCT references. Only 8.7% of RCT citations had no exclusion criteria. The most frequent exclusion-related categories were age <18 years (37.9%), pregnancy (29.5%), moribund status (19.2%), surgical history (14.8%), renal impairment (14.2%), liver dysfunction (12.6%), hemodynamic instability (10.2%), traumatic brain injury (8.4%), allergy (8.2%), and coagulopathy (7.8%). Because the included guidelines primarily targeted adult patients, age <18 years should be interpreted as age-defined non-representation rather than inappropriate exclusion. After excluding this age-defined category, pregnancy, moribund status, surgical history, renal impairment, and liver dysfunction were the most frequent underrepresented categories. Conclusion: Among adult critical care CPGs published between 2015 and 2020, the RCT evidence cited in guideline recommendations frequently did not include several clinically important populations. These findings highlight the need for clinicians to consider the external validity of the underlying evidence when applying guideline recommendations to underrepresented or clinically complex patients.