The Fragility of Significance in the Hip Arthroscopy Literature: A Systematic Review.

The purpose of the present study was to perform the first examination of the utility of p values and the degree of statistical fragility in the hip arthroscopy literature by applying both the Fragility Index (FI) and the Fragility Quotient (FQ) to dichotomous comparative trials. We hypothesized that dichotomous comparative trials evaluating categorical outcomes in the hip arthroscopy literature are statistically fragile.

methodsThe PubMed and MEDLINE databases were queried from 2008-2018 for comparative studies evaluating dichotomous data in the hip arthroscopy literature. The present analysis included both randomized controlled trials (RCTs) and non-RCTs in which dichotomous data and associated p values were reported. Fragility analysis was performed with use of the Fisher exact test until an alteration of significance was determined.

resultsOf the 5,836 studies screened, 4,156 met the search criteria, with 52 comparative studies included for analysis. One hundred and fifty total outcome events with 33 significant (p < 0.05) outcomes and 117 nonsignificant (p ≥ 0.05) outcomes were identified. The final FI incorporating all 150 outcome events from 52 comparative studies was only 3.5 (interquartile range, 2 to 6), with an associated FQ of 0.032 (interquartile range, 0.017 to 0.063). Twenty-two studies (42.3%) either failed to report loss to follow-up (LTF) data or reported LTF greater than the overall FI of 3.5.

conclusionsThe peer-reviewed hip arthroscopy literature may not be as stable as previously thought, as the sole reliance on a threshold p value has proven misleading. We therefore recommend reporting of the FI and FQ, in conjunction with p values, to aid in the evaluation and interpretation of statistical robustness and quantitative significance in future comparative hip arthroscopy studies.