A comparison of frequentist and Bayesian approaches to the Personalised Randomised Controlled Trial (PRACTical)-design and analysis considerations.

backgroundMultiple treatment options frequently exist for a single medical condition with no single standard of care (SoC), rendering a classic randomised trial comparing a specific treatment to a control treatment infeasible. A novel design, the personalised randomised controlled trial (PRACTical), allows individualised randomisation lists and borrows information across patient subpopulations to rank treatments against each other without comparison to a SoC. We evaluated standard frequentist analysis with Bayesian analyses, and developed a novel performance measure, utilising the precision in treatment coefficient estimates, for treatment ranking.

methodsWe simulated trial data to compare four targeted antibiotic treatments for multidrug resistant bloodstream infections as an example. Four patient subgroups were simulated based on different combinations of patient and bacteria characteristics, which required four different randomisation lists with some overlapping treatments. The primary outcome was binary, using 60-day mortality. Treatment effects were derived using frequentist and Bayesian analytical approaches, with logistic multivariable regression. The performance measures were: probability of predicting the true best treatment, and novel proxy variables for power (probability of interval separation) and type I error (probability of incorrect interval separation). Several scenarios with varying treatment effects and sample sizes were compared.

resultsThe Frequentist model and Bayesian model using a strong informative prior, were both likely to predict the true best treatment (

conclusionsUtilising uncertainty intervals on the treatment coefficient estimates are highly conservative, limiting applicability to large pragmatic trials. Bayesian analysis performed similarly to the frequentist approach in terms of predicting the true best treatment.