To Implement or Not to Implement? A Commentary on the Pitfalls of Judging the Value and Risks of Personalized Prognostic Statistical Models.

Unlabelled: Prognostic models in medicine have garnered significant attention, with established guidelines governing their development. However, there remains a lack of clarity regarding the appropriate circumstances for (1) creating and (2) implementing tools based on models with limited performance. This commentary addresses this gap by analyzing the pros and cons of tool development and providing a structured outline that includes critical questions to consider in the decision-making process, based on an example of patients with osteoarthritis. We propose three general justifications for the implementation of survey-based models: (1) mitigation of expectation bias among patients and clinicians, (2) advancement of personalized medicine, and (3) enhancement of existing predictive information sources. Nevertheless, it is crucial to acknowledge that implementing such models is always context-dependent and may harm certain patients, necessitating careful consideration of the withdrawal of tool development and implementation in specific cases. To facilitate the identification of these scenarios, we delineate 16 possibilities following the implementation of a personalized prognostic model and compare the consequences to a current one-size-fits-all treatment recommendation at a population level. Our analysis encompasses the possible patient benefits and harms resulting from implementing or not implementing personalized prognostic models and summarizes them. These findings, together with context-related factors, are important to consider when deciding if, how, and for whom a personalized prognostic tool should be created and implemented. We present a checklist of questions and an Excel sheet calculation table, allowing researchers to weigh the benefits and harms of creating and implementing a personalized prognostic model at a population level against one-size-fits-all standard care in a structured and standardized manner. We condense this into a single value using a uniform Benefit-Risk Score formula. Together with context-related factors, the calculation table and formula are designed to aid researchers in their decision-making process on providing a personalized prognostic tool and deciding for or against its complete or partial implementation. This work serves as a foundation for further discourse and refinement of tool development decisions for prognostic models in health care.