CRIME-Q-a unifying tool for critical appraisal of methodological (technical) quality, quality of reporting and risk of bias in animal research.

backgroundSystematic reviews within the field of animal research are becoming more common. However, in animal translational research, issues related to methodological quality and quality of reporting continue to arise, potentially leading to underestimation or overestimation of the effects of interventions or prevent studies from being replicated. The various tools and checklists available to ensure good-quality studies and proper reporting include both unique and/or overlapping items and/or simply lack necessary elements or are too situational to certain conditions or diseases. Currently, there is no tool available, which covers all aspects of animal models, from bench-top activities to animal facilities, hence a new tool is needed. This tool should be designed to be able to assess all kinds of animal studies such as old, new, low quality, high quality, interventional and noninterventional on. It should do this on multiple levels through items on quality of reporting, methodological (technical) quality, and risk of bias, for use in assessing the overall quality of studies involving animal research.

methodsDuring a systematic review of meningioma models in animals, we developed a novel unifying tool that can assess all types of animal studies from multiple perspectives. The tool was inspired by the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) checklist, the ARRIVE 2.0 guidelines, and SYRCLE's risk of bias tool, while also incorporating unique items. We used the interrater agreement percentage and Cohen's kappa index to test the interrater agreement between two independent reviewers for the items in the tool.

resultsThere was high interrater agreement across all items (92.9%, 95% CI 91.0-94.8). Cohen's kappa index showed quality of reporting had the best mean index of 0.86 (95%-CI 0.78-0.94), methodological quality had a mean index of 0.83 (95%-CI 0.78-0.94) and finally the items from SYRCLE's risk of bias had a mean kappa index of 0.68 (95%-CI 0.57-0.79).

conclusionsThe Critical Appraisal of Methodological (technical) Quality, Quality of Reporting and Risk of Bias in Animal Research (CRIME-Q) tool unifies a broad spectrum of information (both unique items and items inspired by other methods) about the quality of reporting and methodological (technical) quality, and contains items from SYRCLE's risk of bias. The tool is intended for use in assessing overall study quality across multiple domains and items and is not, unlike other tools, restricted to any particular model or study design (whether interventional or noninterventional). It is also easy to apply when designing and conducting animal experiments to ensure proper reporting and design in terms of replicability, transparency, and validity.