Evaluating Data Quality by Proxy: Can We Evaluate All Dimensions of the European Medicines Agency Data Quality Framework for Registry-Based Post-Authorization Safety Studies?

purposeAn increasing number of studies based on secondary data use, including registry-based studies, have been initiated to address post-authorization regulatory commitments. We map differences across tools used in data quality (DQ) assessments, including those embedded in fitness-for-purpose (FFP) assessments, and describe their strengths and limitations for use in DQ assessments of registries considered for use in post-approval safety studies (PASS). We focus on the use case where marketing authorization holders (MAHs) cannot directly analyze patient-level data. Furthermore, we propose complementary measures to improve DQ assessment, including a set of data quality indicators (DQIs).

methodsWe analyzed the extent to which the selected tools used in registry assessments address DQ dimensions and metrics defined in the European Medicines Agency-DQ Framework (EMA-DQF). We specifically considered the use case where DQ assessment was purely based on registry documentation and qualitative due to data access restrictions.

resultsNone of the tools covered all DQ dimensions and metrics; they had limited utility for evaluating extensiveness (i.e., completeness and coverage), semantic coherence, and reliability (i.e., accuracy and plausibility). Furthermore, some supporting document requirements were more useful than others. For example, the data dictionary is useful for evaluating precision and structural coherence. In contrast, publications have limited utility in DQ assessment.

conclusionsWe propose a set of consistent definitions of information requirements and quantitative DQIs that complement existing tools that can be used for evaluating DQ throughout the registry-based PASS lifecycle. If DQIs cannot be evaluated during the preparation of the PASS protocol, uncertainties and their potential impact on the study results need to be acknowledged in the relevant study documents. Additionally, scenario mapping for the removal or replacement of registries with insufficient DQ after PASS initiation is needed.