Are Systematic Reviews and Meta-Analyses Appropriate Tools for Assessing Evolving Medical Device Technologies?

Systematic reviews and meta-analyses (SRMAs) provide unique insights into comparative effectiveness of diabetes treatments. However, use of these analyses may be inappropriate for assessing the value and utility of technologies that involve significant behavioral interventions and encompass rapidly evolving technologies such as real-time continuous glucose monitoring (RT-CGM). The rapid evolution of RT-CGM, compared with the time required for publication of clinical studies used in SRMAs, may preclude differentiation between past and current generations of devices. In addition, the effect of performance and usability differences between the various commercial devices on possible clinical outcomes associated with the devices are often not clearly discussed, and many of the RT-CGM studies assessed in SRMAs do not provide adequate information regarding whether and/or to what degree study subjects and clinicians were trained to use the RT-CGM and utilize the data to adjust therapy. Although numerous clinical studies have shown that the glycemic benefit of RT-CGM is related to the frequency and duration of use, a disproportionate number of RT-CGM studies included in recent SRMAs are based on the results of the intention-to-treat analyses and do not consider this fundamental behavioral component in their conclusions. Given these limitations, the generalizability of SRMA conclusions may be limited, and findings from these reports may significantly underestimate the potential glycemic benefit of current and future devices, posing challenges for coverage and reimbursement. We reviewed the potential limitations of the recent Cochrane Collaboration report on CGM, focusing on the 12 studies that assessed RT-CGM use in adults, children/adolescents or both.