Inconsistency was more prevalent than reported: an empirical study of 57 networks with multiple treatments using the node-splitting approach and a novel interpretation index.

backgroundInconsistency has been reported to be ubiquitous in network meta-analysis. However, this evidence is based on statistical tests for inconsistency with well-documented power limitations. A novel interpretation index was developed that is founded on the Kullback-Leibler divergence measure and warrants a semi-objective decision about the extent of inconsistency as acceptably low or material when statistical tests are underpowered. The prevalence of local inconsistency was investigated using the synergy of the Bayesian node-splitting approach with the newly proposed interpretation index. The results were also contrasted with inferences drawn from the 'standard decision-making approach' about the presence of inconsistency.

methodsThe nmadb R package was considered to obtain the sample of 57 networks on a binary outcome. The Bayesian node-splitting approach was initially applied to each network to estimate the posterior distribution of direct and corresponding indirect effects for each split node alongside the inconsistency factor and between-study standard deviation (

resultsThe interpretation index revealed material inconsistency in 58% of the split nodes and 81% of the networks compared to the 'standard decision-making approach' (whether the 95% credible interval of the inconsistency factor excluded zero inconsistency) that indicated conclusive inconsistency in 4% of the split nodes and 18% of the networks. Material inconsistency was less prevalent for large

conclusionsInconsistency should be interpreted cautiously in the presence of single-study comparisons and substantial statistical heterogeneity, as a true inconsistency may be concealed. Local inconsistency should be expected and quantified using a method that aligns with the evidence structure of the network.