(3 of 4) Network Meta-Analysis Series: Assessing Data
Risk of bias assessment
We will use the tool described in the Cochrane Collaboration Handbook to assess risk of bias in the included studies. The assessment will be performed by two independent reviewers and any disagreement will resolved by consensus. We will evaluate the risk of bias in the following domains: generation of allocation sequence, allocation concealment, blinding of study personnel and participants, blinding of outcome assessor, attrition, selective outcome reporting and other domains, including sponsorship bias. Where inadequate or insufficient details of allocation concealment and other characteristics of trials are provided, we may contact the trial authors to obtain further information.
Assessment of the transitivity assumption
Transitivity is one of the key underlying assumption of NMA. The results can be misleading if the network is substantially intransitive. We will investigate the distribution of clinical and methodological variables that can act as effect modifiers across treatment comparisons. We will investigate if these variables are similarly distributed across studies grouped by comparison. It is generally understood that the inclusion of placebo may violate the transitivity assumption. Consequently, the comparability of placebo-controlled studies with those that provide head-to-head evidence will be examined carefully.
Assessment of inconsistency
Consistency is the agreement between direct and indirect evidence. We will employ local as well as global methods to evaluate consistency. Local methods will be employed to detect evidence loops that are inconsistent or comparisons for which direct and indirect evidence disagree. We will apply the loop-specific approach to evaluate inconsistency within each loop of evidence and a method that separates direct evidence from indirect evidence provided by the entire network. We will also evaluate consistency in the entire network by calculating the I2 for network heterogeneity, inconsistency, and for both.
Tests for inconsistency are known to have low power, and empirical evidence has suggested that 10% of evidence loops published in the medical literature are expected to be inconsistent. Therefore, interpretation of the statistical inference about inconsistency will be carried out with caution and possible sources of inconsistency will be explored even in the absence of evidence for inconsistency.
Exploring heterogeneity and sensitivity analyses
We will explore whether treatment effects for the primary outcomes are robust in subgroup analyses and network meta-regression. Heterogeneity will be investigated with respect to key study characteristic variables such as year of study and PD severity at baseline. The sensitivity of our conclusions for the primary outcomes will be evaluated by subgroups within which the response to treatments may behave differently.
In our next blog…
we’ll discuss creating and implementing appropriate models.