Meta-analysis

Meta-analysis is defined as "a quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc., with application chiefly in the areas of research and medicine."

A meta-analyses is a subset of systematic reviews in which the results of the studies are numerically pooled.

Standards for the reporting of meta-analyses exist.

Validity of meta-analysis
Studies on the validity of meta-analyses conflict. Some of the conflict may be due to the methods used to compare the meta-analyses.

Selecting studies for inclusion
Although meta-analyses in general are very inclusive, arguments exist for only including the best trials.

Cochrane bias scale
The Cochrane Collaboration uses a six item tool.

Jadad score
The Jadad score may be used to assess quality and contains three items:
 * 1) Was the study described as randomized (this includes the use of words such as randomly, random, and randomization)?
 * 2) Was the study described as double blind?
 * 3) Was there a description of withdrawals and dropouts?

Each question is scored one point for a yes answer. In addition, for questions and 2, a point is added if the method was appropriate and a point is deducted if the method is not appropriate (e.g. not effectively randomized or not effectively double-blinded).

Studies with groups having zero events
Excluding studies with zero events total events (zero-total- event trials) or zero events in one treatment group (zero-event-trials) may exaggerate effect sizes. An alternative is to use a continuity correction. Rather than using a constant continuity correction, less bias may occur by correcting with either
 * "a function of the reciprocal of the opposite group arm size"
 * "empirical estimate of the pooled effect size from the remaining studies in the meta-analysis."

Displaying results
Study results may be grouped and displayed with a Forest plot.

Measuring consistency of study results
Consistency can be statistically tested using either the Cochran's Q or I2. The I2 is the "percentage of total variation across studies that is due to heterogeneity rather than chance." These numbers are usually displayed for each group of studies on a Forest plot.

In interpreting of the Cochran's Q, heterogeneity exists if its p-value is < 0.05 or possibly if < 0.10.

The following has been proposed for interpreting I2:
 * Low heterogeneity is I2 = 25%
 * Moderate heterogeneity is I2 = 50%
 * High heterogeneity is I2 = 75%

or according to the Handbook of the Cochrane Collaboration:
 * 0%-40%: might not be important
 * 30%-60%: may represent moderate heterogeneity
 * 50%-90%: may represent substantial heterogeneity
 * 75%-100%: considerable heterogeneity

Statistical methods exist for assessing the importance of subgroups.

Cumulative meta-analysis
Cumulative meta-analysis has been used to show that 25 off 33 randomized controlled trials of streptokinase not necessary and have shown the delay in adoption of evidence by experts.

Individual patient data meta-analysis
An individual patient data meta-analysis is "where analyses are done using original data and outcomes for each person enrolled in relevant studies; these results are then pooled in one analysis as if patients were in a single large study."

Individual patient data meta-analysis (IPD meta-analysis) may have more long lasting results than other meta-analyses.

Network meta-analysis
A network meta-analysis and Bayesian hierarchical models pool studies in order to compare to treatments that have not been directly compared. Network meta-analyses are commonly not well performed and can have misleading conclusions.

Network meta-analyses can be conducted with Bugs and OpenBugs software.

Factors associated with higher quality meta-analyses
Meta-analyses by the Cochrane Collaboration tend to be of higher quality.

Individual data meta-analyses, in which the records from individual patients are pooled together into one dataset, tend to have more stable conclusions.

Factors associated with lower quality meta-analyses
About a third of meta-analyses that happen to precede large randomized controlled trials will conflict with the results of the trial.

Conflict of interest
Meta-analyses produced with a conflict of interest are more likely to interpret results as positive.

Publication bias
Publication bias against negative studies may threaten the validity of meta-analyses that are positive and all the studies included within the meta-analysis are small.

In performing a meta-analyses, a file drawer or a funnel plot analysis may help detect underlying publication bias among the studies in the meta-analysis.

Outcome reporting bias
Meta-analyses in which a smaller proportion of included trials provide raw data for inclusion in the meta-analysis are more likely to be positive. This may be due a bias against reporting negative results.

Obsolescence
The conclusions of meta-analyses may be mitigated by research published after the search date of the meta-analysis. This may occur by the time the meta-analysis has been published. Strategies have been developed for updating meta-analyses.