How to Do a Meta-Analysis: Complete Step-by-Step Tutorial

Define Your Research Question (PICO Framework)

Every meta-analysis begins with a clearly formulated research question. The PICO framework is the gold standard for structuring clinical questions:

Setting Inclusion and Exclusion Criteria

Your PICO question directly determines your eligibility criteria. Before searching the literature, define these precisely:

• Study design: RCTs only? Include observational studies?
• Population specifics: Age range, disease severity, comorbidities
• Intervention details: Dose range, duration, route of administration
• Outcome measurement: How was the outcome measured? Minimum follow-up duration?
• Language: English only, or all languages?
• Publication date: Any time restrictions?

Register Your Protocol

Before starting your search, register your protocol on PROSPERO (International Prospective Register of Systematic Reviews). Registration demonstrates that your review was planned before results were known, reducing the risk of outcome reporting bias. PROSPERO registration is free and increasingly required by journals.

Develop a Literature Search Strategy

A comprehensive, reproducible search strategy is the backbone of any meta-analysis. The goal is high sensitivity (recall) -- it is better to retrieve too many irrelevant articles than to miss relevant ones.

Which Databases to Search

At minimum, search these three databases:

• PubMed / MEDLINE -- The largest biomedical literature database, indexing over 36 million citations. Essential for any health-related meta-analysis.
• Embase -- Stronger coverage of pharmacology, toxicology, and European literature. Approximately 40% of Embase content is unique (not in PubMed).
• Cochrane Central Register of Controlled Trials (CENTRAL) -- The most comprehensive source of reports of RCTs.

Depending on your topic, also consider: Web of Science, Scopus, PsycINFO (psychology), CINAHL (nursing), ClinicalTrials.gov (unpublished trial data), and conference proceedings.

Building Your Search String

Translate each PICO element into search terms. Combine synonyms with OR and PICO elements with AND:

Search Tips for Better Results

• Use both MeSH terms (controlled vocabulary) and free-text keywords to maximize recall
• Include truncation (e.g., diabet*) to capture word variations
• Do NOT use overly restrictive outcome terms in your search -- these cause missed studies
• Search reference lists of included studies and relevant reviews (backward citation searching)
• Search citing articles of key studies (forward citation searching via Google Scholar or Scopus)

Document Everything

Record the exact search string, database, date of search, and number of results for each database. PRISMA 2020 requires this level of transparency, and reviewers will ask for it.

Study Selection and Screening

Following the PRISMA 2020 flow diagram, study selection proceeds in distinct phases:

Phase 1: Deduplication

After searching multiple databases, you will have duplicate records. Use reference management software (Zotero, EndNote, or MetaReview's built-in deduplication) to identify and remove duplicates. Typically, 20-40% of combined results are duplicates.

Phase 2: Title and Abstract Screening

Rapidly screen each unique record based on its title and abstract against your inclusion criteria. At this stage, be inclusive -- if in doubt, keep it for full-text review. Two independent reviewers should screen all records separately.

Phase 3: Full-Text Review

Retrieve the full text of all potentially eligible articles. Read each one carefully against your complete inclusion/exclusion criteria. Record the specific reason for excluding each article (PRISMA 2020 requirement).

Inter-Rater Agreement

Calculate Cohen's kappa coefficient to quantify agreement between the two reviewers:

Disagreements should be resolved through discussion or by consulting a third reviewer. Aim for kappa ≥ 0.80.

Data Extraction

Data extraction is where you systematically pull the quantitative and qualitative information needed from each included study. Accuracy here is critical -- errors in data extraction propagate directly into your meta-analysis results.

What to Extract

Your extraction form should capture:

• Study identifiers: First author, publication year, journal, country
• Study characteristics: Study design (RCT, cohort, case-control), single/multi-center, funding source
• Population: Sample size, age (mean/median), sex distribution, disease severity, inclusion criteria used
• Intervention details: Drug name, dose, route, duration, comparison treatment details
• Outcome data: The numerical results needed to calculate effect sizes (see table below)
• Quality assessment data: Information needed for risk of bias evaluation

Double Extraction

Two reviewers should independently extract data from every study. After extraction, compare the results and resolve any discrepancies. This catches transcription errors, misread tables, and misinterpreted outcome definitions. Studies have shown that single-reviewer extraction has an error rate of 10-30%.

Handling Missing Data

• Contact the original study authors (allow 2-4 weeks for a response)
• Calculate SD from confidence intervals, p-values, or interquartile ranges using established formulas (see Cochrane Handbook Chapter 6)
• If SE is reported instead of SD: SD = SE × √n
• If only median and IQR are reported: use the Wan et al. (2014) method to estimate mean and SD

Choose the Right Effect Size

The effect size measure you choose determines how results are combined and interpreted. Choosing the wrong effect size is one of the most common mistakes in meta-analysis. Here is a decision framework:

Decision Tree

Is your outcome binary (yes/no) or continuous (numerical)?

• Binary outcome (e.g., dead/alive, cured/not cured, event/no event):
  • Cohort study or RCT → Risk Ratio (RR)
  • Case-control study → Odds Ratio (OR)
  • Rare events (<10% incidence) → OR and RR are approximately equal
• Continuous outcome (e.g., blood pressure, pain score, weight):
  • All studies use the same scale/unit → Mean Difference (MD)
  • Studies use different scales measuring the same concept → Standardized Mean Difference (SMD)
• Time-to-event outcome (e.g., overall survival, progression-free survival):
  • Use Hazard Ratio (HR)

Effect Size Comparison Table

For a deeper dive into effect size selection, including formulas and conversion methods, see our dedicated guide: Choosing Effect Sizes: OR, RR, MD, SMD Guide.

Statistical Analysis

This is the computational core of your meta-analysis. Three key decisions must be made: the analytical model, heterogeneity assessment, and how to visualize results.

Fixed-Effect vs. Random-Effects Model

Understanding Heterogeneity

Heterogeneity refers to variability in study results beyond what is expected from sampling error alone. Three key statistics quantify it:

Reading a Forest Plot

The forest plot is the signature visualization of a meta-analysis. Here is how to read one:

• Each row represents one study, labeled with author and year
• The square on each row is the point estimate (effect size) for that study
• The horizontal line through the square is the 95% confidence interval
• Square size reflects the study's weight in the pooled analysis (larger = more weight)
• The diamond at the bottom is the pooled effect estimate; its width shows the 95% CI
• The vertical dashed line is the line of no effect (1.0 for OR/RR/HR, 0 for MD/SMD)
• If a study's CI crosses the line of no effect, that individual study's result is not statistically significant

Subgroup Analysis

When heterogeneity is substantial (I² > 50%), subgroup analysis can help identify sources. Divide studies into groups based on pre-specified characteristics:

• Drug dosage (low vs. high)
• Study design (RCT vs. observational)
• Geographic region (Asia vs. Europe vs. North America)
• Risk of bias (low vs. high)
• Follow-up duration (short-term vs. long-term)

Use the Q-between test (interaction test) to determine if the effect truly differs between subgroups (p < 0.05).

Assess Publication Bias

Publication bias occurs because studies with positive or statistically significant results are more likely to be published than those with null or negative findings. This means the available literature may overestimate the true effect, and your meta-analysis could inherit that bias.

Funnel Plot

A funnel plot graphs each study's effect size (x-axis) against its precision, typically standard error (y-axis, inverted). In the absence of publication bias:

• Studies scatter symmetrically around the pooled effect estimate
• Small studies (low precision, bottom of plot) show greater spread
• Large studies (high precision, top of plot) cluster tightly near the pooled estimate

Asymmetry in the funnel plot -- typically a gap in the bottom-right or bottom-left corner -- suggests that small studies with unfavorable results may be missing.

Statistical Tests for Publication Bias

Trim-and-Fill Method

If publication bias is detected, the trim-and-fill method provides an adjusted estimate. It works by:

1. Identifying asymmetrically unmatched studies on the funnel plot
2. "Trimming" them and recalculating the pooled effect
3. "Filling" in the hypothetically missing studies on the opposite side
4. Recalculating the pooled estimate with the augmented dataset

The adjusted estimate shows what the pooled effect might be if publication bias were absent. A large shift from the original estimate is concerning.

Sensitivity Analysis

Sensitivity analysis tests the robustness of your meta-analysis results. The question it answers: "Would the conclusions change if we made different analytical decisions?"

Leave-One-Out Analysis

The most common sensitivity analysis method. Procedure:

1. Remove one study from the meta-analysis
2. Recalculate the pooled effect with the remaining studies
3. Repeat for every study
4. Compare all results to the original pooled estimate

If removing any single study causes the pooled effect to change direction (e.g., from significant to non-significant, or from favoring intervention to favoring control), that study is influential and must be discussed explicitly.

Other Sensitivity Analyses

• Excluding high risk-of-bias studies: Rerun the analysis with only low/moderate risk-of-bias studies. If results are consistent, conclusions are more robust.
• Model comparison: Compare fixed-effect and random-effects results. If they give substantially different conclusions, heterogeneity is driving the results.
• Cumulative meta-analysis: Add studies chronologically and observe how the pooled estimate evolves over time. Useful for identifying when the evidence stabilized.
• Influence diagnostics: Quantify each study's influence on the pooled estimate using statistics like Cook's distance, DFBETAS, or hat values (available in R's metafor package).
• Excluding outliers: If any study has a point estimate far from the pooled value (e.g., residual z-score > 2), rerun the analysis without it.

Report Results Following PRISMA 2020

The PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement provides a 27-item checklist for transparent reporting. Most medical journals require PRISMA compliance.

Key Sections of a Meta-Analysis Manuscript

1. Introduction: Rationale, objectives, PICO question
2. Methods: Protocol registration, eligibility criteria, databases searched, search strategy, screening process, data extraction methods, effect size and model choices, heterogeneity assessment approach, sensitivity and subgroup analyses planned
3. Results:
   • PRISMA flow diagram (identification, screening, eligibility, included)
   • Characteristics of included studies (Table 1)
   • Risk of bias assessment
   • Pooled effect size (95% CI), p-value
   • Heterogeneity statistics (I², Q, τ²)
   • Forest plot (main analysis)
   • Subgroup analyses with forest plots
   • Sensitivity analysis results
   • Publication bias (funnel plot + Egger's test)
4. Discussion: Summary of evidence, comparison with existing literature, strengths and limitations, implications for practice and research

Writing the Results Paragraph: Template

Here is a standard template for reporting your primary meta-analysis result:

PRISMA 2020 Checklist Highlights

For a complete PRISMA 2020 flow diagram guide, see: PRISMA 2020 Flow Diagram Guide.