Create funnel plots with contour enhancement, Egger's test, and trim-and-fill analysis. Free, no coding, no installation. Export publication-ready SVG or PNG.
A funnel plot is a scatter plot used in meta-analysis to visually assess whether the collection of studies included in your review is likely affected by publication bias -- the tendency for studies with statistically significant or favorable results to be published more readily than studies with null or unfavorable findings. First described by Light and Pillemer in their 1984 book Summing Up: The Science of Reviewing Research, the funnel plot has become one of the most important diagnostic tools in evidence synthesis, alongside the forest plot.
The fundamental principle behind the funnel plot is elegantly simple. In any collection of studies examining the same question, you would expect a natural relationship between study size and the variability of results. Large studies, with their greater statistical power and precision, should produce effect estimates that cluster tightly around the true underlying effect. Small studies, being less precise, should produce effect estimates that scatter more widely. When you plot these studies with effect size on one axis and a measure of precision on the other, the resulting shape should resemble a symmetric, inverted funnel -- wide at the bottom (imprecise small studies) and narrow at the top (precise large studies), with the apex centered on the true effect.
Publication bias is one of the most serious threats to the validity of a meta-analysis. If studies with non-significant results are systematically less likely to be published, the pool of available literature represents a biased sample of all research conducted on the topic. A meta-analysis that synthesizes only published studies will inherit this bias, potentially overestimating the true effect size and leading to incorrect conclusions.
The consequences of undetected publication bias are not merely academic. In clinical medicine, an overestimated treatment effect can lead to adoption of ineffective or harmful interventions. In public health policy, biased evidence synthesis can misallocate resources. In pharmacology, publication bias has contributed to the approval of drugs whose benefits were later found to be smaller than initially reported. The funnel plot, together with formal statistical tests, provides a first line of defense against these outcomes.
When Light and Pillemer introduced the concept in 1984, they proposed plotting sample size against effect size as a simple visual check for bias. The method was refined throughout the 1990s, with important contributions from Begg and Mazumdar (1994), who developed a rank correlation test for funnel plot asymmetry, and Egger and colleagues (1997), who proposed a regression-based test that became the most widely used formal assessment of funnel plot asymmetry. Sterne and Egger (2001) recommended using standard error rather than sample size on the y-axis, as it more directly relates to the expected shape of the funnel under the null hypothesis of no bias. More recently, Peters and colleagues (2008) introduced the contour-enhanced funnel plot, which overlays regions of statistical significance to help distinguish publication bias from other causes of asymmetry.
Today, the funnel plot is a standard component of any well-conducted meta-analysis. The PRISMA 2020 reporting guidelines require authors to assess the risk of bias due to missing results, and funnel plots with accompanying statistical tests are the primary method for doing so. The Cochrane Handbook for Systematic Reviews of Interventions dedicates an entire chapter to detecting reporting biases and recommends funnel plots when the meta-analysis includes at least 10 studies.
The expected funnel shape arises from basic sampling theory. The standard error of an effect estimate decreases as the sample size increases. Studies with larger sample sizes produce more precise estimates (smaller standard errors) that cluster near the true effect. Studies with smaller sample sizes produce less precise estimates (larger standard errors) that are distributed more widely around the true effect. Crucially, in the absence of selection bias, this wider distribution should be symmetric -- some small studies will overestimate the effect and some will underestimate it, with roughly equal probability.
When this symmetry is violated -- when one side of the funnel is emptier than the other -- something has preferentially removed or prevented the publication of studies with certain characteristics. The most common explanation is publication bias: small studies that failed to find a significant effect were filed away in researchers' drawers (the "file drawer problem" described by Rosenthal in 1979) or were never submitted for publication because the authors or editors judged them as uninteresting. The funnel plot makes this absence visible.
Understanding the components of a funnel plot is essential for both creating and interpreting one correctly. Unlike a forest plot, which displays study results in a tabular format, the funnel plot is a true scatter plot where the position of each point conveys two pieces of information simultaneously: the study's effect estimate and its precision.
The horizontal axis represents the effect size estimated by each study. This can be any standard meta-analytic effect measure:
A vertical reference line is drawn at the pooled effect estimate from the meta-analysis. In a symmetric funnel, the points should be distributed evenly on both sides of this line. Some funnel plots also include a second reference line at the null effect (log OR = 0, MD = 0) for context.
The vertical axis represents a measure of study precision. The most common choice is the standard error (SE) of the effect estimate, plotted with smaller values (higher precision) at the top and larger values (lower precision) at the bottom. This orientation produces the characteristic inverted funnel shape with the most precise studies at the apex.
Alternative y-axis measures include:
The Cochrane Handbook and most methodological guidelines recommend using standard error on the y-axis (inverted, with 0 at the top) because it has the most direct relationship with the expected funnel shape and because the pseudo confidence interval bounds form straight lines when SE is used.
Most funnel plots include diagonal lines extending from the apex of the funnel that represent the expected pseudo 95% confidence region. These lines are calculated as the pooled effect estimate plus or minus 1.96 times the standard error at each level of precision. In the absence of heterogeneity and bias, approximately 95% of the study points should fall within these lines. Points falling outside the pseudo confidence limits are more extreme than expected and may warrant investigation.
It is important to note that these are "pseudo" confidence limits, not true statistical confidence intervals. They assume a fixed-effect model with no between-study heterogeneity. When heterogeneity is present (which is common), the scatter of points will naturally extend beyond these limits even in the absence of bias.
A symmetric funnel plot suggests that there is no systematic relationship between study size and effect size. Studies of all sizes are evenly distributed around the pooled estimate, and there is no evidence that studies with particular results are missing. This is the expected pattern when publication and reporting decisions are independent of the results obtained.
An asymmetric funnel plot has a noticeably different density of points on one side compared to the other, typically at the bottom (among small, imprecise studies). The most common pattern in medical meta-analyses is a gap in the bottom-right region (for protective interventions with OR < 1), suggesting that small studies showing no benefit or harm were not published. However, asymmetry can also appear in the bottom-left region, or in other patterns depending on the direction of the expected effect and the specific mechanisms driving the bias.
It is critical to understand that funnel plot asymmetry does not automatically equal publication bias. There are several legitimate reasons why a funnel plot might be asymmetric, and distinguishing between these causes requires careful analysis and contextual judgment. We discuss these alternative explanations in the Interpretation section below.
Over the decades since Light and Pillemer's original proposal, several variations of the funnel plot have been developed to address different analytical questions and to improve the diagnostic value of the visualization. Understanding these variants allows you to choose the most informative funnel plot for your specific situation.
The standard funnel plot is the most basic and widely used version. It plots each study's effect size on the x-axis against its standard error on the y-axis (inverted), with a vertical line at the pooled estimate and diagonal pseudo 95% confidence limits. This is the funnel plot described in most textbooks and the version required by the Cochrane Handbook as a minimum for reporting.
When to use: Always. The standard funnel plot should be the starting point for any assessment of publication bias. It provides a clear, uncluttered view of the distribution of studies and makes asymmetry easy to spot visually. Present it as your primary funnel plot, and use the enhanced versions below for additional diagnostic information.
Limitations: The standard funnel plot relies entirely on visual inspection, which is subjective. Two reviewers may disagree about whether the funnel is asymmetric, especially with fewer than 20 studies. It also does not help distinguish publication bias from other causes of asymmetry.
The contour-enhanced funnel plot, introduced by Peters, Sutton, Jones, Abrams, and Rushton in 2008, is an important advancement that adds shaded regions representing levels of statistical significance to the standard funnel plot. Typically, three contour regions are displayed:
The area outside all contour regions represents non-significant results (p ≥ 0.10).
Why this matters: The contour-enhanced funnel plot addresses a critical limitation of the standard funnel plot by helping you determine why the funnel is asymmetric, not just whether it is asymmetric. The logic works as follows:
When to use: Whenever you detect or suspect funnel plot asymmetry, or as a routine enhancement to the standard funnel plot. Contour-enhanced funnel plots provide strictly more information than standard funnel plots and are increasingly expected by journal reviewers.
The trim-and-fill funnel plot displays the results of the Duval and Tweedie (2000) trim-and-fill method. This method estimates the number of "missing" studies needed to make the funnel plot symmetric and imputes these hypothetical studies on the plot. The imputed studies are typically shown as open circles (to distinguish them from the real studies shown as filled circles), and an adjusted pooled estimate is calculated that includes both real and imputed studies.
The trim-and-fill algorithm works in three iterative steps:
When to use: As a sensitivity analysis when the standard funnel plot shows asymmetry. The trim-and-fill method provides a quantitative estimate of how many studies might be missing and how the pooled estimate would change if those studies were included. However, it should not be treated as a definitive correction for publication bias -- it is an exploratory tool with important limitations (discussed in the Statistical Tests section below).
Some researchers overlay the Egger's regression line on the funnel plot to provide a visual representation of the formal statistical test for asymmetry. This line is fitted by regressing the standardized effect estimates (effect size divided by its standard error, also known as the z-score) against precision (1/SE). When plotted on the standard funnel plot axes, this regression appears as a line whose angle reflects the degree of asymmetry.
If the funnel is symmetric, the regression line should pass through the pooled estimate at the top of the funnel and remain centered. If there is asymmetry, the line will be tilted, with the direction of tilt indicating which side has excess studies. The intercept of the regression (when extrapolated to infinite precision) estimates the pooled effect in the absence of small-study effects.
When to use: As a supplement to the standard funnel plot when you want to show the Egger's test result visually rather than reporting it only as a number in the text. This can be particularly effective in presentations where the audience may not be familiar with the statistical details of Egger's test.
| Funnel Plot Type | Primary Question | Best For | Limitations |
|---|---|---|---|
| Standard | Is the funnel symmetric? | Initial visual assessment, routine reporting | Subjective; cannot distinguish causes of asymmetry |
| Contour-enhanced | Is asymmetry due to publication bias or other factors? | Distinguishing publication bias from heterogeneity | Still partly subjective; requires understanding of significance regions |
| Trim-and-fill | How many studies might be missing and what is the adjusted effect? | Sensitivity analysis; estimating potential impact of bias | Assumes asymmetry is entirely due to bias; unstable with high heterogeneity |
| Egger's regression line | What does the formal asymmetry test look like visually? | Visual communication of Egger's test result | Adds complexity to the plot; may confuse non-statistical audiences |
Interpreting a funnel plot requires both pattern recognition and critical thinking. A common mistake is to equate any asymmetry with publication bias. In reality, funnel plot interpretation is more nuanced, and the implications of asymmetry depend heavily on the context of your meta-analysis.
A symmetric funnel plot displays an even distribution of studies around the pooled estimate at all levels of precision. The scatter of points widens evenly toward the bottom, and there are no obvious gaps on either side. The pseudo 95% confidence lines contain approximately 95% of the studies.
Interpretation: There is no visual evidence of publication bias or small-study effects. This does not prove that publication bias is absent -- it only means that the available evidence does not suggest it. Publication bias can still exist even with a symmetric funnel plot if, for example, large studies are also subject to selective reporting, or if the bias affects all study sizes equally.
In a meta-analysis of a treatment expected to reduce an outcome (OR < 1 or negative MD), the most common asymmetry pattern is a gap in the bottom-right corner of the funnel. This region corresponds to small studies that found no benefit or a harmful effect. The absence of these studies suggests they were either not conducted, not submitted for publication, or rejected by journals because their results were "not interesting."
Interpretation: This pattern is most consistent with classical publication bias. The pooled estimate may be overestimating the true beneficial effect of the intervention because the studies that would have pulled the estimate toward the null are missing. If the contour-enhanced funnel plot shows that the gap falls in regions of statistical non-significance, the case for publication bias is strengthened.
Less commonly, the gap appears in the bottom-left corner, meaning small studies that showed a strong protective effect are missing. This can occur when the research community expects a treatment to be harmful or ineffective, and small studies showing unexpected benefit are viewed skeptically and less likely to be published.
Interpretation: This pattern may indicate publication bias in the opposite direction -- beneficial findings being suppressed. It can also arise when the exposure or intervention being studied is generally considered harmful (e.g., environmental pollutants), and small studies showing no harm are more readily published. Context matters enormously when interpreting this pattern.
Some funnel plots show a "hollow" pattern where studies cluster at the extreme edges of the funnel rather than filling in the center. There are large studies with precise estimates and small studies with extreme estimates, but few medium-sized studies with moderate effects.
Interpretation: This pattern is unusual and may reflect a genuine bimodal distribution of true effects (e.g., the intervention works very well in some contexts and not at all in others) rather than publication bias. It can also occur when studies are drawn from very different populations or use different intervention intensities.
This is one of the most important points in funnel plot interpretation: asymmetry does not equal publication bias. There are several legitimate, non-bias-related reasons why a funnel plot might be asymmetric.
If the true effect genuinely varies across studies and this variation is correlated with study size, the funnel plot will be asymmetric even without any publication bias. For example, smaller studies may use higher doses of a drug (because they can afford to give more medication to fewer patients), leading to genuinely larger effects in smaller studies. This is not bias -- it is a real clinical phenomenon.
Smaller studies often have lower methodological quality: less rigorous randomization, inadequate blinding, higher risk of performance and detection bias. These methodological weaknesses tend to inflate effect sizes. If smaller studies consistently have larger effects because they have lower quality, the funnel will be asymmetric, but the asymmetry reflects quality differences rather than selective publication.
With fewer than 10 to 15 studies, the funnel plot is highly susceptible to chance variation. A few studies landing on one side by random chance can create the appearance of asymmetry. This is why both the Cochrane Handbook and PRISMA guidelines recommend against formal asymmetry testing with fewer than 10 studies.
Rather than entire studies being unpublished, individual outcomes within published studies may be selectively reported or omitted. A study may measure ten outcomes but only report the three that showed significant effects. This creates a form of reporting bias that can produce funnel plot asymmetry, but it operates at the outcome level rather than the study level.
Some effect measures naturally produce asymmetric funnel plots even without bias. For example, odds ratios and risk ratios can show artificial correlation between the effect estimate and its standard error, especially when events are rare or common. Peters' test was specifically developed to address this artifact for binary outcomes.
Visual inspection of funnel plots is inherently subjective. Two experienced researchers may disagree about whether a given funnel plot is asymmetric. To complement the visual assessment, several formal statistical tests have been developed to quantify funnel plot asymmetry. Each test has different statistical properties, assumptions, and appropriate use cases.
Egger's test (Egger, Davey Smith, Schneider, and Minder, 1997) is the most widely used statistical test for funnel plot asymmetry. It performs a weighted linear regression of the standardized effect estimates against their precision.
The model:
Interpretation: If the intercept a is significantly different from zero (p < 0.10), the funnel plot is considered asymmetric. A positive intercept indicates that smaller, less precise studies tend to report larger positive effects than larger studies. A negative intercept indicates the opposite pattern.
Why p < 0.10? The conventional threshold is 0.10 rather than 0.05 because Egger's test has limited statistical power, especially when the number of studies is small. Using a stricter threshold of 0.05 would miss too many genuine cases of asymmetry (high false negative rate). Even at 0.10, the test has poor power with fewer than 10 studies.
Strengths: Well-established, widely recognized, straightforward to compute, and included in most meta-analysis software.
Limitations: Can produce false positives when heterogeneity is substantial, when effects are measured as odds ratios with rare events, or when the number of studies is very small. Not recommended for meta-analyses with fewer than 10 studies.
Begg and Mazumdar's test (1994) takes a non-parametric approach. It calculates the rank correlation (Kendall's tau) between the standardized effect estimates and their variances.
Interpretation: A significant positive correlation (tau > 0, p < 0.10) indicates that studies with larger variances (smaller, less precise studies) tend to report larger effects, which is the pattern expected under publication bias.
Strengths: Non-parametric -- makes fewer distributional assumptions than Egger's test. Less sensitive to outliers.
Limitations: Generally less powerful than Egger's test, meaning it is more likely to miss genuine asymmetry. This lower power makes it less useful as a standalone test. It is best used as a supplementary check alongside Egger's test.
Peters' test (Peters, Sutton, Jones, Abrams, and Rushton, 2006) was developed specifically for meta-analyses of binary outcomes (odds ratios, risk ratios). It addresses a known limitation of Egger's test: when applied to odds ratios, Egger's test can produce false positives because there is an inherent mathematical correlation between the log odds ratio and its standard error, especially when events are rare.
Peters' test regresses the effect estimates against the inverse of the total sample size (1/n) rather than against precision (1/SE), which breaks the spurious correlation that plagues Egger's test for binary outcomes.
When to use: Whenever your meta-analysis uses odds ratios, risk ratios, or other ratio measures derived from 2x2 tables, Peters' test is preferable to Egger's test. For continuous outcomes (MD, SMD), Egger's test remains the standard choice.
The trim-and-fill method (Duval and Tweedie, 2000) goes beyond testing for asymmetry -- it attempts to correct for it by estimating the number and locations of missing studies and providing an adjusted pooled estimate.
Interpretation: If trim-and-fill imputes zero studies, there is no evidence of asymmetry by this method. If it imputes studies and the adjusted pooled estimate changes substantially (e.g., the confidence interval now includes the null), the conclusions of the meta-analysis may be sensitive to publication bias. If the adjusted estimate changes only slightly, the conclusions are robust even if some studies are missing.
Strengths: Provides a tangible, quantitative sensitivity analysis. Easy to understand and explain to non-statistical audiences. Produces a visually informative funnel plot with imputed studies.
Limitations: The method assumes that funnel plot asymmetry is entirely due to publication bias, which is often not the case. It can impute "missing" studies that do not actually exist, leading to an over-correction. It performs poorly when between-study heterogeneity is substantial. The adjusted estimate should be interpreted as a sensitivity analysis, not as the "corrected" truth.
P-curve analysis (Simonsohn, Nelson, and Simmons, 2014) takes a fundamentally different approach to assessing publication bias. Rather than looking at the distribution of effect sizes, it examines the distribution of statistically significant p-values across the included studies. The key insight is that when a real effect exists, the distribution of significant p-values should be right-skewed (more p-values near 0.01 than near 0.05). When significant p-values are obtained through p-hacking, selective reporting, or other questionable research practices, the distribution tends to be flat or left-skewed (more p-values near 0.05).
Strengths: Robust to many forms of publication bias that affect traditional funnel plots. Specifically designed to detect p-hacking and selective reporting. Does not rely on the relationship between effect size and study size.
Limitations: Only uses statistically significant studies, discarding information from non-significant studies. Requires the exact p-values to be reported (or reconstructable) in the original papers. Less established than Egger's test and not yet routinely expected by journal reviewers.
| Test | What It Tests | Best For | Minimum Studies | Threshold |
|---|---|---|---|---|
| Egger's | Linear asymmetry | Continuous outcomes (MD, SMD) | 10+ | p < 0.10 |
| Begg's | Rank correlation | Supplementary to Egger's | 10+ | p < 0.10 |
| Peters' | Asymmetry for binary data | OR, RR from 2x2 tables | 10+ | p < 0.10 |
| Trim-and-fill | Missing studies | Sensitivity analysis | 10+ | N/A (descriptive) |
| P-curve | P-hacking, selective reporting | Questionable research practices | Varies | Binomial/continuous tests |
Researchers have multiple options for generating funnel plots. The right choice depends on your technical comfort level, budget, and analytical requirements. Below is a detailed comparison of the most widely available tools, covering every feature relevant to funnel plot creation and publication bias assessment.
| Feature | MetaReview | R / metafor | RevMan 5 | Stata | CMA | OpenMeta-Analyst |
|---|---|---|---|---|---|---|
| Cost | Free | Free (open source) | Free (Cochrane) | $295-$895/yr | $1,495+ (one-time) | Free (open source) |
| Coding Required | No | Yes (R) | No | Yes (Stata) | No | No |
| Installation | None (browser) | R + packages | Desktop | Desktop | Desktop | Desktop (Java) |
| Standard Funnel Plot | Yes | Yes | Yes | Yes | Yes | Yes |
| Contour Enhancement | Yes | Yes | No | Yes (user-written) | No | No |
| Trim-and-Fill | Yes | Yes | No | Yes | Yes | No |
| Egger's Test | Yes | Yes | No | Yes | Yes | Yes |
| Begg's Test | Yes | Yes | No | Yes | Yes | Yes |
| Peters' Test | Yes | Yes | No | Yes (user-written) | No | No |
| Interactive Plot | Yes | Limited (plotly) | No | No | No | No |
| Export SVG | Yes | Yes | No (image only) | No (PDF/EPS) | No (image only) | No (image only) |
| Export PNG | Yes | Yes | Yes | Yes | Yes | Yes |
| Y-Axis Options | SE, 1/SE, N | All (fully flexible) | SE only | All (flexible) | SE, 1/SE, N | SE only |
| Plot Customization | Moderate | Unlimited | Limited | High | Moderate | Limited |
| Learning Curve | 5 minutes | Days to weeks | 1 hour | Days to weeks | 1-2 hours | 30 minutes |
| Best For | Fast, no-code results | Power users | Cochrane reviews | Statisticians | Funded labs | Legacy projects |
MetaReview stands out as the most complete free, no-code funnel plot maker available. It supports all the key features that researchers need -- standard funnel plots, contour-enhanced funnel plots, trim-and-fill analysis, Egger's test, and Begg's test -- all through a point-and-click interface in the browser. No installation, no coding, no account registration. You enter your data, run the meta-analysis, switch to the funnel plot tab, and have a publication-ready figure ready for export in under 10 minutes.
What sets MetaReview apart from other no-code options like RevMan and CMA is the combination of completeness and accessibility. RevMan does not support contour enhancement, trim-and-fill, or Egger's test. CMA supports these features but costs over $1,000. OpenMeta-Analyst is free but requires Java installation, lacks contour enhancement, and is no longer actively maintained. MetaReview provides all of these capabilities for free, in the browser, with modern UX.
The R package metafor by Wolfgang Viechtbauer remains the gold standard for advanced publication bias assessment. It supports every funnel plot variant, every statistical test, multiple trim-and-fill estimators (R0, L0, Q0), and allows unlimited customization of plot appearance through base R or ggplot2 graphics. If you need meta-regression-based funnel plots, radial plots, or other specialized visualizations, metafor is the only tool that supports them all.
However, this power comes at a cost: you need to be comfortable writing R code. Generating a contour-enhanced funnel plot with trim-and-fill overlay in metafor requires approximately 10-15 lines of code and a solid understanding of the function parameters. For researchers without programming experience, the learning curve is measured in days, not minutes.
RevMan 5 (and RevMan Web) generates basic funnel plots, which is sufficient for Cochrane reviews where the editorial team handles additional analysis. However, RevMan does not support contour enhancement, does not perform trim-and-fill analysis, and does not compute Egger's or Begg's test. If you are using RevMan for a Cochrane review and need these features, you will need to supplement your analysis with another tool -- MetaReview or R -- for the publication bias assessment.
Stata with the metafunnel, metabias, and metatrim commands provides a comprehensive publication bias toolkit, but requires a paid license ($295-$895/year) and familiarity with Stata's command syntax. Comprehensive Meta-Analysis (CMA) is a well-designed desktop application with an intuitive GUI, but its price tag of $1,495 or more puts it out of reach for most individual researchers and students. Both tools are excellent if cost is not a constraint, but MetaReview provides comparable funnel plot capabilities at zero cost.
This section walks you through the complete process of generating a funnel plot for publication bias detection using MetaReview's free online tool. The workflow assumes you already have extracted study data ready. If you also need to create a forest plot, MetaReview generates both from the same data entry -- you do not need to enter data twice.
Open MetaReview in your web browser. No registration or account creation is required. Choose your data type (binary or continuous) and enter the data for each study.
Enter the number of events and total participants for both the intervention and control groups. For example, if a study randomized 150 patients to treatment (23 events) and 148 to control (45 events), enter those four numbers along with the study name.
Enter the mean, standard deviation, and sample size for both groups. For example, if the treatment group has a mean of -2.4, SD of 1.8, and N of 85, and the control group has a mean of -0.6, SD of 2.1, and N of 82, enter those six numbers along with the study name.
Alternatively, click the CSV import button to upload a prepared spreadsheet with all your studies at once. This is the fastest method when you have more than 5-6 studies.
Select your effect size measure (OR, RR, MD, or SMD) and analysis model (fixed-effect or random-effects). The funnel plot requires a completed meta-analysis because it needs the pooled effect estimate and the individual study standard errors, both of which are computed during the meta-analysis.
For most meta-analyses, the random-effects model is the appropriate default choice. MetaReview computes the pooled effect, confidence intervals, heterogeneity statistics (I², Q, tau²), and individual study weights automatically.
After the meta-analysis is complete, switch to the Funnel Plot view. MetaReview generates a standard funnel plot with the following elements:
Take a moment for visual inspection. Look at the overall shape: does it resemble a symmetric inverted funnel? Are there obvious gaps on one side, particularly among the smaller (less precise) studies at the bottom? This initial visual assessment will guide your next steps.
Toggle the contour enhancement option to overlay statistical significance regions on the funnel plot. MetaReview shades three regions:
The unshaded area outside all contours represents non-significant results (p ≥ 0.10).
Now re-examine the funnel. If there are gaps, do they fall in the non-significant (unshaded) areas? If yes, this strengthens the case for publication bias as the cause of the asymmetry. If the gaps are within the significance contours, other explanations (heterogeneity, methodological quality) are more likely.
Click the Egger's test button to perform the formal statistical test for funnel plot asymmetry. MetaReview reports:
How to interpret: If p < 0.10, there is statistically significant evidence of funnel plot asymmetry. If p ≥ 0.10, the test does not detect significant asymmetry. Remember that a non-significant Egger's test does not prove the absence of publication bias -- it only means the test did not detect it, which could be due to genuine absence of bias or insufficient power (especially with fewer than 15-20 studies).
Enable the trim-and-fill method to estimate the number of potentially missing studies and assess the impact on the pooled estimate. MetaReview uses the Duval and Tweedie algorithm and reports:
How to interpret: Compare the original pooled estimate with the adjusted estimate. If the adjusted estimate is substantially different -- for example, if the original OR was 0.60 (95% CI: 0.45-0.80) and the adjusted OR is 0.78 (95% CI: 0.58-1.05) -- this suggests your conclusions may be sensitive to publication bias. If the adjusted estimate is similar to the original -- for example, adjusting from OR 0.60 to OR 0.63 -- your findings are robust to potential missing studies.
Click the download button to export your funnel plot. MetaReview supports two formats:
For journal submission, always choose SVG. If your journal requires TIFF or EPS, export as SVG first and convert using Inkscape (free). The exported plot includes all annotations: contour shading (if enabled), trim-and-fill imputed studies (if enabled), reference lines, and axis labels.
You can generate multiple funnel plot exports -- for example, a standard funnel plot for the main text and a contour-enhanced version with trim-and-fill for a supplementary appendix.
Funnel plots are deceptively simple in concept but frequently misinterpreted in practice. Even experienced meta-analysts make errors that can lead to incorrect conclusions about publication bias. This section covers the most common mistakes and how to avoid them.
This is the single most common mistake in funnel plot analysis. With fewer than 10 studies, the funnel plot is too sparse to reliably distinguish genuine asymmetry from chance variation. A few studies randomly landing on one side can create a pattern that looks like publication bias but is entirely due to sampling fluctuation.
The rule: The Cochrane Handbook explicitly states that funnel plots and formal asymmetry tests (Egger's, Begg's) should not be used when there are fewer than 10 studies. With 10 to 20 studies, interpret with caution. With more than 20 studies, both visual and statistical assessments become more reliable.
What to do instead: If you have fewer than 10 studies, acknowledge in your manuscript that publication bias could not be formally assessed due to the small number of studies. You can still discuss the risk qualitatively by noting whether your search included grey literature, trial registries, and unpublished data, and whether the included studies show any patterns suggestive of selective reporting.
As discussed in the interpretation section, asymmetry has multiple possible causes. Concluding that "the funnel plot was asymmetric, indicating publication bias" is an oversimplification that reviewers will rightfully challenge. True heterogeneity, differences in methodological quality between small and large studies, chance, and the choice of effect measure can all produce asymmetric funnel plots without any publication bias being present.
Better language: "The funnel plot showed asymmetry (Egger's test p = 0.04), which may reflect publication bias, although other explanations such as genuine heterogeneity or methodological differences between small and large studies cannot be ruled out. Contour-enhanced funnel plot analysis suggested that the missing studies would have fallen in regions of statistical non-significance, supporting publication bias as the most likely explanation."
Many researchers generate a standard funnel plot, note asymmetry, and jump straight to trim-and-fill without first using a contour-enhanced funnel plot to assess why the funnel is asymmetric. This misses a crucial diagnostic step. If the asymmetry is in regions of statistical significance, applying trim-and-fill (which assumes the asymmetry is due to publication bias) would be inappropriate and could lead to incorrect "adjustments."
Best practice: Always examine a contour-enhanced funnel plot before applying trim-and-fill. If the gaps are in non-significant regions, trim-and-fill is a reasonable sensitivity analysis. If the gaps are in significant regions, focus on exploring heterogeneity instead.
Presenting a funnel plot figure without any formal statistical test is incomplete. Visual inspection is subjective -- one researcher may see asymmetry where another sees an unremarkable pattern. Statistical tests provide an objective quantitative measure that complements the visual assessment.
What to include: At a minimum, report Egger's test result (intercept, p-value) alongside the funnel plot. If using odds ratios, also report or substitute Peters' test. If asymmetry is detected, report trim-and-fill results (number of imputed studies, adjusted estimate). This combination of visual and statistical evidence is the current standard expected by PRISMA 2020 and most journals.
Meta-analyses of diagnostic test accuracy (DTA) studies use paired sensitivity and specificity as outcomes, not a single effect size. Standard funnel plots, which are designed for a single effect measure, are not appropriate for DTA meta-analyses. The Cochrane Handbook for Diagnostic Test Accuracy Reviews recommends against using standard funnel plots for DTA reviews because the bivariate nature of the data (sensitivity and specificity are correlated) violates the assumptions underlying the funnel plot.
Alternative: For DTA meta-analyses, consider using Deeks' funnel plot asymmetry test (Deeks, Macaskill, and Irwig, 2005), which is specifically designed for diagnostic accuracy studies. This test plots 1/√(effective sample size) against the log diagnostic odds ratio and uses a regression-based test for asymmetry.
Some researchers report the trim-and-fill adjusted estimate as if it were the "true" effect corrected for publication bias. This overstates what the method can deliver. Trim-and-fill is a sensitivity analysis tool, not a bias correction method. It assumes all asymmetry is due to publication bias (which is rarely entirely true), it can impute non-existent studies, and it performs poorly in the presence of substantial heterogeneity.
Correct framing: Present trim-and-fill results as a sensitivity analysis: "After adjusting for potentially missing studies using the trim-and-fill method, the pooled estimate was [adjusted result]. This sensitivity analysis suggests that [conclusions are/are not robust to potential publication bias]." Do not replace the original pooled estimate with the adjusted one.
When reporting funnel plot analyses, some manuscripts fail to specify which y-axis measure was used (SE? 1/SE? sample size?), which asymmetry test was performed, what significance threshold was applied, and whether the trim-and-fill estimator was R0 or L0. These omissions make the analysis non-reproducible.
What to report in Methods: Specify the y-axis measure (standard error is recommended), the asymmetry test(s) used (e.g., Egger's regression test), the significance threshold (p < 0.10), and whether trim-and-fill was planned as a sensitivity analysis. If using MetaReview, you can state: "Funnel plot analysis was performed using MetaReview, including Egger's regression test for asymmetry and the Duval and Tweedie trim-and-fill method as a sensitivity analysis."
A funnel plot with substantial between-study heterogeneity (I² > 50-75%) will naturally show wider scatter of points, and the pseudo 95% confidence limits (which assume no heterogeneity) will contain fewer than 95% of the studies. This extra scatter can mimic or mask asymmetry, making interpretation unreliable.
What to do: If heterogeneity is high, interpret the funnel plot with extreme caution. Consider whether the heterogeneity can be explained by subgroup or meta-regression analysis, and if so, examine funnel plots within homogeneous subgroups. Some methodologists recommend against formal asymmetry testing when I² exceeds 50%, as the tests become unreliable.
Reporting publication bias assessment correctly is a requirement of the PRISMA 2020 statement and a routine expectation of journal reviewers. This section provides a comprehensive guide to what you should include in your Methods section, Results section, and supplementary materials regarding funnel plot analysis.
The PRISMA 2020 (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement, published by Page et al. in 2021, includes specific items related to reporting biases:
Compliance with PRISMA 2020 is mandatory for many journals and is always recommended as a best practice. Failure to address reporting biases is one of the most common deficiencies flagged during peer review of meta-analyses.
In your Methods section, under a subheading such as "Assessment of Reporting Biases" or "Publication Bias Assessment," include the following:
In your Results section, present the publication bias assessment for each primary outcome with 10 or more studies. Include:
When reporting trim-and-fill results, use language that clearly frames them as a sensitivity analysis, not as a correction. Here are examples of appropriate phrasing:
When including a funnel plot as a figure in your manuscript, follow these guidelines:
Example caption: "Figure 3. Funnel plot of 15 studies examining the effect of intervention X on outcome Y. Effect sizes are log odds ratios (x-axis) plotted against standard error (y-axis). The vertical dashed line indicates the pooled estimate (log OR = -0.48). Pseudo 95% confidence limits are shown as diagonal lines. Egger's regression test indicated no significant asymmetry (intercept = -0.42, p = 0.32)."
While PRISMA 2020 provides the general framework, individual journals may have additional requirements or preferences:
Use this checklist before submitting your manuscript to ensure you have covered all the essential elements of publication bias reporting:
| Item | Section | Included? |
|---|---|---|
| Funnel plot assessment method described | Methods | |
| Y-axis measure specified (SE recommended) | Methods | |
| Statistical test(s) named with threshold | Methods | |
| Trim-and-fill planned as sensitivity analysis | Methods | |
| Software named | Methods | |
| Funnel plot figure presented | Results / Supplement | |
| Visual assessment finding described | Results | |
| Egger's/Peters' test result with p-value | Results | |
| Trim-and-fill result (if asymmetry detected) | Results | |
| Conclusion about robustness stated | Discussion | |
| Limitation if <10 studies acknowledged | Discussion |
MetaReview is a free online funnel plot maker with contour enhancement, Egger's test, and trim-and-fill analysis. Detect publication bias in under 10 minutes. No installation, no coding, no cost.
Open MetaReview - It's FreeSee a live example: Aspirin vs Placebo funnel plot (7 RCTs) →
Get notified about new features, publication bias tips, and meta-analysis guides.
No spam. Unsubscribe anytime.
A funnel plot is a scatter plot used in meta-analysis to detect publication bias and other small-study effects. Each study is represented as a point, with the effect size (such as log odds ratio, mean difference, or standardized mean difference) plotted on the x-axis and a measure of study precision (usually the standard error) on the y-axis, with the y-axis inverted so that the most precise studies appear at the top. In the absence of publication bias, the plot should resemble a symmetric inverted funnel: large, precise studies cluster tightly near the pooled estimate at the top, while smaller, less precise studies scatter more widely at the bottom but remain evenly distributed on both sides. The pooled effect estimate from the meta-analysis is drawn as a vertical reference line, and diagonal lines show the pseudo 95% confidence limits. Asymmetry in the funnel -- particularly gaps where small studies with unfavorable results would be expected to appear -- suggests that publication bias may be present, meaning that the available literature represents a biased sample of all research conducted on the topic. Funnel plots are recommended by PRISMA 2020 and the Cochrane Handbook for meta-analyses with 10 or more studies.
Interpreting funnel plot asymmetry requires both visual assessment and formal statistical testing, combined with careful consideration of alternative explanations. Visual inspection looks for gaps or sparse areas in the scatter pattern. The most common bias pattern in medical meta-analyses is a gap in the bottom corner corresponding to small studies with non-significant or unfavorable results, suggesting these studies were not published. However, asymmetry does not automatically equal publication bias. There are several legitimate causes of asymmetry. First, true heterogeneity can create asymmetry if the genuine treatment effect varies systematically with study size -- for example, if smaller studies tend to use higher intervention doses. Second, methodological quality differences between small and large studies can inflate effects in smaller studies without any selective publication. Third, with fewer than 10 studies, apparent asymmetry may simply reflect chance variation. Fourth, the choice of effect measure can create artificial asymmetry, particularly for odds ratios with rare events. To distinguish these causes, use contour-enhanced funnel plots (which overlay statistical significance regions), apply Egger's regression test or Peters' test as formal asymmetry tests, and consider whether the pattern is clinically plausible given what you know about the literature.
Egger's test, published by Matthias Egger, George Davey Smith, Martin Schneider, and Christoph Minder in 1997, is the most widely used formal statistical test for funnel plot asymmetry. The test performs a weighted linear regression where the dependent variable is the standardized normal deviate (the effect size divided by its standard error, which is essentially the z-score) and the independent variable is the precision (the inverse of the standard error, 1/SE). The key parameter is the regression intercept: if the funnel plot is symmetric, the intercept should be close to zero; if the funnel is asymmetric, the intercept will differ significantly from zero. The test uses a conventional significance threshold of p less than 0.10 rather than 0.05 because it has limited statistical power, particularly with fewer than 10 studies. A positive intercept suggests that smaller studies tend to report larger positive effects, consistent with the most common form of publication bias. Egger's test works well for continuous effect measures (mean difference, standardized mean difference) and log-transformed ratio measures, but can produce spurious results when applied to odds ratios with rare events, in which case Peters' test is recommended as an alternative. MetaReview computes Egger's test automatically and reports the intercept, standard error, t-statistic, and p-value.
The trim-and-fill method, developed by Sue Duval and Richard Tweedie and published in 2000, is a non-parametric method for estimating the number and locations of hypothetically missing studies in a meta-analysis due to publication bias, and for calculating an adjusted pooled effect that accounts for these missing studies. The algorithm is iterative: it first estimates the number of asymmetric studies (k0) using rank-based estimators, then "trims" the most extreme small studies from the side of the funnel with excess studies to create a symmetric distribution, re-estimates the pooled effect from the trimmed set, and finally "fills" by adding mirror images of the trimmed studies on the opposite side of the adjusted pooled estimate. The imputed missing studies are typically displayed on the funnel plot as open circles, distinguishing them from the real studies shown as filled circles. The method provides three key outputs: the number of imputed studies, the locations of these studies in the effect size and standard error space, and an adjusted pooled estimate with confidence interval that includes both real and imputed data. While widely used and intuitive, the trim-and-fill method should be treated as a sensitivity analysis rather than a definitive correction for bias. It assumes all asymmetry is due to publication bias, can impute studies that do not actually exist, and performs poorly when between-study heterogeneity is substantial.
The widely accepted minimum is 10 studies for a funnel plot to be meaningfully interpretable. This recommendation comes from both the Cochrane Handbook for Systematic Reviews of Interventions and simulation studies that have evaluated the performance of funnel plots and statistical tests for asymmetry. With fewer than 10 studies, the funnel plot is too sparse to distinguish genuine asymmetry from random variation, and statistical tests like Egger's test and Begg's test have very low power, meaning they will fail to detect even substantial publication bias. Between 10 and 20 studies, funnel plots become more informative but should still be interpreted cautiously, as the visual impression can be misleading and statistical tests remain underpowered. With 20 or more studies, both visual assessment and statistical tests become reasonably reliable. With 30 or more studies, the assessment is most trustworthy. If your meta-analysis includes fewer than 10 studies, do not present a funnel plot or run asymmetry tests. Instead, acknowledge this limitation explicitly in your manuscript and consider alternative approaches to assessing the risk of reporting bias, such as comparing results to registered trial protocols, searching for unpublished data, and qualitatively assessing whether the included studies represent a complete picture of the evidence.
A contour-enhanced funnel plot is an advancement over the standard funnel plot, introduced by Peters, Sutton, Jones, Abrams, and Rushton in 2008. It adds shaded regions representing conventional levels of statistical significance to the standard funnel plot display. Typically, three contour regions are shown: p less than 0.01 (darkest shading), p less than 0.05 (medium shading), and p less than 0.10 (lightest shading), with the area outside all contours representing non-significant results. The critical diagnostic value of the contour-enhanced funnel plot is its ability to help distinguish publication bias from other causes of funnel plot asymmetry. The reasoning is straightforward: if the "missing" studies (the gaps in the funnel) fall predominantly in regions of statistical non-significance (the unshaded area), this supports publication bias as the explanation, because non-significant results are the ones most likely to remain unpublished. Conversely, if the asymmetry or gaps occur within the significance contours, publication bias is a less plausible explanation, and other factors such as genuine heterogeneity, methodological quality differences, or chance variation should be explored. Contour-enhanced funnel plots are increasingly recommended as a standard enhancement to traditional funnel plots because they provide additional diagnostic information with no extra complexity in interpretation. MetaReview supports contour-enhanced funnel plots with a simple toggle option.
Yes, MetaReview is a free, browser-based meta-analysis tool that generates funnel plots without any coding, programming knowledge, or software installation. You open MetaReview in any modern web browser (Chrome, Firefox, Safari, Edge), enter your study data through a point-and-click interface, run the meta-analysis by selecting your effect size and model, and then switch to the funnel plot view. MetaReview supports all the key funnel plot features that researchers need: standard funnel plots with pseudo 95% confidence limits, contour-enhanced funnel plots with significance regions, trim-and-fill analysis with imputed studies and adjusted estimates, Egger's regression test, and Begg's rank correlation test. You can export the funnel plot as SVG (for journal submission) or PNG (for presentations). The entire process from data entry to exported funnel plot takes under 10 minutes. By comparison, creating the same funnel plot in R requires installing R, RStudio, and the metafor package, then writing approximately 10 to 20 lines of code including function calls like funnel(), regtest(), and trimfill(). Stata requires a paid license and knowledge of commands like metafunnel and metabias. MetaReview eliminates all of these barriers while providing the same analytical capabilities.
Egger's test and Begg's test are both formal statistical tests for funnel plot asymmetry, but they differ in their statistical methodology and power. Egger's test (Egger, Davey Smith, Schneider, and Minder, 1997) uses a parametric approach: it performs a weighted linear regression of the standardized normal deviates (effect size divided by standard error) against precision (1/SE) and tests whether the regression intercept differs significantly from zero. Begg's test (Begg and Mazumdar, 1994) uses a non-parametric approach: it calculates Kendall's rank correlation coefficient (tau) between the standardized effect estimates and their variances, testing whether there is a correlation between a study's effect size and its precision. In head-to-head comparisons, Egger's test is generally more powerful -- it is more likely to detect genuine funnel plot asymmetry when it exists. Begg's test makes fewer distributional assumptions and is less sensitive to outlying studies, but its lower statistical power means it misses genuine asymmetry more often. Both tests use a significance threshold of p less than 0.10 because neither has adequate power with small numbers of studies at the conventional 0.05 level. In practice, most researchers report Egger's test as the primary asymmetry test and may include Begg's test as a supplementary analysis. For meta-analyses using odds ratios from binary outcome data, both tests can produce spurious results, and Peters' test (which regresses against inverse total sample size rather than precision) is recommended as a more appropriate alternative.