Adjusting for publication bias in the presence of heterogeneity

Citations Over TimeTop 1% of 2003 papers

Abstract

It is known that the existence of publication bias can influence the conclusions of a meta-analysis. Some methods have been developed to deal with publication bias, but issues remain. One particular method called 'trim and fill' is designed to adjust for publication bias. The method, which is intuitively appealing and comprehensible by non-statisticians, is based on a simple and popular graphical tool called the funnel plot. We present a simulation study designed to evaluate the behaviour of this method. Our results indicate that when the studies are heterogeneous (that is, when they estimate different effects), trim and fill may inappropriately adjust for publication bias where none exists. We found that trim and fill may spuriously adjust for non-existent bias if (i) the variability among studies causes some precisely estimated studies to have effects far from the global mean or (ii) an inverse relationship between treatment efficacy and sample size is introduced by the studies' a priori power calculations. The results suggest that the funnel plot itself is inappropriate for heterogeneous meta-analyses. Selection modelling is an alternative method warranting further study. It performed better than trim and fill in our simulations, although its frequency of convergence varied, depending on the simulation parameters.

Related Papers

• → In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias(2005)494 cited
• → Assessing Publication Bias in Meta-Analyses in the Presence of Between-Study Heterogeneity(2010)172 cited
• → Using Selection Models to Assess Sensitivity to Publication Bias: A Tutorial and Call for More Routine Use(2021)10 cited
• → Meta-Analyses with Binary Outcomes: How Many Studies Need to be Omitted to Detect a Publication Bias?(2008)9 cited
• → In response(2005)