Meta-Analysis Based on Bayesian Model Averaging: Principles and Implementation

Abstract: Meta-analysis serves as an essential statistical methodology to synthesize independent empirical findings. It is deeply embedded across various quantitative disciplines. However, researchers routinely confront significant dilemmas regarding model selection throughout the analytical workflow. When managing across-study heterogeneity, researchers face a strict choice between fixed-effects and random-effects frameworks. Similarly, to mitigate the threat of potential publication bias, researchers choose from a diverse array of disparate correction models. At present, the field lacks a unified, standardized criterion for selecting the single optimal model configuration. Consequently, traditional approaches rely on a single chosen model and neglect model uncertainty. This omission causes overconfident statistical inferences, underestimated standard errors, and potentially biased estimates of the true population effect size. In turn, these estimation errors reduce the replicability of empirical findings.
To resolve these deep-seated methodological vulnerabilities, Bayesian Model Averaging (BMA) offers an innovative and mathematically rigorous paradigm that operates within the broader Bayesian statistical framework. Instead of an arbitrary, ex-ante selection of a single statistical model, BMA directly accommodates model uncertainty. It embeds all theoretically plausible candidate models into a single, cohesive model space. Specifically, BMA treats the model itself as a random variable within a probability space. This formulation effectively manages uncertainty in both the effect size and heterogeneity. By establishing prior probabilities for each model configuration, BMA leverages empirical data to calculate posterior model probabilities. This process quantifies the precise degree of empirical data support that each individual model receives. Ultimately, the final effect size is a weighted average across all models, where posterior probabilities directly determine individual model weights. This approach avoids the inherent biases of single-model selection. It yields a continuous measure of cumulative evidence rather than a binary accept-reject decision.
Researchers operationalize this rigorous approach in meta-analysis as robust Bayesian meta-analysis (RoBMA). RoBMA inherits the core model-averaging principles of BMA. It systematically incorporates multiple publication bias correction models, such as selection models and PET-PEESE (precision-effect test and precision-effect estimate with standard error), into the Bayesian inference framework. It avoids a forced selection of a single optimal model. Instead, the framework weights competing hypotheses within the model space based directly on empirical data. This mechanism provides rigorous quantitative evidence, specifically inclusion Bayes factors (BF), to evaluate 3 critical hypotheses concurrently to govern the validity of literature synthesis. These hypotheses test whether a true population effect exists, whether study-level heterogeneity is present, and whether publication bias contaminates the literature. RoBMA constructs a multi-dimensional model space to map every possible combination of these 3 dimensions (e.g., presence vs. absence of an effect, presence vs. absence of heterogeneity, and presence vs. absence of publication bias). As a consequence, the framework outputs model-averaged posterior estimates that fully incorporate model uncertainty, delivering an exceptionally robust and reliable evaluation of the primary effect size. 
Beyond its conceptual and theoretical advantages, the practical execution of BMA-based meta-analysis has become highly viable due to contemporary computational breakthroughs and software integration. This paper comprehensively outlines the concrete, accessible pathways for applied researchers to deploy these advanced statistical techniques in real-world research scenarios. Specifically, the open-source statistical software JASP seamlessly executes the entire analytical pipeline, which provides an intuitive and user-friendly graphical user interface (GUI) for point-and-click execution. For researchers who require scriptability and reproducibility, specialized open-source packages make the methodology fully available into the R programming language. These programming tools facilitate automated reporting and extensive sensitivity analyses. They effectively lower the technical barriers to advanced Bayesian inference for non-statisticians. This paper effectively bridges the historical divide between intricate Bayesian theory and practical application. It delivers a definitive roadmap for researchers to maximize transparency and credibility in meta-analytic conclusions. Therefore, this methodology offers broad applicability for enhancing the robustness of evidence synthesis across psychological science and various other disciplines.