The Gelman-Rubin statistic allows a statement about the convergence of Monte Carlo simulations.

Definition

${\displaystyle J}$ Monte Carlo simulations (chains) are started with different initial values. The samples from the respective burn-in phases are discarded. From the samples ${\displaystyle x_{1}^{(j)},\dots ,x_{L}^{(j)}}$ (of the j-th simulation), the variance between the chains and the variance in the chains is estimated:

${\displaystyle {\overline {x}}_{j}={\frac {1}{L}}\sum _{i=1}^{L}x_{i}^{(j)}}$ Mean value of chain j

${\displaystyle {\overline {x}}_{*}={\frac {1}{J}}\sum _{j=1}^{J}{\overline {x}}_{j}}$ Mean of the means of all chains

${\displaystyle B={\frac {L}{J-1}}\sum _{j=1}^{J}({\overline {x}}_{j}-{\overline {x}}_{*})^{2}}$ Variance of the means of the chains

${\displaystyle W={\frac {1}{J}}\sum _{j=1}^{J}\left({\frac {1}{L-1}}\sum _{i=1}^{L}(x_{i}^{(j)}-{\overline {x}}_{j})^{2}\right)}$ Averaged variances of the individual chains across all chains

An estimate of the Gelman-Rubin statistic ${\displaystyle R}$ then results as^[1]

${\displaystyle R={\frac {{\frac {L-1}{L}}W+{\frac {1}{L}}B}{W}}}$.

When L tends to infinity and B tends to zero, R tends to 1.

Alternatives

The Geweke Diagnostic compares whether the mean of the first x percent of a chain and the mean of the last y percent of a chain match.

Literature

• Vats, Dootika, and Christina Knudson. "Revisiting the gelman–rubin diagnostic." Statistical Science 36.4 (2021): 518-529. arxiv
• Gelman, Andrew, and Donald B. Rubin. "Inference from iterative simulation using multiple sequences." Statistical science 7.4 (1992): 457-472. pdf

References