Spectral density

The joint-eigenvalue density for the eigenvalues ${\displaystyle \lambda _{1},\dots ,\lambda _{p}\geq 0}$ of a random matrix ${\displaystyle \mathbf {X} \sim W_{p}(\mathbf {I} ,n)}$ is,^[8][9]

${\displaystyle c_{n,p}e^{-{\frac {1}{2}}\sum _{i}\lambda _{i}}\prod \lambda _{i}^{(n-p-1)/2}\prod _{i<j}|\lambda _{i}-\lambda _{j}|}$

where ${\displaystyle c_{n,p}}$is a constant.

In fact the above definition can be extended to any real n > p − 1. If n ≤ p − 1, then the Wishart no longer has a density—instead it represents a singular distribution that takes values in a lower-dimension subspace of the space of p × p matrices.^[10]

Choice of parameters

The least informative, proper Wishart prior is obtained by setting n = p.

The prior mean of W_p(V, n) is nV, suggesting that a reasonable choice for V would be n⁻¹Σ₀⁻¹, where Σ₀ is some prior guess for the covariance matrix.

Log-expectation

The following formula plays a role in variational Bayes derivations for Bayes networks involving the Wishart distribution. From equation (2.63),^[13]

${\displaystyle \operatorname {E} [\,\ln \left|\mathbf {X} \right|\,]=\psi _{p}\left({\frac {n}{2}}\right)+p\,\ln(2)+\ln |\mathbf {V} |}$

where ${\displaystyle \psi _{p}}$ is the multivariate digamma function (the derivative of the log of the multivariate gamma function).

Log-variance

The following variance computation could be of help in Bayesian statistics:

${\displaystyle \operatorname {Var} \left[\,\ln \left|\mathbf {X} \right|\,\right]=\sum _{i=1}^{p}\psi _{1}\left({\frac {n+1-i}{2}}\right)}$

where ${\displaystyle \psi _{1}}$ is the trigamma function. This comes up when computing the Fisher information of the Wishart random variable.

Entropy

The information entropy of the distribution has the following formula:^[11]: 693

${\displaystyle \operatorname {H} \left[\,\mathbf {X} \,\right]=-\ln \left(B(\mathbf {V} ,n)\right)-{\frac {n-p-1}{2}}\operatorname {E} \left[\,\ln \left|\mathbf {X} \right|\,\right]+{\frac {np}{2}}}$

where B(V, n) is the normalizing constant of the distribution:

${\displaystyle B(\mathbf {V} ,n)={\frac {1}{\left|\mathbf {V} \right|^{n/2}2^{np/2}\Gamma _{p}\left({\frac {n}{2}}\right)}}.}$

This can be expanded as follows:

${\displaystyle {\begin{aligned}\operatorname {H} \left[\,\mathbf {X} \,\right]&={\frac {n}{2}}\ln \left|\mathbf {V} \right|+{\frac {np}{2}}\ln 2+\ln \Gamma _{p}\left({\frac {n}{2}}\right)-{\frac {n-p-1}{2}}\operatorname {E} \left[\,\ln \left|\mathbf {X} \right|\,\right]+{\frac {np}{2}}\\[8pt]&={\frac {n}{2}}\ln \left|\mathbf {V} \right|+{\frac {np}{2}}\ln 2+\ln \Gamma _{p}\left({\frac {n}{2}}\right)-{\frac {n-p-1}{2}}\left(\psi _{p}\left({\frac {n}{2}}\right)+p\ln 2+\ln \left|\mathbf {V} \right|\right)+{\frac {np}{2}}\\[8pt]&={\frac {n}{2}}\ln \left|\mathbf {V} \right|+{\frac {np}{2}}\ln 2+\ln \Gamma _{p}\left({\frac {n}{2}}\right)-{\frac {n-p-1}{2}}\psi _{p}\left({\frac {n}{2}}\right)-{\frac {n-p-1}{2}}\left(p\ln 2+\ln \left|\mathbf {V} \right|\right)+{\frac {np}{2}}\\[8pt]&={\frac {p+1}{2}}\ln \left|\mathbf {V} \right|+{\frac {1}{2}}p(p+1)\ln 2+\ln \Gamma _{p}\left({\frac {n}{2}}\right)-{\frac {n-p-1}{2}}\psi _{p}\left({\frac {n}{2}}\right)+{\frac {np}{2}}\end{aligned}}}$

Cross-entropy

The cross-entropy of two Wishart distributions ${\displaystyle p_{0}}$ with parameters ${\displaystyle n_{0},V_{0}}$ and ${\displaystyle p_{1}}$ with parameters ${\displaystyle n_{1},V_{1}}$ is

${\displaystyle {\begin{aligned}H(p_{0},p_{1})&=\operatorname {E} _{p_{0}}[\,-\log p_{1}\,]\\[8pt]&=\operatorname {E} _{p_{0}}\left[\,-\log {\frac {\left|\mathbf {X} \right|^{(n_{1}-p_{1}-1)/2}e^{-\operatorname {tr} (\mathbf {V} _{1}^{-1}\mathbf {X} )/2}}{2^{n_{1}p_{1}/2}\left|\mathbf {V} _{1}\right|^{n_{1}/2}\Gamma _{p_{1}}\left({\tfrac {n_{1}}{2}}\right)}}\right]\\[8pt]&={\tfrac {n_{1}p_{1}}{2}}\log 2+{\tfrac {n_{1}}{2}}\log \left|\mathbf {V} _{1}\right|+\log \Gamma _{p_{1}}({\tfrac {n_{1}}{2}})-{\tfrac {n_{1}-p_{1}-1}{2}}\operatorname {E} _{p_{0}}\left[\,\log \left|\mathbf {X} \right|\,\right]+{\tfrac {1}{2}}\operatorname {E} _{p_{0}}\left[\,\operatorname {tr} \left(\,\mathbf {V} _{1}^{-1}\mathbf {X} \,\right)\,\right]\\[8pt]&={\tfrac {n_{1}p_{1}}{2}}\log 2+{\tfrac {n_{1}}{2}}\log \left|\mathbf {V} _{1}\right|+\log \Gamma _{p_{1}}({\tfrac {n_{1}}{2}})-{\tfrac {n_{1}-p_{1}-1}{2}}\left(\psi _{p_{0}}({\tfrac {n_{0}}{2}})+p_{0}\log 2+\log \left|\mathbf {V} _{0}\right|\right)+{\tfrac {1}{2}}\operatorname {tr} \left(\,\mathbf {V} _{1}^{-1}n_{0}\mathbf {V} _{0}\,\right)\\[8pt]&=-{\tfrac {n_{1}}{2}}\log \left|\,\mathbf {V} _{1}^{-1}\mathbf {V} _{0}\,\right|+{\tfrac {p_{1}+1}{2}}\log \left|\mathbf {V} _{0}\right|+{\tfrac {n_{0}}{2}}\operatorname {tr} \left(\,\mathbf {V} _{1}^{-1}\mathbf {V} _{0}\right)+\log \Gamma _{p_{1}}\left({\tfrac {n_{1}}{2}}\right)-{\tfrac {n_{1}-p_{1}-1}{2}}\psi _{p_{0}}({\tfrac {n_{0}}{2}})+{\tfrac {n_{1}(p_{1}-p_{0})+p_{0}(p_{1}+1)}{2}}\log 2\end{aligned}}}$

Note that when ${\displaystyle p_{0}=p_{1}}$ and ${\displaystyle n_{0}=n_{1}}$we recover the entropy.

KL-divergence

The Kullback–Leibler divergence of ${\displaystyle p_{1}}$ from ${\displaystyle p_{0}}$ is

${\displaystyle {\begin{aligned}D_{KL}(p_{0}\|p_{1})&=H(p_{0},p_{1})-H(p_{0})\\[6pt]&=-{\frac {n_{1}}{2}}\log |\mathbf {V} _{1}^{-1}\mathbf {V} _{0}|+{\frac {n_{0}}{2}}(\operatorname {tr} (\mathbf {V} _{1}^{-1}\mathbf {V} _{0})-p)+\log {\frac {\Gamma _{p}\left({\frac {n_{1}}{2}}\right)}{\Gamma _{p}\left({\frac {n_{0}}{2}}\right)}}+{\tfrac {n_{0}-n_{1}}{2}}\psi _{p}\left({\frac {n_{0}}{2}}\right)\end{aligned}}}$

Characteristic function

The characteristic function of the Wishart distribution is

${\displaystyle \Theta \mapsto \operatorname {E} \left[\,\exp \left(\,i\operatorname {tr} \left(\,\mathbf {X} {\mathbf {\Theta } }\,\right)\,\right)\,\right]=\left|\,1-2i\,{\mathbf {\Theta } }\,{\mathbf {V} }\,\right|^{-n/2}}$

where E[⋅] denotes expectation. (Here Θ is any matrix with the same dimensions as V, 1 indicates the identity matrix, and i is a square root of −1).^[9] Properly interpreting this formula requires a little care, because noninteger complex powers are multivalued; when n is noninteger, the correct branch must be determined via analytic continuation.^[14]

Corollary 1

If z is a nonzero p × 1 constant vector, then:^[15]

${\displaystyle \sigma _{z}^{-2}\,{\mathbf {z} }^{T}\mathbf {X} {\mathbf {z} }\sim \chi _{m}^{2}.}$

In this case, ${\displaystyle \chi _{m}^{2}}$ is the chi-squared distribution and ${\displaystyle \sigma _{z}^{2}={\mathbf {z} }^{T}{\mathbf {V} }{\mathbf {z} }}$ (note that ${\displaystyle \sigma _{z}^{2}}$ is a constant; it is positive because V is positive definite).

Corollary 2

Consider the case where z^T = (0, ..., 0, 1, 0, ..., 0) (that is, the j-th element is one and all others zero). Then corollary 1 above shows that

${\displaystyle \sigma _{jj}^{-1}\,w_{jj}\sim \chi _{m}^{2}}$

gives the marginal distribution of each of the elements on the matrix's diagonal.

George Seber points out that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.^[16]