Multivariate Gaussians

Posted 2020-11-25 eliker

---

A vector-valued random variable $xin mathbb{R}^n$ is said to have a multivariate gaussian ( or normal) distribution with mean $muin mathbb{R}^n$ and covariance matrix $Sigma in mathbb{S}_{++}^n$ if

$$p(x; mu, Sigma)=frac{1}{(2pi)^{n/2}|Sigma |^{1/2}} ext{exp}(-frac{1}{2}(x-mu)^T Sigma ^{-1}(x-mu)). $$

We write this as $x sim mathcal N(mu, Sigma)$

Consider a random vector $x sim mathcal N(mu, Sigma)$, Suppose that the variable in $x$ have been partitioned into two sets $x_A=[x_1 cdots x_r]^T in mathbb{R}^r$ and $x_B =[x_{r+1} cdots x_n]^T in mathbb{R}^{n-r} $, such that

egin{equation*} x= left [ egin{array}{c} x_A \ x_B end{array} ight ] qquad mu= left [ egin{array}{c} mu_A \ mu_B end{array} ight ]  qquad Sigma= left [ egin{array}{cc} Sigma_{AA} & Sigma_{AB} \ Sigma_{BA} & Sigma_{BB} end{array} ight ] end{equation*}

Here, $  Sigma_{AB} = Sigma_{BA}^T $ since $ Sigma = E[(x-mu)(x-mu)^T] = Sigma^T $. The following properties hold:

1. Normalization

$$ int_{xin mathbb{R}^n} p(x; mu, Sigma) dx = 1$$

2.Marginalization

The marginal densities are gaussian:

egin{equation*} p(x_A) = int_{x_B in mathbb{R}^{n-r}} p(x_A, x_B; mu Sigma) dx_B end{equation*}

$$p(x_B) = int_{x_A in mathbb{R}^r} p(x_A, x_B; mu Sigma) dx_A$$

$$x_A sim mathcal N(mu_A, Sigma_{AA})$$

$$x_B sim mathcal N(mu_B, Sigma_{BB})$$

3. Conditioning

The conditional densities are also gaussian

$$p(x_A | x_B) = frac{p(x_A, x_B; mu, Sigma)}{p(x_B)}$$

$$p(x_B | x_A) = frac{p(x_A, x_B; mu, Sigma)}{p(x_A)}$$

$$x_A | x_B sim mathcal N(mu_A + Sigma_{AB} Sigma_{BB}^{-1}(x_B-mu_B), Sigma_{AA} – Sigma_{AB} Sigma_{BB}^{-1} Sigma_{BA} )$$

$$x_B | x_A sim mathcal N(mu_B + Sigma_{BA} Sigma_{AA}^{-1}(x_A-mu_A), Sigma_{BB} – Sigma_{BA} Sigma_{AA}^{-1} Sigma_{AB} )$$

4. Summation

The sum of independent  Gaussian random variables (with same dimensionality), $y sim mathcal N(mu_x. Sigma_{xx})$ and $z sim mathcal N(mu_{z}, Sigma_{zz}) $, is also Gaussian:

$$ y+z sim mathcal N(mu_x + mu_z, Sigma_{xx}+Sigma_{zz}) $$

egin{equation} x=y+z end{equation}