Probstat/notes/chi-squared distribution

This is part of probstat.

Definition

Definition: Let $Z_{1},Z_{2},\ldots ,Z_{n}$ be independent unit normal random variables. A random variable

$X=Z_{1}^{2}+Z_{2}^{2}+\cdots +Z_{n}^{2},$

is called a chi-squared random variable with $n$ degrees of freedom. We also write

$X\sim \chi _{n}^{2}.$

Wikipedia has a nice article on chi-squared distribution which also includes plots of its pdf and cdf.

Properties

Here we states important properties of the chi-squared distribution without proofs.

Expectations and variances

If $X$ is chi-squared with $n$ degrees of freedom, we have that its expectation

$E[X]=n,$

and its variance

$Var(X)=2n.$

Sample variances

The chi-squared distribution is very important when we want to reason about the sample variances (See notes). Under that settings, we sample $X_{1},X_{2},\ldots ,X_{n}$ from a normal population whose mean is $\mu$ and variance is $\sigma ^{2}$.

We first note that if $\mu$ and $\sigma ^{2}$ is known, a term similar to the one we used to compute the same variance is chi-squared. Recall that $(X_{i}-\mu )/\sigma$ is unit normal. Therefore,

$\sum _{i=1}^{n}\left({\frac {X_{i}-\mu }{\sigma }}\right)^{2}={\frac {1}{\sigma ^{2}}}\sum _{i=1}^{n}(X_{i}-\mu )^{2}\sim \chi _{n}^{2}.$

However, usually, $\sigma ^{2}$ is unknown and we are left to the sample variance, defined as

$S^{2}={\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}$.

Note that the only differences between this term and the previous one is the use of sample mean ${\bar {X}}$ instead of the population mean. It can be proved (not here) that in fact, for normal population, $S^{2}$ (with some scaling) is actually a chi-squared random variable with $n-1$ degrees of freedom (instead of $n$ as in the term computed from $\mu$). That is,

$(n-1){\frac {S^{2}}{\sigma ^{2}}}\sim \chi _{n-1}^{2}.$

More over, for normal population, ${\bar {X}}$ and $S^{2}$ are independent. This is very surprising because both quantities are calculated from the same sample.

These two facts are important in the development of the procedure for calculating confidence intervals in case where $\sigma ^{2}$ is unknown. (See notes.)

Two important facts about sample means and sample variances:

1. $(n-1)S^{2}/\sigma ^{2}\sim \chi _{n-1}^{2}.$

2. ${\bar {X}}$ and $S^{2}$ are independent.

Links

• Wikipedia article on the chi-squared distribution