Probstat/notes/sample means and sample variances

Sample

Consider a certain distribution. The mean $\mu$ of the distribution is the expected value of a random variable $X$ sample from the distribution. I.e.,

$\mu =E[X]$.

Also recall that the variance of the distribution is

$\sigma ^{2}=Var(X)=E[(X-\mu )^{2}]=E[X^{2}]=E[X]^{2}.$.

And finally, the standard deviation is $\sigma ={\sqrt {Var(X)}}$.

Suppose that you take $n$ samples $X_{1},X_{2},\ldots ,X_{n}$ independently from this distribution. (Note that $X_{1},X_{2},\ldots ,X_{n}$ are random variables.

Sample means

The statistic

${\bar {X}}={\frac {X_{1}+X_{2}+\cdots +X_{n}}{n}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}$

is called a sample mean. Since $X_{1},X_{2},\dots ,X_{n}$ are random variables, the mean ${\bar {X}}$ is also a random variable.

Thus, we can compute:

$E[{\bar {X}}]=E\left[{\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}E\left[\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}\sum _{i=1}^{n}E[X_{i}]={\frac {1}{n}}n\mu =\mu$

and

$Var({\bar {X}})={\frac {\sigma ^{2}}{n}}.$

Confidence intervals

Recall that the random variable ${\bar {X}}$ is a normal random variable with mean $\mu$ and s.d. $\sigma /{\sqrt {n}}$ Therefore,

${\sqrt {n}}({\bar {X}}-\mu )/\sigma$

will be a unit normal random variable.

We consider how ${\bar {X}}$ deviates from the true mean $\mu$.