- This is part of probstat
Consider a certain distribution. The mean 
of the distribution is the expected value of a random variable 
sample from the distribution. I.e.,
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle \mu =E[X]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4b8d5d31fd9dc7cd9849f1a07a72e82420448133)
.
Also recall that the variance of the distribution is
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle \sigma ^{2}=Var(X)=E[(X-\mu )^{2}]=E[X^{2}]-E[X]^{2}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/413cc5ad0d25f8ab364154479aee3f44d26e3cc0)
And finally, the standard deviation is
Wikimedia Error
Error
Too many requests (f061ab2)
.
Sample Statistics
Suppose that you take 
samples 
independently from this distribution. (Note that are random variables.)
Sample means
The statistic
Wikimedia Error
Error
Too many requests (f061ab2)
is called a sample mean. Since
Wikimedia Error
Error
Too many requests (f061ab2)
are random variables, the mean 
is also a random variable.
We hope that approximates well. We can compute:
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle E[{\bar {X}}]=E\left[{\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}E\left[\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}\sum _{i=1}^{n}E[X_{i}]={\frac {1}{n}}n\mu =\mu }](https://wikimedia.org/api/rest_v1/media/math/render/svg/2df57bd2c3c1cfc8a2937d14ac740a5627a5a58c)
and since are independent, we have that
Wikimedia Error
Error
Too many requests (f061ab2)
Sample variances and sample standard deviations
We can also use the sample to estimate 
.
The statistic
is called a sample variance. The sample standard deviation is
Wikimedia Error
Error
Too many requests (f061ab2)
.
Note that the denominator is
Wikimedia Error
Error
Too many requests (f061ab2)
instead of .
We can show that ![{\displaystyle E[S^{2}]=\sigma ^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7163f09dd5cfeb7bb5f38d475062cc5676f072e5)
.
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle {\begin{array}{rcl}\mathrm {E} [S^{2}]&=&\mathrm {E} \left[{\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}\right]\\&=&\mathrm {E} \left[{\frac {\sum _{i=1}^{n}(X_{i}^{2}-2X_{i}{\bar {X}}+{\bar {X}}^{2}}{n-1}}\right]\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-2\cdot \sum _{i=1}^{n}E[X_{i}{\bar {X}}]+\sum _{i=1}^{n}E[{\bar {X}}^{2}]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-2\cdot \sum _{i=1}^{n}E\left[X_{i}\left((1/n)\sum _{j=1}^{n}X_{j}\right)\right]+\sum _{i=1}^{n}E\left[\left((1/n)\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+n\cdot E\left[\left((1/n)\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+(1/n)\cdot E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\\end{array}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/93650a6d8c6b577a33a2ab19145b5f15d2f395b5)
We note that since 
and 
are independent, we have that
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle E[X_{i}X_{j}]=E[X_{i}]E[X_{j}]=\mu \cdot \mu =\mu ^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3c180ef6fd6cf5f43cac88f3106d697eb57e4d44)
.
Let's deal with the middle term here.
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle {\begin{array}{rcl}\sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]&=&\sum _{i=1}^{n}E[X_{i}X_{i}]+\sum _{i=1}^{n}\sum _{j\neq i}E[X_{i}X_{j}]\\&=&\sum _{i=1}^{n}E[X_{i}^{2}]+\sum _{i=1}^{n}\sum _{j\neq i}E[X_{i}]E[X_{j}]\\&=&nE[X^{2}]+n(n-1)E[X]E[X]\\&=&nE[X^{2}]+n(n-1)\mu ^{2}\\\end{array}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4ef01c98b869c0deb55d07dea156c51f76b4e5b7)
Let's work on the third term which ends up being the same as the middle term.
Wikimedia Error
Error
Too many requests (f061ab2)
![{\displaystyle E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]=E\left[\sum _{j=1}^{n}\sum _{k=1}^{n}X_{j}X_{k}\right]=\sum _{j=1}^{n}\sum _{k=1}^{n}E[X_{j}X_{k}]=nE[X^{2}]+n(n-1)\mu ^{2}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bf78bddcae38881f34e4191e061ae76bce37c466)
Let's put everything together:
![{\displaystyle {\begin{array}{rcl}\mathrm {E} [S^{2}]&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+(1/n)\cdot E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(nE[X^{2}]-(2/n)(nE[X^{2}]+n(n-1)\mu ^{2})+(1/n)(nE[X^{2}]+n(n-1)\mu ^{2})\right)\\&=&{\frac {1}{n-1}}\left(nE[X^{2}]-2E[X^{2}]-2(n-1)\mu ^{2}+E[X^{2}]+(n-1)\mu ^{2}\right)\\&=&{\frac {1}{n-1}}\left((n-1)E[X^{2}]-(n-1)\mu ^{2}\right)\\&=&E[X^{2}]-\mu ^{2}=\sigma ^{2}\\\end{array}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d23da700b512a8307a14076e1ad12f1b64166190)
Summary
Sample means:
Wikimedia Error
Error
Too many requests (f061ab2)
Sample variance:
Properties of sample means and sample variances
Distribution of sample means
While we know basic properties of sample means , if we want to perform other statistical calculation (i.e., computing confidence intervals or testing hypotheses), it is very useful to know the exact distribution of .
For a general population, it will be hard to deal the the distribution of exactly. However, if the population is normal, we are in a very good shape.
Recall the definition of :
Therefore, is a sum of independent normally distributed random variables. A nice property of normal random variables is that the sum of normally distributed random variables remains a normal random variable. Since a normal random variable is uniquely determined by its mean and variance, we have the following observation.
Distribution of sample means of normal populations.
If the population is normally distributed with mean and variance , the distribution of is normal with mean and variance 
.
Examples
Ex1. Suppose that the population has mean
Wikimedia Error
Error
Too many requests (f061ab2)
and variance . If you select a sample of size 20, what is the probability that the sample mean is greater than 17?
Solution:
The sample mean is normal with mean and variance
Wikimedia Error
Error
Too many requests (f061ab2)
. Therefore,
is unit normal.
Note that
We can look at the standard normal table and find out that , for a unit normal random variable Z. Thus, the probability
Wikimedia Error
Error
Too many requests (f061ab2)
which is roughly 1%.
Ex2.
- To be added...
Why do we use normal distributions?
Normal random variables appear very often in our treatment of statistics. This is not just a coincidence. See limit theorems.
![{\displaystyle E[{\bar {X}}]=\mu }](https://wikimedia.org/api/rest_v1/media/math/render/svg/0b24da826c471c03227c6d06047b99ea9209638c)
![{\displaystyle Var[{\bar {X}}]=\sigma ^{2}/n}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7b4eb60e5404746f2244eb4ec7a4c9d07beb2773)
