Central Limit theorems (i.i.d. case), CSIR-NET Mathematical Sciences | Mathematics for IIT JAM, GATE, CSIR NET, UGC NET PDF Download

Throughout the discussion below , let X₁,X₂,... be i.i.d. rv’s, each with finite expected value µ and finite nonzero standard deviation σ. Given n, define  to be the average (X₁+···+X_n)/n, and define Sn to be the sum X₁+···+X_n. Then 

(The equation for the variance of Sn holds because the X_i are independent, so the variance of the sum of the X_i is the sum of the variances.) Now,  = (1/n)Sn, so

REMARKS 1. We can think of the i.i.d. condition as meaning that the X_i are repeated experiments, or alternately random samples, from some given probability distribution.
2. The expected value of the sample average is the same as the expected value of each X_i. This is common sense. We can think of  as an estimate of the true population average µ.
3. As n gets bigger, the spread (standard deviation) of  gets smaller. This is common sense: a bigger sample should give a more reliable estimate of the true population average.

4.  denotes the normal distribution with mean µ and standard deviation σ. THEOREM (Central Limit Theorem) Suppose that X₁,X₂,... is a sequence of i.i.d. rv’s, each with finite expected value µ and finite nonzero standard deviation σ. Let Z_n be the standardized version of , i.e.

Then as n →∞, Zn →N(0,1).
 

REMARKS

1. Note the CLT has an extra assumption (finite variance) which the LLN does not have. The CLT gives more information when it is applicable.

2. The CLT is an incredible law of nature. Under modest assumptions, the process of independent repetition has a universal effect on the averaging process, depending only on the mean and standard deviation of the underlying population.

3. The expression  abbreviates “Z_n converges in distribution to N(0,1) as n →∞”. Informally, this means that if n is large enough, then we have (for all numbers a < b) 

where ≈ means “approximately equals”, and as n goes to infinity, the approximation gets as good as we want. If we want to be completely precise, we express this by saying that for every ε > 0, there exists N such that whenever n ≥ N and a < b, we have 

where Φ is the cumulative distribution function for N(0,1) (which is given approx-imately by the tables in the back of our textbook).

4. There are other ways to express the approximation above: 

where Z is any r.v. which has the standard normal distribution N(0,1). We could also use the notation  to describe this situation. 

RULE OF THUMB

How large should n be for the CLT approximation to be good enough? Really, that depends on the particular r.v. X and on how good “good enough” has to be. One rule of thumb (the rule used, for example, in the Devore text) is that, unless we have explicit information to the contrary, n > 30 is large enough for “good enough”.
 

EXAMPLE

Let us go through the approximations above in an example, with a =−2 and b = 2. We will take the r.v.’s X_i corresponding to flipping a fair coin. So, each X_i equals 0 with probability 1/2, and equals 1 with probability 1/2. For each X_i, µ = .5 and    Let n = 10,000. Then the standard deviation for

Here are the approximations above with these numbers put in. 

If we compute out the last line, we get 

This means: if the experiment is to flip a fair coin 10,000 times: then in about 95% of those experiments, the percentage of the flips which equal heads will be between 49% and 51%.
 

SUMS AND THE CLT

Let us look again at one of the ways to express the CLT: 

Remember,  = (X₁ +···+X_n)/n. If we multiply each element of the inequality on the left by n, we don’t change the truth of the inequality, so we don’t change its probability. So we get 

So, we can also use a normal approximation for the probability that sums lie in some range. (For the special case of coin flipping, we already did this with the normal approximation to the binomial distribution. In fact, the description above explains how the normal approximation to the binomial distribution can be deduced as a consequence of the Central Limit Theorem.)
 

LAW OF LARGE NUMBERS

Let us see that the LLN is a consequence of the CLT, in the case that the CLT applies.

Suppose ε > 0, and we have i.i.d. rv’s as in the Central Limit Theorem. Then 

where Z is any random variable with the standard normal distribution. Therefore for any given ε > 0, no matter how small,

This last statement is one way to state the Law of Large Numbers.