sample size n
sample size n

number of replicates
10000

display

Marc Lavielle,
July 22, 2015

### The Central Limit Theorem

We consider here an infinite sequence $$X_1$$, $$X_2$$, $$X_3$$, …, of independent and identically distributed (i.i.d.) random variables, with mean $$m$$ and finite variance $$\omega^2$$: ${\rm E}(X_i) = m \quad ; \quad {\rm Var}(X_i) = \omega^2 < \infty \ , \quad \text{for } i = 1, 2, \ldots$

For any $$n\geq 1$$ , let $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ be the sample average of $$(X_1, \ldots, X_n)$$.

By the law of large numbers, $$\bar{X}_n$$ converges in probability and almost surely to the expected value $$m$$ as $$n \to \infty$$.

Furthermore, $$\bar{X}_n$$ converges to $$m$$ with speed $$n^{-½}$$, in the sense that the difference between $$\bar{X}_n$$ and $$m$$ decreases as $$n^{-½}$$: $\sqrt{n}(\bar{X}_n - m) = {\cal O}_P(1))$

More precisely, the Central Limit Theorem (CLT) states that, as $$n$$ approaches infinity, the random variable $$\sqrt{n}(\bar{X}_n- m)/\omega$$ converges in distribution to a normal $${\cal N}(0,1)$$:

$\sqrt{n} \frac{\bar{X}_n - m}{\omega} \to {\cal N}(0,1)$

### Some probability distributions

The objective of the application is to “visualize'' the convergence in distribution of $$\bar{X}_n$$ for different distributions of $$X_i$$:

• The normal distribution with mean $$\mu$$ and variance $$\sigma^2$$: $X_i \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = \mu$

• The log-normal distribution with parameters $$\mu$$ and $$\sigma^2$$: $\log(X_i) \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = e^{\mu + \sigma^2/2}$

• The $$\chi^2$$ distribution with $$\nu$$ degree of freedom: $X_i \sim \chi^2(\nu) \quad ; \quad {\rm E}(X_i) = \nu$

• The uniform distribution on [0, b]: $X_i \sim \text{Unif}([0,b]) \quad ; \quad {\rm E}(X_i) = b/2$

• The Poisson distribution with parameter $$\lambda$$: $X_i \sim {\cal P}(\lambda) \quad ; \quad {\rm E}(X_i) = \lambda$

• The binomial distribution with parameters $$k$$ and $$p$$: $X_i \sim B(k,p) \quad ; \quad {\rm E}(X_i) = kp$

1) Rate of convergence

The graph displays one ($$M=1$$) or several ($$M>1$$) replicates of the sequence $$(n^\beta (\bar{X}_n - m)/\omega, n\geq 1)$$.

• Using several probability distributions, display $$M=1,\ 2, \ 10, \ 100$$ replicates of the sequence $$(n^\beta (\bar{X}_n - m)/\omega, n\geq 1)$$ with $$\beta=½$$.

• Select a large number of replicates (e.g. $$N=1000$$) and display the mean and standard deviation of these $$M$$ replicates. What happens when $$\beta<0.5$$? when $$\beta>0.5$$? when $$\beta=0.5$$?

2) Prediction intervals

The graph displays several prediction intervals of $$n^\beta (\bar{X}_n - m)/\omega$$ for $$n=1, 2, \ldots$$. These intervals are computed by Monte Carlo simulation, using $$M=10\,000$$ replicates.

You can change the number of prediction intervals (i.e. the number of bands) and the level of the wider interval. For instance, when the level is $$80\%$$ and the number of bands is $$8$$, then the graph displays the 9 deciles, i.e. the 10th, 20th, …, 90th percentiles of the distribution of $$n^\beta (\bar{X}_n - m)/\omega$$.

These percentiles are compared with the percentiles of a normal $${\cal N}(0,1)$$ (horizontal lines).

• Using the $$\xi^2$$ distribution with 1 d.f., see how the percentiles of the distribution of $$n^\beta (\bar{X}_n - m)/\omega$$ converge to the percentiles of a normal $${\cal N}(0,1)$$ when $$\beta=0.5$$.

• What happens when $$\beta<0.5$$? when $$\beta>0.5$$?

• Use now a normal distribution. Comment on the graph.

• Use now a Pareto distribution with $$c=1.1$$ and $$c=4$$. Comment on the graphs.

3) Limiting distribution

The graph displays the distribution of $$n^\beta (\bar{X}_n - m)/\omega$$ computed by Monte Carlo simulation (histogram obtained using $$M=10\,000$$ replicates).

This distribution is compared with the probability distribution function (pdf) of a normal $${\cal N}(0,1)$$.

• Using the uniform distribution on $$[0, 1]$$, see how the distribution of $$n^\beta (\bar{X}_n - m)/\omega$$ converges to the distribution of a normal $${\cal N}(0,1)$$ when $$\beta=0.5$$.