sample size n
sample size n

number of replicates


Marc Lavielle,
July 22, 2015

The Central Limit Theorem

We consider here an infinite sequence \(X_1\), \(X_2\), \(X_3\), …, of independent and identically distributed (i.i.d.) random variables, with mean \(m\) and finite variance \(\omega^2\): \[{\rm E}(X_i) = m \quad ; \quad {\rm Var}(X_i) = \omega^2 < \infty \ , \quad \text{for } i = 1, 2, \ldots\]

For any \(n\geq 1\) , let \[\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i\] be the sample average of \((X_1, \ldots, X_n)\).

By the law of large numbers, \(\bar{X}_n\) converges in probability and almost surely to the expected value \(m\) as \(n \to \infty\).

Furthermore, \(\bar{X}_n\) converges to \(m\) with speed \(n^{-½}\), in the sense that the difference between \(\bar{X}_n\) and \(m\) decreases as \(n^{-½}\): \[\sqrt{n}(\bar{X}_n - m) = {\cal O}_P(1)) \]

More precisely, the Central Limit Theorem (CLT) states that, as \(n\) approaches infinity, the random variable \(\sqrt{n}(\bar{X}_n- m)/\omega\) converges in distribution to a normal \({\cal N}(0,1)\):

\[ \sqrt{n} \frac{\bar{X}_n - m}{\omega} \to {\cal N}(0,1) \]

Some probability distributions

The objective of the application is to “visualize'' the convergence in distribution of \(\bar{X}_n\) for different distributions of \(X_i\):

  • The normal distribution with mean \(\mu\) and variance \(\sigma^2\): \[X_i \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = \mu\]

  • The log-normal distribution with parameters \(\mu\) and \(\sigma^2\): \[\log(X_i) \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = e^{\mu + \sigma^2/2}\]

  • The \(\chi^2\) distribution with \(\nu\) degree of freedom: \[X_i \sim \chi^2(\nu) \quad ; \quad {\rm E}(X_i) = \nu\]

  • The uniform distribution on [0, b]: \[X_i \sim \text{Unif}([0,b]) \quad ; \quad {\rm E}(X_i) = b/2\]

  • The Poisson distribution with parameter \(\lambda\): \[X_i \sim {\cal P}(\lambda) \quad ; \quad {\rm E}(X_i) = \lambda\]

  • The binomial distribution with parameters \(k\) and \(p\): \[X_i \sim B(k,p) \quad ; \quad {\rm E}(X_i) = kp\]

Some tasks

1) Rate of convergence

The graph displays one (\(M=1\)) or several (\(M>1\)) replicates of the sequence \((n^\beta (\bar{X}_n - m)/\omega, n\geq 1)\).

  • Using several probability distributions, display \(M=1,\ 2, \ 10, \ 100\) replicates of the sequence \((n^\beta (\bar{X}_n - m)/\omega, n\geq 1)\) with \(\beta=½\).

  • Select a large number of replicates (e.g. \(N=1000\)) and display the mean and standard deviation of these \(M\) replicates. What happens when \(\beta<0.5\)? when \(\beta>0.5\)? when \(\beta=0.5\)?

2) Prediction intervals

The graph displays several prediction intervals of \(n^\beta (\bar{X}_n - m)/\omega\) for \(n=1, 2, \ldots\). These intervals are computed by Monte Carlo simulation, using \(M=10\,000\) replicates.

You can change the number of prediction intervals (i.e. the number of bands) and the level of the wider interval. For instance, when the level is \(80\%\) and the number of bands is \(8\), then the graph displays the 9 deciles, i.e. the 10th, 20th, …, 90th percentiles of the distribution of \(n^\beta (\bar{X}_n - m)/\omega\).

These percentiles are compared with the percentiles of a normal \({\cal N}(0,1)\) (horizontal lines).

  • Using the \(\xi^2\) distribution with 1 d.f., see how the percentiles of the distribution of \(n^\beta (\bar{X}_n - m)/\omega\) converge to the percentiles of a normal \({\cal N}(0,1)\) when \(\beta=0.5\).

  • What happens when \(\beta<0.5\)? when \(\beta>0.5\)?

  • Use now a normal distribution. Comment on the graph.

  • Use now a Pareto distribution with \(c=1.1\) and \(c=4\). Comment on the graphs.

3) Limiting distribution

The graph displays the distribution of \(n^\beta (\bar{X}_n - m)/\omega\) computed by Monte Carlo simulation (histogram obtained using \(M=10\,000\) replicates).

This distribution is compared with the probability distribution function (pdf) of a normal \({\cal N}(0,1)\).

  • Using the uniform distribution on \([0, 1]\), see how the distribution of \(n^\beta (\bar{X}_n - m)/\omega\) converges to the distribution of a normal \({\cal N}(0,1)\) when \(\beta=0.5\).