sample size n

number of replicates

display

Marc Lavielle,
July 21, 2015

### The Law of Large Numbers

We consider here an infinite sequence $$X_1$$, $$X_2$$, $$X_3$$, …, of independent and identically distributed (i.i.d.) random variables, with mean $$m$$ and finite variance $$\omega^2$$: ${\rm E}(X_i) = m \quad ; \quad {\rm Var}(X_i) = \omega^2 < \infty \ , \quad \text{for } i = 1, 2, \ldots$

Then, the sample average $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ converges to $$m$$ for $$n \to \infty$$.

• The weak law of large numbers (also called Khintchine's law) states that $$\bar{X}_n$$ converges in probability towards the expected value $$m$$. Indeed, ${\rm E}(\bar{X}_n) = m \quad ; \quad {\rm Var}(\bar{X}_n) = \frac{\omega^2}{n}$

• The strong law of large numbers states that $$\bar{X}_n$$ converges almost surely to the expected value $$m$$.

### Some probability distributions

The objective of the application is to “visualize'' the convergence (in probability) of $$\bar{X}_n$$ for different distributions of $$X_i$$:

• The normal distribution with mean $$\mu$$ and variance $$\sigma^2$$: $X_i \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = \mu$

• The log-normal distribution with parameters $$\mu$$ and $$\sigma^2$$: $\log(X_i) \sim {\cal N}(\mu , \sigma^2) \quad ; \quad {\rm E}(X_i) = e^{\mu + \sigma^2/2}$

• The $$\chi^2$$ distribution with $$\nu$$ degree of freedom: $X_i \sim \chi^2(\nu) \quad ; \quad {\rm E}(X_i) = \nu$

• The uniform distribution on [0, b]: $X_i \sim \text{Unif}([0,b]) \quad ; \quad {\rm E}(X_i) = b/2$

• The Poisson distribution with parameter $$\lambda$$: $X_i \sim {\cal P}(\lambda) \quad ; \quad {\rm E}(X_i) = \lambda$

• The binomial distribution with parameters $$k$$ and $$p$$: $X_i \sim B(k,p) \quad ; \quad {\rm E}(X_i) = kp$

• Using the normal distribution, display one sequence $$(\bar{X}_n, n\geq 1)$$ and see how this sequence converges to $$m=\mu$$ (increase the maximum value of $$n$$).
• Display now $$N$$ replicates $$(\bar{X}_n^{(1)})$$ , $$(\bar{X}_n^{(2)})$$,…, $$(\bar{X}_n^{(N)})$$, for $$N=2, 3, 5, 10, 100, 1000$$. Comment about these graphs.
• Display the empirical mean of the $$N$$ replicates and $$+/-$$ one standard deviation from the mean. How decreases the standard deviation of $$\bar{X}_n$$?