Limit Theorems. STATISTICS Lecture no Department of Econometrics FEM UO Brno office 69a, tel

Transcription

1 STATISTICS Lecture no. 6 Department of Econometrics FEM UO Brno office 69a, tel jiri.neubauer@unob.cz

2 If we repeat some experiment independently we can create using given observed values distribution of relative frequencies and calculate some measures (mean, median, variance... ).

3 If we repeat some experiment independently we can create using given observed values distribution of relative frequencies and calculate some measures (mean, median, variance... ). This distribution (measures) we call sample distribution (sample measures).

4 If we repeat some experiment independently we can create using given observed values distribution of relative frequencies and calculate some measures (mean, median, variance... ). This distribution (measures) we call sample distribution (sample measures). Under particular conditions we can expect that the sample distribution (measures) will converge toward a theoretical distribution (measures). The more repetitions of the experiment the better convergence.

5 Notice that the convergence of the sample values toward theoretical ones is not the convergence in the sense of mathematical convergence, but the probability convergence.

6 Notice that the convergence of the sample values toward theoretical ones is not the convergence in the sense of mathematical convergence, but the probability convergence. The probability convergence if the number of experiments increases, the probability of deviation between sample values and theoretical values decreases.

7 Convergence in probability Definition If the sequence of random variables X 1, X 2,..., X n,... fulfils lim P( X n c < ɛ) = 1, ɛ > 0, n it is said that the sequence {X n } converges in probability to the constant c, we write X n P c.

8 Chebyshev s Inequality Theorem For any random variable X with the mean E(X ), the finite variance D(X ) and for every ɛ > 0 we have P( X E(X ) < ɛ) 1 D(X ) ɛ 2. Chebyshev s inequality is useful fist of all in the theoretical field. It allow us to estimate some probabilities of random variables with unknown distribution.

9 Bernoulli s Theorem Theorem If the random variable X denotes the number of occurrence of the event in the sequence of n independent experiments, where π is the probability of occurrence of the event in one experiment, then for every ɛ > 0 is ( ) lim P X n n π < ɛ = 1.

10 Theorem Let X be a random variable with binomial distribution X B(n, π) a For the standardized random variable U = X nπ nπ(1 π) we have lim P(U u) = Φ(u), n where Φ(u) is the distribution function of the standard normal distribution N(0, 1). a X = X 1, X 2,..., X n, where X i, i = 1..., n, are independent Bernoulli random variables E(X i ) = π, D(X i ) = π(1 π), which means E(X ) = nπ and D(X ) = nπ(1 π).

11 The de Moivre-Laplace theorem says that for n the binomial distribution converges to the normal distribution. Given approximation is acceptable if nπ(1 π) > 9 and 1 n + 1 < π < n n + 1.

12 for proportion Theorem Let X be a random variable with binomial distribution X B(n, π) The random variable X n has the mean E ( ) X n = π and the variance D ( ) X n = π(1 π) n. For the standardized random variable U = X n π π(1 π) n we have lim P(U u) = Φ(u), n where Φ(u) is the distribution function of the standard normal distribution N(0, 1).

13 Theorem Let the random variable be X = X 1 + X X n, where X i, i = 1,..., n are independent random variables with the same distribution with the mean E(X i ) = µ and the finite variance D(X i ) = σ 2, a For the standardized random variable U = X nµ nσ 2 we have lim P(U u) = Φ(u), n where Φ(u) is the distribution function of the standard normal distribution N(0, 1). a E(X ) = nµ and D(X ) = nσ 2

14 for the Mean Theorem Let the random variable X be the mean of n independent random variables X 1, X 2,..., X n, with the same distribution and the mean E(X i ) = µ and the finite variance D(X i ) = σ 2, i = 1,..., n, then E(X ) = µ and D(X ) = σ2 n and for the standardized random variable we have U = X µ n σ lim P(U u) = Φ(u), n where Φ(u) is the distribution function of the standard normal distribution N(0, 1).

15 For M = X X n is: M = n X i as.n(nµ, nσ 2 ), E(M) = nµ, D(M) = nσ 2 i=1

16 For M = X X n is: M = n X i as.n(nµ, nσ 2 ), E(M) = nµ, D(M) = nσ 2 i=1 U = M E(M) D(M) = M nµ nσ 2 as.n(0, 1)

17 For M = X X n is: M = n X i as.n(nµ, nσ 2 ), E(M) = nµ, D(M) = nσ 2 i=1 U = M E(M) D(M) P(M m) = F (m) Φ = M nµ as.n(0, 1) nσ 2 ( ) m nµ nσ 2

18 For M = X X n is: M = n X i as.n(nµ, nσ 2 ), E(M) = nµ, D(M) = nσ 2 i=1 U = M E(M) D(M) = M nµ nσ 2 as.n(0, 1) ( ) m nµ nσ 2 ) P(M m) = F (m) Φ ( P u 1 α/2 < m nµ < u nσ 2 1 α/2 = 1 α

19 For the sample mean X is: n X = 1 n X i as.n(µ, σ2 σ2 n ), E(M) = µ, D(M) = n i=1

20 For the sample mean X is: n X = 1 n X i as.n(µ, σ2 σ2 n ), E(M) = µ, D(M) = n i=1 U = X E(X ) D(X = X µ ) σ n as.n(0, 1)

21 For the sample mean X is: n X = 1 n X i as.n(µ, σ2 σ2 n ), E(M) = µ, D(M) = n i=1 U = X E(X ) D(X = X µ ) σ P(X x) = F (x) Φ n as.n(0, 1) ( x µ ) σ n

22 For the sample mean X is: n X = 1 n X i as.n(µ, σ2 σ2 n ), E(M) = µ, D(M) = n i=1 U = X D(X E(X ) = X µ ) σ n as.n(0, 1) ( P(X x) = F (x) Φ x µ ) σ n ( P u 1 α/2 < x µ ) σ n < u1 α/2 = 1 α

23 Continuity Correction In the case of using the normal distribution as an approximation of a distribution of a discrete random variable, it is recommended to apply so called continuity correction which improves this approximation. If we calculate P(X x) or P(X x) by normal approximation, we get undervalued results. On the contrary if we calculate P(X < x) or P(X > x) by normal approximation, we get overvalued results.

24 Continuity Correction Some examples of continuity correction: before correction x < 3 x 3 x = 5 x 7 x > 7 after correction x < 2.5 x < < x < 5.5 x > 6.5 x > 7.5

25 Example 1 The probability that you hit the target is 0.8. What is the probability that the difference between the number of hits in the sequence of 200 shots and the mean of the this number will not be large than 10?

26 Example 1 The binomial distribution: E(X ) = nπ = = 160 D(X ) = nπ(1 π) = (1 0.8) = 32

27 Example 1 The binomial distribution: E(X ) = nπ = = 160 D(X ) = nπ(1 π) = (1 0.8) = 32 P(150 X 170) = p(150) + p(151) + + p(170) = = ( ) ( ) ( ) = 0.937

28 Example 1 de Moivre-Laplace theorem: ( ) x nπ F (x) Φ nπ(1 π) P(150 X 170) = F (170) F (149) Φ ( ) Φ ( ) = 0.936

29 Example 1 de Moivre-Laplace theorem (with continuity correction): ( ) x nπ F (x) Φ nπ(1 π) P(150 X 170) P(149.5 < X < 170.5) = F (170.5) F (149.5) = = Φ ( ) Φ ( ) = 0.937

30 Example 1 Chebyshev s inequality: P( X E(X ) < ɛ) 1 D(X ) ɛ 2

31 Example 1 Chebyshev s inequality: P( X E(X ) < ɛ) 1 D(X ) ɛ 2 E(X ) = nπ = = 160 D(X ) = nπ(1 π) = (1 0.8) = 32

32 Example 1 Chebyshev s inequality: P( X E(X ) < ɛ) 1 D(X ) ɛ 2 E(X ) = nπ = = 160 D(X ) = nπ(1 π) = (1 0.8) = 32 P( X 160) < 10) = 0.68

33 Example 1 Chebyshev s inequality: P( X E(X ) < ɛ) 1 D(X ) ɛ 2 E(X ) = nπ = = 160 D(X ) = nπ(1 π) = (1 0.8) = 32 P( X 160) < 10) = 0.68 P( X 160) < 11) = 0.736

34 Example 2 In some elections the coalition obtained 52 % of votes. What is the probability that in the public opinion research of the size 2600 respondents the opposition won?

35 Example 2 X... the number of respondents who voted the opposition X B(2600; 0.48)

36 Example 2 X... the number of respondents who voted the opposition X B(2600; 0.48) E(X ) = nπ = = 1248 D(X ) = nπ(1 π) = (1 0.48) =

37 Example 2 P(X > 1300) = 1 P(X [ 1300) = 1 [p(0) + + p(1300)] = (2600 ) = ( ) ] = =

38 Example 2 de Moivre-Laplace theorem: ( ) x nπ F (x) Φ nπ(1 π) P(X > 1300) = 1 P(X 1300) = 1 F (1300) 1 Φ ( ) = =

39 Example 2 de Moivre-Laplace theorem (with continuity correction): ( ) x nπ F (x) Φ nπ(1 π) P(X > 1300) = 1 P(X 1300) 1 P(X < ) = = 1 Φ ( ) = = =