December 19, Probability Theory Instituto Superior Técnico. Poisson Convergence. João Brazuna. Weak Law of Small Numbers

Transcription

1 Simple to Probability Theory Instituto Superior Técnico December 19, 2016

2 Contents Simple to 1 Simple 2 to

3 Contents Simple to 1 Simple 2 to

4 Simple to Theorem - Events with low frequency in a large population follow a distribution even when the probabilities of the events varied. Also known as law of rare events, it was proposed by Ladislaus Josephovich Bortkiewicz in 1898.

5 What do we know? Simple to Bernoulli Distribution X Bernoulli(p) P(X = x) = { p, if x = 1 1 p, if x = 0 where p is the probability of success of a Bernoulli trial. Binomial Distribution If X 1,, X n : then are independent Bernoulli trials; have a constant probability of success p, S = n X j Binomial(n, p)

6 What are we going to see? Simple to Bernoulli Distribution X Bernoulli(p) P(X = x) = { p, if x = 1 1 p, if x = 0 where p is the probability of success of a Bernoulli trial. Binomial Distribution If X 1,, X n : then are independent Bernoulli trials; have eventually different small probabilities of success p j, S = n X j n a p j

7 Bernoulli, Binomial and Distributions Simple to Bernoulli and Binomial Distributions X j Bernoulli(p) S = n X j Binomial(n, p) i.i.d. Bernoulli and Distributions ( ) X j Bernoulli(p j ) S = n a n X j p j indep. Corollary S Binomial(n, p) S a (np)

8 Contents Simple to 1 Simple 2 to

9 Simple to Notation - Random Variables Let us consider a sequence of random variables { X n,j } such that n N and j {1,, n}. This means that for every n N we have n random variables X n,1, X n,2,, X n,n since the index j is a natural number between 1 and n. Notation - Probability of Success p n,j = P(X n,j = 1) Notation - Sum of Interest (also a sequence) S n = X n,1 + X n,2 + + X n,n = n X n,j

10 Theorem Simple to For each n N, let X n,j, with j {1,, n} be independent random variables such that P(X n,j = 1) = p n,j ; P(X n,j = 0) = 1 p n,j. which means that X n,j Bernoulli(p n,j ). Suppose that, as n : n 1 p n,j λ R + ; 2 max j {1,,n} p n,j 0. Then, where S n = X n,1 + + X n,n Z (λ). d Z

11 Auxiliary Lemma 1 Simple to Lemma Let z 1,, z n, w 1,, w n C with modulus not greater than ρ. Then, n z j n w j ρ n 1 n zj w j. Proof By induction over n. For n = 1: z 1 w 1 = ρ 1 1 z 1 w 1. For n > 1: n Induction Hypothesis: z j n w j ρ n 1 for a certain n N. n z j w j

12 Auxiliary Lemma 1 - Proof Simple to Proof (cont.) To prove: n+1 z j n+1 w j ρ n n+1 z j w j for the same n. n+1 n+1 z j w j = n n z n+1 z j w n+1 w j = n n n n = z n+1 z j z n+1 w j + z n+1 w j w n+1 n n z n+1 z j z n+1 w j + n n z n+1 w j w n+1 w j = w j

13 Auxiliary Lemma 1 - Proof Simple to Proof (cont.) n = z n+1 z j n w j + z n+1 w n+1 n I.H. wj n ρ ρ n 1 n+1 zj w j + zn+1 w n+1 ρ n = ρ n z j w j

14 Auxiliary Lemma 2 Simple to Lemma If z C with z 1 then e z (1 + z) z 2. Proof Using Taylor series, e z = 1 + z + z2 2! + z3 3! + z4 4! + = z n n! n=0 e z (1 + z) = z2 2! + z3 3! + z4 4! + = z n n!. n=2

15 Auxiliary Lemma 2 - Proof Simple to Proof (cont.) Since z 1, we have z n z 2, n {2, }. So, e z (1 + z) z 2 = 2 + z3 3! + z4 4! + = z 2 = z 3 + z (1 z ) 2 + = z = = z 2.

16 Theorem - Proof Simple to We are going to prove the result using Levy s Theorem, which states that if ϕ Sn (t) n ϕ Z (t) for all t R and ϕ Z is continuous at t = 0 then S n d Z. For each r.v. X n,j let ϕ n,j be its characteristic function. ( ) ϕ n,j (t) = E e itx n,j = = e it 1 P(X n,j = 1) + e it 0 P(X n,j = 0) = = e it p n,j + 1 (1 p n,j ) = = (1 p n,j ) + p n,j e it

17 Theorem - Proof Simple to So, S n s characteristic function is ) ϕ Sn (t) = E (e itsn = E (e it ) n X n,j = n n ( ) = E e itx n,j = E e itx n,j = indep. = = n ϕ n,j (t) = n n [1 + p n,j ( e it 1) ]. [ (1 p n,j ) + p n,j e it] =

18 Theorem - Proof Remember that if Z (λ), its probability mass function is given by Simple to P(Z = z) = so its characteristic function is ϕ Z (t) = E = ( e itz ) = z=0 {e λ λz z!, if z N 0 0, otherwise e itz P(Z = z) = z=0 e itz λ λz e z! = e λ ) z (λ e it z=0 = e λ e λ eit = e λ(eit 1). z! =

19 Theorem - Proof Simple to To prove that ϕ Sn (t) ϕ Z (t) n we are going to show that ϕ Sn (t) ϕ Z (t) 0. n Consider a sequence of r.v. n Y n p n,j so that ϕ Yn (t) = e n p n,j(e it 1). Since, by assumption, n p n,j λ, we have n ϕ Yn (t) ϕ Z (t) so, by Levy s Theorem,

20 Theorem - Proof Simple to d Y n Z. ϕ Sn (t) ϕ Yn (t) n ( ) = [1 ] + p n,j e it 1 e n n ( = [1 + p n,j e it 1) ] n e p n,j(e it 1) Note that 0 p n,j 1 (it s a probability); e it = 1 (it defines a unitary circle); ( 1 + p n,j e it 1) 1; e p n,j(e it 1) [ ] p = e n,j Re(e it ) 1 1 (exponent 0). p n,j(e it 1) =

21 Theorem - Proof Simple to Using auxiliary lemmas 1 and 2, with max j p n,j 1 2 and e it 1 e it = 2, n n 1 + p n,j = [1 + p n,j ( e it 1) ] ( ) e it 1 n ep n,j(e it 1) n p n,j ( ) e it 2 1 = n e p n,j(e it 1) e p n,j(e it 1) = ( ) [1 ] + p n,j e it L2 1 n p 2 n,j e it 1 2 L1

22 Theorem - Proof Simple to Since n n p n,j max j { pn,j } 2 2 = 4 max j { } n pn,j p n,j 0. p n,j λ and ϕ Yn (t) ϕ Z (t) as n, ϕ Sn (t) ϕ Z (t) = ϕ Sn (t) ϕ Yn (t) + ϕ Yn (t) ϕ Z (t) ϕ Sn (t) ϕ Yn (t) + ϕ Yn (t) ϕ Z (t) = 0 we conclude that ϕ Sn ϕ Z so, by Levy s Theorem, S n d Z.

23 Contents Simple to 1 Simple 2 to

24 Some Simple to

25 World War II Example Simple to Let U n,j Unif [ n, n], indep. { 1, if U n,j ]a, b[ [ n, n] X n,j = 0, otherwise and S n = n X n,j the number of points that land on ]a, b[. So p n,j = P(X n,j = 1) = P(U n,j ]a, b[) = n p n,j = n b a 2n = b a b a max j {1,,n} p n,j = max j {1,,n} 2n S n d Z ( b a 2 b a 1 2n du = b a 2n 2 R + does not depend on n; = b a 2n 0. ),

26 World War II Example Simple to The flying bomb hits in South London during World War II fit a distribution. The area was divided on 576 regions with the same area. The total number of hits was 537. Adapting our last result for R 2, let S be the number of regions with k hits. ( ) S a k P(S = k) Real Values

27 Contents Simple to 1 Simple 2 to

28 Simple to Theorem - For each n N, let X n,j, with j {1,, n} be independent non-negative integer valued random variables such that P(X n,j = 1) = p n,j ; P(X n,j 2) = ɛ n,j. Suppose that, as n : n 1 p n,j λ R + ; 2 max j {1,,n} p n,j 0 ; n 3 ɛ n,j 0. Then, where S n = X n,1 + + X n,n Z (λ). d Z

29 Simple to Proof Let and X n,j = { 1, if X n,j = 1 0, otherwise S n = X n,1 + + X n,n = n X n,j. By the first two assumptions and using the, we conclude that S n d Z.

30 Simple to Proof (cont.) By the third assumption, we know that P(S n S n) 0. Using the Converging Together Lemma, d S n Z.

31 Simple Converging Together Lemma If d X n X ; then Y n d c where c R is a constant, d X n + Y n X + c. to Corollary If then S n d Z ; S n S n d 0, S n d Z.

32 Contents Simple to 1 Simple 2 to

33 Simple to Definition - Counting (in Continuous Time) A family of r.v. { N(t) : t 0 } is said to be a counting process if it represents the total number of events that have occurred up to time t. It must satisfy: N(t) N 0, t 0 ; N(s) N(t), 0 s < t ; N(t) N(s) corresponds to the number of events that occurred in the interval ]s, t], 0 s < t.

34 Simple to Definition - Counting with Independent Increments The counting process { N(t) : t 0 } is said to have independent increments if the number of events that occur in disjoint intervals are independent r.v., that is, if 0 = t 0 < t 1 < < t n then N(t 1 ), N(t 2 ) N(t 1 ),..., N(t n ) N(t n 1 ) are independent r.v. Definition - Counting with Stationary Increments The counting process { N(t) : t 0 } is said to have stationary increments if the distribution of the number of events that occur in any interval depends only on the length of the interval, that is, N(t 2 + s) N(t 1 + s) d = N(t 2 ) N(t 1 ), s 0, 0 t 1 < t 2.

35 Simple to Definition 1 - The counting process { N(t) : t 0 } is said to be a process with rate λ > 0 if: { N(t) : t 0 } has independent and stationary increments; N(t) (λt). This is the usual definition, but it is redundant since the second condition follows from the first one.

36 Simple to Definition 2 - The counting process { N(t) : t 0 } is said to be a process with rate λ > 0 if: lim h 0 N(0) = 0; { N(t) : t 0 } has independent and stationary increments; P[N(h) = 1] = λh + o(h); P[N(h) 2] = o(h) where o(h) stands for functions g 1 (h) and g 2 (h) such that g i (h) h = 0, i {1, 2}. Let us check that this definition implies that N(t) (λt).

37 Simple to Theorem Let { N(t) : t 0 } be a counting process in continuous time. Suppose that: N(0) = 0; { N(t) : t 0 } has independent and stationary increments; P[N(h) = 1] = λh + o(h); P[N(h) 2] = o(h) where o(h) stands for functions g 1 (h) and g 2 (h) such that g lim i (h) h 0 h = 0, i {1, 2}. Then, N(0, t) (λt).

38 Simple to Proof Let X n,j = N ( ) ( ) j n t N j 1 n t. [ ( j p n,j = P(X n,j = 1) = P N n t [ ( ) ] t = P N N(0) = 1 n ) = λ t ( t n + o n n n p n,j = P(X n,j = 1) = ) ( j 1 N n = P [ N ( ) t n ) ] t = 1 = = 1 ] = [ n λ t ( ) ] t n + o λt. n

39 Simple to Proof (cont.) { max p n,j = max λ t n + o ( t n) } = p n,j j {1,,n} j {1,,n} because it does not depend on j. Therefore, as n, max p n,j = λ t ( ) t j {1,,n} n + o 0. n Finally, [ ( j 1 ɛ n,j = P(X n,j 2) = P N n [ ( ) ] t = P N N(0) 2 = o n ) ( j t N ( ) t 0 n ) ] n t 2 =

40 Simple to Proof (cont.) thus n ɛ n,j 0. We are on the conditions of the last theorem, since X n,j are non-negative integer valued r.v. with p n,j = P(X n,j = 1), ɛ n,j = P(X n,j 2) and n 1 p n,j λt R + ; 2 max j {1,,n} p n,j 0 ; n 3 ɛ n,j 0. Therefore, n X n,j d N(t) (λt).

41 Bibliography I Appendix Bibliography Rick Durrett. Probability Theory and. CUP, Isabel Rodrigues. Apontamentos de Complementos de Probabilidades e Estatística. Instituto Superior Técnico, Manuel Cabral Morais. Lecture Notes - Probability Theory. Instituto Superior Técnico, Manuel Cabral Morais. Lecture Notes - Stochastic es. Instituto Superior Técnico, 2014.