1. Review of Probability.

Transcription

1 1. Review of Pobability. What is pobability? Pefom an expeiment. The esult is not pedictable. One of finitely many possibilities R 1, R 2,, R k can occu. Some ae pehaps moe likely than othes. We assign nonnegative numbes p i P [R i ] such that p i 0 and i p i 1. The intepetation is that we know (fom expeience?) that, if we epeated the expeiment a lage numbe of times, these events would occus moe o less in these these ppotions. In othe wods if we epeat the expeiment N times, fo lage N, #(R i ) N p i Often when thee is no eason to pefe one ove the othes we may set P (R i ) 1 k. 1. Examples. Toss a coin. H o T. P (H) P (T ) Thow a die. 1, 2,..., 6. P (1) P (2) P (6) 1 6 Repeated Expeiments. Toss a coin twice. P (HH) P (HT ) P (T H) P (T T ) 1 4. Independence. If P [R i ] p i, then if we epeat twice unde the assumption of independence we have P [R i R j ] P [R i ]P [R j ]. They can be diffeent expeiments. P [R i S j ] P [R i ]P [S j ] You can have many expeiments that ae mutually independent. Fo example fo any sting of length n, P [HHT T HT T H T T ] 1 2 n Absactly X is a finite set of points {x} o {x 1,..., x k } and {p(x)} o {p i } ae numbes adding up to 1. We extend the definition to subsets A X. P (A) i:x i A p i i:x A p(x) P has the popeties 0 P (A) 1. P (X) 1. P ( ) 0 and if A B, then P (A B) P (A) + P (B). 1

2 Wehn diagams. P (A B) P (A) + P (B) P (A B). Mappings. F : X Y. Thee is a natual Q on Y defined by Q(B) P (F 1 A) p i p(x) i:f (x i ) B x:f (x) B If Y R then F is called andom vaiable. Example x {HHT T H} a sting of length n. p(x) 1 2. F is the numbe of heads. {x : F (x) } n has ( n ) stings in it. So ( n ) P [F ] 2 n The heads and tails may have unequal pobabilities. P (H) p and P (T ) 1 p. Then p(x) p F (x) n F (x) (1 p) Theefoe P [F ] ( ) n p (1 p) n Expectations. If X R, then the mean of the distibution p is defined as m x xp (x) Moe geneally if F is andom vaiable then E[F (x)] x F (x)p(x) y yq(y) whee q(y) P [F (x) y] x:f (x)y p(x). If F and G ae andom vaiables then E[aF + bg] ae[f ] + be[g] If P is on X and Q is on Y, then on Z X Y R P Q is the poduct distibution defined by ({x, y}) p(x)q(y). Then it is easy to veify that E[F (x)g(y)] x,y F (x)g(y)p(x)q(y) [ x F (x)p(x)] [ y G(y)q(y)] E[F (x)]e[g(y)] 2

3 Fo the Binomial Distibution m ( ) n p (1 p) n np F F 1 + F F n. Each F i 1 o 0 with pobability p and E[F i ] p and E[F ] n p. Waiting Times; Suppose we have independent tosses with P (H) p and P (T ) 1 p, F is the numbe of ties befoe a Head shows up, (including the last one), then we obtain the Geometic distibution. Fo the Geometic Distibution P [F ] p(1 p) 1 E[F ] p(1 p) 1 1 p Could you have guessed it? Conditional Pobability. P (A B) P (A B) P (B) Example. Dawing without eplacement. We have an un containing ed and g geen ball. A ball is dawn at andom. Then anothe ball is dawn at andom with out eplacement. A is the event that the fist ball is ed. B is the event that the second ball is geen. It is clea that What about P (A B)? P (B A) g g + 1 P (A B) P (A B) P (B) P (A)P (B A) P (B) + g 1 3

4 Bayes ule; B i ae disjoint and thee union is X the whole space. Then P (A) i P (B i A) i P (B i )P (A B i ) In the pevious example P (B) P (B A)P (A) + P (B A c )P (A c ) g g + 1 g + + g 1 g + 1 g g + Conditional Expectation. g g + If F (x) is a andom vaiable on X {x} with pobbailities {p(x)}, the expectation of F on any A can be defined as x A E[F A] F (x)p(x) x A p(x) 1 F (x)p(x) P (A) If X and Y ae two andom vaiables then E[Y X] f(x) x A whee f(x) is defined fo evey x with P [X x] > 0 by the fomula It is easy to check that f(x) E[Y X x] yp [Y y X x] 1 yp [X x, Y y] P [X x] y E[Y ] E[f(X)] E[E[Y X]] Mean and Vaiance. 4

5 Let X {x} be a be a finite set with associated pobabilities {p(x)}. We saw that if Y f(x) is a andom vaiable then, with q(y) P [Y y] x:yf(x) p(x), E[Y ] E[f(x)] yq(y) x f(x)p(x) We can similaly define E[Y 2 ] y y2 q(y). The Vaiance of Y is defined as E[[Y E[Y ]] 2 ] E[Y 2 ] 2[E[Y ]] 2 + [E[Y ]] 2 E[Y 2 ] [E[Y ]] 2 Measues the spead. V (ax + b) a 2 V (X). If X and Y ae independent andom vaiables the V a(x +Y ) V a(x)+ V a(y ). If we expand [X E(X) + Y E(Y )] 2 we get an additional coss tem 2[X E(X)][Y E(Y )] and if X and Y ae independent E[[X E(X)][Y E(Y )]] E[[X E(X)]] E[[Y E(Y )]] 0 Some impotant Discete distibutions. 1. Binomial Distibution. {1, 2,..., n}. P () ( n ) p (1 p) n. Mean np. Vaiance np(1 p). One way to compute is the use of geneating functions. E[e θx ] M(θ) 0 θ E[X ]! e θ ( n M (θ) n(pe θ + 1 p) n 1 pe θ ) p (1 p) n (pe θ + q) n M (θ) n(n 1)(pe θ + 1 p) n 2 p 2 e 2θ + n(pe θ + 1 p) n 1 pe θ Mean M (0) np. Vaiance M (0) [M (0)] 2 n(n 1)p 2 + np n 2 p 2 np np 2 np(1 p) 5

6 2. Geometic Distibution. {0, 1, 2,...,...}. P () p(1 p) 1 M(θ) 1 p(1 p) 1 e θ pe θ 1 (1 p)e θ p p + e θ 1 M (0) 1 p Mean 1 p. Vaiance 1 p 2 M (0) 2 p 2 1 p 1 p 1 p p 2. λ λ 2. Poisson Distibution. {0, 1, 2,...,...}. P () e! M(θ) e λ e λeθ Mean M (0) λ, Vaiance λ. Binomial p << 1, n >> 1 np λ, then as n,p 0, np λ np(1 p) λ. ( n )p (1 p) n λ λ e! Sums of Independent Random vaiables. P [X ] p() P [Y ] q(). X and Y ae independent. π() P [X + Y ] a+b p(a)q(b). π p q is the convolution of p and q. Pobability geneating functions. P (z) p()z. Replace e θ by z. Binomial: (p + qz) n Geomteic: pz 1 (1 p)z Poisson: e λ(z 1). Bin(n, p) Bin(m, p) Bin(n + m, p) P oisson(λ) P oisson(µ) P oisson(λ + µ) [ Negative Binomial: Convolutions of Geometic. P n [X n + ] ( ) n+ 1 (1 p) p n pz 1 (1 p)z Distibution functions. F X (t) P [X t] x t P [X x] 6 ] n

7 If X and Y ae independent and Z max{x, Y }, then F Z (t) F X (t)f Y (t). Assignment 1. k dice ae thown. Assume that all the sides have the same pobability of showing up and the scoes {X 1,..., X k } of the k dice ae independent. What is the pobability distibution of F max 1 i k X i? Calculate E[F ] and V (F ). What happens when k is lage? 7