Probability Reference

Size: px

Start display at page:

Download "Probability Reference"

Isabel Dixon
5 years ago
Views:

1 Pobability Refeece Combiatoics ad Samplig A pemutatio is a odeed selectio. The umbe of pemutatios of items piced fom a list of items, without eplacemet, is! P (; ) : ( )( ) ( + ) {z } ( )! : () -factos Whe selectig with eplacemet, the umbe of possibilities is. A combiatio is a uodeed selectio. The umbe of combiatios of items chose fom a list of di eet items, without eplacemet, is P (; ) ( )( ) ( + )! C(; ) : :! ( )( )!( )! {z } -factos, umeato ad deomiato The umbe of ways to select objects fom di eet items with eplacemet is + ( + )( + ) ( + ) : ( )( ) {z } -factos, above ad below (This is also the umbe of oegative itege solutios of the equatio ad the umbe of ways to distibute idetical objects ito distict boes.) The umbe of distict ways of distibutig objects ito distict classes of size ; ; : : : ;, without eplacemet ad with o ode withi each class, is! : ; ; : : : ;!!! ; whee Biomial Theoem: (a + b) X a b X a b Multiomial Theoem: (a + a + + a ) X a ; ; : : : ; a a ; whee the sum is tae ove all oegative itege values of ; ; : : : ; fo which Stilig s Fomula:! : p (e) o moe accuately! : p + +() e Biomial coe ciet idetities: P P ; + ; ++ ; + ( ) +, fo > 0; P ( ) 0; P m+ ; m p p p P ; mp m+ m + m+ p +p p ;

2 Some useful seies: P P ( + ) P m P Pobability m + P log ( ), fo jj < P ( t) P + t P + t, jtj < P 6 ( + ) ( + ), fo jj < P ()! cosh P 3 4 ( + )! e + (+)! sih If A, B, ad C ae evets (de ed as subsets of the sample space S of all possible outcomes of a epeimet) the P is a pobability measue, whe the followig ae tue: (i) 0 P(A), (ii) P ( S A i) P i P (A i), fo paiwise disjoit A i ; (iii) P(S), P (?) 0. The complemet of a evet is de ed to be A 0 f : Ag, the P (A 0 ) P (A) is the Law of Complemets; P (A [ B) P (A) + P (B) P (A \ B) is the Piciple of Iclusio-Eclusio. Coditioal pobability of A give B, P (AjB) : P (A \ B) ; whe P (B) > 0; P (B) this implies P (A \ B) P (AjB) P (B) P (BjA) P (A) P (B \ A). The evets A ; A ; : : : ; A ae idepedet if P (A \ A \ \ A ) P (A ) P (A ) P (A ) ; fo f ; ; : : : ; g ay subset of :. This implies that A # i ; A # i ; : : : ; A # i s ae idepedet, whee A # ca be eithe A o A 0, sepaately fo each set as :. If B fb ; : g is a patitio of the sample space S, meaig B i \ B j? fo i 6 j ad S i B i S, the the Law of Total Pobability says X P (A) P (AjB i ) P (B i ) ; ad Bayes fomula is i P (B ja) P (AjB ) P (B ) P i P (AjB i) P (B i ) Discete Radom Vaiables. X has pobability mass fuctio pmf f() if (i) f() 0, (ii) P f(), (iii) f( ) P(X ):. X has cumulative distbutio fuctio cdf F () if F () : P(X ) X y f(y); P(a < b) F (b) F (a); f( ) F ( ) F ( ). Cotiuous Radom Vaiables. X has pobability desity fuctio pdf f() if (i) f() 0, (ii) Z b a f() d.. X has cdf F () if F () : Z f() d; P(a X < b) F (b) Z F (a); f() df d. f() d, (iii) P(a < b) 3. The media ~ satis es F (~) ad the pth pecetile p satis es F ( p ) p. The itequatile age is IQR : 0:75 0:5 ad the itedecile age is IDR : 0:90 0:0

3 Discete ad Cotiuous cdfs F () is (i) odeceasig, (ii) Idepedet ad Echageable Radom Vaiables lim F () 0, (iii) lim F (), ad (iv) F () is ight cotiuous.!! The vs X ; X ; : : : ; X ae idepedet if ad oly if the joit pf is the poduct of the magials, i.e., f ( ; ; : : : ; ) f ( )f ( ) f ( ) The vs X ; X ; : : : ; X ae echageable if ad oly if the joit pf is ivaiat ude itechages of its agumets, i.e., f ( ; ; : : : ; ) f ( i ; i ; : : : ; i ) ; fo ay pemutatio (i ; i ; : : : ; i ) of :. Idepedet ad idetically distibuted (iid) vs ae echageable, but idepedece ad echageability, although ovelappig i some aeas, ae distict cocepts. Fo istace, idepedece does ot imply echageability. No does echageability imply idepedece. Epectatio Values By de itio, the epectatio of a fuctio of a v is 8 P < g()f(); E (g(x)) : age(x) : g()f() d; X R X discete, cotiuous. Fo vs of the mied type with pobability fuctio pf f() f disc () + ( )f cot (), you ca oly de e the momets E X X Z f disc () + ( ) f cot () d discete(x) cotiuous(x). th momet is 0 : E (X ) ; the mea is : E (X) ; th cetal momet is : E (X ) ; th absolute deviatio is : E (jx j ); the vaiace is va(x) : : E (X ) E X ( ) ( ) Coe ciet of sewess is : E ((X ) ) 3 ad coe ciet of ecess is E ((X ) ) E ( P a X ) P a E (X ) ; va ( P a X ) P a va(x ) + P P j< a ja cov(x j ; X ). The covaiace ad coelatio ae de ed by: cov(x j ; X ) : E (X j j )(X ) E (X j X ) j : j ; j : co(x j ; X ) j j X cov ai X i ; X b j X j X a i b i va (X i ) + X X (a i b j + a j b i ) cov (X i ; X j ) 4. Coditioal epectatios: E (Y ) E (E(Y jx)) ad va(y ) E (va(y jx)) + va (E(Y jx)) ; whee va (Y jx) : E (Y E(Y jx)) jx. 5. Whe X ad Y ae idepedet vs, va(xy ) va(x) va(y ) + E (X) va(y ) + E (Y ) va(x) i<j 3

4 Geeatig Fuctios Momet geeatig fuctio, mgf: M X (t) : E e tx, M () X (0) 0, M ax+b (t) e bt M X (at), MX 0 (0), MX 00 (0) (M X 0 (0)). Cumulat geeatig fuctio, cgf: K X (t) : log (M X (t)), K 0 X (0), K00 X (0), K 000 X (0) E (X )3, the th cumulat is K () X (0) E (X ). Factoial geeatig fuctio, fgf: P X (s) : E s X, P () X () [] : E (X(X ) (X + )). Chaacteistic fuctio, cf: ' X (!) : E e i!x, ' () X (0) i 0, ' ax+b (!) e ibt X '(a!). Ode Statistics A adom sample of size is a set fx ; X ; : : : ; X g of idepedet ad idetically distibuted (iid) vs. The ode statistics of the adom sample ae de ed to be X (;) X (;) X (;). We assume they ae daw fom a populatio with pdf f() ad cdf F ().. The pdf ad cdf of Y X (;) ae give by g (y) [F (y)] ; ; f(y) [ F (y)] ; X G (y) [F (y)] i [ i F (y)] i i The joit pdfs of two ode statistics, Y X (;) Y s X (s;) ae give by g ;s (y ; y s ) [F (y )] f(y ) [F (y s ) F (y )] s f(y s ) [ F (y s )] s, y y s ; ; s ; ; s ad the pdf of all the ode statistics is g(y ; y ; : : : ; y )! f(y ; y ; : : : ; y ) fo y y y. The pdf ad cdf of the age, R : X (;) X (;), ae give by Tasfomatio of Vaiables f R () ( ) Z [F ( + ) F ()] f()f( + ) d; Z F R () [F ( + ) F ()] f() d If Y u(x) is a smooth oe-to-oe tasfomatio, the G(y) P (Y y) P (u(x) y) P X u (y) F u (y) The coespodig pdf is the deivative: g(y) f ((y)) d dy. If the tasfomatio is ot oe-to-oe, bea up its suppot ito a uio of itevals ove each of which it is oe-to-oe ad apply the pevious fomula to each piece ad sum the esult. E.g., Y X ; G(y) P (Y y) P X y P ( p y X p y) F ( p y) F ( p y) ; so that g(y) p y (f (p y) + f ( p y)) Fo discete vs, the pmf is g(y ) F u (y ) F u (y ) f u (y ). You should ow that fo ay cotiuous v, both U F (X) ad V F (X) ae Uif (0; ). 4

5 If Y : [Y ; Y ; : : : ; Y ] [u (X ; X ; : : : ; X ) ; : : : ; u (X ; X ; : : : ; X )] is a smooth ivetible multivaiate tasfomatio, the use the Jacobia Chage of Vaiable Theoem to wite, g (y ; y ; : : : ; y ) f ( (y); (y); : : : ; ; ; : : : ; (y ; y ; : : : ; y ) ; whee the Jacobia is de ed to ; ; : : : ; (y ; y ; : : : ; y ) : det Fo sums of idepedet vs, use the mgf esult: If S X + X + + X, the M S (t) M X (t)m X (t) M X (t); which fo the iid case educes to M S (t) MX (t). Also fo sums of adom vaiables, the pdf, f(s), of the sum is elated to the pdfs of the idividual X i, p i (), via the covolutio poduct f p p p, whee the poduct is de ed ecusively by (p p ) () : Z p ( y)p (y) dy; p p p p, ad p (p p 3 ) (p p ) p 3. This is usually ot vey useful ecept fo distibutios fo which f(s) ca be moe easily calculated othe ways, e.g., mgfs. (See the last sectio of this efeece sheet.) De itios ad Results If X has cdf F () fo : ad if fo some cdf F () we have lim! F () F () fo all values of at which F () is cotiuous, the the sequece fx g coveges i distibutio to X, which has cdf F (), ad we wite X d! X. If X has mgf M (t), X has mgf M (t), ad thee is a a > 0 such that lim! M (t) M (t) fo all t ( a; a), the X d! X. We say the sequece fx g coveges stochastically to a costat c if the limitig distibutio puts all its mass at the atom fcg, witte X P! c. The sequece fx g coveges i pobability to X if lim! P (jx witte as X P! X. Xj < "), fo ay " > 0. This is If 0 : f! : lim! X (!) X eistsg ad P ( 0 ), the we say that X coveges almost suely a.s. ad we wite X! X. Slutsy s Theoem says: (a) If X P! X, the X d! X: (b) If X P! c, the g(x ) P! g(c), wheeve g is d P d d d cotiuous at c. (c) If X! X ad Y! c, the (i) X + Y! X + c, (ii) X Y! Xc, (iii) X Y! Xc. d (d) If X! X, the fo ay cotiuous fuctio g(y), g (X )! d g(x). Cetal Limit Theoem: (Fom ) If X ; X ; : : : ; X ae iid fom a distibutio with mea ad vaiace <, the P lim Z : lim X i!! p Z N (0; ) (Fom ) If as above, the lim Z X : lim!! p Z N (0; ) 5

6 (Bey-Essee Boud) If, i additio, (E jx i j) + + <, fo some (0; ], the thee is a costat c such that sup P X < z p (z) : R c + The case is most ofte cited: (E jx i j) 3 3 yields sup P X < z p (z) : R ad c :3. Special Discete Radom Vaiables c 3 p ;. Bi(; p), Biomial: X # successes i, a ed umbe of idepedet Beoulli tials with costat p P(Success) : q. b(; ; p) p q ; 0 : ; p; pq; () () p ; M X (t) (pe t + q).. Hype(; N; ), Hypegeometic: X # defectives i samplig items without eplacemet fom a set of N items of which D ae defectives. h(; ; N; D) N D N D D ; N ; maf0; + D Ng : mifd; g N N D N D ; N [] [] D [] N [] 3. Pois(), Poisso: X # of occueces of evets occuig adomly ad idepedetly i a time T ad at a ate whe T. p(; )! e ; 0 : ; ; 0 ( + ) [] ; M X (t) ep e t The Law of Rae Evets tells us that the limit of Bi (; p) as!, p! 0, ad p is Pois (). 4. Bi (; p), Negative Biomial: X # of tials util th success, o NBi(; p): Y # failues util th success X. b (; ; p) p q ; : ; X p ; X q p ; + y f(y) p ( q) y p q y ; y 0 : ; y y Y q p ; Y q p M X (t) p p (e t q) ; M Y (t) ( qe t ) (a) Geo(p), Geometic: X # of tials util the st success. This is Bi (; p). So, f (; p) q :. p, q p, M X (t) p (e t q). p, fo 5. Mult (; p), Multivaiate: X i # of occueces fallig ito categoy i whe the pobability of havig a outcome i each categoy is the same fo each idepedet tial. P (X ) p : ; ; : : : ; p p p ; X i ad ad E (X i ) p i, va (X i ) p i ( p i ), ad cov (X i ; X j ) p i p j fo i 6 j. X p i ; 6

7 Special Cotiuous Radom Vaiables The idicato fuctio is de ed by I (a;b) () ; (a; b) 0; (a; b). Uif(a; b), Uifom: f(; a; b) b a I (a;b)(); ~ (a + b); (b a) ; M X (t) ebt e at t(b a) ; 0 b+ a + (b a) ( + ) ; 0; 6 5. N (; ); Gaussia o Nomal: (; ; ) ( : p ep ~ ; ; 0; 0; M X (t) ep t + t Fo the stadad omal, Z N (0; ), the pdf is (z) p e z ), 3. log N (; ), log Nomal: ep + ; if! e, the! (! ) e ; (! + ) p!,! 4 +! 3 + 3! 6, ~ e ; 0 ep + f (; ; ) p ep (log ) I (0;) () 4. ivg(; ), ivese Gaussia o ivese Nomal:, 3, 3 p, 5, ( ) f (; ; ) p ep ( ) 3 I (0;) whe > 0 ad > 0 ad the mgf is h M X (t) ep whee the semifactoial is de ed by p i ( + t) ; ( 3)!! ()!! 4 6 () ( )!! 3 5 ( ) 5. Cauchy(; ), Cauchy: f() + ( ) ; > 0, ad do ot eist but ~ ad the chaacteistic fuctio jtj ' X (!) ep i! is the oly geeatig fuctio that eists. The paamete is oe half the itequatile age, i.e., IQR (Q 3 Q ) ( 0:75 0:5 ). 6. Ep() Gam(; ), Epoetial: f() e I (0;) (), > 0; ; ; 0!; M X (t) ( t) : This is the distibutio of the time util the et occuece of a adom evet. 7

8 7. Gam(; ), Gamma: f (; ; ) 6, () e I (0;) (), ; > 0, ; ; p, 0 ( + ) () ; M X (t) ( t) If is a positive itege, the this is the distibutio of the time util the th occuece of a adom evet. 8. Gam(; ), Chi-Squae: f(; ) () () e I (0;) (), fo > 0; ; ; Mode ; 0 + ; 8 ; ; M X(t) ( t) 9. Beta(; ), Beta: f (; ; ) ( ) I (0;) () fo ; > 0; B(; ) + ; ( + + )( + ) ; B( + ; ) 0 B(; ) Y Weib(; ), Weibull: f (; ; ) ep I (0;) (); + ; 0 + ; + +. Lap(; ), Laplace o Double Epoetial: f (; ; ) ep f j j g; ~ ;. Logist (; ), Logistic: ; 0; 3; ()! ; M X (t) t e ( ) f (; ; ) s + e ( ) ; F (; ; ) + e et ( ) ; 3 ; 0; :; M X (t) e t B ( t; + t) whee B is the beta fuctio 3. vm (; ), vo Mises: f (; ; ) I 0() ep ( cos ( )) I ( ;) (), whee I 0 () is the modi ed Bessel fuctio of ode 0 ad > 0; ~ ; Some limits ae I () I 0 () ; cf ' X (!) I j!j () I 0 () ei! ; lim f (; ; )!0 I ( ;) () ; lim f (; ; ) p ep ( )o! lim vm (; ) Uif( ; ); lim vm (; ) N (;!0! ) 4. Pa (m; ), Paeto: f (; m; ) m + I (m;) (), with m ad both positive. m, fo > ; ~ m ; m ( ) ( ), fo > 0 m fo < 8

9 5. Et (; ), Eteme Value: cdf F (; ; ) ep e ( ), fo > 0. + ; 6 ; ~ log (log ) ; M X (t) e t ( t), fo t < 6. t, t-distibutio: t N (0; ) whe umeato ad deomiato ae idepedet ad > 0; + f(; ) (+) p B ; t oly has momets up to ode ; 0; ; 0 fo 4 :, ad 6 4 fo 5 :, hece, the mgf does ot eist. 7. F m;, F -distibutio: F m; mm, whe umeato ad deomiato ae idepedet. f(; m; ) mm B m ; (m 8. N ( ; ; ; ; ), Bivaiate omal: whee Q : ) ( + m) (m+) I (0;) (); ; (m + ) m( ) ( 4) f( ; ) p " ep Q ; # + The X N ( ; ), X N ( ; ), Xjy N ; ( ), ad Y j N y ; ( ), whee Also + (y ) ad y + ( ) M X;X (t ; t ) ep t + t + t + t t + t 9. 0 () Nocetal chi-squae : If Z ; Z ; : : : Z ae idepedet N ;, the X 0 X X () whee : is the ocetality paamete The pdf is f (; ; ) X! e ()+ e ()+ + I (0;) fo > 0; : E (X) +, va (X) ( + ), M (t) ( t) ep ft ( t)g, ( )! ( + ) 0 + X () + 0. t 0 () Nocetal t: t 0 () : (N (; )) p ad the pdf is f (; ) e X! + + () p ( + ) (+)! + I (0;) fo >, va (X) + (( ) ) () fo > 9

10 . F 0 m; () Nocetal F : F 0 m; () 0 m () ad the pdf is f (; m; ; ) e X! (m + + ) (m + ) (m+) ( + ) I (m++) (0;) (m + ) ( ) m fo >, va (X) (m + ) + (m + ) (m + ) ( ) ( 4) m ( ) m fo > 4 AdditioTheoems, Divisio Statemets, Miscellaeous Relatios Each of the followig sums ae of idepedet vs of the type idicated.. P Bi( ; p) Bi ( P ; p). P Geo(p) Bi (; p) 3. P Pois( ) Pois ( P ) 4. P Ep() Gam(; ) 5. P Gam( ; ) Gam( P ; ) 6. P a N ( ; ) N P a ; P a 7. fn (0; )g 8. P 9. P X ; : : : ; X iid N (; ), X N (; ) idepedet of (. If X; Y iid N (0; ) the S ) ( ) (a) X jxj Cauchy(0; ) t (b) X+Y X Y (c) U Cauchy(0; ) p XY X 0; N p idepedet of V X Y +Y X +Y. X ; : : : ; X iid Uif(0; ) ) X (;) Beta(; + ) 3. X Uif(0; ) ) log X () 4. X ; X ; : : : ; X iid Ep () ) X (;) Ep () which is also omal. 5. X Gam(; ) ) X 6. X m idepedet of Y ) X X+Y Beta m ; 7. F F m; ) (m)f +(m)f F m; 8. X Beta( ; ) idepedet of Y Beta( ; ) + ) XY Beta( ; + ) + ) XY Beta( ; + ) 9. (X; Y ) N (0; 0; ; ; ) ) Y X Cauchy(0; ) 0. X NBi(; p) ad Y Bi(; p) ) P (X ) P (Y ). I tems of cdfs, this is F X (; ; p) F Y (; ; p). X Gam(; ) ad Y Pois () ) P (X ; ; ) P (Y ; ; ). I tems of cdfs, this is F X (; ; ) F Y ( ; ; ) 0

11 . X log N ( ; ) ad Y log N ( ; ) idepedet, the XY log N ( + ; + ) ad XY log N ( ; ) 3. Fo ay cotiuous v X with cdf F (), the th ode statistic X (;) has cdf G (y) H (F (y); ; + ), whee H is the cdf of a Beta(; + ) v. 4. Gamma fuctio: ( + ) () : R t e t dt; () ; ( + )! whe is a oegative 0 p itege, ad 5. R 0 t e t dt (+) +, > 0 6. Icomplete gamma fuctio: (a; ) : R 0 ta e t dt fo a > 0 ad > 0. P (a; ) : (a; ) (a) is the cdf of the gamma distibutio. The coespodig tail pobability is (a; ) (a) : (a; ) (a) (a) Z t a e t dt 7. Beta fuctio: B (; ) : R 0 t ( t) dt () () (+), fo > 0, > 0 8. Icomplete beta fuctio: B (; ) : R 0 t ( t) dt. I (; ) : B (; ) B (; ) is the cdf of the beta distibutio.

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ