D1.7 Two independent Events: Events A and B are independent if and only if P[AB] = P[A]P[B]. When events A and B have nonzero probabilities, the

Transcription

1 Theores d Defiitios T De Morg s lw reltes ll three bsic opertios: (A B) = A B D Outcoe: A outcoe of experiet is y possible observtio of tht experiet D Sple spce: The sple spce of experiet is the fiest-gri, utully exclusive, collectively exhustive set of ll possible outcoes D3 Evet: A evet is set of outcoes of experiet D4 Evet Spce: A evet spce is collectively exhustive, utully exclusive set of evets T For evet spce B = {B, B, } d y evet A i the sple spce, let C i = A B i For i j, the evets Ci d Cj re utully exclusive d A = C C D5 Axios of Probbility: A probbility esure P[ ] is fuctio tht ps evets i the sple spce to rel ubers such tht Axio : For y evet A, P[A] 0 Axio : P[S] = Axio 3: For y coutble collectio A, A, of utully exclusive evets, P[A A ] = P[A ] + P[A ] + T3 For utully exclusive evets A d A, P[A A ] = P[A ] + P[A ] T4 If A = A A A d A i A j = φ for i j, the P[A] = i= P[A i ] T5 The probbility of evet B = {s, s,, s } is the su of the probbilities of the outcoes cotied i the evet: P[B] = i= P[{s i }] T6 For experiet with sple spce S = {s,, s } i which ech outcoe si is eqully likely, P[s i ] = i T7 The probbility esure P[ ] stisfies () P[φ] = 0 (b) P[A ] = P[A] (c) For y A d B (ot ecessrily disjoit), P[A B] = P[A] + P[B] P[A B] (d) If A B, the P[A] P[B] T8 For y evet A, d evet spce {B, B,, B }, P[A] = i= P[A B i ] D6 Coditiol Probbility: The coditiol probbility of the evet A give the occurrece of the evet B is P[A B] = P[AB] P[B] T9 A coditiol probbility esure P[A B] hs the followig properties tht correspod to the xios of probbility Axio : P[A B] 0 Axio : P[B B] = Axio 3: If A = A A with A i A j = φ for i j, the P[A B] = P[A B] + P[A B] + T0 Lw of Totl Probbility: For evet spce{b, B,, B } with P[B i ] > 0 for ll i, P[A] = P[A B i ]P[B i ] T Byes Theore: P[B A] = P[AB] P[A] = P[A B]P[B] P[A] D7 Two idepedet Evets: Evets A d B re idepedet if d oly if P[AB] = P[A]P[B] Whe evets A d B hve ozero probbilities, the followig foruls re equivlet to the defiitio of idepedet evets: P[A B] = P[A], P[B A] = P[B] Idepedet d disjoit re ot syoys D8 3 Idepedet Evets: A, A, d A3 re idepedet if d oly if () A d A re idepedet, (b) A d A3 re idepedet, (c) A d A3 re idepedet, (d) P[A A A 3 ] = P[A ]P[A ]P[A 3 ] D9 More th Two Idepedet Evets: If 3, the sets A, A,, A re idepedet if d oly if () every set of sets fro A, A,, A re idepedet, (b) P[A A A ] = P[A ]P[A ] P[A ] D0 Fudetl Priciple of Coutig: If subexperiet A hs possible outcoes, d subexperiet B hs k possible outcoes, the there re k possible outcoes whe you perfor both experiets T The uber of k-peruttios of distiguishble objects is () k = ( )( ) ( k + ) =! Eg splig without replceet ( k)! T3 The uber of wys to choose k objects out of distiguishble objects is k = () k =! k! k = 0,,,, D choose k: For iteger 0, we defie k =! k!( k)! 0 otherwise T4 Give distiguishble objects, there re wys to choose with replceet ordered sple of objects Eg splig with replceet T5 For repetitios of subexperiet with sple spce S = {s 0,, s }, there re possible observtio sequeces T6 The uber of observtio sequeces for subexperiets with sple spce S = {0, } with 0 pperig 0 ties d pperig = 0 ties is T7 For repetitios of subexperiet with sple spce S = {s 0,, s }, the uber of legth = observtio sequeces with si pperig i ties is = 0,,! 0!!!! i= k!( k)! = ; i {0,,,},i=0,,,, D Multioil Coefficiet: For iteger 0, we defie = 0,, 0!!! 0 otherwise T8 The probbility of 0 filures d successes i = 0 + idepedet trils is P S 0, = ( p) p = ( p) 0p 0 0 T9 A subexperiet hs sple spce S = {s 0,, s } with P[s i ] = p i For = idepedet trils, the probbility if i occurreces of si, i = 0,,,, is P S 0,, = p 0 0,, 0 p T0 Relibility Probles: () Copoets i series: The probbility tht the opertio succeeds is P[W] = P[W W W ] = p p p = p (b) Copoets i prllel: The probbility tht the prllel opertio succeeds is P[W] = P[W ] = ( p) D Rdo Vrible: A rdo vrible cosists of experiet with probbility esure P[] defied o sple spce S d fuctio tht ssigs rel uber to ech outcoe i the sple spce of the experiet D Discrete Rdo Vrible: X is discrete rdo vrible if the rge of X is coutble set S X = {x, x, } D3 Fiite Rdo Vrible: X is fiite rdo vrible if the rge is fiite set {x, x,, x } D4 Probbility Mss Fuctio (PMF): The probbility ss fuctio (PMF) of the discrete rdo vrible X is P X (x) = P[X = x] T For discrete rdo vrible X with PMF P X (x) d rge SX: () For y x, P X (x) 0 (b) xεs X P X (x) = (c) For y evet B S X, the probbility tht X is i the set B is P[B] = x B P[X = x] = x B P X (x) p x = 0, D5 Beroulli (p) Rdo Vrible: X is Beroulli (p) rdo vrible if the PMF of X hs the for P X (x) = p x =, where the preter p is i the rge 0 < p < Eg coi flip D6 Geoetric (p) Rdo Vrible: X is geoetric (p) rdo vrible if the PMF of X hs the for P X (x) = p( p)x x =,,, where the preter p is i the rge 0 < p < Eg first success D7 Bioil (, p) Rdo Vrible: X is bioil (, p) rdo vrible if the PMF of X hs the for P X (x) = x px ( p) x, where 0 < p < d is iteger such tht Eg uber of heds fro coi flips D8 Pscl (k, p) Rdo Vrible: X is Pscl (k, p) rdo vrible if the PMF of X hs the for P X (x) = x k pk ( p) x k, where 0 < p < d k is iteger such tht k Eg cotiue util 3 filures: P Z (z) = z p3 ( p) z 3 z = 3, 4, 5, x = k, k +, k +,, l, D9 Discrete Uifor (k, l) Rdo Vrible: X is discrete uifor (k, l) rdo vrible if the PMF of X hs the for P X (x) = l k+ where the preters k d l re itegers such tht k < l x = 0,,,, D0 Poisso (α) Rdo Vrible: X is Poisso (α) rdo vrible if the PMF of X hs the for P X (x) = x! where the preter α is i the rge α > 0 A Poisso odel ofte specifies verge rte, λ rrivls per secod d tie itervl, T secods I this tie itervl, the uber of rrivls X hs Poisso PMF with α = λt Eg if λ = queries 0 d T = 0 secods, the α = = queries 5 secod 5 3 D Cuultive Distributio Fuctio (CDF): The cuultive distributio fuctio (CDF) of rdo vrible X is F X (x) = P[X x] α x e α

2 T For y discrete rdo vrible X with rge S X = {x, x, } stisfyig x x, () Goig fro left to right o the x-xis, F X (x) strts t zero d eds t oe (b) The CDF ever decreses s it goes fro left to right (c) For discrete rdo Vrible X, there is jup (discotiuity) t ech vlue of x S X The height of the jup t xi is P X (x i ) (d) Betwee jups, the grph of the CDF of the discrete rdo vrible X is horizotl lie T3 For ll b, F X (b) F X () = P[ < X b] D Mode: A ode of rdo vrible X is uber xod stisfyig P X (x od ) P X (x) for ll x D3 Medi: A edi, xed, of rdo vrible X is uber tht stisfies P[X < x ed ] = P[X > x ed ] D4 Expected Vlue: The expected vlue of X is E[X] = μ X = x S X xp X (x) T4 The Beroulli (p) rdo vrible X hs expected vlue E[X] = p T5 The geoetric (p) rdo vrible X hs expected vlue E[X] = p T6 The Poisso (α) rdo vrible i Defiitio 0 hs expected vlue E[X] = α T7 () For the bioil (, p) rdo vrible X of Defiitio 7, E[X] = p (b) For the Pscl (k, p) rdo vrible X of Defiitio 8, E[X] = k (c) For the p discrete uifor (k, l) rdo vrible X of Defiitio 9, E[X] = k+l T8 Perfor Beroulli trils I ech tril, let the probbility of success be α/, where α > 0 is costt d > α Let the rdo vrible K be the uber of successes i the trils As, P K (k) coverges to the PMF of Poisso (α) rdo vrible D5 Derived Rdo Vrible: Ech sple vlue y of derived rdo vrible Y is theticl fuctio g(x) of sple vlue x of other rdo vrible X We dopt the ottio Y = g(x) to describe the reltioship of the two rdo vribles T9 For discrete rdo vrible X, the PMF of Y = g(x) is P Y (y) = x:g(x)=y P X (x) T0 Give rdo vrible X with PMF P X (x) d the derived rdo vrible Y = g(x), the expected vlue of Y is E[Y] = μ Y = x S X g(x)p X (x) T For y rdo vrible X, E[X μ X ] = 0 T For y rdo vrible X, E[X + b] = E[X] + b D6 Vrice: The vrice of rdo vrible X is Vr[X] = E[(X μ X ) ] = E[X ] (E[X]) D7 Stdrd Devitio: The stdrd devitio of rdo vrible X is X = Vr[X] T3 Vr[X] = E[X ] μ X = E[X ] (E[X]) D8 Moets: For rdo vrible X: () The th oet is E[X ] (b) The th cetrl oet is E[(X μ X ) ] T4 Vr[X + b] = Vr[X] T5 () If X is Beroulli (p), the Vr[X] = p( p), d E[X ] = p (b) If X is geoetric, the Vr[X] = p, d p E[X ] = p (c) If X is bioil (, p), the p Vr[X] = p( p), d E[X ] = p + p( p) (d) If X is Pscl (k, p), the Vr[X] = k( p), d E[X ] = k +k( p) (e) If X is Poisso (α), the Vr[X] = α, d p p E[X ] = α α (f) If X is discrete uifor (k, l), the Vr[X] = (l k)(l k+), d E[X ] = l +l(k+)+k(k ) 6 D9 Coditiol PMF: Give the evet B, with P[B] > 0, the coditiol probbility ss fuctio of X is P X B (x) = P[X = x B] T6 A rdo vrible X resultig fro experiet with evet spce B,, B hs PMF P X (x) = P X Bi (x)p[b i ] x C P X (x) x B, T7 P X B (x) = P[B] 0 otherwise T8 () For y x B, P X B (x) 0 (b) xεb P X B (x) = (c) For y evet C B, P[C B], the coditiol probbility tht X is i the set C, is P[C B] = P X B (x) D0 Coditiol Expected Vlue: The coditiol expected vlue of rdo vrible X give coditio B is E[X B] = μ X B = x B xp X B (x) T9 For rdo vrible X resultig fro experiet with evet spce B,, B, E[X B] = i= E[X B i ]P[B i ] T0 The coditiol expected vlue of Y = g(x) give coditio B is E[Y B] = E[g(X) B] = x B g(x)p X B (x) D3 Cuultive Distributio Fuctio (CDF): The cuultive distributio fuctio (CDF) of rdo vrible X is F X (x) = P[X x] T3 For y rdo vrible X, () F X ( ) = 0, (b) F X () =, (c) P[x < X x ] = F X (x ) F X (x ) D3 Cotiuous Rdo Vrible: X is cotiuous rdo vrible if the CDF F X (x) is cotiuous fuctio x T3 For cotiuous rdo vrible X with PDF f X (x), () f X (x) 0 for ll x, (b) F X (x) = f X (u)du, (c) f X (x)dx =, (d) f X (x) = df X(x) T33 P[x < X x ] = f X (x)dx x x D34 Expected Vlue: The expected vlue of cotiuous rdo vrible X is E[X] = xf X (x)dx T34 The expected vlue of fuctio, g(x), of rdo vrible X is E[g(X)] = g(x)f X (x)dx T35 For y rdo vrible X, () E[X μ X ] = 0, (b) E[X + b] = E[X] + b, (c) Vr[X] = E[X ] μ X, (d) Vr[X + b] = Vr[X] x < b, D35 Uifor Rdo Vrible: X is uifor (, b) rdo vrible if the PDF of X is f X (x) = b where the two preters re > b 0 x, x T36 If X is uifor (, b) rdo vrible, () The CDF of X is F X (x) = < x b, (b) The expected vlue of X is E[X] = b+ (c) The vrice of X is b x > b Vr[X] = (b ), (d) E[X ] = ( b) 3 T37 Let X be uifor (, b) rdo vrible, where d b re both itegers Let K = X The K is discrete uifor ( +, b) rdo vrible D36 Expoetil Rdo Vrible: X is expoetil (λ) rdo vrible if the PDF of X is f X (x) = λe λx x 0, where the preter λ > 0 T38 If X is expoetil (λ) rdo vrible, () F X (x) = e λx x 0, (b) E[X] =, (c) Vr[X] = λ λ, (d) E[X ] = λ T39 If X is expoetil (λ) rdo vrible, the K = X is geoetric (p) rdo vrible with p = e λ D37 Erlg Rdo Vrible: X is Erlg (, λ) rdo vrible if the PDF of X is f X (x) = λ x e λx x 0, where the preter λ > 0, d the preter i= is iteger T30 If X is Erlg (, λ) rdo vrible, the E[X] =, Vr[X] = λ λ, d E[X ] = (+) λ T3 Let K α deote Poisso (α) rdo vrible For y x > 0, the CDF of Erlg (, λ) rdo vrible X stisfies F X (x) = F Kλx ( ) = (λx) k e λx k=0 k! D38 Gussi Rdo Vrible: X is N[μ, ] or Gussi (μ, ) rdo vrible if the PDF of X is f X (x) = e (x μ) π uber d the preter > 0 T3 If X is N[μ, ] or Gussi (μ, ) rdo vrible, the E[X] = μ, Vr[X] = T33 If X is N[μ, ] or Gussi (μ, ) rdo vrible, Y = X + b is Gussi (μ + b, ) D39 Stdrd Norl Rdo Vrible: The stdrd orl rdo vrible Z is the Gussi (0, ) rdo vrible D30 Stdrd Norl CDF: The CDF of the stdrd orl rdo vrible Z is Φ(z) = z e u π du dx, where the preter μ c be y rel T34 If X is Gussi (μ, ) rdo vrible, the CDF of X is F X (x) = Φ( x μ ) The probbility tht X is i the itervl (, b] is [P[ < x b] = Φ b μ Φ( μ )]

3 T35 Φ( z) = Φ(z) D3 Stdrd Norl Copleetry CDF: The stdrd orl copleetry CDF is Q(z) = P[Z > z] = e u π z du = Φ(z) D3 Uit Ipulse (Delt) Fuctio: Let d ε (x) = ε ε/ x ε/, 0 otherwise The uit ipulse fuctio is δ(x) = li ε 0 d ε (x) T36 Siftig Property: For y cotiuous fuctios g(x), g(x)δ(x x 0 )dx = g(x 0 ) 0 x < 0, D33 Uit Step Fuctio: The uit step fuctio is u(x) = x 0 x T37 δ(v)dv = u(x) T38 For rdo vrible X, we hve the followig equivlet stteets: () P[X = x 0 ] = q, (b) P X (x 0 ) = q, (c) F X (x + 0 ) F X (x 0 ) = q, (d) f X (x 0 ) = qδ(0) D34 Mixed Rdo Vrible: X is ixed rdo vrible if d oly if f X (x) cotis both ipulse d ozero, fiite vlues T39 If Y = X, where > 0, the Y hs CDF F Y (y) = F X ( y ), d PDF f Y (y) = f X( y ) T30 Y = X, where > 0 () If X is uifor (b,c), the Y is uifor (b, c) (b) If X is expoetil (λ), the Y is expoetil ( λ ) (c) If X is Erlg (, λ), the Y is Erlg (, λ ) (d) If X is Gussi (μ, ), the Y is Gussi (μ, ) T3 If Y = X + b, the F Y (y) = F X (y b), f Y (y) = f X (y b) T3 Let U be uifor (0,) rdo vrible d let F(x) deote cuultive distributio fuctio with iverse F (u) defied for 0 < u < The rdo vrible X = F (U) hs CDF F X (x) = F(x) D35 Coditiol PDF give Evet: For rdo vrible X with PDF f X (x) d evet B S X with P[B] > 0, the coditiol PDF of X give B is f X B (x) = f X (x) x B, P[B] 0 otherwise T33 Give evet spce {B i } d the coditiol PDFs f X Bi (x), f X (x) = i f X Bi (x)p[b i ] D36 Coditiol Expected Vlue Give Evet: If {x B}, the coditiol expected vlue of X is E[X B] = xf X B (x) dx D4 Joit Cuultive Distributio Fuctio (CDF): The joit cuultive distributio fuctio of rdo vribles X d Y is F X,Y (x, y) = P[X x, Y y] T4 For y pir of rdo vribles, X, Y, () 0 F X,Y (x, y), (b) F X (x) = F X,Y (x, ), (c) F Y (y) = F X,Y (, y), (d) F X,Y (, y) = F X,Y (x, ) = 0, (e) If x x d y y, the F X,Y (x, y) F X,Y (x, y ), (f) F X,Y (, ) = D4 Joit Probbility Mss Fuctio (PMF): The joit probbility ss fuctio of discrete rdo vribles X d Y is P X,Y (x, y) = P[X = x, Y = y] T4 For discrete rdo vribles X d Y d y set B i the X, Y ple, the probbility of the evet {(X, Y)} B} is P[B] = (x,y) B P X,Y (x, y) T43 For discrete rdo vribles X d Y with joit PMF P X,Y (x, y), P X (x) = P X,Y (x, y), P Y (y) = x S X P X,Y (x, y) D43 Joit Probbility Desity Fuctio (PDF): The joit PDF of the cotiuous rdo vribles X d Y is fuctio f X,Y (x, y)with the property F X,Y (x, y) = x y f X,Y (u, v)dvdu T44 f X,Y (x, y) = F X,Y (x,y) x y T45 P[x X x, y < Y y ] = F X,Y (x, y ) F X,Y (x, y ) F X,Y (x, y ) F X,Y (x, y ) T46 A joit PDF f X,Y (x, y) hs the followig properties correspodig to first d secod xios of probbility: () f X,Y (x, y) 0 for ll (x, y), (b) f X,Y (x, y)dxdy = T47 The probbility tht the cotiuous rdo vribles (X, Y) re i A is P[A] = f X,Y (x, y)dxdy A T48 If X d Y re rdo vribles with joit PDF f X,Y (x, y), f X (x) = f X,Y (x, y)dy, f Y (y) = f X,Y (x, y)dx T49 For discrete rdo vribles X d Y, the derived rdo vrible W = g(x, Y) hs PMF P W (w) = (x,y):g(x,y)=w P X,Y (x, y) T40 For cotiuous rdo vribles X d Y, the CDF of W = g(x, Y) is F W (w) = P[W w] = f X,Y (x, y)dxdy g(x,y) w w w T4 For cotiuous rdo vribles X d Y, the CDF of W = x(x, Y) is F W (w) = F X,Y (w, w) = f X,Y (x, y)dxdy T4 For rdo vribles X d Y, the expected vlue of W = g(x, Y) is () Discrete: E[W] = x S X g(x, y)p X,Y (x, y), (b) Cotiuous: E[W] = g(x, y)f X,Y (x, y)dxdy T43 E[g (X, Y) + + g (X, Y)] = E[g (X, Y)] + + E[g (X, Y)] T44 For y two rdo vribles X d Y, E[X + Y] = E[X] + E[Y] T45 The vrice of the su of two rdo vribles is Vr[X + Y] = Vr[X] + Vr[Y] + E[(X μ X )(Y μ Y )] D44 Covrice: The covrice of two rdo vribles X d Y is Cov[X, Y] = E[(X μ X )(Y μ Y )] D45 Correltio: The correltio of X d Y is r X,Y = E[XY] = xyf X,Y (x, y)dxdy T46 () Cov[X, Y] = r X,Y μ X μ Y (b) Vr[X + Y] = Vr[X] + Vr[Y] + Cov[X, Y] (c) If X = Y, Cov[X, Y] = Vr[X] = Vr[Y] d r X,Y = E[X ] = E[Y ] D46 Orthogol Rdo Vribles: Rdo vribles X d Y re orthogol if r X,Y = 0 D47 Ucorrelted Rdo Vribles: Rdo vribles X d Y re ucorrelted if Cov[X, Y] = 0 Orthogol es zero correltio; ucorrelted es zero covrice Note: Idepedece iplies zero covrice, but NOT vice vers D48 Correltio Coefficiet: The correltio coefficiet of two rdo vribles X d Y is ρ X,Y = Cov[X,Y] = Cov[X,Y] Vr[X]Vr[Y] X Y T47 ρ X,Y < 0, T48 If X d Y re rdo vribles such tht Y = X + b, ρ X,Y = 0 = 0, > 0 D49 Coditiol Joit PMF: For discrete rdo vribles X d Y d evet, B with P[B] > 0, the coditiol joit PMF of X d Y give B is P X,Y B (x, y) = P[X = x, Y = y B] P X,Y (x,y) (x, y) B, T49 For y evet B, regio of the X, Y ple with P[B] > 0, P X,Y B (x, y) = P[B] 0 otherwise D40 Coditiol Joit PDF: Give evet B with P[B] > 0, the coditiol joit probbility desity fuctio of X d Y is f X,Y B (x, y) = f X,Y (x,y) (x, y) B, P[B] 0 otherwise T40 Coditiol Expected Vlue: For rdo vribles X d Y d evet B of ozero probbility, the coditiol expected vlue of W = g(x, Y) give B is () Discrete: E[W B] = x S X g(x, y)p X,Y B (x, y), (b) Cotiuous: E[W B] = g(x, y)f X,Y B (x, y)dxdy D4 Coditiol vrice: The coditiol vrice of the rdo vrible W = g(x, Y)is Vr[W B] = E[ W μ W B B] Aother ottio for coditiol vrice is W B T4 Vr[W B] = E[W B] (μ W B ) D4 Coditiol PMF: For y evet Y = y such tht P Y (y) > 0, the coditiol PMF of X give Y = y is P X Y (x y) = P[X = x Y = y] T4 For rdo vribles X d Y with joit PMF P X,Y (x, y), d x d y such tht P X (x) > 0 d P Y (y) > 0, P X,Y (x, y) = P X Y (x y)p Y (y) = P Y X (y x)p X (x)

4 T43 Coditiol Expected Vlue of Fuctio: X d Y re discrete rdo vribles For y y S Y, the coditiol expected vlue of g(x, Y) give Y = y is E[g(X, Y) Y = y] = x S X g(x, y)p X Y (x y) D43 Coditiol PDF: For y such tht f Y (y) > 0, the coditiol PDF of X give {Y = y} is f X Y (x y) = f X,Y(x,y), which iplies f Y X (y x) = f X,Y(x,y) T44 f X,Y (x, y) = f Y X (y x)f X (x) = f X Y (x y)f Y (y) F Y (y) = F Y (y x)f X (x)dx x S X D44 Coditiol Expected Vlue of Fuctio: For cotiuous rdo vribles X d Y d y y such tht f Y (y) > 0, the coditiol expected vlue of g(x, Y) give Y = y is E[g(X, Y) Y = y] = g(x, y)f X Y (x y)dx D45 Coditiol Expected Vlue: The coditiol expected vlue E[X Y] is fuctio of rdo vrible Y such tht is Y = y the E[X Y] = E[X Y = y] T45 Iterted Expecttio: E E[X Y] = E[X] T46 E E[g(X) Y] = E[g(X)] D46 Idepedet Rdo Vribles: Rdo vribles X d Y re idepedet if d oly if () Discrete: P X,Y (x, y) = P X (x)p Y (y), (b) Cotiuous: f X,Y (x, y) = f X (x)f Y (y) T47 For idepedet vribles X d Y, () E[g(X)h(Y)] = E[g(X)]E[h(Y)], (b) r X,Y = E[XY] = E[X]E[Y], (c) Cov[X, Y] = ρ X,Y = 0, (d) Vr[X + Y] = Vr[X] + Vr[Y], (e) E[X Y = y] = E[X] for ll y S Y, (f) E[Y X = x] = E[Y] for ll x S X D47 Bivrite Gussi Rdo Vribles: Rdo vribles X d Y hve bivrite Gussi PDF with preters x μ exp ρ(x μ )(y μ ) + y μ ( ρ ) μ,, μ,, d ρ if f X,Y (x, y) =, where μ π ρ d μ c be y rel ubers, > 0, > 0, d < p < T48 If X d Y re bivrite Gussi rdo vribles i Defiitio 47, X is the Gussi (μ, ) rdo vrible d Y is the Gussi (μ, ) rdo vrible: () f X (x) = π (x μ) e, (b) f Y (y) = π (y μ) e T49 If X d Y re bivrite Gussi rdo vribles i Defiitio 47, the coditiol PDF of Y give X is f Y X (y x) = coditiol expected vlue of Y is μ (x) = μ + ρ (x μ ), d the vrice of Y is = ( ρ ) T430 If X d Y re bivrite Gussi rdo vribles i Defiitio 47, the coditiol PDF of X give Y is f X Y (x y) = f Y (y) (y μ (x)) e π (x μ (y)) e π coditiol expected vlue of X is μ (y) = μ + ρ (y μ ), d the vrice of X is = ( ρ ) T43 Bivrite Gussi rdo vribles X d Y i Defiitio 47 hve correltio coefficiet ρ X,Y = ρ T43 Bivrite Gussi rdo vribles X d Y re ucorrelted if d oly if they re idepedet D5 Multivrite Joit CDF: The joit CDF of X,, X is F X,,X (x,, x ) = P[X x,, X x ] D5 Multivrite Joit PMF: The joit PMF of the discrete rdo vribles X,, X is P X,,X (x,, x ) = P[X = x,, X = x ] D53 Multivrite Joit PDF: The joit PDF of the cotiuous rdo vribles X,, X is the fuctio f X,,X (x,, x ) = F X,,X (x,,x ) x x T5 If X,, X re discrete rdo vribles with joit PMF P X,,X (x,, x ), () P X,,X (x,, x ) 0, (b) x S X T5 If X,, X re cotiuous rdo vribles with joit PDF f X,,X (x,, x ), () f X,,X (x,, x ) 0, (b) x x f X (x), where, give X = x, the, where, give Y = y, the P X,,X (x,, x ) = x S X F X,,X (x,, x ) = f X,,X (u,, u )du du, (c) f X,,X (x,, x )dx dx = T53 The probbility of evet A expressed i ters of the rdo vribles X,, X is () Discrete:P[A] = (x,,x ) A P X,,X (x,, x ), (b) Cotiuous: P[A] = f X,,X (x,, x )dx dx dx A D54 Rdo Vector: A rdo vector is colu vector X = [X X ] Ech X i is rdo vrible D55 Vector Sple Vlue: A sple vlue of rdo is colu vector x = [x x ] The ith copoet, x i, of the vector x is sple vlue of rdo vrible, X i D56 Rdo Vector Probbility Fuctios: () The CDF of rdo vector X is F X (x) = F X,,X (x,, x ) (b) The PMF of discrete rdo vector X is P X (x) = P X,,X (x,, x ) (c) The PDF of cotiuous rdo vector X is f X (x) = f X,,X (x,, x ) D57 Probbility Fuctios of Pir of Rdo Vectors: For rdo vectors X with copoets d Y with copoets: () The joit CDF of X d Y is F X,Y (x, y) = F X,,X,Y,,Y (x,, x, y,, y ); (b) The joit PMF of discrete rdo vectors X d Y is P X,Y (x, y) = P X,,X,Y,,Y (x,, x, y,, y ); (c) The joit PDF of cotiuous rdo vectors X d Y is f X,Y (x, y) = f X,,X,Y,,Y (x,, x, y,, y ) T54 For joit PMF P W,X,Y,Z (w, x, y, z) of discrete rdo vribles W, X, Y, Z, soe rgil PMFs re: () P X,Y,Z (x, y, z) = w S W P W,X,Y,Z (w, x, y, z), (b) P W,Z (w, z) = x S X P W,X,Y,Z (w, x, y, z), (c) P X (x) = w S W z SZ P W,X,Y,Z (w, x, y, z) T55 For joit PDF f W,X,Y,Z (w, x, y, z) of cotiuous rdo vribles W, X, Y, Z, soe rgil PDFs re: () f X,Y,Z (x, y, z) = f W,X,Y,Z (w, x, y, z)dw, (b) f W,Z (w, z) = f W,X,Y,Z (w, x, y, z)dxdy, (c) f X (x) = f W,X,Y,Z (w, x, y, z)dwdydz D58 N Idepedet Rdo Vribles: Rdo Vribles X,, X re idepedet if for ll x,, x, () Discrete: P X,,X (x,, x ) = P X (x )P X (x ) P XN (x ), (b) Cotiuous: f X,,X (x,, x ) = f X (x )f X (x ) f XN (x ) D59 Idepedet d Ideticlly Distributed (iid): Rdo vribles X,, X re idepedet d ideticlly distributed (iid) if () Discrete: P X,,X (x,, x ) = P X (x )P X (x ) P XN (x ), (b) Cotiuous: f X,,X (x,, x ) = f X (x )f X (x ) f XN (x ) D50 Idepedet Rdo Vectors: Rdo Vectors X d Y re idepedet if () Discrete: P X,Y (x, y) = P X (x)p Y (y), (b) Cotiuous: f X,Y (x, y) = f X (x)f Y (y) T56 For rdo vrible W g(x), () Discrete: P W (w) = P[W = w] = x:g(x)=w P X (x), (b) Cotiuous: F W (w) = P[W = w] = f X (x)dx dx g(x) w T57 Let X be vector of iid rdo vribles ech with CDF F X (x) d PDF f X (x) () The CDF d the PDF of Y = x{x,, X } re F Y (y) = (F X (y)), f Y (y) = F X (y) f X (y) (b) The CDF d the PDF of W = i{x,, X } re F W (w) = ( F X (w)), f W (w) = ( F X (w)) f X (w) T58 For rdo vector X, the rdo vribles g(x) hs expected vlue () Discrete: E[g(X)] = g(x)p X (x), (b) Cotiuous: E[g(X)] = g(x)f X (x)dx dx T59 Whe the copoets of X re idepedet rdo vribles, E[g (X )g (X ) g (X )] = E[g (X )]E[g (X )] E[g (X )] T50 Give the cotiuous rdo vector X, defie the derived rdo vector Y such tht Y k = X k + b for costts > 0 d b The CDF d PDF of Y re F Y (y) = F X ( y b,, y b ), f Y(y) = f X( y b,, y b ) T5 If X is cotiuous rdo vector d A is ivertible trix, the Y = AX + b hs PDF f Y (y) = f det(a) X(A (y b)) D5 Expected Vlue Vector: The expected vlue of rdo vector X is colu vector E[X] = μ X = E[X ] E[X ] E[X ] D5 Expected Vlue of Rdo Mtrix: For rdo trix A with the rdo vrible A ij s its i, jth eleet, E[A] is trix with i, jth eleet E[A ij ] D53 Vector Correltio: The correltio of rdo vector X is trix R X with i, jth eleet R X (i, j) = E[X i X j ] I vector ottio, R X = E[XX ] D54 Vector Covrice: The covrice of rdo vector X is trix C X with copoets C X (i, j) = Cov[X i, X j ] I vector ottio, C X = E[(X μ X )(X μ X ) ] T5 For rdo vector X with correltio trix R X, covrice trix C X, d vector expected vlue μ X, C X = R X μ X μ X D55 Vector Cross-Correltio: The cross-correltio of rdo vectors, X with copoets d Y with copoets, is trix R XY with i, jth eleet R XY (i, j) = E[X i Y j ], or, i vector ottio, R XY = E[XY ] x S X x S X

5 D56 Vector Cross-Covrice: The cross-covrice of pir of rdo vectors X with copoets d Y with copoets is trix C XY with i, jth eleet C XY (i, j) = Cov X i, Y j, or, i vector ottio, C XY = E[(X μ X )(Y μ Y ) ] T53 X is -diesiol rdo vector with expected vlue μ X, correltio R X, d covrice C X The -diesiol rdo vector Y = AX + b, where A is trix d b is -diesiol vector, hs expected vlue μ Y, correltio trix R Y, d covrice trix C Y give by: () μ Y = Aμ X + b, (b) R Y = AR X A + (Aμ X )b + b(aμ X ) + bb, (c) C Y = AC X A T54 The vectors X d Y = AX + b hve cross-correltio R XY give by R XY = R X A + μ X b, d cross-covrice C XY give by C XY = C X A D57 Gussi Rdo Vector: X is Gussi (μ X, C X ) rdo vector with expected vlue μ X d covrice C X if d oly if f X (x) = (π) [det (C X )] exp (x μ X ) C X (x μ X ), where det(c X ), the deterit of C X, stisfies det(c X ) > 0 T55 A Gussi rdo vector X hs idepedet copoets if d oly if C X is digol trix T56 Give -diesiol Gussi rdo vector X with expected vlue μ X d covrice C X, d trix A with rk(a) =, Y = AX + b is -diesiol Gussi rdo vector with expected vlue μ Y = Aμ X + b d covrice C Y = AC X A D58 Stdrd Norl Rdo Vector: The -diesiol stdrd orl rdo vector Z is the -diesiol Gussi rdo vector with E[Z] = 0 d C Z = I T57 For Gussi (μ X, C X ) d rdo vector, let A be trix with the property AA = C X The rdo vector Z = A (X μ X ) is stdrd orl rdo vector T58 Give the -diesiol stdrd orl rdo vector Z, ivertible trix A, d -diesiol vector b, X = AZ + b is -diesiol Gussi rdo vector with expected vlue μ X = b d covrice trix C X = AA T59 For Gussi vector X with covrice C X, there lwys exists trix A such tht C X = AA T6 For y set of rdo vribles X,, X, the expected vlue of W N = X + + X is E[W ] = E[X ] + E[X ] + + E[X ] T6 The vrice of W = X + + X is Vr[W ] = i= Vr[X i ] + i= j=i+ Cov X i, X j T63 Whe X,, X re ucorrelted, Vr[W ] = Vr[X ] + + Vr[X ] T64 The PDF of W = X + Y is f W (w) = f X,Y (x, w x)dx = f X,Y (w y, y)dy Note: other wy to fid f W is to plot f X,Y, fid F W fro double itegrls for ech cse, d tke the derivtive of F W T65 Whe X d Y re idepedet rdo vribles, the PDF of W = X + Y is f W (w) = f X (w y)f Y (y)dy = f X (x)f Y (w x)dx D6 Moet Geertig Fuctio (MGF): For rdo vrible X, the oet geertig fuctio (MGF) of X is φ X (s) = E[e sx ] Whe X is cotiuous rdo vrible, φ X (s) = e sx f X (x)dx For discrete rdo vrible Y, the MGF is φ Y (s) = y e sy i S Y ip Y (y i ) T66 A rdo vrible X with MGF φ X (s) hs th oet E[X ] = d φ X (s) ds s=0 T67 The MGF of Y = X + b is φ Y (s) = e sb φ X (s) T68 For set of idepedet rdo vribles X,, X, the oet geertig fuctio of W = X + + X is φ W (s) = φ X (s)φ X (s) φ X (s) Whe X,, X re iid, ech with MGF φ W (s) = [φ X (s)] T69 If K,, K re idepedet Poisso rdo vribles, W = K + + K is Poisso rdo vrible T60 The su of idepedet Gussi rdo vribles W = X + + X is Gussi rdo vrible λ w e λw T6 If X,, X re iid expoetil (λ) rdo vribles, the W = X + + X hs the Erlg PDF f W (w) = w 0, ( )! 0 otherwise T6 Let {X, X, } be collectio of iid rdo vribles, ech with MGF φ X (s), d let N be oegtive iteger-vlued rdo vrible tht is idepedet of {X, X, } The rdo su R = X + + X N hs oet geertig fuctio φ R (s) = φ N (l φ X (s)) T63 For the rdo su if iid rdo vribles R = X + + X N, () E[R] = E[N]E[X], (b)vr[r] = E[N]Vr[X] + Vr[N](E[X]) T64 Cetrl Liit Theore: Give X, X,, sequece of iid rdo vribles with expected vlue μ X d vrice X, the CDF of Z = i= X i μ X hs the property li F Z (z) = φ(z) X D6 Cetrl Liit Theore Approxitio: Let W = X + + X be the su of iid rdo vribles, ech with E[X] = μ X d Vr[X] = X The cetrl liit theore pproxitio to the CDF of W is F W (w) Φ w μ X Eg ppss = 08 Wht is the probbility of fidig 500 cceptble i btch of 600? P[w 500] = P[w < 500] = Φ (500) (600)(08) (600)(08)(0) X D63 De Moivre-Lplce Forul: For bioil (, p) rdo vrible K, P[k K k ] Φ k +05 p Φ p( p) k 05 p p( p) T94 Rdo vribles X d Y hve expected vlues μ X d μ Y, stdrd devitios X d Y, d correltio coefficiet ρ X,Y The optil lier e squre error (LMSE) estitor of X give Y is X L (Y) = Y + b d it hs the followig properties: () = Cov[X,Y] = ρ X Vr[Y] X,Y, b = μ X μ Y, (b) The iiu e squre Y estitio error for lier estite is e L = E (X X L (Y)) = X ρ X,Y, (c) The estitio error X X L (Y) is ucorrelted with Y T95 If X d Y re the bivrite Gussi rdo vribles i Defiitio 47, the optiu estitor of X give Y is the optiu lier estitor i Theore 94 D0 Stochstic Process: A stochstic process X(t) cosists of experiet with probbility esure P[ ] defied o sple spce S d fuctio tht ssigs tie fuctio x(t, s) to ech outcoe s i the sple spce of the experiet D0 Sple Fuctio: A sple fuctio x(t, s) is the tie fuctio ssocited with outcoes s of experiet D03 Eseble: The eseble of stochstic process is the set of ll possible tie fuctios tht c result fro the experiet D04 Discrete-Vlue d Cotiuous-Vlue Processes: X(t) is discrete-vlue process if the set of ll possible vlues of X(t) t ll ties t is coutble set S X ; otherwise X(t) is cotiuous-vlue process D05 Discrete-Tie d Cotiuous-Tie Processes: The stochstic process X(t) is discrete-tie process if X(t) is defied oly for set of tie istts, t = T, where T is costt d is iteger; otherwise X(t) is cotiuous-tie process D06 Rdo Sequece: A rdo sequece X is ordered sequece of rdo vribles X 0, X, T0 Let X deote iid rdo sequece For discrete-vlue process, the sple vector X = [X X k ] hs joit PMF P X (x) = P X (x )P X (x ) P X (x k ) = k i= P X (x i ) For cotiuous-vlue process, the joit PDF of X = [X X k ] is f X (x) = f X (x )f X (x ) f X (x k ) k = i= f X (x i ) D07 Beroulli Process: A Beroulli (p) process X is iid rdo sequece i which ech X is Beroulli (p) rdo vrible D0 The Expected Vlue of Process: The expected vlue of stochstic process X(t) is the deteriistic fuctio μ X (t) = E[X(t)] D0 Autocovrice: The utocovrice fuctio of the stochstic process X(t) is C X (t, τ) = Cov[X(t), X(t + τ)] The utocovrice fuctio of the rdo sequece X is C X [, k] = Cov[X, X +k ] D03 Autocorreltio Fuctio: The utocorreltio fuctio of the stochstic process X(t) is R X (t, τ) = E[X(t)X(t + τ)] The utocorreltio fuctio of the rdo sequece X is R X [, k] = E[X X +k ] T09 The utocorreltio d utocovrice fuctios of process X(t) stisfy C X (t, τ) = R X (t, τ) μ X (t)μ X (t + τ) The utocorreltio d utocovrice fuctios of rdo sequece X stisfy C X [, k] = R X [, k] μ X μ X ( + k) D04 Sttiory Process: A stochstic process X(t) is sttiory if d oly if for ll sets of tie istts t,, t, d y tie differece τ, f X(t ),,X(t )(x,, x ) = f X(t +τ),,x(t +τ)(x,, x ) A rdo sequece X is sttiory if d oly if for y set of iteger tie istts,,, d iteger tie differece k, f X,,X (x,, x ) = f X +k,,x +k (x,, x )

6 T00 Let X(t) be sttiory rdo process For costts > 0 d b, Y(t) = X(t) + b is lso sttiory process T0 For sttiory process X(t), the expected vlue, the utocorreltio, d the utocovrice hve the followig properties for ll t: () μ X (t) = μ X, (b) R X (t, τ) = R X (0, τ) = R X (τ), (c)c X (t, τ) = R X (τ) μ X = C X (τ) For sttiory rdo sequece X the expected vlue, the utocorreltio, d the utocovrice stisfy for ll : ()E[X ] = μ X, (b)r X [, k] = R X [, k] = R X [0, k] = R X [k], (c)c X [, k] = R X [k] μ X = C X [k] D05 Wide Sese Sttiory: X(t) is wide sese sttiory stochstic process if d oly if for ll t, E[X(t)] = μ X, d R X (t, τ) = R X (0, τ) = R X (τ) X is wide sese sttiory rdo sequece if d oly if for ll, E[X ] = μ X, d R X [, k] = R X [, k] = R X [k] Note: Use D03 to test if R X (t, τ) == R X (τ) Note: strictly sttiory iplies WSS, but NOT vice vers T0 For wide sese sttiory process X(t), the utocorreltio fuctio R X (τ) hs the followig properties: ()R X (0) 0, (b)r X (τ) = R X ( τ), (c) R X (0) R X (τ) If X is wide sese sttiory rdo sequece: ()R X [0] 0, (b)r X [k] = R X [ k], (c) R X [0] R X [k] D06 Averge Power: The verge power of wide sese sttiory process X(t) is R X (0) = E[X (t)] The verge power of wide sese sttiory sequece X is R X [0] = E[X ] T03 Let X(t) be sttiory rdo process with expected vlue μ X d utocovrice C X (τ) If C X (τ) dτ <, the X (T), X (T), is ubised, cosistet sequece of estites of μ X D07 Cross-Correltio Fuctio: The cross-correltio of cotiuous-tie rdo processes X(t) d Y(t) is R XY (t, τ) = E[X(t)Y(t + τ)] The cross-correltio of rdo sequeces X d Y is R XY [, k] = E[X Y +k ] D08 Joitly Wide Sese Sttiory Processes: Cotiuous-tie rdo processes X(t) d Y(t) re joitly wide sese sttiory if X(t) d Y(t) re both wide sese sttiory, d the cross-correltio depeds oly o the tie differece betwee the two rdo vribles: R XY (t, τ) = R XY (τ) Rdo sequeces X d Y re both wide sese sttiory d the cross-correltio depeds oly o the idex differece betwee the two rdo vribles: R XY [, k] = R XY [k] T04 If X(t) d Y(t) re joitly wide sese sttiory cotiuous-tie processes, the R XY (τ) = R YX ( τ) If X d Y re joitly wide sese sttiory rdo sequeces, the R XY [k] = R YX [ k] T E[Y(t)] = E h(u)x(t u)du = h(u)e[x(t u)]du T If the iput to LTI filter with ipulse respose h(t) is wide sese sttiory process X(t), the output Y(t) hs the followig properties: ()Y(t) is wide sese sttiory process with expected vlue μ Y = E[Y(t)] = μ X h(u)du, d utocorreltio fuctio R Y (τ) = E[Y(t)Y(t + τ)] = h(u) h(v)r X (τ + u v)dv du (b) X(t) d Y(t) re joitly wide sese sttiory d hve iput-output cross-correltio R XY (τ) = E[X(t)Y(t + τ)] = h(u)r X (τ u)du (c) The output utocorreltio is relted to the iput-output cross correltio by R Y (τ) = h( w)r XY (τ w)dw T3 If sttiory Gussi process X(t) is the iput to LTI filter h(t), the output Y(t) is sttiory Gussi process with expected vlue d utocorreltio give by T D Fourier Trsfor: Fuctios g(t) d G(f) re Fourier trsfor pir if G(f) = g(t)e jπft dt, g(t) = G(f)e jπft df D Power Spectrl Desity: The power spectrl desity fuctio of the wide sese sttiory stochstic process X(t) is T S X (f) = li T E[ X T T(f) ] = li T E T X(t)e jπft dt T T: Wieer-Khitchie: If X(t) is wide sese sttiory stochstic process, R X (τ) d S X (f) re the Fourier trsfor pir S X (f) = R X (τ)e jπfτ dτ, R X (τ) = S X (f)e jπfτ df T3 For wide sese sttiory rdo process X(t), the power spectrl desity S X (f) is rel-vlued fuctio with the followig properties: ()S X (f) 0 for ll f, (b) verge power = S X (f)df = E[X (t)] = R X (0), (c)s X ( f) = S X (f) D5 Cross Spectrl Desity: For joitly wide sese sttiory rdo processes X(t) d Y(t), the Fourier trsfor of the cross-correltio yields the cross spectrl desity S XY (f) = R XY (τ)e jπfτ dτ T6 Whe wide sese sttiory stochstic process X(t) is the iput to lier tie-ivrit filter with trsfer fuctio H(f), the power spectrl desity of the output Y(t) is S Y (f) = H(f) S X (f) Note: wys of fidig R Y (τ), either use T or use T6 to solve for S Y (f) the R Y (τ) = F {S Y (f)} T7 If the wide sese sttiory process X(t) is the iput to lier tie-ivrit filter with trsfer fuctio H(f), d Y(t) is the filter output, the iputoutput cross power spectrl desity fuctio d the output power spectrl desity fuctio re S XY (f) = H(f)S X (f), S Y (f) = H (f)s XY (f) Ie R X (τ) h(t) R X,Y (τ) h( t) R Y (τ) S X (f) H(f) S XY (f) H (f) S Y (f) Specil Topics Foruls A Lier MMSE: X = ρ XY (Y μ Y ) X + μ X A Qutiztio: Let X be cotiuous d uifor (, b) rdo vrible, let Y be the Y qutized output sigl, d let Z = X Y, the Mx Qutiztio Error: X Y x = Δ = b, "power" of the error sigl: L E[Z ] = Vr[Z] + (E[Z]) = (b ) + (E[Z]), SNR = E X = E[Z ] L, d SNR db = 0 log 0 E[X ] = 0 log E[Z ] 0(L), where Δ = b is the qutizer step size d L is the L uber of output levels A3 Gussi Noise Chel: SNR = E[X ] = E[X ] = A E[(X Y) ] E[N ], SNR db = 0log 0 (SNR) = 0log 0 E X = N E[(X Y) ] e jθ e jθ 0log 0 E X = 0log E[N ] 0 A N = 0log 0 A, P[trsissio error] = Q A μ N, E[N ] = N N = A = A SNR N SNR db, where Y is the received 0 0 sigl, Y = X + N, X is the trsit sples, A is the plitude of the Beroulli bit set, d N is the Gussi oise A4 Huff codig produces code C tht is optil (iil) i ters of expected code word legth (iiizes L(C)), Etropy: bits H(x) = x S X P X (x) log P X (x) = E X [log P X (x)] i, D =, source sybol D is the set of fiite legth biry strigs, Expected Legth of Source Code C: L(C) = E X [l(x)] = x S X P X (x)l(x), for good code: H(x) L(C) H(x) +, Efficiecy = H(x) 00%, Ifo Coveyed = L(C) log P X (x) Mth Fcts B0 y e y dy =! B Hlf Agle Foruls: () cos(a + B) = cosacosb siasib, (b) si(a + B) = siacosb + cosasib, (c) cos(a) = cosa sia = 0 cos A = si A, (d) si(a) = siacosa B Products of Siusoids: () siasib = [cos(a B) cos(a + B)], (b) cosacosb = [cos(a B) cos(a + B)], (c) siacosb = [si(a + B) + si(a B)] B3 The Euler Forul: The Euler forul ejθ = cosθ + jsiθ is the source of the idetities () cosθ = ejθ +e jθ, (b) siθ = B4 Fiite Geoetric Series: The fiite geoetric series is q i = + q + q + + q = q+ B5 Ifiite Geoetric Series: Whe q <, i=0 qi = li i=0 q i = B6 q i= iqi Clculus Foruls C d dx (x ) = x l C d l x = C3 d log dx x dx x = u l(u) u + C i=0 = q( q [+( q)]) ( q) B7 If q <, iq i i= = x l q ( q) B8 j j= C4 Itegrtio by Prts: b udv q = (+) j= B9 j b = (+)(+) 6 = uv b vdu C5 du = l u + C C6 u du = u C7 l u du = u l