Probability, Random Variables and Random Processes

Transcription

1 Appedix A robability, Radom Variables ad Radom rocesses I this appedix basic cocepts from probability, radom processes ad sigal theory are reviewed.. robability ad Radom Variables robability Space Ω F Ω is the sample space or set of all possible outcomes. F is a collectio of evets which are subsets of Ω (algebra, field) is a fuctio from F 0 which satisfies i) 0 A A F ii) Ω A F B F A B F ; A F Ω F iii) If A B /0 the A B A B A B F A radom variable X w is a fuctio from Ω to R Ā F X : Ω R that satisfies w Ω : X w x F x R The distributio fuctio F X x of a radom variable is defied as roperties of distributio fuctios (i) F X x a X w b F X b F X a (ii) F X x is cotiuous at x iff X w x 0 X w x w Ω : X w x (iii) lim y x F X y F X x right cotiuous -

2 -2 AENDIX A. ROBABILITY, RANDOM VARIABLES AND RANDOM ROCESSES (iv) lim x F X x lim x F X x 0 If F X x is cotiuous for all x the there exists a fuctio f X x such that F X x x f X u du This fuctio is called the desity fuctio. If a radom variable has a desity fuctio we shall say the radom variable is cotiuous. roperties of desity fuctios (i) X w B B f X u du B (ii) f X x F X x df X x dx R If F X x is piecewise costat with a coutable umber of discotiuities the X is said to be a discrete radom variable. For discrete radom variables we will use their probability mass fuctio p X x Expectatio of a Radom Variable The expectatio of a cotiuous radom variable is E X The expectatio of a discrete radom variable is E X X x x f X x dx xp X x x:p X x 0 If X X 2 X are radom variables the joit distributio F x x is defied as F x x X x X x If these radom variables are (joitly) cotiuous the their joit desity f x x is defied as f x x F x x x x If these radom variables are discrete the the joit probability mass fuctio is p x x X x X x A complex radom variable is a fuctio from Ω to C (Cl is the set of complex umbers) such that RX x r I X x i F x r x i R X : Ω X w Re X w C j Im X w Useful Bouds ) Uio Boud: A B A B M i A i M i A i

3 . -3 Thus 2) Chebyshev Boud: Let m X E X ad σ 2 X E X m X 2 the roof: 3) Cheroff Boud: roof: Sice s 0 X u σ 2 x X X m X δ σ 2 X δ 2 x m X 2 f X x dx x m X δ x m X δ X m X δ 2 δ 2 x m X 2 f X x dx f X x dx δ u e su E e sx s x u 0 x u Let g x u g x e s x u f X x dx 0 g x f X x dx E g X E e s X u e su E e sx Example: Let X X be radom variables. Let H 0 ad H be two evets. Let p 0 x x be the coditioal desity fuctio of X X give H 0 ad p x x be the coditioal desity fuctio of X X give H. Fid a boud o Let Y l p X p 0 X e e Y p X X p X X p 0 X X p l X X p 0 X X 0 H 0 E e sy H 0 A radom variable is Gaussia if the desity fuctio is p X x 2πσ exp p 0 X X H 0 H 0 0 H 0 R exp sl p x p 0 x p 0 x dx 2σ 2 x µ 2 where µ is the mea ad σ 2 is the variace. The characteristic fuctio of a radom variable X is defied as φ X s E e s X. For a Gaussia radom variable the characteristic fuctio is φ X s e s2 σ 2 2 µs

4 -4 AENDIX A. ROBABILITY, RANDOM VARIABLES AND RANDOM ROCESSES Def: A fuctio g : R N R is said to be cocave (covex ) if for ay x R x 2 R ad 0 θ Θg x Θ g x 2 g Θx Θ x 2 where the vector additio is compoet-wise additio. A fuctio g : R R is said to be covex (covex ) if Θg x θ g x 2 g Θx Θ x 2 Jese s Iequality: If f x is a cocave (covex ) fuctio mappig R E f X f E X If f x is a covex (covex ) fuctio mappig R R the E f X f E X R the roof for discrete radom variables: (By iductio) Let X take o values x x 2, with ozero probability E f X p x f x p x 2 f x 2 f p x x p x 2 x 2 f E X where the first iequality is due to the defiitio of covexity. Assume if X is discrete takig values Now let X take values x x 2 x E f X Let α x x the i p x i i p x i f x i f i p x i f x i j p x j E f X α i E f X α f f f p x i f x i i i p x i f x i p x f x p x i α f x i p x f x p x i i α i p x i α x i p x f x i p x i x i p x x i p x i x i Let X X be a radom vector. The covariace matrix of X X is defied to be K X K K 2 K K 2 K K

5 . -5 where ad K i j E X i µ i X j µ j µ i E X i Def: A matrix is said to be oegative defiite if for ay vector a a i.e., i j a i k i ja j 0 ad real ak X a T 0 ad real (positive defiite if strict iequality holds). Claim: The covariace matrix is always oegative defiite. roof: i j E E E i j i a i k i j a j j a j a i E X i µ i X j µ j a i X i µ i a j X j µ j a j X j µ j a i X i µ i i i a i X i µ i Let X X be a real radom vector. The characteristic fuctio of X X 2 X is defied as Ψ X X ν ν E j 2 exp j Def: The radom vector X X is said to be joitly Gaussia if the characteristic fuctio of X X is Ψ X X ν ν exp 0 jν T µ i ν i X i 2 νt Kν where ν T ν ν µ T µ µ ad K is a real symmetric oegative defiite matrix. If K is positive defiite the the joit desity of X X is p x 2π 2 detk 2 exp 2 x µ T K x µ Fact: Let X be a radom vector. The X is joitly Gaussia iff X ca be expressed as WY µ where µ µ µ lr W is ad matrix ad Y Y are idepedet mea zero Gaussia radom variables (the matrix W ca be take to be orthogoal, i.e. the rows of W are orthogoal). K x WK Y W T Now let X be a joitly Gaussia radom vector (of legth ) with mea µ covariace matrix K. Let F be a by matrix. Cosider the radom variable Y XFX T

6 -6 AENDIX A. ROBABILITY, RANDOM VARIABLES AND RANDOM ROCESSES We would like to be able to determie the desity fuctio of this radom variable. Istead, we will determie the characteristic fuctio of this radom variable. The characteristic fuctio is For example, let, the K σ 2 ad Ψ Y ν E exp jνy exp jνµ T F 2 jνk µ det I 2 jνkf Ψ Y ν exp jνµ 2 F 2 jνσ 2 F 2 jνσ 2 F Ivertig this yields the Ricia distributed radom variable. For ν 2 exp sµ provided that Re s 2σ Radom rocesses E exp sy E exp sx 2 Def: A radom process set, X t is a radom variable). Def: The covariace fuctio of a radom process where µ t E X t. Def: A fuctio K s t : R R ad ay fuctio a t js, F the characteristic fuctio becomes 2sσ 2 2sσ 2 X t ;t T is a idexed collectio of radom variables (i.e. for each t T, the idex X t ;t T is defied as K s t E X s µ s X t µ t R is said to be oegative defiite if for ay i a t i K t i t j a t j j 0 (ad is real) ad time istats t t (positive defiite if strict equality holds). Equivaletly a t K t s a s dtds 0 ad is real. Claim: The covariace fuctio is a oegative defiite fuctio. Def: A radom process is said to be Gaussia if for ay ad time istaces t t, X t X t is joitly Gaussia.