Elements of Stochastic Processes Lecture II Hamid R. Rabiee

Transcription

1 Sochasic Processes Elemens of Sochasic Processes Lecure II Hamid R. Rabiee

2 Overview Reading Assignmen Chaper 9 of exbook Furher Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A Firs Course in Sochasic Processes, nd ed., Academic Press, New York, 975.

3 Ouline Basic Definiions Saionary/Ergodic Processes Sochasic Analysis of Sysems Power Specrum

4 Basic Definiions Suppose a se of random variables indexed by a parameer Tracking hese variables wih respec o a parameer consrucs a process ha is called Sochasic Process. ( ) ( X ( ), X ( ),..., X ( ),...) X =, n i.e. The mapping of oucomes o he real (complex) numbers changes wih respec o index.

5 Basic Definiions (con d) In a random process, we will have a family of funcions called an ensemble of funcions X, X, X, 3

6 Basic Definiions (con d) Wih fixed bea, we will have a ime funcion called sample pah. Someimes sochasic properies of a random process can be exraced jus from a single sample pah. (When?)

7 Basic Definiions (con d) Wih fixed, we will have a random variable. Wih fixed and bea, we will have a real (complex) number.

8 Basic Definiions (con d) Example I Brownian Moion Moion of all paricles (ensemble) Moion of a specific paricle (sample pah) Example II Volage of a generaor wih fixed frequency Ampliude is a random variables V (, ) = A( ). cos( + )

9 Basic Definiions (con d) Equaliy Ensembles should be equal for each bea and (, ) Y (, ) X = Equaliy (Mean Square Sense) If he following equaliy holds E [( ( ) ( )) ] X, Y, = 0 Sufficien in many applicaions

10 Basic Definiions (con d) Firs-Order CDF of a random process F X ( x, ) = Pr{ X ( ) x} Firs-Order PDF of a random process f x ( x, ) = F ( x ) X X,

11 Basic Definiions (con d) Second-Order CDF of a random process Second-Order PDF of a random process ( ) ( ) ( ) { } Pr, ;, x X and x X x x F X = ( ) ( ), ;,., ;, x x F x x x x f X X =

12 Basic Definiions (con d) n h order can be defined. (How?) Relaion beween firs-order and secondorder can be presened as f X ( x, ) = f ( x, ;, ) Relaion beween differen orders can be obained easily. (How?) X

13 Basic Definiions (con d) Mean of a random process µ ( ) E[ X ( ) ] = x. f X ( x, ) Auocorrelaion of a random process R = dx = dx ( ) E[ X ( ). X ( )] = x. x. f X ( x, x;, ), dx Fac: ( ) = + R µ, X ( ) X ( ) (Why?)

14 Basic Definiions (con d) Auocovariance of a random process Correlaion Coefficien Example ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) [ ]..,, X X E R C µ µ µ µ = = ( ) ( ) ( ) [ ] ( ) ( ) ( ),,, R R R X X E + + = + ( ) ( ) ( ) ( ),.,,, C C C r =

15 Basic Definiions (con d) Example Poisson Process. e K. k! ( ) = ( ) k Mean µ ( ) =. Auocorrelaion R (, ) = Auocovariance C (, ) =.min( ),

16 Basic Definiions (con d) Complex process Definiion ( ) = X ( ) + i Y ( ) Z. Specified in erms of he join saisics of wo real processes X ( ) and Y ( ) Vecor Process A family of some sochasic processes

17 Basic Definiions (con d) Cross-Correlaion Orhogonal Processes (, ) E[ X ( ) Y ( )] R XY = R XY. ( ) 0, = Cross-Covariance C Uncorrelaed Processes (, ) = R (, ) µ ( ) ( ) XY XY X. µ Y C XY ( ) 0, =

18 Basic Definiions (con d) a-dependen processes Whie Noise ( ) 0 > a C, = ( ) 0 " C, = Variance of Sochasic Process (, ) = ( ) C X

19 Basic Definiions (con d) Exisence Theorem For an arbirary mean funcion µ ( ) For an arbirary covariance funcion C(, ) There exis a normal random process ha is mean is µ ( ) and is covariance is C(, )

20 Ouline Basic Definiions Saionary/Ergodic Processes Sochasic Analysis of Sysems Power Specrum

21 Saionary/Ergodic Processes Sric Sense Saionary (SSS) Saisical properies are invarian o shif of ime origin Firs order properies should be independen of or f X ( x, ) = f X ( x) Second order properies should depends only on difference of imes or # = f ( x x ;, ) f ( x, ;# ) X, = X x

22 Saionary/Ergodic Processes (con d) Wide Sense Saionary (WSS) Mean is consan E [ X ( ) ] = µ Auocorrelaion depends on he difference of imes [ X ( ). X ( + # )] R (# ) E = Firs and Second order saisics are usually enough in applicaions.

23 Saionary/Ergodic Processes (con d) Auocovariance of a WSS process ( ) = R( # ) C # µ Correlaion Coefficien ( ) r # = C C (# ) ( 0)

24 Saionary/Ergodic Processes (con d) Whie Noise If whie noise is an saionary process, why do we call i noise? (maybe i is no saionary!?) a-dependen Process C (# ).$ (# ) C = q (# ) = 0 # a a is called Correlaion Time

25 Saionary/Ergodic Processes (con d) Example SSS Suppose a and b are normal random variables wih zero mean. X ( ) = a. cos( ) + b. sin( ) WSS Suppose has a uniform disribuion in he inerval ( %,% ) X ( ) = a. cos( + )

26 Saionary/Ergodic Processes (con d) Example Suppose for a WSS process &# (# ) = A e R. X(8) and X(5) are random variables E [( ( ) ( )) ] [ ( ) ] X 8 X 5 = E X 8 + E X ( 5) = R( 0) + R( 0) R( 3) [ ] E X ( 5 ). X ( 8) [ ]

27 Saionary/Ergodic Processes (con d) Ergodic Process Equaliy of ime properies and saisic properies. Firs-Order Time average Defined as Mean Ergodic Process E T { X ( ) } = X ( ) = µ = lim T ' X ( ) { X ( ) } E[ X ( ) ] E = Mean Ergodic Process in Mean Square Sense [( ( ) ) ] T & E X µ = 0 ( lim T ' CX (& ). d& = 0 T 0 T T d T

28 Saionary/Ergodic Processes (con d) Slusky s s Theorem A process X() is mean-ergodic ergodic iff lim T ' T T 0 C (# ) d# = 0 Sufficien Condiions a) 0 ( ) < C # d# b) ( ) 0 lim C = T ' #

29 Ouline Basic Definiions Saionary/Ergodic Processes Sochasic Analysis of Sysems Power Specrum

30 Sochasic Analysis of Sysems Linear Sysems y ( ) = Sysem{ x( ) } a. y( ) + b = Sysem{ a. x( ) + b} * a, b Time-Invarian Sysems Linear Time-Invarian Sysems y ( ) = Sysem{ x( ) } y( ) = h( ) * x( ) y ( ) = Sysem{ x( ) } y( # ) = Sysem{ x( # )} *# Where h() is called impulse response of he sysem

31 Sochasic Analysis of Sysems (con d) Memoryless Sysems y ( 0 ) = Sysem{ x( ) } 0 Causal Sysems y ( 0 ) = Sysem{ x( ) } 0 Only causal sysems can be realized. (Why?)

32 Sochasic Analysis of Sysems (con d) Linear ime-invarian sysems Mean [ y( ) ] = E[ h( ) * x( ) ] = h( ) E[ x( ) ] E * Auocorrelaion R * (, ) h( )* R (, ) h ( ) yy = xx *

33 Sochasic Analysis of Sysems (con d) Example I Sysem: y ( ) = 0 x( &). d& Impulse response: Oupu Mean: Oupu Auocovariance: R yy E h ( ) $ ( ). d = U ( ) = = 0 [ y( ) ] E[ x( ) ]. 0 d ( ) = Rxx ( &, ). d&. 0 0, d

34 Sochasic Analysis of Sysems (con d) Example II Sysem: d d ( ) x( ) y = Impulse response: d h $ d ( ) = ( ) Oupu Mean: d E = d Oupu Auocovariance: [ y( ) ] E[ x( ) ] R yy xx,. (, ) = R ( )

35 Ouline Basic Definiions Saionary/Ergodic Processes Sochasic Analysis of Sysems Power Specrum

36 Power Specrum Definiion WSS process X Auocorrelaion ( ) R (# ) Fourier Transform of auocorrelaion S ( ) R( # ) j # d =. e #

37 Power Specrum (con d) Inverse rnasform (# ) S( ) For real processes R S R j =. e # d % ( ) = S( ).cos(.# ) % 0 (# ) = S( ).cos(.# ) 0 d d

38 Power Specrum (con d) For a linear ime invarian sysem S yy * ( ) = H ( ). S ( ). H ( ) = H xx ( ). S ( ) xx Fac (Why?) Var = xx d % [ y( ) ] S ( ). H ( )

39 Power Specrum (con d) Example I (Moving Average) Sysem T + T ( ) = x( & ). T Impulse Response Power Specrum y S d& ( T. ) sin H ( ) = T. ( ) S ( ) yy = xx sin T ( T. ). Auocorrelaion R yy T = ( ). Rxx (# & ) T T & T. d&

40 Power Specrum (con d) Example II Sysem d y = d ( ) x( ) Impulse Response Power Specrum yy H ( ) = i. ( ) = S ( ) S. xx

41 Nex Lecure Saionary Sochasic Processes