Elements of Lecture II Hamid R. Rabiee with thanks to Ali Jalali
Overview Reading Assignment Chapter 9 of textbook Further Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A First Course in Stochastic Processes, nd ed., Academic Press, New York, 975.
Outline Basic Definitions Statistics of Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 3
Basic Definitions Suppose a set of random variables indexed by a parameter Tracking these variables with respect to the parameter constructs a process that is called Stochastic Process. X (t, β)= (X (β), X (β),..., X n (β),...) i.e. The mapping of outcomes to the real (complex) numbers changes with respect to index. 4
Basic Definitions (cont d) A stochastic process x(t) is a rule for assigning to every a function x(t, ). ensemble of functions: Family of all functions a random process generates 5
Basic Definitions (cont d) With fixed (zeta), we will have a time function called sample path. Are sample paths sufficient for estimating stochastic properties of a random process? Sometimes stochastic properties of a random process can be extracted just from a single sample path. (When?) 6
Basic Definitions (cont d) With fixed t, we will have a random variable. With fixed t and zeta, we will have a real (complex) number 7
Basic Definitions (cont d) Example I Brownian Motion Motion of all particles (ensemble) Motion of a specific particle (sample path) 8
Basic Definitions (cont d) Example II Voltage of a generator with fixed frequency Amplitude is a random variable V t, A. cos t 9
Basic Definitions (cont d) Equality Ensembles should be equal for each beta and t t, Y t, Equality (Mean Square Sense) If the following equality holds E Sufficient in many applications X X t, Y t, 0 0
Statistics of First-Order CDF of a random process F X x, t PrX t x First-Order PDF of a random process f x x, t F x t X X,
Statistics of (cont d) Second-Order CDF of a random process Second-Order PDF of a random process, ;,., ;, t t x x F x x t t x x f X X Pr, ;, x t X and x t X t t x x F X
Statistics of (cont d) n th order can be defined. (How?) Relation between first-order and secondorder can be presented as f X x, t f x, ; t, t Relation between different orders can be obtained easily. (How?) X 3
Statistics of (cont d) Mean of a random process X t x. f x t Autocorrelation of a random process R t E X, dx t t EX t. X t x. x. f X x, x; t, t, dx dx Fact: R t t, X ( t) X ( t) (Why?) 4
Statistics of (cont d) Autocovariance of a random process, t Rt, t t. t EX t t. X t t C t Correlation Coefficient C t, t r t, t Ct, t. Ct, t Example E X t X t Rt, t Rt, t Rt t, 5
End of Section Any Question? 6
Outline Basic Definitions (cont d) Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 7
Basic Definitions (cont d) Example Poisson Process. t e K. k! t t k Mean t. t Autocorrelation R t, t. t. t. t. t. t. t t t t t Autocovariance C t, t.mint t, 8
Basic Definitions (cont d) Complex process Definition t X t i Y t Z. Specified in terms of the joint statistics of two real processes X t and Y t Vector Process A family of some stochastic processes 9
Basic Definitions (cont d) Cross-Correlation R XY Orthogonal Processes t t R XY t, t EX t Y t. 0, Cross-Covariance C XY Uncorrelated Processes t t C XY t, t R t, t t t XY X. 0, Y 0
Basic Definitions (cont d) a-dependent processes White Noise t t 0 t a C t, t t 0 t C t, Variance of Stochastic Process t, t t C X
Basic Definitions (cont d) Normal Process t variables x t,..., x ( t and t,...,t. Existence Theorem x ) A Process is called Normal, if the random are jointly normal for any For an arbitrary mean function t For an arbitrary covariance function C t,t There exist a normal random process that its mean is t and its covariance is C t,t n
Outline Basic Definitions Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 3
Stationary Processes Strict Sense Stationary (SSS) Statistical properties are invariant to shift of time origin First order properties should be independent of t or f X x, t f X x Second order properties should depends only on difference of times or t t f X x, x; t, t f X x, x; 4
Stationary Processes (cont d) Wide Sense Stationary (WSS) Mean is constant E X t Autocorrelation depends on the difference of times E X t. X t R First and Second order statistics are usually enough in applications. 5
Stationary Processes (cont d) Autocovariance of a WSS process C R Correlation Coefficient r C C 0 6
Stationary Processes (cont d) White Noise C q. If white noise is an stationary process, why do we call it noise? (maybe it is not stationary!?) a-dependent Process C 0 a a is called Correlation Time 7
Stationary Processes (cont d) Example SSS Suppose a and b are normal random variables with zero mean. X Why is it SSS? WSS Suppose has a uniform distribution in the interval, X Why is it WSS? t a. cos t b. sin t t a. cos t 8
Stationary Processes (cont d) Example Suppose for a WSS process X(8) and X(5) are random variables E R A. e X 8 X 5 E X 8 E X 5 R0 R0 R3 EX 5. X 8 9
End of Section Any Question? 30
Outline Basic Definitions Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 3
Stochastic Analysis of Systems Linear Systems y t g xt a. yt b ga. xt b a, b Time-Invariant Systems y t g xt yt gxt Linear Time-Invariant Systems y t g x t y t h t * x t Where h(t) is called impulse response of the system 3
Stochastic Analysis of Systems (cont d) Memoryless Systems y t g x 0 t 0 Causal Systems y t g xt t 0 0 t Only causal systems can be realized. (Why?) 33
Stochastic Analysis of Systems (cont d) Linear time-invariant systems Mean yt Eht * xt ht Ext E * Autocorrelation R yy * t, t h t * R t, t h t xx * 34
Stochastic Analysis of Systems (cont d) Example I System: y t t 0 x( ). d Impulse response: Output Mean: E h t t t 0 ( t). dt yt Ext. 0 dt U t Output Autocovariance: R yy t t Rxx t, t. d. 0 0, d 35
Stochastic Analysis of Systems (cont d) Example II System: t d dt Impulse response: y xt h t d dt t Output Mean: Output Autocovariance: E d dt yt Ext R yy xx, t. t t, t R t t 36
Outline Basic Definitions Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 37
Power Spectrum Definition WSS process X Autocorrelation t R Fourier Transform of autocorrelation S R. e j d 38
Power Spectrum (cont d) Inverse trnasform R S. e j d For real processes S R S.cos. 0 S.cos. 0 d d 39
Power Spectrum (cont d) For a linear time invariant system S yy * H. S. H H xx. S xx Fact (Why?) Var yt S. H xx d 40
Power Spectrum (cont d) Example I (Moving Average) System t tt Impulse Response Power Spectrum y T tt x. S d yy H sin T. T. S xx sin T T.. Autocorrelation R yy T T. Rxx T T. d 4
Power Spectrum (cont d) Example II System y t d dt xt Impulse Response Power Spectrum H i. S. S yy xx 4
Outline Basic Definitions Stationary Processes Stochastic Analysis of Systems Power Spectrum Ergodic Processes 43
Ergodic Processes Ergodic Process Equality of time properties and statistic properties. First-Order Time average T Defined as EX t X t t lim T X t Mean Ergodic Process E X t EX t Mean Ergodic Process in Mean Square Sense T E X t 0 lim C. d 0 t T T 0 T X 44 T dt T
Ergodic Processes (cont d) Slutsky s Theorem A process X(t) is mean-ergodic iff lim T T T 0 C d 0 Sufficient Conditions a) 0 C d b) lim T C 0 45