16.584: Random (Stochastic) Processes

Size: px

Start display at page:

Download "16.584: Random (Stochastic) Processes"

Brook Stephens
5 years ago
Views:

1 : Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable ζ: Outcome Ex: Noise measurements ; Number of customers accessing a resource; Bit rate of a digital video or audio signal Intensity variation over an image X(t, ζ): Mapping outcomes ζ of an experiment to the real line Ex: X(t) = A cos(ωt + φ) : where A, φ may be Random variables

2 2 Stochastic Processes ζ 3 X(t, ζ) ζ 2 ζ 1 t n X(t n, ζ) t Figure 1: X(t): Continuous time or Discrete time Outcomes of X: Continuous or Discrete (States) of the process X(t;ζ) for a fixed ζ is a time function : Sample of RP X(t;ζ) for a fixed t : RV equal to state of the process at t X(t;ζ) for fixed t and ζ is a number

3 3 Specification of Random Processes X(t, ζ) x 1 x 2 x n t 1 t 2 t n t Figure 2: Joint Distribution (n th order ) F X1,X 2,...,X n (x 1, x 2,...x n ;t 1, t 2,..t n ) = P[X 1 x 1, X 2 x 2,...X n x n ] Joint pdf: f X1,X 2,...,X n (x 1, x 2,...x n ;t 1, t 2,..t n ) First Order: F X (x, t) = P[X(t) x] for a fixed value of t

4 4 Second order Characterization: Moments: Mean, Variance, Correlation, Covariance Functions µ X (t) = E[X(t)] Autocorrelation Function < t < : µ X (t) = xf X(t) (x, t)dx R XX (t 1, t 2 ) = E[X(t 1 )X (t 2 )] = x 1 x 2 f X1,X 2 (x 1, x 2 ;t 1, t 2 )dx 1 dx 2 Autocovariance Function: K XX (t 1, t 2 ) = E[(X(t 1 ) µ X (t 1 ))(X(t 1 ) µ X (t 1 )) ] = R XX (t 1, t 2 ) µ X (t 1 )µ X(t 2 ) Variance: σ 2 X(t) = K XX (t, t) = R XX (t, t) µ X (t) 2 Average Power : R XX (t, t) Correlation Coefficient: ρ X (t 1, t 2 ) = K XX(t 1,t 2 ) σ 2 X (t 1 )σx 2 (t 2) Note: ρ X (t 1, t 2 ) 1

5 5 Stationary Stochastic Processes RP X(t) is Strict Sense Stationary (SSS) if statistical properties are invariant to shift in time origin n th order density : f X1,X 2,...,X n (x, t 1, t 2,..t n ) f X1,X 2,...,X n (x, t 1 +τ,t 2 +τ,..t n + τ) First order density f X (x, t) f X (x, t + τ) for all τ f X (x; t) = f X (x) : Independent of t Second Order : f(x 1, x 2 ;t 1, t 2 ) f(x 1, x 2 ;t 1 + τ, t 2 + τ) : for any τ If τ = t 2 : f(x 1, x 2 ;t 1 + τ, t 2 + τ) f(x 1, x 2 ;t 1 t 2, 0) Therefore : f(x 1, x 2 ;t 1, t 2 ) f(x 1, x 2, τ) where τ = t 1 t 2

6 6 Wide-Sense Stationary (WSS) Processes µ X (t) = µ X : Mean is constant (independent of time ) Autocorrelation depends only on τ = t 1 t 2 R XX (t 1, t 2 ) = R XX (t 1 t 2, 0) R XX (τ, 0) R XX (τ) = E[X(t + τ)x (t)]

7 7 Linear Systems and Cross-Correlation Processes Consider two stochastic processes X(t) and Y (t) For example: X(t) is input to a linear system and Y (t) is the output process System Impulse Response: h(t) 0 < t < : Time- Invariant and Causal X(t) h(t) Y(t) Causal and Time Invariant System Figure 3: Auto and Cross Correlation Functions R XX (t 1, t 2 ) = E [X(t 1 )X (t 2 )] R Y X (t 1, t 2 ) = E [Y (t 1 )X (t 2 )] R XY (t 1, t 2 ) = E [X(t 1 )Y (t 2 )] R Y Y (t 1, t 2 ) = E [Y (t 1 )Y (t 2 )]

8 8 Cross-Correlation Functions R XY (t 1, t 2 ) = E [ X(t 1 ) t 2 X (τ 1 )h ] (t 2 τ 1 )dτ 1 = t 2 R XX(t 1, τ 1 )h (t 2 τ 1 )dτ 1 (1) R XY (t 1, t) = R XX (t 1, t) h (t) (2) R Y X (t 1, t 2 ) = E [ t 1 X(τ 1)h(t 1 τ 1 )dτ 1 X (t 2 ) ] = t 1 R XX(τ 1, t 2 )h(t 1 τ 1 )dτ 1 (3) R Y X (t, t 2 ) = R XX (t, t 2 ) h(t) (4) where the symbol represents convolution operation.

9 9 The correlation of the output signal: R Y Y (t 1, t 2 ) = E [ t 1 x(τ 1)h(t 1 τ 1 )dτ 1 Y (t 2 ) ] = t 1 E [X(τ 1)Y (t 2 )] h(t 1 τ 1 )dτ 1 = t 1 R XY (τ 1, t 2 )h(t 1 τ 1 )dτ 1 (5) R Y Y (t, t 2 ) = R XY (t,t 2 ) h(t) (6) or in terms of R Y X (t 1, t 2 ), R Y Y (t 1, t 2 ) = E [ Y (t 1 ) t 2 X (τ 2 )h ] (t 2 τ 2 )dτ 2 = t 2 E [Y (t 1)X (τ 2 )] h(t 2 τ 2 )dτ 2 = t 2 R Y X(t 1, τ 2 )h (t 2 τ 2 )dτ 2 (7) R Y Y (t,t 2 ) = R Y X (t 1, t) h (t) (8)

10 10 Application to WSS Processes X(t) is WSS Constant Mean : µ X and Autocorrelation: function of τ = t 1 t 2 R XX (t 1, t 2 ) = R XX (t 2 + τ, t 2 ) = R XX (t 2 + τ t 2 ) R XX (t 1, t 2 ) = R XX (τ) (9) R XY (τ) = t 2 R XX(t 1 τ 1 ) h (t 2 τ 1 )dτ 1 = τ R XX(ζ)h (ζ τ)dζ = τ R XX (ζ)h ( (τ ζ))dζ (10) R XY (τ) = R XX (τ) h ( τ) (11) Applying the transformation t 1 τ 1 = ζ

11 11 Similiarly R Y X (τ) = t 1 R XX(τ 1 t 2 )h(t 1 τ 1 )dτ 1 = τ R XX(ζ)h(τ ζ)dζ (12) R Y X (τ) = R XX (τ) h(τ) (13) applying the transformation τ 1 t 2 = ζ Under Stationarity Condition: R Y Y (t 1, t 2 ) = R Y Y (t 1 t 2 ) = R Y Y (τ) In terms of R XY (τ): R Y Y (τ) = t 1 R XY (τ 1 t 2 )h(t 1 τ 1 )dτ 1 = τ R XY (ζ)h(τ ζ)dζ R Y Y (τ) = R XY (τ) h(τ) (14) where the transformation τ 1 t 2 = ζ has been applied.

12 12 In terms of R Y X (τ) : R Y Y (τ) = t 2 R Y X(t 1 τ 2 )h (t 2 τ 2 )dτ 2 R Y X (ζ)h (τ + ζ)dζ = τ R Y Y (τ) = R Y X (τ) h ( τ) (15) where the transformation t 1 τ 2 = ζ has been applied. In completion, we note that on substituting for R XY (τ) and R Y X (τ) in the above equations, R Y Y (τ) may be obtained as, R Y Y (τ) = R XX (τ) h ( τ) h(τ) = R XX (τ) h(τ) h ( τ) (16) R Y Y (τ) = R XX (τ) g(τ) (17) g(τ) = h(τ) h ( τ) (18)

13 13 Power Spectral Density (PSD) Define PSD of X(t) S XX (ω): Fourier Transform (FT) of R XX (τ): S XX (ω) = R XX(τ)e jωτ dτ (19) FT of the convolution R XX (τ) h(τ) is the product S XX (ω)h(ω) where H(ω) is the FT of h(t) Relations derived above in the time domain may be represented in the frequency domain as: R XY (τ) = R XX (τ) h ( τ) S XY (ω) = S XX (ω)h (ω) (20)

14 14 R Y X (τ) = R XX (τ) h(τ) S Y X (ω) = S XX (ω)h(ω) (21) R Y Y (τ) = R XX (τ) h(τ) h ( τ) S Y Y (ω) = S XX (ω)h(ω)h (ω) (22) R Y Y (τ) = R XX (τ) g(τ) S Y Y (ω) = S XX (ω)g(ω) (23) G(ω) = H(ω)H (ω) = H(ω) 2 (24)

EAS 305 Random Processes Viewgraph 1 of 10. Random Processes

EAS 305 Random Processes Viewgraph 1 of 10 Definitions: Random Processes A random process is a family of random variables indexed by a parameter t T, where T is called the index set λ i Experiment outcome