Problem Sheet 1 Examples of Random Processes

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 1 Examples of Random Processes 1. Give examples of situations in which time series can be used for explanation, description, forecasting and control.. Give examples of a continuous and a discrete random process. 3. In the two examples of Q. determine if the processes are quasideterministic or not. 4. Plot a one-dimensional probability density function (PDF) at any discrete time moment t i =iδt (i is a positive integer and Δt is a time interval) for (a) a binary random process of throwing a fair coin, (b) a process of throwing a die at equal time intervals Δt, (c) a process of throwing at equal time intervals Δt a biased coin, in which probability of heads is 0.4 and of tails 0.6. 5. Find mean values and variances at any time moment t i of the processes (a) and (b) from Q. 4. Illustrate their positions on the plots for PDFs by analogy with examples in lecture notes. 6. Find mean value and variance of a continuous random variable α whose PDF is p α (α) = C cos a + π $ # " 4 % & with α # π,0 & and C being some constant. Note, that first of all you $% 4 '( need to find the value of constant C. 1

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet Statistical Characteristics of a Random Process, Stationarity Consider random process X(t): 1. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable uniformly distributed in the interval [ π,π].. X(t)=ξcos(ωt)=f(ξ,t), where ω is constant, ξ is a random variable uniformly distributed in the interval [-1,1]. 3. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable that with equal probability takes two values: -π/4 and π/4. 4. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable uniformly distributed in the interval [0,π/]. 5. Let X(t)=cos(ωt+ϕ), where ω and ϕ are random variables with joint probability density function ωϕ p = 1 8π, ω [8;1], ϕ [ π; π] For processes 1 5 Find mean value and variance of random process X(t). Find autocorrelation function of random process X(t). Is the process wide sense stationary?

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet : Answers 1) X(t) 0, σ X (t) = A, K XX (t, τ) = Ψ XX (t, τ) = A cosωτ ) X(t) 0, σ X (t) = 1 3 cos ωt, K XX (t,τ) = Ψ XX (t,τ) = 1 ( cos(ωt + ωτ) + cosωτ) 6 3) X(t) = A cosωt, σ X (t) = A sin ωt, K XX (t, τ) = A cos(ωτ), Ψ A A XX (t, τ) = cos(ωτ) cos(ωt + ωτ) 4 4 4) X(t) = A π cos(ωt + π /4), σ & X (t) = A 4 ) & ( A ' π + sin(ωt) A * π ) ( 4A ' π +, * K XX (t, τ) = A cos(ωτ) A π sin(ωt + ωτ), Ψ 5) X(t) 0, σ X (t) = 1, K (t, τ) = Ψ 1 XX XX (t, τ) = ( sin(1τ) sin(8τ) ) 8τ XX & A (t, τ) = 4 ) & ( A ' π + cos(ωτ) A * π 4 ) ( A ' π + sin(ωt + ωτ) * 3

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 3 Properties of Autocorrelation and Covariance Prove the following formulae: 1. Ψ XX (t 1,t ) = Ψ XX (t,t 1 ). Ψ XX (t,t + τ) = ( X(t) X(t) ) ( X(t + τ) X(t + τ) ) = X(t)X(t + τ) X(t) X(t + τ) 3. Ψ XY (t,t + τ) = ( X(t) X(t) ) ( Y(t + τ) Y(t + τ) ) = X(t)Y(t + τ) X(t) Y(t + τ) 4. σ X (t) = X ( t) X t ( ) 4

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 4 Ergodicity of a Random Process All processes are as in PS.. Consider random process X(t): 1. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable uniformly distributed in the interval [ π; π].. X(t)=ξcos(ωt)=f(ξ,t), where ω is constant, ξ is a random variable uniformly distributed in the interval [-1; 1]. 3. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable that with equal probability takes two values: -π/4 and π/4. 4. X(t)=Acos(ωt+ϕ)=f(ϕ,t), where A and ω are constants, ϕ is a random variable uniformly distributed in the interval [0; π/]. 5. Let X(t)=cos(ωt+ϕ), where ω and ϕ are random variables with joint probability density function ωϕ p = 1 8π, ω [8;1], ϕ [ π; π] For processes 1 5 Find if the random process X(t) is ergodic with respect to mean value. Find if the random process X(t) is ergodic with respect to variance and covariance. 5

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 4: Answers 1) x(t) = X(t) 0, ( x(t) x )( x(t + τ) x ) = Ψ XX (t, τ) = A cosωτ Yes, this process is ergodic with respect to mean, variance and covariance. ) x(t) = X(t) 0, ( x(t) x )( x(t + τ) x ) = ξ * cosωτ Ψ XX (t, τ) This process is ergodic with respect to mean, but not variance or covariance (compare with results for PS. Q. ). 3 4) x(t) 0, ( x(t) x )( x(t + τ) x ) = A cosωτ The mean values and covariances of these processes obtained by averaging over the ensemble of realizations (see results for PS., Q. 3-4) are different from those obtained by averaging along the single realization. Thus, the processes are not ergodic either with respect to mean, variance or covariance. 5) x(t) = X(t) 0, ( x(t) x )( x(t + τ) x ) = 1 cosω * τ Ψ XX (t, τ) This process is ergodic with respect to mean, but not covariance (compare with results for PS. Q. 5). 6

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 5 Statistical Characteristics of a Random Process, Stationarity More Problems 1. Consider random process X(t)=ξ(t)cos(ωt+ϕ), where ω is constant, ξ(t) is random process that is 1 st order stationary and does not depend on ϕ. ϕ is random variable. Find the conditions that ϕ should satisfy to make random process X(t) wide sense stationary. Hint: consider autocorrelation function of X(t). Let X(t) be a random process with autocorrelation function K XX (τ)=exp(-a τ ), a is a positive constant. Let X(t) modulate the amplitude of a cosine process with random phase Y(t)=X(t)cos(ωt+ϕ). ϕ is a random variable uniformly distributed in the interval [-π; π] and is statistically independent of X(t). Find mean value, autocorrelation function and covariance of Y(t). 3. In problem assume that the only known information about the process X(t) is that it is wide sense stationary and has autocorrelation function C XX (τ). Find mean value of Y(t). Find autocorrelation function of Y(t) in terms of C XX (τ). Is Y(t) wide sense stationary? 4. Let X(t)=Acos(ωt)+Bsin (ωt), where ω is constant, A and B are uncorrelated random variables with zero mean (they may have different distributions). Is X(t) wide sense stationary (WSS)? If the process is not WSS, what additional conditions should be imposed on A and B to make it WSS? 5. Consider a random process X(t)=αt+ξ(t)cos(ωt+ϕ), where ω =const, α is a random variable varying inside [-1; 1] whose probability density distribution (PDD) is p(α)=1/. ξ(t) is a random process with zero mean and covariance function Ψ ξ (t 1,t ) = σ ξ e λ τ, where τ = t 1 t and λ>0; ϕ is random variable varying inside [-π; π] with PDD p(ϕ)=1/(π). Variables ϕ and α, and the process ξ(t) are all statistically independent. Find mean value, variance, autocorrelation and covariance of the process X(t) and determine if X(t) is wide-sense stationary. 7

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 5: Answers 1. cosϕ = 0, sinϕ = 0, cosϕ = 0, sinϕ = 0.. K YY (t, τ) = 1 exp( a τ )cosωτ. 3. Y(t) = 0 K YY (t,τ) = 1 C XX (τ)cosωτ. The process is WSS. % ( % ( 4. X(t) = 0, K XX (t, τ) = cos(ωt + ωτ) ' A B A & * + cosωτ' ) + B & *. The process is not WSS. To ) make the process WSS, variances of A and B should be equal. 5. X(t) 0, σ X (t) = 1 3 t + 1 σ ξ, K XX (t, τ) = Ψ XX (t, τ) = 1 3 t(t + τ) + 1 σ ξ exp( λ τ )cosωτ. The process is not WSS. 8

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 6 Properties of the Fourier Transform (FT) 1. Prove the time-scaling property of the Fourier Transform (FT), i.e. prove the following. Let F(ω) be the FT of the function f(t). Let the argument t of f(t) be multiplied by some real constant a, i.e. let the time be scaled. Then the FT of the function f(at) is equal to 1/ a F(ω/a).. What are the properties of the FT of an odd function f(t)? Find Fourier amplitude and phase spectra for the realizations x(t) in problems 3-7. In all problems sketch the solution. It might be convenient to sketch x(t) first. 3. x(t)=acos(ω0t+ψ) where ψ is not zero. 4. x(t)=asin(ω 0t) 5. Sum of two cosines with different frequencies x(t) = A 1 cos(ω 1 t) + A cos(ω t). 6. Sum of a cosine and a sine with different frequencies x(t) = A 1 cos(ω 1 t) + A sin (ω t). 7. Cosine with periodically modulated amplitude: where m< 1. x(t) = (1+ m cos(ω 1 t))cos(ω t), 8. Find Fourier Transform of unit impulse train of period T, see Figure: f (t) = δ(t nt) = + δ(t + T) + δ(t + T) + δ(t) + δ(t T) + δ(t T) + n= 9

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 6: Answers to selected problems 1.. 3. F(ω) = Aπ[ δ( ω ω 0 ) + δ( ω + ω 0 )], ϕ(ω) = ψ ω ω 0 4. F(ω) = Aπ[ δ( ω ω 0 ) + δ( ω + ω 0 )], ϕ(ω) = π 5. F(ω) = A 1 π δ ω ω 1 6. [ ( ) + δ( ω + ω 1 )] + A π δ( ω ω ) + δ( ω + ω ) ω ω 0 [ ], ϕ(ω) 0 7. Realizations x(t) qualitatively look like in left-hand figure below. [ ( ) + δ( ω + ω )] + m π δ ( ω (ω 1 + ω )) + δ( ω + (ω 1 + ω )) F(ω) = π δ ω ω + m π δ ω (ω 1 ω ) [ ] + [ ( ) + δ( ω + (ω ω ))], ϕ(ω) 0 1 In the right-hand side of the figure the amplitude of FT is shown for the parameters taken as in the left-hand side. ' * 8. I δ(t nt) ( ) +, n= = e jωnt n= 10

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 7 Fourier Transform and Wiener-Khintchine Theorem 1. Consider a rectangular pulse p(t) # 1, 0 t T, p(t) = $ % 0, t < 0 or t > T. a) Calculate the Fourier Transform (FT), F(ω), of p(t). Find and sketch the amplitude Fourier spectrum, F(ω), of p(t) for T=10. b) Consider x(t) = p(t)cosω 0 t. Calculate the FT, F 1 (ω), of x(t) directly using the definition. Compare your result with the one obtained by application of frequencyshifting property of the FT. c) Calculate the squared amplitude Fourier spectrum, F 1 (ω), of x(t). Compare your result with the squared amplitude Fourier spectrum of y(t) = p 1 (t)cosω 0 t, where that was obtained in lecture. $ & 1, T p 1 (t) = t T, % & 0, t > T ', d) Sketch carefully the F 1 (ω) for T=10, ω 0 =5.. a) Calculate the Fourier Transform (FT), F (ω), of z(t) = p(t)sinω 0 t, where p(t) is the same as in Q. 1. Obtain the same result by two methods: from the direct definition of FT using the frequency-shifting property of FT b) Calculate the squared amplitude Fourier spectrum, F (ω), of z(t) from a). Compare your result with the squared amplitude Fourier spectra of x(t) and y(t) of Q. 1. 3. Which of the following functions could be a valid power spectral density, given that a, b, c, and f 0 are real positive numbers? f is frequency in Hz, a) g) af b + f, b) a a + cf b + f, d) a f b, e) b + f, c) a b + jf, h) aδ( f f ) aδ( f + f ), i) cos(3 f ) 0 0. 1+ f a a + f + f, f ) δ( f ) + a b + f, 4. Consider a random process that is 1 st order stationary with zero mean and the power spectral density $ S(ω) = W = const, ω ω 0 0 %, & 0, ω > ω 0 Find autocorrelation function of this process and sketch it. 11

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' 5. Consider a random process X(t)=A 0 cos(ω 0 t+ϕ)=f(ϕ,t), where A 0 and ω 0 are constants, ϕ is a random variable uniformly distributed in the interval [-π; π]. Find power spectral density of X(t) by two methods: using Wiener-Khinchine theorem and from periodograms. Sketch power spectral density. [Hint: autocorrelation function of this process was found within Problem 1 of Problem Sheet.] 6. Consider stationary random process X(t) with zero mean and autocorrelation function Ψ XX (τ) = σ X e α τ, α=const>0. Find power spectral density of this process and sketch it. Find the power of the process. [Hint: you can use the following identity: e ax cos(bx)dx = eax a + b ( acos(bx) + bsin(bx) ) + C. ] 7. Find power spectral density of a wide-sense stationary process whose autocorrelation function is K XX (τ) = e τ cos(4πτ). 8. Consider a random process X(t) that is wide sense stationary with zero mean and with the power spectral density S X (ω) % S X (ω) = W 0 (1 ω ), ω 1, & ' 0, ω >1. W 0 Find autocorrelation function and power of this process. Sketch autocorrelation function. 9. Consider a random process X(t) that is wide sense stationary with zero mean and with the autocorrelation function + K XX (τ) = A $ 1 τ ' - 0& ), T τ T,, % T (. - 0, elsewhere. Find power spectral density and power of this process. Sketch power spectral density. 1

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 7: Answers to selected problems 1. a) F(ω) = 4 # T ω sin % $ ω & ' (, F(ω) = ω sin # % T $ ω & ( ' ( ) 1 ( ) 1 b) F 1 (ω) = 1 * $ j cos (ω ω 0 )T + cos (ω +ω 0, )T & +, % ω ω 0 ω +ω 0 ( ) ( ) ' $ ) + sin (ω ω )T 0 + sin (ω +ω )T '- 0 & ( % ω ω 0 ω +ω ) / 0 (./ $ (ω ω sin 0 )T ' & ) c) F 1 (ω) % ( = ω ω 0 d) ( ) + $ (ω +ω sin 0 )T ' & ) % ( + cos ω T 0 ( ω +ω 0 ) ω ω 0 ( ) [ cos( ω 0 T) cos( ωt) ] The graph for F 1 (ω) is symmetric with respect to zero. 13

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets'. a) F (ω) = 1 * $ j sin ((ω +ω 0 )T) sin ( (ω ω )T 0 )' $ & % ω +ω 0 ω ω ) + cos ( (ω ω )T 0 ) 1 cos (ω +ω 0, )T & +, 0 ( % ω ω 0 ω +ω 0 3. $ (ω ω sin 0 )T ' & ) b) F (ω) % ( = ω ω 0 ( ) + $ (ω +ω sin 0 )T ' & ) % ( cos(ω 0T) ( ω +ω 0 ) ω ω 0 (a) no, since it is not even. (b) yes since it is real, even and positive for all f. (c) no, since it is not even. (d) no, since it can be negative at f <b (e) no, since it is not even (f) yes, since it is real, even and positive for all f. (g) no, since it is not real (h) no, since it is negative at f=-f 0 (i) no, since it is negative when cos(3f) is negative. ( ) 1 [ cos(ω 0T) cos(ωt) ] '- ) / (./ 4. K XX (τ) = W 0 π τ sin(ω 0τ), W P = K XX (0) = lim 0 ω 0 sin(ω 0 τ) = W 0ω 0 τ 0 π ω 0 τ π. The process power can be also estimated as an integral of S(ω) P = 1 S(ω)dω = 1 π π W 0ω 0 = W 0ω 0 π. π 5. S(ω) = A 0 ( δ(ω ω 0 ) +δ(ω +ω 0 )) A π 0 δ( ω + ω 0 ) A π 0 δ( ω ω 0 ) 6. S(ω) = ασ X α +ω P = σ X 14

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' % 1 7. S(ω) = 4 + (4π +ω) + 1 ( ' * & 4 + (4π ω) ) ( ) 8. K XX (τ) = W $ 0 sin τ / ' & ) π % ( τ /) ( ( ) # sin ωt / & 9. S(ω) = A 0 T% ( $ ( ωt /) ' The width of S(ω) is roughly proportional to the difference between the two values of ω closest to 0, at which S(ω)=0: ω 1, T = ±π ω 1, = ± π T ; ω 1 ω = 4π T Thus, the wider the ACF (i.e. the larger the T), the narrower the power spectral density is. 15

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem Sheet 8 Nyquist Theorem, Aliasing 1. A stationary random process X(t) has power spectral density S X (f): $ 7 S X ( f ) = 9 + ( f 3) + 7 & % 9 + ( f + 3), f 10, ' & 0, f >10. A realization of this process is sampled with sampling frequency 1 Hz. Sketch the best power spectral density that could possibly be estimated numerically from this realization if observation time could tend to infinity.. Find the smallest sampling frequency necessary to discretize realizations of the random process X(t) = A(t)cos(ω 0 t + ϕ), where ϕ is a random variable uniformly distributed in [ π; π], and A(t) is a stationary band-width limited random process whose power spectral density S A (f) has the following form: % S A (ω) = W 0(1 ω ), ω 1, & ' 0, ω >1. A(t) and ϕ are statistically independent. Sketch power spectral density S A (f) of the process A(t) and power spectral density S X (f) of X(t), and on the latter plot show the position of the frequency sought. 3. Consider a random process X(t)=A(t)sin(6πt+ϕ), where ϕ is a random variable uniformly distributed in the interval [-π; π], A(t) is a stationary bandwidth-limited random process whose power spectral density S A (f) is given by $ 8(1 f ), f 1, S A ( f ) = % & 0, f >1. A(t) and ϕ are statistically independent. Sketch the power spectral density of the process A(t), S A (f). Find the power spectral density of the process X(t), S X (f). Sketch S X (f) and on this plot show the smallest allowed sampling frequency. Find the power of the process X(t). 4. Consider a random process X(t) = A(t) sin(8πt +ϕ), where ϕ is a random variable uniformly distributed in [0; π], A(t) is a wide-sense stationary random process whose autocorrelation function K AA (τ) is given by + K AA (τ) = $ 1 τ ' - & ), τ 8,, % 8 ( -. 0, τ > 8. Assume that realizations of this process are sampled with sampling step Δt=0.15 sec. Sketch the best power spectral density that could possibly be estimated numerically from a realization of this process if observation time could tend to infinity. 16

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' 5. Assume that a realization of a random process is x(t)=cos(6πt). Let this signal be recorded during time interval [-T; T]. In order to reduce the leakage of power due to finite observation time, one uses a triangular window G(t) before estimating Fourier Spectrum of the original infinitely long signal. The equation of the window function and of the actual signal y(t) from which Fourier Transform is calculated are: $ & G(t) = 1 t % T, t T, y(t) = x(t)g(t). '& 0, t > T, What would be the best estimate of Fourier Transform of x(t), if the sampling step could be taken infinitely small? 6. For all processes considered in Q. 4-9 of Problem Sheet 7 estimate roughly what would be a reasonable sampling frequency for their realizations. You are not required to specify the exact numbers, but only to show a suitable cut-off frequency f c on the sketches of the power spectral densities. 17

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' Problem sheet 8: Answers to selected problems 1. Hint: Calculate the values of S X (f) at several points f, e.g. f=0,1,,3,4,5,6,7,8,9,10. At negative frequencies S X (f) is the same as at the positive ones, because power spectral density is an even function. Solid thick (blue in color) line in the Figure below shows the estimate of the power spectral density sought.. f N = ω 0 + 1 π, 3. S X ( f ) = 1 4 S A ( f + 3) + 1 4 S A ( f 3), P = 4, f N = 8Hz 18

RANDOM'PROCESSES'AND'TIME'SERIES'ANALYSIS.'PART'II:'RANDOM'PROCESSES' '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Problem'Sheets' 4. SX ( f ) = 1 cos(16πf ) 1 1 SA ( f + 4) + SA ( f 4), SA ( f ) = 4 4 8π f 5. S( f ) = 1 $&1 cos(π ( f + 3)T) 1 cos(π ( f 3)T) (& % ) + T &' (π ( f + 3)) (π ( f 3)) &* 19