ECE 636: Systems identification

Size: px
Start display at page:

Download "ECE 636: Systems identification"

Transcription

1 ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain

2 υ ε x S z + y Experimental I/O data > Selection of model type >Selection of criterion >Calculation of model parameters >Model validation

3 Selection of model type: two main model categories Nonparametric (black box): Linear systems impulse response/ frequency response M yn ( ) = hmxn ( ) ( m) Ye jω ( ) He jω j ( ) Xe ω = ( ) m = 0 n Nonlinear systems: Volterra series: Parametric (grey box): Linear systems linear differential/difference equations d y() t dy() t du() t a + a + y() t = b + b0u() t dt dt dt ayn ( ) + ayn ( ) + yn ( ) = bun ( ) + bun ( ) 0 Nonlinear systems: Nonlinear difference equations, block models 3

4 Curve fitting Model complexity, amount of data, criterion of fit: important! wˆ arg min E( w) N = w N E( w) = tk g( xk, w) Ν k =

5 Review of random variable basics Gaussian random variables, central limit theorem Random signals Correlation and covariance functions Stationarity (weak/wide sense and strict), ergodicity Note: Only stationary signals can be ergodic 5

6 Independent/Uncorrelated random variables Independent random variables: pxy (, ) = px ( ) py ( ) In the general case, for N random variables: = N N i= = px (,..., X ) px ( ) Uncorrelated random variables: E{ XY} = E{ X} E{ Y} i IfXYareuncorrelated X,Y uncorrelated, thecovariance betweenthemis them zero: Cov( X, Y ) = E{( X E{ X})( Y E{ Y})} = E{ XY} E{ X} E{ Y} = 0 It also holds that: The normalized quantity: ρ XY Cov( X, Y ) = σ σ X Y Cov( X, Y ) σ Xσ Y is termed the correlation coefficient between the random variables X,Y. It follows that for two uncorrelated r.v. s the correlation coefficient is zero. The correlation coefficient lies between and. 6

7 Vector random variables Last time we reviewed some basic probabilistic measures used for one random variable and for random processes/signals Often we need to describe probabilistically the properties of a set of random variables: random/stochastic vectors Example: We will treat many systems identification problems as a linear regression problem ε u S + y where c will be a set of parameters that will describe the system (e.g. the values of its impulse response h[n] or the coefficients of an autoregressive model with exogenous input a i i, b i i.e.: yt + ayt + + ayt n n = but + + but m m Due to the randomness in the noise term ε these parameters are also random, therefore we will treat them as a random vector and will describe them accordingly. gy In other words if we repeat the estimation procedure with different data records (and consequently noise samples), we will not get the same result 7for c! ( ) ( )... ( ) ( )... ( )

8 Vector random variables For a random vector X the probability distribution function is defined as: P( X x) Prob{ X x,..., Xn xn} = Pr ob{ X x} P( X ) = P(- ) = 0 X The corresponding probability density function is defined as: n P X( x) P( X x) x... xn p ( x ) d x X = Pr ob{ x X < x+ dx,..., xn Xn < xn + dxn} x x xn ' ' P ( x) = p ( x') dx' =... p ( x') dx... dx n and: X X X The marginal probability density function for the element x i is defined as: p ( x )... x p ( x,..., x x,..., x ) dx... dx dx dx = X Xi i i i i+ n i i+ n The joint probability distribution function between two random vectors X and Y is defined as: P ( x, y ) = Prob{ X x, Y y XY } Similarly, the joint probability density function between X and Y is defined as: x y x x ' ' ' ', ) ( ', ') ' '... n y ym XY xy = =... ( ', ')... n... XY x y x y XY x y m P ( p d d p dx dx dy dy 8

9 Vector random variables The expected value of a random vector Χ is defined dfi das: μ =Ε{ X} μi =... xip ( x,..., xn) dx... dx X n or equivalently, l using the marginal pdf for the element x i : μi = xp i X ( x) i i dx i The covariance matrix (dim: nxn) of a random vector X (dim: nx) is defined as: Cov ( X ) = Σ = E {( X μ )( X μ ) T } Diagonal terms Σii: equal to the variance of each vector element σi = E{( Xi μι ) } Non diagonal terms Σij: Covariance between the random vector elements xi and xj The covariance matrix is symmetric and positive semidefinite If the elements of the random vector are uncorrelated, the covariance matrix is diagonal (why?) 9

10 Vector random variables Similarly, the autocorrelation matrix of a random vector X is defined as: R = E{ XX T } = Σ μ T X μ X The covariance matrix (dim: nxm) between two random vectors X (dim: nx), Y (dim: mx) is also defined as: Cov{ XY, } = E{ XY T } Analogously to the case of random variables, two random vectors are termed independent if: p ( x, y) = p ( x) p ( y) XY X Y Two random vectors are termed uncorrelated if: T T E{ XY } = E{ X} E{ Y } Two random vectors are termed orthogonal if: n T E { XY } = E { X Y } = 0 i= i i 0

11 The multidimensional normal distribution A random vector X of dimension Νx is said to follow a multidimensional normal (Gaussian) distribution if the corresponding joint pdf of its elements is: px x x N x μ Σ x μ N ( π ) Σ X N( μσ, ) Τ Σ (,..., ) = exp ( ) ( ) / / Mean value μ Covariance matrix Σ symmetric and positive semidefinite Diagonal elements: Variance of each σ i ( ) element x i ( ) Non diagonal elements: Covariance between xi και xj ( E{( xi μi )( xj μj)} ) N If the elements of X are independent Σ p ( x,..., x ) = p ( x ) = : Σ diagonal Ellipsoid with principal axes determined by the eigenvalues, eigenvectors of Σ σ 0 0 = i i= 0 0 σ Ν N xi μ i = exp N / ( π) σ... σ Ν i = σ i X N X i Multidimensional central limit theorem: vector sum of large number of mutually independent N dimensional r.v. s approaches N dimensional normal distribution

12 The two dimensional normal distribution For N=, if ρ=cov{x,x }/σ σ (correlation coefficient, ρ<): For x, x uncorrelated ρ=0, the principal axes are parallel to the x, x axes. In this case. Therefore, Gaussian (normally) distributed random variables that are uncorrelated are also independent. This is not true in general for other distributions. For ρ=, the distribution is reduced to a one dimensional distribution For Σ=σ Ι, circular contours Any linear transformation of X follows a normal distribution as well, i.e. if p(x)~n(μ,σ) then for Y=A T X we have p(y)~n(a T μ, A T ΣA)

13 The two dimensional normal distribution For increasing ρ 3

14 Sample statistics In practice, the probability distributions of random variables/vectors or random signals are not available. Therefore we can not use the definitions to estimate statistical quantities such as the mean or variance, e.g.: If we have Ν samples {x i } i=,,n of a random variable Χ, we can estimate the mean and variance as the sample mean and sample variance: N N ˆ x = μ = Χ xi ˆ σ N = ( xi x) i= N σ Χ i= These are not the only ways to obtain the estimates. How do we judge if an estimator is good or bad? Eti Estimator t properties An estimator is termed unbiased if its expected value is equal to the true value of the parameter that is estimated, i.e.: An estimator is termed consistent if it converges to its true value for N, i.e.: or equivalently: For the estimate of the mean, if the samples are independent identically distributed (i.i.d.): Ex { } = μ (unbiased) σ X E {( x μx ) } = (consistent) N For the estimate of the variance: N N E{ ˆ σ } σ (biased). Therefore, an unbiased estimate is: ˆ σ Χ = = Χ ( xi x) Χ N t= N It can be also shown that this estimate is consistent

15 Stationary random signals We saw that t for stationary tti random signals, their statistical ttiti properties (mean and variance, correlation/covariance functions) are independent of the time lag t, i.e.: μx mean ϕxx ( τ) = Extxt { ( ) ( + τ)} autocorrelation function ϕxy ( τ) = Extyt { ( ) ( + τ)} cross correlation function γ xx ( τ) = E{( x( t) μx )( x( t+ τ) μx )} = ϕxx ( τ) μx autocovariance function γ xy( τ ) = E{( x( t) μx)( y( t+ τ ) μy)} = ϕxy( τ ) μμ x y cross covariance function For two uncorrelated random processes the cross covariance function is zero: γ xy ( τ ) = 0 The autocorrelation coefficient function is defined as: ϕxx ( τ) ϕxx ( τ) ρxx ( τ) = = ϕxx (0) σx Similarly, the cross correlation coefficient function is: γ xy ( τ ) γ xy ( τ ) ρxy ( τ) = = γ xx (0) γ (0) σ xσ yy y These functions lie between and For ergodic random signals we saw that statistical properties may be calculated as time averages. Therefore: T μx = lim T xtdt ( ) T 0 T ϕxx ( τ) = lim T xtxt ( ) ( ) dt T + τ 0 T ϕxy ( τ) = lim T xt ( ) yt ( + τ) dt T 0

16 Stationary random signals These quantities can be estimated from finite samples in a similar way as before, i.e.: N ˆ μ x = xt () N t= N ˆ ϕxx ( τ) = xnxn ( ) ( + τ) N n= N ˆ ϕxy ( τ) = xny ( ) ( n + τ) N n= It can be shown that these estimates are unbiased and consistent for most random processes, e.g. Gaussian random processes, which are defined as random processes for which the random samples x(τ) follow a multidimensional normal (Gaussian) distribution

17 Examples Autocorrelation (Matlab xcorr)

18 Examples Cross correlation

19 Applications of cross correlation: Estimation of pure time delay

20 Stochastic signals and LTI systems Let the input of a DT LTI system with impulse response h[n] be a wide sense stationary (WSS) signal. The output is: x(t) h(τ) y(t) The input signal is characterized by its mean value and its autocorrelation function What are the corresponding quantities for the output? Since x(t) is WSS: The mean value of the output is: 0 where H(e j0 ) is the frequency response of the system (i.e. the Discrete Time Fourier transform DTFT of its impulse response) evaluated at ω=0: ( j ω He ) = h( τ ) e n= jωτ Therefore, the mean value of y is independent of t also. The value of the frequency response at ω=0 is termed DC gain

21 Stochastic signals and LTI systems The autocorrelation function of the output is: But x(t) is WSS, i.e.: therefore: The output of an LTI system to a WSS random signal is also WSS. By substituting where is the autocorrelation sequence of the deterministic signal h (τ) and is equal to Therefore, the autocorrelation function of the output is the convolution between the autocorrelation function of the input and the deterministic autocorrelation of h[n]

22 Stochastic signals in the frequency domain How do we describe a stochastic signal in the frequency domain? Fourier transform of the autocorrelation function: Power spectrum x(t) () h(τ) y(t) ) Similarly, we define as the DTFTs of respectively. For zero mean random signals: Why power spectrum? From the definition: we can see that the area under the spectrum Φ xx from π to π is proportional to the mean power ofthe signal. For simplicity, we often write: where is the power spectrum or the power spectrum density of the signal For real random signals, therefore the power spectrum is real and an even function of ω, i.e. : Note: In CT the power spectrum is defined as Φ = jωτ xx ( ω) ϕxx ( τ) e dτ

23 Stochastic signals in the frequency domain Similarly the cross power density spectrum between two random signals is defined as the DTFT of the cross correlation function between them The cross power density spectrum is generally x(t) () y(t) ) a complex number and: h(τ) We saw that: Therefore in the frequency domain (convolution property of the DTFT): but Therefore: The output power spectrum is equal to the input power spectrum multiplied by the squared magnitude of the frequency response of the system 3

24 White noise signals White noise signal: Defined as a random signal with zero mean, for which any two samples are independent. The autocorrelation function of a white noise signal is a Dirac delta function: ϕxx ( τ) = Ext { ( + τ) xt ( )} = σxδ( τ) The spectrum of a white noise signal is flat and contains all frequencies (from to ) - analogy with white light: Φ xx( ω) = σ x The mean power of a white noise signal is: White noise is an ideal signal and it exhibits very desirable properties for systems identification (more to follow) If in addition the samples of the white noise signal follow a normal distribution: Gaussian white noise (Matlab: randn) 4

25 Stochastic signals in the frequency domain Example: If Η is an ideal bandpass filter: x(t) H(ω) y(t) What happens if the input signal is white noise with autocorrelation function? Example: Let the input of an LTI system with frequency response is a zero mean white noise signal with. The output spectrum is: 5

26 Stochastic signals and LTI systems The cross correlation between the input and output is: x(t) h(τ) y(t) i.e. the convolution between the input autocorrelation and the inpulse response of the system Direct consequence: Τhe input/output cross spectrum is given by: 6

27 Stochastic signals and LTI systems Example: Let a white noise signal be the input to an x(t) H(ω) y(t) ideal lowpass filter with cutoff frequency ω c Output power spectrum: Output autocorrelation: Mean power of the output: 7

28 Stochastic signals and LTI systems Overall, we have the following relations for an LTI system being driven by a random input Time domain Frequency domain ϕ ( τ) = h( τ)* ϕ ( τ) Φ ( ω) =Η( ω) Φ ( ω) xy xx ϕ ( τ ) = h ( τ)* ϕ ( τ )* h( τ) = ϕ ( τ )* C ( τ ) Φ ( ω ) = Η ( ω ) Φ ( ω ) yy xx xx hh Mean, power, variance μ = μ H (0) y x π {( ( )) } = yy (0) = ( ) xx ( ) π Η Φ π E yt ϕ ω ω dω σ = ϕ (0) μ y yy y Note: All these relations are also valid for continuous time systems (proofs analogous instead of convolution sum >convolution integral) We will use these relations for nonparametric systems identification in the time and frequency domain xy yy xx xx 8

Statistical signal processing

Statistical signal processing Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable

More information

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.

More information

16.584: Random (Stochastic) Processes

16.584: Random (Stochastic) Processes 1 16.584: Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable

More information

3F1 Random Processes Examples Paper (for all 6 lectures)

3F1 Random Processes Examples Paper (for all 6 lectures) 3F Random Processes Examples Paper (for all 6 lectures). Three factories make the same electrical component. Factory A supplies half of the total number of components to the central depot, while factories

More information

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace

More information

Stochastic Processes

Stochastic Processes Elements of Lecture II Hamid R. Rabiee with thanks to Ali Jalali Overview Reading Assignment Chapter 9 of textbook Further Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A First Course in Stochastic

More information

Probability Space. J. McNames Portland State University ECE 538/638 Stochastic Signals Ver

Probability Space. J. McNames Portland State University ECE 538/638 Stochastic Signals Ver Stochastic Signals Overview Definitions Second order statistics Stationarity and ergodicity Random signal variability Power spectral density Linear systems with stationary inputs Random signal memory Correlation

More information

EAS 305 Random Processes Viewgraph 1 of 10. Random Processes

EAS 305 Random Processes Viewgraph 1 of 10. Random Processes EAS 305 Random Processes Viewgraph 1 of 10 Definitions: Random Processes A random process is a family of random variables indexed by a parameter t T, where T is called the index set λ i Experiment outcome

More information

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random

More information

EEM 409. Random Signals. Problem Set-2: (Power Spectral Density, LTI Systems with Random Inputs) Problem 1: Problem 2:

EEM 409. Random Signals. Problem Set-2: (Power Spectral Density, LTI Systems with Random Inputs) Problem 1: Problem 2: EEM 409 Random Signals Problem Set-2: (Power Spectral Density, LTI Systems with Random Inputs) Problem 1: Consider a random process of the form = + Problem 2: X(t) = b cos(2π t + ), where b is a constant,

More information

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides were adapted

More information

IV. Covariance Analysis

IV. Covariance Analysis IV. Covariance Analysis Autocovariance Remember that when a stochastic process has time values that are interdependent, then we can characterize that interdependency by computing the autocovariance function.

More information

Introduction to Probability and Stochastic Processes I

Introduction to Probability and Stochastic Processes I Introduction to Probability and Stochastic Processes I Lecture 3 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark Slides

More information

A Probability Review

A Probability Review A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in

More information

SRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS

SRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS 1. Define random process? The sample space composed of functions of time is called a random process. 2. Define Stationary process? If a random process is divided

More information

Chapter 2 Random Processes

Chapter 2 Random Processes Chapter 2 Random Processes 21 Introduction We saw in Section 111 on page 10 that many systems are best studied using the concept of random variables where the outcome of a random experiment was associated

More information

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl. E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,

More information

5 Operations on Multiple Random Variables

5 Operations on Multiple Random Variables EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y

More information

Fig 1: Stationary and Non Stationary Time Series

Fig 1: Stationary and Non Stationary Time Series Module 23 Independence and Stationarity Objective: To introduce the concepts of Statistical Independence, Stationarity and its types w.r.to random processes. This module also presents the concept of Ergodicity.

More information

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes

More information

Definition of a Stochastic Process

Definition of a Stochastic Process Definition of a Stochastic Process Balu Santhanam Dept. of E.C.E., University of New Mexico Fax: 505 277 8298 bsanthan@unm.edu August 26, 2018 Balu Santhanam (UNM) August 26, 2018 1 / 20 Overview 1 Stochastic

More information

Introduction to Probability and Stocastic Processes - Part I

Introduction to Probability and Stocastic Processes - Part I Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark

More information

14 - Gaussian Stochastic Processes

14 - Gaussian Stochastic Processes 14-1 Gaussian Stochastic Processes S. Lall, Stanford 211.2.24.1 14 - Gaussian Stochastic Processes Linear systems driven by IID noise Evolution of mean and covariance Example: mass-spring system Steady-state

More information

EE538 Final Exam Fall :20 pm -5:20 pm PHYS 223 Dec. 17, Cover Sheet

EE538 Final Exam Fall :20 pm -5:20 pm PHYS 223 Dec. 17, Cover Sheet EE538 Final Exam Fall 005 3:0 pm -5:0 pm PHYS 3 Dec. 17, 005 Cover Sheet Test Duration: 10 minutes. Open Book but Closed Notes. Calculators ARE allowed!! This test contains five problems. Each of the five

More information

Lecture 4: FT Pairs, Random Signals and z-transform

Lecture 4: FT Pairs, Random Signals and z-transform EE518 Digital Signal Processing University of Washington Autumn 2001 Dept. of Electrical Engineering Lecture 4: T Pairs, Rom Signals z-transform Wed., Oct. 10, 2001 Prof: J. Bilmes

More information

Massachusetts Institute of Technology

Massachusetts Institute of Technology Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.011: Introduction to Communication, Control and Signal Processing QUIZ, April 1, 010 QUESTION BOOKLET Your

More information

ECE534, Spring 2018: Solutions for Problem Set #5

ECE534, Spring 2018: Solutions for Problem Set #5 ECE534, Spring 08: s for Problem Set #5 Mean Value and Autocorrelation Functions Consider a random process X(t) such that (i) X(t) ± (ii) The number of zero crossings, N(t), in the interval (0, t) is described

More information

Random signals II. ÚPGM FIT VUT Brno,

Random signals II. ÚPGM FIT VUT Brno, Random signals II. Jan Černocký ÚPGM FIT VUT Brno, cernocky@fit.vutbr.cz 1 Temporal estimate of autocorrelation coefficients for ergodic discrete-time random process. ˆR[k] = 1 N N 1 n=0 x[n]x[n + k],

More information

Gaussian, Markov and stationary processes

Gaussian, Markov and stationary processes Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November

More information

for valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I

for valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I Code: 15A04304 R15 B.Tech II Year I Semester (R15) Regular Examinations November/December 016 PROBABILITY THEY & STOCHASTIC PROCESSES (Electronics and Communication Engineering) Time: 3 hours Max. Marks:

More information

System Identification & Parameter Estimation

System Identification & Parameter Estimation System Identification & Parameter Estimation Wb3: SIPE lecture Correlation functions in time & frequency domain Alfred C. Schouten, Dept. of Biomechanical Engineering (BMechE), Fac. 3mE // Delft University

More information

Lecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process

Lecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process Lecture Notes 7 Stationary Random Processes Strict-Sense and Wide-Sense Stationarity Autocorrelation Function of a Stationary Process Power Spectral Density Continuity and Integration of Random Processes

More information

Problems on Discrete & Continuous R.Vs

Problems on Discrete & Continuous R.Vs 013 SUBJECT NAME SUBJECT CODE MATERIAL NAME MATERIAL CODE : Probability & Random Process : MA 61 : University Questions : SKMA1004 Name of the Student: Branch: Unit I (Random Variables) Problems on Discrete

More information

Stochastic Process II Dr.-Ing. Sudchai Boonto

Stochastic Process II Dr.-Ing. Sudchai Boonto Dr-Ing Sudchai Boonto Department of Control System and Instrumentation Engineering King Mongkuts Unniversity of Technology Thonburi Thailand Random process Consider a random experiment specified by the

More information

Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes

Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes Klaus Witrisal witrisal@tugraz.at Signal Processing and Speech Communication Laboratory www.spsc.tugraz.at Graz University of

More information

UCSD ECE 153 Handout #46 Prof. Young-Han Kim Thursday, June 5, Solutions to Homework Set #8 (Prepared by TA Fatemeh Arbabjolfaei)

UCSD ECE 153 Handout #46 Prof. Young-Han Kim Thursday, June 5, Solutions to Homework Set #8 (Prepared by TA Fatemeh Arbabjolfaei) UCSD ECE 53 Handout #46 Prof. Young-Han Kim Thursday, June 5, 04 Solutions to Homework Set #8 (Prepared by TA Fatemeh Arbabjolfaei). Discrete-time Wiener process. Let Z n, n 0 be a discrete time white

More information

x. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).

x. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ). .8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics

More information

UCSD ECE153 Handout #40 Prof. Young-Han Kim Thursday, May 29, Homework Set #8 Due: Thursday, June 5, 2011

UCSD ECE153 Handout #40 Prof. Young-Han Kim Thursday, May 29, Homework Set #8 Due: Thursday, June 5, 2011 UCSD ECE53 Handout #40 Prof. Young-Han Kim Thursday, May 9, 04 Homework Set #8 Due: Thursday, June 5, 0. Discrete-time Wiener process. Let Z n, n 0 be a discrete time white Gaussian noise (WGN) process,

More information

Physics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester

Physics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester Physics 403 Parameter Estimation, Correlations, and Error Bars Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Best Estimates and Reliability

More information

Modern Navigation. Thomas Herring

Modern Navigation. Thomas Herring 12.215 Modern Navigation Thomas Herring Estimation methods Review of last class Restrict to basically linear estimation problems (also non-linear problems that are nearly linear) Restrict to parametric,

More information

EE482: Digital Signal Processing Applications

EE482: Digital Signal Processing Applications Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 11 Adaptive Filtering 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/

More information

Chapter 4 Random process. 4.1 Random process

Chapter 4 Random process. 4.1 Random process Random processes - Chapter 4 Random process 1 Random processes Chapter 4 Random process 4.1 Random process 4.1 Random process Random processes - Chapter 4 Random process 2 Random process Random process,

More information

Name of the Student: Problems on Discrete & Continuous R.Vs

Name of the Student: Problems on Discrete & Continuous R.Vs Engineering Mathematics 05 SUBJECT NAME : Probability & Random Process SUBJECT CODE : MA6 MATERIAL NAME : University Questions MATERIAL CODE : JM08AM004 REGULATION : R008 UPDATED ON : Nov-Dec 04 (Scan

More information

Probability and Statistics for Final Year Engineering Students

Probability and Statistics for Final Year Engineering Students Probability and Statistics for Final Year Engineering Students By Yoni Nazarathy, Last Updated: May 24, 2011. Lecture 6p: Spectral Density, Passing Random Processes through LTI Systems, Filtering Terms

More information

Discrete time processes

Discrete time processes Discrete time processes Predictions are difficult. Especially about the future Mark Twain. Florian Herzog 2013 Modeling observed data When we model observed (realized) data, we encounter usually the following

More information

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering Stochastic Processes and Linear Algebra Recap Slides Stochastic processes and variables XX tt 0 = XX xx nn (tt) xx 2 (tt) XX tt XX

More information

Multivariate Random Variable

Multivariate Random Variable Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate

More information

EE538 Final Exam Fall 2007 Mon, Dec 10, 8-10 am RHPH 127 Dec. 10, Cover Sheet

EE538 Final Exam Fall 2007 Mon, Dec 10, 8-10 am RHPH 127 Dec. 10, Cover Sheet EE538 Final Exam Fall 2007 Mon, Dec 10, 8-10 am RHPH 127 Dec. 10, 2007 Cover Sheet Test Duration: 120 minutes. Open Book but Closed Notes. Calculators allowed!! This test contains five problems. Each of

More information

Signals and Spectra (1A) Young Won Lim 11/26/12

Signals and Spectra (1A) Young Won Lim 11/26/12 Signals and Spectra (A) Copyright (c) 202 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version.2 or any later

More information

Properties of the Autocorrelation Function

Properties of the Autocorrelation Function Properties of the Autocorrelation Function I The autocorrelation function of a (real-valued) random process satisfies the following properties: 1. R X (t, t) 0 2. R X (t, u) =R X (u, t) (symmetry) 3. R

More information

Discussion Section #2, 31 Jan 2014

Discussion Section #2, 31 Jan 2014 Discussion Section #2, 31 Jan 2014 Lillian Ratliff 1 Unit Impulse The unit impulse (Dirac delta) has the following properties: { 0, t 0 δ(t) =, t = 0 ε ε δ(t) = 1 Remark 1. Important!: An ordinary function

More information

2 Quick Review of Continuous Random Variables

2 Quick Review of Continuous Random Variables CALIFORNIA INSTITUTE OF TECHNOLOGY Control and Dynamical Systems CDS b R. M. Murray Stochastic Systems 8 January 27 Reading: This set of lectures provides a brief introduction to stochastic systems. Friedland,

More information

UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, Practice Final Examination (Winter 2017)

UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, Practice Final Examination (Winter 2017) UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, 208 Practice Final Examination (Winter 207) There are 6 problems, each problem with multiple parts. Your answer should be as clear and readable

More information

Properties of LTI Systems

Properties of LTI Systems Properties of LTI Systems Properties of Continuous Time LTI Systems Systems with or without memory: A system is memory less if its output at any time depends only on the value of the input at that same

More information

where r n = dn+1 x(t)

where r n = dn+1 x(t) Random Variables Overview Probability Random variables Transforms of pdfs Moments and cumulants Useful distributions Random vectors Linear transformations of random vectors The multivariate normal distribution

More information

LTI Systems (Continuous & Discrete) - Basics

LTI Systems (Continuous & Discrete) - Basics LTI Systems (Continuous & Discrete) - Basics 1. A system with an input x(t) and output y(t) is described by the relation: y(t) = t. x(t). This system is (a) linear and time-invariant (b) linear and time-varying

More information

Atmospheric Flight Dynamics Example Exam 2 Solutions

Atmospheric Flight Dynamics Example Exam 2 Solutions Atmospheric Flight Dynamics Example Exam Solutions 1 Question Given the autocovariance function, C x x (τ) = 1 cos(πτ) (1.1) of stochastic variable x. Calculate the autospectrum S x x (ω). NOTE Assume

More information

ECE 636: Systems identification

ECE 636: Systems identification ECE 636: Systems identification Lectures 7 8 onparametric identification (continued) Important distributions: chi square, t distribution, F distribution Sampling distributions ib i Sample mean If the variance

More information

Continuous Random Variables

Continuous Random Variables 1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables

More information

Adaptive Filter Theory

Adaptive Filter Theory 0 Adaptive Filter heory Sung Ho Cho Hanyang University Seoul, Korea (Office) +8--0-0390 (Mobile) +8-10-541-5178 dragon@hanyang.ac.kr able of Contents 1 Wiener Filters Gradient Search by Steepest Descent

More information

Multiple Random Variables

Multiple Random Variables Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An

More information

ECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen. D. van Alphen 1

ECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen. D. van Alphen 1 ECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen D. van Alphen 1 Lecture 10 Overview Part 1 Review of Lecture 9 Continuing: Systems with Random Inputs More about Poisson RV s Intro. to Poisson Processes

More information

Stochastic Processes. A stochastic process is a function of two variables:

Stochastic Processes. A stochastic process is a function of two variables: Stochastic Processes Stochastic: from Greek stochastikos, proceeding by guesswork, literally, skillful in aiming. A stochastic process is simply a collection of random variables labelled by some parameter:

More information

Gaussian Basics Random Processes Filtering of Random Processes Signal Space Concepts

Gaussian Basics Random Processes Filtering of Random Processes Signal Space Concepts White Gaussian Noise I Definition: A (real-valued) random process X t is called white Gaussian Noise if I X t is Gaussian for each time instance t I Mean: m X (t) =0 for all t I Autocorrelation function:

More information

Stochastic Processes. Chapter Definitions

Stochastic Processes. Chapter Definitions Chapter 4 Stochastic Processes Clearly data assimilation schemes such as Optimal Interpolation are crucially dependent on the estimates of background and observation error statistics. Yet, we don t know

More information

CCNY. BME I5100: Biomedical Signal Processing. Stochastic Processes. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Stochastic Processes. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Stochastic Processes Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

ECE-340, Spring 2015 Review Questions

ECE-340, Spring 2015 Review Questions ECE-340, Spring 2015 Review Questions 1. Suppose that there are two categories of eggs: large eggs and small eggs, occurring with probabilities 0.7 and 0.3, respectively. For a large egg, the probabilities

More information

Solutions. Number of Problems: 10

Solutions. Number of Problems: 10 Final Exam February 9th, 2 Signals & Systems (5-575-) Prof. R. D Andrea Solutions Exam Duration: 5 minutes Number of Problems: Permitted aids: One double-sided A4 sheet. Questions can be answered in English

More information

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d)

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d) Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d) Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides

More information

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows. Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage

More information

Deterministic. Deterministic data are those can be described by an explicit mathematical relationship

Deterministic. Deterministic data are those can be described by an explicit mathematical relationship Random data Deterministic Deterministic data are those can be described by an explicit mathematical relationship Deterministic x(t) =X cos r! k m t Non deterministic There is no way to predict an exact

More information

Expectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Expectation. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,

More information

16.584: Random Vectors

16.584: Random Vectors 1 16.584: Random Vectors Define X : (X 1, X 2,..X n ) T : n-dimensional Random Vector X 1 : X(t 1 ): May correspond to samples/measurements Generalize definition of PDF: F X (x) = P[X 1 x 1, X 2 x 2,...X

More information

Homework 3 (Stochastic Processes)

Homework 3 (Stochastic Processes) In the name of GOD. Sharif University of Technology Stochastic Processes CE 695 Dr. H.R. Rabiee Homework 3 (Stochastic Processes). Explain why each of the following is NOT a valid autocorrrelation function:

More information

5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors

5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we

More information

Lecture 13: Discrete Time Fourier Transform (DTFT)

Lecture 13: Discrete Time Fourier Transform (DTFT) Lecture 13: Discrete Time Fourier Transform (DTFT) ECE 401: Signal and Image Analysis University of Illinois 3/9/2017 1 Sampled Systems Review 2 DTFT and Convolution 3 Inverse DTFT 4 Ideal Lowpass Filter

More information

Econometría 2: Análisis de series de Tiempo

Econometría 2: Análisis de series de Tiempo Econometría 2: Análisis de series de Tiempo Karoll GOMEZ kgomezp@unal.edu.co http://karollgomez.wordpress.com Segundo semestre 2016 II. Basic definitions A time series is a set of observations X t, each

More information

Random Process. Random Process. Random Process. Introduction to Random Processes

Random Process. Random Process. Random Process. Introduction to Random Processes Random Process A random variable is a function X(e) that maps the set of experiment outcomes to the set of numbers. A random process is a rule that maps every outcome e of an experiment to a function X(t,

More information

Part IV Stochastic Image Analysis 1 Contents IV Stochastic Image Analysis 1 7 Introduction to Stochastic Processes 4 7.1 Probability................................... 5 7.1.1 Random events and subjective

More information

LECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity.

LECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity. LECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity. Important points of Lecture 1: A time series {X t } is a series of observations taken sequentially over time: x t is an observation

More information

Probability and Statistics

Probability and Statistics Probability and Statistics 1 Contents some stochastic processes Stationary Stochastic Processes 2 4. Some Stochastic Processes 4.1 Bernoulli process 4.2 Binomial process 4.3 Sine wave process 4.4 Random-telegraph

More information

P 1.5 X 4.5 / X 2 and (iii) The smallest value of n for

P 1.5 X 4.5 / X 2 and (iii) The smallest value of n for DHANALAKSHMI COLLEGE OF ENEINEERING, CHENNAI DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING MA645 PROBABILITY AND RANDOM PROCESS UNIT I : RANDOM VARIABLES PART B (6 MARKS). A random variable X

More information

Consider the joint probability, P(x,y), shown as the contours in the figure above. P(x) is given by the integral of P(x,y) over all values of y.

Consider the joint probability, P(x,y), shown as the contours in the figure above. P(x) is given by the integral of P(x,y) over all values of y. ATMO/OPTI 656b Spring 009 Bayesian Retrievals Note: This follows the discussion in Chapter of Rogers (000) As we have seen, the problem with the nadir viewing emission measurements is they do not contain

More information

ELEC546 Review of Information Theory

ELEC546 Review of Information Theory ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random

More information

PROBABILITY AND RANDOM PROCESSESS

PROBABILITY AND RANDOM PROCESSESS PROBABILITY AND RANDOM PROCESSESS SOLUTIONS TO UNIVERSITY QUESTION PAPER YEAR : JUNE 2014 CODE NO : 6074 /M PREPARED BY: D.B.V.RAVISANKAR ASSOCIATE PROFESSOR IT DEPARTMENT MVSR ENGINEERING COLLEGE, NADERGUL

More information

Chapter 6. Random Processes

Chapter 6. Random Processes Chapter 6 Random Processes Random Process A random process is a time-varying function that assigns the outcome of a random experiment to each time instant: X(t). For a fixed (sample path): a random process

More information

Expectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda

Expectation. DS GA 1002 Probability and Statistics for Data Science.   Carlos Fernandez-Granda Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,

More information

ECE Homework Set 3

ECE Homework Set 3 ECE 450 1 Homework Set 3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3

More information

Massachusetts Institute of Technology

Massachusetts Institute of Technology Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.11: Introduction to Communication, Control and Signal Processing QUIZ 1, March 16, 21 QUESTION BOOKLET

More information

Module 3. Convolution. Aim

Module 3. Convolution. Aim Module Convolution Digital Signal Processing. Slide 4. Aim How to perform convolution in real-time systems efficiently? Is convolution in time domain equivalent to multiplication of the transformed sequence?

More information

Stochastic process for macro

Stochastic process for macro Stochastic process for macro Tianxiao Zheng SAIF 1. Stochastic process The state of a system {X t } evolves probabilistically in time. The joint probability distribution is given by Pr(X t1, t 1 ; X t2,

More information

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else ECE 450 Homework #3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3 4 5

More information

If we want to analyze experimental or simulated data we might encounter the following tasks:

If we want to analyze experimental or simulated data we might encounter the following tasks: Chapter 1 Introduction If we want to analyze experimental or simulated data we might encounter the following tasks: Characterization of the source of the signal and diagnosis Studying dependencies Prediction

More information

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science : Discrete-Time Signal Processing

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science : Discrete-Time Signal Processing Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.34: Discrete-Time Signal Processing OpenCourseWare 006 ecture 8 Periodogram Reading: Sections 0.6 and 0.7

More information

TSKS01 Digital Communication Lecture 1

TSKS01 Digital Communication Lecture 1 TSKS01 Digital Communication Lecture 1 Introduction, Repetition, and Noise Modeling Emil Björnson Department of Electrical Engineering (ISY) Division of Communication Systems Emil Björnson Course Director

More information

Review of Probability

Review of Probability Review of robabilit robabilit Theor: Man techniques in speech processing require the manipulation of probabilities and statistics. The two principal application areas we will encounter are: Statistical

More information

Chapter 5 Frequency Domain Analysis of Systems

Chapter 5 Frequency Domain Analysis of Systems Chapter 5 Frequency Domain Analysis of Systems CT, LTI Systems Consider the following CT LTI system: xt () ht () yt () Assumption: the impulse response h(t) is absolutely integrable, i.e., ht ( ) dt< (this

More information

Each problem is worth 25 points, and you may solve the problems in any order.

Each problem is worth 25 points, and you may solve the problems in any order. EE 120: Signals & Systems Department of Electrical Engineering and Computer Sciences University of California, Berkeley Midterm Exam #2 April 11, 2016, 2:10-4:00pm Instructions: There are four questions

More information

EE482: Digital Signal Processing Applications

EE482: Digital Signal Processing Applications Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 11 Adaptive Filtering 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/

More information

3. Probability and Statistics

3. Probability and Statistics FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important

More information