ECE 636: Systems identification
|
|
- Russell Lawson
- 6 years ago
- Views:
Transcription
1 ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain
2 υ ε x S z + y Experimental I/O data > Selection of model type >Selection of criterion >Calculation of model parameters >Model validation
3 Selection of model type: two main model categories Nonparametric (black box): Linear systems impulse response/ frequency response M yn ( ) = hmxn ( ) ( m) Ye jω ( ) He jω j ( ) Xe ω = ( ) m = 0 n Nonlinear systems: Volterra series: Parametric (grey box): Linear systems linear differential/difference equations d y() t dy() t du() t a + a + y() t = b + b0u() t dt dt dt ayn ( ) + ayn ( ) + yn ( ) = bun ( ) + bun ( ) 0 Nonlinear systems: Nonlinear difference equations, block models 3
4 Curve fitting Model complexity, amount of data, criterion of fit: important! wˆ arg min E( w) N = w N E( w) = tk g( xk, w) Ν k =
5 Review of random variable basics Gaussian random variables, central limit theorem Random signals Correlation and covariance functions Stationarity (weak/wide sense and strict), ergodicity Note: Only stationary signals can be ergodic 5
6 Independent/Uncorrelated random variables Independent random variables: pxy (, ) = px ( ) py ( ) In the general case, for N random variables: = N N i= = px (,..., X ) px ( ) Uncorrelated random variables: E{ XY} = E{ X} E{ Y} i IfXYareuncorrelated X,Y uncorrelated, thecovariance betweenthemis them zero: Cov( X, Y ) = E{( X E{ X})( Y E{ Y})} = E{ XY} E{ X} E{ Y} = 0 It also holds that: The normalized quantity: ρ XY Cov( X, Y ) = σ σ X Y Cov( X, Y ) σ Xσ Y is termed the correlation coefficient between the random variables X,Y. It follows that for two uncorrelated r.v. s the correlation coefficient is zero. The correlation coefficient lies between and. 6
7 Vector random variables Last time we reviewed some basic probabilistic measures used for one random variable and for random processes/signals Often we need to describe probabilistically the properties of a set of random variables: random/stochastic vectors Example: We will treat many systems identification problems as a linear regression problem ε u S + y where c will be a set of parameters that will describe the system (e.g. the values of its impulse response h[n] or the coefficients of an autoregressive model with exogenous input a i i, b i i.e.: yt + ayt + + ayt n n = but + + but m m Due to the randomness in the noise term ε these parameters are also random, therefore we will treat them as a random vector and will describe them accordingly. gy In other words if we repeat the estimation procedure with different data records (and consequently noise samples), we will not get the same result 7for c! ( ) ( )... ( ) ( )... ( )
8 Vector random variables For a random vector X the probability distribution function is defined as: P( X x) Prob{ X x,..., Xn xn} = Pr ob{ X x} P( X ) = P(- ) = 0 X The corresponding probability density function is defined as: n P X( x) P( X x) x... xn p ( x ) d x X = Pr ob{ x X < x+ dx,..., xn Xn < xn + dxn} x x xn ' ' P ( x) = p ( x') dx' =... p ( x') dx... dx n and: X X X The marginal probability density function for the element x i is defined as: p ( x )... x p ( x,..., x x,..., x ) dx... dx dx dx = X Xi i i i i+ n i i+ n The joint probability distribution function between two random vectors X and Y is defined as: P ( x, y ) = Prob{ X x, Y y XY } Similarly, the joint probability density function between X and Y is defined as: x y x x ' ' ' ', ) ( ', ') ' '... n y ym XY xy = =... ( ', ')... n... XY x y x y XY x y m P ( p d d p dx dx dy dy 8
9 Vector random variables The expected value of a random vector Χ is defined dfi das: μ =Ε{ X} μi =... xip ( x,..., xn) dx... dx X n or equivalently, l using the marginal pdf for the element x i : μi = xp i X ( x) i i dx i The covariance matrix (dim: nxn) of a random vector X (dim: nx) is defined as: Cov ( X ) = Σ = E {( X μ )( X μ ) T } Diagonal terms Σii: equal to the variance of each vector element σi = E{( Xi μι ) } Non diagonal terms Σij: Covariance between the random vector elements xi and xj The covariance matrix is symmetric and positive semidefinite If the elements of the random vector are uncorrelated, the covariance matrix is diagonal (why?) 9
10 Vector random variables Similarly, the autocorrelation matrix of a random vector X is defined as: R = E{ XX T } = Σ μ T X μ X The covariance matrix (dim: nxm) between two random vectors X (dim: nx), Y (dim: mx) is also defined as: Cov{ XY, } = E{ XY T } Analogously to the case of random variables, two random vectors are termed independent if: p ( x, y) = p ( x) p ( y) XY X Y Two random vectors are termed uncorrelated if: T T E{ XY } = E{ X} E{ Y } Two random vectors are termed orthogonal if: n T E { XY } = E { X Y } = 0 i= i i 0
11 The multidimensional normal distribution A random vector X of dimension Νx is said to follow a multidimensional normal (Gaussian) distribution if the corresponding joint pdf of its elements is: px x x N x μ Σ x μ N ( π ) Σ X N( μσ, ) Τ Σ (,..., ) = exp ( ) ( ) / / Mean value μ Covariance matrix Σ symmetric and positive semidefinite Diagonal elements: Variance of each σ i ( ) element x i ( ) Non diagonal elements: Covariance between xi και xj ( E{( xi μi )( xj μj)} ) N If the elements of X are independent Σ p ( x,..., x ) = p ( x ) = : Σ diagonal Ellipsoid with principal axes determined by the eigenvalues, eigenvectors of Σ σ 0 0 = i i= 0 0 σ Ν N xi μ i = exp N / ( π) σ... σ Ν i = σ i X N X i Multidimensional central limit theorem: vector sum of large number of mutually independent N dimensional r.v. s approaches N dimensional normal distribution
12 The two dimensional normal distribution For N=, if ρ=cov{x,x }/σ σ (correlation coefficient, ρ<): For x, x uncorrelated ρ=0, the principal axes are parallel to the x, x axes. In this case. Therefore, Gaussian (normally) distributed random variables that are uncorrelated are also independent. This is not true in general for other distributions. For ρ=, the distribution is reduced to a one dimensional distribution For Σ=σ Ι, circular contours Any linear transformation of X follows a normal distribution as well, i.e. if p(x)~n(μ,σ) then for Y=A T X we have p(y)~n(a T μ, A T ΣA)
13 The two dimensional normal distribution For increasing ρ 3
14 Sample statistics In practice, the probability distributions of random variables/vectors or random signals are not available. Therefore we can not use the definitions to estimate statistical quantities such as the mean or variance, e.g.: If we have Ν samples {x i } i=,,n of a random variable Χ, we can estimate the mean and variance as the sample mean and sample variance: N N ˆ x = μ = Χ xi ˆ σ N = ( xi x) i= N σ Χ i= These are not the only ways to obtain the estimates. How do we judge if an estimator is good or bad? Eti Estimator t properties An estimator is termed unbiased if its expected value is equal to the true value of the parameter that is estimated, i.e.: An estimator is termed consistent if it converges to its true value for N, i.e.: or equivalently: For the estimate of the mean, if the samples are independent identically distributed (i.i.d.): Ex { } = μ (unbiased) σ X E {( x μx ) } = (consistent) N For the estimate of the variance: N N E{ ˆ σ } σ (biased). Therefore, an unbiased estimate is: ˆ σ Χ = = Χ ( xi x) Χ N t= N It can be also shown that this estimate is consistent
15 Stationary random signals We saw that t for stationary tti random signals, their statistical ttiti properties (mean and variance, correlation/covariance functions) are independent of the time lag t, i.e.: μx mean ϕxx ( τ) = Extxt { ( ) ( + τ)} autocorrelation function ϕxy ( τ) = Extyt { ( ) ( + τ)} cross correlation function γ xx ( τ) = E{( x( t) μx )( x( t+ τ) μx )} = ϕxx ( τ) μx autocovariance function γ xy( τ ) = E{( x( t) μx)( y( t+ τ ) μy)} = ϕxy( τ ) μμ x y cross covariance function For two uncorrelated random processes the cross covariance function is zero: γ xy ( τ ) = 0 The autocorrelation coefficient function is defined as: ϕxx ( τ) ϕxx ( τ) ρxx ( τ) = = ϕxx (0) σx Similarly, the cross correlation coefficient function is: γ xy ( τ ) γ xy ( τ ) ρxy ( τ) = = γ xx (0) γ (0) σ xσ yy y These functions lie between and For ergodic random signals we saw that statistical properties may be calculated as time averages. Therefore: T μx = lim T xtdt ( ) T 0 T ϕxx ( τ) = lim T xtxt ( ) ( ) dt T + τ 0 T ϕxy ( τ) = lim T xt ( ) yt ( + τ) dt T 0
16 Stationary random signals These quantities can be estimated from finite samples in a similar way as before, i.e.: N ˆ μ x = xt () N t= N ˆ ϕxx ( τ) = xnxn ( ) ( + τ) N n= N ˆ ϕxy ( τ) = xny ( ) ( n + τ) N n= It can be shown that these estimates are unbiased and consistent for most random processes, e.g. Gaussian random processes, which are defined as random processes for which the random samples x(τ) follow a multidimensional normal (Gaussian) distribution
17 Examples Autocorrelation (Matlab xcorr)
18 Examples Cross correlation
19 Applications of cross correlation: Estimation of pure time delay
20 Stochastic signals and LTI systems Let the input of a DT LTI system with impulse response h[n] be a wide sense stationary (WSS) signal. The output is: x(t) h(τ) y(t) The input signal is characterized by its mean value and its autocorrelation function What are the corresponding quantities for the output? Since x(t) is WSS: The mean value of the output is: 0 where H(e j0 ) is the frequency response of the system (i.e. the Discrete Time Fourier transform DTFT of its impulse response) evaluated at ω=0: ( j ω He ) = h( τ ) e n= jωτ Therefore, the mean value of y is independent of t also. The value of the frequency response at ω=0 is termed DC gain
21 Stochastic signals and LTI systems The autocorrelation function of the output is: But x(t) is WSS, i.e.: therefore: The output of an LTI system to a WSS random signal is also WSS. By substituting where is the autocorrelation sequence of the deterministic signal h (τ) and is equal to Therefore, the autocorrelation function of the output is the convolution between the autocorrelation function of the input and the deterministic autocorrelation of h[n]
22 Stochastic signals in the frequency domain How do we describe a stochastic signal in the frequency domain? Fourier transform of the autocorrelation function: Power spectrum x(t) () h(τ) y(t) ) Similarly, we define as the DTFTs of respectively. For zero mean random signals: Why power spectrum? From the definition: we can see that the area under the spectrum Φ xx from π to π is proportional to the mean power ofthe signal. For simplicity, we often write: where is the power spectrum or the power spectrum density of the signal For real random signals, therefore the power spectrum is real and an even function of ω, i.e. : Note: In CT the power spectrum is defined as Φ = jωτ xx ( ω) ϕxx ( τ) e dτ
23 Stochastic signals in the frequency domain Similarly the cross power density spectrum between two random signals is defined as the DTFT of the cross correlation function between them The cross power density spectrum is generally x(t) () y(t) ) a complex number and: h(τ) We saw that: Therefore in the frequency domain (convolution property of the DTFT): but Therefore: The output power spectrum is equal to the input power spectrum multiplied by the squared magnitude of the frequency response of the system 3
24 White noise signals White noise signal: Defined as a random signal with zero mean, for which any two samples are independent. The autocorrelation function of a white noise signal is a Dirac delta function: ϕxx ( τ) = Ext { ( + τ) xt ( )} = σxδ( τ) The spectrum of a white noise signal is flat and contains all frequencies (from to ) - analogy with white light: Φ xx( ω) = σ x The mean power of a white noise signal is: White noise is an ideal signal and it exhibits very desirable properties for systems identification (more to follow) If in addition the samples of the white noise signal follow a normal distribution: Gaussian white noise (Matlab: randn) 4
25 Stochastic signals in the frequency domain Example: If Η is an ideal bandpass filter: x(t) H(ω) y(t) What happens if the input signal is white noise with autocorrelation function? Example: Let the input of an LTI system with frequency response is a zero mean white noise signal with. The output spectrum is: 5
26 Stochastic signals and LTI systems The cross correlation between the input and output is: x(t) h(τ) y(t) i.e. the convolution between the input autocorrelation and the inpulse response of the system Direct consequence: Τhe input/output cross spectrum is given by: 6
27 Stochastic signals and LTI systems Example: Let a white noise signal be the input to an x(t) H(ω) y(t) ideal lowpass filter with cutoff frequency ω c Output power spectrum: Output autocorrelation: Mean power of the output: 7
28 Stochastic signals and LTI systems Overall, we have the following relations for an LTI system being driven by a random input Time domain Frequency domain ϕ ( τ) = h( τ)* ϕ ( τ) Φ ( ω) =Η( ω) Φ ( ω) xy xx ϕ ( τ ) = h ( τ)* ϕ ( τ )* h( τ) = ϕ ( τ )* C ( τ ) Φ ( ω ) = Η ( ω ) Φ ( ω ) yy xx xx hh Mean, power, variance μ = μ H (0) y x π {( ( )) } = yy (0) = ( ) xx ( ) π Η Φ π E yt ϕ ω ω dω σ = ϕ (0) μ y yy y Note: All these relations are also valid for continuous time systems (proofs analogous instead of convolution sum >convolution integral) We will use these relations for nonparametric systems identification in the time and frequency domain xy yy xx xx 8
Statistical signal processing
Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable
More informationStochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno
Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.
More information16.584: Random (Stochastic) Processes
1 16.584: Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable
More information3F1 Random Processes Examples Paper (for all 6 lectures)
3F Random Processes Examples Paper (for all 6 lectures). Three factories make the same electrical component. Factory A supplies half of the total number of components to the central depot, while factories
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationStochastic Processes
Elements of Lecture II Hamid R. Rabiee with thanks to Ali Jalali Overview Reading Assignment Chapter 9 of textbook Further Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A First Course in Stochastic
More informationProbability Space. J. McNames Portland State University ECE 538/638 Stochastic Signals Ver
Stochastic Signals Overview Definitions Second order statistics Stationarity and ergodicity Random signal variability Power spectral density Linear systems with stationary inputs Random signal memory Correlation
More informationEAS 305 Random Processes Viewgraph 1 of 10. Random Processes
EAS 305 Random Processes Viewgraph 1 of 10 Definitions: Random Processes A random process is a family of random variables indexed by a parameter t T, where T is called the index set λ i Experiment outcome
More informationELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random
More informationEEM 409. Random Signals. Problem Set-2: (Power Spectral Density, LTI Systems with Random Inputs) Problem 1: Problem 2:
EEM 409 Random Signals Problem Set-2: (Power Spectral Density, LTI Systems with Random Inputs) Problem 1: Consider a random process of the form = + Problem 2: X(t) = b cos(2π t + ), where b is a constant,
More informationParametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes
Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides were adapted
More informationIV. Covariance Analysis
IV. Covariance Analysis Autocovariance Remember that when a stochastic process has time values that are interdependent, then we can characterize that interdependency by computing the autocovariance function.
More informationIntroduction to Probability and Stochastic Processes I
Introduction to Probability and Stochastic Processes I Lecture 3 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark Slides
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationSRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS
UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS 1. Define random process? The sample space composed of functions of time is called a random process. 2. Define Stationary process? If a random process is divided
More informationChapter 2 Random Processes
Chapter 2 Random Processes 21 Introduction We saw in Section 111 on page 10 that many systems are best studied using the concept of random variables where the outcome of a random experiment was associated
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More information5 Operations on Multiple Random Variables
EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y
More informationFig 1: Stationary and Non Stationary Time Series
Module 23 Independence and Stationarity Objective: To introduce the concepts of Statistical Independence, Stationarity and its types w.r.to random processes. This module also presents the concept of Ergodicity.
More informationENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University
ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes
More informationDefinition of a Stochastic Process
Definition of a Stochastic Process Balu Santhanam Dept. of E.C.E., University of New Mexico Fax: 505 277 8298 bsanthan@unm.edu August 26, 2018 Balu Santhanam (UNM) August 26, 2018 1 / 20 Overview 1 Stochastic
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More information14 - Gaussian Stochastic Processes
14-1 Gaussian Stochastic Processes S. Lall, Stanford 211.2.24.1 14 - Gaussian Stochastic Processes Linear systems driven by IID noise Evolution of mean and covariance Example: mass-spring system Steady-state
More informationEE538 Final Exam Fall :20 pm -5:20 pm PHYS 223 Dec. 17, Cover Sheet
EE538 Final Exam Fall 005 3:0 pm -5:0 pm PHYS 3 Dec. 17, 005 Cover Sheet Test Duration: 10 minutes. Open Book but Closed Notes. Calculators ARE allowed!! This test contains five problems. Each of the five
More informationLecture 4: FT Pairs, Random Signals and z-transform
EE518 Digital Signal Processing University of Washington Autumn 2001 Dept. of Electrical Engineering Lecture 4: T Pairs, Rom Signals z-transform Wed., Oct. 10, 2001 Prof: J. Bilmes
More informationMassachusetts Institute of Technology
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.011: Introduction to Communication, Control and Signal Processing QUIZ, April 1, 010 QUESTION BOOKLET Your
More informationECE534, Spring 2018: Solutions for Problem Set #5
ECE534, Spring 08: s for Problem Set #5 Mean Value and Autocorrelation Functions Consider a random process X(t) such that (i) X(t) ± (ii) The number of zero crossings, N(t), in the interval (0, t) is described
More informationRandom signals II. ÚPGM FIT VUT Brno,
Random signals II. Jan Černocký ÚPGM FIT VUT Brno, cernocky@fit.vutbr.cz 1 Temporal estimate of autocorrelation coefficients for ergodic discrete-time random process. ˆR[k] = 1 N N 1 n=0 x[n]x[n + k],
More informationGaussian, Markov and stationary processes
Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November
More informationfor valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I
Code: 15A04304 R15 B.Tech II Year I Semester (R15) Regular Examinations November/December 016 PROBABILITY THEY & STOCHASTIC PROCESSES (Electronics and Communication Engineering) Time: 3 hours Max. Marks:
More informationSystem Identification & Parameter Estimation
System Identification & Parameter Estimation Wb3: SIPE lecture Correlation functions in time & frequency domain Alfred C. Schouten, Dept. of Biomechanical Engineering (BMechE), Fac. 3mE // Delft University
More informationLecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process
Lecture Notes 7 Stationary Random Processes Strict-Sense and Wide-Sense Stationarity Autocorrelation Function of a Stationary Process Power Spectral Density Continuity and Integration of Random Processes
More informationProblems on Discrete & Continuous R.Vs
013 SUBJECT NAME SUBJECT CODE MATERIAL NAME MATERIAL CODE : Probability & Random Process : MA 61 : University Questions : SKMA1004 Name of the Student: Branch: Unit I (Random Variables) Problems on Discrete
More informationStochastic Process II Dr.-Ing. Sudchai Boonto
Dr-Ing Sudchai Boonto Department of Control System and Instrumentation Engineering King Mongkuts Unniversity of Technology Thonburi Thailand Random process Consider a random experiment specified by the
More informationFundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes
Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes Klaus Witrisal witrisal@tugraz.at Signal Processing and Speech Communication Laboratory www.spsc.tugraz.at Graz University of
More informationUCSD ECE 153 Handout #46 Prof. Young-Han Kim Thursday, June 5, Solutions to Homework Set #8 (Prepared by TA Fatemeh Arbabjolfaei)
UCSD ECE 53 Handout #46 Prof. Young-Han Kim Thursday, June 5, 04 Solutions to Homework Set #8 (Prepared by TA Fatemeh Arbabjolfaei). Discrete-time Wiener process. Let Z n, n 0 be a discrete time white
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationUCSD ECE153 Handout #40 Prof. Young-Han Kim Thursday, May 29, Homework Set #8 Due: Thursday, June 5, 2011
UCSD ECE53 Handout #40 Prof. Young-Han Kim Thursday, May 9, 04 Homework Set #8 Due: Thursday, June 5, 0. Discrete-time Wiener process. Let Z n, n 0 be a discrete time white Gaussian noise (WGN) process,
More informationPhysics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester
Physics 403 Parameter Estimation, Correlations, and Error Bars Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Best Estimates and Reliability
More informationModern Navigation. Thomas Herring
12.215 Modern Navigation Thomas Herring Estimation methods Review of last class Restrict to basically linear estimation problems (also non-linear problems that are nearly linear) Restrict to parametric,
More informationEE482: Digital Signal Processing Applications
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 11 Adaptive Filtering 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/
More informationChapter 4 Random process. 4.1 Random process
Random processes - Chapter 4 Random process 1 Random processes Chapter 4 Random process 4.1 Random process 4.1 Random process Random processes - Chapter 4 Random process 2 Random process Random process,
More informationName of the Student: Problems on Discrete & Continuous R.Vs
Engineering Mathematics 05 SUBJECT NAME : Probability & Random Process SUBJECT CODE : MA6 MATERIAL NAME : University Questions MATERIAL CODE : JM08AM004 REGULATION : R008 UPDATED ON : Nov-Dec 04 (Scan
More informationProbability and Statistics for Final Year Engineering Students
Probability and Statistics for Final Year Engineering Students By Yoni Nazarathy, Last Updated: May 24, 2011. Lecture 6p: Spectral Density, Passing Random Processes through LTI Systems, Filtering Terms
More informationDiscrete time processes
Discrete time processes Predictions are difficult. Especially about the future Mark Twain. Florian Herzog 2013 Modeling observed data When we model observed (realized) data, we encounter usually the following
More informationProf. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides
Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering Stochastic Processes and Linear Algebra Recap Slides Stochastic processes and variables XX tt 0 = XX xx nn (tt) xx 2 (tt) XX tt XX
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationEE538 Final Exam Fall 2007 Mon, Dec 10, 8-10 am RHPH 127 Dec. 10, Cover Sheet
EE538 Final Exam Fall 2007 Mon, Dec 10, 8-10 am RHPH 127 Dec. 10, 2007 Cover Sheet Test Duration: 120 minutes. Open Book but Closed Notes. Calculators allowed!! This test contains five problems. Each of
More informationSignals and Spectra (1A) Young Won Lim 11/26/12
Signals and Spectra (A) Copyright (c) 202 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version.2 or any later
More informationProperties of the Autocorrelation Function
Properties of the Autocorrelation Function I The autocorrelation function of a (real-valued) random process satisfies the following properties: 1. R X (t, t) 0 2. R X (t, u) =R X (u, t) (symmetry) 3. R
More informationDiscussion Section #2, 31 Jan 2014
Discussion Section #2, 31 Jan 2014 Lillian Ratliff 1 Unit Impulse The unit impulse (Dirac delta) has the following properties: { 0, t 0 δ(t) =, t = 0 ε ε δ(t) = 1 Remark 1. Important!: An ordinary function
More information2 Quick Review of Continuous Random Variables
CALIFORNIA INSTITUTE OF TECHNOLOGY Control and Dynamical Systems CDS b R. M. Murray Stochastic Systems 8 January 27 Reading: This set of lectures provides a brief introduction to stochastic systems. Friedland,
More informationUCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, Practice Final Examination (Winter 2017)
UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, 208 Practice Final Examination (Winter 207) There are 6 problems, each problem with multiple parts. Your answer should be as clear and readable
More informationProperties of LTI Systems
Properties of LTI Systems Properties of Continuous Time LTI Systems Systems with or without memory: A system is memory less if its output at any time depends only on the value of the input at that same
More informationwhere r n = dn+1 x(t)
Random Variables Overview Probability Random variables Transforms of pdfs Moments and cumulants Useful distributions Random vectors Linear transformations of random vectors The multivariate normal distribution
More informationLTI Systems (Continuous & Discrete) - Basics
LTI Systems (Continuous & Discrete) - Basics 1. A system with an input x(t) and output y(t) is described by the relation: y(t) = t. x(t). This system is (a) linear and time-invariant (b) linear and time-varying
More informationAtmospheric Flight Dynamics Example Exam 2 Solutions
Atmospheric Flight Dynamics Example Exam Solutions 1 Question Given the autocovariance function, C x x (τ) = 1 cos(πτ) (1.1) of stochastic variable x. Calculate the autospectrum S x x (ω). NOTE Assume
More informationECE 636: Systems identification
ECE 636: Systems identification Lectures 7 8 onparametric identification (continued) Important distributions: chi square, t distribution, F distribution Sampling distributions ib i Sample mean If the variance
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationAdaptive Filter Theory
0 Adaptive Filter heory Sung Ho Cho Hanyang University Seoul, Korea (Office) +8--0-0390 (Mobile) +8-10-541-5178 dragon@hanyang.ac.kr able of Contents 1 Wiener Filters Gradient Search by Steepest Descent
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen. D. van Alphen 1
ECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen D. van Alphen 1 Lecture 10 Overview Part 1 Review of Lecture 9 Continuing: Systems with Random Inputs More about Poisson RV s Intro. to Poisson Processes
More informationStochastic Processes. A stochastic process is a function of two variables:
Stochastic Processes Stochastic: from Greek stochastikos, proceeding by guesswork, literally, skillful in aiming. A stochastic process is simply a collection of random variables labelled by some parameter:
More informationGaussian Basics Random Processes Filtering of Random Processes Signal Space Concepts
White Gaussian Noise I Definition: A (real-valued) random process X t is called white Gaussian Noise if I X t is Gaussian for each time instance t I Mean: m X (t) =0 for all t I Autocorrelation function:
More informationStochastic Processes. Chapter Definitions
Chapter 4 Stochastic Processes Clearly data assimilation schemes such as Optimal Interpolation are crucially dependent on the estimates of background and observation error statistics. Yet, we don t know
More informationCCNY. BME I5100: Biomedical Signal Processing. Stochastic Processes. Lucas C. Parra Biomedical Engineering Department City College of New York
BME I5100: Biomedical Signal Processing Stochastic Processes Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not
More informationECE-340, Spring 2015 Review Questions
ECE-340, Spring 2015 Review Questions 1. Suppose that there are two categories of eggs: large eggs and small eggs, occurring with probabilities 0.7 and 0.3, respectively. For a large egg, the probabilities
More informationSolutions. Number of Problems: 10
Final Exam February 9th, 2 Signals & Systems (5-575-) Prof. R. D Andrea Solutions Exam Duration: 5 minutes Number of Problems: Permitted aids: One double-sided A4 sheet. Questions can be answered in English
More informationParametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d)
Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d) Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationDeterministic. Deterministic data are those can be described by an explicit mathematical relationship
Random data Deterministic Deterministic data are those can be described by an explicit mathematical relationship Deterministic x(t) =X cos r! k m t Non deterministic There is no way to predict an exact
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More information16.584: Random Vectors
1 16.584: Random Vectors Define X : (X 1, X 2,..X n ) T : n-dimensional Random Vector X 1 : X(t 1 ): May correspond to samples/measurements Generalize definition of PDF: F X (x) = P[X 1 x 1, X 2 x 2,...X
More informationHomework 3 (Stochastic Processes)
In the name of GOD. Sharif University of Technology Stochastic Processes CE 695 Dr. H.R. Rabiee Homework 3 (Stochastic Processes). Explain why each of the following is NOT a valid autocorrrelation function:
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationLecture 13: Discrete Time Fourier Transform (DTFT)
Lecture 13: Discrete Time Fourier Transform (DTFT) ECE 401: Signal and Image Analysis University of Illinois 3/9/2017 1 Sampled Systems Review 2 DTFT and Convolution 3 Inverse DTFT 4 Ideal Lowpass Filter
More informationEconometría 2: Análisis de series de Tiempo
Econometría 2: Análisis de series de Tiempo Karoll GOMEZ kgomezp@unal.edu.co http://karollgomez.wordpress.com Segundo semestre 2016 II. Basic definitions A time series is a set of observations X t, each
More informationRandom Process. Random Process. Random Process. Introduction to Random Processes
Random Process A random variable is a function X(e) that maps the set of experiment outcomes to the set of numbers. A random process is a rule that maps every outcome e of an experiment to a function X(t,
More informationPart IV Stochastic Image Analysis 1 Contents IV Stochastic Image Analysis 1 7 Introduction to Stochastic Processes 4 7.1 Probability................................... 5 7.1.1 Random events and subjective
More informationLECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity.
LECTURES 2-3 : Stochastic Processes, Autocorrelation function. Stationarity. Important points of Lecture 1: A time series {X t } is a series of observations taken sequentially over time: x t is an observation
More informationProbability and Statistics
Probability and Statistics 1 Contents some stochastic processes Stationary Stochastic Processes 2 4. Some Stochastic Processes 4.1 Bernoulli process 4.2 Binomial process 4.3 Sine wave process 4.4 Random-telegraph
More informationP 1.5 X 4.5 / X 2 and (iii) The smallest value of n for
DHANALAKSHMI COLLEGE OF ENEINEERING, CHENNAI DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING MA645 PROBABILITY AND RANDOM PROCESS UNIT I : RANDOM VARIABLES PART B (6 MARKS). A random variable X
More informationConsider the joint probability, P(x,y), shown as the contours in the figure above. P(x) is given by the integral of P(x,y) over all values of y.
ATMO/OPTI 656b Spring 009 Bayesian Retrievals Note: This follows the discussion in Chapter of Rogers (000) As we have seen, the problem with the nadir viewing emission measurements is they do not contain
More informationELEC546 Review of Information Theory
ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random
More informationPROBABILITY AND RANDOM PROCESSESS
PROBABILITY AND RANDOM PROCESSESS SOLUTIONS TO UNIVERSITY QUESTION PAPER YEAR : JUNE 2014 CODE NO : 6074 /M PREPARED BY: D.B.V.RAVISANKAR ASSOCIATE PROFESSOR IT DEPARTMENT MVSR ENGINEERING COLLEGE, NADERGUL
More informationChapter 6. Random Processes
Chapter 6 Random Processes Random Process A random process is a time-varying function that assigns the outcome of a random experiment to each time instant: X(t). For a fixed (sample path): a random process
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationECE Homework Set 3
ECE 450 1 Homework Set 3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3
More informationMassachusetts Institute of Technology
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.11: Introduction to Communication, Control and Signal Processing QUIZ 1, March 16, 21 QUESTION BOOKLET
More informationModule 3. Convolution. Aim
Module Convolution Digital Signal Processing. Slide 4. Aim How to perform convolution in real-time systems efficiently? Is convolution in time domain equivalent to multiplication of the transformed sequence?
More informationStochastic process for macro
Stochastic process for macro Tianxiao Zheng SAIF 1. Stochastic process The state of a system {X t } evolves probabilistically in time. The joint probability distribution is given by Pr(X t1, t 1 ; X t2,
More informationECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else
ECE 450 Homework #3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3 4 5
More informationIf we want to analyze experimental or simulated data we might encounter the following tasks:
Chapter 1 Introduction If we want to analyze experimental or simulated data we might encounter the following tasks: Characterization of the source of the signal and diagnosis Studying dependencies Prediction
More informationMassachusetts Institute of Technology Department of Electrical Engineering and Computer Science : Discrete-Time Signal Processing
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.34: Discrete-Time Signal Processing OpenCourseWare 006 ecture 8 Periodogram Reading: Sections 0.6 and 0.7
More informationTSKS01 Digital Communication Lecture 1
TSKS01 Digital Communication Lecture 1 Introduction, Repetition, and Noise Modeling Emil Björnson Department of Electrical Engineering (ISY) Division of Communication Systems Emil Björnson Course Director
More informationReview of Probability
Review of robabilit robabilit Theor: Man techniques in speech processing require the manipulation of probabilities and statistics. The two principal application areas we will encounter are: Statistical
More informationChapter 5 Frequency Domain Analysis of Systems
Chapter 5 Frequency Domain Analysis of Systems CT, LTI Systems Consider the following CT LTI system: xt () ht () yt () Assumption: the impulse response h(t) is absolutely integrable, i.e., ht ( ) dt< (this
More informationEach problem is worth 25 points, and you may solve the problems in any order.
EE 120: Signals & Systems Department of Electrical Engineering and Computer Sciences University of California, Berkeley Midterm Exam #2 April 11, 2016, 2:10-4:00pm Instructions: There are four questions
More informationEE482: Digital Signal Processing Applications
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 11 Adaptive Filtering 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More information