Model structure. Lecture Note #3 (Chap.6) Identification of time series model. ARMAX Models and Difference Equations
|
|
- Harry Hudson
- 6 years ago
- Views:
Transcription
1 System Modeling and Identification Lecture ote #3 (Chap.6) CHBE 70 Korea University Prof. Dae Ryoo Yang Model structure ime series Multivariable time series x [ ] x x xm Multidimensional time series (temporal+spatial) Model classification SISO/MIMO Linear/onlinear Deterministic/Stochastic For linear discrete-time systems Difference equation and ARMAX models ransfer function model State-space model - -3 Chap.6 Identification of ime- Series Model Identification of time series model Model structure Parametric estimation Least square method Excellent properties when disturbances are uncorrelated Otherwise, there may be systematic errors and bias More sophisticated methods are needed Handling correlated disturbances Extension of linear regression ARMAX Models and Difference Equations ARMAX (AutoRegressive Moving Average with exogenous input) model A( z ) y z d B( z ) u C( z ) w where d is time delay and A( z ) a z a z an z A nb B( z ) b bz b z bn z B nc Cz ( ) cz cz c z 0 with unnown parameters [ a a n b b n c c n w] ( ) (AR) d nc A B C Az y w y C( z ) w (MA or FIR) Az ( ) y z Bz ( ) u w (ARX) Az ( ) y Cz ( ) w (ARMA) Az y Bz u Cz w ( ) ( ) ( ) (ARIMAX or CARIMA) - Integrated Controlled -4
2 ARMAX: General model ARX: Controlled autoregressive model and Linear regression model when disturbance is measured AR: Model harmonics compounded with noise, and runcated impulse response (FIR) model ARMA: Model-based spectrum analysis ARIMAX: For nonzero means and drift or nonstationary disturbance cases Az y Bz u Cz w ( ) ( ) ( ) A ( z ) y ( ) ( )( ) B z u C z z w Integrated white noise: onstationary (random wal) ransfer Function Models ransfer function models * y H u( z) u H v( z) v ( E{ vv i j} v ij) Bz ( ) Cz ( ) Az ( ) y u w F( z ) D( z ) Bz ( ) y u (Output error (OE) model) v F( z ) Bz ( ) Cz ( ) y u (Box-Jenins (BJ) model) w F( z ) D( z ) OE model: o assumption on disturbance sequence {v } BJ model: Filtered white noise {w } sequence by C/D -5-7 Prediction Error Method (PEM) Methods to predict y based on previous data and the identified model PEM min E( y ( ) y ) min E ( ) It minimizes the variance of the prediction steps ahead of the output y where the prediction is based on the present data. y ( A) y Bu Cw ( Ay ) Bu ( Cw ) w ( A) y Bu ( C)( C ( Ay Bu )) w z w E{( y y ) } E{( z w y ) } E{( z y ) } E{ w} w Minimum attainable variance -6 Difference between output error and prediction error y ay bu (Output error method) y ay bu (Prediction error method) Output error relies more on the accuracy of future output modeling. Prediction error uses actual output. Output error identification is a nonlinear estimation problem. Prediction error identification is a linear estimation problem. Algorithm for OE identification. Least square identification to find initial estimate of F and B. M : F( z ) y B( z ) u v. Filter the data according to F F y y / F ( z ), u u / F ( z ) Prewhitening filter 3. Subsequence estimation of F and B from the model. F F M : F( z ) y B( z ) u v 4. Repeat -3 until the estimate converges. -8
3 Comparison of error models for identification -9 For ARMAX model A( z ) y d z B( z ) u C( z ) w log L( ) 0.5log(( ) det ) 0.5 ( ) ( ) v v y where y ay an y n bu dbu A A dn B Unnown v cv cn CvnC v [ y yn u ] A d udn v B vn C [ a an b b ] A n c c B nc he empirical lielihood function when v = v I ( v unnown) log L(, ) ( / )log( ) ( / ) log( ) (/ ) ( ) v v v ( /)log( ) ( /)log( v) (/ v) V( ) log L(, v) (/ v) V( ) 0 4 log L(, ) (/ ) ( ) ( / ) 0 v v V v v ( ( ( i V ( V ) i i ) i ) i ) v ( V ) 0 (/ V ) ( ) ( ( )) ( ) (ewton-raphson method) - Maximum-Lielihood Method Select estimate so that the observation Y is most probable. max py ( ) py ( ) Lielihood function Example 6.4 Y v ( Ev { } 0 and Evv { } 0) / Assume that pv ( ) (( ) det ) v exp( 0.5 vv v) / Lielihood function: pv ( ) (( ) det v) exp( 0.5( Y) v ( Y)) log L( ) log p( ) 0.5log(( ) det v) 0.5 v If the model is linear in parameters with normally distributed white noise, the maximum lielihood estimate is same as Marov estimate. Cramer-Rao lower bound log L log L log L Cov( ) E E Example 6.5 LS and ML identification S y ay bu w cw : For colored noise, ML identification performs better than LS. LS can estimates only a and b. Local minimum Example 6.6 Pseudolinear regression S: A( z ) y B( z ) u C( z ) w Estimate high order polynomials A and B by least squares he computed residual sequence { } yields a good approximation of white noise sequence {w }. Extend the regressor with { } and then estimate the polynomials of A, B and C using least squares identification. It is also called two-step linear regression. Fisher information matrix -0-3
4 Kalman Filter State-space model x x u v Ev { } 0, Ee { } 0 y Cx Du e E{ vv } R, E{ ee } R, P(0) E{ x0x0} R0 Optimal estimate of x based on the input-output data min J( x) E{( x x ) } Kalman filter (Kalman-Bucy filter) Kalman filter will minimize the above minimization when v and e are independent and normally distributed. x x u ( ) K y Cx K PC ( R CPC ) P P R PC ( R CPC ) CP Cases for time-varying parameters v Ev { } 0, Ee { } 0 y e K ( y ) K P( R P) P P R P ( R P ) P Excellent for time-varying systems R and R are important design parameter that should match the temporal variations of and the observation noise, respectively Derivation he prediction error x x x he prediction error dynamics x ( KCx ) v Ke he mean prediction error E{ x } ( KC) E{ x} he mean square prediction error Ex { x } E{[( KCx ) v Ke ][( KCx ) v Ke ] } ( KCExx ) { }( KC ) v KeK Let P E{ xx } and Q e CPC P P KCP PC K v KQK P P v PC Q CP ( K PC Q ) Q ( K PC Q ) Minimization of P + gives K PC ( e CPC ) P P v PC ( e CPC ) CP (Riccati equation) Instrumental Variable Method Correlation between the regressors and the prediction error leads to bias of the parameter estimates obtained from least-square solutions to the linear regression problem Replace regressor by some other variable Z: IV method In order to mae the estimator consistent EZv { } 0 ran( Z ) p z ( Z ) ZY z z z Cov( ) E{( )( ) } ( Z ) Z vz( Z) he instrumental variable abeshould oudbec chosen sothat they eyae are simultaneously uncorrelated with v and highly correlated with
5 Example 6.8 S: y 0.9y 0.uw 0.7w ( Ew { } 0, Ew { } w) Biased least-square estimate of parameters y u [ ab] ( ) Y[ ] Instrumental variable y u z az bu z u Z z ( Z ) Z Y [ ] z u Shows reduced bias 0 Example 6.9 u0 u For a choice of IV Z u u z ( Z ) Z Y [ ] 047] Gives very poor estimate. It might be difficult to choose appropriate instrumental variables. hus, an iterative procedure are usually used. -7 Some Aspects of Application Prefiltering, smoothing, prewhitening f f Y ( z) F( z ) Y( z), U ( z) F( z ) U( z) f f M : Az ( ) y Bz ( ) u ( v For periodic variation, use F(z 0 w) )= z d when d is the period of trend. Bias reduction rend elimination y ( y )/, u ( u )/ M : Az ( )( y y) Bz ( )( u u) v Differentiation i i of data M : Az ( ) y Bz ( ) u v M : Az ( ) y Bz ( ) u v It gives improved accuracy Offset estimation via an extra parameter M : Az ( ) y Bz ( ) u ( v w) 0 It introduces new noise correlation Extra parameter -9 Example 6.0 (he Yule-Waler equations) Consider the AR process S: A( z ) y w ; ( E{ w } 0, E{ w } ) w * Cyy ( ) E{ y y } E{( ai y i w ) y } ae i { y i y } E{ w y } i i ac ( ), 0 i yy i i w Cyy ( ) ac ( ), 0 i i yy i Choosing numbers M> n A and p>n A and [ y yn ] A z [ y yp]/ M ( p,, M p) Cyy ( i) Cyy ( i) Cyy ( i) a Cyy ( i) Cyy () i Cyy ( i) Cyy ( i ) a Cyy ( i ) Cyy ( i p) Cyy ( i p) Cyy ( i p ) an C ( ) A yy i p z ( Z ) ZY ( Z ) ( Z Y ) -8 Convergence and Consistency Convergence in L p p,, 0<p< p p lim E{ x x } 0 Convergence almost surely lim P{ x x, n, 0} n Convergence in probability P{ x x, 0} 0 Central limit theorem Let {x } be a sequence of independent random variables with common distribution function F with finite mean and variance. X has a limiting normal distribution with mean 0 and variance as. S dist If S x, then X ormal(0,) -0 5
6 Efficient estimate, E{( ) } E{( ) } for any other estimate Consistent estimate Probability limit lim E{( ) } 0 lim P {, 0} 0 plim Unbiased and asymptotically unbiased estimates E{ } (Unbiased estimate) lim E{ } (Asymptotically unbiased estimate) - 6
6.435, System Identification
SET 6 System Identification 6.435 Parametrized model structures One-step predictor Identifiability Munther A. Dahleh 1 Models of LTI Systems A complete model u = input y = output e = noise (with PDF).
More informationLecture Note #7 (Chap.11)
System Modeling and Identification Lecture Note #7 (Chap.) CBE 702 Korea University Prof. Dae Ryoo Yang Chap. Real-time Identification Real-time identification Supervision and tracing of time varying parameters
More informationEECE Adaptive Control
EECE 574 - Adaptive Control Basics of System Identification Guy Dumont Department of Electrical and Computer Engineering University of British Columbia January 2010 Guy Dumont (UBC) EECE574 - Basics of
More informationSGN Advanced Signal Processing: Lecture 8 Parameter estimation for AR and MA models. Model order selection
SG 21006 Advanced Signal Processing: Lecture 8 Parameter estimation for AR and MA models. Model order selection Ioan Tabus Department of Signal Processing Tampere University of Technology Finland 1 / 28
More informationCONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Prediction Error Methods - Torsten Söderström
PREDICTIO ERROR METHODS Torsten Söderström Department of Systems and Control, Information Technology, Uppsala University, Uppsala, Sweden Keywords: prediction error method, optimal prediction, identifiability,
More informationIdentification of ARX, OE, FIR models with the least squares method
Identification of ARX, OE, FIR models with the least squares method CHEM-E7145 Advanced Process Control Methods Lecture 2 Contents Identification of ARX model with the least squares minimizing the equation
More informationSystem Modeling and Identification CHBE 702 Korea University Prof. Dae Ryook Yang
System Modeling and Identification CHBE 702 Korea University Prof. Dae Ryook Yang 1-1 Course Description Emphases Delivering concepts and Practice Programming Identification Methods using Matlab Class
More informationLecture Note #6 (Chap.10)
System Modeling and Identification Lecture Note #6 (Chap.) CBE 7 Korea University Prof. Dae Ryoo Yang Chap. Model Approximation Model approximation Simplification, approximation and order reduction of
More informationMODEL PREDICTIVE CONTROL and optimization
MODEL PREDICTIVE CONTROL and optimization Lecture notes Model Predictive Control PhD., Associate professor David Di Ruscio System and Control Engineering Department of Technology Telemark University College
More informationEL1820 Modeling of Dynamical Systems
EL1820 Modeling of Dynamical Systems Lecture 9 - Parameter estimation in linear models Model structures Parameter estimation via prediction error minimization Properties of the estimate: bias and variance
More information12. Prediction Error Methods (PEM)
12. Prediction Error Methods (PEM) EE531 (Semester II, 2010) description optimal prediction Kalman filter statistical results computational aspects 12-1 Description idea: determine the model parameter
More informationCHBE507 LECTURE II MPC Revisited. Professor Dae Ryook Yang
CHBE507 LECURE II MPC Revisited Professor Dae Ryook Yang Fall 2013 Dept. of Chemical and Biological Engineering Korea University Korea University III -1 Process Models ransfer function models Fixed order
More informationRECURSIVE ESTIMATION AND KALMAN FILTERING
Chapter 3 RECURSIVE ESTIMATION AND KALMAN FILTERING 3. The Discrete Time Kalman Filter Consider the following estimation problem. Given the stochastic system with x k+ = Ax k + Gw k (3.) y k = Cx k + Hv
More informationf-domain expression for the limit model Combine: 5.12 Approximate Modelling What can be said about H(q, θ) G(q, θ ) H(q, θ ) with
5.2 Approximate Modelling What can be said about if S / M, and even G / G? G(q, ) H(q, ) f-domain expression for the limit model Combine: with ε(t, ) =H(q, ) [y(t) G(q, )u(t)] y(t) =G (q)u(t) v(t) We know
More information6.435, System Identification
System Identification 6.435 SET 3 Nonparametric Identification Munther A. Dahleh 1 Nonparametric Methods for System ID Time domain methods Impulse response Step response Correlation analysis / time Frequency
More information11. Further Issues in Using OLS with TS Data
11. Further Issues in Using OLS with TS Data With TS, including lags of the dependent variable often allow us to fit much better the variation in y Exact distribution theory is rarely available in TS applications,
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationEconometrics II - EXAM Answer each question in separate sheets in three hours
Econometrics II - EXAM Answer each question in separate sheets in three hours. Let u and u be jointly Gaussian and independent of z in all the equations. a Investigate the identification of the following
More informationEconomics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
More informationElements of Multivariate Time Series Analysis
Gregory C. Reinsel Elements of Multivariate Time Series Analysis Second Edition With 14 Figures Springer Contents Preface to the Second Edition Preface to the First Edition vii ix 1. Vector Time Series
More informationARIMA Modelling and Forecasting
ARIMA Modelling and Forecasting Economic time series often appear nonstationary, because of trends, seasonal patterns, cycles, etc. However, the differences may appear stationary. Δx t x t x t 1 (first
More informationBasic concepts in estimation
Basic concepts in estimation Random and nonrandom parameters Definitions of estimates ML Maimum Lielihood MAP Maimum A Posteriori LS Least Squares MMS Minimum Mean square rror Measures of quality of estimates
More informationEE482: Digital Signal Processing Applications
Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu EE482: Digital Signal Processing Applications Spring 2014 TTh 14:30-15:45 CBC C222 Lecture 11 Adaptive Filtering 14/03/04 http://www.ee.unlv.edu/~b1morris/ee482/
More informationParametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d)
Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes (cont d) Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides
More informationComputer Exercise 1 Estimation and Model Validation
Lund University Time Series Analysis Mathematical Statistics Fall 2018 Centre for Mathematical Sciences Computer Exercise 1 Estimation and Model Validation This computer exercise treats identification,
More informationChapter 9: Forecasting
Chapter 9: Forecasting One of the critical goals of time series analysis is to forecast (predict) the values of the time series at times in the future. When forecasting, we ideally should evaluate the
More informationTIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.
TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION
More informationTime Series Analysis
Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Input-Output systems The z-transform important issues
More informationLecture 7: Discrete-time Models. Modeling of Physical Systems. Preprocessing Experimental Data.
ISS0031 Modeling and Identification Lecture 7: Discrete-time Models. Modeling of Physical Systems. Preprocessing Experimental Data. Aleksei Tepljakov, Ph.D. October 21, 2015 Discrete-time Transfer Functions
More informationLinear Model Under General Variance Structure: Autocorrelation
Linear Model Under General Variance Structure: Autocorrelation A Definition of Autocorrelation In this section, we consider another special case of the model Y = X β + e, or y t = x t β + e t, t = 1,..,.
More informationTime Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference
More informationTime Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY PREFACE xiii 1 Difference Equations 1.1. First-Order Difference Equations 1 1.2. pth-order Difference Equations 7
More informationIDENTIFICATION OF A TWO-INPUT SYSTEM: VARIANCE ANALYSIS
IDENTIFICATION OF A TWO-INPUT SYSTEM: VARIANCE ANALYSIS M Gevers,1 L Mišković,2 D Bonvin A Karimi Center for Systems Engineering and Applied Mechanics (CESAME) Université Catholique de Louvain B-1348 Louvain-la-Neuve,
More informationSystem Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2012 F, FRI Uppsala University, Information Technology 30 Januari 2012 SI-2012 K. Pelckmans
More informationSystem Identification, Lecture 4
System Identification, Lecture 4 Kristiaan Pelckmans (IT/UU, 2338) Course code: 1RT880, Report code: 61800 - Spring 2016 F, FRI Uppsala University, Information Technology 13 April 2016 SI-2016 K. Pelckmans
More informationECE 636: Systems identification
ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain υ ε x S z + y Experimental
More informationTesting Linear Restrictions: cont.
Testing Linear Restrictions: cont. The F-statistic is closely connected with the R of the regression. In fact, if we are testing q linear restriction, can write the F-stastic as F = (R u R r)=q ( R u)=(n
More informationSystem Identification
System Identification Lecture : Statistical properties of parameter estimators, Instrumental variable methods Roy Smith 8--8. 8--8. Statistical basis for estimation methods Parametrised models: G Gp, zq,
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Financial Econometrics / 49
State-space Model Eduardo Rossi University of Pavia November 2013 Rossi State-space Model Financial Econometrics - 2013 1 / 49 Outline 1 Introduction 2 The Kalman filter 3 Forecast errors 4 State smoothing
More informationEL1820 Modeling of Dynamical Systems
EL1820 Modeling of Dynamical Systems Lecture 10 - System identification as a model building tool Experiment design Examination and prefiltering of data Model structure selection Model validation Lecture
More informationLecture Note #6 (Chap.10)
System Modeling and Identification Lecture Note #6 (Chap.) CBE 7 Korea University Prof. Dae Ryook Yang Chap. Model Approximation Model approximation Simplification, approximation and order reduction of
More information1. The Multivariate Classical Linear Regression Model
Business School, Brunel University MSc. EC550/5509 Modelling Financial Decisions and Markets/Introduction to Quantitative Methods Prof. Menelaos Karanasos (Room SS69, Tel. 08956584) Lecture Notes 5. The
More informationCBE495 LECTURE IV MODEL PREDICTIVE CONTROL
What is Model Predictive Control (MPC)? CBE495 LECTURE IV MODEL PREDICTIVE CONTROL Professor Dae Ryook Yang Fall 2013 Dept. of Chemical and Biological Engineering Korea University * Some parts are from
More informationAdvanced Process Control Tutorial Problem Set 2 Development of Control Relevant Models through System Identification
Advanced Process Control Tutorial Problem Set 2 Development of Control Relevant Models through System Identification 1. Consider the time series x(k) = β 1 + β 2 k + w(k) where β 1 and β 2 are known constants
More informationInstrumental Variables and Two-Stage Least Squares
Instrumental Variables and Two-Stage Least Squares Generalised Least Squares Professor Menelaos Karanasos December 2011 Generalised Least Squares: Assume that the postulated model is y = Xb + e, (1) where
More informationCramér-Rao Bounds for Estimation of Linear System Noise Covariances
Journal of Mechanical Engineering and Automation (): 6- DOI: 593/jjmea Cramér-Rao Bounds for Estimation of Linear System oise Covariances Peter Matiso * Vladimír Havlena Czech echnical University in Prague
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53
State-space Model Eduardo Rossi University of Pavia November 2014 Rossi State-space Model Fin. Econometrics - 2014 1 / 53 Outline 1 Motivation 2 Introduction 3 The Kalman filter 4 Forecast errors 5 State
More informationRoss Bettinger, Analytical Consultant, Seattle, WA
ABSTRACT DYNAMIC REGRESSION IN ARIMA MODELING Ross Bettinger, Analytical Consultant, Seattle, WA Box-Jenkins time series models that contain exogenous predictor variables are called dynamic regression
More informationDynamic Regression Models (Lect 15)
Dynamic Regression Models (Lect 15) Ragnar Nymoen University of Oslo 21 March 2013 1 / 17 HGL: Ch 9; BN: Kap 10 The HGL Ch 9 is a long chapter, and the testing for autocorrelation part we have already
More informationESTIMATION ALGORITHMS
ESTIMATIO ALGORITHMS Solving normal equations using QR-factorization on-linear optimization Two and multi-stage methods EM algorithm FEL 3201 Estimation Algorithms - 1 SOLVIG ORMAL EQUATIOS USIG QR FACTORIZATIO
More informationVariations. ECE 6540, Lecture 10 Maximum Likelihood Estimation
Variations ECE 6540, Lecture 10 Last Time BLUE (Best Linear Unbiased Estimator) Formulation Advantages Disadvantages 2 The BLUE A simplification Assume the estimator is a linear system For a single parameter
More informationWiener Filtering. EE264: Lecture 12
EE264: Lecture 2 Wiener Filtering In this lecture we will take a different view of filtering. Previously, we have depended on frequency-domain specifications to make some sort of LP/ BP/ HP/ BS filter,
More informationPure Random process Pure Random Process or White Noise Process: is a random process {X t, t 0} which has: { σ 2 if k = 0 0 if k 0
MODULE 9: STATIONARY PROCESSES 7 Lecture 2 Autoregressive Processes 1 Moving Average Process Pure Random process Pure Random Process or White Noise Process: is a random process X t, t 0} which has: E[X
More informationMultivariate ARMA Processes
LECTURE 8 Multivariate ARMA Processes A vector y(t) of n elements is said to follow an n-variate ARMA process of orders p and q if it satisfies the equation (1) A 0 y(t) + A 1 y(t 1) + + A p y(t p) = M
More informationTime Series Examples Sheet
Lent Term 2001 Richard Weber Time Series Examples Sheet This is the examples sheet for the M. Phil. course in Time Series. A copy can be found at: http://www.statslab.cam.ac.uk/~rrw1/timeseries/ Throughout,
More informationLeast-squares data fitting
EE263 Autumn 2015 S. Boyd and S. Lall Least-squares data fitting 1 Least-squares data fitting we are given: functions f 1,..., f n : S R, called regressors or basis functions data or measurements (s i,
More informationLECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
More informationStatistical Filtering and Control for AI and Robotics. Part II. Linear methods for regression & Kalman filtering
Statistical Filtering and Control for AI and Robotics Part II. Linear methods for regression & Kalman filtering Riccardo Muradore 1 / 66 Outline Linear Methods for Regression Gaussian filter Stochastic
More informationParameter Estimation
Parameter Estimation Consider a sample of observations on a random variable Y. his generates random variables: (y 1, y 2,, y ). A random sample is a sample (y 1, y 2,, y ) where the random variables y
More informationSystem Identification
System Identification Arun K. Tangirala Department of Chemical Engineering IIT Madras July 26, 2013 Module 6 Lecture 1 Arun K. Tangirala System Identification July 26, 2013 1 Objectives of this Module
More informationEEG- Signal Processing
Fatemeh Hadaeghi EEG- Signal Processing Lecture Notes for BSP, Chapter 5 Master Program Data Engineering 1 5 Introduction The complex patterns of neural activity, both in presence and absence of external
More informationLeast Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD
Least Squares Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 75A Winter 0 - UCSD (Unweighted) Least Squares Assume linearity in the unnown, deterministic model parameters Scalar, additive noise model: y f (
More informationNew Introduction to Multiple Time Series Analysis
Helmut Lütkepohl New Introduction to Multiple Time Series Analysis With 49 Figures and 36 Tables Springer Contents 1 Introduction 1 1.1 Objectives of Analyzing Multiple Time Series 1 1.2 Some Basics 2
More informationTHE TRANSLATION PLANES OF ORDER 49 AND THEIR AUTOMORPHISM GROUPS
MATHEMATICS OF COMPUTATION Volume 67, Number 223, July 1998, Pages 1207 1224 S 0025-5718(98)00961-2 THE TRANSLATION PLANES OF ORDER 49 AND THEIR AUTOMORPHISM GROUPS C. CHARNES AND U. DEMPWOLFF Abstract.
More informationChapter 3: Maximum Likelihood Theory
Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood
More informationEcon 623 Econometrics II Topic 2: Stationary Time Series
1 Introduction Econ 623 Econometrics II Topic 2: Stationary Time Series In the regression model we can model the error term as an autoregression AR(1) process. That is, we can use the past value of the
More informationBox-Jenkins ARIMA Advanced Time Series
Box-Jenkins ARIMA Advanced Time Series www.realoptionsvaluation.com ROV Technical Papers Series: Volume 25 Theory In This Issue 1. Learn about Risk Simulator s ARIMA and Auto ARIMA modules. 2. Find out
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationCONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Recursive Algorithms - Han-Fu Chen
CONROL SYSEMS, ROBOICS, AND AUOMAION - Vol. V - Recursive Algorithms - Han-Fu Chen RECURSIVE ALGORIHMS Han-Fu Chen Institute of Systems Science, Academy of Mathematics and Systems Science, Chinese Academy
More information1 EM algorithm: updating the mixing proportions {π k } ik are the posterior probabilities at the qth iteration of EM.
Université du Sud Toulon - Var Master Informatique Probabilistic Learning and Data Analysis TD: Model-based clustering by Faicel CHAMROUKHI Solution The aim of this practical wor is to show how the Classification
More informationOnline monitoring of MPC disturbance models using closed-loop data
Online monitoring of MPC disturbance models using closed-loop data Brian J. Odelson and James B. Rawlings Department of Chemical Engineering University of Wisconsin-Madison Online Optimization Based Identification
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationPractical Spectral Estimation
Digital Signal Processing/F.G. Meyer Lecture 4 Copyright 2015 François G. Meyer. All Rights Reserved. Practical Spectral Estimation 1 Introduction The goal of spectral estimation is to estimate how the
More informationIntroduction to Estimation Methods for Time Series models Lecture 2
Introduction to Estimation Methods for Time Series models Lecture 2 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 2 SNS Pisa 1 / 21 Estimators:
More informationStatistics 910, #5 1. Regression Methods
Statistics 910, #5 1 Overview Regression Methods 1. Idea: effects of dependence 2. Examples of estimation (in R) 3. Review of regression 4. Comparisons and relative efficiencies Idea Decomposition Well-known
More informationLecture 4a: ARMA Model
Lecture 4a: ARMA Model 1 2 Big Picture Most often our goal is to find a statistical model to describe real time series (estimation), and then predict the future (forecasting) One particularly popular model
More informationLecture 4: Types of errors. Bayesian regression models. Logistic regression
Lecture 4: Types of errors. Bayesian regression models. Logistic regression A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting more generally COMP-652 and ECSE-68, Lecture
More informationSystem Identification & Parameter Estimation
System Identiication & Parameter Estimation Wb30: SIPE Lecture 9: Physical Modeling, Model and Parameter Accuracy Erwin de Vlugt, Dept. o Biomechanical Engineering BMechE, Fac. 3mE April 6 00 Delt University
More informationClassical Decomposition Model Revisited: I
Classical Decomposition Model Revisited: I recall classical decomposition model for time series Y t, namely, Y t = m t + s t + W t, where m t is trend; s t is periodic with known period s (i.e., s t s
More information5 Transfer function modelling
MSc Further Time Series Analysis 5 Transfer function modelling 5.1 The model Consider the construction of a model for a time series (Y t ) whose values are influenced by the earlier values of a series
More informationLMS and eigenvalue spread 2. Lecture 3 1. LMS and eigenvalue spread 3. LMS and eigenvalue spread 4. χ(r) = λ max λ min. » 1 a. » b0 +b. b 0 a+b 1.
Lecture Lecture includes the following: Eigenvalue spread of R and its influence on the convergence speed for the LMS. Variants of the LMS: The Normalized LMS The Leaky LMS The Sign LMS The Echo Canceller
More informationEconomics 620, Lecture 4: The K-Variable Linear Model I. y 1 = + x 1 + " 1 y 2 = + x 2 + " 2 :::::::: :::::::: y N = + x N + " N
1 Economics 620, Lecture 4: The K-Variable Linear Model I Consider the system y 1 = + x 1 + " 1 y 2 = + x 2 + " 2 :::::::: :::::::: y N = + x N + " N or in matrix form y = X + " where y is N 1, X is N
More informationAdvanced Econometrics I
Lecture Notes Autumn 2010 Dr. Getinet Haile, University of Mannheim 1. Introduction Introduction & CLRM, Autumn Term 2010 1 What is econometrics? Econometrics = economic statistics economic theory mathematics
More informationp(z)
Chapter Statistics. Introduction This lecture is a quick review of basic statistical concepts; probabilities, mean, variance, covariance, correlation, linear regression, probability density functions and
More informationKalman Filter. Predict: Update: x k k 1 = F k x k 1 k 1 + B k u k P k k 1 = F k P k 1 k 1 F T k + Q
Kalman Filter Kalman Filter Predict: x k k 1 = F k x k 1 k 1 + B k u k P k k 1 = F k P k 1 k 1 F T k + Q Update: K = P k k 1 Hk T (H k P k k 1 Hk T + R) 1 x k k = x k k 1 + K(z k H k x k k 1 ) P k k =(I
More informationNonlinear Parameter Estimation for State-Space ARCH Models with Missing Observations
Nonlinear Parameter Estimation for State-Space ARCH Models with Missing Observations SEBASTIÁN OSSANDÓN Pontificia Universidad Católica de Valparaíso Instituto de Matemáticas Blanco Viel 596, Cerro Barón,
More informationSubspace Identification Methods
Czech Technical University in Prague Faculty of Electrical Engineering Department of Control Engineering Subspace Identification Methods Technical Report Ing. Pavel Trnka Supervisor: Prof. Ing. Vladimír
More information4 Derivations of the Discrete-Time Kalman Filter
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof N Shimkin 4 Derivations of the Discrete-Time
More informationLecture 11 FIR Filters
Lecture 11 FIR Filters Fundamentals of Digital Signal Processing Spring, 2012 Wei-Ta Chu 2012/4/12 1 The Unit Impulse Sequence Any sequence can be represented in this way. The equation is true if k ranges
More informationMarcel Dettling. Applied Time Series Analysis SS 2013 Week 05. ETH Zürich, March 18, Institute for Data Analysis and Process Design
Marcel Dettling Institute for Data Analysis and Process Design Zurich University of Applied Sciences marcel.dettling@zhaw.ch http://stat.ethz.ch/~dettling ETH Zürich, March 18, 2013 1 Basics of Modeling
More informationGeneralized Least Squares
Generalized Least Squares Upuntilnow,wehaveassumedthat Euu = ¾ I Now we generalize to let Euu = where is restricted to be a positive de nite, symmetric matrix ommon Examples Autoregressive Models AR()
More informationFree Meixner distributions and random matrices
Free Meixner distributions and random matrices Michael Anshelevich July 13, 2006 Some common distributions first... 1 Gaussian Negative binomial Gamma Pascal chi-square geometric exponential 1 2πt e x2
More informationIntroduction to Estimation Methods for Time Series models. Lecture 1
Introduction to Estimation Methods for Time Series models Lecture 1 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 1 SNS Pisa 1 / 19 Estimation
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationMaximum Likelihood Diffusive Source Localization Based on Binary Observations
Maximum Lielihood Diffusive Source Localization Based on Binary Observations Yoav Levinboo and an F. Wong Wireless Information Networing Group, University of Florida Gainesville, Florida 32611-6130, USA
More informationCALIFORNIA INSTITUTE OF TECHNOLOGY Control and Dynamical Systems. CDS 110b
CALIFORNIA INSTITUTE OF TECHNOLOGY Control and Dynamical Systems CDS 110b R. M. Murray Kalman Filters 14 January 2007 Reading: This set of lectures provides a brief introduction to Kalman filtering, following
More informationRECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK
RECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK TRNKA PAVEL AND HAVLENA VLADIMÍR Dept of Control Engineering, Czech Technical University, Technická 2, 166 27 Praha, Czech Republic mail:
More informationMatlab software tools for model identification and data analysis 10/11/2017 Prof. Marcello Farina
Matlab software tools for model identification and data analysis 10/11/2017 Prof. Marcello Farina Model Identification and Data Analysis (Academic year 2017-2018) Prof. Sergio Bittanti Outline Data generation
More informationMassachusetts Institute of Technology Department of Electrical Engineering and Computer Science : Discrete-Time Signal Processing
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.34: Discrete-Time Signal Processing OpenCourseWare 006 ecture 8 Periodogram Reading: Sections 0.6 and 0.7
More informationMultivariate Time Series Analysis and Its Applications [Tsay (2005), chapter 8]
1 Multivariate Time Series Analysis and Its Applications [Tsay (2005), chapter 8] Insights: Price movements in one market can spread easily and instantly to another market [economic globalization and internet
More information