Cramér-Rao Bounds for Estimation of Linear System Noise Covariances
|
|
- August Wade
- 5 years ago
- Views:
Transcription
1 Journal of Mechanical Engineering and Automation (): 6- DOI: 593/jjmea Cramér-Rao Bounds for Estimation of Linear System oise Covariances Peter Matiso * Vladimír Havlena Czech echnical University in Prague Faculty of Electrical Engineering Department of Control Engineering Abstract he performance of Kalman filter depends directly on the noise covariances which are usually not nown and need to be estimated Several estimation algorithms have been published in past decades but the measure of estimation quality is missing he Cramér-Rao bounds represent limitation of quality of parameter estimation that can be obtained from given data In this article he Cramér-Rao bounds for noise covariance estimation of linear time-invariant stochastic system will be derived wo different linear system models will be considered Further the performance of earlier published methods will be discussed according to the Cramér-Rao bounds he analogy between the Cramér-Rao bounds and the Riccati equation will be pointed out Keywords Cramér-Rao bounds Kalman Filter oise covariance estimation Introduction he Cramér-Rao (CR) bounds represent a limitation of the estimation quality of unnown parameters from the given data If the variance of the estimates reaches CR bounds it can be stated that the estimation algorithm wors optimally In this paper we will derive CR bounds for the estimation of noise covariances of a linear stochastic system he noise covariances are tuning parameters of Kalman filter and the filter performance depends on them However they are not nown in general and hence tuning of a Kalman filter remains a challenging tas he very first methods for noise covariance estimation were published in 7s by Mehra[] Recently new methods were proposed by Odelson et al [3] and further modified by Rajamani et al[45] and Aesson et al[6] A maimum lielihood approach was described in[7] In Section 5 these two approaches will be compared to the CR bounds and their performance will be discussed In the last part of the paper relationship between the Riccati equation and the CR bounds will be pointed out Deriving of the noise covariance CR bounds is based on[8] and[9] Additional information about CR bounds and Bayesian estimation can be found in[] Cramér-Rao Bounds * Corresponding author: petermatiso@felcvutcz (Peter Matiso) Published online at Copyright Scientific & Academic Publishing All Rights Reserved his paper was presented at the 8 th IFAC World Congress Italy [online] he tas is to estimate a vector of parameters from a set of measured data y y y y he upper inde emphasizes that vector y contains data samples he conditioned probability density function (pdf) p( y ) is assumed to be nown Further the nabla operator is defined as () he Fisher information matri (FIM) J( ) for parameters is defined as ( ) ln J p y y () where operator y is the conditioned epected value defined as f( ) f( ) p( y ) dy (3) y Alternatively we can define a posterior FIM (and posterior CR bounds) using joint probability p( y ) instead of the conditioned one where p( y ) p( y ) p Probability function p represents prior information about the parameter vector Assume having an unbiased estimate of parameter vector ˆ obtained by an arbitrary method When the true value of parameter vector is compared to the unbiased estimated set the following inequality holds [8] where J represents the Cramér-Rao bound Inequality A B means that A B is a positive semidefinite matri Based on (4) the following statement can be (4)
2 Journal of Mechanical Engineering and Automation (): 6-7 concluded Assuming that CR bound is reachable any estimation algorithm woring optimally (in the sense of the smallest covariance of obtained estimates) must give estimates whose variance is equal to the Cramér-Rao bound If the CR bound is reachable then the optimal result can be achieved by a maimum lielihood approach Consider a scalar linear stochastic system a v (5) y c e where and y represent the state and the measured output respectively Stochastic inputs v and e are described by normal distributions vt () Q et () R (6) Using the notation of[9] the logarithms of the conditioned probability of the state measurements and unnown parameters can be epressed as y py i i ln p ln i (7) ln p ln i p i i he following matrices are used for FIM calculation K ln p (8) K ln p (9) K ln p () K ln p () K ln p () K ln p (3) L y ln p y (4) L ln p y y (5) L ln p y y (6) Further recursive formulas for FIM of the state and unnown parameters are defined as J J L K J K K (7) J K K K where J K J K (8) and K K J J L J J L (9) () Initial conditions of the recursive algorithm are set as J K J K ln p () J A ln p θ θ θ θ where K and A represent prior information If no prior probability function is nown then A K he final form of FIM of the state and parameters is J J J () J J 3 Cramér-Rao Bounds for oise Covariances Estimation Consider a scalar stochastic system given by (5) where a he noise sources are not correlated and are defined by (6) he probabilities used in (7) can be defined as p Q a Q (3) p y R c R ow matrices ()-(6) can be calculated using the following general integral formulas assuming e d ( ) e d ( ) ( ) e d e d 3 4 Further logarithm of the Gaussian function is given by ln ( P) ln ln P (5) P CR bounds of Q R estimates are to be found considering a scalar system with two unnown parameters he nabla operator in this case is of the form (6) Q R where Q R At the first step the arguments of the epectation operators in ()-(6) are calculated as follows (4)
3 8 Peter Matiso et al: Cramér-Rao Bounds for Estimation of Linear System oise Covariances a a ln p Q (7) a ln p Q (8) a 3 ln p Q Q (9) ln p y y c (3) 3 R R et matrices ()-(6) are obtained using integral formulas (4) as K (3) i K i (3) K Q (33) L (34) L R (35) A ln p (36) It can be seen from (3)-(35) that several of the matrices are zero which significantly simplifies formula (7) In par- ticular matri J in () is zero and this fact allows us to calculate FIM for unnown covariances independently on the states he resulting FIM can be written as J J L K (37) where J represents the prior information about unnown parameters Q R If no prior information is available it can be set to zero From the above equations and the form of matrices in (3)-(35) it can be seen that FIM is calculated for each covariance independently Formula for FIM is of the form J J Q (38) R where it can be assumed that J A particular closed form solution to recursive formula (38) is given by J Q R 4 Cramér-Rao Bounds for a System in Innovation Form (39) In Section a stochastic system was defined using (5) he system has two independent sources of uncertainty the process noise and the measurement noise It is assumed the process and the measurement noises to be normal with zero mean and covariances Q R respectively In non-scalar case variables Q R are matrices and contain redundant information oise covariances represent properties of the noise sources Considering a higher order system allows us to model the same number of noise sources as the sum of numbers of states and outputs Matri Q would have the same dimension as the number of states and matri R would have a dimension equal to the number of outputs However from the observed output data only the number of noise sources equal to the number of outputs can be recognized herefore we can alternatively use the so-called innovation form of stochastic system that represents a minimal form[] or [] he disadvantage of such formulation comes from the fact that we loose the physical bacground of the noise sources and their structure However in many cases we only have output data at hand and no other additional information about the stochastic inputs In such case the innovation form of the stochastic system can be used a K (4) y c where K is a steady state Kalman gain and () is the stationary innovation process hen ( ) R (4) he Cramér-Rao bounds for K R ε estimation are derived using the same formulas as for the system (5) Analogously to formulas (7)-(3) we define a 3 a KR ln p a (4) a KR a KR ln p (43) a KR ln p a a 4 3 K RK RK (44) a a 3 3 RK R RK
4 Journal of Mechanical Engineering and Automation (): 6-9 ln p y c y (45) 3 R R he epected values are calculated using integral formulas (4) Matrices ()-(6) are of the form K (46) K (47) K KR K (48) KR R L (49) L (5) R he formula for FIM calculation is of the form (37) Substituting (48) and (5) to (37) leads to the resulting formula K KR J (5) KR R variance of the estimates is calculated he dependence of the variance of the estimated Q-parameters on the number of used data samples is shown in Figure J(Q) Figure Cramér-Rao bound for Q estimation and variance of estimated Q using ALS and Maimum lielihood method In Figure an analogous result for R estimation is shown We can state that ML method converges faster to the CR bound than ALS ML algorithm gives better estimates in case of covariance Q than ALS even with very small sets of data (around 5 samples) CR bound ML method ALS method CR bound ML method ALS method 5 Comparison of the Methods for oise Covariance Estimation J(R) In the previous sections we derived the CR bounds for noise covariances estimation In this chapter a comparison of ALS (Autoregressive least squares) method and maimum lielihood method will be presented he ALS method [3-5] uses the assumption that estimated autocorrelation function calculated from the finite set of data converges to the true autocorrelation function In[7] it is shown that the assumption of fast convergence is not correct and this fact can significantly limit the quality of the estimates In the same paper a maimum lielihood approach is demonstrated using a simple maimum seeing method We can compare performance of both approaches according to the CR bounds he ALS algorithm prior setting contains the optimal Kalman filter gain and the maimum lag for autocorrelation calculation d 5 he ML method searches the maimum in the interval to he grid is logarithmic and contains 8 8 points Consider a system of the form (5) where a 5 c Q and R he estimation algorithm uses time samples and is repeated 3 times hen the resulting Figure Cramér-Rao bound for R estimation and variance of estimated R using ALS and Maimum lielihood method 6 Relationship between Cramér-Rao Bounds and Riccati Equation In this section we will point out the relationship between Riccati equation (RE) and CR bounds It is well nown that under some conditions Kalman filter is optimal It was stated in the previous section that CR bounds represent a limit of estimation quality herefore it can be epected that RE should converge to CR bounds for state estimation of a linear system Using formulas (8)-() and (4) the CR bounds for state estimation can be easily obtained ichavsý et al (998) We have
5 Peter Matiso et al: Cramér-Rao Bounds for Estimation of Linear System oise Covariances K A Q A K A Q K Q L C R C (5) ow the recursive formula for FIM calculation Šimandl et al () is of the form J K K J K L K (53) Using (5) we obtain resulting formula J Q Q A J A Q A C R C A Q (54) Riccati equation is of the form P AP A AP C CP C R CP A Q (55) It can be stated that P and matri Moreover if initial values J converge to the same P J then trajectories of P and J are the same his fact proves the optimality of Kalman filter Equivalency of equations (54) and (55) is proved in Appendi 7 Conclusions In this paper Cramér-Rao bounds for the noise covariance estimation were derived for two different stochastic models From (3)-(35) it can be seen that matrices K K and L are zero that means that the information between state and noise covariances is not correlated It can be concluded that the covariances and the states can be estimated independently of each other Another observation can be done on matrices (33) and (35) It can be seen that there is no correlation between information about the covariances hat means that CR bounds can be calculated for each covariance separately Considering limit cases Q converging to zero and R being arbitrary large CR bound for Q estimation will converge to zero either It means that there eists a method that can estimate Q up to an arbitrary level of accuracy CR bounds can be used as a quality measure for all newly proposed estimating algorithms We have compared a maimum lielihood approach with ALS method of Odelson et al (5) according to the CR bounds Based on these eamples (Figure and Figure ) it can be observed that the CR bounds for the noise covariances converge to zero relatively fast even for small sets of data ( to 4 samples) In Section 6 we pointed out a relationship between Fisher information matri recursive formula and Riccati equation Equivalence of the equations is proved in Appendi We concluded that this is another proof of optimality of Kalman filter APPEDIX Consider two recursive equations (56) and (57) that are Fisher information matri recursive equation and discrete Riccati equation Both equations are equivalent and P J for sufficient large not necessarily J Q Q A J A Q AC R C A Q (56) P AP A AP C CP C R CP A Q (57) he right side of equation (56) is modified using substitution J P Q Q A J A Q AC R C A Q Q A A QA P A Q AC R C A Q Q A A QA P A Q AC R C P A Q AC R C A Q Q A A QA P C R C P A Q AC R C A Q ow both sides of the equation are inverted so the left side is P and the right side is of the form Q A A QA P C R C P AQ ACR C AQ QA P A Q A C R C P CR C A P CR C AQ A A Q QA P A Q A C R C AP CR C A Q Further an inversion lemma (58) will be applied on the epression A BD C A A B D CA B CA (59) he final formula is of the form P CR C P AP A AP C CP C R CP A Q which is eactly Riccati equation ACKOWLEDGEMES his wor was supported by the Czech Science Foundation grant o P3//353 REFERECES [] Mehra RK (97) On the identification of variances and adaptive Kalman filtering IEEE rans on Automat Control AC 5 pp [] Mehra R K (97) Approaches to Adaptive Filtering IEEE
6 Journal of Mechanical Engineering and Automation (): 6- rans on Automat Control AC 7 pp [3] Odelson B J Rajamani M R and Rawlings J B (5) A new autocovariance least squares method for estimating noise covariances Automatica Vol 4 pp [4] Rajamani M R and Rawlings J B (8) Estimation of the disturbance structure from data using semidefinite programming and optimal weighting Automatica Vol 45 pp 7 [5] Rajamani M R and Rawlings J B [online] hewiscedu/software/als [6] Aesson B M Jorgensen J B Poulsen K and Jorgensen S B (8) A generalized autocovariance least squares method for Kalman filter tuning Journal of Process control Vol 8 pp [7] Matiso P and Havlena V () oise covariances estimation for Kalman filter tuning Proceedings of IFAC AL- COSP urey [8] ichavsý P Muravchi C H and ehorai A (998) Posterior Cramér-Rao Bounds for Discrete-ime onlinear Filtering IEEE rans on Signal Processing Vol 46 pp [9] Šimandl M Královec J and ichavsý P () Filtering predictive and smoothing Cramér-Rao bounds for discrete-time nonlinear dynamic systems Automatica Vol 37 pp [] Candy J V (9) Bayesian Signal Processing John Wiley & Sons Inc USA [] Ljung L (987) System Identification he heory for he User Prentice Hall USA [] Kailath Sayed A H and Hassibi B () Linear Estimation Prentice Hall USA [3] Anderson B D O and Moore J (979) Optimal filtering Dover Publications USA [4] Matiso P (9) Estimation of covariance matrices of the noise of linear stochastic systems Diploma thesis Czech echnical University in Prague [5] Matiso P and Havlena V () Cramér-Rao bounds for estimation of linear system noise covariances Proceedings of the 8 th IFAC World Congress Italy [6] Papoulis A (99) Probability Random Variables and Stochastic Processes hird edition Mc Graw-Hill Inc USA
Basic concepts in estimation
Basic concepts in estimation Random and nonrandom parameters Definitions of estimates ML Maimum Lielihood MAP Maimum A Posteriori LS Least Squares MMS Minimum Mean square rror Measures of quality of estimates
More informationRobust Adaptive Estimators for Nonlinear Systems
Abdul Wahab Hamimi Fadziati Binti and Katebi Reza () Robust adaptive estimators for nonlinear systems. In: Conference on Control and Fault-olerant Systems (Sysol) --9 - --. http://d.doi.org/.9/sysol..6698
More informationLecture Note 2: Estimation and Information Theory
Univ. of Michigan - NAME 568/EECS 568/ROB 530 Winter 2018 Lecture Note 2: Estimation and Information Theory Lecturer: Maani Ghaffari Jadidi Date: April 6, 2018 2.1 Estimation A static estimation problem
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationState Estimation of Linear and Nonlinear Dynamic Systems
State Estimation of Linear and Nonlinear Dynamic Systems Part II: Observability and Stability James B. Rawlings and Fernando V. Lima Department of Chemical and Biological Engineering University of Wisconsin
More informationPERIODIC KALMAN FILTER: STEADY STATE FROM THE BEGINNING
Journal of Mathematical Sciences: Advances and Applications Volume 1, Number 3, 2008, Pages 505-520 PERIODIC KALMAN FILER: SEADY SAE FROM HE BEGINNING MARIA ADAM 1 and NICHOLAS ASSIMAKIS 2 1 Department
More informationPrediction, filtering and smoothing using LSCR: State estimation algorithms with guaranteed confidence sets
2 5th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) Orlando, FL, USA, December 2-5, 2 Prediction, filtering and smoothing using LSCR: State estimation algorithms with
More informationON MODEL SELECTION FOR STATE ESTIMATION FOR NONLINEAR SYSTEMS. Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof
ON MODEL SELECTION FOR STATE ESTIMATION FOR NONLINEAR SYSTEMS Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof Delft Center for Systems and Control, Delft University of Technology, Mekelweg 2, 2628 CD
More informationCzech Technical University in Prague
Czech echnical University in Prague Faculty of Electrical Engineering Doctoral hesis Peter Matisko Prague, 2013 Estimation of the stochastic properties of controlled systems "here is nothing more practical
More informationDESIGNING A KALMAN FILTER WHEN NO NOISE COVARIANCE INFORMATION IS AVAILABLE. Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof
DESIGNING A KALMAN FILTER WHEN NO NOISE COVARIANCE INFORMATION IS AVAILABLE Robert Bos,1 Xavier Bombois Paul M. J. Van den Hof Delft Center for Systems and Control, Delft University of Technology, Mekelweg
More informationFUNDAMENTAL FILTERING LIMITATIONS IN LINEAR NON-GAUSSIAN SYSTEMS
FUNDAMENTAL FILTERING LIMITATIONS IN LINEAR NON-GAUSSIAN SYSTEMS Gustaf Hendeby Fredrik Gustafsson Division of Automatic Control Department of Electrical Engineering, Linköpings universitet, SE-58 83 Linköping,
More informationCONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Recursive Algorithms - Han-Fu Chen
CONROL SYSEMS, ROBOICS, AND AUOMAION - Vol. V - Recursive Algorithms - Han-Fu Chen RECURSIVE ALGORIHMS Han-Fu Chen Institute of Systems Science, Academy of Mathematics and Systems Science, Chinese Academy
More informationAlgorithm for Multiple Model Adaptive Control Based on Input-Output Plant Model
BULGARIAN ACADEMY OF SCIENCES CYBERNEICS AND INFORMAION ECHNOLOGIES Volume No Sofia Algorithm for Multiple Model Adaptive Control Based on Input-Output Plant Model sonyo Slavov Department of Automatics
More informationAn insight into noise covariance estimation for Kalman filter design
Preprints of the 19th World Congress The International Federation of Automatic Control An insight into noise covariance estimation for Kalman filter design Simone Formentin, Sergio Bittanti (IFAC Fellow)
More informationMini-Course 07 Kalman Particle Filters. Henrique Massard da Fonseca Cesar Cunha Pacheco Wellington Bettencurte Julio Dutra
Mini-Course 07 Kalman Particle Filters Henrique Massard da Fonseca Cesar Cunha Pacheco Wellington Bettencurte Julio Dutra Agenda State Estimation Problems & Kalman Filter Henrique Massard Steady State
More informationDiscrete-Time H Gaussian Filter
Proceedings of the 17th World Congress The International Federation of Automatic Control Discrete-Time H Gaussian Filter Ali Tahmasebi and Xiang Chen Department of Electrical and Computer Engineering,
More informationState Estimation of Linear and Nonlinear Dynamic Systems
State Estimation of Linear and Nonlinear Dynamic Systems Part I: Linear Systems with Gaussian Noise James B. Rawlings and Fernando V. Lima Department of Chemical and Biological Engineering University of
More informationVariations. ECE 6540, Lecture 10 Maximum Likelihood Estimation
Variations ECE 6540, Lecture 10 Last Time BLUE (Best Linear Unbiased Estimator) Formulation Advantages Disadvantages 2 The BLUE A simplification Assume the estimator is a linear system For a single parameter
More informationEstimation, Detection, and Identification CMU 18752
Estimation, Detection, and Identification CMU 18752 Graduate Course on the CMU/Portugal ECE PhD Program Spring 2008/2009 Instructor: Prof. Paulo Jorge Oliveira pjcro @ isr.ist.utl.pt Phone: +351 21 8418053
More information2 Introduction of Discrete-Time Systems
2 Introduction of Discrete-Time Systems This chapter concerns an important subclass of discrete-time systems, which are the linear and time-invariant systems excited by Gaussian distributed stochastic
More information6.435, System Identification
SET 6 System Identification 6.435 Parametrized model structures One-step predictor Identifiability Munther A. Dahleh 1 Models of LTI Systems A complete model u = input y = output e = noise (with PDF).
More informationCONTROL SYSTEMS, ROBOTICS, AND AUTOMATION - Vol. V - Prediction Error Methods - Torsten Söderström
PREDICTIO ERROR METHODS Torsten Söderström Department of Systems and Control, Information Technology, Uppsala University, Uppsala, Sweden Keywords: prediction error method, optimal prediction, identifiability,
More informationUTILIZING PRIOR KNOWLEDGE IN ROBUST OPTIMAL EXPERIMENT DESIGN. EE & CS, The University of Newcastle, Australia EE, Technion, Israel.
UTILIZING PRIOR KNOWLEDGE IN ROBUST OPTIMAL EXPERIMENT DESIGN Graham C. Goodwin James S. Welsh Arie Feuer Milan Depich EE & CS, The University of Newcastle, Australia 38. EE, Technion, Israel. Abstract:
More informationFisher Information Matrix-based Nonlinear System Conversion for State Estimation
Fisher Information Matrix-based Nonlinear System Conversion for State Estimation Ming Lei Christophe Baehr and Pierre Del Moral Abstract In practical target tracing a number of improved measurement conversion
More informationDesign of FIR Smoother Using Covariance Information for Estimating Signal at Start Time in Linear Continuous Systems
Systems Science and Applied Mathematics Vol. 1 No. 3 2016 pp. 29-37 http://www.aiscience.org/journal/ssam Design of FIR Smoother Using Covariance Information for Estimating Signal at Start Time in Linear
More informationNonlinear Parameter Estimation for State-Space ARCH Models with Missing Observations
Nonlinear Parameter Estimation for State-Space ARCH Models with Missing Observations SEBASTIÁN OSSANDÓN Pontificia Universidad Católica de Valparaíso Instituto de Matemáticas Blanco Viel 596, Cerro Barón,
More informationKalman-Filter-Based Time-Varying Parameter Estimation via Retrospective Optimization of the Process Noise Covariance
2016 American Control Conference (ACC) Boston Marriott Copley Place July 6-8, 2016. Boston, MA, USA Kalman-Filter-Based Time-Varying Parameter Estimation via Retrospective Optimization of the Process Noise
More informationA New Approach to Tune the Vold-Kalman Estimator for Order Tracking
A New Approach to Tune the Vold-Kalman Estimator for Order Tracking Amadou Assoumane, Julien Roussel, Edgard Sekko and Cécile Capdessus Abstract In the purpose to diagnose rotating machines using vibration
More informationState Estimation by IMM Filter in the Presence of Structural Uncertainty 1
Recent Advances in Signal Processing and Communications Edited by Nios Mastorais World Scientific and Engineering Society (WSES) Press Greece 999 pp.8-88. State Estimation by IMM Filter in the Presence
More informationMaximum Likelihood Diffusive Source Localization Based on Binary Observations
Maximum Lielihood Diffusive Source Localization Based on Binary Observations Yoav Levinboo and an F. Wong Wireless Information Networing Group, University of Florida Gainesville, Florida 32611-6130, USA
More informationLinear Prediction Theory
Linear Prediction Theory Joseph A. O Sullivan ESE 524 Spring 29 March 3, 29 Overview The problem of estimating a value of a random process given other values of the random process is pervasive. Many problems
More informationTime Series: Theory and Methods
Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary
More informationOn the convergence of the iterative solution of the likelihood equations
On the convergence of the iterative solution of the likelihood equations R. Moddemeijer University of Groningen, Department of Computing Science, P.O. Box 800, NL-9700 AV Groningen, The Netherlands, e-mail:
More informationRECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK
RECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK TRNKA PAVEL AND HAVLENA VLADIMÍR Dept of Control Engineering, Czech Technical University, Technická 2, 166 27 Praha, Czech Republic mail:
More informationOnline monitoring of MPC disturbance models using closed-loop data
Online monitoring of MPC disturbance models using closed-loop data Brian J. Odelson and James B. Rawlings Department of Chemical Engineering University of Wisconsin-Madison Online Optimization Based Identification
More informationSimultaneous state and input estimation with partial information on the inputs
Loughborough University Institutional Repository Simultaneous state and input estimation with partial information on the inputs This item was submitted to Loughborough University's Institutional Repository
More informationInformation, Covariance and Square-Root Filtering in the Presence of Unknown Inputs 1
Katholiee Universiteit Leuven Departement Eletrotechnie ESAT-SISTA/TR 06-156 Information, Covariance and Square-Root Filtering in the Presence of Unnown Inputs 1 Steven Gillijns and Bart De Moor 2 October
More informationML ESTIMATION AND CRB FOR NARROWBAND AR SIGNALS ON A SENSOR ARRAY
2014 IEEE International Conference on Acoustic, Speech and Signal Processing ICASSP ML ESTIMATION AND CRB FOR NARROWBAND AR SIGNALS ON A SENSOR ARRAY Langford B White School of Electrical and Electronic
More informationFIR Filters for Stationary State Space Signal Models
Proceedings of the 17th World Congress The International Federation of Automatic Control FIR Filters for Stationary State Space Signal Models Jung Hun Park Wook Hyun Kwon School of Electrical Engineering
More informationLecture 8: Bayesian Estimation of Parameters in State Space Models
in State Space Models March 30, 2016 Contents 1 Bayesian estimation of parameters in state space models 2 Computational methods for parameter estimation 3 Practical parameter estimation in state space
More informationIdentification of State and Measurement Noise Covariance Matrices using Nonlinear Estimation Framework
Journal of Physics: Conference Series PAPER OPEN ACCESS Identification of State and Measurement Noise Covariance Matrices using Nonlinear Estimation Framework To cite this article: Oliver Kost et al 215
More informationp(x ω i 0.4 ω 2 ω
p( ω i ). ω.3.. 9 3 FIGURE.. Hypothetical class-conditional probability density functions show the probability density of measuring a particular feature value given the pattern is in category ω i.if represents
More informationEstimators as Random Variables
Estimation Theory Overview Properties Bias, Variance, and Mean Square Error Cramér-Rao lower bound Maimum likelihood Consistency Confidence intervals Properties of the mean estimator Introduction Up until
More informationMaximum Likelihood Modeling Of Orbits Of Nonlinear ODEs
Maimum Likelihood Modeling Of Orbits Of onlinear ODEs orsten Söderström, orbjörn Wigren and Emad Abd-Elrady Department of Systems and Control, Information echnology, Uppsala Univ., PO Bo 337, SE-75 05
More informationTIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.
TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION
More informationEstimation, Detection, and Identification
Estimation, Detection, and Identification Graduate Course on the CMU/Portugal ECE PhD Program Spring 2008/2009 Chapter 5 Best Linear Unbiased Estimators Instructor: Prof. Paulo Jorge Oliveira pjcro @ isr.ist.utl.pt
More informationOPTIMAL CONTROL AND ESTIMATION
OPTIMAL CONTROL AND ESTIMATION Robert F. Stengel Department of Mechanical and Aerospace Engineering Princeton University, Princeton, New Jersey DOVER PUBLICATIONS, INC. New York CONTENTS 1. INTRODUCTION
More informationA NOVEL OPTIMAL PROBABILITY DENSITY FUNCTION TRACKING FILTER DESIGN 1
A NOVEL OPTIMAL PROBABILITY DENSITY FUNCTION TRACKING FILTER DESIGN 1 Jinglin Zhou Hong Wang, Donghua Zhou Department of Automation, Tsinghua University, Beijing 100084, P. R. China Control Systems Centre,
More informationModelling using ARMA processes
Modelling using ARMA processes Step 1. ARMA model identification; Step 2. ARMA parameter estimation Step 3. ARMA model selection ; Step 4. ARMA model checking; Step 5. forecasting from ARMA models. 33
More informationSquare-Root Algorithms of Recursive Least-Squares Wiener Estimators in Linear Discrete-Time Stochastic Systems
Proceedings of the 17th World Congress The International Federation of Automatic Control Square-Root Algorithms of Recursive Least-Squares Wiener Estimators in Linear Discrete-Time Stochastic Systems Seiichi
More informationOn the convergence of the iterative solution of the likelihood equations
On the convergence of the iterative solution of the likelihood equations R. Moddemeijer University of Groningen, Department of Computing Science, P.O. Box 800, NL-9700 AV Groningen, The Netherlands, e-mail:
More informationECONOMETRIC METHODS II: TIME SERIES LECTURE NOTES ON THE KALMAN FILTER. The Kalman Filter. We will be concerned with state space systems of the form
ECONOMETRIC METHODS II: TIME SERIES LECTURE NOTES ON THE KALMAN FILTER KRISTOFFER P. NIMARK The Kalman Filter We will be concerned with state space systems of the form X t = A t X t 1 + C t u t 0.1 Z t
More informationFiltering for Linear Systems with Error Variance Constraints
IEEE RANSACIONS ON SIGNAL PROCESSING, VOL. 48, NO. 8, AUGUS 2000 2463 application of the one-step extrapolation procedure of [3], it is found that the existence of signal z(t) is not valid in the space
More informationcovariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of
Index* The Statistical Analysis of Time Series by T. W. Anderson Copyright 1971 John Wiley & Sons, Inc. Aliasing, 387-388 Autoregressive {continued) Amplitude, 4, 94 case of first-order, 174 Associated
More informationExpressions for the covariance matrix of covariance data
Expressions for the covariance matrix of covariance data Torsten Söderström Division of Systems and Control, Department of Information Technology, Uppsala University, P O Box 337, SE-7505 Uppsala, Sweden
More informationA Comparison of Particle Filters for Personal Positioning
VI Hotine-Marussi Symposium of Theoretical and Computational Geodesy May 9-June 6. A Comparison of Particle Filters for Personal Positioning D. Petrovich and R. Piché Institute of Mathematics Tampere University
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationAuxiliary signal design for failure detection in uncertain systems
Auxiliary signal design for failure detection in uncertain systems R. Nikoukhah, S. L. Campbell and F. Delebecque Abstract An auxiliary signal is an input signal that enhances the identifiability of a
More informationSteady-state DKF. M. Sami Fadali Professor EE UNR
Steady-state DKF M. Sami Fadali Professor EE UNR 1 Outline Stability of linear estimators. Lyapunov equation. Uniform exponential stability. Steady-state behavior of Lyapunov equation. Riccati equation.
More informationHammerstein System Identification by a Semi-Parametric Method
Hammerstein System Identification by a Semi-arametric ethod Grzegorz zy # # Institute of Engineering Cybernetics, Wroclaw University of echnology, ul. Janiszewsiego 11/17, 5-372 Wroclaw, oland, grmz@ict.pwr.wroc.pl
More informationMobile Robot Localization
Mobile Robot Localization 1 The Problem of Robot Localization Given a map of the environment, how can a robot determine its pose (planar coordinates + orientation)? Two sources of uncertainty: - observations
More informationFurther Results on Model Structure Validation for Closed Loop System Identification
Advances in Wireless Communications and etworks 7; 3(5: 57-66 http://www.sciencepublishinggroup.com/j/awcn doi:.648/j.awcn.735. Further esults on Model Structure Validation for Closed Loop System Identification
More informationSimultaneous Input and State Estimation for Linear Discrete-time Stochastic Systems with Direct Feedthrough
52nd IEEE Conference on Decision and Control December 10-13, 2013. Florence, Italy Simultaneous Input and State Estimation for Linear Discrete-time Stochastic Systems with Direct Feedthrough Sze Zheng
More informationResearch Article Weighted Measurement Fusion White Noise Deconvolution Filter with Correlated Noise for Multisensor Stochastic Systems
Mathematical Problems in Engineering Volume 2012, Article ID 257619, 16 pages doi:10.1155/2012/257619 Research Article Weighted Measurement Fusion White Noise Deconvolution Filter with Correlated Noise
More informationConstrained State Estimation Using the Unscented Kalman Filter
16th Mediterranean Conference on Control and Automation Congress Centre, Ajaccio, France June 25-27, 28 Constrained State Estimation Using the Unscented Kalman Filter Rambabu Kandepu, Lars Imsland and
More informationPOSTERIOR CRAMÉR-RAO BOUNDS ANALYSIS FOR PASSIVE TARGET TRACKING 1
Vol5 No JOURNAL OF ELECRONICS(CHINA) January 8 POSERIOR CRAMÉR-RAO BOUNDS ANALYSIS FOR PASSIVE ARGE RACKING Zhang Jun Zhan Ronghui (School of Electronic Science and Engineering, National Univ of Defense
More informationState Estimation using Moving Horizon Estimation and Particle Filtering
State Estimation using Moving Horizon Estimation and Particle Filtering James B. Rawlings Department of Chemical and Biological Engineering UW Math Probability Seminar Spring 2009 Rawlings MHE & PF 1 /
More informationDetection & Estimation Lecture 1
Detection & Estimation Lecture 1 Intro, MVUE, CRLB Xiliang Luo General Course Information Textbooks & References Fundamentals of Statistical Signal Processing: Estimation Theory/Detection Theory, Steven
More information2.7 The Gaussian Probability Density Function Forms of the Gaussian pdf for Real Variates
.7 The Gaussian Probability Density Function Samples taken from a Gaussian process have a jointly Gaussian pdf (the definition of Gaussian process). Correlator outputs are Gaussian random variables if
More informationBayesian Approach 2. CSC412 Probabilistic Learning & Reasoning
CSC412 Probabilistic Learning & Reasoning Lecture 12: Bayesian Parameter Estimation February 27, 2006 Sam Roweis Bayesian Approach 2 The Bayesian programme (after Rev. Thomas Bayes) treats all unnown quantities
More informationKalman Filters with Uncompensated Biases
Kalman Filters with Uncompensated Biases Renato Zanetti he Charles Stark Draper Laboratory, Houston, exas, 77058 Robert H. Bishop Marquette University, Milwaukee, WI 53201 I. INRODUCION An underlying assumption
More informationOn Identification of Cascade Systems 1
On Identification of Cascade Systems 1 Bo Wahlberg Håkan Hjalmarsson Jonas Mårtensson Automatic Control and ACCESS, School of Electrical Engineering, KTH, SE-100 44 Stockholm, Sweden. (bo.wahlberg@ee.kth.se
More informationOPTIMAL ESTIMATION of DYNAMIC SYSTEMS
CHAPMAN & HALL/CRC APPLIED MATHEMATICS -. AND NONLINEAR SCIENCE SERIES OPTIMAL ESTIMATION of DYNAMIC SYSTEMS John L Crassidis and John L. Junkins CHAPMAN & HALL/CRC A CRC Press Company Boca Raton London
More informationLeast Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD
Least Squares Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 75A Winter 0 - UCSD (Unweighted) Least Squares Assume linearity in the unnown, deterministic model parameters Scalar, additive noise model: y f (
More informationEstimation of the Disturbance Structure from Data using Semidefinite Programming and Optimal Weighting
2 TWMCC Texas-Wisconsin Modeling and Control Consortium 1 Technical report number 2007-02 Estimation of the Disturbance Structure from Data using Semidefinite Programming and Optimal Weighting Murali R
More informationSIGMA POINT GAUSSIAN SUM FILTER DESIGN USING SQUARE ROOT UNSCENTED FILTERS
SIGMA POINT GAUSSIAN SUM FILTER DESIGN USING SQUARE ROOT UNSCENTED FILTERS Miroslav Šimandl, Jindřich Duní Department of Cybernetics and Research Centre: Data - Algorithms - Decision University of West
More informationLikelihood Bounds for Constrained Estimation with Uncertainty
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 5 Seville, Spain, December -5, 5 WeC4. Likelihood Bounds for Constrained Estimation with Uncertainty
More informationESTIMATOR STABILITY ANALYSIS IN SLAM. Teresa Vidal-Calleja, Juan Andrade-Cetto, Alberto Sanfeliu
ESTIMATOR STABILITY ANALYSIS IN SLAM Teresa Vidal-Calleja, Juan Andrade-Cetto, Alberto Sanfeliu Institut de Robtica i Informtica Industrial, UPC-CSIC Llorens Artigas 4-6, Barcelona, 88 Spain {tvidal, cetto,
More informationA SQUARE ROOT ALGORITHM FOR SET THEORETIC STATE ESTIMATION
A SQUARE ROO ALGORIHM FOR SE HEOREIC SAE ESIMAION U D Hanebec Institute of Automatic Control Engineering echnische Universität München 80290 München, Germany fax: +49-89-289-28340 e-mail: UweHanebec@ieeeorg
More informationPARAMETER ESTIMATION AND ORDER SELECTION FOR LINEAR REGRESSION PROBLEMS. Yngve Selén and Erik G. Larsson
PARAMETER ESTIMATION AND ORDER SELECTION FOR LINEAR REGRESSION PROBLEMS Yngve Selén and Eri G Larsson Dept of Information Technology Uppsala University, PO Box 337 SE-71 Uppsala, Sweden email: yngveselen@ituuse
More informationEECE Adaptive Control
EECE 574 - Adaptive Control Basics of System Identification Guy Dumont Department of Electrical and Computer Engineering University of British Columbia January 2010 Guy Dumont (UBC) EECE574 - Basics of
More informationAdaptive Two-Stage EKF for INS-GPS Loosely Coupled System with Unknown Fault Bias
Journal of Gloal Positioning Systems (26 Vol. 5 No. -2:62-69 Adaptive wo-stage EKF for INS-GPS Loosely Coupled System with Unnown Fault Bias Kwang Hoon Kim Jang Gyu Lee School of Electrical Engineering
More informationState Space and Hidden Markov Models
State Space and Hidden Markov Models Kunsch H.R. State Space and Hidden Markov Models. ETH- Zurich Zurich; Aliaksandr Hubin Oslo 2014 Contents 1. Introduction 2. Markov Chains 3. Hidden Markov and State
More informationEVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER
EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER Zhen Zhen 1, Jun Young Lee 2, and Abdus Saboor 3 1 Mingde College, Guizhou University, China zhenz2000@21cn.com 2 Department
More informationAn Interpretation of the Moore-Penrose. Generalized Inverse of a Singular Fisher Information Matrix
An Interpretation of the Moore-Penrose 1 Generalized Inverse of a Singular Fisher Information Matrix Yen-Huan Li, Member, IEEE, Ping-Cheng Yeh, Member, IEEE, arxiv:1107.1944v4 [cs.i] 6 Aug 2012 Abstract
More informationLTI Systems, Additive Noise, and Order Estimation
LTI Systems, Additive oise, and Order Estimation Soosan Beheshti, Munther A. Dahleh Laboratory for Information and Decision Systems Department of Electrical Engineering and Computer Science Massachusetts
More informationRecursive Least Squares for an Entropy Regularized MSE Cost Function
Recursive Least Squares for an Entropy Regularized MSE Cost Function Deniz Erdogmus, Yadunandana N. Rao, Jose C. Principe Oscar Fontenla-Romero, Amparo Alonso-Betanzos Electrical Eng. Dept., University
More informationDetection & Estimation Lecture 1
Detection & Estimation Lecture 1 Intro, MVUE, CRLB Xiliang Luo General Course Information Textbooks & References Fundamentals of Statistical Signal Processing: Estimation Theory/Detection Theory, Steven
More informationAchieving State Estimation Equivalence for Misassigned Disturbances in Offset-Free Model Predictive Control
Achieving State Estimation Equivalence for Misassigned Disturbances in Offset-Free Model Predictive Control Murali R. Rajamani BP Research and Technology, Naperville, IL 05 James B. Rawlings Dept. of Chemical
More informationConditional Posterior Cramér-Rao Lower Bounds for Nonlinear Sequential Bayesian Estimation
1 Conditional Posterior Cramér-Rao Lower Bounds for Nonlinear Sequential Bayesian Estimation Long Zuo, Ruixin Niu, and Pramod K. Varshney Abstract Posterior Cramér-Rao lower bounds (PCRLBs) 1] for sequential
More informationADAPTIVE FILTER THEORY
ADAPTIVE FILTER THEORY Fourth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada Front ice Hall PRENTICE HALL Upper Saddle River, New Jersey 07458 Preface
More informationAdaptive Filter Theory
0 Adaptive Filter heory Sung Ho Cho Hanyang University Seoul, Korea (Office) +8--0-0390 (Mobile) +8-10-541-5178 dragon@hanyang.ac.kr able of Contents 1 Wiener Filters Gradient Search by Steepest Descent
More informationGrowing Window Recursive Quadratic Optimization with Variable Regularization
49th IEEE Conference on Decision and Control December 15-17, Hilton Atlanta Hotel, Atlanta, GA, USA Growing Window Recursive Quadratic Optimization with Variable Regularization Asad A. Ali 1, Jesse B.
More informationNonlinear State Estimation! Particle, Sigma-Points Filters!
Nonlinear State Estimation! Particle, Sigma-Points Filters! Robert Stengel! Optimal Control and Estimation, MAE 546! Princeton University, 2017!! Particle filter!! Sigma-Points Unscented Kalman ) filter!!
More informationElements of Multivariate Time Series Analysis
Gregory C. Reinsel Elements of Multivariate Time Series Analysis Second Edition With 14 Figures Springer Contents Preface to the Second Edition Preface to the First Edition vii ix 1. Vector Time Series
More informationIdentification of Disturbance Covariances Using Maximum Likelihood Estimation
TWCCC Texas Wisconsin California Control Consortium Technical report number 2014-02 Identification of Disturbance Covariances Using Maximum Likelihood Estimation Megan A. Zagrobelny and James B. Rawlings
More informationFinancial Econometrics
Financial Econometrics Nonlinear time series analysis Gerald P. Dwyer Trinity College, Dublin January 2016 Outline 1 Nonlinearity Does nonlinearity matter? Nonlinear models Tests for nonlinearity Forecasting
More informationSIMON FRASER UNIVERSITY School of Engineering Science
SIMON FRASER UNIVERSITY School of Engineering Science Course Outline ENSC 810-3 Digital Signal Processing Calendar Description This course covers advanced digital signal processing techniques. The main
More informationv are uncorrelated, zero-mean, white
6.0 EXENDED KALMAN FILER 6.1 Introduction One of the underlying assumptions of the Kalman filter is that it is designed to estimate the states of a linear system based on measurements that are a linear
More informationGaussian Process for Internal Model Control
Gaussian Process for Internal Model Control Gregor Gregorčič and Gordon Lightbody Department of Electrical Engineering University College Cork IRELAND E mail: gregorg@rennesuccie Abstract To improve transparency
More information