A Simplified Derivation of Scalar Kalman Filter using Bayesian Probability Theory

Similar documents
Chandrasekhar Type Algorithms. for the Riccati Equation of Lainiotis Filter

Bayesian Methods: Introduction to Multi-parameter Models

Discrete-Time Systems, LTI Systems, and Discrete-Time Convolution

THE KALMAN FILTER RAUL ROJAS

Lainiotis filter implementation. via Chandrasekhar type algorithm

Recursive Updating Fixed Parameter

System Diagnostics using Kalman Filter Estimation Error

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

13.4 Scalar Kalman Filter

3/8/2016. Contents in latter part PATTERN RECOGNITION AND MACHINE LEARNING. Dynamical Systems. Dynamical Systems. Linear Dynamical Systems

ECE 8527: Introduction to Machine Learning and Pattern Recognition Midterm # 1. Vaishali Amin Fall, 2015

Recurrence Relations

Chapter 6 Sampling Distributions

Lecture 2: Monte Carlo Simulation

Dept. of Biomed. Eng. BME801: Inverse Problems in Bioengineering Kyung Hee Univ.

Lectures on Stochastic System Analysis and Bayesian Updating

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY

MATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4

Frequency Response of FIR Filters

Lecture 4. Hw 1 and 2 will be reoped after class for every body. New deadline 4/20 Hw 3 and 4 online (Nima is lead)

Overview of Estimation

Chapter 6 Principles of Data Reduction

Introduction to Signals and Systems, Part V: Lecture Summary

Bayesian Control Charts for the Two-parameter Exponential Distribution

Chapter 10 Advanced Topics in Random Processes

Statistical Noise Models and Diagnostics

ECON 3150/4150, Spring term Lecture 3

Local ensemble Kalman filtering in the presence of model bias

R. van Zyl 1, A.J. van der Merwe 2. Quintiles International, University of the Free State

Regression and generalization

CSE 527, Additional notes on MLE & EM

Chapter 12 EM algorithms The Expectation-Maximization (EM) algorithm is a maximum likelihood method for models that have hidden variables eg. Gaussian

Random Signals and Noise Winter Semester 2017 Problem Set 12 Wiener Filter Continuation

Uncertainties in Seismicity Models: Towards Bayesian Earthquake Forecasting

Standard Bayesian Approach to Quantized Measurements and Imprecise Likelihoods

Statistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons

GRAPHING LINEAR EQUATIONS. Linear Equations ( 3,1 ) _x-axis. Origin ( 0, 0 ) Slope = change in y change in x. Equation for l 1.

1.010 Uncertainty in Engineering Fall 2008

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

6.867 Machine learning, lecture 7 (Jaakkola) 1

REGRESSION (Physics 1210 Notes, Partial Modified Appendix A)

5.1 Review of Singular Value Decomposition (SVD)

(all terms are scalars).the minimization is clearer in sum notation:

Minimax Estimation of the Parameter of Maxwell Distribution Under Different Loss Functions

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science. BACKGROUND EXAM September 30, 2004.

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Properties and Hypothesis Testing

ELEG3503 Introduction to Digital Signal Processing

ECE-S352 Introduction to Digital Signal Processing Lecture 3A Direct Solution of Difference Equations

An Introduction to Randomized Algorithms

Lecture 11 and 12: Basic estimation theory

Direction of Arrival Estimation Method in Underdetermined Condition Zhang Youzhi a, Li Weibo b, Wang Hanli c

Maximum likelihood estimation from record-breaking data for the generalized Pareto distribution

On an Application of Bayesian Estimation

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition

Problem Set 4 Due Oct, 12

Chapter 6: Determinants and the Inverse Matrix 1

Signal Processing in Mechatronics

Sequential Monte Carlo Methods - A Review. Arnaud Doucet. Engineering Department, Cambridge University, UK

Correlation Regression

On Orlicz N-frames. 1 Introduction. Renu Chugh 1,, Shashank Goel 2

independence of the random sample measurements, we have U = Z i ~ χ 2 (n) with σ / n 1. Now let W = σ 2. We then have σ 2 (x i µ + µ x ) 2 i =1 ( )

1 Inferential Methods for Correlation and Regression Analysis

Section 14. Simple linear regression.

Mathematical Modeling of Optimum 3 Step Stress Accelerated Life Testing for Generalized Pareto Distribution

by Paul D. Abrm"l.2, Jr. SIMULTANEOUS ESTIMATION OF / IN LINEAR DYNAMICAL SYSTEMS Electronics Research Ce~2ter Cam bridge, MUSS.

The z-transform. 7.1 Introduction. 7.2 The z-transform Derivation of the z-transform: x[n] = z n LTI system, h[n] z = re j

AN OPEN-PLUS-CLOSED-LOOP APPROACH TO SYNCHRONIZATION OF CHAOTIC AND HYPERCHAOTIC MAPS

Grouping 2: Spectral and Agglomerative Clustering. CS 510 Lecture #16 April 2 nd, 2014

Estimation for Complete Data

RAINFALL PREDICTION BY WAVELET DECOMPOSITION

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5

Spectral Analysis. This week in lab. Next classes: 3/26 and 3/28. Your next experiment Homework is to prepare

5. Fast NLMS-OCF Algorithm

The standard deviation of the mean

Mechatronics. Time Response & Frequency Response 2 nd -Order Dynamic System 2-Pole, Low-Pass, Active Filter

Statistical Inferences for Lomax Distribution Based on Record Values (Bayesian and Classical)

Statistical Pattern Recognition

Mixtures of Gaussians and the EM Algorithm

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE

SDS 321: Introduction to Probability and Statistics

Exponential Families and Bayesian Inference

HUMAN COMPUTER INTERACTION

Multilevel ensemble Kalman filtering

DIGITAL SIGNAL PROCESSING LECTURE 3

Filter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and

MANEUVERING TARGET TRACK PREDICTION MODEL

Expectation and Variance of a random variable

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

FIR Filter Design: Part II

Topics Machine learning: lecture 2. Review: the learning problem. Hypotheses and estimation. Estimation criterion cont d. Estimation criterion

Lecture 3. Digital Signal Processing. Chapter 3. z-transforms. Mikael Swartling Nedelko Grbic Bengt Mandersson. rev. 2016

A Slight Extension of Coherent Integration Loss Due to White Gaussian Phase Noise Mark A. Richards

Parameter, Statistic and Random Samples

Factor Analysis. Lecture 10: Factor Analysis and Principal Component Analysis. Sam Roweis

The Expectation-Maximization (EM) Algorithm

Random Variables, Sampling and Estimation

Linear Regression Demystified

Regression and correlation

Transcription:

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede A Simplified Derivatio of Scalar Kalma Filter usig Bayesia Proaility Theory Gregory Kyriazis Imetro - Electrical Metrology Divisio

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede. Itroductio. Liear dyamical models 3. Estimatio 4. Coclusios

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede. Itroductio 960 R. E. Kalma - A ew approach to liear filterig ad predictio prolems 963 R. E. Kalma - New methods i Wieer filterig theory Kalma receives Natioal Medal of Sciece Octoer 7 009 I may estimatio prolems especially those ivolvig dyamical systems oservatios are made sequetially i time ad up-to-date parameter estimates are required. The recursive solutio to the discrete-time liear estimatio prolem was first pulished y Kalma. The estimatio algorithm is called Kalma filter.

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede Kalma filter The Kalma filterig algorithm made it possile to avigate precisely over log distaces ad time spas. Kalma s algorithm is used extesively i all avigatio systems for deep-space exploratio. It has also ee applied to forecastig (P. J. Harriso ad C. F. Steves 976). Historically this sigal estimatio prolem was viewed as filterig arrowad sigals from widead oise; hece the ame filterig for sigal estimatio. Kalma used orthogoality to derive his filterig algorithm (R. E. Kalma 960). Kalma s equatios have ofte ee derived usig iovatios. The cocept of iovatios or the upredictale part of oservatios was itroduced y Kailath (968). Kalma s equatios were derived usig a Bayesia approach for the first time i NASA TR R-35 (96). A similar approach was used y Y. C. Ho ad R. C. K. Lee (964). A simplified derivatio is preseted here.

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede. Liear dyamical models Liear dyamical models are state-space models whose state upredictale vagaries with time are descried proailistically X Z X Z a - w a Z ~ N 0 Z w w ~ N 0 Oservatio equatio System equatio oservatio sequece Gauss-Markov sequece of ukow process states kow series of costats (liear relatioship) kow series of costats (first-order differece equatio) oise sequece system oise drivig fuctio Both w ad are white ad mutually ucorrelated.

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede. Estimatio X Z ukow values of the correspodig quatities possile values of those ukows Let d = {d X = x } with d 0 descriig the iitial availale iformatio icludig the values of a - ad. All the iformatio d aout the ukow state is ecoded y the posterior PDF at ad used to derive the ew posterior oce the data sample X = x is received at. It is show i the sequel how to evolve from the posterior PDF at to the posterior at. It is assumed that iitially at time = iformatio cocerig the state Z 0 was descried as 0 N 0 0 Z ~ ν (mea ad variace kow)

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede Prior PDF for the process state at ~ exp Z d p ~ a a This follows immediately from the system equatio ad the properties of the Gaussia distriutio. Posterior PDF for the process state at exp p Z d

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede A samplig distriutio with ukow locatio ad scale parameters is assiged that descries the prior kowledge aout the oise. As a fuctio of those parameters the samplig distriutio is the termed the likelihood of the parameters give the oserved data. Likelihood for Z give that X = x l x exp x ; Noise ad prior iformatio are comied at each time with Bayes theorem i.e. posterior prior likelihood. O comiig the prior ad the likelihood completig the square ad lumpig the terms that do ot deped o ito the proportioality costat

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede exp Z d p Posterior PDF for the process state at ~ x The posterior mea is the weighted average of the prior mea ad x / with the weights eig ad / respectively. For fast recursive estimatio oe is just iterested i the estimate ad the associated ucertaity. I dyamical prolems each state ca e estimated from the last previous estimate ad the ew data sample received. Thus oly the last estimate ad associated ucertaity eed to e stored.

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede σ We ow have all the equatios required to recursively geerate the solutio to the estimatio prolem: Recursive equatios ν x a a It is assumed that the state estimate at a ad x are all kow at. est estimate of the state at ucertaity associated with the estimate

6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede Scalar Kalma filter a a x a If a - does ot vary with ad w ad are oth statioary that is ad are oth costats the oth ad will approach limits as approaches ifiity. By defiig the Kalma gai as the state estimate at ad the associated ucertaity ecome

Tale I. Parameter estimates for a sigal cotaiig five frequecies. 6th Iteratioal Workshop o Aalysis of Dyamic Measuremets Jue -3 0 Göteorg Swede 4. Coclusio The Bayesia approach preseted here provides a simpler ad more direct derivatio of the solutio to the prolem of recursive estimatio of the state of a firstorder liear dyamical system. It was show that as expected the solutio is Kalma s filterig algorithm.