STATIC AND DYNAMIC RECURSIVE LEAST SQUARES

Size: px
Start display at page:

Download "STATIC AND DYNAMIC RECURSIVE LEAST SQUARES"

Transcription

1 STATC AND DYNAMC RECURSVE LEAST SQUARES 3rd February Problem #1: additional information Problem At step we want to solve by least squares A 1 b 1 A 1 A 2 b 2 A 2 A x b, A := A, b := b 1 b 2 b with weight matrix W := W 1 W 2 W meaning that measurements at different times (steps) are independent Sizes Matrices A j are m j n We assume that they have maximum column ran Matrices W j are symmetric and positive definite Simple case m j = m for all j, meaning that the number of observations is constant Magnitudes We assume that n is small, whereas m j can be relatively large The aim To approximate A x b we solve (A W A ) 1 A W b The idea is to use x 1 to calculate x in a shorter time 1

2 Notation We write := A j W j A j = A W A j=1 Theorem (A recursion) Beginning with x 1 = P 1 A 1 W 1 b 1, P 1 = (A 1 W 1 A 1 ) 1 the solution is given by the recurrent calculations j = 1,, x j = x j 1 + P j A 1 j W j (b j A j x j 1), Pj = j 1 }{{} + A j W j A j =: K j Proof t is a straightforward verification Remar The vector b j A j x j 1 is the residual of the observations at level j by taing into account the values of the parameters at level j 1 A practical algorithm := A 1 W 1 A 1 x :=solve(, A 1 W 1 b 1 ) for j = 1 : := + A j W j A j r := b j A j x δ :=solve(, A j W j r) x := x + δ end for % residual of recent calculations with new data Computational features Explicit assembling of could be avoided if iterative methods were to be used 2 Problem # 2: dynamic least squares The problem Least squares solution to A 0 F 0 A 1 F 1 F 1 A x 0 x 1 x 1 x b 0 0 b b 2

3 with weight matrix W = W 0 W1 W 1 W W nterpretation Observations in different times (for different magnitudes) are contained in the equations A j x j b j whereas x j+1 F j x j is an approximate dynamic model, relating the sets of parameters Simplified notations A 0 F 0 A := A 1 F 1 For matrices we write, S := F 1 A A 0 F 0 A 1 F 1 F 1 Names smoothed values (at earlier states): x j for j < ; filtered value (at the current state): x ; predicted value (next state in the future, not yet observed): x +1 := F x Theorem (Kalman filter recurrence) Starting with x 0 0 := (A 0 W 0 A 0 ) 1 A 0 W 0 b 0, P 0 0 := (A 0 W 0 A 0 ) 1, the calculations for x at different values of can be carried out by doing: where x 1 := F 1 x 1 1, x := x 1 + K (b A x 1 ) P 1 := F 1 1 F 1 + W 1 K := P 1 A (A P 1 A + W 1 ) 1 Proof See Section 4 P := ( K A )P 1 3

4 Names For the two steps of the method we write: prediction x 1 := F 1 x 1 1 correction x := x 1 + K (b A x 1 ) For the matrices intervening in computations predicted covariance P 1 gain matrix K corrected covariance P Sizes in the typical simple case The number of parameters in the dynamical system is fixed, ie, x j R m for all j, The number of observations in each times step is fixed, ie, b j R n for all j m = #{parameters}, n = #{observations} Hence the sizes of the matrices are as follows: W j is m m Ŵj is n n F j is m m A j is n m P and P 1 are m m K is m n (acts on observations returning parameters) Computational features When n is large and m is small We have to compute C = A P 1 A + W 1, which is n n Typically we have W 1 instead of W To compute K (b A x 1 ) = P 1 A C 1 (b A x 1 ) we solve an n n system To compute K A = P 1 A C 1 A we solve m systems n n All these calculations can be done in parallel 4

5 Steady state We remar that the steady state case, where the dynamics reduces to x j+1 x j, j that is, F j =, j is not equivalent to the static case (Problem #1) 3 Normal equations of the Kalman filter A first observation n the normal equations A W A x = A W b we can write A W A = D 0 U 0 U0 D 1 U 1 U 1 D 1 U 1 U 1 D A 0 W 0 b 0, A A 1 W 1 b 1 W b = = A W b c 0 c 1 c where (taing W 0 = 0) D j := W j + A j W j A j + F j W j+1 F j U j := F j W j+1 D := W + A W A The symbol mars the only element of the matriz which depends on the size () Notice also that for the next time step D = D + F W +1 F Another loo at the normal equations is symmetric bloc tridiagonal D 0 U 0 U0 D 1 U 1 U1 D 1 U 1 U 1 D The system x 0 x 1 x 1 x = c 0 c 1 c 1 c 5

6 Reduced equations Before considering the inclusions of observations at time (A x b ) we can consider the reduced system A 0 F 0 y 0 b 0 A 1 y 1 F 1 0 y 1 b 1 y 0 F 1 Theorem The vector (x 0 1, x 1 1,, x 1 1, F 1 x 1 1 }{{} x 1 is the weighted least square solution to the reduced system ) Proof Denote S as above, and W = W 0 W1 W 1 to the weight matrix The matrix for the normal equations is D 0 U 0 U0 D 1 U 1 S W S = U1 = D 1 U 1 U 1 W = A 1 W 0 0 1A F W F 1 and the right hand side is (c 0,, c 1, 0) t is simple to prove that the proposed solution satisfies the equations W U 1 U 1 W 6

7 A direct bloc method f we only want to solve the last unnown x = x, we can apply an elimination technique to reach the upper bloc triangular form E 0 U 0 x 0 d 0 E 1 U 1 x 1 d 1 = E 1 U 1 E by using the well nown tridiagonal algorithm x 1 x d 1 d E 0 := D 0 d 0 := c 0 for j = 1 : 1 E j := D j Uj 1E 1 j 1 U j 1 d j := c j Uj 1E 1 j 1 d j 1 end for E := D U 1 E 1 1 U 1 d := c U 1 E 1 1 d 1 x :=solve(e, d ) The same elimination process applied to the reduced equations yields E 0 U 0 y 0 d 0 E 1 U 1 y 1 d 1 = where E 1 U 1 H y 1 y d 1 e H := W U 1E 1 1 U 1 = E A W A e := U 1E 1 1 d 1 = d A W b = d c The next step The equality D = D + F W +1 F implies, that the next step only requires the following calculations E := E + F W +1 F D +1 := W +1 + A +1W +1 A +1 U := F W +1 E +1 := D +1 U E 1 U d +1 := c +1 U E 1 d 7

8 The method as a whole Notice that we can write all the steps in the preceding recursive way Notice however that this scheme does not use the solutions at previous states, unlie the true Kalman filter implementation D 0 := A 0 W 0 A 0 E 0 := D 0 d 0 := A 0 W 0 b 0 x 0 :=solve(e 0, d 0 ) for j = 1 : end for E j 1 = E j 1 + F j 1 W j F j 1 D j := W j + A j W j A j U j 1 := F j 1 W j E j := D j U j 1E 1 j 1 U j 1 d j := A j W j b j U j 1E 1 j 1 d j 1 x j :=solve(e j, d j ) Writing the program in a more computational way, we can see the memory requirements of the algorithm A := A 0 ; b := b 0 W := W 0 ; D := A W A E := D d := A W b x :=solve(e, d) for j = 1 : F := F j 1 ; W := W j U := F W E = E + F W F A := A j ; b := b j ; W := W j D := W + A W A E := D U E 1 U d := A W b U E 1 d x :=solve(e, d) end for % storage of observations % storage of weights % storage of the state equation 8

9 4 A proof of the Kalman filter recursion Lemma Let C and P be invertible Then the inverse of Q := P P B (BP B + C 1 ) 1 BP is Q 1 = + B CB Proof t can be easily proven by direct verification Also we can easily see that [ ] [ ] [ ] C 1 + BP B BP 0 P B = P 0 Q ( is used in positions we are not interested in) nverting both sides [ ] [ ] [ ] C CB 0 B C + B = CB 0 Q 1 Comparison of blocs (2, 2) gives the result Theorem For all = E, 1 = H Proof The couple (H, E ) is given by the recurrence H = W W F 1(E 1 + F 1 W F 1 ) 1 F 1 W, E = H + A W A starting at E 0 = A 0 W 0 A 0 On the other hand, the couple (P 1, P ) satisfies the recurrence P 1 = W 1 + F 1 1 F 1, P = P 1 P 1 A (A P 1 A + W 1 ) 1 A P 1 starting with P 0 0 = (A 0 W 0 A 0 ) 1 The lemma clearly shows that both recurrences are equivalent Theorem For all x = ( K A )x 1 + K b Proof Notice first that by the preceding result and the form of the corresponding normal equations and their solutions x = d 1 x 1 = d A W b 9

10 and also ( K A ) = 1 The result is then equivalent to showing that but P d = P d P A W b + K b P A W = K as can be deduced from the definiyos of P and K Further reading and sources Gilbert Strang ntroduction to Applied Mathematics Wellesley Cambridge Gilbert Strang Linear Algebra, Geodesy and GPS Wellesley Cambridge Neil Gershhenfeld The Nature of Mathematical Modeling Cambridge University Press, 1999 (chapter 15) by F J Sayas Mathematics, Ho!, May, 2001 Mathematics, Ho! is a personal project for individual and comunal achievement in higher Maths among Mathematicians, with a bias to Applied Mathematics t is a collection of class notes and small courses do not claim entire originality in these, since they have been much influenced by what report as references Conditions of use You are allowed to use these pages and to share this material This is by no means aimed to be given to undergraduate students, since the style is often too dry Anyway you are requested to quote this as a source whenever you mae extensive use of it Note also that this is permanently wor in progress 10

Time-Varying Systems and Computations Lecture 3

Time-Varying Systems and Computations Lecture 3 Time-Varying Systems and Computations Lecture 3 Klaus Diepold November 202 Linear Time-Varying Systems State-Space System Model We aim to derive the matrix containing the time-varying impulse responses

More information

Derivation of the Kalman Filter

Derivation of the Kalman Filter Derivation of the Kalman Filter Kai Borre Danish GPS Center, Denmark Block Matrix Identities The key formulas give the inverse of a 2 by 2 block matrix, assuming T is invertible: T U 1 L M. (1) V W N P

More information

MATH 802: ENUMERATIVE COMBINATORICS ASSIGNMENT 2

MATH 802: ENUMERATIVE COMBINATORICS ASSIGNMENT 2 MATH 80: ENUMERATIVE COMBINATORICS ASSIGNMENT KANNAPPAN SAMPATH Facts Recall that, the Stirling number S(, n of the second ind is defined as the number of partitions of a [] into n non-empty blocs. We

More information

Representation of doubly infinite matrices as non-commutative Laurent series

Representation of doubly infinite matrices as non-commutative Laurent series Spec. Matrices 217; 5:25 257 Research Article Open Access María Ivonne Arenas-Herrera and Luis Verde-Star* Representation of doubly infinite matrices as non-commutative Laurent series https://doi.org/1.1515/spma-217-18

More information

Some Formulas for the Principal Matrix pth Root

Some Formulas for the Principal Matrix pth Root Int. J. Contemp. Math. Sciences Vol. 9 014 no. 3 141-15 HIKARI Ltd www.m-hiari.com http://dx.doi.org/10.1988/ijcms.014.4110 Some Formulas for the Principal Matrix pth Root R. Ben Taher Y. El Khatabi and

More information

Welsh s problem on the number of bases of matroids

Welsh s problem on the number of bases of matroids Welsh s problem on the number of bases of matroids Edward S. T. Fan 1 and Tony W. H. Wong 2 1 Department of Mathematics, California Institute of Technology 2 Department of Mathematics, Kutztown University

More information

Least Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD

Least Squares. Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter UCSD Least Squares Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 75A Winter 0 - UCSD (Unweighted) Least Squares Assume linearity in the unnown, deterministic model parameters Scalar, additive noise model: y f (

More information

Linear Recurrence Relations for Sums of Products of Two Terms

Linear Recurrence Relations for Sums of Products of Two Terms Linear Recurrence Relations for Sums of Products of Two Terms Yan-Ping Mu College of Science, Tianjin University of Technology Tianjin 300384, P.R. China yanping.mu@gmail.com Submitted: Dec 27, 2010; Accepted:

More information

Math 60. Rumbos Spring Solutions to Assignment #17

Math 60. Rumbos Spring Solutions to Assignment #17 Math 60. Rumbos Spring 2009 1 Solutions to Assignment #17 a b 1. Prove that if ad bc 0 then the matrix A = is invertible and c d compute A 1. a b Solution: Let A = and assume that ad bc 0. c d First consider

More information

A Note on the Determinant of Five-Diagonal Matrices with Fibonacci Numbers

A Note on the Determinant of Five-Diagonal Matrices with Fibonacci Numbers Int. J. Contemp. Math. Sciences, Vol. 3, 2008, no. 9, 419-424 A Note on the Determinant of Five-Diagonal Matrices with Fibonacci Numbers Hacı Civciv Department of Mathematics Faculty of Art and Science

More information

ELA

ELA Volume 18, pp 564-588, August 2009 http://mathtechnionacil/iic/ela GENERALIZED PASCAL TRIANGLES AND TOEPLITZ MATRICES A R MOGHADDAMFAR AND S M H POOYA Abstract The purpose of this article is to study determinants

More information

The k-fibonacci matrix and the Pascal matrix

The k-fibonacci matrix and the Pascal matrix Cent Eur J Math 9(6 0 403-40 DOI: 0478/s533-0-0089-9 Central European Journal of Mathematics The -Fibonacci matrix and the Pascal matrix Research Article Sergio Falcon Department of Mathematics and Institute

More information

F denotes cumulative density. denotes probability density function; (.)

F denotes cumulative density. denotes probability density function; (.) BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models

More information

ON THE MATRIX EQUATION XA AX = X P

ON THE MATRIX EQUATION XA AX = X P ON THE MATRIX EQUATION XA AX = X P DIETRICH BURDE Abstract We study the matrix equation XA AX = X p in M n (K) for 1 < p < n It is shown that every matrix solution X is nilpotent and that the generalized

More information

ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3

ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ISSUED 24 FEBRUARY 2018 1 Gaussian elimination Let A be an (m n)-matrix Consider the following row operations on A (1) Swap the positions any

More information

Linear Prediction Theory

Linear Prediction Theory Linear Prediction Theory Joseph A. O Sullivan ESE 524 Spring 29 March 3, 29 Overview The problem of estimating a value of a random process given other values of the random process is pervasive. Many problems

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY MOORE PENROSE INVERSE OF BIDIAGONAL MATRICES. I

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY MOORE PENROSE INVERSE OF BIDIAGONAL MATRICES. I PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical and Mathematical Sciences 205, 2, p. 20 M a t h e m a t i c s MOORE PENROSE INVERSE OF BIDIAGONAL MATRICES. I Yu. R. HAKOPIAN, S. S. ALEKSANYAN 2, Chair

More information

1 Basic Combinatorics

1 Basic Combinatorics 1 Basic Combinatorics 1.1 Sets and sequences Sets. A set is an unordered collection of distinct objects. The objects are called elements of the set. We use braces to denote a set, for example, the set

More information

2 Introduction of Discrete-Time Systems

2 Introduction of Discrete-Time Systems 2 Introduction of Discrete-Time Systems This chapter concerns an important subclass of discrete-time systems, which are the linear and time-invariant systems excited by Gaussian distributed stochastic

More information

DATA ASSIMILATION FOR FLOOD FORECASTING

DATA ASSIMILATION FOR FLOOD FORECASTING DATA ASSIMILATION FOR FLOOD FORECASTING Arnold Heemin Delft University of Technology 09/16/14 1 Data assimilation is the incorporation of measurement into a numerical model to improve the model results

More information

Dependence and independence

Dependence and independence Roberto s Notes on Linear Algebra Chapter 7: Subspaces Section 1 Dependence and independence What you need to now already: Basic facts and operations involving Euclidean vectors. Matrices determinants

More information

Lecture 4: Linear Algebra 1

Lecture 4: Linear Algebra 1 Lecture 4: Linear Algebra 1 Sourendu Gupta TIFR Graduate School Computational Physics 1 February 12, 2010 c : Sourendu Gupta (TIFR) Lecture 4: Linear Algebra 1 CP 1 1 / 26 Outline 1 Linear problems Motivation

More information

JACOBI S ITERATION METHOD

JACOBI S ITERATION METHOD ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes

More information

Counting Peaks and Valleys in a Partition of a Set

Counting Peaks and Valleys in a Partition of a Set 1 47 6 11 Journal of Integer Sequences Vol. 1 010 Article 10.6.8 Counting Peas and Valleys in a Partition of a Set Toufi Mansour Department of Mathematics University of Haifa 1905 Haifa Israel toufi@math.haifa.ac.il

More information

Yi Wang Department of Applied Mathematics, Dalian University of Technology, Dalian , China (Submitted June 2002)

Yi Wang Department of Applied Mathematics, Dalian University of Technology, Dalian , China (Submitted June 2002) SELF-INVERSE SEQUENCES RELATED TO A BINOMIAL INVERSE PAIR Yi Wang Department of Applied Mathematics, Dalian University of Technology, Dalian 116024, China (Submitted June 2002) 1 INTRODUCTION Pairs of

More information

There is a unique function s(x) that has the required properties. It turns out to also satisfy

There is a unique function s(x) that has the required properties. It turns out to also satisfy Numerical Analysis Grinshpan Natural Cubic Spline Let,, n be given nodes (strictly increasing) and let y,, y n be given values (arbitrary) Our goal is to produce a function s() with the following properties:

More information

Non-stationary extremal eigenvalue approximations in iterative solutions of linear systems and estimators for relative error

Non-stationary extremal eigenvalue approximations in iterative solutions of linear systems and estimators for relative error on-stationary extremal eigenvalue approximations in iterative solutions of linear systems and estimators for relative error Divya Anand Subba and Murugesan Venatapathi* Supercomputer Education and Research

More information

VISION TRACKING PREDICTION

VISION TRACKING PREDICTION VISION RACKING PREDICION Eri Cuevas,2, Daniel Zaldivar,2, and Raul Rojas Institut für Informati, Freie Universität Berlin, ausstr 9, D-495 Berlin, German el 0049-30-83852485 2 División de Electrónica Computación,

More information

Generalized Akiyama-Tanigawa Algorithm for Hypersums of Powers of Integers

Generalized Akiyama-Tanigawa Algorithm for Hypersums of Powers of Integers 1 2 3 47 6 23 11 Journal of Integer Sequences, Vol 16 (2013, Article 1332 Generalized Aiyama-Tanigawa Algorithm for Hypersums of Powers of Integers José Luis Cereceda Distrito Telefónica, Edificio Este

More information

Autocorrelation Functions in GPS Data Processing: Modeling Aspects

Autocorrelation Functions in GPS Data Processing: Modeling Aspects Autocorrelation Functions in GPS Data Processing: Modeling Aspects Kai Borre, Aalborg University Gilbert Strang, Massachusetts Institute of Technology Consider a process that is actually random walk but

More information

POINT VALUES AND NORMALIZATION OF TWO-DIRECTION MULTIWAVELETS AND THEIR DERIVATIVES

POINT VALUES AND NORMALIZATION OF TWO-DIRECTION MULTIWAVELETS AND THEIR DERIVATIVES November 1, 1 POINT VALUES AND NORMALIZATION OF TWO-DIRECTION MULTIWAVELETS AND THEIR DERIVATIVES FRITZ KEINERT AND SOON-GEOL KWON,1 Abstract Two-direction multiscaling functions φ and two-direction multiwavelets

More information

A Matrix Theoretic Derivation of the Kalman Filter

A Matrix Theoretic Derivation of the Kalman Filter A Matrix Theoretic Derivation of the Kalman Filter 4 September 2008 Abstract This paper presents a matrix-theoretic derivation of the Kalman filter that is accessible to students with a strong grounding

More information

Solutions for Chapter 3

Solutions for Chapter 3 Solutions for Chapter Solutions for exercises in section 0 a X b x, y 6, and z 0 a Neither b Sew symmetric c Symmetric d Neither The zero matrix trivially satisfies all conditions, and it is the only possible

More information

2. Introduction to commutative rings (continued)

2. Introduction to commutative rings (continued) 2. Introduction to commutative rings (continued) 2.1. New examples of commutative rings. Recall that in the first lecture we defined the notions of commutative rings and field and gave some examples of

More information

Cramér-Rao Bounds for Estimation of Linear System Noise Covariances

Cramér-Rao Bounds for Estimation of Linear System Noise Covariances Journal of Mechanical Engineering and Automation (): 6- DOI: 593/jjmea Cramér-Rao Bounds for Estimation of Linear System oise Covariances Peter Matiso * Vladimír Havlena Czech echnical University in Prague

More information

Pairs of matrices, one of which commutes with their commutator

Pairs of matrices, one of which commutes with their commutator Electronic Journal of Linear Algebra Volume 22 Volume 22 (2011) Article 38 2011 Pairs of matrices, one of which commutes with their commutator Gerald Bourgeois Follow this and additional works at: http://repository.uwyo.edu/ela

More information

Elements of Multivariate Time Series Analysis

Elements of Multivariate Time Series Analysis Gregory C. Reinsel Elements of Multivariate Time Series Analysis Second Edition With 14 Figures Springer Contents Preface to the Second Edition Preface to the First Edition vii ix 1. Vector Time Series

More information

ORBIT 14 The propagator. A. Milani et al. Dipmat, Pisa, Italy

ORBIT 14 The propagator. A. Milani et al. Dipmat, Pisa, Italy ORBIT 14 The propagator A. Milani et al. Dipmat, Pisa, Italy Orbit 14 Propagator manual 1 Contents 1 Propagator - Interpolator 2 1.1 Propagation and interpolation............................ 2 1.1.1 Introduction..................................

More information

Formula for the inverse matrix. Cramer s rule. Review: 3 3 determinants can be computed expanding by any row or column

Formula for the inverse matrix. Cramer s rule. Review: 3 3 determinants can be computed expanding by any row or column Math 20F Linear Algebra Lecture 18 1 Determinants, n n Review: The 3 3 case Slide 1 Determinants n n (Expansions by rows and columns Relation with Gauss elimination matrices: Properties) Formula for the

More information

Actions and Identities on Set Partitions

Actions and Identities on Set Partitions Actions and Identities on Set Partitions The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher Marberg,

More information

Fisher Information Matrix-based Nonlinear System Conversion for State Estimation

Fisher Information Matrix-based Nonlinear System Conversion for State Estimation Fisher Information Matrix-based Nonlinear System Conversion for State Estimation Ming Lei Christophe Baehr and Pierre Del Moral Abstract In practical target tracing a number of improved measurement conversion

More information

A Piggybacking Design Framework for Read-and Download-efficient Distributed Storage Codes

A Piggybacking Design Framework for Read-and Download-efficient Distributed Storage Codes A Piggybacing Design Framewor for Read-and Download-efficient Distributed Storage Codes K V Rashmi, Nihar B Shah, Kannan Ramchandran, Fellow, IEEE Department of Electrical Engineering and Computer Sciences

More information

Velocity (Kalman Filter) Velocity (knots) Time (sec) Kalman Filter Accelerations 2.5. Acceleration (m/s2)

Velocity (Kalman Filter) Velocity (knots) Time (sec) Kalman Filter Accelerations 2.5. Acceleration (m/s2) KALMAN FILERING 1 50 Velocity (Kalman Filter) 45 40 35 Velocity (nots) 30 25 20 15 10 5 0 0 10 20 30 40 50 60 70 80 90 100 ime (sec) 3 Kalman Filter Accelerations 2.5 Acceleration (m/s2) 2 1.5 1 0.5 0

More information

Matrix Secant Methods

Matrix Secant Methods Equation Solving g(x) = 0 Newton-Lie Iterations: x +1 := x J g(x ), where J g (x ). Newton-Lie Iterations: x +1 := x J g(x ), where J g (x ). 3700 years ago the Babylonians used the secant method in 1D:

More information

Hand Written Digit Recognition using Kalman Filter

Hand Written Digit Recognition using Kalman Filter International Journal of Electronics and Communication Engineering. ISSN 0974-2166 Volume 5, Number 4 (2012), pp. 425-434 International Research Publication House http://www.irphouse.com Hand Written Digit

More information

Elementary Linear Algebra, Second Edition, by Spence, Insel, and Friedberg. ISBN Pearson Education, Inc., Upper Saddle River, NJ.

Elementary Linear Algebra, Second Edition, by Spence, Insel, and Friedberg. ISBN Pearson Education, Inc., Upper Saddle River, NJ. 2008 Pearson Education, Inc., Upper Saddle River, NJ. All rights reserved. APPENDIX: Mathematical Proof There are many mathematical statements whose truth is not obvious. For example, the French mathematician

More information

Sprague-Grundy Values of the R-Wythoff Game

Sprague-Grundy Values of the R-Wythoff Game Sprague-Grundy Values of the R-Wythoff Game Albert Gu Department of Mathematics Carnegie Mellon University Pittsburgh, PA 15213, U.S.A agu@andrew.cmu.edu Submitted: Aug 6, 2014; Accepted: Apr 10, 2015;

More information

Iterative Learning Control for Tailor Rolled Blanks Zhengyang Lu DCT

Iterative Learning Control for Tailor Rolled Blanks Zhengyang Lu DCT Iterative Learning Control for Tailor Rolled Blans Zhengyang Lu DCT 2007.066 Master Internship Report Supervisors: Prof. Maarten Steinbuch (TU/e) Camile Hol (Corus) Eindhoven University of Technology Department

More information

Analytical Study on Method of Solving Polynomial Inequalities

Analytical Study on Method of Solving Polynomial Inequalities IJSTE - International Journal of Science Technology & Engineering Volume 3 Issue 11 May 2017 ISSN (online): 2349-784X Analytical Study on Method of Solving Polynomial Inequalities Mamo Shigute Lecturer

More information

and the compositional inverse when it exists is A.

and the compositional inverse when it exists is A. Lecture B jacques@ucsd.edu Notation: R denotes a ring, N denotes the set of sequences of natural numbers with finite support, is a generic element of N, is the infinite zero sequence, n 0 R[[ X]] denotes

More information

M.A. Botchev. September 5, 2014

M.A. Botchev. September 5, 2014 Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev

More information

Some families of identities for the integer partition function

Some families of identities for the integer partition function MATHEMATICAL COMMUNICATIONS 193 Math. Commun. 0(015), 193 00 Some families of identities for the integer partition function Ivica Martinja 1, and Dragutin Svrtan 1 Department of Physics, University of

More information

Rook Polynomials In Higher Dimensions

Rook Polynomials In Higher Dimensions Grand Valley State University ScholarWors@GVSU Student Summer Scholars Undergraduate Research and Creative Practice 2009 Roo Polynomials In Higher Dimensions Nicholas Krzywonos Grand Valley State University

More information

State Estimation of Linear and Nonlinear Dynamic Systems

State Estimation of Linear and Nonlinear Dynamic Systems State Estimation of Linear and Nonlinear Dynamic Systems Part II: Observability and Stability James B. Rawlings and Fernando V. Lima Department of Chemical and Biological Engineering University of Wisconsin

More information

Computing the pth Roots of a Matrix. with Repeated Eigenvalues

Computing the pth Roots of a Matrix. with Repeated Eigenvalues Applied Mathematical Sciences, Vol. 5, 2011, no. 53, 2645-2661 Computing the pth Roots of a Matrix with Repeated Eigenvalues Amir Sadeghi 1, Ahmad Izani Md. Ismail and Azhana Ahmad School of Mathematical

More information

Analysis of Carleman Representation of Analytical Recursions

Analysis of Carleman Representation of Analytical Recursions JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 224, 8190 1998 ARTICLE NO. AY985986 Analysis of Carleman Representation of Analytical Recursions G. Berolaio Department of Physics, Bar-Ilan Uniersity,

More information

Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4

Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4 Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix

More information

Background on Linear Algebra - Lecture 2

Background on Linear Algebra - Lecture 2 Background on Linear Algebra - Lecture September 6, 01 1 Introduction Recall from your math classes the notion of vector spaces and fields of scalars. We shall be interested in finite dimensional vector

More information

AN ALGEBRAIC INTERPRETATION OF THE q-binomial COEFFICIENTS. Michael Braun

AN ALGEBRAIC INTERPRETATION OF THE q-binomial COEFFICIENTS. Michael Braun International Electronic Journal of Algebra Volume 6 (2009) 23-30 AN ALGEBRAIC INTERPRETATION OF THE -BINOMIAL COEFFICIENTS Michael Braun Received: 28 February 2008; Revised: 2 March 2009 Communicated

More information

Math 4377/6308 Advanced Linear Algebra

Math 4377/6308 Advanced Linear Algebra 3.1 Elementary Matrix Math 4377/6308 Advanced Linear Algebra 3.1 Elementary Matrix Operations and Elementary Matrix Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/

More information

c 1998 Society for Industrial and Applied Mathematics

c 1998 Society for Industrial and Applied Mathematics SIAM J MARIX ANAL APPL Vol 20, No 1, pp 182 195 c 1998 Society for Industrial and Applied Mathematics POSIIVIY OF BLOCK RIDIAGONAL MARICES MARIN BOHNER AND ONDŘEJ DOŠLÝ Abstract his paper relates disconjugacy

More information

State Estimation by IMM Filter in the Presence of Structural Uncertainty 1

State Estimation by IMM Filter in the Presence of Structural Uncertainty 1 Recent Advances in Signal Processing and Communications Edited by Nios Mastorais World Scientific and Engineering Society (WSES) Press Greece 999 pp.8-88. State Estimation by IMM Filter in the Presence

More information

Residual statics analysis by LSQR

Residual statics analysis by LSQR Residual statics analysis by LSQR Residual statics analysis by LSQR Yan Yan, Zhengsheng Yao, Gary F. Margrave and R. James Brown ABSRAC Residual statics corrections can be formulated as a linear inverse

More information

FERMENTATION BATCH PROCESS MONITORING BY STEP-BY-STEP ADAPTIVE MPCA. Ning He, Lei Xie, Shu-qing Wang, Jian-ming Zhang

FERMENTATION BATCH PROCESS MONITORING BY STEP-BY-STEP ADAPTIVE MPCA. Ning He, Lei Xie, Shu-qing Wang, Jian-ming Zhang FERMENTATION BATCH PROCESS MONITORING BY STEP-BY-STEP ADAPTIVE MPCA Ning He Lei Xie Shu-qing Wang ian-ming Zhang National ey Laboratory of Industrial Control Technology Zhejiang University Hangzhou 3007

More information

PRIME-REPRESENTING FUNCTIONS

PRIME-REPRESENTING FUNCTIONS Acta Math. Hungar., 128 (4) (2010), 307 314. DOI: 10.1007/s10474-010-9191-x First published online March 18, 2010 PRIME-REPRESENTING FUNCTIONS K. MATOMÄKI Department of Mathematics, University of Turu,

More information

A Local Inverse Formula and a Factorization

A Local Inverse Formula and a Factorization A Local Inverse Formula and a Factorization arxiv:60.030v [math.na] 4 Oct 06 Gilbert Strang and Shev MacNamara With congratulations to Ian Sloan! Abstract When a matrix has a banded inverse there is a

More information

City, University of London Institutional Repository

City, University of London Institutional Repository City Research Online City, University of London Institutional Repository Citation: Zhao, S., Shmaliy, Y. S., Khan, S. & Liu, F. (2015. Improving state estimates over finite data using optimal FIR filtering

More information

DETERMINISTIC REPRESENTATION FOR POSITION DEPENDENT RANDOM MAPS

DETERMINISTIC REPRESENTATION FOR POSITION DEPENDENT RANDOM MAPS DETERMINISTIC REPRESENTATION FOR POSITION DEPENDENT RANDOM MAPS WAEL BAHSOUN, CHRISTOPHER BOSE, AND ANTHONY QUAS Abstract. We give a deterministic representation for position dependent random maps and

More information

A Fast Implicit QR Eigenvalue Algorithm for Companion Matrices

A Fast Implicit QR Eigenvalue Algorithm for Companion Matrices A Fast Implicit QR Eigenvalue Algorithm for Companion Matrices D. A. Bini a,1 P. Boito a,1 Y. Eidelman b L. Gemignani a,1 I. Gohberg b a Dipartimento di Matematica, Università di Pisa, Largo Bruno Pontecorvo

More information

hal , version 1-29 Feb 2008

hal , version 1-29 Feb 2008 Compressed Modular Matrix Multiplication Jean-Guillaume Dumas Laurent Fousse Bruno Salvy February 29, 2008 Abstract We propose to store several integers modulo a small prime into a single machine word.

More information

Aggregate Set-utility Fusion for Multi-Demand Multi-Supply Systems

Aggregate Set-utility Fusion for Multi-Demand Multi-Supply Systems Aggregate Set-utility Fusion for Multi- Multi- Systems Eri P. Blasch BEAR Consulting 393 Fieldstone Cir, Fairborn, O 4534 eri.blasch@sensors.wpafb.af.mil Abstract Microeconomic theory develops demand and

More information

Information Formulation of the UDU Kalman Filter

Information Formulation of the UDU Kalman Filter Information Formulation of the UDU Kalman Filter Christopher D Souza and Renato Zanetti 1 Abstract A new information formulation of the Kalman filter is presented where the information matrix is parameterized

More information

are the q-versions of n, n! and . The falling factorial is (x) k = x(x 1)(x 2)... (x k + 1).

are the q-versions of n, n! and . The falling factorial is (x) k = x(x 1)(x 2)... (x k + 1). Lecture A jacques@ucsd.edu Notation: N, R, Z, F, C naturals, reals, integers, a field, complex numbers. p(n), S n,, b(n), s n, partition numbers, Stirling of the second ind, Bell numbers, Stirling of the

More information

AMS Mathematics Subject Classification : 65F10,65F50. Key words and phrases: ILUS factorization, preconditioning, Schur complement, 1.

AMS Mathematics Subject Classification : 65F10,65F50. Key words and phrases: ILUS factorization, preconditioning, Schur complement, 1. J. Appl. Math. & Computing Vol. 15(2004), No. 1, pp. 299-312 BILUS: A BLOCK VERSION OF ILUS FACTORIZATION DAVOD KHOJASTEH SALKUYEH AND FAEZEH TOUTOUNIAN Abstract. ILUS factorization has many desirable

More information

WORD: EXAMPLE(S): COUNTEREXAMPLE(S): EXAMPLE(S): COUNTEREXAMPLE(S): WORD: EXAMPLE(S): COUNTEREXAMPLE(S): EXAMPLE(S): COUNTEREXAMPLE(S): WORD:

WORD: EXAMPLE(S): COUNTEREXAMPLE(S): EXAMPLE(S): COUNTEREXAMPLE(S): WORD: EXAMPLE(S): COUNTEREXAMPLE(S): EXAMPLE(S): COUNTEREXAMPLE(S): WORD: Bivariate Data DEFINITION: In statistics, data sets using two variables. Scatter Plot DEFINITION: a bivariate graph with points plotted to show a possible relationship between the two sets of data. Positive

More information

NEURAL NETWORKS FOR TRANSMISSION OVER NONLINEAR MIMO CHANNELS

NEURAL NETWORKS FOR TRANSMISSION OVER NONLINEAR MIMO CHANNELS NEURAL NETWORKS FOR TRANSISSION OVER NONLINEAR IO CHANNELS by AL UKHTAR AL-HINAI A thesis submitted to the Department of Electrical and Computer Engineering in conformity with the requirements for the

More information

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Worst Case and Average Case Behavior of the Simplex Algorithm

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Worst Case and Average Case Behavior of the Simplex Algorithm Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebrasa-Lincoln Lincoln, NE 68588-030 http://www.math.unl.edu Voice: 402-472-373 Fax: 402-472-8466 Topics in Probability Theory and

More information

Discrete Structures Lecture Sequences and Summations

Discrete Structures Lecture Sequences and Summations Introduction Good morning. In this section we study sequences. A sequence is an ordered list of elements. Sequences are important to computing because of the iterative nature of computer programs. The

More information

Minimum Repair Bandwidth for Exact Regeneration in Distributed Storage

Minimum Repair Bandwidth for Exact Regeneration in Distributed Storage 1 Minimum Repair andwidth for Exact Regeneration in Distributed Storage Vivec R Cadambe, Syed A Jafar, Hamed Malei Electrical Engineering and Computer Science University of California Irvine, Irvine, California,

More information

Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016

Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016 Orthogonal Projection and Least Squares Prof. Philip Pennance 1 -Version: December 12, 2016 1. Let V be a vector space. A linear transformation P : V V is called a projection if it is idempotent. That

More information

Numerical Method for Fuzzy Partial Differential Equations

Numerical Method for Fuzzy Partial Differential Equations Applied Mathematical Sciences, Vol. 1, 2007, no. 27, 1299-1309 Numerical Method for Fuzzy Partial Differential Equations M. Afshar Kermani 1 and F. Saburi Department of Mathematics Science and Research

More information

Adaptive ensemble Kalman filtering of nonlinear systems. Tyrus Berry and Timothy Sauer George Mason University, Fairfax, VA 22030

Adaptive ensemble Kalman filtering of nonlinear systems. Tyrus Berry and Timothy Sauer George Mason University, Fairfax, VA 22030 Generated using V3.2 of the official AMS LATEX template journal page layout FOR AUTHOR USE ONLY, NOT FOR SUBMISSION! Adaptive ensemble Kalman filtering of nonlinear systems Tyrus Berry and Timothy Sauer

More information

Consider the following example of a linear system:

Consider the following example of a linear system: LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x + 2x 2 + 3x 3 = 5 x + x 3 = 3 3x + x 2 + 3x 3 = 3 x =, x 2 = 0, x 3 = 2 In general we want to solve n equations

More information

Fraction-free Computation of Matrix Padé Systems

Fraction-free Computation of Matrix Padé Systems Fraction-free Computation of Matrix Padé Systems Bernhard Becermann, Laboratoire d Analyse Numérique et d Optimisation, UFR IEEA M3, USTL Flres Artois, F 59655 Villeneuve d Ascq CEDEX, France, e-mail:

More information

Huber s M-estimation in relative GPS positioning: computational aspects. 1 Introduction

Huber s M-estimation in relative GPS positioning: computational aspects. 1 Introduction Huber s M-estimation in relative GPS positioning: computational aspects X-W Chang, Y Guo School of Computer Science, McGill University, 3480 University Street, Montreal, Quebec, Canada H3A 2A7 Email: chang@csmcgillca,

More information

arxiv:math/ v1 [math.co] 7 Jul 2004

arxiv:math/ v1 [math.co] 7 Jul 2004 Newton s Method as a Formal Recurrence arxiv:math/0407115v1 [math.co] 7 Jul 2004 Hal Canary, Carl Eduist, Samuel Lachterman, Brendan Younger February 1, 2008 Abstract Iterating Newton s method symbolically

More information

Pafnuty Lvovich Chebyshev, a Russian mathematician,

Pafnuty Lvovich Chebyshev, a Russian mathematician, DELVING deeper Benjamin Sinwell The Chebyshev Polynomials: Patterns and Derivation Pafnuty Lvovich Chebyshev, a Russian mathematician, is famous for his wor in the area of number theory and for his wor

More information

Part I State space models

Part I State space models Part I State space models 1 Introduction to state space time series analysis James Durbin Department of Statistics, London School of Economics and Political Science Abstract The paper presents a broad

More information

Stationary Probabilities of Markov Chains with Upper Hessenberg Transition Matrices

Stationary Probabilities of Markov Chains with Upper Hessenberg Transition Matrices Stationary Probabilities of Marov Chains with Upper Hessenberg Transition Matrices Y. Quennel ZHAO Department of Mathematics and Statistics University of Winnipeg Winnipeg, Manitoba CANADA R3B 2E9 Susan

More information

October 7, :8 WSPC/WS-IJWMIP paper. Polynomial functions are renable

October 7, :8 WSPC/WS-IJWMIP paper. Polynomial functions are renable International Journal of Wavelets, Multiresolution and Information Processing c World Scientic Publishing Company Polynomial functions are renable Henning Thielemann Institut für Informatik Martin-Luther-Universität

More information

Differential Resultants and Subresultants

Differential Resultants and Subresultants 1 Differential Resultants and Subresultants Marc Chardin Équipe de Calcul Formel SDI du CNRS n 6176 Centre de Mathématiques et LIX École Polytechnique F-91128 Palaiseau (France) e-mail : chardin@polytechniquefr

More information

Krylov Subspace Methods that Are Based on the Minimization of the Residual

Krylov Subspace Methods that Are Based on the Minimization of the Residual Chapter 5 Krylov Subspace Methods that Are Based on the Minimization of the Residual Remark 51 Goal he goal of these methods consists in determining x k x 0 +K k r 0,A such that the corresponding Euclidean

More information

Maths for Signals and Systems Linear Algebra for Engineering Applications

Maths for Signals and Systems Linear Algebra for Engineering Applications Maths for Signals and Systems Linear Algebra for Engineering Applications Lectures 1-2, Tuesday 11 th October 2016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE LONDON

More information

the method of steepest descent

the method of steepest descent MATH 3511 Spring 2018 the method of steepest descent http://www.phys.uconn.edu/ rozman/courses/m3511_18s/ Last modified: February 6, 2018 Abstract The Steepest Descent is an iterative method for solving

More information

17 Solution of Nonlinear Systems

17 Solution of Nonlinear Systems 17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m

More information

For final project discussion every afternoon Mark and I will be available

For final project discussion every afternoon Mark and I will be available Worshop report 1. Daniels report is on website 2. Don t expect to write it based on listening to one project (we had 6 only 2 was sufficient quality) 3. I suggest writing it on one presentation. 4. Include

More information

Chapter 2: Linear Independence and Bases

Chapter 2: Linear Independence and Bases MATH20300: Linear Algebra 2 (2016 Chapter 2: Linear Independence and Bases 1 Linear Combinations and Spans Example 11 Consider the vector v (1, 1 R 2 What is the smallest subspace of (the real vector space

More information

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M. TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION

More information

Math x + 3y 5z = 14 3x 2y + 3z = 17 4x + 3y 2z = 1

Math x + 3y 5z = 14 3x 2y + 3z = 17 4x + 3y 2z = 1 Math 210 1. Solve the system: x + y + z = 1 2x + 3y + 4z = 5 (a z = 2, y = 1 and x = 0 (b z =any value, y = 3 2z and x = z 2 (c z =any value, y = 3 2z and x = z + 2 (d z =any value, y = 3 + 2z and x =

More information

{ ϕ(f )Xt, 1 t n p, X t, n p + 1 t n, { 0, t=1, W t = E(W t W s, 1 s t 1), 2 t n.

{ ϕ(f )Xt, 1 t n p, X t, n p + 1 t n, { 0, t=1, W t = E(W t W s, 1 s t 1), 2 t n. Innovations, Sufficient Statistics, And Maximum Lielihood In ARM A Models Georgi N. Boshnaov Institute of Mathematics, Bulgarian Academy of Sciences 1. Introduction. Let X t, t Z} be a zero mean ARMA(p,

More information