Filtering and Identification

Transcription

1 Filtering and Identification Day 1 - Lecture 1: Introduction and refreshment LA Michel Verhaegen 1/42

2 Smart Optics Systems Star Telescope / Collimator Plane wavefront Turbulent Atmosphere Disturbed wavefront Deformable mirror Controller Tip tilt mirror Beam splitter Camera Wavefront sensor Adaptive Optics Active correction of wavefront aberrations by a deformable mirror. What is needed from a control engineer? 2/42

3 Lithography Challenge: Aberration correction due to deformations in the mirrors caused by the heating of the Light Source (pm accuracy for 32nm technology! Internships possible! 3/42

4 Photondetector Confocal pin hole Microscopy Lens Pin hole conjugated to the focal point (rejection of out-of-focus emission) 3-dimensional pointwise scanning (image formed by points) Laser source Dichroic beam splitter Confocal to widefield: Objective 3D scanning Focal plane Specimen (Image courtesy: AU: Airy-disk U 4/42

5 Teaching Staff (DCSC) Lecturer Prof.dr.ir. Michel Verhaegen Teaching Assistants (TA s): Jonas Calimer Karl Granström 5/42

6 Objectives of the course After studying this course you should be able to derive estimation, filtering and identification a algorithms based on the the linear least squares method a And control (H 2, etc.) 6/42

7 Course material Book: Filtering and System Identification: An Introduction, by Michel Verhaegen and Vincent Verdult, Cambridge University Press, hand-outs or local blackboard? 7/42

8 Outline of the course This intensive course will run for a week; with morning lectures and homework in the afternoon. Day 1: LA review and Deterministic Linear Least Squares Day 2: Stochastic Least Squares and Kalman filtering 8/42

9 Outline of the course (C td) Day 3: Use of the Kalman filter and optimal predictors for input-output models Day 4: Deterministic Subspace Identification and a framework for consistency analysis Day 5: Instrumental variables in Subspace identification and probing some future developments No Homework! 9/42

10 Exam Four sets of homeworks: Hand-in sets on morning of the next day to the Lecturer. 10/42

11 Filtering and identification Let s start! 11/42

12 System identification? in a general context The art to extract missing information by inspection with the goal to... in a scientific context The art to extract mathematical models from measurements derived by experimentation with physical phenomenon one wants to understand/control (GOAL!) 12/42

13 Identification cycle DATA GENERATION MODEL SELECTION AND ESTIMATION MODEL VALIDATION USE THE MODEL 13/42

14 Mathematical ingredients? (Linear) Least Squares min x ǫ T ǫ y = Fx+ǫ y Matrix Theory Probability Theory Signal/System Theory Domain Knowledge F f 2 f 1 14/42

15 Overview Linear Algebra (LA) The matrix concept! The Usefull matrix factorization: The SVD A Quick view on its potential! Matrix-crimes The Useful matrix Lemma 15/42

16 Matrix theory: Some history Matrix is Latin for womb (matrix = mögel, grogrund,matris) Chinese used matrix methods already in [200 BC 300AD]. 1. They used concepts like determinants of a table of numbers 2. Determinant was long known to be invented by Japanese Seki Kowa /42

17 Matrix theory: Some history The term Matrix was first introduced by James Sylvester 1850 Mr. J.J. Sylvester on a new Class of Theorems www history.mcs.st and.ac.uk/ history/ Phil. Mag. S. 6, Vol 37, No. 251, Nov /42

18 Definition of a matrix What it is not? 18/42

19 Definition of a matrix A matrix A R m n is a two-dimensional table of numbers: a 11 a 12 a 1n a 21 a 22 a [ ] 2n A =... = a 1 a 2 a n a m1 a m2 a mn with a ij R, a i R m. 19/42

20 A matrix represents a (linear) mapping A matrix is (also) a mapping between two Euclidean vector spaces: A : R n R m : x R n, y R m : Ax = y R n "A" m R /42

21 The Four key spaces of a linear mapping R n m The linear mapping: A : R n R "A" ( domain ) R m ( Image or Range space ) is characterized by four 0 0 subspaces: range(a) = {y R m : y = Ax for some x R n } range(a T ) = {x R n : x = A T y for some y R m } ker(a) = {x R n : Ax = 0} ker(a T ) = {y R m : A T y = 0} The rank of A equals the dimension of range(a). 21/42

22 Special class of matrices Definition: An square matrix Q R n n is orthogonal if Q T Q = QQ T = I n This means: 1. Each column vector of an orthogonal matrix has length? 2. Two different column (row) vectors of an orthogonal matrix satisfy? 3. What is the inverse of an orthogonal matrix? 4. And many more useful (numerical) advantages... 22/42

23 Overview Linear Algebra (LA) The matrix concept! The Usefull matrix factorization: The SVD A Quick view on its potential! Matrix-crimes The Useful matrix Lemma 23/42

24 The Singular value decomposition (SVD) The SVD-Theorem: Let A R m n, then there exists a pair of orthogonal matrices: ] U = [u 1 u m R m m : UU T = U T U = I m ] V = [v 1 v n R n n : VV T = V T V = I n such that, U T AV = [ ] Σ R m n, Σ = diag(σ 1,,σ p ) with σ 1 σ 2 σ p 0 and p = min(m,n). 24/42

25 A = Example SVD }{{} U [U,Sigma,V]=svd(A); A = } {{ } Σ } {{ } V T Column vectors of the matrix U: left singular vectors Column vectors of the matrix V : right singular vectors Diagonal elements of Σ: the singular values T 25/42

26 RangeDemo.m 26/42

27 Observations from RangeDemo.m ( ) columns of A lie in a plane R 3 dim span col (A) = 2 # non-zero singular values (sv s) = 2 the left singular vectors u 1,u 2 corresponding to the non-zero singular values: 2 A = σ i u i vi T i=1 form an orthogonal basis for span col (A). the left singular vector u 3 corresponding to the zero singular value (i = 3) is a basis for ker(a T ). the left (and right) singular vectors are orthogonal and are of unit length. 27/42

28 The four key subspaces Let the SVD of the matrix A be given as, [ ] A = U 1 U 2 Σ 1 0 V 1 T 0 0 V2 T with Σ1 > 0 then, since Ax = ( U 1 ( Σ1 (V T 1 x))), range(a) = {y R m : y = Ax for some x R n } = span(u 1 ) Further, since for x = V 2 α : Ax = U 1 Σ 1 V T 1 V 2α = 0, ker(a) = {x R n : Ax = 0} = span(v 2 ) R n "A" V 1 U 1 V 2 U m R 28/42

29 The SVD: the workhorse for reliable calculations Contrary to the eigenvalue decomposition, the determinant, etc. the SVD allows for a numerically reliable calculus. Example: Checking the singularity of a matrix A: The notion det(a) is often used to signal the singularity of a matrix. This is only true in the case it is exactly zero! 29/42

30 Checking Singularity (Ct d) Example: Consider the square matrix: A =.... R n n Then det(a) equals 1. But the condition number of the matrix A defined as: κ α (A) = A α A 1 α for α = 1,2, and A α = sup x 0 Ax α x α equals: κ (A) = n2 n 1 30/42

31 Condition number of a matrix Definition: For a general matrix A R m n (m n), its condition number κ 2 (A) (in short κ(a)) is given as: κ(a) = A 2 A 2 where A denotes the pseudo-inverse of a matrix, i.e. satisfying, AA A = A A AA = A (AA ) T = AA (A A) T = A A If A is full rank, then A = (A T A) 1 A T. Exercise: Check that κ(a) = σ 1 σ n! 31/42

32 Overview Linear Algebra (LA) The matrix concept! The Usefull matrix factorization: The SVD A Quick view on its potential Matrix-crimes The Useful matrix Lemma 32/42

33 Optimal low rank approximation Theorem: Let the SVD in the SVD-theorem be given and let k < rank(a) and let the following approximation A k of A be given: A k = k i=1 σ i u i v T i then, min rank(b)=k A B 2 = A A k 2 = σ k+1 33/42

34 Spiegelman.m 34/42

35 Overview Linear Algebra (LA) The matrix concept! The Usefull matrix factorization: The SVD A Quick view on its potential Matrix-crimes The Useful matrix Lemma 35/42

36 a Matrix-crimes: Syntax crimes 1. Non-compatibility of dimensions: A + B when A R 2 3 and B R 3 3 and the same for A T B. 2. Matrix products do (in general) not commute: AB BA. 3. Matrix inverse of the product of matrices: (AB) 1 A 1 B 1 in stead of (AB) 1 = B 1 A 1 - provided inverses exist! 4. (A+B) 2 A 2 +2AB +B 2! a Typical violations of Stanford students [S. Boyd - EE 263], our TUD students too often join the club... 36/42

37 Matrix-crimes: Semantic crimes Matrix expressions that simply do not make sense. Examples: 1. Let x R n, then xx T exists but (xx T ) 1 not, why? 2. If the matrix Q R m n for m > n, then QQ T can never be the identity matrix. 37/42

38 Overview Linear Algebra (LA) The matrix concept! The Usefull matrix factorization: The SVD A Quick view on its potential Matrix-crimes The Useful matrix Lemma 38/42

39 Lemma 2.3 p. 19 Schur Complements: Block Triangular Factorizations Let the block matrix A R n n (symmetric) be invertible, then a very useful matrix factorization of matrix consisting of different blocks is the following (C R m m ): A B I 0 = A 0 I A 1 B B T C B T A 1 I 0 C B T A 1 B 0 I Therefore the following holds, A B 0 A > 0 B T and C B T A 1 B 0 C 39/42

40 Given Exercise [ ] A B B T (A R n n andc R m m ) C (symmetric) with A > 0 and C B T A 1 B > 0, then show: ( [ ] A B ) rank B T = n+m C 40/42

41 Lemma 2.3 p. 19 Schur Complements: Block Triangular Factorizations When C is invertible, then we have: A B = I BC 1 A BC 1 B T 0 I 0 B T C 0 I 0 C C 1 B T I Therefore the following holds, A B 0 C > 0 B T and A BC 1 B T 0 C The condition Matrix 0 among others means that a square root of the matrix exists: Matrix = Matrix 1/2 Matrix T/2 41/42

42 Summary of Lecture 1 To start the discovery tour for retrieving system information from measured data records: What we just have done is a brief review of linear algebra. Next we briefly review probability theory and filtering of stochastic processes! We will also start with analysing the derterministic least squares problem! Reading of the course book of first Day Lecture: Study Chapters 1, 2( ), 3, 4( ) 42/42