Discrete Ill Posed and Rank Deficient Problems. Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 1

Transcription

1 Discrete Ill Posed and Rank Deficient Problems Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 1

2 Definitions Overview Inversion, SVD, Picard Condition, Rank Deficient, Ill-Posed Classical Regularization Tikhonov and the Discrete Smoothing Norms Discretization Quadrature and Galerkin Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 2

3 Discretization Overview 2 Discrete Picard Condition Quadrature Galerkin Other Techniques TSVD CG Interation Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 3

4 Inversion Ax=b Solve for x, when A and b are known (or for A, when x and b are known) via Inverse, Psuedo-Inverse x= A 1 b x=a t b Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 4

5 SVD SVD(A) σ in non-increasing order or GSVD(A,L) n A=U V T = i=1 U T U =VV T =I n u i i v i T A T A=V 2 V T AA T =U 2 U T Singular Values Singular Vectors Left & Right (2) Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 5

6 Rank Deficient Gap in the Singular Values Approx. Zero Noise! Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 6

7 No gap in values Ill Posed Lots of small values Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 7

8 (Discrete) Picard Condition 1 0 Continuous K s,t f t =g s K s, t = i=1 f t = i=1 i=1 i u i s v i t SVE SVD Discrete Ax=b n A=U V T = i=1 n u i, g v i t x=a t i b= i=1 u i, g i 2... u i T b i u i i v i T v i denominator must decrease faster than numerator or we get an unbounded solution Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 8

9 Regularization First regularize, then discretize. Minimize residual norm with Constrained Values, with Constrained Size, Minimize Size, with Constrained Residual Minimize Residual and Size 1 r= 0 K s,t f t g s min r 2 2, x S min r 2 2, w min w 2 2, r 2 2 Tikhonov! min{ r w 2 2 } Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 9

10 Tikhonov Regularization min{ Ax b L x x } Solved by Least Squares using SVD Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 10

11 Discrete Smoothing Norms Choices for L : Identity Matrix Weighted Diagonal Discrete Derivative Approximations Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 11

12 L = I The identity matrix... Additional constraint: Minimize the absolute value of the solution min{ Ax b I x x } Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 12

13 L = diag(w) A diagonal matrix of weights on x... Additional constraint: Minimize a weighted selection of values of the solution min{ Ax b W x x } W =diag w Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 13

14 L = L1 or L2 A banded matrix approximating a derivative operator... Additional constraint: Minimize total variation in values of the solution (but still allow steep gradients) min{ Ax b L 1 x x } Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 14

15 Priors Allow the addition of a priori information about the result min{ Ax b L x x } Using no prior is the same as a zero prior Weight solution towards expectation Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 15

16 L curve under-smoothing over-smoothing (fake plot) Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 16

17 Discretization of Integral Quadrature 1 0 n t dt j=1 Equations choose w j w j t j a ij =w j K s,t b i =g s i Galerkin 1 a ij = 0 choose, 1 K s,t i s j t ds dt b i = 0 g s i s ds Raliegh-Ritz If =, K is symetric, and nodes are co-located Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 17

18 Solution (fake plots) Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 18

19 Discussion [ Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 19

20 References P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, 1998 Wikipedia: Inverse Problems, visited Jan 28, 2009 J Kaipio, E Somersalo, Statistical and Computational Inverse Problems, Springer, 2005 Alistair Boyle, Feb 2009, SYS5906: Directed Studies Inverse Problems 20