Linear Inverse Problems. A MATLAB Tutorial Presented by Johnny Samuels

Size: px

Start display at page:

Download "Linear Inverse Problems. A MATLAB Tutorial Presented by Johnny Samuels"

Charlene Houston
5 years ago
Views:

1 Linear Inverse Problems A MATLAB Tutorial Presented by Johnny Samuels

2 What do we want to do? We want to develop a method to determine the best fit to a set of data: e.g.

3 The Plan Review pertinent linear algebra topics Forward/inverse problem for linear systems Discuss well-posedness Formulate a least squares solution for an overdetermined system

4 Linear Algebra Review Represent m linear equations with n variables: a11y1+ a12y2 + + a1nyn = b1 a y + a y + + a y = b a y + a y + + a y = b n n 2 m1 1 m2 2 mn n m a11 a1n y1 b1 = am 1 a mn y n b m A = m x n matrix, y = n x 1 vector, b = m x 1 vector If A = m x n and B = n x p then AB = m x p (number of columns of A = number of rows of B) A y b

5 Linear Algebra Review: Example y1 y2 + 3y3 + y4 = 2 3y 3y + y = y + y 2y = y y y = y b A y Solve system using MATLAB s backslash operator \ : A = [ ; ; ]; b=[2;-1;3]; y = A\b

6 Linear Algebra Review: What does it mean? y + y = y1+ y2 = 1 Graphical Representation: 1 1 y1 1 = 2 1 y 2 1 A y b

7 Linear Algebra Review: Square Matrices A = square matrix if A has n rows and n columns The n x n identity matrix = = 0 1 If there exists A 1 s.t. A A= I then A is invertible I

8 Linear Algebra Review Square Matrices cont. If a b A = c d then A 1 1 d b = ad bc c a Compute (by hand) and verify (with MATLAB s * command) the inverse of 2 1 A = 1 3

9 Linear Algebra Review: One last thing The transpose of a matrix A with entries A ij is defined as A ji and is denoted as A T that is, the columns of A T are the rows of A Ex: A = implies T A 1 0 = Use MATLAB s to compute transpose

10 Forward Problem: An Introduction We will work with the linear system Ay = b where (for now) A = n x n matrix, y = n x 1 vector, b = n x 1 vector The forward problem consists of finding b given a particular y

11 Forward Problem: Example g = 2y : Forward problem consists of finding g for a given y If y = 2 then g = A = What if and y =? What is the forward problem for vibrating beam? 1 1

12 Inverse Problem For the vibrating beam, we are given data (done in lab) and we must determine m, c and k. In the case of linear system Ay=b, we are provided with A and b, but must determine y

13 Inverse Problem: Example g = 2y : Inverse problem consists of finding y for a given g 1 1 If g = 10 then 2 2y = y = 2 10= Ay = b A Ay= A b y= A b y y2 = y 3 2 A Use A\b to determine y y b 1 y y2 = y

14 Well-posedness The solution technique y= A b produces the correct answer when Ay=b is wellposed Ay=b is well-posed when 1. Existence For each b there exists an y such that Ay=b 2. Uniqueness A 1 3. Stability is continuous Ay=b is ill-posed if it is not well-posed 1 Ay = Ay y = y

15 Well-posedness: Example In command window type y=well_posed_ex(4,0) y is the solution to y y2 1 = y y 4 1 A y b K = condition number; the closer K is to 1 the more trusted the solution is

16 Ill-posedness: Example In command window type y=ill_posed_ex(4,0) y is the solution to Hy 1 1 = 1 1 where H = Examine error of y=ill_posed_ex(8,0) Error is present because H is ill-conditioned

17 What is an ill conditioned system? A system is ill conditioned if some small perturbation in the system causes a relatively large change in the exact solution Ill-conditioned system:

18 Ill-Conditioned System: Example II y = y A y b 1 y2 y? =? y y2.066 y 1? = y2? = A y b

19 What is the effect of noisy data? Data from vibrating beam will be corrupted by noise (e.g. measurement error) Compare: 1. y=well_posed_ex(4,0) and z=well_posed(4,.1) 2. y=well_posed_ex(10,0) and z=well_posed(10,.2) 3. y=ill_posed_ex(4,0) and z=ill_posed(4,.1) 4. y=ill_posed_ex(10,0) and z=ill_posed(10,.2) How to deal with error? Stay tuned for next talk

20 Are we done? What if A is not a square matrix? In this case does not exist A 1 Focus on an overdetermined system (i.e. A is m x n where m > n) Usually there exists no exact solution to Ay=b when A is overdetermined In vibrating beam example, # of data points will be much larger than # of parameters to solve (i.e. m > n)

21 Overdetermined System: Example x = xi = x1 + x2 + + x 2 n i= 1 Minimize n 2 2 T ( ) ( ) Ay b = Ay b Ay b

22 Obtaining the Normal Equations T We want to minimize φ ( y) = ( Ay b) ( Ay b) : ( T ) T ( ) ( ) ( ) φ y = A Ay b + Ay b A T ( ) ( ) 2 ( T T A Ay Ab ) φ ( y) is minimized when y solves ( ) 1 T = A Ay b + A Ay b T T T T = A Ay Ab+ A Ay Ab = T T y = A A Ab provides the least squares solution T T A Ay = T Ab

23 Least Squares: Example Approximate the spring constant k for Hooke s Law: l is measured lengths of spring, E is equilibrium position, and F is the resisting force ( ) = y b l E k F T ( ) ( ) A ( ) 1 ( ) k = l E l E l E F least_squares_ex.m determines the least squares solution to the above equation for a given data set T

24 What did we learn? Harmonic oscillator is a nonlinear system, so the normal equations are not directly applicable Many numerical methods approximate the nonlinear system with a linear system, and then apply the types of results we obtained here

Linear Inverse Problems

SAMSI/CRSC Undergraduate Workshop Linear Inverse Problems Ariel Cintrón-Arias Tuesday, May 22, 2007 LINEAR INVERSE PROBLEMS 1 Inner product (1, 1, 5, 7) β 1 β 2 β 3 β 4 = 1β 1 + 1β 2 + 5β 3 + 7β 4 In general,