Numerical Linear Algebra

Transcription

1 Numerical Analysis, Lund University, Numerical Linear Algebra Unit 8: Condition of a Problem Numerical Analysis, Lund University Claus Führer and Philipp Birken

2 Numerical Analysis, Lund University, A problem and its sensitivity Let X, Y be normed vector spaces and f : X Y a (not necessarily linear) function. We consider f as a solution operator to a mathematical problem and call for x X the task to compute f (x ) a problem for the in-data x. Definition f (x ) is called well-conditioned if f (x + δx) f (x ) Y is small for all small δx X. We quantify this more on the next slides.

3 Numerical Analysis, Lund University, Absolute condition number Definition Let δf (x ) := f (x + δx) f (x ). We call ˆκ(x ) := lim sup δ 0 δx X δ δf Y δx X the absolute condition number of the problem f (x ). Note: If f is differentiable with a Jacobian J we have ˆκ(x ) = J(x ) X,Y (express δf by a Taylor expansion of f and neglect higher order terms.)

4 Numerical Analysis, Lund University, Relative condition number Definition Let δf (x ) := f (x + δx) f (x ). We call κ(x ) := lim sup δ 0 δx X δ δf f δx x the relative condition number of the problem f (x ). Note: If f is differentiable with a Jacobian J we have κ(x ) = J(x ) X,Y f (x ) x. A problem is well-conditioned if κ(x ) 10 6 else it is ill-conditioned. (This is a rule of thumb not a quantitative statement.)

5 Numerical Analysis, Lund University, Examples 1. A harmless example: f (x) = x 2. This has κ(x) f (x) = x 1 x 2, x R 2. The relative condition number is in -norm 2 κ(x) = x 1 x2 / max( x 1, x 2 ) Note, that subtraction of nearly equal values is extremely sensitive with respect to perturbations. 3. Wilkinson s example: Compute the roots ξ i = i of the polynomial p(x) = Π 20 i=1(x i) = a 20 x 20 + a 1 x + a 0

6 Numerical Analysis, Lund University, Wilkinson s example (Cont.) We note a 15 = = We investigate (experimentally) how ξ 15 = 15 is affected by changing a 15 by 0.1 (relative change ). Experimentally we obtain (ξ 15 + δξ 15 ) = j which gives the estimate κ

7 Wilkinson s example (Cont.) In Python... chars from sympy import * chars from scipy import * chars from scipy. linalg import eig chars chars x = symbols ( x ) chars chars def p(x): chars "\" " chars Wilkinson polynomial chars "\" " chars p=1 chars for i in range (1,21 ): chars p *=(x-i) chars return p chars p( 1) # a numeric value chars p( x) # a symbolic value chars p(15) chars pp= expand (p(x)) chars coeff =[ pp. coeff (x,i) for i in range (0,21 )] Numerical Analysis, Lund University,

8 Numerical Analysis, Lund University, Companion matrix In Python... chars def comp_ matrix ( coeff ): chars n= len ( array ( coeff )) -1 chars c= zeros ((n,n)) chars c[:,-1]= - array ( coeff [: -1]) chars c=c+ diag ( ones ((n-1,)), -1) chars return c chars chars c= comp_matrix ( coeff ) chars print ( The roots of the polynomial \n,sort ( eig (c)[0]))

9 Numerical Analysis, Lund University, Perturbing the problem We change now one coefficient... chars c15 = coeff [15] chars coeff [ 15 ]+= 0. 1 # perturbation chars rel_in_error = abs (0.1/ c15 ) chars c= comp_matrix ( coeff ) chars newroot15 = sort ( eig (c)[0])[ 14] chars rel_out_error = abs ( newroot15-15) chars condition_15 = rel_out_error / rel_in_error

10 Numerical Analysis, Lund University, Condition Number of Matrix-Vector Multiplication Consider the problem f : x Ax (matrix vector multiplication) κ = lim sup δ 0 δx X δ A(x+δx) Ax Ax δx x = lim sup δ 0 δx X δ Aδx δx Ax x = A x Ax In the special case that A is invertible, there exists a b with x = A 1 b and consequently x Ax = A 1 b b A 1. Thus κ A A 1. In the homework we will see that for some special x we have equality, i.e. the estimate is sharp.

11 Numerical Analysis, Lund University, Solving a linear equation system Consider the problem f : b A 1 b (i.e. solving Ax = b) This is a matrix-vector multiplication problem with A 1. We get κ A A 1 by interchanging A with A 1 on the slide before.

12 Numerical Analysis, Lund University, Solving a linear equation system, A as input Consider the problem f : A A 1 b (i.e. solving Ax = b) A is now problem input and subject to perturbations δa b = (A + δa)(x + δx) = }{{} Ax +Aδx + δax + δaδx }{{} =b 0 Thus, Aδx + δax = 0 (assympotically for small perturbations). δx = A 1 (δa)x δx A 1 δa x

13 Numerical Analysis, Lund University, Solving a linear equation system, A as input (Cont.) From this we get δx x δa A A 1 δa δa = A A 1 (The same estimate as before when we considered b as input.)

14 Numerical Analysis, Lund University, Condition Number of a Matrix The number κ A = A A 1 is called the condition number of a matrix In the 2-norm it can be expressed by the singular values of A: κ A = σ 1 σ n where σ 1 is the largest and σ n the smallest singular value of A.