Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b =1 c = max x (1c and these norms satsfy > 0 ff x 0 (1d 0 = 0 (1e cx = c where c = any scalar (1f x + y y (1g For matrces we would lke the norm to satsfy Ax A and the addtonal requrements Ax A for some partcular x (2a (2b A 0 A = 0 ff a j = 0 ca = c A where c s a real number A + B A + B (trangle nequalty AB A B (Schwartz nequalty (2c (2d (2e (2f (2g 1
The most popular norms are L m (A = ( a m j 1/m j (3a A A = max a,j (the absolute norm (3b,j A E = L 2 (A = a 2 j = tr(aa T (Eucldan, Frobenus (3c A S = max λ(aa T (Spectral (3d ρ(a = max (λ (A, = A S f A = A T (spectral radus (3e Snce tr(aa T =sum of egenvalues, clearly A S < A E Unless otherwse specfed, A means the Eucldean norm 11 Theorems Some useful relatons are: a A S = max x Ax (4a b A S ρ(a (4b c If A S < 1, I + A s non sngular and (4c (I + A 1 1 S 1 A S (4d d (I + A 1 I S 1 n A E A S A E A S 1 A S (4e (4f e If A s non sngular and A 1 B S < 1 then (A + B 1 A 1 S k A 1 S where (4g k = A 1 B S 1 A 1 B S 2 Condton Number The condton number s a measure of the effect on an approxmate nverse of a matrx A (or on an approxmate soluton of Ax = b, when the elements are changed slghtly Several dfferent condton numbers have been suggested 1 Turng s M-condton number M(A = n max,j a j max j a 1 j The average value for M(A s n log(n 2
2 Turnng s N-condton number, N(A = n 1 A A 1 where A = (,j a2,j 1/2 The average value for N(A s n 3 Von Neumann P-condton number P(A = λ max /λ mn The average value for P(A s n When a matrx egenvalues are close to each other, the matrx s well condtoned for solvng lnear equatons However t s ll condtoned for determnng the egenvalues 4 Determnant of the normalzed matrx The th row of A s dvded by ( n j=1 a2,j 1/2 If the determnant s small compared to ±1, the system s ll-condtoned The best condtoned matrces are orthogonal matrces, ther N-condton numbers are 1 and ther M condton numbers are about log(n For matrces wth elements chosen randomly from a normal dstrbuton, the N condton numbers are of the order of n and ther M condton numbers are of the order of n log(n 21 Relatons between the condton numbers 22 Insght nto Condton Numbers M(A n 2 N(A M(A (5 M(A P(A nm(a n (6 Condton numbers are somewhat mpractcal snce they requre ether nvertng the matrx, solvng for egenvalues, or calculatng the determnant all expensve operatons However, consder a small change n the rght hand sde vector, b, that causes a change n the soluton, x It can be shown that f the condton number s defned as A A 1 that x cond(a b b (7 Lkewse f a small change A causes a change x, then x A cond(a A 1 cond(a A A (8 provded that A A 1 < 1 3
23 Condton Number n terms of norms Let Ax M = max x, m = mn x and defne the condton number as Ax (9a Cond(A M m (9b If A(x + x = b + b, then b M and b m x and thus x < Cond(A b b and t can be shown that f we defne (10 that Cond(A = A S A 1 S (11 Cond(A < n 2 λ max λ mn = n 2 ρ(aρ(a 1 (12 If a 1 j s large, A 1 E >> 0, e, ll condtoned But snce A 1 S < A 1 E, A 1 S s not a good measure of ll condtonng However A 2 S = max λ (AA T 1 n A 2 E we have A E n A S A E (13 So A S s an acceptable measure of ll condtonng 24 Does makng A symmetrc mprove the soluton It s often suggested that the set of lnear equatons, Ax = b be solved by multplyng by the transpose of A, A T, gvng but A T Ax = A T b (14a P(A T A P(A and N(A T A N(A (14b so makng the orgnal problem nto one wth a symmetrc matrx actually worsens the method 4
25 Example Let A = ( 41 28 97 66 b = ( 41 97 ( 1 x = 0 (15 Now lettng ( 411 b = 970 ( 001 b = 000 ( 034 then x = 097 ( 066 and x = 097 (16 (17 The one norms are: b = 138, = 1, b = 001, x = 163 and Eq (7 gves or 163 A A 1 72464e 04 (18 A A 1 = 2249 (19 Now the value 2249 s only an approxmaton because dfferent choces of b wll gve dfferent results The P condton number s P(A = λ max = 107093 = 1515 (20 λ mn 00093 26 Matlab and SVD condton numbers Another defnton s that the condton number s the rato of the largest and smallest sngular values obtaned from SVD In ths case, t equals 1623 The condton number obtaned usng condest(a n Matlab s 2249 Also cond(a = cond(a, 2 = cond(a, fro = 1623 and condest(a = cond(a, 1 = cond(a, nf = 2249 Matlab also computes a condton number based upon the 1-norm, =2249, based upon the 2-norm for nvertng A, = 1623, and for computng the egenvalues, = 11893 5
3 Egenvalues and Egenvectors Gven a matrx A, f the vector x s such that Ax = λ x (21 where λ s a scalar, x s sad to be an egenvector and λ s the egenvalue That s for Ae = λe = λie (22a (A λie = 0 (22b e > 0 (22c A λi = 0 (22d If A s n n, there wll be n egenvalues and egenvectors, some of whch may be duplcates The egenvalues satsfy a λ(ab = λ(ba (23a b λ(a 1 = 1/λ(A (23b c λ(a + B = λ(a + λ(b (23c d λ(a m = λ m (A (23d If A = A T, and A s real, λ s real, the egenvectors are lnearly ndependent and span the space In addton λ max a j, aj 2, max j a j, max j a j (24 31 Condton Number related to Egenvalues Snce b must belong to the space the A spans, we can expand t n terms of the egenvectors, e Now b = β 1 e 1 + + β n e n (25 Ae 1 =λ 1 e 1 A 1 Ae 1 =λ 1 A 1 e 1 gvng e 1 =λ 1 A 1 e 1 6
A 1 e 1 = 1 λ 1 e 1 Thus x = A 1 b = β 1 λ 1 e + + β n λ n e n (27 and f a small change n b alters β j and f λ j = λ mn 0, A 1 b stretches the soluton x by a huge amount n the drecton of e j and ntroduces a large error n the soluton 4 Hlbert and Vandermonde Matrces The Vandermonde determnant s defnded as V,j = a j 1 or 1 a 1 a1 n 1 1 a 2 a n 1 1 a n an n 1 2 (28 and arses from nterpolatng the polynomal a 0 + a 1 x + + an 1 on equally spaced data ponts In Matlab the call A = vander(1 : 05 : 3 wll produce the matrx wth a largest value of 81 and a condton number of cond(a = 21217 10 4, condest(a = 39795 10 4 and an nverse wth a largest value of 10 3 Scalng the matrx has no effect on the condton number The Hlbert matrx 1 1/2 1/3 1/(m + 1 1/2 1/3 1/4 1/(m + 2 1/(m + 1 1/(m + 2 1/(m + 3 1/(2m + 1 (29 arses from fttng the polynomal f(x = m j=0 a jφ j (x where φ j (x = x j and s a classc example of an llcondtoned matr 7