Solutions to Problems of Numerical Methods I.

Transcription

1 Solutons to Problems of Numercal Methods I Csaba J Hegedüs ELE, Faculty of Informatcs Budapest, 5 November egyzet az ELE Informatka Kar 5 év Jegyzetpályázatának támogatásával készült Lektorálta: Dr baffy József, Óbuda Egyetem

2 Problems n Chapter Let the set of machne numbers be M (5, 4, 4) Identfy the specal machne numbers! Map the followng numbers: /5, 37, 367, 7, 78 nto ths set! k 5 / / 3 35 t 5 ε M / 3 35, ε, fl( / 5) fl( ), / 5 < ε, fl( 78), because 78 > M + t k 5 4 ( ) ( ) 6 / 55 M, fl(37), fl(367), How should we convert 87 nto a ternary number of base 3? Dvson by 3 for the nteger part and multplcaton by 3 for the fracton part: 3 and How the machne epslon s modfed, f choppng s appled nstead of roundng? It wll be twce as much because now the last dgt s uncertan Problems n Chapter Show that for all nduced norm I holds May the Frobenus norm be an nduced norm? pply defnton: norm max Ix If x n, n I R then I F n such that t may not be an nduced If s nvertble then x x s also a vector norm We have to check norm condtons: ) x x only f x It s needed here that x only f x If has an nverse, the only soluton for the frst equaton s x x ) λx λ x λ x 3) he trangle nequalty s also nherted from the frst vector norm: ( x + y) x + y 3 matrx condton number may not be less then for nduced norms I cond( ) 4 Usng the -norm, the condton number of orthogonal or untary matrces s equal to

3 H H For untary matrces U U such that λmax ( U U ) λmax ( I) he case for orthogonal matrces s smlar 5 Prove: ab a b ab a b ab a b he -norm s the column norm: ab max a b a b he -norm s the row norm: ab max a b a b Frst soluton for the -norm: ab max ab x a max b x and t s maxmal f x x vectors x and b are n the same drecton, then x b / b In that case the result s a b Second soluton We explot the fact that the nonzero egenvalues of bb and b b are equal: ρ( ) ρ( ) ρ( ) ab ba ab a bb a b b a b, where ρ s the spektral radus 6 U U I (orthogonal) U ( ) ( ) ( ) U ρ U U ρ UU ρ, because the egenvalues are unchanged f nterchangng matrces n a product, see the Remark after the spectral norm n the text 7 B ± B + B B + B + B B + B Interchangng and B gves B + B Combnng the two gves the statement for + B Changng B to B gves the result for B 8 3,??? 4 max{ 6,5, } 6 { } max 6, 7 7 ( ) / 4 / / / λ max ( ) λmax ( ) λmax ( ) / 9 he -norm can be gven by the spectral radus of spectral radus, t s less than any norm of : { } ρ( ) spectral radus and usng the heorem on the

4 Check the nequalty x x (It s consstent wth the -norm) F Observe that the Frobenus norm of vectors as specal matrces s equal to the -norm, therefore t s the consequence of the fourth norm condton for matrx norms If then ρ( ) spectral radus, that s, the spectral norm s the mnmal norm for symmetrc matrces ( spectral norm) { } ρ ρ λ max λmax ( ) spectral radus ( ) ( ) ( ) bsolute sgn s not needed, because the egenvalues are squared If then, p, p It s the consequence of the prevous problem For symmetrc matrces the -norm s mnmal 3 U U I (orthogonal) U F F We use the dentty: trace( B) trace( B) : U trace U U trace UU trace F F F F 4 B B f and B B See the Remark after the spectral norm n the text It follows that matrces can be cyclcally permuted f there are more matrces n a product such that the egenvalue s unchanged: ( ) λmax ( ) ( ) λmax ( ) λ max B B B B λ max B B B B 5 cond ( ) cond ( ) ( ) λ ( ) ( ) λ ( ) cond ( ) λ / / max max λ max max cond ( ) 6 P P, p,,, where P s a permutaton matrx p p p Matrx P s orthogonal such that the result comes from Problem 6 for p For the column norm ( p ) there s no change n P because only the columns are permuted In

5 P rows are permuted that can not change the -norm of a column, therefore the column of maxmal norm should be the same Smlar argument apples for the nfnty norm Problems n Chapter 3 3 Perform a dyadc multplcaton wth two vectors Explan that t should have rank Whch method s smpler to multply by a dyad? a) Form ab then compute x b) Compute b x frst and then multply vector a wth that scalar Every column of the resultng matrx s a scalar multple of a therefore the columns are lnearly dependent a) Formng ab needs n flops, further computng x needs n addtonal flops, the operaton count s 3n flops b) Computng b x needs n flops and multplyng vector a wth a scalar stll requres n flops, altogether 3n flops Clearly, ths latter one s smpler 3 Consder the permutaton matrx Π [ e, e4, e3, e ] Check that ts transpose gves the nverse Prove ths fact n general! How can we store ths permutaton matrx n a vector? Check e e4 [ e, e4, e3, e ] I 4, e 3 e where I 4 s the dentty of order 4 he general case s smlar: e e [ e, e,, e ] I e n n n For storng a permutaton matrx n a vector, chose a vector a [ n] and store Π a 33 Check: F a e of (33) ( ) a e e a F a a a a + e e e a 34 Wth the ad of formula (35), show that the determnant of a matrx wll not change f a scalar multple of a column s added to another column of the matrx pply the theorem on the determnant of the product of two matrces! ake the determnant n (35): ( I + αe e ) because the second multpler s a specal trangular matrx havng s n the dagonal such that ts determnant s k 35 Form the rank- sum of DB, where D [ d δ ] s a dagonal matrx, (only the dagonal elements are nonzero)

6 n n n DB e e DB e e d B d e e B 36 pplyng the scalar product and the dyadc product forms of matrx multplcaton, show m, n that tr( B) tr( B),, B R! Observe that for vectors a, b tr( ab ) b a holds hen ( ) ( ) ( ) ( ) tr B tr e e B tr e Be tr B ( ) ( ) 37 Let matrx be nvertble Gve the expanson of vector x n terms of the columns of ( ) x x 38 Collect the vectors of a borthogonal system nto matrces [ a, a,, a n ] and B [ b, b,, b n ] Check that B s a dagonal matrx! How can we gve vector x as a lnear combnaton of vectors a? nd how can we gve the expanson wth the ad of vectors b? pplyng defnton of borthogonalty gves B D a b α δ D herefore the nverse of s D B and x ( D B x) he nverse of B s D, hence the second answer s: x B( D x) 39 Check: f P s a proecton, then I P s also a proecton ( )( ) I P I P I P + P I P 3 plane has normal vector s and ts defnng equaton s s x σ Introduce the proecton P I ss / s s Show that for all vectors y the operaton Py + σ s / s s produces a vector n the plane It s enough to check the statement: ( ) s Py + σ s / s s s I ss / s s y + σ s s / s s + σ 3 Show wth the prevous matrx P : Py s, n other words Py s perpendcular to s Gve the vector that connects Py + σ s / s s and y! s P therefore the statement follows for any y he connectng vector s parallel to the normal vector: s y y ( Py + σ s / s s) y y + σ ss y / s s σ s s s 3 Show that the backward dentty J [ enen e ], where the columns of the unt matrx are gven n reverse order, s nvolutory What proecton wll t defne for n,3?

7 In fact, J s a symmetrc permutaton matrx, therefore t s nvolutory he proectons defned are: / / / / and / / / / 33 Show that matrx ( )( ) I x y x y /( x y) ( x y) wll reflect vectors x and y nto each other, f they are dfferent and have the same length: ssume x x y y σ hen x x y y ( x y)( x y) ( x y) x σ y x I ( ) ( ) x x x y x x y y ( x y) ( x y) σ y x σ y x he other reflecton can be checked smlarly 34 We have the possblty to reflect vector x wth the prevous matrx nto vector y ± σ e, where σ x x How should we choose the sgn of σ to avod cancellaton error n the denomnator? ssume y σ e such that the sgn s attached to sgma From the prevous problem the denomnator now s σ ( σ e x) here wll be no cancellaton for sgn( σ ) sgn( e x) 35 Introduce F I + UV, where the unt matrx s modfed by the n l matrces U and V, that s, they have l < n columns If F s nvertble, show that F I U ( I + V U ) V (Sherman-Morrson-Woodbury formula) holds, where I l s a unt matrx of sze l l It s enough to check: ( )( ( l ) ) FF I + UV I U I + V U V I + UV U ( I + V U ) V UV U ( I + V U ) V l l ( ( l ) ( l ) ) ( l )( l ) I + U I I + V U V U I + V U V ( ) I + U I I + V U I + V U V I l Problems n Chapter 4 4 Usng LU-decomposton, solve the followng lnear system: x

8 , L, U, x Fnd the operaton count for x, LUx, U L x he factorzatons gven n Secton 3 may be appled for the last case ll of them needs n flops, the last one requres stll n dvsons 43 Usng Problem 35, show that the matrx of (46) can be nverted by takng the negatve of the block Smlar result for the upper trangular case can be found by transposton We look for the nverse of the matrx n (46): L I ( E E ) E I E such that usng the Sherman-Morrson-Woodbury formula he matrx n the mddle wll be a unt matrx because of E and t remans only to change sgn 44 Let L L to a larger lower trangular matrx ssumng that the dagonal blocks are nvertble, apply the parttoned nverse to get L be a lower trangular matrx, whch s complemented by a block row [ ] L L L L L L L L pplyng the parttoned nverse formula (49): L L + ( L L ) LL I I L + L LL L 45 By usng the block parttoned form, check the determnant dentty ( ) It can be seen from the block decomposton formula: I I, I ( ) I 46 Wth the ad of the prevous problem, check the denttes a b I + b a I + ab

9 Compare t wth the approach gven n Example E33! We use the formula of Problem 45 Chosng for the left upper element gves the last determnant akng the Schur complement ( ), where s the dentty matrx wll lead to the formula n the mddle s seen, ths approach s much smpler 47 What s the domnant term n the operaton count of the Gauss-Jordan factorzaton? We have to do roughly n( n k) flops n the k -th step, see heorem 3 Summng up for k,, n yelds for the domnant term 3 n flops , b, LU, x b L?, U? x? L 3, U, x Problems n Chapter 5 5 We have the Cholesky-decomposton LL Gve the operaton count for computng x x f matrx s used n the computaton! How can we decrease the number of operatons f x LL x s used? o compute x requres n flops and the remanng scalar product n flops But the computaton of L x nvolves n( n ) flops plus n flops for sclar product computaton herefore the second approach needs half as many operatons 5 We can avod square roots, f we use the form LDL, where L has unt dagonal and D s a dagonal matrx Elaborate the steps of ths decomposton! hs method can also be used for ndefnte matrces f the pvot elements n D happen to be large enough hs tme we proceed smlarly as n LU-decomposton If a step s ready, we dvde the row of the pvot wth the pvot and save pvot n a dagonal matrx For symmetrc matrces dvson s not necessary, because the U part s the transpose of L

10 53 Show that the row dagonal domnance s preserved f the matrx s multpled from the left by a nonsngular dagonal matrx lso, t s preserved f two rows and columns wth the same row and column numbers are nterchanged dagonal matrx on the left multples rows wth a number If that number s nonzero, the rato between the entres wll be the same such that dagonal domnance s kept Interchangng the same two rows and columns wll keep the dagonal element n dagonal poston, only the order of row elements s changed that wll ntroduce no effect n dagonal domnance 54 Show that for the LU-decomposton of essentally dagonally domnant matrces (by row): strct dagonally domnance takes place n the th step for the k-th row, f there was strct dagonal domnance n the -th row and the element a < k was not zero It s enough to check the frst step Strct dagonal domnance n the frst row means that the full absolute contrbuton of the frst column element wll be less than the frst column element, f t s nonzero ( ) k, 55 Show that dagonal domnance by columns s also nherted n LU-decomposton hs tme we attach the dvsor to columns and consder the contrbuton of the left out row elements n a column he only change s that the reasonng s done to columns 56 If we are gven a new rght vector b, whch data should be preserved and whch data should be recomputed n both algorthms (fast LU and passage)? Fast LU: he second row n (65) should be recomputed for b elements he off-dagonal and pvot elements are needed n the computaton Passage: he f elements should be recomputed Stll g s are neede for the soluton 57 Prove that the trdagonal matrx n (6) s postve defnte, because t has a LL - decomposton In fact, dagonal domnance wll take place n the actual rows when dong LU-decomposton he second pvot s / 3 / he thrd one s / 3 4 / 3 ssumng the n -st pvot s n / ( n ), nductvely one gets ( n ) / n (n n + ) / n ( n + ) / n for the next pvot, such that Cholesk-decomposton s possble LL, L?

11 , L 4 Problems n Chapter 6 6 Prove formula (65)! Multplyng two factors shows, the statement s true for akng a next factor wll add a new term only n the sum because of orthogonalty of the vectors Inductvely, one term wll be added for an arbtrary by the same reasonng 6 Show that r, + s of (67) and (68) are equal! It s because the proectons for q q PP P z do not make change n the -th orthogonal vector: 63 Collect the orthogonal vectors nto matrx Q [ qq q ] Derve the formula: PP P ( ) I Q Q Q Q pply the weghted dyadc sum of Sect 34 for the last formula of (65) See also: Problem Let matrx m n R have lnearly ndependent columns Check that ( ) I s also a proecton and applyng t to a vector, the resultng vector wll be orthogonal to all columns of It s enough to check the proecton condton ( ) proecton Really, ( ) P z I z ( ) P for ( ) as I P s also a 65 One can elaborate the varant of GS orthogonalsaton, when the q s are normed vectors, q Rewrte formulas for that case!

12 In ths case Q Q I holds such that Q R from (69) ll dvsors n the proectons are s and the + -st vector s gven by +, + + +, +, + + r q a q r, r q a,, where r +, + s found from the condton that q + s normalzed 66 Havng a QR -decomposton of, how can we solve the lnear system x b? Frst step: form Q x Rx Q b Second step: solve Rx R s upper trangular 67 Make the QR -decomposton of 3 3 4! Q b from below for x, because We may choose [ ] for q because all elements of the frst column can be dvded by 3 and thus the computaton s smpler Now q q 6 he proecton for the second column results n: 4 q, q 6 q q From the thrd column ( 6) q3 6 and QR Gve QR -decomposton n modfed Gram-Schmdt style! 3 3 / 3 5/3 3 4 /3 / 3 - -/3 QR 5 / / Now we calculate the scalar products of the frst column wth all columns n the ntal matrx and the result s shown under the columns On top of the columns these numbers are dvded by the squared norm of the frst column (9) Next the frst column s multpled by the numbers above the second and thrd columns and the resultng vector s subtracted from the correspondng vectors In other words, form the dyad by the frst column and the row vector on top of the matrx and subtract smlarly to LU-decomposton Repeat the same procedure for the second matrx that has by one less columns he thrd matrx s zero and fnally the decomposton s gven Observe, the row vectors above the

13 ntermedate matrces gve the nonzero row elements of R s seen, there are only two lnearly ndependent vectors n ths set 69 Let the rnold method s performed so that the orthogonal vectors are normed Show that f s symmetrc, then the upper Hessenberg matrx H s also symmetrc, e trdagonal he matrx form of the recurson s Q QH + h, q e, where Q ( q q q ) + + Multplyng from the left by Q gves Q Q H hat shows the symmetrcty of H for symmetrc and then the Hessenberg matrx should be trdagonal 6 Let the startng vector x be the sum of three egenvectors of havng dfferent egenvalues How many new vectors can be generated by the rnold method? Let the startng vector be x αu + αu + α3u3, where α s are nonzero scalars and the egenvectors wth egenvalues the Krylov vectors can be arranged n a matrx as λ Introduce matrx U [ α u α u α u ] λ λ x x x U λ λ λ3 λ u s are Wth these ddng a next column to the transposed Vandermonde matrx would not ncrease rank, therefore the maxmal possble number of lnearly ndependent vectors s 3 Problems n Chapter 7 7 Let be an upper Hessenberg matrx such that all subdagonal elements are nonzero Show that there s only one Jordan block to each egenvalue Consder λi H, where λ s an egenvalue hen the rank loss may be no more than, because the subdetermnant by deletng frst row, last column s nonzero he rank loss shows the number of Jordan blocks to an egenvalue 7 Show f the egenvalues of are λ s, then u λu u λ u r / σ F 73 Check: σ Let have the sngular value decomposton s the maxmal egenvalue of Further has egenvalues / λ s H V Σ U, where, U V are untary hen σ H H H H UΣV VΣ U UΣ U, such that σ gves the -norm H H H H H F tr( ) tr( UΣV V Σ U ) tr( UΣ U ) tr( U UΣ ) tr( Σ ) 74 he matrx s dagonally domnant, f the Gershgorn dsks do not have zero In that case all rad are less than the belongng absolute dagonal element and that s ust the condton for dagonal domnance

14 75 Dagonally domnant matrces are nvertble Yes, because no Gershgorn dsk contans zero egenvalue 76 he rank of the matrx s at least as large as the number of those Gershgorn dsks, whch do not contan zero hese dsks belong to dagonally domnant rows hen t s possble to fnd a submatrx from these rows that s nonsngular, therefore the rank s at least equals to the number of such rows 77 Gershgorn dsks can also be found wth respect to columns f usng left egenvectors n the dervaton Equvalently, consder the transposed matrx 78 By usng Gershgorn s theorem and dagonal smlarty transform, decde f matrx s nvertble: Choose D dag[ ], then DD s dagonally domnant 79 Show that the egenvalues of a matrx are: λ a + a a a ± + a a, It s the root formula of the characterstc polynomal ( ) Prove that 7 U U / m, where U U Λ λ a + a + a a a a U UΛ and takng norms gves the result, where m s the smallest absolute egenvalue 7 U U M U Λ U and f M s the largest absolute egenvalue, the result follows by takng norms 7 cond( U ) cond( U ) cond( Λ ) Multplyng the same sdes of the prevous nequaltes gve the result Problems n Chapter 8 8 Let LU be a rank-factorsaton What s the orthogonal proecton to Im( )?

15 Im( ) s the subspace spanned by the columns of L such that the orthogonal proecton to t s gven by + ( LL L L L) L 8 What s the orthogonal proecton to the null space of? Gve the dstance of x to Nul( ) n two-norm! he null space of s gven by the orthogonal proecton I + pplyng the dstance + + theorem: dst (Nul( ), x) I ( I ) x x 83 lne passes ponts r and r Gve the dstance of vector x from ths lne! he drecton vector of the lne s: d r r Vector r x s a dstance of x from a pont of the lne such that s has a component parallel to d and an other one, perpendcular to d he length of the perpendcular component gves the dstance from the lne: ( ) dst ( lne, x) I dd / d d ( r x) 84 Show that f a matrx s nvertble then ts nverse and pseudonverse are equal It comes from the frst Penrose condton: + and multplyng by the nverse of gves: I Gve the orthogonal proecton nto Im( )! P he row vectors of n Problem 85 are the normal vectors of two planes Gve the orthogonal proecton nto the ntersecton of the two planes! Vector x s n the ntersecton of the two planes f x he orthogonal proecton nto the ntersecton of planes s gven by ( ) an arbtrary vector + P I I because Pz for 87 r [ ] What s the dstance of vector r from the ntersecton of the two planes n the prevous problem? ( ) ( ) dst (ntersecton, r) I P r r , rank( ) +? (Use LU-decomposton!)

16 3 LU-decomposton gves: LU 5 4 It s a rank factorzaton f both L 3 and U have rank two, such that two columns n L and two rows n U are needed For the pseudonverse ( ) ( ) U L U UU L L L needs to be computed 89 What s the pseudosoluton of x b f b [ ] and matrx comes from Problem 88 8 Show I, f the columns of are lnearly ndependent + 8 Derve the relaton ( ) + ( ) from the four Pendrose condtons 8 Matrx has an approxmate egenvector x Fnd the belongng approxmate egenvalue λ from the condton that x λx s mnmal Gve formula for λ! Problems n Chapter 9 9 Check f the base ponts are located symmetrcally to x, then α,,, hold and the polynomals are alternatng even and odd functons 9 Fnd the orthogonal polynomals p, p, p for the base ponts {,,,,}! 93 he Chebyshev polynomals are also orthogonal and they can be generated by the followng recurson:, x, x lthough they are not monc now, yet t s the famlar form Expand n+ n n 4x 3x + wth Chebyshev polynomals! 94 k P( x) ( + ) ( x) Gve a skllful way of computng the sum at the pont x! b 95 Show that ( p, p ) µ ββ β, where µ ( p, p) α( x) dx s the -th moment a 96 Show that the prncpal mnors of the trdagonal matrx x α β β x α β n β n x αn

17 have the same recursons as orthogonal polynomals wth parameters Problems n Chapter α and β How should we modfy Jacob teraton, f the matrx s dagonally domnant wth respect to columns? Show that heorem 3 can be reformulated for the case when the matrx s dagonally domnant wth respect to columns 3 Elaborate estmate Hba! hvatkozás forrás nem található for the GS-teraton! What happens to Jacob and GS teraton f nstead of dagonal domnance we have equalty n some equatons? nd f equalty takes place n the last row? BJ? B GS? pplyng heorem 3 show: max, see also a ( α β ) Hba! hvatkozás forrás nem található, f s strctly dagonally domnant by rows How can we modfy statement for dagonal domnance wth respect to columns? 6 ssumng D + ωl s dagonally domnant by rows, prove by applyng heorem 3 B GS ω + ωβ ( ω) max ( ) ωα 7 If D < holds, then we can derve an nequalty smlar to that of heorem 3 by usng (5), because of the equalty D D ( + D) ( I + D ) D Show that ( + D) holds for nduced norms Is that necessary that D be a dagonal matrx? For the matrx of Example 4 whch method gves a better estmate?