4. Linear systems of equations. In matrix form: Given: matrix A and vector b Solve: Ax = b. Sup = least upper bound

Transcription

1 4. Lnea systes of eqatons a a a a 3 3 a a a a 3 3 a a a a 3 3 In at fo: a a a3 a a a a3 a a a a3 a Defnton ( vecto no): On a vecto space V, a vecto no s a fncton fo V to e set of non-negatve eal nes at oeys e ee postlates: ( ) 0 fo any V, and 0 f and only f 0 ( ) fo eal ( ) y y fo, yv n Coonly sed nos fo V R : / () L no : ( ) L no p p : p ( ) L no Ma : / p 3 Gven: at and vecto Solve: = det 0 ( s non-sngla, ests) ee ests a nqe solton soltons det0 ( s sngla, does not est) no solton. Gass elnaton eod. LU decoposton 3. Jaco teaton eod 4. Gass-Sedel teaton eod 5. Relaaton eods 6. Congate gadent eods 7. Pecondtone Defnton ( at no): at no sodnatng to a vecto no s defned y Sp : R,. Sp = least ppe ond L no : Ma a L no: Ma whee 's ae e sngla vales of * ae egenvales of * = congate tanspose of Ma f all egenvales of ae eal and postve. 4

2 Defnton (Condton ne) - Def s called e condton ne of a at. B B Sp y y = f all egenvales ae eal and postve a n Well/Ill-posed pole Consde he elatve eo esltng n solvng de to a dsted s Ill-condtoned f s lage (solton s senstve to a slght vaaton of ). Well-condtoned f s of sall to odeate agntde. 5 7 Well/Ill-posed pole Gven:, ests Sppose at e vecto s peted to e, If and, y how ch s dffent fo? Bacwad Ssttton (Uppe angla Mat) a a a a a 0 a a a a 0 0 a a a 0 0 a, a, a, a, a, a 3, 3, 33 3, a a a,, a a a,,, a a fo,,,, 6 8

3 fowad Ssttton (Lowe angla Mat) a a a a3 a3 a a, a, a, 0 0 a, a, a, a, 0 a, a, a, a, a a a a a a fo,3,, 9 heefoe, afte e do loop, we have E : a a a a () () () () () 3 3 E : 0 a a a () () () () 3 3 E : 0 a a a () () () () 3 3 E : 0 a a a () () () () 3 3 w pdated coeffcents () a a,,,, () () () a, a,3,, a a a a a,,, 3,, Gassan Elnaton Meod E : a a a a 3 3 E : a a a a 3 3 E : a a a a E 3 3 : a a a a 3 3 SEP : povded a 0, DO a a E E fo,3,, a a a a a3 3 a a a a a 3 3 a a a () () () () a a a a a () a a () 0 SEP : povded a 0, DO a a E E fo 3,4,, (3) () (3) () () () () E : a a a a (3) (3) (3) (3) (3) 3 3 E : 0 a a a (3) (3) (3) (3) 3 3 E : 0 0 a a (3) (3) (3) E : 0 0 a a (3) (3) (3) 3 3 E : 0 0 a a (3) () () () () (3) (3) (3) 3 3 a a a,,,, a a,,, a a a a a,, 3,4,, (3) () (3) () (3) () () () () a a, 3,4,,

4 () SEP : epeat e pocess (povded a 0) ntl E : a a a a ( ) ( ) ( ) ( ) ( ) 3 3 E : 0 a a a ( ) ( ) ( ) ( ) 3 3 E : 0 0 a a ( ) ( ) ( ) E : a a ( ) ( ) ( ) 3 3 E : a ( ) ( ) 3 U ( ) acwad ssttton () scaled pvotng : choose p y copang e elatve agntdes of a nstead of e asolte agntde,.e. p and ap a S Ma a, Ma S S solte vales: 89. > 8.9 Relatve vales 89./ < 8.9/ p Eaple scalng s sed fo copason t not eally copted. 3 5 Pvotng --- wheneve a =0 o s vey sall () pvotng : ntechange e ow w e p ow whee p s e sallest ntege > and a p 0 () aal pvotng : pvotng not only n ows t also n colns at step: seach fo p, q sch at p, q and e.g. 3, a 0 33 E : a a a a (3) (3) (3) (3) (3) 3 3 (3) (3) (3) (3) : (3) (3) 3: E a a a E a E : 0 0 a3 3 a E : 0 0 a33 a () patal pvotng : choose p n sch a way at (3) (3) (3) (3) (3) (3) p a Ma a and p 4 a pq Ma a, ntechange e ow w e p ow ntechange e coln w e p coln 6

5 Eaple: = 3 E : E : E : Choose p = 4 and q = 5 E E : ' E E : ' E E : ' otce q ntechanges w as well new solton:,,,,,, LU L UX Ly () fowad ssttton Ly y y y, () acwad ssttton U y y n n, y, 7 9 Mat factozaton (LU decoposton) Gven:, ost often t t t t PROPERY: has a dagonal stcte (ost nonzeo eleents ae on e an dagonal.) FID: = LU, whee L(U) s a lowe (ppe) tangla at L,,,3, 0,,,3, U , 3, 33 3, 3,, If so, we can solve = y acwad/fowad ssttton 8 Geneal LU-decoposton , , , 3,,,3 0, 0 0 0,,,3,, # of constants = degees of feedo = + Eaple: asse assgned fo,,, By osevaton, a a fo,,,,, 0

6 Geneal LU-decoposton ,,,3, ,,, 3, , , 3, , 33 3, 3 3,, specal cases: : Doolttle's factozaton : Cot's factozaton possle only f s U L : Cholesy's factozaton eal,syetc, and postve-defnte at s postve-defnte f > 0 fo any 0. a a fo,3,, a 3 3 a a3 fo 3,4,, heoe: If Gassan elnaton can e pefoed on e lnea syste = wot ow ntechanges, en e at can e factoed nto =LU. 3 a a a a,,,,,,,,, a fo,,, a,,,, a,,,, Gassan elnaton pocedes SEP : povded a 0, DO a a E E fo,3,, M () It can e shown at a a , , () () M M () () () L

7 Gassan elnaton pocedes SEP : povded a 0, DO a a E E fo 3, 4,, M () () () () () () a a , , It can e shown at M M () () () () (3) (3) () L 5 () () ( ) ( ) M M M M ,,,3 0,,,3, () () ( ) ( ) L M M M M ( ) ( ) () () M M M M U LU L 7 Gassan elnaton pocedes SEP : povded a 0, DO a a E E ( ) ( ) ( ) ( ),,, a a a a a 0 a a a a 0 0 a a a a a a ( ) ( ) ( ) ( ) ( ) 3,, ( ) ( ) ( ) ( ) 3,, ( ) ( ) ( ) 33 3, 3, ( ) ( ),, ( ), U ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) () () M M M M M M M ( ) ( ) () () M M M M U () () ( ) ( ) M M M M U 6 Iteatve technqe ~ good fo lage, spase atces Gven: SEP: tae a gess fo e solton, say SEP: copte e eo (esde) SEP3: too lage? ceate a new gess.e. geneate a seqence n soe way so at e seqence conveges to e eal solton cteon fo stoppng e teaton: () () () 8

8 est staton: 0 as.e. 0 as fo =,,,. teatve schees: (sally) Gven soe at and vecto c, c heoe 0 n Fo any R, e seqence defned y c 0 fo each and c0 conveges to e nqe solton of c 0 f and only f. Moeove, he spectal ads () of a at s defned as e a asolte vale of egenvales of e at,.e. Ma. schee constcts e at and e vecto c n sch a way at e solton of =+c s also e solton of =. 9 Jaco teaton eod fo E : aaa a 3 3 povded a 0 f a =0, do eodeng le pvotng a a a a a,,,, a Wte DLU whee D, LU, ae e dagonal, lowe tangla, and ppe tangla pats of. D LU D D LU 3 c < show> c c c cc c cc ccc 0 c as f 0 c c f heefoe c as.e. c o c 30 Jaco teaton eod DO =,,, (fowad loop) a a a ( ) ( ) ( ) D D LU ( ) ( ) Jaco D LU cjaco D ( ) c Gven dffeent atces DLU dffeent dffeent 3

9 Gass-Sedel teaton eod Jaco eod: DO =,,, (fowad loop) a a ( ) ( ) ( ) a Conseqently,,,, ae copted efoe. Gass-Sedel : ( ) ( ) ( ) ( ) ( ) ( ) ( ) when coptng, eplace y fo,,,. a a ( ) ( ) ( ) a D L U ( ) ( ) ( ) ltenatve fo of Gass-Sedel teaton eod (, ) e esltng vecto afte e step and Defne efoe e step dng e teaton (, ) ( ) ( ) ( ) ( ) ( ) ( ),,,,, Defn e,, gong to do : a a ( ) ( ) ( ) a ts esdal vecto : (, ) (, ) (, ) (, ) coponent : a ( ) ( ) a a ( ) ( ) ( ) a a a D L U ( ) ( ) ( ) ( DL ) U ( ) (, ) ( ) ( ) ( ) coponent : a a a In patcla, we ae nteested n e coponent, ( ) ( ) DL DL U GassSedel GassSedel DL U c DL (0) heoe: If s stctly dagonally donant, en fo any choce of, o e Jaco and Gass-Sedel eods gve seqences at conveges to e nqe solton of. ( ) 0 ( ) a a a (, ) ( ) ( ) ( ) a a a (, ) ( ) ( ) ( ) a eefoe (, ) ( ) ( ) a ~ Gass-Sedel teaton eod ~ 34 36

10 Befoe pdatng e coponent of : (, ) ( ) ( ) ( ) coponent : a a a fte pdatng e coponent of : (, ) ( ) ( ) ( ) coponent : a a a gan, we ae nteested n e coponent, a a a (, ) ( ) ( ) ( ) a a 0 ( ) ( ) (, ) ( ) Gass-Sedel eod enfoces 0 to otan and n sch a way, epects to have a convegence. 37 (, ) ( ) ( ) a ( ) ( ) ( ) ( ) a a a a ( ) ( ) ( ) a a a D L U ( ) ( ) ( ) ( ) D D L U ( ) ( ) ( ) ( ) ( ) ( D L DU ) elaaton elaaton DL DU c DL 39 Relaaton eod Whee an teaton eod conveges o not s pole-dependent. eaple: Gass-Sedel eod (, ) ( ) ( ) a dvege each te coect too ch? convege t slowly coect too lttle? Can we adst e aont of coecton at each te? W a cetan postve vale of, one can pefo (, ) ( ) ( ) a 0< < : nde-elaaton ( dvege) > : ove-elaaton ( convege slowly) = : Gass-Sedel eod 38 Mnzaton Pole Gven: =, whee s syetc and postve-defnte (s all egenvales of ae eal and postve) defne : R R, fo all R he solton of nzes e fncton. <show> = a loo fo e n: 0 a a 0 a a 0 a a 0 40

11 Gadent teaton eod (D nzaton) Sppose s e cent teate and p R s a gven decton vecto. et teate: p w a vale of whch nzes. e.. Mn p R Wanted 0 p p p p p p 0 p p p p p p 4 Gadent teaton eod (D nzaton) Step : tae an ntal gess and tae p Step : f >, p p p p p snce () deceases ostly along e negatve gadent decton. 43 Gadent teaton eod (D nzaton) p p p p p p 0 0 LHS p p p p p p p p p 0 Popetes of Gadent teaton eod heoe: Fo any ntal teate 0, e seqence of e gadent 0 * eod conveges to e solton and satsfes e eo estates * * 0 a n whee e -no s defned as. he salle, e faste e eod conveges. 4 44

12 Congate decton eod In e gadent teaton eod: p p p 0 p 0 0 enfoced fo one each te Congate decton eod p0 = 0 p p0 p 0 p p p p0 = 0 p = 0 p 0 0 he gadent eod seaches e n of n a decton p. ~ hese seachng dectons p ay not e e fastest pa leadng to e n. 0 p p 0 0 p p p 0 0 p p p 0 0 p p 0 p p Congate decton eod 0 p0, 0 0 0p0 p, p Sppose we have oved along soe decton p 0 to a n and now popose to ove along soe new decton p. We don t want f e new coecton spols o nzaton along e p 0 decton o not. =: choose a pope so at = 0 p 0 0 =: choose a pope so at p = 0 and p = 0 0 Congate decton eod Ipoveent: select seachng dectons n a patcla way sch at p 0 ae lnealy ndependent, e.g. ae e oogonal p p =0 fo 0,,,, p p =0 fo and 0, 0 0 hs fo any R, p 0 0.e p s a ass of R and ae e coodnates of w espect to e ass

13 Congate Gadent eod one way to geneate a set of oongal vectos p statng: p0 0 0 p teate: p p 0 p p eslt: If p p 0 fo,,, en p p 0 fo all. <show> fo : p p p p p p 0 p p p p p p p p p p 0 ecept 0 ( s syetc) p p 49 Congate Gadent eod Step : tae an ntal gess and tae p 0 0 p p 0 0 DO Step : and p fo 0 Step 3: and p p fo ED DO * c * heoe: 0, c c a n, 5 Congate Gadent eod Step : tae an ntal gess and tae p 0 0 p p 0 0 DO Step : p and p fo 0 p Step 3: and p p fo 0 p p ED DO heoe: he congate decton algo ns at ost teatons w = *. * c * heoe: 0, c c a n, 50 Pecondtoned Gadent eod Sppose M R R s syetc and postve-defne. Consde = / / M M / / / / M M M M s syetc and postve-defnte ow, nstead of solvng =, we solve he convegence ate s now detened y 5