Introduction to Simulation - Lecture 5. QR Factorization. Jacob White. Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Transcription

1 Introducton to Smulaton - Lecture 5 QR Factorzaton Jacob Whte hanks to Deepak Ramaswamy, Mchal Rewensk, and Karen Veroy

2 Sngular Example LU Factorzaton Fals Strut Jont Load force he resultng nodal matrx s SINGULAR, but a soluton exsts!

3 Sngular Example LU Factorzaton Fals v v v3 v4 he resultng nodal matrx s SINGULAR, but a soluton exsts!

4 Sngular Example Recall weghted sum of columns vew of systems of equatons x b x b M M MN = xn bn xm + xm + + x M = b N N M s sngular but b s n the span of the columns of M

5 Orthogonal columns mples: M M = 0 j j Orthogonalzaton If M has orthogonal columns Multplyng the weghted columns equaton by th column: M xm + xm + + x M = M b ( ) N N Smplfyng usng orthogonalty: x M M = M b x = ( ) M b ( M ) M

6 Orthogonalzaton Orthonormal M - Pcture M s orthonormal f: M M = 0 j and M M = j Pcture for the two-dmensonal case M M b Non-orthogonal Case M x x Orthogonal Case b M

7 Orthogonalzaton QR Algorthm Key Idea x b x b M M M N = xn bn Orgnal Matrx y b y b Q Q QN = yn bn Matrx wth Orthonormal Columns Qy = b y = Q b How to perform the converson?

8 Orthogonalzaton Projecton Formula Gven M, M, fnd Q= M rm so that M ( ) Q= M M rm = 0 M M r = M M M Q r M

9 Orthogonalzaton Normalzaton Formulas smplfy f we normalze Q = M = M Q Q = M M r Now fnd Q = M rq so that Q Q = 0 r = Q M Fnally Q = Q = Q Q r Q

10 Orthogonalzaton How was a x matrx converted? Snce Mx should equal Qy, we can relate x to y x y M M = xm + x M = Q Q = yq + y Q x y M = r Q M = r Q + r Q r r x y 0 r = x y

11 Orthogonalzaton he x QR Factorzaton x r r x b M M Q Q x = = 0 r x b Upper rangular Orthonormal wo Step Solve Gven QR Step ) QRx = b Rx = Q b = b Step ) Backsolve Rx = b

12 Orthogonalzaton he General Case 3x3 Case M M M3 M M rm M3 r 3M r3m o Insure the thrd column s orthogonal M ( M r ) 3M r M M M r M r M = 3 3 = 0 ( )

13 ( r3 r ) ( r3 r ) Orthogonalzaton M M M M M M M M = Must Solve Equatons for Coeffcents n 3x3 Case 3 3 = M M M M r M M 3 3 M M M M r = 3 M M 3

14 Orthogonalzaton Must Solve Equatons for Coeffcents o Orthogonalze the Nth Vector M M M M N r, N M M N = M M M M r M M N N N N, N N N N nner products requres N 3 work

15 3x3 Case Orthogonalzaton Use prevously orthogonalzed vectors M M M3 M M r Q M3 r3q r 3Q o Insure the thrd column s orthogonal Q M Q Q = 0 r = Q M ( r r ) Q M Q Q = 0 r = Q M ( r r )

16 Basc Algorthm Modfed Gram-Schmdt For = to N For each Source Column r = M M N Normalze N N = Q = M r For j = + to N { r j M j Q M M r Q j j j operatons For each target Column rght of source N = 3 ( N ) N N operatons

17 Basc Algorthm By Pcture Q Q Q Q 3 N r r r r 0 r r r 0 0 r r r 3 N 3 N 33 3N NN

18 Basc Algorthm By Pcture QM MQ MQ 3 QM 44 r r r3 r4 r r3 r4 r 33 r 34 r 44

19 Basc Algorthm Zero Column Q What f a Column becomes Zero? M M N Matrx MUS BE Sngular! r r r3 r N ) Do not try to normalze the column. ) Do not use the column as a source for orthogonalzaton. 3) Perform backward substtuton as well as possble

20 Basc Algorthm Zero Column Contnued Resultng QR Factorzaton 0 Q 0 Q3 Q N 0 r r r r r r r 3 N 33 3N NN

21 Sngular Example Recall weghted sum of columns vew of systems of equatons x b x b M M MN = xn bn wo Cases when M s sngular xm + xm + + x M = b N N Case ) b span{ M,.., MN} b span{ Q,.., QN} Case ) b span{ M,.., M }, How accurate s x? N

22 Mnmzaton Vew Alternatve Formulatons Defnton of the Resdual R: R x b Mx ( ) Fnd x whch satsfes Mx = b Mnmze over all x ( ) R x R x = R x ( ) ( ) ( ) N = Equvalent f b span cols M { ( )} ( ) ( ) Mx = b and mnx R x R x = 0 Mnmzaton extends to non-sngular or nonsquare case!

23 Mnmzaton Vew One-dmensonal Mnmzaton Suppose x= xe and therefo re Mx= xme = xm One dmensonal Mnmzaton R x R x = b xme b xme d dx ( ) ( ) ( ) ( ) = bb xbme + x Me Me ( ) ( ) ( ) ( ) ( ) R x R x = bme ( ) + x Me Me = 0 bme x = e M Me Normalzaton

24 Mnmzaton Vew One-dmensonal Mnmzaton, Pcture Me = M bme x b x = e M Me e One dmensonal mnmzaton yelds same result as projecton on the column!

25 Now Mnmzaton Vew wo-dmensonal Mnmzaton x= xe + x e and Mx= xme + x Me Resdual Mnmzaton R x R x = b x Me x Me b x Me x Me ( ) = b b xb ( ) Me + x Me Me ( ) xb ( ) Me + x Me Me + x x Me Me ( ) ( ) ( ) ( ) Couplng erm ( ) ( )

26 Mnmzaton Vew wo-dmensonal Mnmzaton Contnued More General Search Drectons x= v p + v p and Mx= vmp + v Mp span p, p = span e, e { } { } R x R x = b b vb Mp + v Mp Mp ( ) ( ) ( ) ( ) If Couplng erm p M Mp = vb Mp v Mp Mp + + v v Mp Mp ( ) ( ) ( ) ( ) 0 Mnmzatons Decouple!!

27 Mnmzaton Vew Formng M M orthogonal Mnmzaton Drectons th search drecton equals M M orthogonalzed unt vector p e rjpj p M Mpj j= = = r = j ( Mp ) ( Me ) j ( Mp ) ( Mp ) j j 0 Use prevous orthogonalzed Search drectons

28 Mnmzaton Vew Mnmzng n the Search Drecton Decoupled mnmzatons done ndvdually ( ) Mnmze: ( ) v Mp Mp vb Mp Dfferentatng: v Mp Mp b Mp = 0 v = ( ) ( ) Mp bmp ( ) ( ) Mp

29 Mnmzaton Vew Mnmzaton Algorthm For = to N For each arget Column p = e For j = to - For each Source Column left of target r r = j Mp p p r x= x+ v p p j M Mp j j Mp p p r p Orthogonalze Search Drecton Normalze search drecton

30 Mnmzaton and QR Comparson Q Q Q N Orthonormal M M M e r p e ( ) r p r e rn N p ( e ) rne N M M Orthonormal

31 Search Drecton Orthogonalzed { e }, e unt vectors search drectons,, e {,, } N p p N Unt Vectors Search Drectons M M Orthogonalzaton Could use other sets of startng vectors { bmbmb,,, } {,, p } p N Krylov-Subspace Search Drectons M M Orthogonalzaton Why?

32 Summary QR Algorthm Projecton Formulas Orthonormalzng the columns as you go Modfed Gram-Schmdt Algorthm QR and Sngular Matrces Matrx s sngular, column of Q s zero. Mnmzaton Vew of QR Basc Mnmzaton approach Orthogonalzed Search Drectons QR and Length mnmzaton produce dentcal results Mentoned changng the search drectons