Solving Systems of Equations

Size: px

Start display at page:

Download "Solving Systems of Equations"

Jeremy Gilmore
5 years ago
Views:

PGE : Formultio d Solutio i Geosystems Egieerig Dr. Blhoff Solvig Systems of Equtios Numericl Methods with MTLB, Recktewld, Chpter 8 d Numericl Methods for Egieers, Chpr d Cle, 5 th Ed.

1 PGE : Formultio d Solutio i Geosystems Egieerig Dr. Blhoff Solvig Systems of Equtios Numericl Methods with MTLB, Recktewld, Chpter 8 d Numericl Methods for Egieers, Chpr d Cle, 5 th Ed., Prt Three, Chpters 9 to d pplied Numericl Methods with MTLB, Chpr, d Ed., Prt Three, Chpters 8 to Solvig Systems of Equtios My Sciece & Egieerig problems require the solutio to systems of equtios b b b b Geerl Rule: Need s my (uique) equtios s ukows Guide to puttig equtios i Nturl Form:. Write the equtios i their turl form. Idetify Ukows, d order them. Isolte the ukows. Write Equtios i Mtri form (=b)

2 I Geerl: For =b, we eed sme umber of equtios s ukows ( squre mtri) N equtios = N ukows M equtios > N ukows OVERDETERMINED M equtios < N ukows UNDERDETERMINED Specil Mtrices Digol hs zeros for ll elemets ecept i=j c C dig( c, c,, c ) c Iverse hs property tht - =I d - =I (Impt for solvig systems of equtios) b I b b Symmetric mtri is equl to its trspose 5 6

More Specil Mtrices Tridigol mtrices re squre with ozero etries o mi digol, d the digols bove d below the mi digol b c b c b c b c Positive Defiite mtrices hve ll positive eigevlues ( symmetric,

3 More Specil Mtrices Tridigol mtrices re squre with ozero etries o mi digol, d the digols bove d below the mi digol b c b c b c b c Positive Defiite mtrices hve ll positive eigevlues ( symmetric, digolly domit mtri is lwys positive defiite) Digolly domit mes tht the sum of the bsolute Vlue of the off digols is less th the bsolute vlue of the mi digols ii ij j ji m m m 5 Smll Systems of Equtios 8. Isolte Ukows 8 8 ( ). Grphicl pproch 8. Determits d Crmer s Rule 6

Requiremets for solutio Need s my (uique) equtios s ukows Hvig squre mtri ( equtios d ukows) is t good eough The mtri must be o-sigulr, or the rk() must equl The rows must be lierly

4 Requiremets for solutio Need s my (uique) equtios s ukows Hvig squre mtri ( equtios d ukows) is t good eough The mtri must be o-sigulr, or the rk() must equl The rows must be lierly idepedet 7 Guss Elimitio is Fudmetl procedure for solvig systems of lier equtios Digol systems re esy: Cosider the followig:, b Bck Substitutio = -5/5 = - = 6/ = = -/ = - 8

5 Solvig Trigulr Systems: Bck Substitutio Upper trigulr mtri hs zeros i ll positios below the mi digol. Lower trigulr mtri hs zeros bove the mi digol 9, b The Solutio to the lst equtio: 8 Now tht is kow, it is substituted ito the d equtio: Filly d re substituted ito the st equtio: 9 9 few properties of Systems of Equtios You c write the equtios i y order 8 8 You c lwys multiply both sides by sclr 8 / 6 You c dd/subtrct two equtios to replce eistig oe

6 Nïve Gussi Elimitio (No Pivotig) lwys write system i terms of ugmeted mtri b b b Use Forwrd Elimitio to crete upper trigulr mtri Use sclr multiplictio d row subtrctio/dditio to mke these elemets Zero b b b Solve usig Bck Substitutio Nïve Gussi Elimitio i Spshot Step : Forwrd Elimitio Step : Bck Substitutio b b b b b b b b b b b b b b b b b b

7 How to do Step (Forwrd Elimitio)? b Keep Eq b b Multiply Eq by / d subtrct the product from Eq Multiply Eq by / d subtrct the product from Eq Eq Eq Eq Eq Eq Eq Eq New Coefficiets b b b b b b b b How to do Step (Forwrd Elimitio)? b Keep Eq b Keep Eq b Multiply Eq by / d subtrct the product from Eq Eq Eq Eq Eq Eq b b b b New Coefficiets b b b Look t the emple for Forwrd Elimitio

8 How to do Step (Bck Substitutio)? b b b or b b b Substitute d bck i Eq to fid Substitute bck i Eq to fid Fid from Eq Bck Substitutio b b b ( b ( b ) / b ) / Bck Substitutio Look t the emple for Bck Substitutio Pitflls i Guss Elimitio. Divisio by Zero Pivotig Prtil (Row) Pivotig Colum Pivotig Full Pivotig combies both Roud-off error Lots of subtrctios/dditios cuse error propgtio Pivotig helps Guss Elimitio ot good for N> few hudred. Ill-Coditioed Systems Smll chges i coefficiets result i lrge chges i solutio wide rge of swers c pproimtely stisfy the equtios Roud off error cuses smll chges i coefficiets resultig i lrge solutio errors! 6

9 Prtil Pivotig Sometimes Guss Elimitio results i zero t the pivot Switch two rows t tht poit to void the problem (Row Pivotig) Be sure to move the right-hd side lso This WILL NOT chge the, solutio vector C lso echge colums DO NOT move the right-hd side! This WILL chge the, solutio vector Echgig colums d rows is FULL PIVOTING Robust lgorithms pivot eve whe there is o zero to void roudoff 7 Commo Questios Why c I just multiply row by rbitrry umber? Becuse we multiply both sides of the equtio by the sme umber. So they re equivlet Why c I dd/subtrct two equtios d still hve vlid system? Sice the equtios re lierly idepedet, I get ew idepedet equtio i the vector spce If I multiply row by sclr d the dd/subtrct, should t the multiplied-row chge lso i your mtri? It ctully does t mtter, becuse they re equivlet. It is ofte esier to keep the origil equtio I get differet upper-trigulr mtri whe I perform GE You c get differet mtri depedig o whether you try to dd or subtrct durig forwrd-elimitio 8

10 Review :Lots of egieerig problems ivolve simulteous lier equtios, b Must be N idepedet equtios for N ukows for uique solutio Guss Elimitio is systemtic pproch to obti solutio. Forwrd Elimitio: Series of row opertios used to crete upper trigulr system b b b u u u d u u d u d. Bck Substitutio: Sequetilly clculte N, N-, =b /, d so o 9 Pitflls i Guss Elimitio Sometimes zero ppers i pivot poit Eve smll umbers (er zero) c cuse problem Roud off error c ccumulte with ll those subtrctios Pivotig (row d colum) is used to help with pitflls Switch y two rows (be sure to chge b vector lso) Switch y two colums ( b does t chge, but does) Good progrmmig prctice:. Systemticlly elimites rows. Icludes pivotig

11 LU decompositio steps i Guss Elimitio. Forwrd elimitio O(N ) opertio. Bck substitutio O(N ) opertio Commo to hve to solve =b lots of times, where b chges but ever does Do t wt to wste effort cretig the upper trigulr mtri every time How do I crete it oce without messig with b? 5 & 5 Overview of LU Decompositio We wt to solve X B OR X B Forwrd Elimitio gives u u u d u u d u d UX D We c ssume tht lower-trigulr mtri eists such tht. L U X D X B Multiplyig out we see LU ND LD B

12 Overview of LU Decompositio LU decompositio: Bsic Strtegy (=b). Decompose ito lower [L] d upper [U] trigulr mtri: []=[L][U] /. Solve for ew vector, d usig forwrd substitutio step: [L]d=b. Solve for, usig bck substitutio step: d d 6 / d [U]=d 6 6

13 The Mtri Iverse Hs the followig property: If kow, c be used to solved systems of equtios C be computtiolly difficult, but we c solve series of uit vectors o the right-hd side (RHS); =b. LU decompositio is idel, sice b chges but does t. I I b b b * solutio vectors become the colums of the mtri iverse 5 Mtri Norm ssocites sclr with mtri Norms mesure of the mgitude of the elemets of vector or Mtri Lots of orms for vectors, but L, L d L orms commo For Mtrices we hve lots of orms too L, L d L orms little differet for mtrices Frobeius Norm is commo L 6

14 Frobeius Norm reshpes mtri ito vector d computes orm m ij F i j Emple: Vector : Mtri : ] [.7 ) ( 5.77 ) ( ) ( )... 5 ( F Iduced mtri orms use vector orm to defie relted mtri orm Note: Similr to Vector Norms, there re some other Mtri Norms other th Frobeius Norm. Ech of these Mtri Norms hs its ow defiitio. e.g. (which is clled Colum-Sum Norm) is the mimum vlue betwee summtio of bsolute vlues of ech colum 9 m(9,,8,8) )) ),( ),( 5 ),( m(( 5 You mostly use Frobeius orms d ot other oes.

15 Mtri Coditio Number Cod[ ] Mtri iverse, -, must be kow Coditio # vlue Idetity mtri is ectly; y other mtri greter th If it is pproimtely, the its well-coditioed. If much lrger th, it is ill-coditioed d proe to lot of error I c get differet swer bsed o the orm used 5 6 ; 7 7 Cod [ ] (.9)(5.65) Some Mtlb Built-i fuctios Geerl Method for solvig =b >> = \b (POWERFUL MTLB TOOL!!) Clculte the iverse - >> iv() Determie the orm of mtri, >> orm(,p) % p c be,, if, or fro (frobeius orm) Clculte the coditio umber of mtri >> cod(,p) % p c be,, if, or fro (frobeius orm) Perform LU decompositio of mtri >> [L,U,P] = lu()

Direct Methods for solvig systems of equtios Guss Elimitio d LU decompositio re direct (elimitio) methods Forwrd elimitio to get upper trigulr mtri Substitutio to solve for -vector Some pitflls d

16 Direct Methods for solvig systems of equtios Guss Elimitio d LU decompositio re direct (elimitio) methods Forwrd elimitio to get upper trigulr mtri Substitutio to solve for -vector Some pitflls d drwbcks to direct methods Slow for systems lrger th few hudred (elimitio is O(N ) process) Roudoff error c propgte for lrge equtios eve with pivotig Need ltertive for lrge equtios (>) Idirect (Itertive) Methods offer ltertive to elimitio Ide: Guess vector () d systemticlly refie the root Sort of like fied-poit itertio Write ech of the N equtios eplicitly i terms of,, etc. Solve for ech ew i usig old guess vlues Itertive methods (tht re quite similr) Jcobi Guss-Seidel Successive (Over)-Reltio (SOR)

Covergece Criteri Digol Domit gurtees covergece I ech row, the bsolute vlue of the mi digol is greter th sum of the bsolute vlue of off-digols Sufficiet coditio to gurtee covergece but ot ecessry

17 Covergece Criteri Digol Domit gurtees covergece I ech row, the bsolute vlue of the mi digol is greter th sum of the bsolute vlue of off-digols Sufficiet coditio to gurtee covergece but ot ecessry (Possible tht it will coverge eve if is t digolly domit) Digol Domit ii j j, ji ij Jcobi Itertive method Iitil vlue ssumed for ll ukows, () ={ (), (), N () } Secod estimte, (), is foud by substitutig () ito the right-hd side I geerl (k+)th estimte is clculted from the (k)th estimte j k ( k ) i bi ij j i,,..., ii j ji

Covergig whe re you doe? Cotiue util differeces b/w vlues obtied i itertios re smll (i.e. whe the swer is t chgig much ymore) Stop whe pproimte reltive error of ll the ukows is smller th tolerce k k i i k i tol i,,.

18 Covergig whe re you doe? Cotiue util differeces b/w vlues obtied i itertios re smll (i.e. whe the swer is t chgig much ymore) Stop whe pproimte reltive error of ll the ukows is smller th tolerce k k i i k i tol i,,... Creful it is possible the solutio is t chgig much, but still is t cceptble swer 5 Guss-Seidel Method Jcobi method wits util the ed of itertio; the updtes etire solutio vector, (k), ll t oce Guss Seidel is little more greedy Updte the vlue of i (k) s soo s you clculte it I itertio (k), use recet vlues of i s Guss-Seidel is fster, but my ot coverge s well b j ( k) ( k) j j j ( ) j i j k ( k) ( k) i bi ijj ijj i,,... ii j ji b j ( k) ( k) j j 6 j

19 Successive Reltio ew ew old i i i Reltio is slight modifictio to Guss-Seidel i order to ehce covergece fter ech vlue of is computed, the vlue is modified with weighted verge of previous d preset itertios If =, (- ) is equl to zero d the result is umodified (regulr Guss Seidel) For from to, more weight is plced o previous vlue Used to mke o-coverget system coverge or dmpe oscilltios Clled uderreltio For from to, more weight is plced o preset vlue ssume movig i right directio, but too slow Clled overreltio, or SOR 7 Direct (Elimitio) versus Idirect Methods Types Guss Elimitio LU Decompositio Some others ccurcy Direct No error if frctios used But ROUNDOFF errors occur i computig Errors compoud with subtrctios Speed Elimitio is O(N ) problem Double the equtios, Eight times the time! Itertive Types Jcobi Guss-Seidel Successive Over-Reltio ccurcy Depeds o # of itertios TRUNCTION error due to usig fiite # itertios Roudoff error smll Speed Speed depeds o umber of itertios d N Usully fster th Elimitio techiques for N > 8

20 Review Systems of Lier Equtios Commo i Sciece d Egieerig (d college footbll!) Direct Methods error domited by roudoff Guss Elimitio LU decompositio Itertive Methods error domited by tructio Jcobi Guss Seidel (w/ or w/o reltio) 9 Systems of No-Lier Equtios Cosider: y y y 57 C be re-writte s: f y y (, ) f y y y (, ) 57 I geerl: Solutio cosists of set of -vlues tht result i ll equtios equl to zero f (,,, ) f (,,, ) f (,,, )

Newto s Method used to solve roots of olier equtio Guess Clculte the fuctio d the slope (i.e. derivtive) Fid updted Derived from Tylor series Geerl Solutio

21 Newto s Method used to solve roots of olier equtio Guess Clculte the fuctio d the slope (i.e. derivtive) Fid updted Derived from Tylor series Geerl Solutio for systems of equtios f f f, i, i f f f f, i, i f (- ) i, i, f f f f i i J f New guess vector Jcobi of Prtil derivtives Old guess Fuctio vector t old guess

22 Compriso of D to multi-dimesios D solutio i i f '( i) f( i) Multidimesiol solutio J f i i i, f() re vectors i multidimesios Jcobi, J, the mtri of prtil derivtives Evlute the Jcobi t curret guess, i J J Solutio Strtegy (Guess vector i ). Clculte the vector f( i ), d J (mtri of prtil derivtives). Solve J = -f (gives =J - f) Guss Elimitio LU decompositio Guss Seidel, etc. f (,,, ) f (,,, ) f (,,, ). Updte i+ = i + & Iterte J f i i i

23 Itertios Cotiue itertios while itertios re ot chgig much Similr to Guss Seidel ( i+ - i )/ i is smll Choose lrgest vlue for error Should be creful it is t locl mi We relly wt f()= (but f is vector) Could use dot product f*f < tol 5 Covergece of Multidimesiol Newto Method Like D, Newto s method coverges QUDRTICLLY er the root No gurtee it will coverge to right root (multiple solutios?) Good egieer tries to choose good guess vlue for Eve the, it my still diverge 6

Numerical Methods (CENG 2002) CHAPTER -III LINEAR ALGEBRAIC EQUATIONS. In this chapter, we will deal with the case of determining the values of x 1

Numerical Methods (CENG 2002) CHAPTER -III LINEAR ALGEBRAIC EQUATIONS. In this chapter, we will deal with the case of determining the values of x 1 Numericl Methods (CENG 00) CHAPTER -III LINEAR ALGEBRAIC EQUATIONS. Itroductio I this chpter, we will del with the cse of determiig the vlues of,,..., tht simulteously stisfy the set of equtios: f f...