GG313 GEOLOGICAL DATA ANALYSIS

Transcription

1 GG33 GEOLOGICAL DATA ANALYSIS 3 GG33 GEOLOGICAL DATA ANALYSIS LECTURE NOTES PAUL WESSEL SECTION 3 LINEAR (MATRIX ALGEBRA OVERVIEW OF MATRIX ALGEBRA (or All you ever wated to kow about Lear Algebra but were afrad to ask... A matrx s smply a rectagular array of "elemets" arraged a seres of m rows ad colums. The order of a matrx s the specfcato of the umber of rows by the umber of colums. Elemets of a matrx are gve as aj, where the value of specfes the row posto ad the value of j specfes the colum posto; so a j detfes the elemet at posto,j. A elemet ca be a umber (real or complex, algebrac expresso, or (wth some restrctos a matrx or matrx expresso. For example: A a a 3 a 3 a 3 a 33 a 4 a 4 a 43 Ths matrx, A, has order 4 x 3, the elemet a 3, 3 0, etc. The otato for matrces s ot always cosstet but s usually oe of the followg schemes: matrx - desgated by a bold letter (most commo; captal letter; or letter wth uder-score, brackets or hat (^. Order s also sometmes gve: A (4,3 A s 4 x 3. order - always gve as row x colum but uses letters,m,p dfferetly: (row xm(colum or m(row x (colum elemet - most commoly a j wth row; j colum; (sometmes k,l,p Advatages of matrx algebra maly les the fact that t provdes a cocse ad smple method for mapulatg large sets of umbers or computatos, makg t deal for computers. Also (l Compact form of matrces allows coveet otato for descrbg large tables of data; ( Operatos allow complex relatoshps to be see whch would otherwse be obscured by the shear sze of the data (.e., t ads clarfcato; ad (3 Most matrx mapulato volves just a few stadard operatos for whch stadard subroutes are readly avalable. As a coveto wth data matrces (.e., the elemets represet data values, the colums usually represet the dlferet varables (e.g., oe colum cotas temperatures, aother salty, etc. whle rows cota the samples (e.g., the values of the varable at each depth or tme. Sce there are usually more samples tha varables, such data matrces are usually rectagular havg more rows (m tha colums ( - order m x where m >. Colum vector s a matrx cotag oly a sgle colum of elemets:

2 GG33 GEOLOGICAL DATA ANALYSIS 3 a a a Row vector s a matrx cotag oly a sgle row of elemets: a T a...a Sze of a vector s smply the umber of elemets t cota (, both examples above. Null matrx, wrtte as 0 or 0 (m, has all elemets equal to 0 - t plays the role of zero matrx algebra. Square matrx has the same umbers of rows as colums, so order s x. Dagoal matrx s a square matrx wth zero all postos except alog the prcpal (or leadg dagoal: or D d j 0 for j o zero for j Ths type of matrx s mportat for scalg rows or colums of other matrces. The Idetty matrx (I s a dagoal matrx wth all of the ozero elemets equal to oe. Wrtte as I or I; t plays the role of matrx algebra (A I I A A. A Lower tragular matrx (L s a square matrx wth all elemets equal to zero above the prcpal dagoal: or L j 0 for < j o zero for j A Upper tragular matrx s a square matrx wth all elemets equal to zero below the prcpal dagoal u j 0 for < j o zero for j (If oe multples tragular matrces of the same form, the result s a thrd matrx of the same form. We also have Fullv populated matrx whch s a matrx wth all of ts elemets ozero, Sparse matrx whch s a matrx wth oly a small proporto of ts elemets ozero, ad Scalar whch smply s a umber (.e., matrx wth a sgle elemet.

3 GG33 GEOLOGICAL DATA ANALYSIS 3 3 Matrx traspose (or traspose of a matrx s obtaed by terchagg the rows ad colums of a matrx. So row becomes colum ad colum j becomes row j (also the order of the matrx s reversed. A ; A T A dagoal matrx s ts ow traspose: D T D. I geeral, we fd a j a j Symmetrc matrx s a square matrx whch s symmetrcal about ts prcpal dagoal, so a j a j. Therefore a symmetrcal matrx s equal to ts ow traspose. Skew symmetrc matrx s a matrx whch: A A T symmetrc a j a j So A T A a 0(prcpaldagoal elemetsare zero A skew symmetrc Ay square matrx ca be splt to the sum of a symmetrc ad skew symmetrc matrx: A A + A T + A A T Basc Matrx Operatos: Matrx addto ad subtracto requres matrces of the same order sce ths operato smply volves addto or subtracto of correspodg elemets. So, f A + B C, A a a b b + b + b a ; B b b ; C + b a + b a 3 a 3 b 3 b 3 a 3 + b 3 a 3 + b 3 ad ( A + B B + A ((A + B+ C A +(B +C where all matrces are the same order. Scalar multplcato of a matrx s multplyg a matrx by a costat (scalar:

4 GG33 GEOLOGICAL DATA ANALYSIS 3 4 A a a 3 a 3 a a 3 a 3 where s a scalar. Every elemet s multpled by the scalar. Scalar product (or dot product or er product s the product of vectors of the same sze. a b where a s a row vector (or the traspose of a colum vector of legth, b s a colum vector (or the traspose of a row vector, also of legth, ad s the scalar product of a b. The: a a a 3 ; b b b b 3 ad b + a b + a 3 b 3 Some people lke to vsualze ths multplcato as: x x x x 3 4 b Fg. 3-. Dot product of two vectors. a [ 4 5 ] [ 3 ] Coceptually, ths product ca be thought of as multplyg the legth of oe vector by the compoet of the other vector whch s parallel to the frst: b a b cos Fg. 3-. Graphcal meag of the dot product of two vectors. Thk of b as a force ad a as the magtude of dsplacemet whch s equal to work the drecto of a. Thus: where a b a b cos( a

5 GG33 GEOLOGICAL DATA ANALYSIS 3 5 x x + x x The maxmum prcple says that the ut vector ( makg a a maxmum s that ut vector potg the same drecto as a: If a the cos( cos(0 ad a a cos ( a a. Ths s equally true where d s ay vector of a gve magtude - that vector whch parallels d wll gve the largest scalar product. Parallel vectors thus have cos(, the a b a b ad a b (.e., vectors are parallel f oe s smply a scalar multple of the other - ths property comes from equatg drecto coses where: a b Perpedcular vectors have cos( cos 90 0, so a b 0, where a b. Squarg vectors s smply: a a a T forrowvectors a a T a forcolumvectors Matrx multplcato requres "coformable" matrces. Coformable matrces are oes whch there are as may colums the frst as there are rows the secod: C (m, A (m,p B (p, So, the product matrx C s of order m x ad has elemets cj: p c j a k b kj k Ths s exteso of scalar product - ths case, each elemet of C s the scalar product of a row vector A ad colum vector B. c c c c a a a b b 3 a a b b 3 b 3 b 3 I "box form": c b + b + 3 b 3

6 GG33 GEOLOGICAL DATA ANALYSIS 3 6 Elemets of C are scalar products of correspodg vectors as show here. p B m A m C Fg The matrx product of two matrces. p Order of multplcato s mportat usually A B B A ad uless A ad B are square matrces or the order of A T s the same as the order of B (or vce versa, oe of the two products ca ot eve be formed. Order s specfed by statg: A s pre -multpled by B (for B A Multple products: A s post -multpled by B (for A B D A B C A B C (The order whch the pars are multpled s ot mportat mathematcally. Computatoal cosderatos: C (m A (m,p B (p, volves m p multplcatos ad m (p - addtos, so: E (m, [A (m, p B (p, q ] C (q, gves m p q multplcatos [D (m, q ] C (q, gves m q multplcatos ad E (m, A (m,p [B (p,q C (q, ] gves p q multplcatos A (m,p [D (p, ] gves m p multplcatos

7 GG33 GEOLOGICAL DATA ANALYSIS 3 7 Therefore: (A B C mq(p + totalmultplcatos A (B C p(m + qtotal multplcatos If both A ad B are 00 x 00 matrces ad C s 00 x, the m 00, p 00, q 00, ad. Multplyg usg form l volves ~l x 0 6 multplcatos, whereas form volves x 0 4 ; so computg B C frst, the pre-multplyg by A saves almost a mllo multplcatos ad almost a equal umber of addtos ths example. Therefore order s extremely mportat computatoally for both speed ad accuracy (more operatos lead to a greater accumulato of roud-off errors. Traspose of matrx product s smply multplcato by traspose of the dvdual matrces reverse order: D A B C D T C T B T A T Multplcato bv I leaves the matrx uchaged: A I I A A :I A: :AI Pre-multplcato bv a dagoal matrx: C D A, where D s a dagoal matrx. C s the A matrx wth each row scaled by a dagoal elemet of D: D d d d 33 ; A 3 a a 3 a 3 a 3 a 33 C d d 3 d d a d a 3 d a 3 d 33 a 3 d 33 a 33 d 33 each elemet d each elemet d each elemet d 33 Post-multplcato bv a dagoal matrx produces a matrx whch each colum has bee scaled by a dagoal elemet. C A D C d d 3 d 33 d a d a 3 d 33 a 3 d a 3 d a 33 d 33 where each colum A has bee scaled by the correspodg dagoal matrx elemets, d. The determat of a matrx s a sgle umber represetg a property of a square matrx

8 GG33 GEOLOGICAL DATA ANALYSIS 3 8 (depedet upo what the matrx represets. The ma use here s for fdg the verse of a matrx or solvg smultaeous equatos. Symbolcally, the determat s usually gve as det A, A or A (to dfferetate from magtude. Calculato of a x determat s gve by: A a a Ths s the dfferece of the cross products. The calculato of a x determat s gve by A m m + 3 m 3 ( m where m s the determat wth the frst row ad colum mssg; m s the determat wth the frst row ad secod colum mssg; etc. (The determat of matrx s just the partcular elemet. A example of a 3 x 3 determat follows. A 3 a a 3 a 3 a 3 a 33 m 3 a a 3 a 3 a 3 a 33 a a 33 a 3 a 3 m 3 a a 3 a 3 a 3 a 33 a 33 a 3 a 3 So m 3 3 a a 3 a 3 a 3 a 33 a 3 a a 3 A m m + 3 m 3 (a a 33 a 3 a 3 ( a 33 a 3 a ( a 3 a a 3 For a 4 x 4 determat, each m would be a etre expaso gve above for the 3 x 3 determat oe quckly eeds a computer. A sgular matrx s a square matrx whose determat s zero. A determat s zero f: (l ay row or colum s zero; ( ay row or colum s equal to a lear combato of the other rows or colums. For example: A where row 3(row - row 3.

9 GG33 GEOLOGICAL DATA ANALYSIS 3 9 A (a a 33 a 3 a 3 ( a 33 a 3 a ( a 3 a a 3 [( 4 0( 3] 6[( 4 0(5]+4[( 3 (5] The degree of clusterg symmetrcally about the prcpal dagoal s aother (of may propertes of a determat. The more the clusterg, the hgher the value of the determat. Matrx dvso ca be thought of as multplyg by the verse. Cosder scalar dvso: x b x b xb whch we ca wrte bb Matrces ca be effectvely dvded by multplyg by the verse matrx. Nosgular square matrces may have a verse symbolzed as A -l ad AA -l. The calculato of matrx verse s usually doe usg elmato methods o the computer. For a smple x matrx, ts verse s gve by: A A a A example follows. A ; A Soluto of Smultaeous Equatos AA I A system of smultaeous equatos ukows: x + x + 3 x x 4 b x + a x + a 3 x 3 + a 4 x 4 b a 3 x + a 3 x + a 33 x 3 + a 34 x 4 b 3 a 4 x + a 4 x + a 43 x 3 + a 44 x 4 b 4 ca be wrtte as where Ax b A 3 4 a a 3 a 4 a 3 a 3 a 33 a 34 a 4 a 4 a 43 a 44 coeffcetmatrx

10 GG33 GEOLOGICAL DATA ANALYSIS 3 0 x x x x 3 x 4 ; b b b b 3 b 4 The so A Ax A b (pre multplygboth sdesby A Ix x A b gves the soluto for values of x l, x, x 3, x 4 whch solve the system. The followg example solves for smultaeous equatos. Cosder equatos ukows (e.g., equatos of les the x-y plae: I matrx form ths traslates to: 5x + 7x 9 3x x A x To solve ths matrx, we eed the verse of A: x x 9 b The x A - b where A A b or, box form: 9 :b A :x So, the values x l ad x solve the above system, or

11 GG33 GEOLOGICAL DATA ANALYSIS 3 Computatoal cosderatos x x Whle ths approach may seem burdesome, t s good because t s extremely geeral ad allows a smple hadlg ad a straght forward soluto to very large systems. However, t s true that drect (elmato methods to the soluto are fact qucker for fully populated matrces: l Soluto to verse matrx approach volves 3 multplcatos for the verso ad m more multplcatos to fsh the soluto, where s the umber of equatos per set, ad m s the umber of sets of equatos (each of the same form but dfferet b matrx. The total umber of multplcatos s 3 + m. Soluto to drectly solvg equatos volves 3 /3 + m. So, whle the matrx form s easy to hadle, oe should ot ecessarly always use t bldly. We wll cosder may stuatos for whch matrx solutos are deal. For sparse or symmetrcal matrces, the above relatoshps may ot hold. The rak of a matrx s the umber of learly depedet vectors t cotas (ether row or colum vectors: A Sce row 3 -(row - (row or col 3 col - /4(col ad col 4 -(col + 3/4(col, the matrx A has rak (.e., t has oly learly depedet vectors, depedet of whether vewed by rows or colums. The rak of a matrx product must be less tha or equal to the smallest rak of the matrces beg multpled A (rak B (rak C (rak Therefore, (from aother agle, f a matrx has rak r, tha ay matrx factor of t must have rak of at least r. Sce the rak caot be greater tha the smallest of m or, a mx matrx, ths defto also lmts the sze (order of factor matrces. (That s, oe caot factor a matrx of rak, to matrces of whch ether s of less tha rak, so m ad of each factor must also be. The trace of a square matrx s smply the sum of the elemets alog the prcpal dagoal. It s symbolzed as tr A. Ths property s useful calculatg varous quattes from matrces. Submatrces are smaller matrx parttos of the larger supermatrx: Supermatrx F A B C D Such parttog wll frequetly be useful. Other useful matrx propertes:

12 GG33 GEOLOGICAL DATA ANALYSIS 3. A T T A. A A 3. A T A T A T 4. D ABC,the D C B A ;recall that D ABC; D T C T B T A T Ths "reversal rule" for verse products may be useful for elmatg or mmzg the umber of matrx verses requrg calculato. We wll look at a few examples of matrx mapulatos. For data matrx A: ad ut row vector j: A j (l Compute the mea of each colum vector A (each colum has legth 3: x c 3 ja j A j The :A j : :x c ( Compute the mea of each row vector A: The x r 3 Aj T Aj T : j T A : : x r Last tme we looked at the matrx equato ad we foud that the soluto could be wrtte A x b

13 GG33 GEOLOGICAL DATA ANALYSIS 3 3 x A b For a momet, let us just cosder the left had sde A. x. For ay x, ths product gves a ew vector y. We ca say that x s trasformed to gve y. Ths s a lear trasformato sce there are oly lear terms the matrx multplcato,.e., the vector y s y x + x + 3 x 3 x + a x + a 3 x 3 a 3 x + a 3 x + a 33 x 3 Thus we call the operato T(x A. x a lear trasformato. If we stck to three or less dmesos, t s possble to graphcally vsualze vectors ad operatos o them. Fgure 3-4 shows a arbtrary vector x ad the result y of the lear trasformato y A. x. y x Fg The vector x s trasformed to aother vector y usg a lear trasformato. Obvously, as we pck aother x, we get aother y. We mght wat to kow f there are certa vectors that, whe beg operated o, returs a vector the same drecto, possbly loger or shorter tha the orgal x. I other words, are there a x that satsfes Ax x (3. We call the egevalue ad x the egevector. We ca rewrte ths as A x x 0 A x I x 0 A I x 0 or B x 0 I geeral, the soluto to ths equato ca be wrtte x B -l. : x B B or

14 GG33 GEOLOGICAL DATA ANALYSIS 3 4 B x 0 So apart from the trval soluto x [0 0 0], the aswer s gve whe We kow the determat of B s B 0 B 3 a a 3 a 3 a 3 a 33 0 Wrtg out what the determat s ad settg t to zero gves a polyomal of order. For 3 ths wll geeral gve a cubc equato; for a quadratc equato must be solved. The solutos,... etc. are called the egevalues of A, ad the equato B 0 s called the characterstc equato. For example, gve A let us fd ts egevalues. We set A I or We easly solve for : ± 4(, So the egevalues are, -. We ow kow what must be for (3. to be satsfed, but what about the vectors x? We stll have't foud what they must be, but we wll substtute the value for (3.. Usg frst, we fd Fd A x x (A I x x x 0 0 or 5x 6x 0 45x 8x 0 whch both gve

15 GG33 GEOLOGICAL DATA ANALYSIS 3 5 x 5 x So x t 5 where t s ay scalar. Smlarly, for -, we fd (A + I. x x x 0 0 whch reduces to whch gves 3x x 0 x t 3 It may happe that the characterstc equato gves solutos that are magary. However, f the matrx s symmetrc t wll always yeld real egevalues, ad as log as the matrx A s ot sgular, all the wll be o-zero ad the correspodg egevectors wll be orthogoal. The techque we've used apples to matrces of ay sze, but fdg the roots of large polyomals s paful. Usually, the are foud by matrx mapulatos that volve successve approxmatos to the x. Ths s of course oly practcal o a computer. If we restrcted our atteto to -D geometry, certa propertes of egevalues ad egevectors wll be clearer. Cosder the matrx A A (4,8 (8,4 Fg Graphcal represetato of two vectors the x-y plae. We ca regard the matrx as two row vectors [4 8] ad [8 4]. Let us fd the egevalues ad egevectors of A :

16 GG33 GEOLOGICAL DATA ANALYSIS The egevectors are: 8 ± , x x x x 0 0 8x +8x 0 x x e T x x x x 0 0 8x +8x 0 x x e T We fd that the egevectors defe the mor ad major axs of the ellpse whch goes through the two pots defed by (4,8 ad (8,4. The legth of these axes are gve by the absolute values of the egevalues, ad 4. e e λ 4 λ Fg The egevectors, scaled by the egevalues, ca be see to represet the major ad mor axes of the ellpse that goes through the two data vectors (8, 4 ad (4, 8.

17 GG33 GEOLOGICAL DATA ANALYSIS 3 7 It s customary to ormalze the egevectors so that ther legth s uty. I our case we fd e T ad e T The axes of the ellpse are the smply v e v e Sce the sg of the egevector s determate we choose to make all egevalues postve ad thus place the mus-sg sde e. You'll otce that v l. v 0,.e., they are orthogoal. The egevectors make up the colums a ew matrx V V e e Let us expad the egevalue equato (3. A. x x to a full matrx equato. We have A e e A e e We ca combe these two matrx equatos to oe. where A V V 0 0 From ths we may lear two thgs. l Post multply by V - : A V V A V V The egevalue-egevector operato let us splt a symmetrc matrx A to a dagoal matrx (wth the egevalues alog the dagoal ad the matrx V (wth the egevectors as rows ad ts verse V -. Pre-multplyg by V - : V A V Ths operato trasforms the A matrx to a dagoal matrx. It correspods to a rotato of the coordate axes whch the egevectors V becomes the ew coordate axes. Relatve to the ew coordates, coveys the same formato as A does the old coordates. Because s a smple dagoal matrx, the rotato (trasformato makes the relatoshps betwee rows ad colums A much clearer. Smple Regresso ad Curve Fttg Whereas a terpolat fts each data pot exactly, t s frequetly advatageous to produce a smoothed ft to the data - ot exactly fttg each pot, but producg a "best" ft. A popular (ad

18 GG33 GEOLOGICAL DATA ANALYSIS 3 8 coveet method for producg such fts s kow as the method of least squares. The method of least squares produces a ft of a specfed (usually cotuous bass to a set of data pots whch mmzes the sum of the squared msmatch (error betwee the ftted curve ad data. The error ca be measured as Fgure 3-: Ths regresso of y o x s the most commo method. Less commo methods (more work volved s regresso of x o y ad orthogoal regresso (whch we wll retur to later. y error x Fg Graphcal represetato of the regresso errors used least-squares procedures. y error y error x Fg Two other regresso methods: regressg x o y ad orthogoal regresso. x Least squares smple example Cosder fttg a sgle "best" lear slope to data pots. Ths ca be a scatter plot of y(t, x(t plotted at smlar values of t; or a smple f(x relatoshp. At ay rate, y s cosdered a fucto of x. We wsh to ft a le of the form y + a (x (3. ad must therefore determe a value for ad a whch produces a le that mmzes the sum of the squared errors (x 0 s specfed beforehad. So mmze (y computed y observed Ideally, for each observato y at x we should have

19 GG33 GEOLOGICAL DATA ANALYSIS a (x y + a (x y + a (x 3 y 3 + a (x y There are may more equatos ( - oe for each observed value of y tha ukows ( - ad a. Such a system s over-determed ad there exsts o uque soluto (uless all the y 's do le exactly o a sgle le, whch case ay two equatos wll uquely determe ad a. I matrx form (.e., Ax b: (x (x (x a y y y (3.3 Sce A s a o-square matrx t has o verse, so the equato caot be verted ad solved. Cosder stead the error the ft at each pot: + a (x y e +a (x y e + a (x y e We wsh to determe the values for ad a that mmze e Ths wll mmze the varace of the resduals about the regresso le ad gve the leastsquares ft. Notato: S E(,a e ( e T e matrxform So, E(,a ad the mmum of ths fucto (wth respect to the two ukow coeffcets ca be determed usg smple dfferetal calculus, where E(,a a E(a,a a 0 (at themmum (3.4 E a a e + a (x y +a (x y 0 E a a e a + a (x y

20 GG33 GEOLOGICAL DATA ANALYSIS 3 0 +a (x y (x 0 These two equatos ca ow be expaded to ther dvdual terms formg what are kow as the ormal equatos: + a (x y ( x + a (x y (x The ormal equatos thus provde a system of equatos ukows whch ca be uquely solved. Rearragg, ( x + a (x y + a (x y (x Notce that all sums are sums of kow values that sum to smple costats. Solvg: substtute ths to the secod equato: y a (x y a (x (x + a (x y (x Now solve for a : (x a (x y + a (x y (x a (x (x y (x y (x Fally, a y (x y (x (x x 0 (x Substtute a to the frst equato to solve for. So, we compute the sums y, y (x, (x, ad (x ad substtute to the above equato to gve ad a producg the best ft. I matrx form the ormal equatos are: (x (x (x a y (3.5 y (x

21 GG33 GEOLOGICAL DATA ANALYSIS 3 So, Nx B, e.g., of the form Ax b. Sce N s square ad of full rak, ths equato s solved the stadard maer: or N Nx N B x N B Ths problem was smple eough ( x to solve brute force for ad a. For larger systems ths becomes mpractcal ad a matrx soluto to the o-square A x b equato must be sought. We wll ext look at the geeral lear least-squares problem ad fd a soluto matrx otato. Geeral Lear Least Squares We have looked at a few specal cases where we have sought to ft a "model" to "data" a least-squares sese. Fttg a straght le to the x-y pots was a very smple example of ths techque. We wll ow look at the more geeral problem of fdg the coeffcets for ay lear combato of bass fuctos that fts some data a least squares sese. There are umerous stuatos where ths s eeded: Stuato Model Data Curve Fttg Coeffcets of polyomals, s/cos, etc. Pots x-y plae Gravty modelg Destes of subsurface polygos Gravty observatos Hypoceter locato Small pertubatos to hypoceter Sesmc arrval tmes posto Whle the bass fuctos these cases are all vastly dfferet, they are all used a lear combato to ft the observed data. We wll therefore take tme to vestgate how such a problem s set up, ad how t ca all be smplfed wth matrx algebra. Geeral (lear least squares. Cosder the least squares fttg of ay cotuous bass of the form For example x, x, x 3,, x m polyomal bass Fourer se bass x x 0 x x x 3 x x s (π x/ T x s (4π x/ T x 3 s (6π x/ T

22 GG33 GEOLOGICAL DATA ANALYSIS 3 x m x m x j s (mπ x/ T We desre to ft a equato of the form y x +a x + + a m x m to a data set of data pots, where > m, by mmzg E: E (e ( x +a x + +a m y where y s the observed value. We ca wrte ths explctly: (3.6 x + a x + + a m x m y e x + a x + + a m x m y e x + a x + + a m x m y e where x j s the j'th x fucto of the bass, evaluated at the value x. To mmze E, we set E(a j a j 0 (3.7 So E( a ( x + a x + + a m y ( x +a x + +a m y x 0 or E(a a E(a m a m a ( x +a x + + a m y ( x +a x + +a m y x 0 Rearragg these ormal equatos gves a ( x +a x + + a m y m ( x +a x + +a m y 0 E(a j (a x +a x + + a m y x j 0 j

23 GG33 GEOLOGICAL DATA ANALYSIS 3 3 x +a x x + + a m x y x x x +a x + + a m x y x or (for each j : x +a x + + a m y x x j + a x x j + + a m x j y x j Ths provdes a closed system of m ormal equatos, oe for each coeffcet, e.g., E(a j a j 0 for j,,..., m. I matrx form x x x x x x x x x x a a m y x y x y or smply N x B where N s the (kow coeffcet matrx, x the vector wth the ukows (a j, ad B cotas weghted sums of kow (observed quattes. Solve for the a j the x vector (N s square ad of full rak: N N x N B x N B where x s the soluto for the a j. The resultg a j values are the oes whch satsfy (3.7 ad therefore the same oes whch, combato wth the chose bass, produce the "best" ft to the data pots such that (3.6 s mmzed. Last tme we foud the soluto to a geeral lear least squares problem led us to the matrx form

24 GG33 GEOLOGICAL DATA ANALYSIS 3 4 x x x x x x x x x x a a m y x y x y or smply N x B (3.8 where N s the (kow coeffcet matrx, x the vector wth the ukows (a j, ad B cotas weghted sums of kow (observed quattes. Solve for the a j the x vector (N s square ad of full rak: N N x N B x N B (3.9 where x s the soluto for the a j. The resultg a j values are the oes whch satsfy (3.7 ad therefore the same oes whch, combato wth the chose bass, produce the "best" ft to the data pots such that (3.6 s mmzed. We wll ow look at a smpler approach to the same problem usg matrx algebra. We have x x x x x m x m x x x m a a m y y y 3 y e e e 3 e A x b e Sce m <, the A matrx s rectagular. Ths ca be wrtte x x x m y x x x m y x x x m y a a m e e e 3 e C X e These matrces descrbe the system lsted earler. We wsh to fd the a j values whch mmze E e T e. The matrx form ca be parttoed as A b x e

25 GG33 GEOLOGICAL DATA ANALYSIS 3 5 A b x A x + ( b A x b e We wll, for ow, refer to the [A -b] matrx as the C matrx ad the [x ] T colum vector as the X vector. So, we have C X e (recall C s a x m+l rectagular matrx. The 'th error, e, s the dot product: C X e x x y a a m a a m x x y where C s the 'th row vector C. The squared 'th error, e s the or e T e C X T C X X T C T C X e T e a a m x x y x x y a a m where we have used the reversal rule for trasposed products. The sum of the e over all the 's s thus e T e X T C T C X a a m x x y x x y a a m X T C T The product C T C ca be computed to form a ew matrx R. Sce C T m+,c,m+ R m+,m+ the resultg R matrx s square ad symmetrc. So, To mmze E, we fd C E e T e X T C T C X X T R X X E a j a j a j X T R X X T a j For the d coeffcet (as a example, we get R X + X T R X a j

26 GG33 GEOLOGICAL DATA ANALYSIS 3 6 Thus, the partal dervatve of the error s X T X a a T E a R a a m + a a m R For all coeffcets we set all the partal dervatves to zero: where E a j a j 0 X T R X +X T R X X T X T a j I m O m where I m s the m x m detty matrx ad O m the ull vector of legth m. Cosder frst R ( C T C: C: x x x m y x x x m y x x x m y C T : x x x x x x x x x x x x x y x x y x :R x m x m x m y y y x x y y y x x y y The R matrx should look famlar. Cosder the parttoed matrx multplcato:

27 GG33 GEOLOGICAL DATA ANALYSIS 3 7 R C T C (m x A ( x m -b m+ m+ m+ A T (m x A T A (m x m T -A b m+ -b T -b T A b T b Notce that A T A s matrx N of the ormal equatos ad -A T b (-b T A T equal matrx B. Because R s symmetrcal (so R R T we have X T R R X T So E(a j a j X T R X + X T R X 0 X T R X 0 X T R X 0 Cosder X T R :

28 GG33 GEOLOGICAL DATA ANALYSIS 3 8 m+ R -A b TAT A m+ -b T A b T b m+ m+ T Ẋ I 0 (m x m m A T A T -A b m Always multpled by zero N B Ad X T R X : x m+ m+ T A A T -A b m T Ax - A Nx - B T A b Fally, sce X T R X A T A x A T b 0 A T A x A T b A T A A T A x A T A A T b or x A T A A T b x N B (3.0

29 GG33 GEOLOGICAL DATA ANALYSIS 3 9 as before. Therefore, the ukow values of the a j ( the x vector ca be solved for drectly from the system: x + a x + + a m x m y x + a x + + a m x m y x + a x + + a m x m y or smply A x b where A s of order x m wth > m. The least squares soluto the becomes x A T A A T b where A T A s a square matrx of full rak, wth order r m, ad thus vertable. Fttg a straght le, revsted We wll aga cosder the best-fttg le problem, ths tme wth errors the y-values. We wat to measure how well the model agrees wth the data, ad for ths purpose wll use the fucto,.e. a,b y a bx (3. Mmzg wll gve the best weghted least squares soluto. Aga we set the partal dervatves to 0: y 0 a bx a (3. b 0 y a bx x Let us defe the followg: S S x x S y y S xx x S xy x y The (3. reduces to as + bs x S y Wth we fd as x + bs xx S xy SS xx S x

30 GG33 GEOLOGICAL DATA ANALYSIS 3 30 a S xx S y S x S xy b SS xy S x S y (3.3 All ths s swell but we must also estmate the ucertates a ad b. For the same we may get large dffereces errors a ad b. Although ot show here, cosderato of propagato of errors shows that the varace f the value of ay fucto s f f y (3.4 For our le we ca drectly fd the dervatves of a ad b wth respect to y from (3.3: Fgure 3-9. The ucertaty the le ft depeds to a large extet o the dstrbuto of the x-postos. Isertg ths result to (4.7 the gves a S S x xx x y b Sx S x y a S S x xx x S xx S xx S x x + S x x S xx S xxs x x + S x x S xxs S xxs x + S xs xx ad S xx S xx S S x S xx (3.5 Sx S x b S x SS x x + S x

31 GG33 GEOLOGICAL DATA ANALYSIS 3 3 S x SS x x + S x S S xx SS x + S x S S S xxs S x S (3.6 Smlarly, we ca fd the covarace ab from ab a y The, the correlato coeffcet becomes b y S x r S x SS xx (3.7 It s therefore useful to shft the org to x where r 0. Fally, we must check f the ft s meagful. We use the value computed ad get crtcal χ α for - degrees of freedom whch we use to see f exceeds ths value. If t does't we may say the ft s sgfcat at the level. What f some data costrats are more relable tha others? We may gve that resdual more weght tha the others: e e e I geeral, we ca use weghts w for each error so that the ew error e ' e w. We do ths by troducg a weght matrx w whch s a dagoal matrx: w e w w w Now the sum of the squared errors, E, becomes E e T e e T w T w e e T W e where we have troduced W w T w: Sce we have w T e w(a T x - b we obta E (w A x w b T (w A x w b(x T A T wt b T wt (w A x w b x T A T w T w A x x T A T w T w b b T w T w A x + b T w T w b We substtute for W. The

32 GG33 GEOLOGICAL DATA ANALYSIS 3 3 E a j 0 x T A T W A x + x T A T W A x x T A T W b b T W A x Sce x oly cotas the a j, we have x T x I. We fd A T W A x + x T A T W A A T W b b T W A 0 Aga, the d ad 4th term are the traspose of the st ad 3rd. Because each term represet a symmetrcal matrx our equato reduces to A T W A x A T W b 0 whch gves us the weghted lear least squares soluto x A T W A A T W b (3.8