5 Multivariate Analysis of Spectra

Transcription

1 5 Multvrte Anlyss of Spectr 5.1 Introducton Spectrum mens here: We mesure the ntensty for certn wve lengths. Such functon chrcterzes chemcl mxture (or s specl cse pure sustnce). There re mny spectr n chemstry. For some of them, pure sustnces hve spectrum tht conssts of sngle pek. As long s the peks re not overlppng, we cn dentfy the dfferent components of mxture nd ther proportons. NIR-Spectr (ner nfrred): The NIR-Spectr of pure sustnces s ny functon wth some more or less chrcterstc peks. Hence, t s rther dffcult to dentfy the type nd the quntty of the dfferent components sed on the spectrum of chemcl mxture. On the other sde, these spectr re very chep: No extr processng s needed, they cn e mesured on-lne. Exmple c Qulty Control v NIR-Spectr We hve dt of reflectons of NIR-wves on 52 grnulte smples wth wve length 1100, 1102, 1104,..., 2500 nm. Fgure 5.1.c shows the spectr n centered form; for ech wve length j the medn vlue med (X (j) ws sutrcted from the X (j) s. Wl c Z ) Tle 5.1.c: Dt for the exmple NIR-spectr (for wvelengths lrger thn 1800nm). Questons There re outlers. Are there other structures? The mount of n ctve ngredent ws determned wth chemcl nlyss. Cn we estmte t suffcently ccurte wth the spectrum? d In other pplctons we mesure spectr to follow recton on-lne. It s used for estmtng the order of recton nd to determne potentl ntermedte products nd recton constnts, determnng the end of process, montorng process. We cn lso utomtclly montor slow processes of ll knds. For exmple stockkeepng: Are there ny (unwnted) gng effects? 48

2 5.2 Multvrte Sttstcs: scs 49 Spectrum, med centered Wve length Fgure 5.1.c: NIR-Spectr of grnulte smples, centered t the medn curve. e For ech oservton (smple) we hve mny vrles ( whole spectrum). Questons Is t resonle to plot the dfferent smples on plne or s t possle to ctch most nformton nd see structure from just few dmensons (nsted of usng ll vrles)? Cn we dentfy dmensons (wth techncl nterpretton) n the hgh-dmensonl spce tht contn most of the nformton? Is t possle to dentfy nd to quntfy the dfferent components of chemcl mxture sed on ts spectrum? For regresson nlyss, 70 vrles (or 700 t hgher resoluton) re too much f we only hve 52 oservtons. How should we reduce dmensonlty? 5.2 Multvrte Sttstcs: scs Notton The vector X = [X (1), X (2),..., X (m) ] T denotes the th spectrum. It s pont n m-dmensonl spce. Hence, for ech oservton we mesure m dfferent qunttes. Remrk In sttstcs nd prolty theory vectors re usully column vectors. Row vectors re denoted y the symol T (trnsposed vector). Ths s nconvenent n sttstcs ecuse the dt mtrx X = [X (j) ], tht conssts of n oservtons of m vrles s ult up the other wy round: The th row contns the vlues for the th oservton. For most pplctons ths s useful tle (see e.g. the desgn mtrx of lner regresson model). Here, t s often the other wy round: In tle of spectr, column often contns sngle spectrum (.e., t s one oservton of spectrum).

3 50 5 Multvrte Anlyss of Spectr Defntons We defne the followng qunttes for n m-dmensonl rndom vector X = [X (1), X (2),..., X (m) ] T R m. Expectton µ R m µ = (µ 1,..., µ m ) T, where µ k = E[X (k) ], k = 1,..., m. In other words: vector tht conssts of the (unvrte) expecttons. We wrte µ X n stutons where we lso hve other rndom vrles. Covrnce Mtrx Σ R m m Σ s n m m mtrx wth elements [ ] Σ jk = Cov(X (j), X (k) ) = E (X (j) µ j )(X (k) µ k ). We lso use the notton Vr(X) or Cov(X). Note tht Σ jj = Cov(X (j), X (j) ) = Vr(X (j) ). Ths mens tht the dgonl elements of the mtrx re the vrnces. Σ jk Corr(X (j), X (k) ) = Σ jj Σ kk. Agn, sometmes we wrte Σ X f we wnt to pont out tht ths s the covrnce mtrx tht corresponds to X. c d Lner Trnsformtons For smple (one-dmensonl) rndom vrle: Y = + X, where, R. Expectton: E[Y ] = + µ X. Vrnce: Vr(Y ) = 2 σ 2 X. For rndom vectors: Y = + X, where R m, R m m. Expectton: E[Y ] = + µ X. Covrnce: Cov(Y ) = Σ X T. Remrk The multvrte norml dstruton X N (µ, Σ) s fully chrcterzed y the men µ nd the covrnce mtrx Σ. It s the most common dstruton n multvrte sttstcs. See e.g. Chpter 15.3 n Sthel (2000). Fgure 5.2.d llustrtes two two-dmensonl norml dstrutons wth the contours of ther denstes. The men vector s responsle for the locton of the dstruton nd the covrnce mtrx for the shpe of the contours. e Estmtors µ = [ X (1), X (2),..., X (m)] T = vector of mens 1 n Σ = (X µ)(x µ) T n 1 =1 = mtrx of the emprcl vrnces nd covrnces. Ths mens tht Σ jk = 1 n 1 n (X (j) =1 X (j) )(X (k) X (k) ). The covrnce mtrx plys crucl role n multvrte models tht re sed on the norml dstruton or tht wnt to model lner reltonshps.

4 5.3 Prncpl Component Anlyss (PCA) 51 z (2) x (2) z (1) x (1) Fgure 5.2.d: Contours of the prolty denstes for stndrd norml (left) nd generl (rght) multvrte norml dstruton. 5.3 Prncpl Component Anlyss (PCA) Our gol s dmensonlty reducton. We re lookng for few dmensons n the m-dmensonl spce tht cn expln most of the vrton n the dt. We defne vrton n the dt s the sum of the ndvdul m vrnces m Vr(X (j) ) = 1 n m (j) ( X ) 2, n 1 j=1 =1 j=1 where X (j) re the centered oservtons: X (j) = X (j) X (j). We wnt to fnd new coordnte system wth certn propertes. Ths wll led to new ss vectors k ( k = 1), the so clled prncpl components. The ndvdul components of these ss vectors re clled lodngs. new coordntes Z (k) = X T k, the so clled scores (projectons of the dt on the drectons ove). Wht propertes should the new coordnte system hve? The frst ss vector 1 should e chosen such tht Vr(Z (1) ) s mxml. The second ss vector 2 should e orthogonl to the frst one ( T 2 1 = 0) such tht Vr(Z (2) ) s mxmzed. And so on... Fgure 5.3. llustrtes the de usng two-dmensonl dstruton. To summrze, we re performng trnsformton to new vrles Z = T (X µ), where the trnsformton mtrx s orthogonl. It cn e shown tht s the mtrx of (stndrdzed) egenvectors nd λ k re the egenvlues of Σ X. Rememer tht Σ X s symmetrc mtrx nd therefore we cn decompose t nto Σ X = D T,

5 52 5 Multvrte Anlyss of Spectr where s the mtrx wth the egenvectors n the dfferent columns nd D s the dgonl mtrx wth the egenvlues on the dgonl (ths s fct from lner lger). Therefore we hve Vr(Z) = T ΣX = D = λ 1 λ 2 λ m 0. λ λ λ m Hence, the ndvdul components of Z re uncorrelted nd the frst component of Z hs lrgest vrnce. y constructon t holds tht λ 1 = Vr(Z (1) ). It s the mxml vrnce of projecton: λ 1 = mx : =1 ( Vr(X )). Accordngly for λ m : It s the smllest vrnce. ecuse the λ k re the egenvlues of Σ X, we know from lner lger tht m m λ k = k=1 k=1 Σ kk = m k=1 Vr(X (j) ). Hence kj=1 λ j mj=1 λ j s the proporton of the totl vrnce tht s explned y the frst k prncpl components. Of course we cn lwys go ck to the orgnl dt usng the new vrles y dong smple ck-trnsformton X µ = ( T ) 1 Z = Z = m k=1 Z (k) (k). Grphcl Representton y reducng dmensonlty t gets eser to vsulze the dt. For tht reson we only consder the frst two (or three) components nd forget out the other ones. Fgure 5.3. () llustrtes the frst two components for the NIR-spectr exmple (for techncl resons we only consder wve lengths lrger thn 1800 nm). We cn see 5 outlers they were lredy vsle n the spectr. Fgure 5.3. () shows the frst three components of prncpl component nlyss wthout the outlers.

6 5.3 Prncpl Component Anlyss (PCA) 53 log (wdth) Prncpl Component Prncpl Component Fgure 5.3.: Prncpl component rotton. Comp y th q G w u v j F C I K L M x E RJ N r s T c S n O g z A D mh d kp l f o e V Y Z Comp. 1 X W P U Q Fgure 5.3.: () Sctterplot of the frst two prncpl components for the exmple NIR-spectr.

7 54 5 Multvrte Anlyss of Spectr Y V Z Comp.1 q e o l E C I f da v u w ty pk z R D r m N JLK G H s M h g x j n F O c S T V Z Y o ldh e k m O pdza n SN r T JML VY Y V Z I s u Z EKC f R qh g x j vw F G ty y Ht q G u w j v C F K I xm c r L E JR S T n sn O g z h md A p k d l f o e Comp.2 O h o S T n MNL mr sp z J D ua I K d k l RH E f e Cq gx j vw F G yt c c c c y t qh G w u j vc F K I x E LM R Z r J T V Y s n S N O g kl pd h m DzA f o e q ec l E o yt vw f d R ua G H kk pd z IJ rl g j sm Fx M N h n O S T V Z Y Comp Fgure 5.3.: () Sctterplot mtrx of the frst three prncpl components for the exmple NIR-spectr wthout the 5 outlers. c d PCA s sutle for mny multvrte dt sets. If we re nlyzng spectr we hve the specl cse tht the vrles (the ntenstes of dfferent wvelengths) hve specl orderng. Hence, we cn plot ech oservton s functon. We cn lso llustrte the prncpl component drectons (the lodngs) k s spectr! Sclng Issues If the vrles re mesured n dfferent unts, they should e stndrdzed to (emprcl) vrnce 1 (otherwse comprng vrnces doesn t mke sense). Ths leds to PCA (= egennlyss) of the correlton- nsted of the covrnce mtrx. For spectr ths s not useful ecuse wvelengths wth very vrle ntenstes contn the most mportnt nformton. If we would stndrdze the vrles n tht setup, we would down-weght these vrles compred to the unstndrdzed dt set. e Choosng the numer p of components: (p < m) 2 (mye 3) for llustrtonl purposes. Plot the explned vrnce (egenvlues) n decresng order nd look for rekpont ( scree plot : plot λ k vs. k), see Fgure 5.3.e. Expln 95% of the vrnce : The sum of the egenvlues p j=1 λ j should e 95% of the totl sum m j=1 λ j.

8 5.3 Prncpl Component Anlyss (PCA) 55 Spectrum Comp. 1 Comp. 2 Comp Wve length Fgure 5.3.c: Spectr of lodngs of the frst three prncpl components for the exmple NIRspectr. Vrnces 0.0e e e 05 NIR spectr wthout 5 outlers Vrnces of prnc. components Comp.1 Comp.3 Comp.5 Comp.7 Comp.9 Fgure 5.3.e: Vrnces of the prncpl components (scree plot) for the exmple NIR-spectr. ut: Vrnce m j=1 λ j = m j=1 Vr(X (j) ) s the sum of ll vrnces. There could e (mny) nose vrles mong them! Restrcton to the frst p prncpl components: In the trnsformton formul (5.3.) we smply gnore the lst m p terms: X µ = X + Ê, X = p k=1 Ths cn e nterpreted n the followng two wys. Z (k) (k), Ê = m k=p+1 Z (k) (k). In Lner Alger termnology: The dt mtrx of the X s the est pproxmton of the dt mtrx of the X µ f we restrct ourselves to mtrces wth rnk p (n the sense of the so-clled Froenus norm of mtrces: E 2 = j E2 j ). In sttstcl termnology: We were lookng for p vrles Z (k) = m j=1 kj X (j), k = 1,... p, such tht the dfferences E = X X of X = p k=1 Z(k) (k) show mnml vrnce (n the sum): mj=1 Vr(E (j) ) = m k=p+1 λ k s mnml (there wll e no etter choce thn the vrles Z (k) ).

9 56 5 Multvrte Anlyss of Spectr 5.4 Lner Mxng Models, Fctor Anlyss Model for Spectr Let c k e the spectrum of the chemcl component k nd consder mxture of the components wth coeffcents s = [s (k) ]. For the th mxture we hve the coeffcents s. Accordng to Lmert-eer the spectrum of the th mxture s X = k c (k) s (k) + E = C s + E where E re mesurement errors. C s the mtrx of spectr c k (n the dfferent columns). Ths looks very smlr to 5.3.e. The dfferences re C not orthogonl X nsted of X µ, not centered E rndom vector (mesurement error) s (k) 0 or C jk 0, X (j) 0 f we use the orgnl spectr. c Ths model cn e used for mny pplctons where there re m mesurements tht re lner supermpostons of p < m components. Exmples re: Chemcl elements n rocks tht consst of severl ed-rocks. Trce elements n sprng wter tht rn through dfferent sol lyers. If the source profles (spectr) c k re known, the contrutons s (k) cn e estmted for ech oservton seprtely usng lner regresson. However, t s more nterestng f oth the source profles nd ther contrutons hve to e estmted from dt. Ths cn e cheved usng comnton of sttstcl methods, professonl expertse nd pplcton specfc propertes. 5.5 Regresson wth Mny Predctors In the ntroductory exmple out NIR-spectr we dscussed the queston whether we cn predct the mount of n ctve ngredent sed on spectrum. Hence, we hve response vrle Y nd severl predctors [x (1),..., x (m) ]. If we set up lner regresson model we fce the prolem tht there re mny more predctors thn oservtons. Hence, t s not possle to ft full model (t would led to perfect ft). A possle remedy s to use stepwse regresson: We strt wth just one predctor nd dd the most sgnfcnt predctor n the next step (untl some stoppng crteron s met). Exmple: Grnulte Smples. Y = yeld. n = 44 (wthout outlers ). Tle 5.5. shows computer output. For comprson: Smple correlton etween L2450 nd yeld: r = 0.57, R 2 = etter known re the followng methods to hndle the prolem of hvng too mny predctors 1. Prncpl Component-Regresson, 2. Rdge Regresson 3. New methods lke Lsso, Elstc Net,...

10 5.5 Regresson wth Mny Predctors 57 Vlue Std. Error t vlue Pr(> t ) Sgnf (Intercept) *** L *** L ** L *** L *** L *** L *** L *** Resdul stndrd error: on 36 degrees of freedom Multple R-Squred: Tle 5.5.: Computer output for regresson model fter vrle selecton wth stepwse forwrd. c d Prncpl Component-Regresson PCA of the predctors leds to new vrles [Z (1),..., Z (p) ]. The prncpl components re usully selected wthout exmnng the reltonshp wth the response Y. Vrnt of rown rown (1993): Select them ccordng to smple correlton wth Y! Rdge Regresson An esy wy to ensure tht the mtrx X T X (tht needs to e nvertle for lest squres) s non-sngulr s to dd dgonl mtrx λi, ledng to β λ = (X T X + λi ) 1 X T Y.

11 logrphy tes, D. M. nd Wtts, D. G. (1988). Nonlner regresson nlyss nd ts pplctons, Wley Seres n Prolty nd Mthemtcl Sttstcs, Wley, New York. ennett, J. H. (ed.) (1971). Collected Ppers of R. A. Fscher; , Vol. I, The Unversty of Adelde. oen, J. R. nd Zhn, D. A. (1982). Wdsworth Inc. elmont. The Humn Sde of Sttstcl Consultng, ortz, J. (2005). Sttstk für Sozlwssenschftler, 6 edn, Sprnger, erln. ox, G. E. P. nd Drper, N. R. (1987). Emprcl Model-uldng nd Response Surfces, Wley Seres n Prolty nd Mthemtcl Sttstcs, Wley, New York. ox, G. E. P., Hunter, W. G. nd Hunter, J. S. (1978). Sttstcs for Expermenters, Wley, N. Y. rown, P. J. (1993). Oxford, UK. Crroll, R. nd Ruppert, D. (1988). Wley, New York. Mesurement, Regresson, nd Clrton, Clrendon Press, Trnsformton nd Weghtng n Regresson, Chtfeld, C. (1996). The Anlyss of Tme Seres; An Introducton, Texts n Sttstcl Scence, 5 edn, Chpmn nd Hll, London, NY. Dnel, C. (1976). Applctons of Sttstcs to Industrl Expermentton, Wley Seres n Prolty & Mthemtcl Sttstcs, Wley, New York. Dnel, C. nd Wood, F. S. (1980). Fttng Equtons to Dt, 2 edn, Wley, N. Y. 1st ed Federer, W. T. (1972, 1991). Sttstcs nd Socety: Dt Collecton nd Interpretton, Sttstcs: Textooks nd Monogrphs, Vol.117, 2 edn, Mrcel Dekker, N.Y. Hrmn, H. H. (1960, 1976). Modern Fctor Anlyss, 3 edn, Unversty of Chcgo Press, Chcgo. Hrtung, J., Elpelt,. nd Klösener, K. (2002). Sttstk. Lehr- und Hnduch der ngewndten Sttstk, 13 edn, Oldenourg, München. Hogln, D. C., Mosteller, F. nd Tukey, J. W. (1991). Fundmentls of Explortory Anlyss of Vrnce, Wley, N. Y. Hogg, R. V. nd Ledolter, J. (1992). Appled Sttstcs for Engneers nd Physcl Scentsts, 2 edn, Mxwell Mcmlln Interntonl Edtons. Huet, S., ouver, A., Gruet, M.-A. nd Jolvet, E. (1996). Sttstcl Tools for Nonlner Regresson: A Prctcl Gude wth S-Plus Exmples, Sprnger-Verlg, New York. Lwley, D. N. nd Mxwell, A. E. (1963, 1967). Fctor Anlyss s Sttstcl Method, utterworths Mthemtcl Texts, utterworths, London. Lnder, A. nd erchtold, W. (1982). Sttstsche Methoden II: Vrnznlyse und Regressonsrechnung, rkhäuser, sel. Med, R. (1988). The desgn of experments, Cmrdge Unversty Press, Cmrdge. 58

12 logrphy 59 Myers, R. H. nd Montgomery, D. C. (1995). Response Surfce Methodology; Process nd Product Optmzton Usng Desgned Experments, Wley Seres n Prolty nd Sttstcs, Wley, NY. Petersen, R. G. (1985). Desgn nd Anlyss of Experments, Sttstcs Textooks nd Monogrphs, Mrcel Dekker, N.Y. Rpold-Nydegger, I. (1994). Untersuchungen zum Dffusonsverhlten von Anonen n croxylerten Cellulosememrnen, PhD thess, ETH Zurch. Rtkowsky, D. A. (1989). Hndook of Nonlner Regresson Models, Mrcel Dekker, New York. Renner, R. M. (1993). The resoluton of compostonl dt set nto mxtures of fxed source compostons, Appled Sttstcs Journl of the Royl Sttstcl Socety C 42: Schs, L. (2004). Angewndte Sttstk, 11 edn, Sprnger, erln. Scheffe, H. (1959). The Anlyss of Vrnce, Wley, N. Y. Seer, G. nd Wld, C. (1989). Nonlner regresson, Wley, New York. Sthel, W. A. (2000). Sttstsche Dtennlyse: Ene Enführung für Nturwssenschftler, 3 edn, Veweg, Wesden. Swnourne, E. S. (1971). Anlyss of Knetc Dt, Nelson, London.