Chapter 5 Matrix Approach to Simple Linear Regression

Transcription

1 Chapter 5 Matrx Approach to Smple Lear Regresso Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people, tems, plats, amals,...) ad colums wll represet attrbutes or characterstcs The dmeso of a matrx s t umber of rows ad colums, ofte deoted as r x c (r rows by c colums) Ca be represeted full form or abbrevated form: a a a j a c a a a j a c A aj,..., r; j,..., c a a aj a c a r ar arj arc Specal Types of Matrces Square Matrx: Number of rows = # of Colums b b A 8 4 B b b Vector: Matrx wth oe colum (colum vector) or oe row (row vector) d 57 d C 4 D E' 7 3 ' f f f3 d F 3 r c 8 d4 Traspose: Matrx formed by terchagg rows ad colums of a matrx (use "prme" to deote traspose) G ' G 3 5 h h c h hr H h j,..., r; j,..., c ' h j,..., ;,..., rc H j c r cr hr hrc h c h rc

2 Matrx Equalty: Matrces of the same dmeso, ad correspodg elemets same cells are all equal: 4 6 b b A 0 = B b 4, b 6, b, b 0 b b Example: Bollywood Box Offce Data Data orgal (o-trasformed) uts. Respose Vector: Y' Desg Matrx: Y Y Y Y Y Y Y ' As we wll see below, the matrx cludes a colum of s, ths wll allow for the tercept term.

3 Y Each row represets a dvdual flm. There are =55 rows both ad Y.

4 Matrx Addto ad Subtracto Addto ad Subtracto of Matrces of Commo Dmeso: C 0 D 4 6 C D C D a a c b b c A rc a j,..., r; j,..., c b B j,..., r; j,..., c rc ar a rc br b rc a b a c b c A B aj b j,..., r; j,..., c rc a rbr arc b rc a b a c b c AB aj b,..., ;,..., rc j r j c ar br arc b rc Regresso Example: Y E Y Y E Y E Y Y E Y Y E Y sce Y E Y ε Y E Y ε EY Y E Y Y EY EY Matrx Multplcato Multplcato of a Matrx by a Scalar (sgle umber): 3() 3() 6 3 k 3 A 7 ka 3( ) 3(7) 6 Multplcato of a Matrx by a Matrx (#cols( A) = #rows( B)): If c r : A B A B AB = ab,..., r ; j,..., c ra ca rb cb ra cb ab c r th th j sum of the products of the A B elemets of row of A ad j colum of B: 5 3 A 3 B (3) 5() ( ) 5(4) 6 8 A B AB 3(3) ( )() 3( ) ( )(4) 7 7 0(3) 7() 0( ) 7(4) j A B c If c r c : A B AB = ab = a b,..., r ; j,..., c A B j k kj A B ra ca rb cb ra cb k

5 Note that the (,j) th elemet of AB s the sumproduct of the th row of A ad the j th colum of B. Also, t s mportat to remember that ulke matrx addto ad subtracto, matrx multplcato s ot elemet by elemet. Examples of Matrx Multplcato Smultaeous Equatos: a x a x y a x a x y ( equatos: x, x ukow): a x a x y ax ax y a a x y a a x y A = Y Sum of Squares: Regresso Equato (Expected Values): Matrces used smple lear regresso (that geeralze to multple regresso): YY ' Y Y Y Y Y Y Y ' Y 0 Y Y 0 0 ' Y β Y Y 0

6 Example: Bollywood Box Offce Data Y'Y ' 'Y Specal Types of Matrces Symmetrc Matrx: Square matrx wth a traspose equal to tself: A = A': A A' A Dagoal Matrx: Square matrx wth all off-dagoal elemets equal to 0: b 0 0 A b 0 B Note:Dagoal matrces are symmetrc (ot vce versa) b 3 Idetty Matrx: Dagoal matrx wth all dagoal elemets equal to (acts lke multplyg a scalar by ): 0 0 a a a3 a a a3 I 0 0 A a a a IA AI A a a a a3 a3 a 33 a3 a3 a 33 Scalar Matrx: Dagoal matrx wth all dagoal elemets equal to a sgle umber" k k k k I 0 0 k k Vector ad matrx ad zero-vector: J 0 Note: ' r ' r rr r r r All of these matrces have mportat roles regresso aalyss, partcularly whe we have multple predctor varables. Software packages (cludg R ad ECEL (to a lesser extet)) ca be used to make all relevat matrx computatos. Lear Depedece ad Matrx Rak Lear Depedece: Whe a lear combato of the colums (rows) of a matrx produces a zero vector (oe or more colums (rows) ca be wrtte as lear fucto of the other colums (rows)) Rak of a matrx: Number of learly depedet colums (rows) of the matrx. Rak caot exceed the mmum of the umber of rows or colums of the matrx. rak(a) m(r A,c a ) A matrx s full rak f rak(a) = m(r A,c a ) rr J rr

7 3 A 3 Colums of are learly depedet rak( ) = 4 A A A A 0 A A 4 3 B 0 0 Colums of are learly depedet rak( ) = 4 B B B B 0 B B For all well posed regresso problems, ad wll be full rak. Matrx Iverse Note: For scalars (except 0), whe we multply a umber, by ts recprocal, we get : (/)= x(/x)=x(x - )= I matrx form f A s a square matrx ad full rak (all rows ad colums are learly depedet), the A has a verse: A - such that: A - A = A A - = I A 4 A A A I / B 0 / 0 B / 6 BB - 4/ / I / Obtag verses of x matrces s very smple. For matrces that are larger tha x, we wll use R ad/or ECEL to obta verses whe ecessary. Note that all software packages are terally dog the computatos lear regresso programs.

8 Computg the Iverse of a x Matrx a a A full rak (colums/rows are learly depedet) a a Determat of A A a a a a Note: If A s ot full rak (for some value k): a ka a ka A aa aa kaa aka 0 ' a a - A Thus does ot exst f s ot full rak a a A A A Whle there are rules for geeral r r matrces, we wll use computers to solve them Regresso Example: Note: ' ' '

9 Example: Bollywood Box Offce Data SS ' 55(765.43) Solvg Smultaeous Equatos wth a Matrx Iverse AY = C where A ad C are matrces of of costats, Y s matrx of ukows A AY A C Y = A C (assumg A s square ad full rak) Equato : y 6y 48 Equato : 0y y 6 y 48 - A 0 Y y C Y = A C A ( ) 6(0) Y = A C Note the wsdom of watg to dvde by A at ed of calculato! Serous roudg errors ca occur whe dvdg by the determat too early maual computatos. Some Useful Matrx Results (Assumg Coformablty of Matrces to Operatos All rules assume that the matrces are coformable to operatos: Addto Rules: A B B A ( A B) C A ( B C) Multplcato Rules: (AB)C A(BC) C( A B) CA + CB k( A B) ka kb k scalar Traspose Rules: ( A ')' A ( A B )' A ' B' ( AB)' B'A' (ABC)' = C'B'A' Iverse Rules (Full Rak, Square Matrces): (AB) = B A (ABC) = C B A (A ) = A (A') = (A )'

10 Radom Vectors ad Matrces Show for case of =3, geeralzes to ay : Radom varables: Y, Y, Y Y Y Y Y 3 3 E Y Expectato: EY E Y I geeral: EYj E Y,..., ; j,..., p p E Y 3 Varace-Covarace Matrx for a Radom Vector: Y E Y Y E Y E Y Y E Y ' E Y E Y Y EY Y EY Y3 EY 3 Y3 E Y 3 Y EY Y EY Y EY Y EY Y3 E Y 3 Y EY Y EY Y EY Y EY Y3 E Y 3 Y3 EY3 Y EY Y3 EY 3Y EY Y3 E Y 3 3 E Lear Regresso Example ( = 3) Error terms are assumed to be depedet, wth mea 0, costat varace : j E 0, 0 j ε E ε σ ε 0 0 I Y = β + ε E Y E β + ε β E ε β σ Y σ β + ε σ ε 0 0 I

11 Mea ad Varace of Lear Fuctos of Y A matrx of fxed costats k Y radom vector a a Y W ay... a Y W AY radom vector: k W a a Y W a Y... a Y k k k k k E W E W EW k E a Y... a Y a E Y... a E Y EakY... aky akey... akey a a E Y AE Y a a E Y k k σ W = E AY - AE Y AY - AE Y ' = E A Y - E Y A Y - E Y ' = E A Y - E Y Y - E Y 'A' = AE Y - E Y Y - E Y ' A' = Aσ Y A' Multvarate Normal Dstrbuto Geeral Case Y Y Y μ = E Y Σ = σ Y Y Multvarate Normal Desty fucto: f / / - Y Σ exp Y - μ 'Σ Y - μ Y ~ Nμ, Σ ~,,..., Y N Y, Y j Note, f A s a (full rak) matrx of fxed costats: W = AY ~ N Aμ, AΣA' j j

12 Bvarate Normal Desty wth Smple Lear Regresso Matrx Form Smple Lear Regresso Model: Y,..., Y 0 Y 0 Y 0 Defg: 0 Y 0 Y Y 0 0 Y= β ε 0 Y = β + ε sce: β EY Assumg costat varace, ad depedece of error terms : σ Y = σ ε I 0 0 Further, assumg ormal dstrbuto for error terms : Y~ N β, I

13 Least Squares Estmato Matrx Form Q Q Normal equatos obtaed from:, ad settg each equal to 0: b b Y 0 0 b b Y Note: I matrx form: ' 0 Y b0 'Y Defg b b Y 'b = 'Y b = ' 'Y - Based o matrx form: Q Y - β ' Y - β = Y'Y - Y'β - β''y + β''β Y ' Y 0Y Y 0 0 Q Y 0 0 ( Q) ' Y ' β Q Y 0 Settg ths equal to zero, ad replacg wth b ' b ' Y Aga, the key result that we wll be usg repeatedly throughout multple regresso s that: b = ( ) - Y. Y wll always be a x vector of resposes, ad wll deped o what predctor varables are cotaed a model, but wll always have rows. Example: Bollywood Box Offce Data ' 'Y 55(765.43) b 55(765.43) (578.35) ( 47)(9097.5) (765.43) ( 47)(578.35) ( 55)(9097.5) Compare these estmates wth those from Chapter.

14 Ftted Values ad Resduals Matrx Form Y b0 b e Y Y Y b0 b Y b 0 b Note that whle H s hghly useful matrx computatos, t s x. Thus, we wll rarely prt t out (except very small datasets), ad t s very dffcult to work wth ECEL. The mportat take-away here s that whle the elemets of Y ad are assumed to be depedet, the elemets of I Matrx form: - - Y b = ' 'Y = HY H = ' ' b0 b Y H s called the "hat" or "projecto" matrx, ote that H s dempotet ( HH = H) ad symmetrc( H = H' ): Y Y Y Y Y Y Y Y e Y - Y = Y - b = Y - HY = (I - H)Y Y Y Y Y Y ad e are ot (H s ot a dagoal matrx) HH = ' ' ' ' 'I ' ' ' ' H H' = ' ' ' = ' ' = H Note: - E Y = E HY = HE Y = Hβ = ' 'β = β σ Y = Hσ IH' H e E = E I H Y = I H E Y = I H β β - β = 0 σ e = I H σ I I H ' I H MSE MSE s Y H s e = I H

15 Aalyss of Varace Total (Corrected) Sum of Squares: SSTO Y Y Y Y Note: Y ' Y Y Y'JY J SSTO Y ' Y Y'JY Y ' I J Y SSE e'e = Y - b sce b''y = Y'' ' Y ' Y - b = Y'Y - Y'b - b''y + b''b = Y'Y - b''y = Y' I - H Y - 'Y = Y'HY ' SSR SSTO SSE Y b''y Y'HY Y'JY Y H J Y Note that SSTO, SSR, ad SSE are all QUADRATIC FORMS: Y'AY for symmetrc, dempotet matrces A Example: Bollywood Box Offce Data Y Y'IY Y Y'JY b 'Y b''y Y'HY SSTO Y' I J Y SSE Y' I H Y SSR Y' H J Y

16 Ifereces Lear Regresso MSE Recall: b b = ' 'Y Þ E b = ' 'E Y = ' 'β = β σ b = ' 'σ Y ' = σ ' 'I ' = σ ' s b ' s ' - MSE MSE MSE MSE MSE MSE Estmated Mea Respose at : Y h b0 b h h h s Y h h h h 'b ' MSE Predcted New Respose at : - Y h b0 b h h'b s pred MSE h' ' h h s b ' ' h h h - Example: Bollywood Box Offce Data ' (765.43) SSE MSE (765.43) b s sb s b Compare these wth the results from Chapter.

17 R Program for Matrx Computatos Bollywood Data bbo <- read.csv(" header=t) attach(bbo) ames(bbo) Y <- Budget; <- Gross <- legth(y) 0 <- rep(,) <- as.matrx(cbd(0,)) Y <- as.matrx(y,col=) # Colum of s for matrx # Form the -matrx (=55 rows, Cols) # Form the Y-vector (=55 rows, col) # Notes: t() = traspose of, %*% = matrx multplcato, solve(a) = A(-) ( <- t() %*% ) # Obta ' matrx ( rows, cols) (Y <- t() %*% Y) # Obta 'Y vector ( rows, col) (I <- solve()) # Obta (')(-) matrx ( rows, cols) (b <- I %*% Y) # Obta b-vector ( rows, col) Y_hat <- %*% b # Obta the vector of ftted values (=55 rows, col) e <- Y - Y_hat # Obta the vector of resduals (=55 rows, col) prt(cbd(y_hat,e)) H <- %*% I %*% t() # Obta the Hat matrx J_ <- matrx(rep(/,),col=) # Obta the (/)J matrx (=55 rows, =55 cols) I_ <- dag() # Obta the detty matrx (=55 rows, =55 cols) (SSTO <- t(y) %*% (I_ - J_) %*% Y) # Obta Total Sum of Squares (SSTO) # SSTO ca also be computed as: (SSTO <- (t(y) %*% Y) - (t(y) %*% (I_ - J_) %*% Y)) (SSE <- t(y) %*% (I_ - H) %*% Y) # Obta Error Sum of Squares (SSE) # SSE ca also be computed as: (SSE <- (t(y) %*% Y) - (t(b) %*% Y)) (SSR <- t(y) %*% (H - J_) %*% Y) # Obta Regresso Sum of Squares (SSR) # SSR ca also be computed as: (SSR <- (t(b) %*% Y) - (t(y) %*% J_ %*% Y)) (MSE <- SSE/(-)) # Obta MSE = s (s_b <- MSE[,] * I) # Obta s{b}, must use MSE[,] ad * to do scalar multplcato (_h <- matrx(c(,0),col=)) # Create _h vector, for case where budget=0 (Y_hat_h <- t(_h) %*% b) # Obta the ftted value whe budget=0 (s_yhat_h <- t(_h) %*% s_b %*% _h) # Obta s{y_hat_h} (s_pred <- MSE + (t(_h) %*% s_b %*% _h)) # Obta s{pred}

18 R Output for Matrx Computatos Bollywood Data > ( <- t() %*% ) # Obta ' matrx ( rows, cols) > > (Y <- t() %*% Y) # Obta 'Y vector ( rows, col) [,] > > (I <- solve()) # Obta (')(-) matrx ( rows, cols) e e-06 > > (b <- I %*% Y) # Obta b-vector ( rows, col) [,] > > Y_hat <- %*% b # Obta the vector of ftted values (=55 rows, col) > > e <- Y - Y_hat # Obta the vector of resduals (=55 rows, col) > > prt(cbd(y_hat,e)) [,] [,] [,] [,] [54,] [55,] > Cotued Below

19 > > H <- %*% I %*% t() # Obta the Hat matrx > > J_ <- matrx(rep(/,),col=) # Obta the (/)J matrx (=55 rows, =55 cols) > > I_ <- dag() # Obta the detty matrx (=55 rows, =55 cols) > > (SSTO <- t(y) %*% (I_ - J_) %*% Y) # Obta Total Sum of Squares (SSTO) [,] [,] > # SSTO ca also be computed as: > # (SSTO <- (t(y) %*% Y) - (t(y) %*% (I_ - J_) %*% Y)) > > (SSE <- t(y) %*% (I_ - H) %*% Y) # Obta Error Sum of Squares (SSE) [,] [,] > # SSE ca also be computed as: > # (SSE <- (t(y) %*% Y) - (t(b) %*% Y)) > > (SSR <- t(y) %*% (H - J_) %*% Y) # Obta Regresso Sum of Squares (SSR) [,] [,] > # SSR ca also be computed as: > # (SSR <- (t(b) %*% Y) - (t(y) %*% J_ %*% Y)) > > (MSE <- SSE/(-)) # Obta MSE = s [,] [,] > > (s_b <- MSE[,] * I) # Obta s{b}, must use MSE[,] ad * to do scalar multplcato > > (_h <- matrx(c(,0),col=)) # Create _h vector, for case where budget=0 [,] [,] [,] 0 > > (Y_hat_h <- t(_h) %*% b) # Obta the ftted value whe budget=0 [,] [,] > > (s_yhat_h <- t(_h) %*% s_b %*% _h) # Obta s{y_hat_h} [,] [,].5905 > > (s_pred <- MSE + (t(_h) %*% s_b %*% _h)) # Obta s{pred} [,] [,]

20 E(Y) Models wth Multple Predctors Chapter 6 Multple Regresso I Most Practcal Problems have more tha oe potetal predctor varable Goal s to determe effects (f ay) of each predctor, cotrollg for others Ca clude polyomal terms to allow for olear relatos Ca clude product terms to allow for teractos whe effect of oe varable depeds o level of aother varable Ca clude dummy varables for categorcal predctors Frst-Order Model wth Predctors Y 0 E 0 E Y Plae 3-dmesos E(Y)= Iterpretato of Regresso Coeffcets Addtve: E{Y} = Mea of 0 Itercept, Mea of Y whe = =0 Slope wth Respect to (effect of creasg by ut, whle holdg costat) Slope wth Respect to (effect of creasg by ut, whle holdg costat)

21 These ca also be obtaed by takg the partal dervatves of E{Y} wth respect to ad, respectvely Iteracto Model: E{Y} = Whe = 0: Effect of creasg by : ()+ 3 ()(0)= Whe = : Effect of creasg by : ()+ 3 ()()= + 3 The effect of creasg depeds o level of, ad vce versa Geeral Lear Regresso Model wth p- Predctor Varables Y... 0 p, p p Y 0 k k k p Y where: k k 0 k 0 E 0 E Y... (Hyperplae p-dmesos) 0 p p p Smple lear regresso Normalty, depedece, ad costat varace for errors: 0, ~ NID 0, Y ~ N..., Y, Y 0 j p p j Example: Factors Effectg Ar Permeablty of Wove Fabrcs Y Average ar permeablty cm /s/cm 3 Warp Yar Desty (eds/cm) Weft Yar Desty (pcks/cm) Mass per Ut Area grams/cm 3 Graphc Source: Wkpeda Data Source: A. Ca, S. Vasslads, M. Ragouss, I. Tarakcoglu (007). Predcto of the Ar Permeablty of Wove Fabrcs Usg Neural Networks, Iteratoal Joural of Clothg Scece ad Techology, Vol. 9, #, pp. 8-35

22 ID warp weft mass mea_ap sd_ap Data represet the mea of each of 5 assessmets of ar permeablty. The stadard devato (sd_ap) s ot used ow, the respose, Y, s mea_ap. The stadard devato wll be used later for weghted least squares. SUMMARY OUTPUT Regresso Statstcs Multple R R Square Adjusted R Squar Stadard Error Observatos 30 All umbers o ths table are reproduced below. The Stadard Error the Regresso Statstcs porto s s = MSE ANOVA df SS MS F gfcace F Regresso E-0 Resdual Total Coeffcetsadard Erro t Stat P-value Lower 95%Upper 95% Itercept E warp weft mass

23 Scatterplot Matrx Note that the bottom row plots Y versus each of the varables. Ar permeablty teds to decrease as each predctor creases. We wll see later that the overall model s a good ft, whle dvdual coeffcets are ot sgfcat (due to the strog correlato betwee weft ad mass). The relatoshps betwee ar permeablty ad weft ad mass appear to be olear. Specal Types of Varables/Models p- dstct umerc predctors (attrbutes) Y = Sales, =Advertsg, =Prce Categorcal Predctors Idcator (Dummy) varables, represetg m- levels of a m level categorcal varable Y = Salary, =Experece, = f College Grad, 0 f Not Polyomal Terms Allow for beds the Regresso Y=MPG, =Speed, =Speed Trasformed Varables Trasformed Y varable to acheve learty Y =l(y) Y =/Y Iteracto Effects Effect of oe predctor depeds o levels of other predctors Y = Salary, =Experece, = f Coll Grad, 0 f Not, 3 = E{Y} =

24 No-College Grads ( =0): E{Y} = (0) + 3 (0)= 0 + College Grads ( =): E{Y} = () + 3 ()= ( 0 + )+( + 3 ) Respose Surface Models E{Y} = Note: Although the Respose Surface Model has polyomal terms, t s lear wth respect to the Regresso parameters Matrx Form of Multple Regresso Model Y...,..., 0 p, p Matrx Form: Y, p Y Y, p Y p, p 0 β ε E ε p p σ ε 0 0 I Y β ε E Y = E β + ε β σ Y I p p p p p p Note that smple lear regresso s the specal case where p- =.

25 Least Squares Estmato of Regresso Coeffcets Goal: Mmze: Q Y 0... p, p Obta Estmates of,,..., that mmze Q b, b,..., b Example: Factors Effectg Ar Permeablty of Wove Fabrcs ' 'Y INV(') b Note that the elemets of ad Y are: 0 p 0 p b0 b Normal Equatos: ' ' - b Y b ' 'Y pp p p p b p Maxmum Lkelhood also leads to the same estmator b : L Y β, exp 0... p, p / sce maxmzg L volves mmzg Y... 0 p, p 3 Y 3 Y ' 'Y 3 Y Y

26 Ftted Values ad Resduals Ftted Values: Y e Y Y Resduals: e Y e e p p - - Y b ' 'Y = HY H = ' ' H = H' = HH p p - ' e = Y Y Y b Y ' 'Y = I H Y I H I H I H I H MSE σ Y = σ HY = Hσ Y H' = σ H s Y H MSE σ e = σ I H Y = I H σ Y I H ' = σ I H s e I H Example: Factors Effectg Ar Permeablty of Wove Fabrcs Y-hat e e vs Y-hat e

27 Aalyss of Varace Sums of Squares Y Y Y Y Y Y Y Y Y Y b HY Y JY Y Y Y Y SSTO Y Y Y Y ' Y Y Y' I J Y SSE Y Y SSE Y Y ' Y Y Y' I H Y Y'Y - b''y MSE p SSR SSR Y Y Y Y ' Y Y Y' H J Y b''y Y' JY MSR p E MSE p p k kk k k ' kk ' kk ' k k k ' k E MSR SS SS SS k k k ' k' E MSR E MSE E MSR E MSE... 0 p Example: Factors Effectg Ar Permeablty of Wove Fabrcs Y Y'IY Y 39.3 Y'JY b 'Y b''y Y'HY SSTO Y' I J Y SSE Y' I H Y MSE SSR Y' H J Y MSR

28 ANOVA Table, F-test, ad R Aalyss of Varace (ANOVA) Table Source df Sum of Squares Mea Square Regresso p SSR SSR b''y - Y' JY MSR p Error p SSE Y'Y - b''y SSE MSE p Total SSTO Y'Y - Y' JY Test of H :... 0 E Y H : Not all 0 0 p 0 A k MSR Test Statstc: F* Rejecto Rego: F* F ; p, p P-value= PrF p, p F * MSE SSR SSE Coeffcet of Multple Determato: R Correlato: R R SSTO SSTO SSE p Adjusted-R SSE places a "pealty" o models wth extra predctors SSTO p SSTO Example: Factors Effectg Ar Permeablty of Wove Fabrcs H0 3 H A k : 0 : Not all TS : F* RR : F* F.95;3, P value: P F 3, R R Adjusted-R

29 Ifereces Regardg Regresso Parameters Y = β + ε E ε = 0 σ ε = I E Y = β σ Y = I E b = E ' 'Y = ' 'E Y = ' 'β = β p p p p b b, b b, b s b s b, b s b, b b, b b b, b s b, b s b s b, b σ b s b bp, b0 bp, b bp s bp, b s bp, b s bp σ b = σ ' 'Y ' 'σ Y ' ' ' ' ' ' ' MSE s b = ' bk k ~ t p 00% CI for b t ; p s b s b k - k k k bk Test of H0 : k 0 H A : k 0 Test Statstc: t* s b Rejecto Rego: t * t ; p P-value=Pr t p t * k Smultaeous 00% CI for : k ; g s s g p b t p sb k Example: Factors Effectg Ar Permeablty of Wove Fabrcs s b MSE ' 3.9 t.975, Test for : TS : t* ( ) % CI for : ( ) , Test for : TS : t* ( ) % CI for : ( ) , Test for 3: TS : t3* ( ) % CI for : ( ) , Note: Idvdually, o terms are sgfcat, but as a group they are. We wll uderstad why later.

30 Estmatg Mea Respose at Specfc -levels Gve set of levels of,..., :,..., ' ' ' ' h h h 00% CI for E h Y : Y h t ; p s h Y p h h, p β b h ' ' h EYh h Y h h p hp, E Y β Y σ b ' s Y MSE ' h h h h h h h Example: Factors Effectg Ar Permeablty of Wove Fabrcs Estmatg the mea at Warp= = 60,Weft= =35, Mass= 3 = % Cofdece Rego for Regresso Surface: Y hws Y h W pf ; p, p 00% CI for several ( g) E Y : Y Bs Y B t ; p g h h h ' h ' h ' Y h hb sy ' h h 3.9(.586) 4.88 ' 95% CI for β : (4.88) (4.5503,.55) h

31 Predctg New Respose(s) at Specfc -levels Gve set of levels of,..., :,..., pred MSE p h h, p β b h ' ' h EYh h Y h h p s hp, ' h - ' h mea of m observatos (at same -levels): s predmea MSE m 00% CI for Yh (ew) : Y h t ; p spred ' h - ' g Y Y h Ss S gf g p Scheffe: 00% CI for several ( ) : pred ;, h(ew) Boferro: 00% CI for several ( g) Yh (ew) : Y h Bspred B t ; p g Example: Factors Effectg Ar Permeablty of Wove Fabrcs Predctg a ew Ar Permeablty observato at Warp= = 60,Weft= =35, Mass= 3 = 5 h ' h ' h ' Y h hb s ' h pred 3.9( 0.586) % PI for Y : (7.0450) (.6, 7.889) h( ew) Presumably, ar permeablty measuremets caot be egatve, so we would terpret the predcto terval as beg (0, 7.889)

32 R Program for Chapter 6 Examples Ar Permeablty arperm <- read.csv("e:\\blue_drve\\sta40\\arperm_wove_reg.csv", header=true) attach(arperm) ames(arperm) Y <- mea_ap <- warp <- weft 3 <- mass <- legth(y) 0 <- rep(,) <- as.matrx(cbd(0,,,3)) # Form the -matrx (=30 rows, 4 Cols) Y <- as.matrx(y,col=) # Form the Y-vector (=30 rows, col) p <- col() # Notes: t() = traspose of, %*% = matrx multplcato, solve(a) = A(-) ( <- t() %*% ) (Y <- t() %*% Y) (I <- solve()) # Obta ' matrx (4 rows, 4 cols) # Obta 'Y vector (4 rows, col) # Obta (')(-) matrx (4 rows, 4 cols) (b <- I %*% Y) # Obta b-vector (4 rows, col) Y_hat <- %*% b e <- Y - Y_hat # Obta the vector of ftted values (=30 rows, col) # Obta the vector of resduals (=30 rows, col) prt(cbd(y_hat,e)) H <- %*% I %*% t() J_ <- matrx(rep(/,),col=) I_ <- dag() # Obta the Hat matrx # Obta the (/)J matrx (=30 rows, =30 cols) # Obta the detty matrx (=30 rows, =30 cols) (SSTO <- t(y) %*% (I_ - J_) %*% Y) # Obta Total Sum of Squares (SSTO) # SSTO ca also be computed as: # (SSTO <- (t(y) %*% Y) - (t(y) %*% (I_ - J_) %*% Y)) (SSE <- t(y) %*% (I_ - H) %*% Y) # Obta Error Sum of Squares (SSE) # SSE ca also be computed as: # (SSE <- (t(y) %*% Y) - (t(b) %*% Y)) (SSR <- t(y) %*% (H - J_) %*% Y) # Obta Regresso Sum of Squares (SSR) # SSR ca also be computed as: # (SSR <- (t(b) %*% Y) - (t(y) %*% J_ %*% Y)) (MSE <- SSE/(-p)) (s_b <- MSE[,] * I) multplcato se_b <- sqrt(dag(s_b)) # Obta MSE = s # Obta s{b}, must use MSE[,] ad * to do scalar # Obta SE's of dvdual regresso coeffcets Cotued Below

33 se_b <- sqrt(dag(s_b)) # Obta SE's of dvdual regresso coeffcets prt(cbd((b-qt(.975,-p)*se_b),(b+qt(.975,-p)*se_b))) # Prt CI's for Beta coeffcets (_h <- matrx(c(,60,35,5),col=)) =60,=35,3=5 # Create _h vector, for case where (Y_hat_h <- t(_h) %*% b) # Obta the ftted value whe budget=0 (s_yhat_h <- t(_h) %*% s_b %*% _h) # Obta s{y_hat_h} (s_pred <- MSE + (t(_h) %*% s_b %*% _h)) # Obta s{pred} ### Prt 95% CI for Mea at _h ad 95% PI for Idvdual Observato prt(cbd((y_hat_h-qt(.975,-p)*sqrt(s_yhat_h)),(y_hat_h+qt(.975,-p)*sqrt(s_yhat_h)))) prt(cbd((y_hat_h-qt(.975,-p)*sqrt(s_pred)),(y_hat_h+qt(.975,-p)*sqrt(s_pred)))) R Output > ( <- t() %*% ) # Obta ' matrx (4 rows, 4 cols) > > (Y <- t() %*% Y) # Obta 'Y vector (4 rows, col) [,] > > (I <- solve()) # Obta (')(-) matrx (4 rows, 4 cols) > > (b <- I %*% Y) # Obta b-vector (4 rows, col) [,] > (SSTO <- t(y) %*% (I_ - J_) %*% Y) # Obta Total Sum of Squares (SSTO) [,] [,] > > (SSE <- t(y) %*% (I_ - H) %*% Y) # Obta Error Sum of Squares (SSE) [,] [,] > > (SSR <- t(y) %*% (H - J_) %*% Y) # Obta Regresso Sum of Squares (SSR) [,] [,] Cotued Below

34 > (MSE <- SSE/(-p)) # Obta MSE = s [,] [,] 3.98 > > (s_b <- MSE[,] * I) # Obta s{b}, must use MSE[,] ad * to do scalar multplcato > > se_b <- sqrt(dag(s_b)) # Obta SE's of dvdual regresso coeffcets > > prt(cbd((b-qt(.975,-p)*se_b),(b+qt(.975,-p)*se_b))) # Prt CI's for Beta coeffcets [,] [,] > > (_h <- matrx(c(,60,35,5),col=)) # Create _h vector, for case where =60,=35,3=5 [,] [,] [,] 60 [3,] 35 [4,] 5 > > (Y_hat_h <- t(_h) %*% b) # Obta the ftted value whe budget=0 [,] [,] > > (s_yhat_h <- t(_h) %*% s_b %*% _h) # Obta s{y_hat_h} [,] [,] > > (s_pred <- MSE + (t(_h) %*% s_b %*% _h)) # Obta s{pred} [,] [,] > > ### Prt 95% CI for Mea at _h ad 95% PI for Idvdual Observato > prt(cbd((y_hat_h-qt(.975,-p)*sqrt(s_yhat_h)),(y_hat_h+qt(.975,p)*sqrt(s_yhat_h)))) [,] [,] [,] > prt(cbd((y_hat_h-qt(.975,-p)*sqrt(s_pred)),(y_hat_h+qt(.975,-p)*sqrt(s_pred)))) [,] [,] [,] > Ay dffereces betwee these output values ad those wth the chapter are due to the fact that I used 5 decmal places o calculatos.

35 Chapter 7 Multple Regresso II Extra Sums of Squares For a gve dataset, the total sum of squares remas the same, o matter what predctors are cluded (whe o mssg values exst amog varables) As we clude more predctors, the regresso sum of squares (SSR) creases (techcally does ot decrease), ad the error sum of squares (SSE) decreases SSR + SSE = SSTO, regardless of predctors model Whe a model cotas just, deote: SSR( ), SSE( ) Model Cotag, : SSR(, ), SSE(, ) Predctve cotrbuto of above that of : SSR( ) = SSE( ) SSE(, ) = SSR(, ) SSR( ) Exteds to ay umber of Predctors Deftos ad Decomposto of SSR,,,,,, SSTO SSR SSE SSR SSE SSR SSE 3 3,,,, SSR SSR SSR SSE SSE SSR SSR SSR SSE SSE 3,,, 3,,,, 3, SR,, SSR SSE SSE,, SSR SSR SSR SSE SSE SSR S 3 3 3, SSR SSR SSR SSR SSR,, 3 3,,, 3 3,,,, SSR SSR SSR SSR SSR SSR SSR SSR SSR SSR SSR 3 3

36 Example: Factors Effectg Ar Permeablty of Wove Fabrcs The followg partal ANOVA tables are for all 7 possble models cotag at least oe of the 3 predctors. Predctors SSR SSE dfr dfe , , , ,, ,,,, 3,, 3, , SSTO SSR SSE SSR SSE SSR SSE SSR SSE SSR, SSE, SSR SSR SSR SSR SSR SSR SSR, SSR,, SSR, SSR, SSR,, SSR , ,,, SSR SSR SSR SSR SSR SSR SSR SSR SSR SSR,, SSR SSR SSR, SSR,, SSR SSR SSR, SSR SSR SSR Note that as the # of predctors creases, so does the ways of decomposg SSR

37 ANOVA Sequetal Sum of Squares Ths s a parttog of the Regresso sum of squares the full model, to ts sequetal sums of squares for the varables the order of ther appearace the regresso program. For the case of 3 predctors, etered the order:,, 3 : Source of Varato SS df MS Regresso SSR(,,3) 3 MSR(,,3) SSR() MSR() SSR( ) MSR( ) 3, SSR(3,) MSR(3,) Error SSE(,,3) -4 MSE(,,3) Total SSTO - SSR SSR MSR MSR SSR 3, MSR 3, SSR,, 3 MSR,, 3 3 SSR, 3 MSR, 3 Example: Factors Effectg Ar Permeablty of Wove Fabrcs Source of Varato SS df MS Regresso , Error Total Note: =

38 Extra Sums of Squares & Tests of Regresso Coeffcets (Sgle k ) Full Model: Y ~ NID 0, H : 0 H : 0 Reduced Model: Y 0 3 A 3 0 Geeral Lear Test: F* SSE( R) SSE( F) dfr df f SSE( F) df f SSE F SSE 3 dff 4 SSE R SSE dfr Full Model: ( ),, Reduced Model: ( ), 3 SSE( R) SSE( F) SSE, SSE,, SSR, df df R F SSR 3, MSR 0 3, H F* ~ F, 4 SSE,, 3 MSE,, 3 4 Rejecto Rego: F* F ;, 4 P value: P F, 4 F * Example: Factors Effectg Ar Permeablty of Wove Fabrcs H0 : 3 0 H A : MSR 3, 9.5 MSE,, F* F.95;, P value PF, Ths F-test gves the exact same result as the t-test from Chapter 6. t * F * t.975, 4 F.95;, 4

39 Extra Sums of Squares & Tests of Regresso Coeffcets (Multple k ) Full Model: Y ~ NID 0, H : 0 H : ad/or 0 Reduced Model: Y 0 3 A 3 0 Geeral Lear Test: F* SSE( R) SSE( F) dfr df f SSE( F) df f Full Model: SSE( F) SSE,, 3 dff 4 SSE R SSE dfr Reduced Model: ( ) SSE( R) SSE( F) SSE SSE,, SSR, df df R F Example: Factors Effectg Ar Permeablty of Wove Fabrcs SSR, 3 MSR, F F SSE,, 3 MSE,, 3 4 H0 3 * ~, 4 Rejecto Rego: F* F ;, 4 P value: P F, 4 F * H0 : 3 0 H A : ad/or MSR, MSE,, F* 6.65 F.95;, P value PF, Note that the dvdual t-tests for Wear ( ) ad Mass ( 3 ) were ot sgfcat, but whe we test them smultaeously, they are hghly sgfcat. Ths s due to the fact that they are hghly correlated, ad both related to Ar Permeablty (Y). We wll look at ths more detal the secto o Multcollearty.

40 Extra Sums of Squares & Tests of Regresso Coeffcets (Geeral Case) Full Model: Y... ~ NID 0, H 0 p, p :... 0 H : At least oe of q p A q p Reduced Model: Y... q p 0 q, q SSE( R) SSE( F) dfr df F Geeral Lear Test: F* SSE( F) df F p F q R q p q p q Full Model: SSE( F) SSE,..., df p Reduced Model: SSE( R) SSE,..., df q SSE( R) SSE( F) SSE,..., SSE,..., SSR,...,,..., df df q p p q R F SSR q,..., p,..., q p q MSR,..., 0,..., H q p q F* ~ F p q, p SSE,..., MSE,..., p p p Rejecto Rego: F* F ; p q, p P-value P F p q; p F * Sce there are oly three predctors the Ar Permeablty model, we wll ot do ths as a example. Other Lear Tests Suppose frm has two types of advertsg: prt, $000s ad teret, $000s as well as promotoal expedtures, $000s : 3 Let Sales = Y, they vary ther expedtures o perods ad observe sales each (Prce s costat) Y ~ NID 0, Test of equal effects of creasg each put by ut (say $000s): H : H : H s False 0 3 A 0 Full Model: Y 0 df F Reduced Model: Y Y Y W W df 0 3 R

41 Suppose frm has two types of advertsg: prt, $000s ad teret, $000s as well as promotoal expedtures, $000s : 3 Let Sales = Y, they vary ther expedtures o perods ad observe sales each (Prce s costat) Y ~ NID 0, Test that Mea sales whe all puts=0 s $0,000 ad effect of creasg by ut s : H : 0, H : H s False A F R (all uts are $000s) Full Model: Y df 4 Reduced Model: Y 0 Y 0 U U Y 0 df ( o tercept) 3 Coeffcets of Partal Determato-I Proporto of Varato Explaed by or more varables, ot explaed by others Regresso of Y o : Y 0 Uexplaed: Varato Explaed: SSR SSE SSTO SSR Regresso of Y o : Y 0 Uexplaed: Varato Explaed: SSR SSE SSTO SSR Regresso of Y o, : Y 0 Varato Explaed: SSR, Uexplaed: SSE, SSTO SSR, Proporto of Varato Y, Not Explaed by, that s Explaed by : R Y SSE SSE, SSR, SSR SSR, SSR SSR SSE SSE SSTO SSR SSE Proporto of Varato Y, Not Explaed by, that s Explaed by : R Y SSE SSE, SSR, SSR SSR, SSR SSR SSE SSE SSTO SSR SSE

42 Coeffcets of Partal Determato-II R R R Y 3 Y 3 Y 3 SSR,, 3 SSE, SSE,, SSR,, SSR, SSE, SSE, 3 3 SSR,, SSR, SSR, SSTO SSR, SSE, R SSR,, SSR, SSR, SSTO SSR, SSE, SSE 3 3 SSR, SSR, SSTO SSR, SSE, 3 SSE SSE SSE,, SSR,, SSR 3 3 Y 3 SSR,, SSR SSR, SSTO SSR SSE 3 3 Coeffcet of Partal Correlato: R sg R sg Y Y f 0 Y 0 f 0 Y 0 Example: Factors Effectg Ar Permeablty of Wove Fabrcs,, SSR SSE SSR, SSE, 843. SSR SSR SSE RY SSR 3, SSE, 843. RY SSR, SSE R Y

43 Stadardzed Regresso Model Useful removg roud-off errors computg ( ) - Makes easer comparso of magtude of effects of predctors measured o dfferet measuremet scales Coeffcets represet chages Y ( stadard devato uts) as each predctor creases SD (holdg all others costat) Sce all varables are cetered, o tercept term Stadardzed Radom Varables: Scaled to have mea=0, SD= Y Y k k Y Y k k sy sk k,..., p s s Y Correlato Trasformato: * Y Y * k k Y k k,..., p sy sk Stadardzed Regresso Model: Y... * * * * * * p, p s Y * Note: k k k,..., p 0 Y... p p sk k Stadardzed Regresso Model: Y... * * * * * * p, p * * * Y r r, p ry, p * r r Y, p r * * *' * ( p) Y *' * Y r Y r ( p) ( p) ( p) * *, p * Y r p, rp, ry, p Ths results from: k k * k k k s k sk s k s k * * k k ' k k k ' k' k k ', k ' k s k k' s r s k ' sksk ' s ks k ' k * * Y Y k k sy, Y Y k k k Y r s s s s sy s k Yk Y k Y k k Y kk '

44 Stadardzed Regresso Model: Y... * * * * * * p, p * * * Y r r, p ry, p * Y r r, p r * * *' * Y *' * Y r Y r ( p) rp, rp, ry, p ( p) ( p) ( p) * *, p * Y Y *' * * *' * * *' * *' * * b Y b Y b rry ( p) ( p) ( p) ( p) b s b k,..., p b Y b... b Y * k k 0 p p sk Example: Factors Effectg Ar Permeablty of Wove Fabrcs warp weft mass arperm warp* weft* mass* arperm*

45 *'* *'Y* s_y b s_ b s_ b s_ b ybar 3.04 INV(*'*) b* xbar xbar xbar Note that the correlato betwee Weft ad Mass s That leads us to Multcollearty. Multcollearty Cosder model wth Predctors (ths geeralzes to ay umber of predctors) Y = Whe ad are ucorrelated, the regresso coeffcets b ad b are the same whether we ft smple regressos or a multple regresso, ad: SSR( ) = SSR( ) SSR( ) = SSR( ) Whe ad are hghly correlated, ther regresso coeffcets become ustable, ad ther stadard errors become larger (smaller t-statstcs, wder CI s ), leadg to strage fereces whe comparg smple ad partal effects of each predctor Estmated meas ad Predcted values are ot affected Example: Factors Effectg Ar Permeablty of Wove Fabrcs Warp, Weft Warp, Mass Coeffcetsadard Erro t Stat P-value Coeffcetsadard Erro t Stat P-value Itercept Itercept warp warp weft mass Weft, Mass Warp, Weft, Mass Coeffcetsadard Erro t Stat P-value Coeffcetsadard Erro t Stat P-value Itercept Itercept weft warp mass weft mass Compare the stadard errors of the Weft coeffcet: chagg from 0.33 to depedg o whether Mass s the model (a 6-fold crease whe Mass s cluded). Smlarly, stadard error for the Mass coeffcet chages from wthout Weft to wth Weft.

46 R Program for Chapter 7 Examples Ar Permeablty arperm <- read.csv("e:\\blue_drve\\sta40\\arperm_wove_reg.csv", header=true) attach(arperm) ames(arperm) plot(arperm[,:5]) ### Scatterplot Matrx of,,3,y ssto <- sum((mea_ap-mea(mea_ap))) ### Total Sum of Squares ########## Ft all 7 possble Models ap.mod3 <- lm(mea_ap ~ warp + weft + mass) summary(ap.mod3) aova(ap.mod3) ap.mod <- lm(mea_ap ~ warp + weft) #summary(ap.mod) #aova(ap.mod) ap.mod3 <- lm(mea_ap ~ warp + mass) #summary(ap.mod3) #aova(ap.mod3) ap.mod3 <- lm(mea_ap ~ weft + mass) #summary(ap.mod3) #aova(ap.mod3) ap.mod <- lm(mea_ap ~ warp) #summary(ap.mod) #aova(ap.mod) ap.mod <- lm(mea_ap ~ weft) #summary(ap.mod) #aova(ap.mod) ap.mod3 <- lm(mea_ap ~ mass) #summary(ap.mod3) #aova(ap.mod3) ####### Obta SSE ad SSR for each model (devace=sse) sse.x <- devace(ap.mod); ssr.x <- ssto-sse.x sse.x <- devace(ap.mod); ssr.x <- ssto-sse.x sse.x3 <- devace(ap.mod3); ssr.x3 <- ssto-sse.x3 sse.xx <- devace(ap.mod); ssr.xx <- ssto-sse.xx sse.xx3 <- devace(ap.mod3); ssr.xx3 <- ssto-sse.xx3 sse.xx3 <- devace(ap.mod3); ssr.xx3 <- ssto-sse.xx3 sse.xxx3 <- devace(ap.mod3); ssr.xxx3 <- ssto-sse.xxx3 #### Compute Sequetal Sums of Squares ssr.xx-ssr.x ### SSR( ) ssr.xx-ssr.x ### SSR( ) ssr.xxx3-ssr.xx ### SSR(3,) #### Test H0: B=B3=0 aova(ap.mod,ap.mod3) ### Compue Coeffcets of Partal Determato (ssr.xx-ssr.x)/sse.x ### R(Y ) (ssr.xxx3-ssr.xx)/sse.xx ### R(Y3 ) Cotued Below

47 #### Correlato Trasformato ad Stadardzed Regresso Coeffcets y.corr <- (mea_ap-mea(mea_ap))/(sqrt(9)*sd(mea_ap)) x.corr <- (warp-mea(warp))/(sqrt(9)*sd(warp)) x.corr <- (weft-mea(weft))/(sqrt(9)*sd(weft)) x3.corr <- (mass-mea(mass))/(sqrt(9)*sd(mass)) xstar <- matrx(cbd(x.corr,x.corr,x3.corr),col=3) ystar <- matrx(y.corr) bstar <- solve(t(xstar) %*% xstar) %*% t(xstar) %*% ystar bstar ##### Regresso coeffcet estmates for all ad 3 varable models summary(ap.mod) summary(ap.mod3) summary(ap.mod3) summary(ap.mod3) R Output > ap.mod3 <- lm(mea_ap ~ warp + weft + mass) > summary(ap.mod3) Call: lm(formula = mea_ap ~ warp + weft + mass) Resduals: M Q Meda 3Q Max Coeffcets: Estmate Std. Error t value Pr(> t ) (Itercept) e-07 *** warp weft mass Resdual stadard error: o 6 degrees of freedom Multple R-squared: , Adjusted R-squared: 0.8 F-statstc: o 3 ad 6 DF, p-value:.678e-0 > aova(ap.mod3) Aalyss of Varace Table Respose: mea_ap Df Sum Sq Mea Sq F value Pr(>F) warp ** weft e- *** mass Resduals Cotued Below

48 > ssr.xx-ssr.x ### SSR( ) [] > ssr.xx-ssr.x ### SSR( ) [] > ssr.xxx3-ssr.xx ### SSR(3,) [] > > aova(ap.mod,ap.mod3) Aalyss of Varace Table Model : mea_ap ~ warp Model : mea_ap ~ warp + weft + mass Res.Df RSS Df Sum of Sq F Pr(>F) e-0 *** > (ssr.xx-ssr.x)/sse.x ### R(Y ) [] > > (ssr.xxx3-ssr.xx)/sse.xx ### R(Y3 ) [] > bstar <- solve(t(xstar) %*% xstar) %*% t(xstar) %*% ystar > bstar [,] [,] [,] [3,] > summary(ap.mod) lm(formula = mea_ap ~ warp + weft) Coeffcets: Estmate Std. Error t value Pr(> t ) (Itercept) e-08 *** warp *** weft e- *** > summary(ap.mod3) lm(formula = mea_ap ~ warp + mass) Coeffcets: Estmate Std. Error t value Pr(> t ) (Itercept) e-07 *** warp mass e- *** > summary(ap.mod3) lm(formula = mea_ap ~ weft + mass) Coeffcets: Estmate Std. Error t value Pr(> t ) (Itercept) e-09 *** weft mass *** > summary(ap.mod3) lm(formula = mea_ap ~ warp + weft + mass) Coeffcets: Estmate Std. Error t value Pr(> t ) (Itercept) e-07 *** warp weft mass

49 Chapter 8 Models for Quattatve ad Qualtatve Predctors Polyomal Regresso Models Useful Settgs: True relato betwee respose ad predctor s polyomal True relato s complex olear fucto that ca be approxmated by polyomal specfc rage of -levels Models wth Predctor: Icludg p polyomal terms model, creates p- beds d order Model: E{Y} = 0 + x + x (x = cetered ) 3 rd order Model: E{Y} = 0 + x + x + 3 x 3 Respose Surfaces wth (or more) predctors d order model wth Predctors: E Y x x x x x x x x 0

50

51 % RED Modelg Strateges Use Extra Sums of Squares ad Geeral Lear Tests to compare models of creasg complexty (hgher order) Use codg fttg models (cetered/scaled) predctors to reduce multcollearty whe coductg testg. Keep lower order terms wheever hgher order polyomals ad teractos are cluded a model, eve f ot sgfcat. Ft models orgal uts, or back-trasform for plottg o orgal scale * (see below for quadratc) For Respose Surfaces, clude multple replcates at ceter pot for goodess-of-ft tests Cetered varables: 0 0 Y b b x b x b b b ' ' ' 0 0 o b b b b b b b b b b b b b b b Example: Relatoshp Betwee We Color (% Red) ad Athocya Glucosdes (GLU) A study related varous types of color parameters ad pheolc compouds three varetes of we (Temprallo, Gracao, ad Caberet Sauvgo) over a 6 moth storage perod. The followg plot s of the percet red color (Y) versus Athocya Glucosdes (). Source: M. Moagas, P. J. Martı-Alvarez,, B. Bartolome C. Gomez-Cordoves (006). Statstcal terpretato of the color parameters of red wes fucto of ther pheolc composto durg agg bottle, Europea Food Research Techology, Vol., pp %Red versus GLU GLU GLU GLUC GLUC %Red GLU GLUC ad GLUC are the cetered, ad squared cetered values of GLU Fttg the Regresso Model: Y 0 x x ~ NID 0, x

52 % RED Coeffcetsadard Erro t Stat P-value Lower 95%Upper 95% Itercept E GLUC GLUC E E-05 Note that all coeffcets are sgfcat. Also, the tercept represets the ftted value at x=0, or equvaletly, the ftted value at the mea GLU level. The ftted equato, ad back-trasformed coeffcets are: Y x x b ' 0 ' ' (8.77) ( )(8.77) b ( )(8.77) b Y Below s the model ft wth the u-cetered (orgal) GLU values (dffereces are due to roudg): Coeffcetsadard Erro t Stat P-value Lower 95%Upper 95% Itercept E GLU GLU E E %Red versus GLU GLU

53 Regresso Models wth Iteracto Term(s) Iteracto Effect (Slope) of oe predctor varable depeds o the level other predctor varable(s) Formulated by cludg cross-product term(s) amog predctor varables Varable Models: E{Y} = E Y 0 (0) E Y 0 (0) Testg Hypothess of o teracto: H : 0 H : A 3

54 Example Respose Surface Relatg 3 Factors to Color Itesty Idgo Dye Appled to Cotto Source: M.B. Tcha, N. Meks, N. Drr, M. Kechda, ad M.F. Mhe (03). A promsg route to dye cotto by dgo wth a ecologcal exhausto process: A dyeg process optmzato based o a respose surface methodology, Idustral Crops ad Products, Vol. 46, pp A expermet was coducted relatg = Temperature (Celsus, Levels=35,60,00), = Tme (Mutes, Levels=30,60,90,0), ad 3 = Catozg Aget (Percetage, Levels=0,4,0,5,0) to Y = Color Yeld (K/S rato). The expermet was made up of = 60 rus. To obta the same results as the authors, we wll use the orgal data levels, ot cetered or scaled levels. The secod order respose surface s of the form: E Y ECEL Output: Regresso Statstcs Multple R R Square Adjusted R Square Stadard Error 0.66 Observatos 60 ANOVA df SS MS F gfcace F Regresso Resdual Total Coeffcetsadard Erro t Stat P-value Lower 95%Upper 95% Itercept Temp Tme Cato Temp Tme E E-05 Cato TempTme E E-05 TempCat E Note that all regresso coeffcets are sgfcat, mplyg a very complex surface four dmesos. To observe the surface, we wll use the rsm package R.

55 R Program dye <- read.csv("e:\\blue_drve\\sta40\\dye_cotto_rsm.csv", header=true) attach(dye); ames(dye) stall.packages("rsm") lbrary(rsm) dye.rsm <- rsm(ks ~ SO(Temp,Tme,Cato)) summary(dye.rsm) drop(dye.rsm) cotour(dye.rsm, ~ Temp + Tme, mage=true) cotour(dye.rsm, ~ Temp + Cato, mage=true) cotour(dye.rsm, ~ Tme + Cato, mage=true) R Text Output > dye.rsm <- rsm(ks ~ SO(Temp,Tme,Cato)) > summary(dye.rsm) Call: rsm(formula = KS ~ SO(Temp, Tme, Cato)) Estmate Std. Error t value Pr(> t ) (Itercept) e e <.e-6 *** Temp e-0.668e <.e-6 *** Tme e e *** Cato 4.099e e e-09 *** Temp:Tme -.806e e * Temp:Cato 8.976e e * Tme:Cato e e Temp e e <.e-6 *** Tme -.36e e * Cato e e e- ** Multple R-squared: , Adjusted R-squared: F-statstc: o 9 ad 50 DF, p-value: <.e-6 Aalyss of Varace Table Respose: KS Df Sum Sq Mea Sq F value Pr(>F) FO(Temp, Tme, Cato) < e-6 TWI(Temp, Tme, Cato) PQ(Temp, Tme, Cato) < e-6 Resduals Lack of ft Pure error Statoary pot of respose surface: Temp Tme Cato > drop(dye.rsm) Df Sum of Sq RSS AIC <oe> FO(Temp, Tme, Cato) TWI(Temp, Tme, Cato) PQ(Temp, Tme, Cato) Note that the frst ANOVA table gves the sequetal sums of squares, the secod oe (drop) gves the partal sums of squares, each group gve all the others. I ths case, sequetal makes more sese due to the herarchy wth ma effects (FO = Frst order terms tested frst). Hgher order terms are (TWI = Two-Way Iteractos ad PQ = Polyomal Quadratc). Statoary pot gves the best levels for each varable to maxmze Y.

56 Cotour Plots for all Pars of Varables (at Best Level of Thrd Varable):

57 Data (Splt to 3 Groups of Colums for Readablty) Temp Tme Cato KS Temp Tme Cato KS Temp Tme Cato KS Qualtatve Predctors Ofte, we wsh to clude categorcal varables as predctors (e.g. geder, rego of coutry, ) Trck: Create dummy (dcator) varable(s) to represet effects of levels of the categorcal varables o respose Problem: If varable has c categores, ad we create c dummy varables, the model s ot full rak whe we clude tercept Soluto: Create c dummy varables, leavg oe level as the cotrol/basele/referece category Iteractos ca be geerated betwee qualtatve ad quattatve predctors May models wll cota multple qualtatve predctors.