Contents. 2 2 factorial design 4

Transcription

1 Contents TAMS38 - Lecture 10 Response surface methodology Lecturer: Zhenxia Liu Department of Mathematics - Mathematical Statistics 12 December, factorial design Polynomial Regression model First degree polynomial regression / first order response surface design Second degree polynomial regression / second order response surface design Response surface methodology Example 1 Central composite design - CCD Example 2 The coming tasks 2 2 factorial design 3 A factorial design can be analyzed using ANOVA or regression analysis. How? We start with studying a 2 2 design with two observations per combination. A We have the model B y 1, 1,1, y 1, 1,2 y 1,1,1, y 1,1,2 1 y 1, 1,1, y 1, 1,2 y 1,1,1, y 1,1,2 Y ijk = µ + τ i + β j + (τβ) ij + ε ijk where ε ijk N(0, σ) and independent for i = 1, 1, j = 1, 1 and k = 1, factorial design 4 We write the model Y ijk = µ + τ i + β j + (τβ) ij + ε ijk as Y 1, 1,1 Y 1,1,1 Y 1, 1,1 Y 1,1,1 Y 1, 1,2 Y 1,1,2 Y 1, 1,2 Y 1,1,2 A B AB = x 1 x 2 x 1 x 2 We rewrite the model as µ τ 1 β 1 (τβ) 11 Y = µi + τ 1 A + β 1 B + (τβ) 11 AB + ε Y = µ + τ 1 x 1 + β 1 x 2 + (τβ) 11 x 1 x 2 + ε + ε 1, 1,1 ε 1,1,1 ε 1, 1,1 ε 1,1,1 ε 1, 1,2 ε 1,1,2 ε 1, 1,2 ε 1,1,2 (1)

2 Regressin model 5 Base on the model Y = µ + τ 1 x 1 + β 1 x 2 + (τβ) 11 x 1 x 2 + ε. We get the following regression model Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + ε. (2) Where x 1 = 1, 1, x 2 = 1, 1 and ε N(0, σ). Now we use the model (2) above. Let every x i vary between 1 to 1 continuously, i=1,2, then we will have a response surface. (-1,1) (1,1) (-1,-1) (1,-1) Second degree Polynomial regression 7 First degree Polynomial regression 6 First degree Polynomial regression/first order response surface design Model: Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + ε Where x i [ 1, 1], i = 1, 2 and ε N(0, σ). Note: Two factors are in 2-dimensional space, and the response is in the 3rd dimension. The interaction term gives some curvature to the response surface, but sometimes it is not enough to explain the surface. Polynomial regression 8 Second degree Polynomial regression/second order response surface design Model: Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + β 11 x β 22 x ε Where x i [ 1, 1], i = 1, 2 and ε N(0, σ). Note: Two factors are in 2-dimensional space, and the response is in the 3rd dimension. The quadratic terms give the curvature to the response surface. If the curvature exists, then the response gets optimum, i.e. maximum or minimum. How to test curvature? First degree Polynomial regression for 2 k design Model: k Y = β 0 + β j x j + β ij x i x j + ε j=1 {i,j i<j} Where x j [ 1, 1], j = 1,..., k and ε N(0, σ). Second degree Polynomial regression for 2 k design Model: k y = β 0 + β j x j + k β ij x i x j + β jj x 2 j + ε j=1 (i,j):i<j j=1 Where x j [ 1, 1], j = 1,..., k and ε N(0, σ).

3 Response surface Methodology 9 Response surface methodology - RSM Response surface methodology (RSM) explores the relations between explanatory variables and response variable. The goal of RSM is to find an optimum response. How? We test the curvature, i.e. quadratic term.we add more observations in the center point. Then we compare the mean value for the observations in the corner points ȳ F and the mean value for the observations in the center point ȳ C. Why? Remark: One reason that we take extra observations in the center point is that these observations will not change the estimators of the main and interaction effects for the original 2 k design. Response surface Methodology 11 For the 2 2 factorial design we have the model and we get the following model where β 0 = µ. y ij = µ + τ i + β j + (τβ) ij + ε ij, Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + ε, In the corner points, we have ( ) ˆβ 0 = ˆµ = ȳ.. = ȳ F ȲF N µ, σ2 n F. In the center point, we have y C1,..., y CnC observations. Y Ck N(µ +, σ }{{} 2 )., then =µ C ˆM C = ȲC N (µ C, σ2 n C ). Response surface Methodology 10 We assume that there are n F observations in the corners, then we get the mean ȳ F, thus the corresponding point estimator ȲF. We also assume that there are n C observations in the center point, then we get the mean ȳ C, thus the corresponding point estimator ȲC. To test curvature we will compare the means ȳ F and ȳ C. Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + β 11 x β 22x ε The expectations for these means are E(ȲF ) = β 0 + β 11 + β 22 and E(ȲC) = β 0 while both of them are β 0 in the first degree polynomial regression model. If the mean values E(ȲF ) and E(ȲC) are not equal, which indicates that the quadratic terms are needed, i.e., the curvature exists. Response surface Methodology 12 We want to compare ˆµ = ȳ F and ˆµ C = ȳ C. Ȳ F ȲC N If µ = µ C, i.e., no curvature, then and ( µ µ C, σ 2 ( 1 n F + 1 n C Ȳ F ȲC 0 N(0, 1) σ 1 n F + 1 n C ) 2 (ȲF ( ȲC ) χ 2 (1). σ 2 1 n F + 1 n C )).

4 Response surface Methodology 13 Define SS P Q = (ȳ F ȳ C ) 2 1 n F + 1 = n F n C (ȳ F ȳ C )2, n n C + n F C (P Q= pure quadratic). Hypothesis H 0 : no curvature, i.e., = 0 a) If we have one observation per combination, then the test statistic v P Q = SS P Q SS P Q /1 s 2 = (n C 1)s 2 /(n C 1), where s 2 = 1 nc n C 1 k=1 (y Ck ȳ C ) 2, i.e., the sample variance for the center observations. If v P Q > F α (1, n C 1), then reject H 0, i.e. there exists curvature.. Example 1 15 Here I have let Minitab generate a 2 2 -design with five observations in the central point. Then I have found the response values (y) and did analysis. Response surface Methodology 14 b) For the general case, we have the test statistic v P Q = SS P Q SS E /dfe If v P Q > F α (1, dfe), then reject H 0, i.e. there exists curvature. Note: Case (b) is the generalized case of case (a). Example 1 - continued 16 Test on curvature with significance level α = v P Q = 1.04 < 7.71 = F 0.05 (1, 4), don t reject H 0. Hence, the curvature is not significant.

5 Example 1 - continued 17 MTB > Name C1 "StdOrder" C2 "RunOrder" C3 "CenterPt" C4 "Blocks" C5 "A" C6 "B" MTB > FFDesign 2 4; SUBC> CPBlocks 5; SUBC> CTPT CenterPt ; SUBC> Randomize; SUBC> SOrder StdOrder RunOrder ; SUBC> Alias 2; SUBC> XMatrix A B. Full Factorial Design Factors: 2 Base Design: 2, 4 Runs: 9 Replicates: 1 Blocks: 1 Center pts (total): 5 All terms are free from aliasing. Example 1 - continued 19 Example 1 - continued 18 MTB > FFactorial C7 = C5 C6 C5*C6; SUBC> Design C5 C6 C4; SUBC> Order C1; SUBC> InUnit 1; SUBC> Levels ; SUBC> CTPT C3; SUBC> FitC; SUBC> Brief 2; SUBC> Alias. Factorial Fit: C7 versus A, B Estimated Effects and Coefficients for C7 (coded units) Term Effect Coef SE Coef T P Constant A B A*B Ct Pt S = PRESS = MAI * - li.u TAMS38 - Lecture 10 R-Sq = 82.42% R-Sq(pred) = *% R-Sq(adj) = 64.85% Central composite design 20 Analysis of Variance for C7 (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects Way Interactions Curvature Residual Error Total When we want to estimate the model including curvature we need more observations. The question is how to choose new observations. For a 2 2 design, we have the model Y = β 0 + β 1 x 1 + β 2 x 2 + β 12 x 1 x 2 + β 11 x β 22 x ε, We have six parameters and we need at least six observations to do the estimation of them. There are several ways to extend design so that one can estimate a second-order model (see book, Section 11.4). Central composite design (CCD) is used to estimate the parameters of second order surface design - second degree polynomial design.

6 Central composite design 21 Central composite design 22 x 3 Central composite design - CCD A Central composite design is one type of response surface design. It has 2k additional points, if there are k factors. x 2 (0, α) x 2 Usually we take axial points with coded values (±α, 0,..., 0), (0, ±α,..., 0),... (0, 0,..., ±α). How to choose α? The CCD is rotatable by the choice of α = n 1/4 F, where n F = is the number of points used in the corners. ( α, 0) (0, α) (α, 0) x 1 x 1 A CCD is spherical by the choice of α = k. Central composite designs (CCD) for k = 2 and k = 3. Example 2 23 Example 2 - continued 24 In a study one wants to find the best combination of time (t) and temperature (T ) which produce the maximum amount of a specific substancee in a chemical process. We think that the best combination is around the time t = 75 minutes and temperature T = 130 C. Let the time vary from 70 to 80 minutes and the temperature from to C as in the Table 1. We have coded our variables as x 1 = t 75 minutes, x 2 = T 130 C 5 minutes 2.5 C. Table 1: Result from the first design with three center points Variables in Variables in Respons: original units coded units amount run time temp x 1 x 2 y

7 Example 2 - continued 25 Example 2 - continued 26 We will use a 2 2 factorial design with three center points; see Figure 1. The design is called a first-order design and is good for testing first order polynomial, Y = β 0 + β 1 x 1 + β 2 x 2 + ε. We use Minitab command: Stat/DOE/Factorial/ Create Factorial Design... Figure 1. MTB > Name C1 "StdOrder" C2 "RunOrder" C3 "CenterPt" C4 "Blocks" C5 "A" C6 "B" MTB > FFDesign 2 4; SUBC> CPBlocks 3; SUBC> CTPT CenterPt ; SUBC> Randomize; SUBC> SOrder StdOrder RunOrder ; SUBC> Alias 2; SUBC> XMatrix A B. Example 2 - continued 27 Example 2 - continued 28 We put the measured y-values in proper rows of column c7 (Y). C1 C2 C3 C4 C5 C6 C7 StdOrder RunOrder CenterPt Blocks A B Y , , , , , , ,3 (a) Test on interaction between A and B with significance level (b) Test on curvature with significance level Analyzing according to a complete model of factorial design with special expectation of the center points. Stat/DOE/Factorial/Analyze Factorial Design... MTB > FFactorial Y = C5 C6 C5*C6; SUBC> Design C5 C6 C4; SUBC> Order C1; SUBC> InUnit 1; SUBC> Levels ; SUBC> CTPT C3; SUBC> FitC; SUBC> Brief 2; SUBC> Alias.

8 Example 2 - continued 29 Example 2 - continued 30 Factorial Fit: Y versus A; B Estimated Effects and Coefficients for Y (coded units) Term Effect Coef SE Coef T P Constant 61,8000 1,000 61,80 0,000 A 4,7000 2,3500 1,000 2,35 0,143 B 9,0000 4,5000 1,000 4,50 0,046 A*B -1,3000-0,6500 1,000-0,65 0,582 Ct Pt 0,5000 1,528 0,33 0,775 S = 2 PRESS = * R-Sq = 92,93% R-Sq(pred) = *% R-Sq(adj) = 78,80% Analysis of Variance for Y (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 103, ,090 51, ,89 0,072 2-Way Interactions 1 1,690 1,690 1,6900 0,42 0,582 Curvature 1 0,429 0,429 0,4286 0,11 0,775 Residual Error 2 8,000 8,000 4,0000 Total 6 113,209 (a) Interaction seems to be negligible. (b) Curvature seems to be negligible. Are these true? We do analysis according to the linear model with Stat/Regression/Regression... Example 2 - continued 31 Regression Analysis: Y versus x1; x2 The regression equation is Y = 62,0 + 2,35 x1 + 4,50 x2 Predictor Coef SE Coef T P Constant 62,0143 0, ,16 0,000 x1 2,3500 0,7952 2,96 0,042 x2 4,5000 0,7952 5,66 0,005 S = 1,59049 R-Sq = 91,1% R-Sq(adj) = 86,6% Analysis of Variance Source DF SS MS F P Regression 2 103,090 51,545 20,38 0,008 Residual Error 4 10,119 2,530 Total 6 113,209 Example 2 - continued 32 Observe that we obtain the same coefficients of A and B as in the previous analysis. We now have an approximately linear relation y = x x 2. We now want to find the steepest ascent. Suppose that we are in (x 01, x 02 ) and should move to (x 11, x 12 ). The distance between these points is r = (x 11 x 01 ) 2 + (x 12 x 02 ) 2. We want to move in a direction so the change in the response y is maximal. We have y 0 = x x 02, y 1 = x x 12.

9 Example 2 - continued 33 Example 2 - continued 34 Furthermore, y 1 y 0 = 2.35(x 11 x 01 ) (x 12 x 02 ) scalar product [ ( )((x 11 x 01 ) 2 + (x 12 x 02 ) 2 ) ] 1/2 = c( ) Cauchy-Schwarz-ineq. wih equality when x 11 x 01 = c 2.35 and x 12 x 02 = c Choose c = If x 1 increase with 1 then increase x 2 with Note: Start in the center point (0,0) with y 0 = and take a step in the steepest ascent, se Figure 2 and Table 2. Table 2: Path of steepest ascent. Coded variables Time (min) Temp ( C) x 1 x 2 t T run observation ,6, (mean) Example 2 - continued 35 Example 2 - continued 36 When a new step doesn t give a better value, we stop and try to fit a new response surface with a new factorial design. Table 3: Result from the second design with two center points Variables in Variables in Response: original units coded units amount run tid temp x 1 x 2 y Figur 2.

10 Example 2 - continued 37 Example 2 - continued 38 Factorial Fit: Y versus A; B Estimated Effects and Coefficients for Y (coded units) Term Effect Coef SE Coef T P Constant 82,975 1,025 80,93 0,008 A -4,050-2,025 1,025-1,98 0,298 B 2,650 1,325 1,025 1,29 0,419 A*B -9,750-4,875 1,025-4,75 0,132 Ct Pt 5,275 1,776 2,97 0,207 S = 2,05061 PRESS = * R-Sq = 97,37% R-Sq(pred) = *% R-Sq(adj) = 86,84% P AB = and P P Q = indicates that we have some small curvature and interaction, but very few degrees of freedom for the SS E. We extend the design to a spherical CCD. Analysis of Variance for Y (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 23,425 23,425 11,713 2,79 0,390 2-Way Interactions 1 95,062 95,062 95,062 22,61 0,132 Curvature 1 37,101 37,101 37,101 8,82 0,207 Residual Error 1 4,205 4,205 4,205 Total 5 159,793 Example 2 - continued 39 Example 2 - continued. 40 Table 4: Resultat from the extended design Variables in Variables in Response: original units coded units amount run tid temp x 1 x 2 y Resultat from the second design with extra observations We put the measurements to Minitab. C1 C2 C3 C4 C5 C6 x1 x2 x1*x2 x1**2 x2**2 Y 1-1, , ,8 2 0, , ,7 3 0, , ,8 4 1, , ,4 5-1, , ,2 6 1, , ,5 7-1, , ,3 8 1, , ,2 9 0, , ,2 10 0, , ,5 11 0, , ,0 12 0, , ,0

11 Example 2 - continued 41 Now we can fit the second order polynomial with STAT/ Regression/Regression... The regression equation is Y = 87,4-1,38 x1 + 0,362 x2-4,87 x1*x2-2,14 x1**2-3,09 x2**2 Predictor Coef SE Coef T P Constant 87,375 1,002 87,22 0,000 x1-1,3837 0,7084-1,95 0,099 x2 0,3620 0,7084 0,51 0,628 x1*x2-4,875 1,002-4,87 0,003 x1**2-2,1437 0,7920-2,71 0,035 x2**2-3,0937 0,7920-3,91 0,008 S = 2,00365 R-Sq = 88,7% R-Sq(adj) = 79,2% Analysis of Variance Source DF SS MS F P Regression 5 188,189 37,638 9,38 0,008 Residual Error 6 24,088 4,015 Total ,277 Example 2 - continued 43 Example 2 - continued 42 We now have y = x x x 1 x x x 2 2, which we can be maximized by completing to squares y = ( x x x 1 x 2 + x x 2 ) 2 [ ( = x ) 2 ( ) x ( ) ] x x x x [ = (x x 2 ) x 2 ] x 2 The coming tasks 44 = [ (x x 2 ) ( x ) ] [ (x x 2 ) 2 = ] (x ) , with equality for x 2 = 3.00 and x 1 = x 2. Hence, optimum points are x 1 = 3.74, x 2 = Lab 6 is on Tuesday, Dec. 19th. Download and print the labs. Work in a group of two or three. Once you have done or tried all the tasks, you can queue. Then a teacher will check your solutions and may ask a few questions. If the teacher thinks your solutions and replies are OK, then you will sign a passing sheet for each lab. Home Assignment 3 must be submitted by 5 PM on Friday, Dec. 15th. And y max =

12 Thank you 45 Thank you!