A. R. Manson, R. J. Hader, M. J. Karson. Institute of Statistics Mimeograph Series No. 826 Raleigh

Transcription

1 F 'F MINIMJM BIAS ESTIMATION.AlfD EXPERIMENTAL DESIGN APPLIED TO UNIVARIATE rolynomial M)DELS by A. R. Manson, R. J. Hader, M. J. Karson Institute of Statistics Mimeograph Series No. 826 Raleigh

2 MINDIJM BIAS ESTIMATION AND EXPERIMENTAL DEEIGN APPLIED TO UNIVARIATE POLINOOIAL MODELS A. R. MANSON AND R. J. HADER Department of Statistics North Carolina State University Raleigh, North Carolina M. J. KARSON Graduate School of Business Administration Universtty of Michigan Ann Arbor, Michigan Minimum. bias estimation, as introduced by the authors in [4], is applied to those combinations of univariate polynomial responses and polynomial approximating functions of lower degree which are of primary practical importance. Designs in the experimental region of interest R: (-1, +1] which give smallest integrated variance V are presented. Examples of contour plots of constant V versus design levels: show regions of experimental designs which give smaller integrated mean square error than is obtained by using standard least squares estil!lcltion for designs which attain minimum. bias B via choice of design moments. KEY WORDS f Minimum. Bias Estimation Experimental Design Polynomial Models Polynomial Approximating Functions

3 INTRODUCTION In the approximation of response relationships, several authors have taken both variance and bias errors into consideration (for example, see [11, (21, [3], and (4). In a previous paper [41, minimum bias estimation of a polynomial response model was introduced. The present paper concerns itself with the design implications associated with minimum bias estimation in the univariate case. It is necessary to review briefly the notation and results of r41, where it is assumed that the response variable ~ is related to an independent variable via a polynomial of degree d + k - 1, in which ls~ is a row vector containing p0l<ters of x up through order d - 1; ls~ is row vector containing powe:cs of x from order d through d + k - 1; and ~l and ~2 are column vectors of the corresponding regression coefficients. Equation (1.1) is to be approximated by using observations of the response variable taken at selected levels of the single, ind.ependent variable x to fit the polynomial of degree d - 1 y(~) = x'b..-,...,);:,1 (1.2) The estimation criterion adopted is that of minimizing the squared bias integrated over a specific region of interest R in the independent variable, i.e., it is desired to minimize (1.3 )

4 - 3 - \ Furthermore, subject to minimizing B, the estimation method used should then minimize the integrated variance of the fitted polynomial y(x), NO S,. V =:2 R Var[Y(~ldx, (j (1.4) for any fixed experimental design. In the univariate case presented here, the region of interest R will be the interval [-1, IJ on x. The symbol a used in equations (1.3) and (1.4) is defined via its inverse as..1 J (1 = Rdx. The primary criterion selected is motivated by a desire to use simple low order polynomial approximations to estimate response relationships but to achieve some measure of robustness or protection against the possible existence of higher order polynomial terms. The expected mean square error integrated CNer R is simply the sum of V and B (1. e., J = V + B). Minimization of this sum requires advance knowledge of the regression coefficients which is not generally available. Results obtained by Box and Draper in (11 and [2J indicate that unless V is quite large with respect to B, optimum designs for minimizing the sum of V and B are very close to those obtained by assuming that V = O. Their approach used standard least squares estimation in the approximating polynomial Y(x). In view of the dominating importance of B, we have chosen to use a method of estimation aimed directly at minimization of B. Another reason for desiring to minimize B comes from a numerical analysis viewpoint. one approximates a polynomial response by another polynomial of If

5 - 4 - lower degree, then it seems reasonable to use the best approximating polynomial, apart from experimental error considerations. In (4J it is shown that an estimator of the form...1 b = A(X'XrX' 1, (1.6) will minimize B and subject to giving minimum B will minimize V for any fixed design. In equation (1.6) the matrix X contains the full set of powers of x in (1.1) for the given design; y is the column vector of... observations on the response variable; and where I d is a d by d identity matrix, (1.8) and By the integration of a matrix, we in:g;>ly integration in an element by element sense. The notation (x'xf used in equation (1.6) designates the ordinary inverse of XiX if it is non-singular and the generalized inverse if XiX is singular. If XiX is singular, then it is required that be estimable for a given design to be "admissible". inverse satisfying the single requirement Any generalized

6 - 5 - (x/x){x'xf (XiX).. XiX,. will suffice to make the estimator.2 1 unique. Furthermore" for Y' and 11 which are polynomials 1n x it can be shown that W 1 1s a positive.. definite matrix and therefore bas an ordinar1 inv~rsej exists for all adaiasib~e designs.,. The estimator Y' - mini.jm.jm bias estimator for 11- thus w~lw2 = ~~l with.el given in (1.6) is referred to as the It obviously does not depend on advance knowledge of the true regression coefficients of (1.1), it is unique" minimizes B and subject to giving Min B gives min.:1.mum V for any fixed design for which ~ is estimable" and contains the method of standard least squares estimation as a speciaj. case when The same minimum B is attained for any design for which.afj,.., is estimable" name~ (l.lo) where (l.ll) By the standard least squares estimator we mean (J..12) where Xl is that part of the X.. (X :X ) matrix which involves only the 1 2 values of terms in ~i. Box and Draper [1] showed that Min B is attained :.".", : ".

7 - 6 - using the standard least squares estimator and requiring that the experimental design used have moments satisfying the requirement Designs which satisfy (1.1.3) are special cases of the larger class of./ designs for which ~ is estimable, for which the same Min B is attained,..., using minimum. bias estimation. The additiona! design flexibility obtained is one of the main advantages of the minimum bias estimation method. One of the more obvious ways of using this flexibility is to choose designs which will minimize V aver all admissible designs. In addition, it is possible to obtain a smeller value of intei~rated mean scjuo.ce error J using minimum bias estimation than is obtained using standard least squares estimation for those designs which satisfy the Box-Draper requirement given in equation (1.1.3). In the following sections we shall treat the design problem of minimizing V aver the class of admissible designs for the 8ituations where (d =2; k =1, 2, 3), (d =3; k =1, 2), and (d =4; k = 1). The model situation (d = 2; k = 1,2,3) implies that y(x) is a,..., linear polynomial in x while 11 is either a quadratic polynomial in x (k = 1), a cubic polynomial in x (k = 2), or a quartic polynomial in x (k = 3). Our general procedure shall be: using the minimum bias estimator find the design which minimizes V aver the class of admissible designs, i.e.,. those for which JfJ is estimable.,...,

8 LINEAR APPROXIMATING POLYNOMIAL This section is concerned with situations where the linear approximating polynomial (d = 2) (2.1) is expected to give a reasonable fit to the true response. Three polynomial responses will be investigated; quadratic (k = 1), cubic (k = 2), and quartic (k = 3). To specify a particular approximating polynomial, true response, and design setting, a triple of the form (d, k, N) will be used, e.g., (d, k, N) = (2, 1, 4) will,. be used to designate a situation where y is of degree d - 1 = 1 as in (2.1), ~ is of degree d + k - 1 = 2 (quadratic) where k represents the,. difference in degree of the polynomials for y and ~, and N = 4 is the total number of observations of the response. 2.1 Quadratic Response (k = 1) form Tbe true response is assumed to be of degree 2 (quadratic) of tbe We shall restrict our attention to designs symmetric about the origin in terms of the x, in determining those designs which will u minimize V given in equation (1.4). simplified since The expression for V may be to give,. I I - 2 Var y = ~la(x X) A/~CT."

9 - 8 - N f1 I (, )- I I - I V = 2' -1 ~1A X X A ~1dx = N trace [A(X X) A W 1 ) If XiX is non-singular, then for (d, k, N) = (2, 1, N) where and v = 9c-6a+l (2.1.4) 2 3a ' 9(c-a ) N 2 a = t x /N u u=1 N 4 c = 1: x /N. u u=1 Note that non-singularity of XiX requires that 0 < a 2 < c s Na 2 /2 for real valued x. u The expression for V given in (2.1.4) may be verified by evaluating equation (2.1.3) using the appropriate matrices, A, (X/X,-~ and WI which are given in Section 1 of Appendix A. To determine the Xu for an N point design so as to minimize the V of equation (2.1.4) one simply minimizes V over a and c. However, the equation ~: = 0 gives (3a - 1)2 = 0, which does not contain c. Therefore, V is minimized (for any value of a) when c is either at its maximum value or at its minimum. and will in general give V = co If c = a 2 then XiX will be singular (except in the special case of a =1/3 which wi 11 be treated below). Hence, }liin V with respect to a and c occurs when c = Max c =lile?"!2.. Designs which have c =Na 2 /2 must have N. - 2 of. the Xu at zer~, 0r...e value of x at +.1" and one value of x at -t where t 2 =Na '2. I.n fa.ct u u I

10 - 9-1 fn Min V = M[N +31a + 9" a 1, c which can be minimized with respect to a by the usual techniques to give Min V = (18-N)/8 for 2 ~ N ~ 5 a,c (2.1.8) which occurs when a = 4/(18-3N). For N ~ 6, V possess no relative minima for finite values of a, in which situation one should choose the maximum. possible value of a in order to decrease V. In the limit as a approaches infinity Min V = HI (N-2) a,c If one wishes to confine the experimental points to the region of interest R: (-1, 1], then Min V = 4(:~N-4 )/9(N-2) for N ~ 6. a,c In situations where XiX is singular (for non-trivial designs), the singularity occurs when c = a 2 (which will always be the case when N = 2 and occurs for N > 2 whenever only two of the design points are distinct). These situations require a = 1/3 for estimability of I/J and such a value of a gives V = 2.,... Designs having \(C' = a = 1/3 i must have all Xu equal in magnitude and thus have all non-zero levels of x at +.t. These designs will satisfy the Box-Draper condition of u - (1.3.3) and hence the minimum bias estimator.21 will coincide with the *,. standard least squares estimator Eol for this y, 11 XIX is singular. combination whenever

11 Table 1 gives the values of Min V for 2 s: N s: 7, and all designs given in Table 1 have N - 2 values of x u at zero, one value of x u at +1., and one value of Xu at -1.. Figure 1 shows how V varies with the single non-zero design level, TABLE 1 Design Parameter Values Which Minimize V for (d, k, N) = (2, 1, N) N a c = «&2/ Absolute Min V Min V in R: [-1, 1], 2 1/3 1/q 1/ /9 81Z( 2/ /3 8/9 4/ (1.=1) 5 4/3 40/9 10/ (1.=1) 6 co ro co (1.=1) 7 co co co ll1ll (1.=1) Note that for N ~ 4, the absolute Min V does not occur for a design in R, in which case the design having smallest V in R is given. For N s: 5, designs which were non-symmetric in the x were u investigated to see if smaller values of V than those given in Table 1 could be obtained. The results showed that no non-symmetric designs exist which will give smaller V than the best obtained for the corresponding symmetric designs of N observations. If one were to use the standard least squares estimator and b* = (XiX )-lx- l -1 1: *, * y =!Jl>J.'

12 - II v 2.0 o 7 N= ! 0 5 J..l J, Figure 1 V versus the design level t for (d, k, N) = (2, 1, N)

13 then which can be shown by writing out the elements of the above matrices in summation form. Designs which satisfy the Box-Draper conditions for Min B given in equation (1.13), via the sufficient conditions always have For d = 2 if equation (1.13) is satisfied, in any way, then V * = 2. However, for d > 2 it is possible to achieve V * < d for designs satisfying equation (1.13). For the particular estimator-model situation (d, k, N) = (2, 1, N), it is necessary that a = 1/3 in order for a design to have moments satisfying (1.13). Such designs lie outside R for any N ~ 4 and hence no design exists in R for N ~ 4 which will give Min B using standard least squares estimation. For (d, k, N) = (2, 1, N) 1t is always possible to find designs in R with V < (V* = 2) for N ~ 3. In fact, Figure 1 shows an infinity of designs for each N ~ 3 for which V < 2, so that is is not necessary to choose the design level t so as to minimize V. Rather, one should choose t in a range to give v < 2, and to satisfy whatever other design criteria one might have. This additional flexibility is available in addition to attaining Min B and V < 2 which means that it 1s possible to obtain smaller, integrated mean square error, J = V + B, than is possible using any design which

14 - ~ - will satisfy the requirements of (1.13) wi. thout requiring any advance knowledge of the parameters f:. 1 or ~. 2.2 Cubic Response (k = 2) The true response is now assumed to be a polynomial in x of degree, (cubic), of the form Methods analagous to those used in Section 2.1 give v = 2 c -6a; e-30 c ;9 a = N tr[a(x/x)-a/w ) 9-(c.a. ) 75(ae-c) l for (d, k, N) = (2, 2, N) when XiX is non-singular, where a and c are defined in (2.1.5) and (2.1.6) and N 6 u u=l e = 1: x IN (2.2.3 ) See Section 2 of Appendix A for the appropriate matrices used in equation (2.2.2). The method of minimizing V over choice of design is similar to that used in Section 2.1. It is desired to determine the values of the design moments a, c, and e which minimize V. For the x u levels of the design to be real valued, the range of c is dependent on the value of a and similarly, the range of e is a function of both a and c. However, ~~ = 0 implies that (5c... 3a)2 = 0 which does not contain e. Hence, Min V occurs either when e is at its maximum value (in terms of a and c) or at its minimum value. A check shows that maximum e gives Min V. Thus, to minimize V for any values of a and c, one must maximize e. This can be accomplished by symmetric designs which have one value of Xu at each of.:!: 1-, m/2 values of Xu at each 2

15 N { tt. = 2m a- (2.2.4) and m = (N-2)/2 if N"is an even integer ~ = (N-3)/2 if Nis an ~~d integer. At this point minimization of V over a and c becomes intractible by analytic methods. AccordinglY, numerical optimization methods were used to determine the values of a and c (or alternatively t l and t 2 ) which give Min V for N = 4 through N = 15. For values of N which have Min V designs outside of R: [-1, IJ, the design in R which gives smallest V is listed in Table 2, although such a design may not have the "optimum allocation" of design points required for absolute Min V. Table 2 also lists the designs in R which have singular XiX (1. e., rank = 3), allow estimation of Af', and give smallest V among such ~ designs, provided such smallest V is less than V = 2. Those designs which have XiX of rank 3 would not allow fitting the assumed true, cubic response by a full cubic polynomial and hence a polynomial of * lesser degree would be required. Minimum bias estimation allows polynomial modela which can not be approximated by polynomials of the same degree to be fitted by lower degree polynomials and still obtain

16 * Min B, a value of V < (V = 2), and a lower value of integrated mean square error J than is obtained using standard least squares estimation for designs satisfying the conditions of equation (1.1-'). TABLE 2 Designs in R with Smallest V for (d, k, N) = (2, 2, H) H NO N 1.t 1 N 2 1,2 V Rank of X'X * li '* ~87 3* * * ' * <: o * <: ' * 1-' l235 3* * ; ' * Designs having singular XiX matrices While Table 2 lists the single designs in R having smallest V, there are, for each value of H, an infinite number of designs located in a region in the Cel' 2) plane surrounding the optimal design for which V < (V * = 2). Figure 6 of [4] shows the contours of constant V for N = 4. Any design characterized by values of Cel' 2) ic.aide the

17 v = 2 contour will give a s~er integrated. ill.ean square error thani! standard least squares estimation were used and the design were required to satisfy the Box-Draper conditions for Min B given in equation (1. ~). See Table Bl of Appendix B for additiona"l Hin B design. If one desired to use the add.!tional flexibility obtained using the minimum bias estimator to achieve equal spacing of design levels (for example) then it is possible to minimize V along the line t = 2 3t (for N even) or along t = 2.t (for N odd). See [4J for an l 2 l example. Table 2 presents only the optimal designs for Min V. There are alternative allocations ot design points which lead to values of V which are less than V * = 2. For example, if N = 8, a symmetric design could have been obtained by taking two points at x = 0, three points at each ot x =!t l = , and one point at each of x =!t 2 = which would have given V = which does not differ appreciably trom Min V = 1.883~5 given in Table 2. Hence, it is possible to use non-optimal allocation ot the N design observations and still obtain a large class of designs for which V < 2 and which therefore will have smaller integrated mean square" error than designs satisfying (l.~). ", 2.3 Quartic Responses (k =3) The fitting ot a quartic po).ynomial response of the form (2.3.1),. by a linear y is ot somewhat less interest from a practical viewpoint. However1 it is presented tor completeness.

18 The expression obtained for V is quite lengthy and will be om1tted since minimization of V was accomplished numerically. Section 3 of ~AppendiX A gives the matrices (X / xrl, A, end W 1 required to evaluate V for non-singular XiX. Table 3 gives the symmetric designs in R: [-1, 1] of the form NO observations at x = 0, N l observations at each of x =!.1IJ! and N 2 observations at each of x =.! 1,2' which minimize V and which have XiX matrices of full rank (5). The table also contains designs in R giving smallest V for singular XiX matrices of rank ; and 4 provided such smallest V is less than V * = 2. The conditions on the non-zero levels of x for estimability of }fj in cases of singular X'X matrices,... are: ti =0 and.t~ =3/5 for XiX of rank; Note that for N = 4, the best four point design gives V = *. compared to V = 2 for the umque four point design which will satisfy equation (l.l,}) having 1,1 = ; and 1. 2 = 0.7g(654. If one were to use minimum bias estimation and desire equal spacing of design levels for N = 4, then the optimal design would have 1,1 = ;7, 1,2 = and V = See Table B2 of Appendix B for additional Min B designs. 2.4 Comparison If one denotes by Vl' V2' and V;, the values of V obtained when k = 1, 2, and ; respectively, then it is easy to show that V1 s; V2 :l: V; for any fixed design in the admissible cj.ass (i. e., ~ estimable).

19 TABLE 3 Designs in R with Smallest V for (d, It, N) = (2, 3.. N) B NO IV 1 t 1 N 2 1,2 V Rank of X'X * * o g-f * CX)CX) o * * * CX) * f * g-f * O.81cm o * * * * * o gT 1 90l235 3* o g-f * * Designs having singular X'X matrices This is true because in adding terms of higher degree to the assumed polynomial model one produces positive covariances between models differing in degree. Hence, VarG) will increase as k increases for a fixed design, and therefore.. V must also increase as k increases. It is possible to achieve a measure of robustness when fitting a response by a linear polynomial, in the sense that designs exist which

20 A for the same linear y will simultaneously g1ve minimum B whether the true response model is the quadratic polynomial (k = 1) of (2.1.1) or the cubic polynomial. (k = 2) of (2.2.1). Of course, Min B could be obtained for both k = 1 and 2 using different estimators ~l)' but this will in general increase V 2 since V 2 ~ V l The condition for which a,. linear polynomial y may be used to achieve simultaneous Min B for k = 1 or k =2 is c = 3a/5 or N 4 3 N 2 1: x =-5 1: x u u u=l u=l Such designs will of course have V = V (which is as small as V l 2 2 can be for a given design). That particular design having smallest V1 = V2 for a N = 4 point design occurs when the design has one point at each of x =,!.t l = o.3539ll and, cne point at each of x =.:.tt 2 = giving V l = V 2 = This value of V l = V 2 is still less than V * = 2, which is the best that can be achieved by designs satisfying the conditiona of (1..13) and using standard least squares estimation. iee [51 for an alternative derivation of (2.4.1). By carrying the logic one step further it is possible to determine... the conditions for which the same linear y may be used to simultaneously protect against the true model being quadratic, cubic, or quartic (i.e., k = 1, 2, or 3). However, it is felt that such a requirement is not as realistic in a practical sense as obtaining simultaneous protection against the response being a polyncdial of It = 1. or 2 degrees higher than the. approximating pol;ynomial

21 QUADRATIC APPROXIMATING POLYNOMIAL (d = 3) Let us now extend the results of the previous section to the situaion whe-re a response is approximated by a quadratic polynomial in x, of the form Attention will be confined to true response models which are either cubic (k = 1) or quartic (k = 2) polynomials. 3.1 Cubic Response (k = 1) When the true response is a polynomial of k = 1 degree higher than "- that of y, of the form (3.1.1) we may evaluate V using the appropriate matrices from Section 4 of - Appendix :A -to obtain V = N tr(a(x'x)-la'w ] =!5c-].0~+3 + g?e-30 c ;9 a 1 15(c-a) 75(ae-c) (3.1.2) This expression is valid for non-singular X/X (L e., c.; a 2 ano.- ae.; c 2 ). Using techniques identical to those used in Section 2.2, it can be shown that 9Y = 0 does not contain e and that Min V occurs when e is at oe e its maximum value. Hence minimization of V was then accomplished numerically over the levels of a and c or equivalently over design levels t l and t 2 The resulting designs were frequently located outside of R: [-1, 11; so the I'best" symmetric designs in R, in terms of smallest V, are given in Table 4. The symmetric allocation of design

22 observations considered for the entries of Table 4 is No points at x = 0, N 1 points at each of x =.!ll'.~ ad N points at each of 2 x =.:!:l2; so that N = No + 2N + 2N as in Section 2.~. 1 2 Note that each of the designs listed in Table 4 have V ~ (V* = ~)... * which is the value of V obtained by the standard least squares estimator for designs satisfying the sufficient conditiona for Min B glven in equation (2.1.9). For d = ~ the minimum value of V* is rather than V* =~. This value of Min V* is attained for designs which satisfy the necessary and sufficient conditions for Min B given in equation (l.13). These designs have a = 0.~76544 and c = TABLE 4 Designs in R with Smallest V for (d, k, N) = (3, l, N) N NO N 1 t l N 2 t 2 V Rank of XiX 4 0 l 0.~J.6012 l g l l ~ ~2 2 R~4l o o o ~* 7 l l l.oooooo l * l l l.oooooo ,} 4 9 ~ 2 0.6~04l0 l l.oooooo ~ ~.oooooo ~* lo 0 ~ 0 ~96~~~ lo ~ ~* 11 1 ~ ~ ~ 2 l.oooooo ~.OOOOOO ~* 13 l l.oooooo 2.6l l111 3* ~ ~* l o 2.636~18 4 l ~.OOOOOO ~* * Designs having singul.ar X'X matrices

23 Each design in Table 4 having non-singular XiX may be specified as a point in (t l,.t 2 ) space for a given allocation of the N observations. Such a point is surrounded by a region of points corresponding to designs (of the same allocation) which have V < V *, so that additional flexibillty is available if one merely requires that V be less than V* rather than requiring Min V. Figure 2 shows contours of constant V for (d, k, N) = (;, 1, 5). Should equal spacing of design levels be desired, the best such design would have NO = N l = N 2 = 1 with t l = O. 464z77, t 2 = ;, and V = 2.7;;991. It should be noted that designs for which neither t l nor t 2 equal one are, in fact, l\1in V designs since absolute Min V is attained in R. Also, some of the l1best" V allocations of design points differ from the "optimal l1 allocation for obtaining Min V because absolute Min V is not attained in R. See Table B3 of Appendix B for additional Min B designs. 3.2 Quartic Response (k = 2) The true response is assumed to be a polynomial in x of degree 4 (quartic), of the form (;.2.1) The expression Obtained for V when XiX is non-singular is extremely. messy and is not presented here since minirti7.ation of V over choice of design was done numerically. The appropriate matrices required to evaluate are given in Section 5 of Appendix: A. The type of symmetric design

24 r o. 'r Figure 2 '2 Contours of constant V for (d, k, N) = (3, 1, 5)

25 considered was again restricted to that used in Section '.1, namely NO observations at x = 0, N l observations at each of x =!.ll' and N 2 observations at x =!./'2. For XiX to be non-singular (of rank 5) for such designs, it is necessary that NO' N l, N2" t l and 1,2 be non-zero and that /'11: 1. 2 When X'X is singular, ~ is estimable only when XiX is of rank 4, i.e., when NO = 0 with all other requirements for nonsingul.arity holding. This requires an even number of total observations N for symmetric designs of the type considered. Table 5 gives the designs in R having smallest V for both singular and nonsingular X'X matrices provided such smallest V obtained is less than or equal to d = 3, the value of V* which would be obtained by the standard least squares estimator for any design satisfying the sufficient conditions for Min B given in equation (2.1.9) However, it is of interest to note that it is not possible to satisfy even the necessary and sufficient conditions for Min B given in equation (1.13) for (d, k" N) = (3, 2, N) using 5 point, symmetric designs of the type considered. ThUS, the minimum bias estimation results have no direct comparison with the standard least squares,. approach for this y, 'Tl combination. Once again, it should be pointed out that for brevity, only the designs having smallest V are shown in Table 5. The design coordinates (ll' 1. 2 ) for each design shown in Table 5 are surrounded by a res!0n of designs each having V ~ d, and in most cases these regions are quite extensive. Furthermore, nearly all symmetric allocations of the N design observations to the levels 0,!.L,!.1. have smallest l 2 values of V ~ d which are in turn surrounded by regions of designs

26 25 - TABLE 5 Designs in R with Smallest V for (d" k" N) = (3" 2" N) N' No 1f N 2 "2 V Rank of XiX , * ll * l.J * ;? * "" * l * * Designs for which X'X is singular having V ~ d. Only that symmetric allocation producing the smallest value of V is shown in Table 5. For example" for N = 15 if one takes NO = 5 observations at x = 0" N 1 = 4 observations at each of x =!,t1 = " and N 2 = 2 observations at each of x 2 Q!,t2 = " then the value of V obtained is V = which is not appreciably larger than the smallest value. of V listed in Table 5 for N = 15. Thus" a wide range of choices is available in allocating the N observations to, the five design points. See Table B4 of Appendix B for additional Min B designs. 3.3 Comparison of k = 1 and k = 2 Let V be the value of l V when 11 is cubic (k = 1) as in equation (3.1.1) and V 2 be the value of V when 11 is quartic (k = 2) as in equation ( ~). Then it can be shown tha.t" for any fixed design"

27 that V l S; V 2 Conditions for equality of V l and V 2 are not simple, as was the 8ituation for d = 2. Indeed, there is no continuum in the (t, l t ) plane for which V 2 l = V For example, if N 2 = 5 with No = N l = N 2 = 1, then V l = V 2 at only two isolated points in the (t, l t ) plane. These are at (ll 2 = , l2 = ) giving V l = V 2 = and at (ll = , l2 = ) giving.,,:.. V1 = V2 = The former of the two points is reasonably close to the design giving Min V l in R, namely ( 1 = , t 2 = ) with V1 = and is very close to the design giving Min V2 in R, namely (t l = , t 2 = ) with V 2 = Hence, if one were fitting a response with a quadratic approximating polynomial (d = 3) and wanted to obtain minimum bias B for true models of degree 3(k = 1) and 4(k = 2) using the same estimator, then the 5 point design with (t l = ~ t 2 = ) would be the best choice of such designs. Figures 2 and 3 illustrate the contours of constant V for k = 1 and k = 2, respectively. These two figures have the Min V design locations in the (tl' l2) plane marked with small circles as well as the two points at which V1 and V2 are equal. 4. CUBIC APPROXIMNrING POLYNOMIAL This section is concerned with the situation where a third order (cubic) approximating polynomi.al (d = 4), is expected to fit the true response 11, with reasonable closeness. The true response 11 will be assumed to be a quartic polynomial in x(k = 1),

28 - 'Z t Figure 3 Contours of constant V for (d, k, N) = (3, 2, 5)

29 of the form: (4.2) Once again the expression is quite 1.engthy and canplicated in terms of design moments and will be amitted. See Section 6 of Append;ix A for the ap:propriate matrices for eval.uating V when X'X is non-singu1.ar. Minimization of V over admissib1.e designs in R which are symmetric and have five 1.evels of the form considered in Sections 301. and 3.2 for both singular (rank 4) and nonsingu1.ar XiX was accomplished numerical.1y, with the results shown in Tab1.e 6. The design moment conditions required for estimability of /fj,.., when XiX is singu1.ar (rank 4) are: (4.3) and (4.4) These conditions are also the requirements for designs to satis1'y equation (1..13) when standard 1.east squares estimation is used, and therefore give V = V* when X'X is singular of rank four. "

30 TABLE 6 Designs in R with Smallest V for (d, k, N) = (4, 1, N) N NO N 1 l1 N 2.t 2 V Rank of XiX ;; l.l * ;; O. 86J..J., * d> ;; J., o 4* o J., * o o lJ., o 4* o o. gr01b W * d> > * Designs for which X'X is singular. These are also the designs which give Min B when the standard least squares estimator is used. 5. SUMMARY I In this paper we have presented the application of "minimum bias combinations of primary practical importance,. where both y and 11 are polynomials in a single, independent variate.,. For fitting a linear polynanial (y) to either a quadratic or a cubic estimation" to those y, 11 polynomial resonse (11), the symmetric designs which give absolute minimum integrated va:dance (V) are determined. For all other y., 11 combinations considered, the type of design considered was limited to a five point design having NO observations at x = 0, N 1 observations at each of x =.:.t 1 ' and N 2 observations at each of x =.!.l2" When the designs gi.ving absolute minimum V are outside of R: [-1,. 1] those

31 designs in R which give smaj.lest V are presented. It is seen that minimum bias estimation in combination with proper experimental. design: (i) l.eads to smaller integrated mean square error than is obtained by standard l.east squares estimation for designs which satisfy the Box-Draper conditiona for Min B as given in equation (1.13) j (ii) does not require any prior information concerning the unknown regression coefficients; (iii) allows a great deal. of flexibility with respect to design levels and with respect to the allocation of observations to the design levels j (iv) allows a type of robustness to be achieved, in that one may A use a single estimator (y) and obtain Min B simul.taneously for each of several. possible "true models" (11); (v) A allows Min B to be attained for many y, 11 combinations by designs having a minimal. number of l.evels, for which the true model coul.d not be fitted by standard least squares I estimation because of singularity of the design matrix, X X; (vi ) and is reasonably simple to apply. REFERENCES [11 Box, G. B. P. and Draper, N. R., A Basis for the Selection of a Response Surface Design. Jour. ~ ~ M$OC. 54: , (1959). (21 Box, G. E. P. and Draper, N. R., The Choice of a Second order Rotatable Design. Biometrika 50: , (1963).

32 [3J [4J [5J David, H. A. and Arens, B. E., Optimal Spacing in Regression Analysis. Annals of ~. stat. 30:J , (1949). Karson, M. J., Manson, A. R., and Hader, R. J. Minimum Bias Estimation and Experimental Design for Response Surfaces. Technometrics 11: , (1969). Karson, M. J., Design Criterion for Minimum Bias Estimation of Response Surfaces. Jour. Amer. Stat. Assoc. 65: , (1970). APPENDIX A 1. (d, k, N) = (2, 1, Nj c -a --2' 0, 1 2 c-a c-a 3 (X'Xr 1 = N W 1 = A, and 0, a' -a 1 ~ ~, 0, 2 c-a c-a 2. (d, k, N) = (2, 2, N) c 0 -a --2', --2' 0, c-a c-a 1 e c 3 2' 2' A and ex'xfl = N- 1 ae-c ae-v = a 1 - -'-2' 0, --2' 0, 5 c-a c-a -c a 0 2' 0, 2' ae-c ae-c For W 1 see Section 1 of Appendix A.,

33 (d, k, N) = (2, 3, N) A = L 0 5 and cf-e 2 b.,, ce-af 0 0 b., 2 ae-c b. 0, e -c 0, 2' 2' ae-c ae-c (x'xf 1 1 ce-af f-c2 ac-e = N- 0 0 b. b. b. where -, --,, -c a 0, 0 0 2' 2' ae-c ae-c 2 2 ae-c ac-e c-a 0, 0, b. b. b., 0 See Section 1 of Appendix A for WI 4. (d, k, N) = (3, 1, N) W-= and A = L " See Section 2 of Appendix A for (X'xf 1.

34 5. Cd, k, N) = (3, 2, N) A = ~ For WI see Section 4 of Appendix A and for (XiX) see Section 3 of Appendix A. 6. (d, k, N) = (4, 1, N.l _L W = 3" 5 and A = See Section 3 of Appendix A for (x / Xf 1.,

35 APPENDIX B,..- TABLE Bl * N) = (2, 2, N) Alternative Designs in R Having V ~ (V = d) for (d, k, N NO N 1 1,1 N 2 1,2 V Rank of XiX ,6 3 0.~ ~ o * * * XiX Designs having singular matrices

36 TABLE B2 * for (d, N) = (2, 3, N) Alternative Designs in R Having V ~ (V = d) k, N NO N 1 1,1 N 2 1,2 V Rank of X'X * Ou oo 58l';~)'4./.j' * * * ) , * ,, ou * * Designs which have singular X'X lnatrices

37 TABLE B5 Alternative Designs in R Having V ~ (V* = d) for(d, k, N) = (4, 1, N) N NO N 1 1,1 N 2 1,2 V o All the designs in this table have non-singular X'X matrices