2 FRED J. HICKERNELL the sample mean of the y (i) : (2) ^ 1 N The mean square error of this estimate may be written as a sum of two parts, a bias term

Size: px

Start display at page:

Download "2 FRED J. HICKERNELL the sample mean of the y (i) : (2) ^ 1 N The mean square error of this estimate may be written as a sum of two parts, a bias term"

Oswin Ball
5 years ago
Views:

1 GOODNESS OF FIT STATISTICS, DISCREPANCIES AND ROBUST DESIGNS FRED J. HICKERNELL Abstract. The Cramer{Von Mises goodness-of-t statistic, also known as the L 2 -star discrepancy, is the optimality criterion for an experimental design for the location model with misspecication. This connection between goodnessof-t statistics, discrepancies and experimental designs is shown to hold in greater generality. 1. Introduction Experimental designs tell the experimenter how to set the levels for various controllable factors in an experiment so as to eciently determine how these factors aect the response. Mathematically speaking, let s denote the number of factors, and let be a measurable subset of R s corresponding to the experimental domain or space of feasible levels of the factors. An experimental design is a nite set of points P. Several criteria have been proposed to choose good designs. Orthogonal designs choose the columns of the design matrix to be orthogonal, and so the information matrix is diagonal. Optimal designs minimize or maximize some function of the information matrix. The optimality criterion depends on the assumed model for the response in terms of the explanatory variables. Since the true model may not be known in advance, robust designs are of interest. For example, consider the location model with misspecication: (1) y g(x) + ; g(x) + h(x); where y denotes the response, x is the vector of factors, g(x) is some function of the factors, and denotes random noise with zero mean and variance 2. The function g(x) is decomposed into its mean value with respect to some given distribution F(x), g(x) df (x); and the misspecication, h(x). For a given design P fz (1) ; : : : ; z (N) g, one may observe the corresponding response values, y (i) g(z (i) ) + (i), and then estimate the unknown constant by Key words and phrases. Cramer { Von Mises, low discrepancy, reproducing kernel Hilbert spaces. This research was supported by a Hong Kong RGC grant 94-95/38 and HKBU FRG grant 95-96/II-01. 1

2 2 FRED J. HICKERNELL the sample mean of the y (i) : (2) ^ 1 N The mean square error of this estimate may be written as a sum of two parts, a bias term depending on the size of the misspecication, and a variance term depending on the size of the noise: i1 y (i) : (3) Ef(? ^) 2 g [Err(h)] N ; where the square root of the bias term is Err(h) h(x) df (x)? 1 N i1 h(z (i) )? 1 N i1 h(z (i) ) Err(g): For the location model, the positions of the design points aect only the bias term, and not the variance term. The bias term can be further analyzed using either a worst-case or an averagecase (Bayesian) approach. It can be shown that either approach produces an optimality criterion that is equivalent to a goodness-of-t statistic measuring how close the empirical distribution of the design, P, F P (x) 1 N z2p 1 fzxg; approximates F(x). Here 1 A is the indicator function of A, and z (z 1 ; : : : ; z s ) x (x 1 ; : : : ; x s ) means that z j x j for all j. The purpose of this article is to show this equivalence explicitly. Note that the optimality criteria or goodness-of- t statistics appearing here are called discrepancies in the numerical quadrature literature. 2. The Optimality Criterion Is a Discrepancy The rst step is to show that an optimal, robust design for the location model is one that minimizes the discrepancy. This is done by determining how large (3) can be for the average-case or worst-case h(x). For the average-case analysis the function of the factors, g(x), is assumed to be in a space of random functions A with zero mean and covariance kernel K(x; t): E g2a[g(x)] 0; E g2a[g(x)g(t)] K(x; t) 8x; t 2 C s : The mean value of the bias term in (3) is: E g2a[err(g)] 2 E g2a " g(x) df (x)? 1 N z2p K(x; t) df (x) df(t)? 2 N + 1 N 2 z;z 0 2P K(z; z 0 ) f(z) # 2 z2p K(z; t) df (t) K(x; t) d[f (x)? F P (x)] d[f(t)? F P (t)]:

3 GOODNESS OF FIT STATISTICS, DISCREPANCIES AND ROBUST DESIGNS 3 The right-hand side of this equation is called the square discrepancy, and depends on the design as well as the covariance kernel: (4) [D(P ; K)] 2 K(x; t) d[f (x)? F P (x)] d[f(t)? F P (t)]: Therefore, an optimal robust design for the location model is one that minimizes the discrepancy. Note from the formulas above that the computational work required to evaluate the discrepancy is at worst O(N 2 ), the work required to compute a double sum. The worst-case analysis assumes that the function of the factors, g(x), lies in a Hilbert space, W, with inner product h; ; i W and reproducing kernel K(x; t). The reproducing kernel is a function on, such that, (5) g(t) hg; K(; t)i W 8t 2 ; g 2 W: The reproducing kernel is always positive denite and symmetric in its arguments, that is, (6a) (6b) i;k1 K(x; t) K(t; x) 8x; t 2 ; a (i) a (k) K(x (i) ; x (k) ) 0 8a (i) 2 R; x (i) 2 : The bias term in (3) is the square of Err(g), which is a linear functional on W. By the Riesz Representation Theorem, this linear functional has a representer,, such that Err(g) h; gi for all g 2 W. Because W has a reproducing kernel, it is easy to nd this representer: (x) h; K(; x)i W Err(K(; x)): The Cauchy-Schwarz inequality then implies that (7) [Err(g)] 2 [Err(h)] 2 h; hi 2 W kk 2 W khk 2 W : This upper bound is attained when the misspecication, h, is proportional to. The rst term in (7) is just the square discrepancy using (4), since Therefore, the worst-case analysis gives (8) kk 2 W h; i W Err() [D(P ; K)] 2 : sup Ef(? ^) 2 g [D(P ; K)] g2w N : khk1 A robust design which is optimal against the worst misspecication minimizes the discrepancy. Theorem 1. Consider the location model with misspecication (1). Then E g2aef(? ^) 2 g sup Ef(? ^) 2 g [D(P ; K)] g2w N ; khk1 where A is a space of random functions with covariance kernel K, W is a Hilbert space of functions with reproducing kernel K, and D(P ; K) is the discrepancy de- ned in (4). Thus, the optimal design for this model is the one which minimizes the discrepancy. Moreover, given any continuous kernel, K, that satises conditions (6), it is possible to construct a set of functions, A, for which K is the covariance

4 4 FRED J. HICKERNELL kernel, and there exists a unique Hilbert space, W, for which K is the reproducing kernel. The proof of the rst part of this theorem is given in the paragraphs above, while the proof of the latter part is given in [6]. See [6] for a more detailed discussion of covariance and reproducing kernels. Suppose that the experimental domain is the s-dimensional unit cube, [0; 1] s, (perhaps, after suitable scaling), and that the underlying distribution, F, is the uniform distribution, F unif (x) x 1 x s : Furthermore, suppose that the covariance/reproducing kernel is chosen to be K(x; t) sy j1 [1? max(x j ; t j )] : This means that the space A consists of Brownian sheets (multidimensional generalizations of Brownian motions), and the space W consists of all functions g satisfying the s s 2 L 2 ([0; 1] s ) and gj xj 1 0 8j: Under these assumptions the discrepancy dened in (4) is: [D 2(P )] 2 [F unif (x)? F P (x)] 2 dx [0;1] s s 1? 2 sy 1? zj 2 3 N 2 z2p j1! + 1 N 2 sy z;z 0 2P j1 [1? max(z j ; z 0 j)]: This is known as the L 2 -star discrepancy in the numerical quadrature literature, and also as the Cramer-Von Mises goodness-of-t statistic [1]. The averagecase analysis for this particular kernel was done in [8] and the worst-case analysis in [9]. In the next section the connection between discrepancies and goodness-of-t statistics is generalized. 3. The Discrepancy is a Goodness-of-Fit Statistic From the denition in (4) it is clear that the discrepancy measures how far away the empirical distribution function F P is from F. Since K(x; t) is positive-denite, the discrepancy becomes small if and only if F P approximates F well. In fact, one may dene an inner product on a space of signed measures, M, in terms of the reproducing kernel. Let M be the following space of signed measures: M signed measure F (x) : K(x; t) df (x) df (t) < +1 ; and dene an inner product on M as follows: (9) hf; Gi M K(x; t) df (x) dg(t) 8F; G 2 M: In this case the discrepancy is just the norm of F? F P : D(P ; K) kf(x)? F P (x)k M :

5 GOODNESS OF FIT STATISTICS, DISCREPANCIES AND ROBUST DESIGNS 5 That is, any discrepancy is a goodness-of-t statistic that takes the form of the norm of F? F P, where the norm is induced by a certain inner product dened in terms of the kernel K. The converse is also true. Suppose M is some Hilbert space of signed measures containing all point mass measures Ffzg(x) 1 fzxg. One may then dene a kernel in terms of the inner product on M, (10) K(z; z 0 ) hffzg; Ffz 0 gi M 8z; z 0 2 : This kernel satises conditions (6) by the denition of inner product. Also, equation (9) holds automatically when F and G are measures assigning unit mass to a single point. Furthermore, equality (9) holds for any signed measures in F; G 2 M by writing F and G as limits of linear combinations of point mass measures. The results of the preceding two paragraphs give the following theorem: Theorem 2. The discrepancy as dened in (4) takes the form kf? F P k M where the norm is induced by the inner product, h; i M dened in terms of the kernel in (9). Conversely, if M is a Hilbert space of signed measures containing all measures of the form Ffzg(x) 1 fzxg, then kf? F P k M D(P ; K) for the kernel dened in (10). 4. Conclusion The previous two sections show the strong relationship between robust optimal design criteria for location models, discrepancies, and goodness-of-t statistics. There are several practical consequences of these results: i. The uniform design [2, 7] spreads design points evenly over the experimental domain by minimizing the discrepancy. Theorem 1 makes the connection between optimal design for the misspecied location model and the uniform design. ii. Any goodness-of-t statistic arising from an inner product is equivalent to the discrepancy dened in (4) by Theorem 2. The formula for the discrepancy provides a way of computing the goodness-of-t statistic in only O(N 2 ) operations. For example, the multidimensional Cramer-Von Mises goodness-of-t statistic (L 2 -star discrepancy) compares the proportion of experimental points in the box [0; x) to its volume, and may be written as: [D 2(P )] 2 [0;1]s ( [0;x) d[f unif? F P ]) 2 dx: Because the box is anchored at the origin, this statistic changes when the set of experimental points, P, is reected through the plane x j 12 passing through the center of the cube for any j. Therefore, one might wish to modify this goodness-of-t statistic to consider all boxes [t; x) [0; 1) s, that is, [D(P )] 2 [0;1] 2s ; tx ( [t;x) d[f unif? F P ]) 2 dt dx:

6 6 FRED J. HICKERNELL This L 2 -unanchored discrepancy is not easy to compute in the above form, but using Theorem 2, one can obtain the relevant kernel as K(z; z 0 ) [0;1] 2s ; tx ( [t;x) dffzg [t;x) dffz 0 g ) dt dx sy j1 [min(z j ; z 0 j)? z j z 0 j]: The formula for the corresponding discrepancy then follows from (4) as s 1 2 sy [D(P )] 2 zj (1? z? j ) + 1 sy [min(z 12 N 2 N 2 j ; zj) 0? z j zj]: 0 z2p j1 z;z 0 2P j1 This formula was obtained earlier by [5]. iii. By Theorem 2, goodness-of-t statistics may be used as discrepancies, which indicate the error of numerical quadrature. Conversely, discrepancies in the numerical quadrature literature (see, for example, [3, 4]) may be used as goodness-of-t statistics. Acknowledgments The author would like to thank Sung-Nok Chiu, Gang Wei, Min-Yu ie, and Rong-ian Yue for their valuable comments and suggestions. References 1. R. B. D'Agostino and M. A. Stephens (eds.), Goodness-of-t techniques, Marcel Dekker, New York, K. T. Fang and F. J. Hickernell, The uniform design and its applications, Bull. Inst. Internat. Statist., 50 th Session, Book 1 (1995), 333{ F. J. Hickernell, A generalized discrepancy and quadrature error bound, Math. Comp. 67 (1998), 299{ , Lattice rules: How well do they measure up?, Pseudo- and Quasi-Random Point Sets (P. Hellekalek and G. Larcher, eds.), Lecture Notes in Statistics, Springer-Verlag, New York, 1998, to appear. 5. W. J. Moroko and R. E. Caisch, Quasi-random sequences and their discrepancies, SIAM J. Sci. Comput. 15 (1994), 1251{ G. Wahba, Spline models for observational data, SIAM, Philadelphia, Y. Wang and K. T. Fang, A note on uniform distribution and experimental design, Kexue Tongbao (Chinese Sci. Bull.) 26 (1981), 485{ H. Wozniakowski, Average case complexity of multivariate integration, Bull. Amer. Math. Soc. 24 (1991), 185{ S. K. aremba, Some applications of multidimensional integration by parts, Ann. Polon. Math. 21 (1968), 85{96. address: fred@hkbu.edu.hk, Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

E(s 2 )-OPTIMALITY AND MINIMUM DISCREPANCY IN 2-LEVEL SUPERSATURATED DESIGNS

Statistica Sinica 12(2002), 931-939 E(s 2 )-OPTIMALITY AND MINIMUM DISCREPANCY IN 2-LEVEL SUPERSATURATED DESIGNS Min-Qian Liu and Fred J. Hickernell Tianjin University and Hong Kong Baptist University