Nested Latin Hypercube Designs with Sliced Structures

Transcription

1 Communications in Statistics - Theory and Methods ISSN: (Print) X (Online) Journal homepage: Nested Latin Hypercube Designs with Sliced Structures Hao Chen & Min-Qian Liu To cite this article: Hao Chen & Min-Qian Liu (2015) Nested Latin Hypercube Designs with Sliced Structures, Communications in Statistics - Theory and Methods, 44:22, , DOI: / To link to this article: Accepted author version posted online: 27 Jun Published online: 27 Jun Submit your article to this journal Article views: 35 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at Download by: [Min-Qian Liu] Date: 25 November 2015, At: 09:14

2 Communications in Statistics Theory and Methods, 44: , 2015 Copyright Taylor & Francis Group, LLC ISSN: print / X online DOI: / Nested Latin Hypercube Designs with Sliced Structures HAO CHEN AND MIN-QIAN LIU LPMC and Institute of Statistics, Nankai University, Tianjin, China 1. Introduction In computer experiments, space-filling designs with a sliced structure or nested structure have received much recent interest and been studied separately. However, it is likely that designs with both structures are needed in some situations, but there are no suitable designs so far. In this paper, we construct a special class of nested Latin hypercube designs with sliced structures, in such a design, a small sliced Latin hypercube design is nested within a large one. The construction method is easy to implement and the number of factors is flexible. Numerical simulations show the usefulness of the newly proposed designs. Keywords Computer experiment; Nested Latin hypercube design; Sliced Latin hypercube design; Sliced permutation; Space-filling design. Mathematics Subject Classification Primary 62K15; Secondary 62K20. Since many physical experiments in modern business, engineering, and sciences are so expensive, time-consuming, and even cannot be executed, the corresponding computer experiments are much more interesting and becoming more and more important. For example, days or weeks will be taken to run the fluidized bed process in the food industry one time (Dewettinck et al., 1999), while only several minutes will be enough for a run of the corresponding computer experiment. Now let us review some designs which have been developed for computer experiments. To reduce the variance of the estimate of total mean, space-filling designs were proposed, including Latin hypercube designs (LHDs), uniform designs, and so on (Santner et al., 2003; Fang et al., 2005). An n q matrix is called an LHD, denoted by L(n, q), if each of its columns includes n uniformly spaced levels. Such a design achieves the maximum stratification when projected onto any univariate dimension. Recently, computer experiments with multiple levels of accuracy have been considered. As stated in Qian et al. (2009b), a large computer program can be run at different levels Received November 3, 2012; Accepted March 8, Address correspondence to Min-Qian Liu, LPMC and Institute of Statistics, Nankai University, Tianjin , China; mqliu@nankai.edu.cn Color versions of one or more of the figures in the article can be found online at

3 4722 Chen and Liu of sophistication with vastly varying computational time, so multiple experiments with various levels of fidelity have become prevalent in practice. The study of this kind of issues involves two aspects: data modeling (Kennedy and O Hagan, 2000; Reese et al., 2004; Qian et al., 2006; Qian and Wu, 2008) and experimental planning (Qian et al., 2009a,b; Qian, 2009; Qian and Ai, 2010; Haaland and Qian, 2010; He and Qian, 2011; Sun et al., 2013, 2014; Yang et al., 2014). Nested space-filling designs were first proposed by Qian et al. (2009b) for such computer experiments. Nested Latin hypercube designs (NLHDs, Qian, 2009) are special nested space-filling designs. Without loss of generality, consider a computer experiment with two levels of accuracy, the low-accuracy experiment (LE) and high-accuracy experiment (HE). The LE is fast but inaccurate, while the HE is slow but accurate. Designs for LE and HE are denoted by D l and D h, respectively. This pair of matrices (D h,d l ) is called an NLHD, denoted by NL(n, m, q), if D h D l while D l and D h (after level collapsing sometimes) are both LHDs, where n and m are the numbers of runs of D l and D h, respectively, and q is the number of their columns. For simplicity, we only use D l to denote an NLHD in some cases when no confusion is caused. In addition, computer experiments with qualitative and quantitative factors have also been considered. As claimed in Qian and Wu (2009), the early work of computer experiments assumes that the input factors are all quantitative (Santner et al., 2003; Fang et al., 2005). In fact, some factors can be qualitative. For example, a data center computer experiment involves qualitative factors like diffuser height and hot-air return-event location (Schmidt et al., 2005). The study of this kind of computer experiments also involves two aspects: data modeling (Qian et al., 2008; Han et al., 2009) and experimental planning (Qian and Wu, 2009; Qian, 2012; Huang et al., 2014; Yin et al., 2014; Yang et al., 2014; Cao and Liu, 2015). Particularly, sliced Latin hypercube designs (SLHDs) were proposed by Qian (2012) for such computer experiments. An n q matrix S is called an SLHD, denoted by SL(n, t, q), if S is an L(n, q) and it can be partitioned into slices S i for i = 1,...,t, each of which is an L(m, q) with m = n/t (sometimes level collapsing is needed). Note that there exist some situations where both the LE and HE have qualitative and quantitative factors. However, neither NLHDs nor SLHDs are suitable for such situations. The purpose of this paper is to construct a new kind of NLHDs, which include slices in both D l and D h, to solve the problem reasonably. In fact, such NLHDs are also SLHDs, and we call them the nested sliced Latin hypercube designs (NSLHDs, for simplicity). A pair of matrices (D h,d l ) is called an NSLHD, denoted by NSL(n, m, s, q), if D h D l while D l is an SL(n, st, q) and D h is an SL(m, s, q) (sometimes level collapsing is needed), where n and m are the numbers of runs of D l and D h, respectively, q is the number of their columns, and t = n/m, s m. Similar to the NLHD, we only use D l to denote an NSLHD sometimes when no confusion is caused. In this paper, the D h in any NLHD and slices in any SLHD need level collapsing to be LHDs. The paper is organized as follows. In Sec. 2, we propose a construction of NSLHDs. Section 3 studies the relationships among NLHDs, SLHDs, and NSLHDs. In Sec. 4, some simulations are provided to illustrate the usefulness and advantages of the newly constructed designs. Some concluding remarks are presented in Sec Construction First, we introduce some definitions and notation. For a vector V of length n, letv (i) be the ith element of V, i = 1,...,n. For any real number r, r denotes the smallest integer greater than or equal to r, and for a real vector or matrix M, M is similarly defined. Denote

4 Nested Latin Hypercube Designs 4723 by A\B the set of elements which belong to A but not B. For a design D = (d ij ) with n runs and q factors, let d i be the ith run and d j be the jth factor, for i = 1,...,n, j = 1,...,q. Assume the levels of each factor of D are a permutation on Z n ={1,...,n}, then we can obtain samples from D as follows: X ij = d ij e ij,i= 1,...,n, j = 1,...,q, (1) n where e ij is a random number from uniform distribution U[0, 1]. A permutation on Z n is a rearrangement of 1,...,n, and all n! rearrangements are equally probable. A nested permutation on Z n is a special permutation on Z n, denoted by π np = (π np (1),...,π np (n)), where the first m elements of π np is a permutation on Z m after the level collapsing operation π np (i)/t with n = mt and i = 1,...m (Qian, 2009). A sliced permutation of t slices on Z n, denoted by π sp, is also a special permutation on Z n, which can be divided into t groups and each group is a permutation on Z m under the same operation as for the nested permutation. Now, let us present the construction of NSLHDs. Here, without loss of generality, we only consider designs for computer experiments with two levels of accuracy, and for the cases of three or more levels of accuracy, the construction can be generalized naturally. Suppose we want to construct an NSL(n, m, s, q). The construction can be carried out as follows. Algorithm 1. Step 1. Divide the n elements of Z n into m blocks, b 1,...,b m, where b i ={z Z n z/t =i}, i= 1,...,m, t = n/m. Set u = 1, v= 1. Step 2. For i = 1,...,m, draw one element e i randomly from b i, and the left elements of b i constitute a new set, still denote it by b i.letb u = (e 1,...,e m ). Step 3. Update u = u + 1. If u t, goto Step 2; otherwise, we get t columns B 1,...,B t. Step 4. Construct t sliced permutations of s slices on Z m, πsp k for k = 1,...,t. Step 5. Compare the elements of B k /t with the elements of πsp k for k = 1,...,t. If B k (i)/t =πsp k (j), then replace π sp k (j) with B k(i), i = 1,...,m, j = 1,...,m, k = 1,...,t. Denote the new vector generated from πsp k by H k,k= 1,...,t. Step 6. Stacking H 1,...,H t run by run gives a column g v = (H 1,...,H t ). Step 7. Update v = v + 1. If v q, setu = 1, goto Step 2; otherwise, we get q columns g 1,...,g q.letg = (g 1,...,g q ). It can be easily verified that the matrix G generated by Algorithm 1 is an NSL(n, m, s, q), in which an m q SLHD with s slices is nested within an n q SLHD with ts slices. These properties are stated in the following theorem with the proof being omitted. Theorem 1. For G constructed by Algorithm 1, let G 1 be the submatrix of G consisting of the first m rows. Then (i) G 1 is an SL(m, s, q), G is an SL(n, ts, q);

5 4724 Chen and Liu (ii) (G 1,G) is an NL(n, m, q), thus (G 1,G) is an NSLHD(n, m, s, q). An example is given to illustrate the above construction. Example 1 (Construction of an NSL(12, 6, 2, 2)). Let n = 12, m = 6, t = 2, s = 2, and q = 2. Following Step 1, divide Z 12 into six blocks: b 1 ={1, 2}, b 2 = {3, 4}, b 3 ={5, 6}, b 4 ={7, 8}, b 5 ={9, 10}, and b 6 ={11, 12}. From Steps 2 and 3, suppose we get B 1 = (1, 4, 5, 8, 10, 11) and then B 2 = (2, 3, 6, 7, 9, 12). By Step 4, suppose we obtain two sliced permutations S 1 = (1, 4, 6, 2, 3, 5) and S 2 = (1, 4, 5, 6, 2, 3). Obviously, B 1 (1)/2 =S 1 (1), B 1 (2)/2 =S 1 (4), B 1 (3)/2 =S 1 (5), B 1 (4)/2 =S 1 (2), B 1 (5)/2 =S 1 (6), and B 1 (6)/2 =S 1 (3). So H 1 (1) = B 1 (1) = 1, H 1 (4) = B 1 (2) = 4, H 1 (5) = B 1 (3) = 5, H 1 (2) = B 1 (4) = 8, H 1 (6) = B 1 (5) = 10, and H 1 (3) = B 1 (6) = 11, then H 1 = (1, 8, 11, 4, 5, 10). Similarly, we get H 2 = (2, 7, 9, 12, 3, 6). Then the first column of G is obtained, i.e., g 1 = (H 1,H 2 ) = (1, 8, 11, 4, 5, 10, 2, 7, 9, 12, 3, 6). In the same way, we can obtain the other column of G, say, g 2 = (8, 2, 11, 3, 6, 9, 5, 12, 1, 4, 7, 10). Finally, we get a design G = Make a level collapsing operation G/2, wehave G/2 = So, if we take G 1 to be the first six rows of G, we know that (G 1,G)isanNL(12, 6, 2). Meanwhile, if we make an operation G/4, then G/4 = We can see that G is an SL(12, 4, 2) and G 1 is an SL(6, 2, 2). So the small SLHD G 1 is nested within the large SLHD G, and (G 1,G)isanNSL(12, 6, 2, 2). 3. Relationships among NLHDs, SLHDs, and NSLHDs We now demonstrate the relationships among the NLHDs (Qian, 2009), SLHDs (Qian, 2012), and designs constructed in this paper. First, we can easily have Corollary 1. (i) An NL(n, n/2,q) is an SL(n, 2,q), while an NL(n, n/t, q) may not be an SL(n, t, q) when t>2; (ii) An SL(n, s, q) is an NL(n, n/s, q); (iii) When s 1, t= n/m > 1, annsl(n, m, s, q) is an NL(n, m, q), and when t = n/m 1, s > 1, an NSL(n, m, s, q) is an SL(n, ts, q); (iv) In an NSL(n, m, s, q), there exists a small SLHD SL(m, s, q) that is nested within a large SLHD SL(n, st, q). Whereas, such a feature may not hold for either an NL(n, m, q) or an SL(n, s, q).

6 Nested Latin Hypercube Designs 4725 Specially, let us compare an NL(27, 9, 2), an SL(27, 3, 2), and an NSL(27, 9, 3, 2) in the following example for illustration. Example 2. Let n = 27, m= 9, t= 3, s= 3, and q = 2. First, we obtain an NL(27, 9, 2) following the construction in Qian (2009), an SL(27, 3, 2) following the construction in Qian (2012), and an NSL(27, 9, 3, 2) following Algorithm 1, denoted by N, S, and G, respectively, where N = ( ) , ( ) S =, ( ) G = After the operations N/3, S/3, and G/3, respectively, we get N/3 =, S/3 =, G/3 = And after N/9, S/9, and G/9, respectively, we have N/9 =, S/9 =, G/9 = From N/3, it can be seen that N is an NL(27, 9, 2), but it is not an SLHD. While S/3 indicates that S is also an NL(27, 9, 2). Through G/3, we can get the conclusion that G is not only an NL(27, 9, 2), but also an SL(27, 3, 2). In addition, G/9 implies that an SL(9, 3, 2) is nested within an SL(27, 3, 2), however, neither N nor S have this property by looking at N/9 or S/9. In fact, G is also an SL(27, 9, 2). The structures of these three kinds of designs can also be viewed intuitively by the scatter plots of N, S, and G in Fig. 1. In Fig. 1, the symbols, +,,,,,,, and represent the runs 1 3, 4 6, 7 9, 10 12, 13 15, 16 18, 19 21, 22 24, and of N, S, and G, respectively. Obviously, we can see that any of the three designs has one-dimensional uniformity when projected onto any one dimension, but only in Fig. 1(c), any three identical symbols appear in different rows and different columns when considering the 3 3 grids.

7 4726 Chen and Liu 4. Simulations Figure 1. (a) Scatter plot of N; (b) scatter plot of S; and (c) scatter plot of G. In this section, we provide some numerical illustrations, which consider computer experiments not only with HE and LE, but also with quantitative and qualitative factors. In the following two examples, we assume the real models are known so that the real data are available, and we use the Matlab toolbox DACE (Lophaven et al., 2002) to fit Gaussian process (GP) meta-models. Now, let us briefly introduce the GP model with quantitative factors (Qian et al., 2006). Suppose that n vectors of input variable values for q covariates, denoted by D = (d ij ) = (d 1,...,d n ), are involved in a computer experiment, and the corresponding response values of which is y = (y(d 1 ),...,y(d n )). Then the data consist of an n q design matrix and n 1 response vector. The GP model has the following structure: y(d i ) = f (d i ) β + ε(d i ),i= 1,...,n, where f (d) = (f 1 (d),...,f p (d)) is a vector of pre-specified regression functions and β = (β 1,...,β p ) is a vector of unknown coefficients. The residual ε(d) isassumedtobe

8 a stationary GP with covariance: Nested Latin Hypercube Designs 4727 cov(ε(d i ),ε(d j )) = σ 2 R(d i,d j ), where R(d i,d j ) is the Gauss correlation function, whose popular form is the following product exponential correlation function (Santner et al., 2003) and will be used in this paper: { R(d i,d j ) = exp q θ h d ih d jh 2}, h=1 where θ = (θ 1,...,θ q ) is a vector of scale parameters. When quantitative and qualitative factors exist simultaneously in a computer experiment, Qian et al. (2008) proposed an integrated analysis that assumes a single GP model across different values of quantitative and qualitative factors. Since the main purpose of this paper is to present a new kind of designs, so we do not focus on the modeling skill, and still use the independent analysis, which is simple and enough to compare the three kinds of designs (NLHDs, SLHDs, and NSLHDs) though inferior than integrated analysis (Qian et al., 2008). In the independent analysis, distinct GP models are used to model the data, which are collected at different level combinations of the qualitative factors. Our simulations are carried out according to the following algorithm. Algorithm 2 Step 1. Set parameters n, m, s, t, q for generating designs, theta0, lob, upb, the regression functions and the correlation function for the DACE, and provide the real response functions. Here, theta0 is the initial guess for θ, lob, and upb are the bounds for θ. Step 2. Generate designs, get samples through (1), and obtain training data using the real response functions. Step 3. Build a base surrogate GP model using the data collected from LE. Step 4. Fit a GP model, named the difference GP model, using the differences of the responses from common runs of HE and LE. Here, for a common run, the difference of the responses is set to be the difference obtained by subtracting the response of LE from the response of HE. Step 5. Adjust the base surrogate model by adding the difference GP model to it and obtain an accurate GP model. Step 6. Predict the responses using the accurate model at the untried points for HE (in an NLHD (D h,d l ), the runs of D l \D h are called the untried points for HE) and random points in the experimental region, respectively. Step 7. Compute the root mean squared errors (RMSEs) at the two kinds of points in Step 6, respectively. Step 8. Repeat Steps 2 7 a certain number of times and present the boxplots of the RMSEs. For comparison, we do the same work for NLHDs, SLHDs, and NSLHDs. As usually assumed for computer experiments, no errors are considered for the outputs in this paper. First, let us see an example involving one qualitative factor, z 1, and one quantitative factor, x 1.

9 4728 Chen and Liu Example 3 (One quantitative factor and one qualitative factor). Without loss of generality, we use NSL(12, 6, 2, 1) s, NL(12, 6, 1) s, and SL(12, 4, 1) s and get the training data using the real response functions. For the qualitative factor, z 1, its levels are taken to be (0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1). The design for HE of an SLHD is assumed to have two slices so that the run number of the design for HE is equivalent to those of an NSLHD and an NLHD. There are two different levels of accuracy, and the real response functions of the LE and HE are assumed to be { ( x1 ) cos(7πx l(x 1,z 1 ) = 1 /2), if z 1 = 0, (1 + 3x 1 ) cos(7πx 1 /2), if z 1 = 1, { exp{1.4x1 } cos(7πx h(x 1,z 1 ) = 1 /2), if z 1 = 0, exp{3x 1 } cos(7πx 1 /2), if z 1 = 1, and respectively, where h(x 1,z 1 ) is taken from Qian et al. (2008). In the DACE, we assume the parameters theta0 = 10, lob = 0.01, itupb = 10, and use the Gauss correlation function corrgauss and the second order polynomial regression function regpoly2 for building the surrogate GP model (in Step 3 of Algorithm 2), and use the corrgauss and the first order polynomial regression function regpoly1 for building the difference GP model (in Step 4 of Algorithm 2). After Steps 2 6, we compute the RMSEs at the untried points for HE and 100 random points in the experimental region {0, 1} [0, 1]. For the quantitative factor, x 1, we sample its 100 values from [0, 1] randomly; for the qualitative factor, z 1, its values are taken to be (0, 1, 0, 1 ), where 0 is a zero vector of length 25, and 1 is defined similarly. After repeating 1000 times, the boxplots of RMSEs at the two different kinds of predicting points are given in Figs. 2 and 3, respectively. From Figs. 2 and 3, we can see that the NSLHDs are a little superior than the NLHDs and SLHDs in RMSEs at both the 100 random points in {0, 1} [0, 1] and the untried points for HE. We have also computed the mean and standard deviation values of the 1000 RMSEs, which are listed in Table 1. It can be seen that the NSLHDs have the smallest values of mean and standard deviation of the RMSEs at both kinds of predicting points, Figure 2. Boxplots of RMSEs at 100 random points in {0, 1} [0, 1] in Example 3.

10 Nested Latin Hypercube Designs 4729 Figure 3. Boxplots of RMSEs at the untried points for HE in Example 3. which also indicate that the NSLHDs have the best performance among the three kinds of designs. Remark 1. We should notice that the run number is so limited that the structures of the NLHDs, SLHDs, and NSLHDs can be very close to each other. Among the 1000 times of repeat, there must be a certain number of times when the NLHDs and SLHDs happen to be NSLHDs. This may be the reason why the results have no obvious differences. The slight superiority of NSLHDs over NLHDs may be explained from the imposing of the sliced structures on both the D h and D l for an NSLHD, and the superiority over SLHDs may be explained from the uniformity. We have computed the mean values of the centered L 2 -discrepancies (Hickernell, 1998) for the 1000 NL(12, 6, 1) s, SL(12, 4, 1) s, and NSL(12, 6, 2, 1) s, respectively, and find that they are all However, when we just consider the D h, i.e., the first six rows of each design for HE, the mean values of the centered L 2 -discrepancies are , , and for the NLHDs, SLHDs, and NSLHDs, respectively. Theoretically, the D h in an NSLHD or NLHD is an LHD, but that in an SLHD may not be. Therefore, the points of the D h in an NSLHD or NLHD are usually more uniform than those of the D h in an SLHD. With the sliced structures of the D h s and Table 1 Mean and deviation standard values of RMSEs in Example 3 At 100 random points At the untried points Mean Standard deviation Mean Standard deviation NLHDs SLHDs NSLHDs

11 4730 Chen and Liu D l s and better uniformity of the D h s, it is anticipated that the NSLHDs have a better performance in model adjustment and prediction in general. Next, we provide another example to compare the three kinds of designs involving one qualitative factor, z 1, and two quantitative factors, x 1 and x 2. Example 4 (Two quantitative factors and one qualitative factor). We consider NL(24, 12, 2) s, SL(24, 4, 2) s, and NSL(24, 12, 2, 2) s and assign the levels of z 1 as (0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1). Assume the real response functions of the LE and HE are { ( (x1 x l(x 1,x 2,z 1 ) = 2 )) cos(7πx 2 /2) + sin(5π(x 2 x 1 )), if z 1 = 0, and (1 + 3(x 1 x 2 )) cos(5πx 2 ) + sin(5π(x 2 x 1 )), if z 1 = 1, { exp{1.4(x1 x h(x 1,x 2,z 1 ) = 2 )} cos(7πx 2 /2) + sin(10π(x 2 x 1 ) 2 ), if z 1 = 0, exp{3(x 1 x 2 )} cos(5πx 2 ) + sin(10π(x 2 x 1 ) 2 ), if z 1 = 1, respectively. In the DACE, we assume the values of the parameters theta0 = [5, 5], lob = [0.01, 0.01], upb = [10, 10], and use the Gauss correlation function corrgauss, the second order polynomial regression function regpoly2 and the first order polynomial regression function regpoly1 for building the GP models as we do in Example 3. In Step 6 of Algorithm 2, 100 random points are sampled from experimental region {0, 1} [0, 1] 2, where the two quantitative factors are randomly sampled from [0, 1] 2 and the qualitative factor is arranged to be (0, 1, 0, 1 ), where 0 and 1 are as defined in Example 3. After repeating 1000 times, the boxplots of RMSEs at the untried points for HE and at 100 random points are shown in Figs. 4 and 5, respectively. From Figs. 4 and 5, we can see that the RMSEs of the NSLHDs are slightly smaller than those of the SLHDs and NLHDs, which coincide with the conclusions of Examples 3. The mean and standard deviation values of the 1000 RMSEs are listed in Table 2, which also imply that the models obtained by using the NSLHDs achieve the best prediction Figure 4. Boxplots of RMSEs at the untried points for HE in Example 4.

12 Nested Latin Hypercube Designs 4731 Figure 5. Boxplots of RMSEs at 100 random points in Example 4. Table 2 Mean and deviation standard values of RMSEs in Example 4 At 100 random points At the untried points Mean Standard deviation Mean Standard deviation NLHDs SLHDs NSLHDs performance, and the NSLHD may be a better choice for modeling in some situations where both the HE and LE have quantitative and qualitative factors. 5. Concluding Remarks In this paper, we combine the advantages of the SLHDs and NLHDs and propose a construction for a special class of LHDs, called NSLHDs, each of which is a large SLHD containing a small SLHD. The applications of NSLHDs due to their special structures are explored through two illustrative examples, i.e., Examples 3 and 4. In the examples, we compare the prediction performance through the RMSEs at the untried points for HE and at random points in the experimental region, respectively. The simulation results indicate that the NSL- HDs usually perform better than the SLHDs and NLHDs in situations where both quantitative and qualitative factors exist for computer experiments with multiple levels of accuracy. Acknowledgments The authors thank the reviewers for their constructive comments on an early version of this paper.

13 4732 Chen and Liu Funding This work was supported by the National Natural Science Foundation of China (Grant Nos and ), the 131 Talents Program of Tianjin, and the Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant No ). References Cao, R. Y., Liu, M. Q. (2015). Construction of second-order orthogonal sliced Latin hypercube designs. J. Complexity 31: Dewettinck, K., Visscher, A. D., Deroo, L., Huyghbbaet, A. (1999). Modeling the steady-state thermodynamic operation point of top-spray fluidized-bed processing. J. Food Eng. 39: Fang, K. T., Li, R., Sudjianto, A. (2005). Design and Modeling for Computer Experiments. New York: CRC Press. Haaland, B., Qian, P. Z. G. (2010). An approach to constructing nested space-filling designs for multi-fidelity computer experiments. Stat. Sinica 20: Han, G., Santner, T. J., Notz, W. I., Bartel, D. L. (2009). Prediction for computer experiments having quantitative and qualitative input variables. Technometrics 51: He, X., Qian, P. Z. G. (2011). Nested orthogonal array-based Latin hypercube designs. Biometrika 98: Hickernell, F. J. (1998). A generalized discrepancy and quadrature error bound. Math. Comput. 67: Huang, H. Z., Yang, J. F., Liu, M. Q. (2014). Construction of sliced (nearly) orthogonal Latin hypercube designs. J. Complexity 30: Kennedy, M. C., O Hagan, A. (2000). Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1 13. Lophaven, S. N., Nielsen, H.B., Sondergaard, J. (2002). A Matlab kriging toolbox DACE. Version 2.5. Available at Qian, P. Z. G. (2009). Nested Latin hypercube designs. Biometrika 96: Qian, P. Z. G. (2012). Sliced Latin hypercube designs. J. Am. Stat. Assoc. 107: Qian, P. Z. G., Ai, M. Y. (2010). Nested Lattice sampling: a new sampling scheme derived by randomizing nested orthogonal arrays. J. Am. Stat. Assoc. 105: Qian, P. Z. G., Ai, M. Y., Wu, C. F. J. (2009a). Construction of nested space-filling designs. Ann. Stat. 37: Qian, P. Z. G., Seepersad, C., Joseph, R., Allen, J., Wu, C. F. J. (2006). Building surrogate models based on detailed and approximate simulations. ASME Trans. J. Mech. Design 128: Qian, P. Z. G., Tang, B., Wu, C. F. J. (2009b). Nested space-filling designs for computer experiments with two levels of accuracy. Stat. Sinica 19: Qian, P. Z. G., Wu, C. F. J. (2008). Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50: Qian, P. Z. G., Wu, C. F. J. (2009). Sliced space-filling designs. Biometrika 96: Qian, P. Z. G., Wu, H. Q., Wu, C. F. J. (2008). Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics 50: Reese, C. S., Wilson, A. G., Hamada, M., Martz, H. F., Ryan, K. J. (2004). Integrated analysis of computer and physical experiments. Technometrics 46: Santner, T. J., Williams, B. J., Notz, W. I. (2003). The Design and Analysis of Computer Experiments. New York: Springer. Schmidt, R. R., Cruz, E. E., Iyengar, M. K. (2005). Challenges of data center thermal management. IBM J. Res. Develop. 49: Sun, F. S., Liu, M. Q., Qian, P. Z. G. (2014). On the construction of nested space-filling designs. Ann. Stat. 42:

14 Nested Latin Hypercube Designs 4733 Sun, F. S., Yin, Y. H., Liu, M. Q. (2013). Construction of nested space-filling designs with two levels of accuracy using difference matrices. J. Stat. Plan. Inf. 143: Yang, J. Y., Liu, M.Q., Lin, D. K. J. (2014). Construction of nested orthogonal Latin hypercube designs. Stat. Sinica 24: Yang, X., Chen, H., Liu, M. Q. (2014). Resolvable orthogonal array-based uniform sliced Latin hypercube designs. Statist. Probab. Lett. 93: Yin, Y. H., Lin, D. K. J., Liu, M. Q. (2014). Sliced Latin hypercube designs via orthogonal arrays. J. Stat. Plan. Inf. 149: