e-companion ONLY AVAILABLE IN ELECTRONIC FORM Electronic Companion Stochastic Kriging for Simulation Metamodeling

Transcription

1 OPERATIONS RESEARCH doi /opre ec e-companion ONLY AVAILABLE IN ELECTRONIC FORM informs 009 INFORMS Electronic Companion Stochastic Kriging for Simulation Metamodeling by Bruce Ankenman, Barry L. Nelson, and Jeremy Staum, Operations Research, doi /opre

2 ec Proofs EC.1. MSE-optimal Stochastic Kriging Predictor Here we prove the results in Section for stochastic kriging when Σ M, Σ ε and β 0 are known. Most of these results parallel similar derivations in Stein (1999 for standard kriging, to which the reader can refer for missing details. We consider linear predictors of the form λ 0 (x 0 +λ(x 0 Ȳ where λ 0 (x 0 and λ(x 0 are weights that depend on x 0 and will be chosen to give minimum MSE for predicting Y(x 0 =β 0 + M(x 0. To simplify the notation we drop the dependence of λ 0 and λ on x 0. The MSE of the linear predictor is MSE = E (Y(x 0 λ 0 λ Ȳ ] =E (β 0 λ 0 λ 1 k β 0 ] +E (M(x 0 λ Ȳ + λ 1 k β 0 ] +E (β 0 λ 0 λ 1 k β 0 (M(x 0 λ Ȳ + λ 1 k β 0 ] =(β 0 λ 0 λ 1 k β 0 +E (M(x 0 λ Ȳ + λ 1 k β 0 ] =(β 0 λ 0 λ 1 k β 0 +Σ M (x 0, x 0 +λ Σ M + Σ ε ] λ λ T Σ M (x 0,. Now following the same steps as Stein (1999, Chapter 1., we can show that the MSE-optimal weights are λ 0 = β 0 λ 1 k β 0 and λ =Σ M + Σ ε ] 1 Σ M (x 0,. With these weights, the estimator is Ŷ(x 0 =β 0 + Σ M (x 0, Σ M + Σ ε ] 1 ( Ȳ β 0 1 k. By direct substitution the MSE is MSE =Σ M (x 0, x 0 Σ M (x 0, Σ M + Σ ε ] 1 Σ M (x 0, =Σ M (x 0, x 0 Σ M (x 0, Σ 1 M Σ M(x 0, +Σ M (x 0, ΞΣ M (x 0,. (EC.1 The final step follows from Henderson and Searle (1981, which shows that Ξ will be a positive definite matrix that depends on Σ ε and Σ M and will disappear if Σ ε = 0.

3 ec3 EC.. Impact of CRN for k> Design Points As a second illustration of the impact of CRN on stochastic kriging, suppose that 1 ρ ρ Σ M + Σ ε = τ ρ 1 ρ I + V ρρ 1 This matrix is invertible in closed form using Theorem in Graybill (1969 (see also Appendix EC.5. Therefore, the MSE of the stochastic kriging estimator (everything known can be obtained by plugging Σ M, Σ ε and Σ M (x 0, =τ (r 0,r 0,...,r 0 into (7 to obtain which is increasing in ρ. EC.3. Joint Distribution MSE = τ (1 kr0τ (1 + (k 1ρV + τ The multivariate normality of (Y(x 0, Ȳ(x 1,...,Ȳ(x k follows directly from the response surface model (3 and Assumption 1. They imply that CovY j (x i, Y l (x h ] = τ + V(x i, i= h, j = l CovM(x i +ε j (x i, M(x h +ε l (x h ] = τ, i= h, j l τ R M (x i x j ; θ i h and similarly CovY(x 0, Y j (x i ] = CovM(x 0, M(x i +ε j (x i ] = τ R M (x 0 x i ; θ. Since the ε j (x i are independent across replications and design points, averaging the n i replications at design point x i only affects CovȲ(x i, Ȳ(x i ] = VarȲ(x i ] = τ + V(x i /n i. Therefore, Y(x 0 ( ] Ȳ(x 1. MVN τ R M (x 0, ; θ β 0 1 k+1,. (EC. R M (x 0, ; θ τ R M (θ + Diag{V(x i /n i } Ȳ(x k The conditional expectation of Y(x 0 given (Ȳ(x 1,...,Ȳ(x k then follows immediately from standard results for the multivariate normal distribution (e.g., Stein 1999, Appendix A.

4 ec4 EC.4. Proof of Theorem 1 We first show that for any fixed positive definite covariance matrix Σ ε, the predictor Ŷ (x 0 =β 0 + Σ M (x 0, Σ M + Σ ε] 1 ( Ȳ β 0 1 k is an unbiased predictor. This follows immediately from EŶ (x 0 Y(x 0 ] = β 0 +0 β 0 =0. Next notice that under Model (3 we have S (x i = 1 n i 1 n i j=1 ( Yj (x i Ȳ(x i = 1 n i 1 n i j=1 (ε j (x i ε j (x i. Therefore, under Assumption 1, V(x i =S (x i is independent of Ȳ(x i =β 0 + M(x i + ε j (x i by well-known properties of the normal distribution (recall that M is also independent of ε j (x i. Then since Σ ε = Diag { V(x1 /n 1, V(x } /n,..., V(x k /n k we have ] ] E Ŷ(x 0 Y(x 0 =E E (Ŷ(x 0 Y(x 0 Σ ε =Eβ 0 β 0 ]=0. EC.5. Variance Inflation Example of Section 3.1 As in 3.1, suppose that 1 r r Σ M = τ r 1 r......, rr 1 Σ M (x 0, =τ (r 0,r 0,...,r 0 with r 0,r 0, Σ ε =(V/nI, and we have an independent estimator V Vχ n 1/(n 1. Let γ = V/τ be the ratio of the intrinsic variance to the extrinsic variance. A key to the result is noting that we can write Σ M + Σ ε = qq T + τ Diag{1 r, 1 r,...,1 r} + V n I where q T =( r, r,..., r. This matrix is invertible in closed form using Theorem of Graybill (1969. Specialized to this case { } Σ M + Σ ε ] 1 1 = Diag τ (1 r+v/n,..., 1 τ (1 r+v/n τ r τ (1 r+kτ r + V/n τ (1 r+v/n (EC.3

5 ec5 where indicates a matrix of appropriate dimension with all elements the same. We obtain the MSE of Ŷ(x 0, the stochastic kriging estimator with Σ ε (i.e., V known, MSE = τ (1 kr0 1+(k 1r + γ n by plugging Σ M, Σ ε and Σ M (x 0, into (7 and using the known inverse (EC.3. The MSE of Ŷ(x 0 is derived by substituting V for V and noting that since Ŷ(x 0 is unbiased for any V independent of Ȳ (see Theorem 1 and the proof in Section EC.4 ] ] MSE Ŷ(x 0 =VarŶ(x 0 ] ] =E Var (Ŷ(x 0 V +Var E (Ŷ(x 0 V ] =E Var (Ŷ(x 0 V =E Var β 0 + τ (r 0,r 0,...,r 0 Σ M + V ] 1 (Ȳ n I β0 1 k V ( = τ 1+(k 1r + γ E 1+ n kr 0 kr0 ( (. 1+(k 1r + γ V 1+(k 1r + γ V n V n V The last step requires writing out the conditional variance, recalling that Ȳ and V are independent, and employing several tedious applications of Theorem of Graybill (1969. EC.6. Maximum Likelihood Estimators The log likelihood function for (β 0,τ, θ given (Ȳ(x 1,...,Ȳ(x k follows directly from the joint normal distribution (EC.: l(β 0,τ, θ= ln (π k/] 1 ln τ R M (θ+σ ε ] 1 (Ȳ β0 ] 1 k τ 1 (Ȳ β0 R M (θ+σ ε 1 k. For brevity, let Σ = τ R M (θ+σ ε. Taking derivatives, and applying the matrix calculus results ( and (3 gives l(β 0,τ, θ β 0 = β 0 1 T k Σ 1 1 k + 1 T k Σ 1 Ȳ = 1 T k Σ 1 (Ȳ β 0 1 k = 0 (EC.4

6 ec6 l(β 0,τ, θ τ = 1 trace Σ 1 R M (θ ] l(β 0,τ, θ θ p 1 (Ȳ { β0 1 k Σ 1 R M (θσ 1}( Ȳ β 0 1 k = 0 (EC.5 = 1 ] trace Σ 1 τ R M (θ θ p 1 } (Ȳ β0 1 k { Σ 1 τ R M (θ (Ȳ β0 Σ 1 1 k = 0 (EC.6 θ p where θ p is the pth element of θ. Observe that β 0 that solves (EC.4 is explicit given Σ. To obtain starting solutions to search for the MLEs we set R M (θ =I and simplify (EC.4 and (EC.5. EC.7. Estimated MSE To obtain the MSE estimator (5, we first assume Σ M = τ R M (θ and Σ ε are known, so that only β 0 need be estimated. An expression for MSE(x 0 then follows precisely the development in Stein (1999, 1.5 with τ R M (θ+σ ε playing the role of K, the covariance matrix of the random field. Our estimator MSE(x 0 is then obtained by plugging in τ R M ( θ and Σ ε obtained via maximum likelihood estimation. EC.8. Derivation of IMSE-Optimal Design The first step in deriving the IMSE-optimal allocation of replications is to note that MSE(x 0 ; n =Σ M (x 0, x 0 Σ M (x 0, Σ(n 1 Σ M (x 0, = τ τ 4 r(x 0 Σ(n 1 r(x 0 k = τ τ ] 4 Σ(n 1 r i,j i(x 0 r j (x 0 i,j=1 where τ r(x 0 =(Σ M (x 0, x 1, Σ M (x 0, x,...,σ M (x 0, x k. Therefore, k IMSE(n =τ τ ] 4 Σ(n 1 i,j i,j=1 = τ τ 4 1 W Σ(n 1] 1 x 0 X r i (x 0 r j (x 0 dx 0 where is the Hadamard product of matrices. From this we can form the Lagrangian L(n=τ τ 4 1 W Σ(n 1] + λ(n 1 n.

7 ec7 The first-order optimality conditions are L(n n i = τ 4 1 W ] Σ(n 1 1 λ =0 n i for i = 1,,...,k. Careful application of matrix calculus gives ] Σ(n 1 = Σ(n 1 Σ(n Σ(n 1 n i n i Σ(n = V(x i J (ii n i n i where J (ii is a k k matrix with 1 in position (i, i and zeroes elsewhere. The result then follows by solving the first-order conditions. References Graybill, F. A Matrices with Applications in Statistics, nd edition, Wadsworth, Belmont, CA. Henderson, H. V. and S. R. Searle On deriving the inverse of a sum of matrices. SIAM Review 3, Stein, M. L Interpolation of Spatial Data: Some Theory for Kriging. Springer, NY.