Package GPfit. R topics documented: April 2, Title Gaussian Processes Modeling Version Date

Transcription

1 Title Gaussian Processes Modeling Version Date Package GPfit April 2, 2015 Author Blake MacDoanld, Hugh Chipman, Pritam Ranjan Maintainer Hugh Chipman Description A computationally stable approach of fitting a Gaussian Process (GP) model to a deterministic simulator. Imports lhs (>= 0.5), lattice (>= ) License GPL-2 NeedsCompilation no Repository CRAN Date/Publication :44:08 R topics documented: GPfit-package corr_matrix GP_deviance GP_fit plot.gp predict.gp print.gp summary.gp Index 17 1

2 2 corr_matrix GPfit-package Gaussian Process Modeling Description Details A computationally stable approach of fitting a Gaussian process (GP) model to simulator outputs. It is assumed that the input variables are continuous and the outputs are obtained from scalar valued deterministic computer simulator. This package implements a slightly modified version of the regularized GP model proposed in Ranjan et al. (2011). For details see MacDonald et al. (2015). A new parameterization of the Gaussian correlation is used for the ease of optimization. This package uses a multi-start gradient based search algorithm for optimizing the deviance (negative 2*log-likelihood). For a complete list of functions, use library(help="gpfit"). The main function for fitting the GP model is GP_fit. Author(s) Blake MacDoanld, Hugh Chipman, Pritam Ranjan Maintainer: Hugh Chipman <hugh.chipman@acadiau.ca> References MacDonald, K.B., Ranjan, P. and Chipman, H. (2015). GPfit: An R Package for Fitting a Gaussian Process Model to Deterministic Simulator Outputs. Journal of Statistical Software, 64(12), Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4), Santner, T.J., Williams, B., and Notz, W. (2003), The design and analysis of computer experiments, Springer Verlag, New York. corr_matrix Power Exponential or Matern Correlation Matrix Description Computes the power exponential or Matern correlation matrix for a set of n design points in d- dimensional input region and a vector of d correlation hyper-parameters.

3 corr_matrix 3 Usage corr_matrix(x, beta, corr=list(type="exponential",power=1.95)) Arguments X the (n x d) design matrix beta a (d x 1) vector of correlation hyper-parameters in (, ) corr Details a list that specifies the type of correlation function along with the smoothness parameter. The default corresponds to power exponential correlation with smoothness parameter "power=1.95". One can specify a different power (between 1.0 and 2.0) for the power exponential, or use the Matern correlation function, specified as corr=list(type = "matern", nu=(2*k+1)/2), where k {0, 1, 2,... The power exponential correlation function is given by R ij = d k=1 exp( 10β k x ik x jk power ). The Matern correlation function given by Santner, Williams, and Notz (2003) is R ij = d 1 k=1 Γ(ν)2 (2 ν x ν 1 ik x jk 10 β k ) ν κ ν (2 ν x ik x jk 10 β k ), where κ ν is the modified Bessel function of order ν. Value Note The (n x n) correlation matrix, R, for the design matrix (X) and the hyper-parameters (beta). Both Matern and power exponential correlation functions use the new β parametrization of hyperparameters given by θ k = 10 β k for easier likelihood optimization. That is, beta is a log scale parameter (see MacDonald et al. (2015)). Author(s) Blake MacDonald, Hugh Chipman, Pritam Ranjan References MacDonald, K.B., Ranjan, P. and Chipman, H. (2015). GPfit: An R Package for Fitting a Gaussian Process Model to Deterministic Simulator Outputs. Journal of Statistical Software, 64(12), Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4),

4 4 GP_deviance Santner, T.J., Williams, B., and Notz, W. (2003), The design and analysis of computer experiments, Springer Verlag, New York. Examples ## 1D Example - 1 n = 5; d = 1; set.seed(3); beta = rnorm(1); R = corr_matrix(x,beta); print(r) ## 1D Example - 2 beta = rnorm(1) R = corr_matrix(x,beta,corr = list(type = "matern")) print(r) ## 2D example - 1 n = 10; d = 2; set.seed(2); beta = rnorm(2); R = corr_matrix(x,beta,corr = list(type = "exponential", power = 2)); print(r) ## 2D example - 2 beta = rnorm(2) R = corr_matrix(x,beta,corr = list(type = "matern", nu = 5/2)) print(r) GP_deviance Computes the Deviance of a GP model Description Evaluates the deviance (negative 2*log-likelihood), as defined in Ranjan et al. (2011), however the correlation is reparametrized and can be either power exponential or Matern as discussed in corr_matrix. Usage GP_deviance(beta, X, Y, nug_thres = 20, corr = list(type="exponential",power=1.95))

5 GP_deviance 5 Arguments beta X Y nug_thres corr a (d x 1) vector of correlation hyper-parameters, as described in corr_matrix the (n x d) design matrix the (n x 1) vector of simulator outputs a parameter used in computing the nugget. See GP_fit. a list of parameters for the specifing the correlation to be used. See corr_matrix. Value the deviance (negative 2*log-likelihood) Author(s) Blake MacDonald, Hugh Chipman, Pritam Ranjan References Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4), See Also corr_matrix for computing the correlation matrix; GP_fit for estimating the parameters of the GP model. Examples ## 1D Example 1 n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); beta = rnorm(1); GP_deviance(beta,x,y); ## 1D Example 2 n = 7; d = 1; y <- log(x+0.1)+sin(5*pi*x);

6 6 GP_fit beta = rnorm(1); GP_deviance(beta,x,y,corr=list(type="matern",nu=5/2)) ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2; n = 10; d = 2; beta = rnorm(2); GP_deviance(beta,x,y); GP_fit Gaussian Process Model fitting Description Usage For an (n x d) design matrix, X, and the corresponding (n x 1) simulator output Y, this function fits the GP model and returns the parameter estimates. The optimization routine assumes that the inputs are scaled to the unit hypercube [0, 1] d. GP_fit(X, Y, control = c(200*d, 80*d, 2*d), nug_thres = 20, trace = FALSE,maxit = 100,corr = list(type="exponential",power=1.95), optim_start = NULL) Arguments X Y control nug_thres trace the (n x d) design matrix the (n x 1) vector of simulator outputs. a vector of parameters used in the search for optimal beta. See Details. a parameter used in computing the nugget. See Details. logical, if TRUE, will provide information on the optim runs maxit the maximum number of iterations within optim, defaults to 100

7 GP_fit 7 corr optim_start a list of parameters for the specifing the correlation to be used. See corr_matrix. a matrix of potentially likely starting values for correlation hyperparameters for the optim runs, i.e., initial guess of the d-vector beta Details This function fits the following GP model, y(x) = µ + Z(x), x [0, 1] d, where Z(x) is a GP with mean 0, V ar(z(x i )) = σ 2, and Cov(Z(x i ), Z(x j )) = σ 2 R ij. Entries in covariance matrix R are determined by corr and parameterized by beta, a d-vector of parameters. For computational stability R 1 is replaced with R 1 δ lb, where R δlb = R+δ lb I and δ lb is the nugget parameter described in Ranjan et al. (2011). The parameter estimate beta is obtained by minimizing the deviance using a multi-start gradient based search (L-BFGS-B) algorithm. The starting points are selected using the k-means clustering algorithm on a large maximin LHD for values of beta, after discarding beta vectors with high deviance. The control parameter determines the quality of the starting points of the L-BFGS-B algoritm. control is a vector of three tunable parameters used in the deviance optimization algorithm. The default values correspond to choosing 2*d clusters (using k-means clustering algorithm) based on 80*d best points (smallest deviance, refer to GP_deviance) from a 200*d - point random maximin LHD in beta. One can change these values to balance the trade-off between computational cost and robustness of likelihood optimization (or prediction accuracy). For details see MacDonald et al. (2015). The nug_thres parameter is outlined in Ranjan et al. (2011) and is used in finding the lower bound on the nugget (δ lb ). Value an object of class GP containing parameter estimates beta and sig2, nugget parameter delta, the data (X and Y), and a specification of the correlation structure used. Author(s) Blake MacDonald, Hugh Chipman, Pritam Ranjan References MacDonald, K.B., Ranjan, P. and Chipman, H. (2015). GPfit: An R Package for Fitting a Gaussian Process Model to Deterministic Simulator Outputs. Journal of Statistical Software, 64(12), Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4), See Also plot.gp for plotting in 1 and 2 dimensions; predict.gp for predicting the response and error surfaces;

8 8 GP_fit optim for information on the L-BFGS-B procedure; GP_deviance for computing the deviance. Examples ## 1D Example 1 n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); print(gpmodel) ## 1D Example 2 n = 7; d = 1; y <- log(x+0.1)+sin(5*pi*x); print(gpmodel, digits = 4) ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2; n = 30; d = 2; print(gpmodel)

9 plot.gp 9 plot.gp Plotting GP model fits Description Usage Plots the predicted response and mean squared error (MSE) surfaces for simulators with 1 and 2 dimensional inputs (i.e. d = 1,2). ## S3 method for class GP plot(x, M = 1, range = c(0, 1), resolution = 50, colors = c( black, blue, red ), line_type = c(1, 2), pch = 20, cex = 1, legends = FALSE, surf_check = FALSE, response = TRUE,...) Arguments x M a class GP object estimated by GP_fit the number of iterations for use in prediction. See predict.gp range the input range for plotting (default set to [0, 1]) resolution the number of points along a coordinate in the specified range colors line_type pch cex legends For 1 Dimensional Plotting a vector of length 3 assigning colors[1] to training design points, colors[2] to model predictions, and colors[3] to the error bounds a vector of length 2 assigning line_type[1] to model predictions, and line_type[2] to the error bounds a parameter defining the plotting character for the training design points, see pch for possible options in par a parameter defining the size of the pch used for plotting the training design points, see cex for possible options in par a parameter that controls the inclusion of a legend; by default it is FALSE surf_check response For 2 Dimensional Plotting logical, switch between 3d surface and 2d level/contour plotting, the default of FALSE implies level/contour plotting logical, switch between predicted response and error (MSE) plots, the default of TRUE displays the response surface... additional arguments from wireframe or levelplot Author(s) Blake MacDonald, Hugh Chipman, Pritam Ranjan

10 10 plot.gp References Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4), See Also GP_fit for estimating the parameters of the GP model; predict.gp for predicting the response and error surfaces; par for additional plotting characters and line types for 1 dimensional plots; wireframe and levelplot for additional plotting settings in 2 dimensions. Examples ## 1D Example 1 n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); plot.gp(gpmodel) ## 1D Example 2 n = 7; d = 1; y = log(x+0.1)+sin(5*pi*x); ## Plotting with changes from the default line type and characters plot.gp(gpmodel, resolution = 100, line_type = c(6,2), pch = 5) ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2;

11 predict.gp 11 n = 30; d = 2; library(lattice); ## Basic level plot plot.gp(gpmodel) ## Adding Contours and increasing the number of levels plot.gp(gpmodel, contour = TRUE, cuts = 50, pretty = TRUE) ## Plotting the Response Surface plot.gp(gpmodel, surf_check = TRUE) ## Plotting the Error Surface with color plot.gp(gpmodel, surf_check = TRUE, response = FALSE, shade = TRUE) predict.gp Model Predictions from GPfit Description Usage Computes the regularized predicted response ŷ δlb,m (x) and the mean squared error s 2 δ lb,m (x) for a new set of inputs using the fitted GP model. ## S3 method for class GP predict(object, xnew = object$x,m=1,...) Arguments object xnew M Details Value a class GP object estimated by GP_fit the (n_new x d) design matrix of test points where model predictions and MSEs are desired the number of iterations. See Details... for compatibility with generic method predict The value of M determines the number of iterations (or terms) in approximating R 1 R 1 δ lb,m. The iterative use of the nugget δ lb, as outlined in Ranjan et al. (2011), is used in calculating ŷ δlb,m (x) and s 2 δ lb,m (x), where R 1 δ,m = M t=1 δt 1 (R + δi) t. Returns a list containing the predicted values (Y_hat), the mean squared errors of the predictions (MSE), and a matrix (complete_data) containing xnew, Y_hat, and MSE

12 12 predict.gp Author(s) Blake MacDonald, Hugh Chipman, Pritam Ranjan References Ranjan, P., Haynes, R., and Karsten, R. (2011). A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data, Technometrics, 53(4), See Also GP_fit for estimating the parameters of the GP model; plot.gp for plotting the predicted and error surfaces. Examples ## 1D Example n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); xvec <- seq(from=0,to=1,length.out=10); GPprediction = predict.gp(gpmodel,xvec); yhat = GPprediction$Y_hat; mse = GPprediction$MSE; completedata = GPprediction$complete_data; completedata; ## 1D Example 2 n = 7; d = 1; y <- log(x+0.1)+sin(5*pi*x); set.seed(1) x = maximinlhs(n,d) y = computer_simulator(x) xvec <- seq(from=0,to=1,length.out=10); GPmodel = GP_fit(x,y) GPprediction = predict.gp(gpmodel,xvec); yhat = GPprediction$Y_hat; mse = GPprediction$MSE; completedata = GPprediction$complete_data;

13 print.gp 13 ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2; n = 10; d = 2; xvector = seq(from=0,to=1,length.out=10); xvec = expand.grid(x = xvector, y=xvector); xvec = as.matrix(xvec); GPprediction = predict.gp(gpmodel,xvec); yhat = GPprediction$Y_hat; mse = GPprediction$MSE; completedata = GPprediction$complete_data; print.gp GP model fit Summary Description Prints the summary of a class GP object estimated by GP_fit Usage ## S3 method for class GP print(x,...) Arguments x Details a class GP object estimated by GP_fit... for compatibility with generic method print Prints the summary of the class GP object. It returns the number of observations, input dimension, parameter estimates of the GP model, lower bound on the nugget, and the nugget threshold parameter (described in GP_fit).

14 14 summary.gp Author(s) See Also Blake MacDonald, Hugh Chipman, Pritam Ranjan GP_fit for more information on estimating the model; print for more description on the print function. Examples ## 1D example n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); print(gpmodel) ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2; n = 10; d = 2; print(gpmodel,digits = 3) summary.gp Summary of GP model fit Description Prints the summary of a class GP object estimated by GP_fit

15 summary.gp 15 Usage ## S3 method for class GP summary(object,...) Arguments Details object a class GP object estimated by GP_fit... for compatibility with generic method summary prints the summary of the GP object (object), by calling print.gp Author(s) See Also Blake MacDonald, Hugh Chipman, Pritam Ranjan print.gp for more description of the output; GP_fit for more information on estimating the model; summary for more description on the summary function. Examples ## 1D example n = 5; d = 1; computer_simulator <- function(x){ x = 2*x+0.5; y = sin(10*pi*x)/(2*x) + (x-1)^4; set.seed(3); summary(gpmodel) ## 2D Example: GoldPrice Function x1=4*x[,1] - 2; x2=4*x[,2] - 2; t1 = 1 + (x1 + x2 + 1)^2*(19-14*x1 + 3*x1^2-14*x2 + 6*x1*x2 + 3*x2^2); t2 = 30 + (2*x1-3*x2)^2*(18-32*x1 + 12*x1^2 + 48*x2-36*x1*x2 + 27*x2^2); y = t1*t2; n = 10; d = 2;

16 16 summary.gp summary(gpmodel)

17 Index Topic Deviance GP_deviance, 4 Topic Gaussian Correlation corr_matrix, 2 Topic Gaussian Process Model Prediction predict.gp, 11 Topic Gaussian Process GP_fit, 6 Topic Matern Correlation corr_matrix, 2 Topic Power Exponential Correlation corr_matrix, 2 corr_matrix, 2, 4, 5, 7 GP_deviance, 4, 7, 8 GP_fit, 2, 5, 6, 10, GPfit (GPfit-package), 2 GPfit-package, 2 legend, 9 levelplot, 9, 10 optim, 6 8 par, 9, 10 plot.gp, 7, 9, 12 predict, 11 predict.gp, 7, 9, 10, 11 print, 13, 14 print.gp, 13, 15 summary, 15 summary.gp, 14 wireframe, 9, 10 17