Bayesian Optimization Under Uncertainty

Transcription

1 Bayesian Optimization Under Unertainty Justin J. Beland University o Toronto justin.beland@mail.utoronto.a Prasanth B. Nair University o Toronto pbn@utias.utoronto.a Abstrat We onsider the problem o robust optimization, where it is sought to design a system suh that it sustains a speiied measure o perormane under unertainty. This problem is hallenging sine modeling a omplex system under unertainty an be expensive and or most real-world problems robust optimization will not be omputationally viable. In this paper, we propose a Bayesian methodology to eiiently solve a lass o robust optimization problems that arise in engineering design under unertainty. The entral idea is to use Gaussian proess models o loss untions (or robustness metris) together with appropriate aquisition untions to guide the searh or a robust optimal solution. Numerial studies on a test problem are presented to demonstrate the eiay o the proposed approah. 1 Introdution Consider a salar output o an expensive omputer simulation (x 1, x 2 +δ, ξ), where x 1 X 1 R d1 and x 2 X 2 R d2 an be preisely ontrolled (ontrol ators) while δ Y R d2 and ξ Z R d ξ are random variables (noise ators) with the speiied joint probability density untion p(δ, ξ). Now, suppose that we seek the minima o subjet to the set o inequality onstraints, j (x 1, x 2 + δ, ξ) 0, j = 1,..., d, by varying the ontrol ators. This problem an be posed as a robust optimization problem where we seek to minimize some measure o loss suh that the optimum is least sensitive to the noise ators δ and ξ. The robust optimization problem that we onsider is o the orm x = arg min J (x) s.t. Pr[ j 0] 1 η, j = 1,..., d, (1) x {x 1,x 2} where J : X 1 X 2 R denotes a loss untion (or a robustness metri) [1] and η [0, 1] is a user deined parameter that ontrols the probability o onstraint satisation. Optimization problems o this orm are enountered in many areas suh as the design o iruits, airrat and automotive omponents [2, 3]. The primary ous o the present work is to develop eiient Bayesian optimization (BO) methods or solving (1). It is well known that Bayesian methods are well suited or loating the minima o omplex optimization problems on a limited omputational budget, partiularly when gradient inormation is not available and the underlying untion is orrupted by noise [4 7]. To the best o our knowledge, BO methods have not been ormulated or robust optimization problems o the orm onsidered here. The key hallenge is that we don t have aess to observations o J due to omputational resoure limitations, instead we only have the ability to query and j, j = 1,..., d. In this paper, we present a methodology to estimate loss untions that are o interest in optimization under unertainty using Gaussian proess (GP) models [8] onditioned on observations o and j, j = 1,..., d. Subsequently, we propose aquisition untions that an be used to iteratively onverge to the minima o (1). Finally, we present numerial studies to demonstrate the perormane o the proposed algorithm. 31st Conerene on Neural Inormation Proessing Systems (NIPS 2017), Long Beah, CA, USA.

2 2 Bayesian Optimization Under Unertainty In this setion we outline the proposed BO strategy to solve (1). To simpliy our notation, we introdue two new variables: x = {x 1, x 2 + δ, ξ} deined over the produt spae X R d1+d2+d ξ and ζ = {δ, ξ} deined over the produt spae = Y Z. In addition, we will use the notation x i to denote the ith observation o x and x 1:t to denote t observations. Lastly, we denote the set o optimization variables (or ontrol ators) {x 1, x 2 } by x X where X = X 1 X 2. Note that Y and Z are the image spaes o δ and ξ respetively. In the BO under unertainty ramework, the loss untion J is speiied suh that the noise ators ζ are integrated out and deisions an be made entirely in the deision spae X. One possibility is to use Bayes risk [9] as the loss untion, i.e. J (x) = ( x)p(ζ)dζ. (2) Here, the loss untion is the irst statistial moment o over the spae given a setting or the ontrol ators x. Alternatively, one may deine J as the seond-order statistial moment given by ( x)2 p(ζ)dζ. This measure o loss simultaneously minimizes Bayes risk and the variane o [10]. The proposed ramework or BO under unertainty an aommodate a wide variety o alternative robustness metris, suh as the aggregate o the mean and the variane [11], the minimax priniple [3] and horsetail mathing [12]. Next, we speiy a zero mean GP prior over the untion with a ovariane untion k pr : X X R. In other words, we approximate as a untion o x 1, x 2 and ξ. Note that it is not neessary to expliitly model the dependene o on δ sine this noise ator an be interpreted to be a perturbation to x 2. Now, suppose that we have gathered the t observations y 1:t = ( x 1:t ) + ɛ, where ɛ N (0, ν) denotes measurement noise. Subsequently, by onditioning the prior on the t observations we obtain the posterior distribution GP(µ pos, kpos ), where µpos ( x) = k( x)t (K + νi) 1 y 1:t is the posterior mean and k pos ( x, x ) = k pr ( x, x ) k( x) T (K + νi) 1 k( x ) denotes the posterior ovariane. The elements o the ovariane matrix K R t t are given by K pq = k pr ( xp, x q )+νδ pq where δ pq denotes the Kroneker delta and k( x) = [k pr ( x, x1 ),..., k pr ( x, xt )] T is the vetor o ross ovarianes. The hyperparameters in the prior ovariane untion and the noise variane an be estimated using an empirial or ully Bayesian approah [8]. Consider the ase when the loss untion J is deined as Bayes risk (see (2)). Sine the loss untion is a linear operator applied to it ollows that, where µ pos k pos J (x, x ) = J (x) = J GP(µ pos J, kpos J ), (3) µ pos ( x)p(ζ)dζ = z(x)t (K + νi) 1 y 1:t, (4) k pos ( x, x )p(ζ, ζ )dζdζ = z(x, x ) z(x) T (K + νi) 1 z(x ), (5) z(x) : X R t with its ith omponent deined as z i (x) = kpr ( x, xi )p(ζ)dζ, and z : X X R is given by z(x, x ) = k pr ( x, x )p(ζ, ζ )dζdζ. (6) The integrals that appear in (4) - (6) an be evaluated analytially provided that the ovariane untion k pr ( x, x ) and the joint distribution p(ζ) are separable with respet to their input arguments. When this is not easible, a multivariate sparse quadrature sheme an be used to approximate these integrals [13]. We note that i J is not a linear operator applied to then the estimator or the loss untion is no longer a GP, or example, i J (x) = ( x)2 p(ζ)dζ. In this speii ase, we an apply GP inerening to the integrand 2 so that the estimator or J is a GP. For a more general nonlinear loss untion, we an onstrut a Gaussian approximation o J using a sampling proedure or a sparse quadrature sheme. 2

3 To guide the searh towards the minimizer x while ensuring a high probability o onstraint satisation, we require a strategy to identiy the point x t+1 = {x t+1 1, x t+1 2 } X, where we should next query the deterministi omputer model. However, this inormation alone is not suiient to evaluate and j, j = 1,..., d, as we must also selet a setting or ξ t+1 Z. We then require the aquisition untion αx : X R to guide the optimization and αξ : Z R to identiy an appropriate setting or ξ t+1. Loating the query point x t+1 is aomplished by maximizing αx(x). To avoid sampling ar away rom the easible region we selet x t+1 suh that Pr[ j (x t+1 1, x t δ, ξ) 0], j = 1,..., d is high beore querying the omputer model [14]. The proposed aquisition untion an be written as d αx(x) = α x (x) Pr[ j (x 1, x 2 + δ, ξ) 0], (7) j=1 where one example o α x : X R is the probability o improvement (POI) riterion [15]. This means α x = Pr[J J ], where J denotes a target or the loss untion. Another possibility is the expeted improvement (EI) riterion [16], whih is deined as E[max(0, J J )]. It is also possible to use the lower onidene bound (LCB) [17] deined as µ pos J (x) βkpos J (x, x) with β R+. The LCB is well suited to problems where the goal is to minimize regret but may suggest new points that do not explore beyond a loal minima. It is worth noting that it is also possible to use inormation gain metris as andidate aquisition untions or use in the BO under unertainty ramework [18 20]. I we have GP models or all o the onstraints then the produt terms appearing in (7) an be eiiently approximated using the ideas presented in [21, 22]. Finally, we onsider the ase when ξ t+1 is seleted suh that the preditive apability o the GP models or the objetive and onstraint untions are improved. One way to proeed urther would be to assume that x t+1 is ixed and deine the aquisition untion as αξ (ξ) = kpos ({xt+1, ξ}, {x t+1, ξ}). By maximizing this aquisition untion we selet the loation in the image spae Z where the model is least aurate. An alternative would be to express αξ (ξ) as the aggregate o both the variane o the objetive untion and all o the onstraints. Another option would be to integrate over the ontrol variables rom the GP posterior variane and maximize αξ (ξ) = X kpos ( x, x)dx to obtain the setting or ξ t+1. Again, it would be possible to amalgamate some ombination o the variane o eah onstraint j, j = 1,..., d and integrate over X. With the new points seleted, we query the objetive and onstraint untions and then augment the dataset D 1:t = { x 1:t, y 1:t, 1:t y } with the new observations { x t+1, y t+1, t+1 y } to obtain D t+1, where x i ontains {x i, ξ i } and i y R d denotes the ith vetor o noise orrupted onstraint observations. The key steps o the proposed methodology are outlined in Algorithm 1. Algorithm 1: Bayesian Optimization Under Unertainty D 1:t = { x 1:t, y 1:t, 1:t y } // Initialize training dataset with t samples while ost budget do GP(µ pos J GP(µ pos J, kpos J ) j GP(µ pos j, k pos, kpos ) onditioning on D 1:t j ) onditioning on D 1:t GP(µ pr, kpr ) x t+1 = arg max x αx(x) D 1:t+1 D 1:t { x t+1, y t+1, t+1 y } t t + 1 GP(µ pr j, k pr j ), j = 1,..., d ξ t+1 = arg max ξ α ξ(ξ) 3 Numerial Studies We present numerial studies involving the minimization o the Branin untion [23] under unertainty. In partiular, we rewrite the Branin untion as (x + δ), where x [ 5, 10] [0, 15] and δ is a vetor o uniormly distributed random variables deined over the interval [ δ b, δ b ]. For this study, we hoose Bayes risk as the loss untion. Sine J is inexpensive to evaluate preisely using a quadrature sheme or this partiular problem, we illustrate the ontour plots o Bayes risk while varying δ b R 2 as shown in 3

4 Figure 1. For the ase when δ b is a vetor o zeros as shown in Figure 1a we reover the orig- (a) δ b = [0, 0]T (b) δ b = [1, 1]T () δ b = [2, 2]T (d) δ b = [3, 3]T Figure 1: Bayes risk or the Branin untion where red stars indiate global minima. inal Branin untion with three global minima. In the senario where δ b > 0 there exists a single global minima as illustrated in Figures 1b, 1 and 1d. Now, suppose that we seek to minimize J using the methodology presented in Setion 2. I the prior distribution p( ) is deined by a zero mean untion and the squared exponential ovariane untion pos [8] then we an derive losed orm expressions or µpos J and kj. We study the perormane o POI, EI ı and UCB using the gap metri G = (J (x ) J (x ))/(J (xı ) J (x? )), where J (xı ) denotes the Bayes risk evaluated at the irst query point and J (x ) is the minimum observed value in J (x1:t ) [24]. A omparison o the results obtained or δ b = [1, 1]T are shown in Figure 2. We initialize the dataset D1:10 with 10 random query points and arry out 25 independent runs. Additionally, in Figure 2d, (a) αx : POI (b) αx : EI () αx : LCB (d) αx : random Figure 2: Convergene o the gap metri G or various aquisition untions. we use a random number generator as the aquisition untion or omparison. For this setting o p(δ), the POI aquisition untion perorms onsistently well while the EI riterion explores more in the earlier iterations, thus requiring additional evaluations o to reah the global minima. When ompared with the randomized aquisition untion, all methods have lower unertainty in the later iterations. 4 Conlusions and Future Work We propose an eiient ramework or solving onstrained optimization problems where the objetive or the onstraint untions are sensitive to unertainty. We irst speiy a loss untion suh that deisions an be made entirely in the spae o the ontrol ators. However, i evaluating the underlying objetive and onstraint untions are expensive, then solving the robust optimization problem an be omputationally demanding. It was shown that by speiying a GP prior over the objetive untion, estimating the loss untion beomes tratable and in some ases the mean and ovariane untions an be expressed analytially. Similarly, using GP models or the onstraints, the probability o onstraint satisation an be eiiently approximated. Finally, we showed that update points used to query the expensive objetive and onstraint untions an be seleted by maximizing speiied aquisition untions. The ous o ongoing work is on the ormulation o alternative aquisition untions to identiy multiple query points in parallel as well as the use o gradient observations to aelerate onvergene. Aknowledgments: This researh is unded by an NSERC Disovery Grant and the Canada Researh Chairs program. 4

5 Reerenes [1] S. M. Göhler, T. Eiler, and T. J. Howard, Robustness metris: Consolidating the multiple approahes to quantiy robustness, Journal o Mehanial Design, vol. 138, no. 11, p , [2] M. S. Phadke, uality engineering using robust design. Prentie Hall PTR, [3] H.-G. Beyer and B. Sendho, Robust optimization a omprehensive survey, Computer methods in applied mehanis and engineering, vol. 196, no. 33, pp , [4] J. Mokus, Appliation o Bayesian approah to numerial methods o global and stohasti optimization, Journal o Global Optimization, vol. 4, no. 4, pp , [5] D. R. Jones, A taxonomy o global optimization methods based on response suraes, Journal o global optimization, vol. 21, no. 4, pp , [6] J. Snoek, H. Larohelle, and R. P. Adams, Pratial Bayesian optimization o mahine learning algorithms, in Advanes in neural inormation proessing systems, 2012, pp [7] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, Taking the human out o the loop: A review o bayesian optimization, Proeedings o the IEEE, vol. 104, no. 1, pp , [8] C. E. Rasmussen and C. K. I. Williams, Gaussian proesses or mahine learning. MIT press Cambridge, 2006, vol. 1. [9] J. O. Berger, Statistial deision theory and Bayesian analysis. Springer Siene & Business Media, [10] G. Taguhi, Introdution to quality engineering: Designing quality into produts and proesses [11] T. K.-L. Chen W Allen J and F Mistree, A proedure or robust design: Minimizing variations aused by noise ators and ontrol ators, ASME Journal o Mehanial Design, vol. 118, pp , [12] L. Cook and J. Jarrett, Horsetail mathing: A lexible approah to optimization under unertainty, Engineering Optimization, pp. 1 19, [13] T. Gerstner and M. Griebel, Numerial integration using sparse grids, Numerial algorithms, vol. 18, no. 3, pp , [14] M. J. Sasena, Flexibility and eiieny enhanements or onstrained global design optimization with kriging approximations, PhD thesis, General Motors, [15] H. J. Kushner, A new method o loating the maximum point o an arbitrary multipeak urve in the presene o noise, Journal o Basi Engineering, vol. 86, no. 1, pp , [16] J. Mokus, V. Tiesis, and A. Zilinskas, The appliation o Bayesian methods or seeking the extremum, Towards Global Optimization, vol. 2, pp , [17] N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger, Gaussian proess optimization in the bandit setting: No regret and experimental design, arxiv preprint arxiv: , [18] P. Hennig and C. J. Shuler, Entropy searh or inormation-eiient global optimization, Journal o Mahine Learning Researh, vol. 13, no. Jun, pp , [19] J. M. Hernández-Lobato, M. Gelbart, M. Homan, R. Adams, and Z. Ghahramani, Preditive entropy searh or Bayesian optimization with unknown onstraints, in International Conerene on Mahine Learning, 2015, pp [20] Z. Wang and S. Jegelka, Max-value entropy searh or eiient Bayesian optimization, arxiv preprint arxiv: , [21] J. Bet, D. Ginsbourger, L. Li, V. Piheny, and E. Vazquez, Sequential design o omputer experiments or the estimation o a probability o ailure, Statistis and Computing, vol. 22, no. 3, pp , [22] R Shöbi, B. Sudret, and S. Marelli, Rare event estimation using polynomial-haos kriging, ASCE-ASME Journal o Risk and Unertainty in Engineering Systems, Part A: Civil Engineering, vol. 3, no. 2, p. D , [23] A. Torn and A. Zilinskas, Global optimization. Springer-Verlag New York, In., [24] E. Brohu, Interative Bayesian optimization: Learning user preerenes or graphis and animation, PhD thesis, University o British Columbia,