Iterative Improvement plus Random Restart Stochastic Search by X. Hu, R. Shonkwiler, and M. C. Spruill School of Mathematics Georgia Institute of Tech

Transcription

1 Iterative Improvement plus Random Restart Stochastic Search by X. Hu, R. Shonkwiler, and M. C. Spruill School of Mathematics Georgia Institute of Technology Atlanta, GA 0 shonkwiler@math.gatech.edu, spruill@math.gatech.edu Keywords: Monte Carlo Methods, Multistart Methods, Markov Chains, Parallelization. AMS Subject Classiæcation: 5K0. Running head: Iterative Improvement plus Random Restart Mail proofs to: R. Shonkwiler School of Mathematics Georgia Institute of Technology Atlanta, GA 0 December 8, 99

2 Abstract The multistart or iterative improvement plus random restart èi R è method for global optimization combines local search with random initialization. This method partitions the solution space into basins of attraction to the local optima. The method's convergence rate to a global optimizer as the number of iterations t increases is given as s çt in terms of two scalar parameters, acceleration s, and retention ç. Acceleration is estimated as s é =ç é. The expected running time of the algorithm in order to ænd a basin containing a global optimizer is estimated as E ç s, ç : There is associated with every I R search a fundamental polynomial f of low degree having a unique root ç é. Acceleration and retention are given in terms of f and ç according to ç = s = çèç, èf 0 èçè : ç,fèè The coeæcients of f can be estimated during the course of a run. Under mild conditions on the class of potential objective functions, F, the most eæective startèrestart distribution ç is the uniform distribution. Parallel execution of an I R search using m independent and identical processes èiipè improves the convergence rate to è=s m èç mt ; since s é, IIP parallel using m processes converges more than m times faster than a single process. Hence accelerated convergence can even be achieved by simulating m processes in code on a single processor. The method and our results are illustrated on two example problems.

3 Iterative Improvement plus Random Restart Stochastic Search. Introduction Given a real valued function C deæned on a domain æ, we analyze restart methods for ænding some point x æ optècè the set of global minimizers of C over æ, Cèx æ è ç Cèxè; x æ: Referring to C as a cost, assume it is any function from a class F of functions all having domain æ. In what follows, we may impose other conditions on F as well. If C is suæciently smooth, powerful numerical methods exist for identifying local minimizers, èdennis and Schnabel, 98è. Often these methods utilize local derivative, or gradient, information to construct a sequence of points x 0 ;x ;x ;:::! x on which the function values v i = Cèx i è decrease until a local minimum is determined to within some tolerance. If C has numerous local minima, locating a global minimizer is problematic and success depends on the choice of starting value x 0. Inspired by this gradient model, we seek to generalize it, even though derivatives may not be deæned for functions in F. Given C F, throughout this work we assume there is a èdeterministicè algorithm g such that èè for every x æ, gèxè æ and Cègèxèè ç Cèxè; èè and èè g is not cyclic, that is, if x = g k èxè for some ké0, then x = gèxè =g èxè=:::=g k èxè: We refer to g as an èiterativeè improvement algorithm. In a given application problem, g might bethe implementation of some heuristic which improves the cost on each iteration. Thus, as above, an improvement algorithm applied to its own output generates a sequence of points x 0, x = gèx 0 è, x = g èx 0 è, ::: in æ on which the cost values are non-increasing. The limit of such a sequence terminates in a local minimizer relative to C èwhich could also be a global minimizer as wellè beyond which improvement is not possible by the application of g alone. All points of the domain which are attracted to a given local minimizer deæne its basin. That is to say, the relation x ç y if and only if lim k! g k èxè =lim j! g j èyè is an equivalence relation on æ. The resulting equivalence classes, B i, i = 0; ;:::, are the basins of æ relative to g. The settling point or local minimizer x of basin B is given as the limit, lim k! g k èxè, where x is any point of B. Once a local minimum has been reached, for instance when gèxè = x, the process must be restart. For this, we use a æxed probability distribution ç over æ. Upon reaching a

4 local minimum, the search is continued by selecting a restart point x 0 0 æ with probability çèx 0 0è. We also use ç to start the search; thus ç is the startèrestart distribution. We assume that ç is not degenerate, that is çèxè é 0 for all x æ. Our search process is described by a random variable Xètè on æ. To start, Xèè is selected with probability çèxèèè. Then for t = ; ;:::;t we take Xèt +è=gèxètèè where t is the least t such that Xèt +è = Xètè. The sequence f;:::;t g is the ærst epoch. Next Xèt +è is ère-èselected with probability çèxèt + èè and so on. The startèrestart times are ;t ;t ;:::, a strictly increasing sequence, and deæne the epochs. èno restart check is made to previous epochs, i.e. that the re-selected Xèt i +è equals Xèt i è, i =;;:::.è We also deæne the running minimizer random variable W ètè æ as follows. Let vt æ = minfcèxèçèè : ç ç ç tg, hence vt æ is the best observation up to time t, or the current best observation. Put W ètè =Xèçèwhere ç = minfs : CèXèsèè = vt æ g. Evidently CèW ètèè = vt æ for all t; if the current best observation has been encountered multiple times, W ètè is the ærst point to achieve it. In what follows we will assume that the domain æ is a ænite set, let N be its cardinality, N = jæj. Nevertheless our results also hold for iterative improvement applied to smooth cost functions on domains in Euclidean space provided the number of improvement iterations are kept ænite. This is always the case in a computer implementation. The reason is that the results depend only on the number of iterations taken to ëænd" local minima and not on the points themselves. We summarize our results here. Iterative improvement plus random restart èi R è, is a homogeneous Markov Chain on æ. Its convergence rate to a global optimizer is s çt, that is PrëW ètè = optècèë ç s çt! 0 as t!; where s and ç are two scalar parameters we call acceleration and retention respectively. Acceleration is estimated as sé=ç é. The expected running time of the algorithm in order to ænd a basin containing a global minimizer is estimated as E ç s, ç : There is associated with every I R search a fundamental polynomial fèçè =r 0 ç+r ç +:::+r d ç d+, ; of low degree having a unique root ç é. Acceleration and retention are given in terms of f and ç according to ç = s = çèç, èf 0 èçè : ç,fèè

5 The coeæcients of f can be estimated during the course of a run. Under mild conditions on the class F, the most eæective startèrestart distribution ç is the uniform distribution over æ. Parallel execution of an I R search using m independent and identical processes èiipè improves the convergence rate to è=s m èç mt ; since s é, IIP parallel using m processes converges more than m times faster than a single process. Hence improvement can be achieved by simulating m processes in code on a single processor. The method and our results are illustrated on two example problems. A recent survey of the literature on global stochastic optimization can be found in èschoen, 990è. What we have called random restart methods are included there as the simplest instances of what Schoen calls multistart methods. èsolis and Wets, 98è study convergence for more complicated random restart methods in which the probability distribution for choosing the next starting point can depend on the evolution of the search. èboender and Rinnooy Kan, 98è study stopping rules. Our studies do not address these issues. ètíorn and Zelinskas, 989è present in their survey some numerical results on parallelization for a collection of methods and end by recommending future study of Monte Carlo and geometric rather than algorithmic parallelization. Our investigation of random restarts is in one such recommended area, multiple CPU Monte Carlo parallelization, and continues the work of èshonkwiler and Van Vleck, 99è using the same techniques.. Running time estimation When the number of improvement iterations is ænite, as wehave assumed, I R search has a simple graph theoretic representation. Forest of trees model Let the vertices of the graph correspond to the points of the domain. And let there be a directed edge from x to y if and only if y = gèxè. In this representation, the points of each basin are organized as a rooted tree with the local minimizer serving as the root. By the depth of a tree we mean its maximum path length; the number of vertices in such a path is the depth plus. The graphical representation of the entire space is therefore a forest of trees, one for each basin. The settling point ofat least one basin is a global minimizer. We shall refer to such a basin as a goal basin. By the min-bin M we mean the union of goal basins. The incorporation of random restart modiæes our graph model by adding an edge from every local minimizer to every point of the domain. As noted above, the random variable Xètè is a Markov chain on æ. Letting the row vector æ t denote the distribution of Xètè, and P the transition probability matrix, standard Markov chain theory provides that æ t+ = æ P t 5

6 where æ is the starting distribution ç. Ordering the points of æ according to M ærst, then basin B and so on, the transition matrix P assumes the form P = M 0 ::: 0 L B ::: L L L ::: B n 5 : èè By overload of notation, we also use B i to denote the matrix corresponding to basin B i, i = ;:::;n. The same for the sub-matrix M. By ordering the points, x i;j B i, j = ;:::;jb i j, of each such basin beginning at the root node and working up the tree, each sub-matrix B i assumes of the form B i = çèx i; è çèx i; è çèx i; è ::: çèx i;jb ijè 0 0 ::: ::: ::: 0 the 's are in the lower triangle but not necessarily on the sub-diagonal. The blocks designated by L in èè, for instance the one in the kth column, are generic for the form L = çèx k; è çèx k; è ::: çèx k;jb kjè 0 0 ::: : 0 0 ::: 0 In putting these blocks together to form P, the row corresponding to a local minimum pertains to a restart with distribution ç. Finally, M is the block corresponding to the min-bin which can be comprised of one or more goal basins. Its structure, similar to the B i, consists of 's in the lower triangular part on each row not corresponding to a global minimizer. In most problems, global minima are not recognized as such when encountered. Indeed, this aspect of the search problem engenders the theory of stopping times. Of course the running minimizer W ètè will remain constant henceforth when the global minimum is found, but the search will press on. If in fact global minimizers are not recognized, then the rows of P stemming from a global minimizer are identical to those corresponding to a local minimizer, namely, the distribution deæned by ç. There are situations however when a global minimizer is known as such when found. These situations typically occur in inverse problems in which the precise set of inputs are sought giving some known output. In such a case the minimizer is an absorbing state for the chain since one stops searching 5 ;

7 at that point; therefore the corresponding row is the delta function æ ij equal to if i = j and 0 otherwise. There is also a circumstance in which the terms of these rows don't matter, such as when computing the expected hitting time of the chain to a global minimizer; a subject we take up next. Expected hitting time Let H be the hitting time random variable to M, that is, the number of iterations t until Xètè M, and let E be its expectation. Knowing E is important because the min-bin will lead, deterministically, to a point of optècè. Further E can be found by the method of èshonkwiler and Van Vleck, 99è because ^P, the matrix which results from P when the rows and columns of the min-bin elements are deleted, is a primitive matrix èsee nextè. In just the same way that ^P is the min-bin deleted version of P, let ^æt be the min-bin deleted version of æ t. Theorem. Let ç be the Perron-Frobenius eigenvalue of ^P and ç be its next largest eigenvalue èin modulusè. Let ç be the corresponding normalized right eigenvector and put s ==è^æ æ çè èdot productè. Then there is a æxed polynomial hètè such that æ ç ç t ç t PrèXèt +è =Mè, sæ éhètè jç j ; t=;;:::: èè ç Proof. Take M to be the goal in èshonkwiler and Van Vleck, 99è. The results of that work imply the conclusion provided the deleted matrix ^P is primitive. But it is clear from the forest of trees graph èwith the restart edges includedè that each non-min-bin state communicates with every other, possibly through a restart. And, since every local minimizer can follow itself, the process is both irreducible and aperiodic. Hence ^P is a primitive matrix. Corollary. With terms as in the Theorem, for some integer K, æ æ PrèW ètè = optècèè, çt sç d m è æ = O t K ç jç j ç ç t! : èè Proof. Since at most a ænite number of additional steps after entering M lead to optècè, say d m èsee belowè, we can write Hence from the Theorem we have æ PrèXètè optècèè ç PrèXèt, d m è Mè: æ çd m ç t PrèW ètè = optècèè, sæ ç hèt, d mè æ ç ç jç j ç d m ç jç j ç ç t

8 from which the conclusion follows. Remark. The probability vector! acts something like a stationary distribution since! ^P = ç!; the relative probabilities among the states remain æxed, rather the overall probability of being in a non min-bin state decreases by the factor ç. Thus it is clear that retention is the asymptotic probability that the process does not ænd M ètaken as the goal hereè in a given iteration; that is the probability that the process is retained in the non min-bin basins. In almost all large problems retention is very close to èbut just less thanè ; values çé:99999 are the rule. Remark. Since expected hitting time is given by from èè an estimate of E is Deænition. hitting time distribution, chdètè. E = X t= E ç s PrèXètè = Mè;, ç : è5è We refer to the probability PrèW ètè = optècèè as the complementary. Fundamental Polynomial Because of the special structure of P, both retention and acceleration can be calculated directly. In the forest of trees model, it is clear that all states which are a given number of steps from a settling point are equivalent as far as ænding a global minimizer is concerned. Therefore we construct a simpliæed but equivalent graph model. The equivalent graph of the iterative improvement process consists of two linear directed graphs, one for the min-bin and one for all other points of æ; in essence, two unbranched trees. The root of the min-bin tree identiæes to optècè which we may also denote as opt 0 ècè. The next node above this one identiæes to all vertices of the min-bin connected by an edge to a global minimizer. Denote this subset of æ by opt ècè; that is to say opt ècè =fxæ:gèxèoptècèg. In a similar fashion, deæne opt j+ ècè =fxæ:gèxèopt j ècèg: Thus all points of opt j ècè are j steps from a global minimizer and will identify to the jth vertex above the root node of the min-bin tree. The depth, d m of this tree is equal to the largest depth of a goal basin tree of the previous model. 8

9 Similarly the root node of the non min-bin tree identiæes with the set of local minimizers which are not global minimizers. Denote this set by Lopt 0 ècè èor just LoptèCèè. Recursively deæne Lopt j+ ècè =fxæ:gèxèlopt j ècèg j =0;;:::: The jth vertex above the root vertex of the non min-bin tree represents the set Lopt j ècè. Let d denote the depth of this tree; d is equal to the largest depth among the non-goal basins of the previous model. The weight of all edges of these two, as yet, disconnected unbranched trees is. Adding restart modiæes the graph by adding an edge from the root vertex of the non min-bin tree to all other vertices. The weight of these new edges is easy to ægure and depends on the restart measure ç. In particular let q j = çèopt j ècèè, j =0;;:::;d m and let r j = çèlopt j ècèè, j = 0; ;:::;d. These are the weights applied to the restart edges. That is, the probability of restarting j steps from a global minimizer is q j and j steps from a local minimizer is r j. Under the equivalency, the min-bin deleted transition probability matrix becomes P 0 = èthe 's are on the sub-diagonal hereè. r 0 r r ::: r d, r d 0 0 ::: ::: ::: ::: 0 Theorem. The characteristic polynomial of P 0 is 5 ; èè,ç d+ + r 0 ç d + r ç d, + :::+r d, ç+r d : èè Proof. Expand detèp 0, çiè by minors along the ærst row. Deænition. In what follows, we shall ænd it easier to work with a slightly diæerent form of this polynomial; set ç ==ç and multiply through by ç d+. We refer to the result, fèçè, as the fundamental polynomial of P, fèçè =r 0 ç+r ç +:::+r d, ç d +r d ç d+, : è8è By construction, every root of èè is also a root of the fundamental polynomial and conversely. Note that the degree of the fundamental polynomial, d +, is the number of distinct steps in the deepest non goal basin; usually far smaller than the dimension of P. 9

10 Theorem. All non-zero eigenvalues of the original matrix ^P are eigenvalues of P 0. Therefore, the reciprocal of every eigenvalue of P 0 is a root of è8è, and conversely, the reciprocal of every root of è8è is an eigenvalue of P 0. Proof. We show that two nodes may be merged. Since the complete merging of nodes can be achieved by successively merging two at a time, this will prove the theorem. Hence suppose nodes x s and x t lead to x m. Then both rows s and t of ^P have the form èæm è= è0 0 ::: 0 0 ::: 0è with the in the mth position and 0's elsewhere. Under the equivalence, assume that node t is merged with node s whose new chance of being selected for restart is therefore the sum ç s + ç t. Row s of P 0 will be èæ m è while column s of P 0 will be the sum of columns s and t of ^P, namely çs + ç t for rows corresponding to local minima and 0 for the other rows. To simplify, wemay èand doè re-index both matrices. For ^P write xs ærst and x t next followed by the other nodes in their same order. Then ^P, çi will be,ç 0 ::: 0 0 ::: 0 0,ç ::: 0 0 ::: 0 p p ::: p ;m, p m p ;m+ ::: p n p n p n ::: p n;m, p nm p n;m+ ::: p nn, ç Expanding detè ^P, çiè by the ærst row in minors gives è,çè det è,è m+ det,ç ::: 0 0 ::: 0 p ::: p ;m, p m p ;m+ ::: p n p n ::: p n;m, p nm p n;m+ ::: p nn, ç ,ç ::: 0 0 ::: 0 p p ::: p ;m, p ;m+ ::: p n p n p n ::: p n;m, p n;m+ ::: p nn, ç Now expanding each sub-determinant by its ærst row in minors gives è,çè è è,è m det è,çè det p, ç ::: p n p n ::: p nn, ç p p, ç ::: p ;m, p ;m+ ::: p n p n p n ::: p n;m, p n;m+ ::: p nn, ç 0 5 :! 5,

11 è,è m+ è,çè det p p, ç ::: p ;m, p ;m+ ::: p n p n p n ::: p n;m, p n;m+ ::: p nn, ç Factoring,ç from all members and adding the last two determinants, since all but their ærst columns are identical, we get è,è m det è,çè det p, ç ::: p n p n ::: p nn, ç p + p p, ç ::: p ;m, p ;m+ ::: p n p n +p n p n ::: p n;m, p n;m+ ::: p nn, ç But this ænal result is exactly the expansion of detèp 0, çiè for the reduced matrix P 0 proving the theorem. In reducing the degree of the characteristic polynomial by one, only the zero root,ç =0has been lost; it occurred in the factor step above. 5 : 5 : Deænition. Theorem. Let ç 0 be the probability oflanding in the min-bin, ç 0 = dmx j=0 For the fundamental polynomial, çèopt j ècèè: fèè =,ç 0 ; è9è in particular, its value at is negative. Moreover, f has a unique root ç é. Thus the Perron-Frobenius eigenvalue ç, that is retention, equals reciprocal ç ç = ç : Proof. Since a restart must select some node, ç 0 + r 0 + r + :::+r d = from which the ærst conclusion follows. Since f!as ç!and f 0 èçè =r 0 +r ç+r ç +:::+dr d, ç d, +èd+èr d ç d ; è0è is positive for all ç ç 0, the second conclusion also follows. The Perron-Frobenius eigenvalue of ^P èand also of P 0 è is its largest eigenvalue in modulus, is real and less than ; hence it must be reciprocal ç. Deænition. We refer to ç as the principal root of the fundamental polynomial.

12 The right Perron-Frobenius eigenvector, ç, of ^P is easily calculated. From èè we get the recursion equations From this we ænd ç 0 = çç ç = çç.=. ç n, =çç n : ç k = ç k ç 0 ; k =;:::;n: Similarly, from èè we get the following recursion equations for the components of the left Perron-Frobenius eigenvector,!,! 0 r 0 +! = ç! 0! 0 r +! = ç!.+.=. èè! 0 r n, +! n =ç! n,! 0 r n = ç! n : From these we get equations for the components in terms of! 0 ;! n =! 0 çr n! n, =! 0 èçr n, + ç r n è.=. èè! =! 0 èçr + ç r + :::+ç n r n è: The sum of the system èè in conjunction with the normalization, P! i =, gives! 0 X r i + nx! 0 è, ç 0 è+,! 0 ==ç: i! i ==ç Solve for! 0 to get! 0 = ç, çç 0 : èè

13 Further recall that under the normalization, P! i ç i =;hence, using èè and fèçè =0, From this we get that! 0 ç 0 =+ç r +ç r +:::+nç n+ r n! 0 ç 0 = çr 0 +ç r +ç r +:::+èn+èç n+ r n! 0 ç 0 ç = r 0 +çr +ç r +:::+èn+èç n r n! 0 ç 0 ç =f0 èçè: ç 0 =! 0 çf 0 èçè : Finally we calculate s ==èçææ 0 è where æ 0 is the non min-bin partition vector of the starting distribution èfor the equivalent processè; thus Therefore So The foregoing prove the following theorem. æ 0 =èr 0 r ::: r d è: s = r 0ç 0 + r ç + :::+r d ç d =r 0 ç 0 +r çç 0 + r ç ç 0 + :::+r d ç d ç 0 = ç 0 ç = ç! 0 f 0 èçè s = çèç, èf 0 èçè : èè,fèè Theorem 5. Acceleration s is given by equation èè in terms of the fundamental polynomial and its principal root. Theorem. For iterative improvement with restart, s ç ç, é. Proof. The secant line joining the points è;fèèè and èç; 0è has slope,fèè=èç, è. By the Mean Value Theorem, for some point éçéç, But since f 00 é 0 on ë; è, it follows that f 0 èçè=,fèè ç, : f 0 èçè éf 0 èçè:

14 Hence s ç = f 0 èçè è,fèè=èç, è é : Run time estimation of retention, acceleration and hitting time Remark. Returning to the fundamental polynomial, we notice that its coeæcients are the various probabilities for restarting a given distance from a local minimum. Thus the linear coeæcient is the probability of restarting on a local minimum, the quadratic coeæcient is the probability ofrestarting one iteration from a local minimum and so on. As a result, it is possible to estimate the fundamental polynomial during a run by keeping track of the number of iterations spent in the downhill processes. Using the estimate of the fundamental polynomial, estimates of retention and acceleration and hence also expected hitting time can be aæected. As a run proceeds, the coeæcient estimates converge to their right values and so does the estimate of E, see x.. Hitting time to a global minimizer. Let E denote the expected hitting time directly to optècè calculated from a start up. By combining the hitting time E to the min-bin together with the conditional distribution of landing in the min-bin, E can be calculated. Thus E = E + dmx j= jçèopt j ècèè: Alternatively, E can be computed directly from the equivalent matrix description. The full equivalent matrix can be written ç A Z Q where Z is the èd m +èæèd+ è matrix of 0's, P 0 is as above, A is the matrix with 's on the sub-diagonal èbeginning with the second row, a = è and Q is the èd +èæèd m +è matrix Q = Hence the optècè deleted matrix is P 0 ç q 0 q q ::: q d m, q d ::: ::: 0 0 ç ç A 0 Z çp = 0 Q 0 P 0 5 : è5è

15 where Z 0 is Z with the ærst row deleted, Q 0 is Q with the ærst column deleted and A 0 = 0 0 ::: ::: : 0 0 ::: 0 Theorem. The matrix ç P has a unique eigenvalue, ç ç, largest in modulus; it is equal to ç. The corresponding normalized left eigenvector is è a a ::: a d m, a d m z!è where! is the left eigenvector of P 0 from above and a j =! 0 èçq j + ç q j+ + :::+ç d m,j+ q d mè z = The normalized right eigenvector is zç where ç is the right eigenvector of P 0 as above. Proof. Due to the block matrix Z 0 of 0's, we have 5 dm, X j= a j : detè ç P, çiè = detèa 0, çiè detèp 0, çiè: But the ærst of these works out to be è,çè d m contributing dm 0's to the eigenvalues of ç P. Hence the non zero eigenvalues of ç P are the same as those of P 0. The remaining conclusions are veriæed by direct calculation. The convergence toward ç! occurs provided the process starts in the non min-bin states; such a starting vector æ has 0's for its ærst d m elements. Hence the dot product çæç = ^æç ==s. So acceleration and retention for P 0 is the same as that for ^P. Further, PrèXètè = optècèè = çæ t = ç t è=sèç! = çt s as before. 5

16 5. Choice of a Probability Measure for Restart In this section arguments in favor of using the uniform measure on the state space for restarting are presented. The sense in which this is a good measure is this: if a random restart method is to be used to ænd a global minimum of a function f, and if this function is unknown èpointwise evaluations can be made of f and possibly some of its derivatives or other functionals, but the general behavior of f on æ is not knownè and could be any one from a suæciently large collection of functions, then by using any other measure for restarting, the worst case is worse than using the uniform measure. Choosing the restart measure is a game theoretic problem. We start by assuming there is a way to rank the performance of the search process when applied to a speciæc cost function. This performance, æ, might be taken as reciprocal expected hitting time èso smaller times correspond to higher performancesè or possibly as reciprocal median hitting time. We assume æ, F, and g to be given leaving æ = æèç; Cè to depend on the choice of restart distribution and cost. Certainly the more the objective class is restricted, the more the restart distribution can be crafted. Since the calculation of expected hitting time depends in a complicated way on g, and therefore on speciæc cases, we will simplify matters by taking æèç; Cè to be ç 0 = çèm C è, the probability of restarting in the min-bin for cost C, which is an estimate of reciprocal expected hitting time. Next, there must be a criterion for interpreting performance diæerences. One possibility, and the one we will use, is mini-max; that is we select the restart distribution which maximizes the performance of the worst performing objective function. This is the right criterion against an intelligent adversary who would always choose the worst cost function for a given restart distribuition. An admissible cost function x Cost Fig.

17 Example. Let æ be the interval ë0; ë ç IR and take F to be all continuous, piecewise linear functions on æ such that: èè all local maxima lie on the x-axis strictly outside ë; ë, èè all line segments have slope equal to æ except the initial slope may be less than - and the ænal slope may exceed +, and èè at least one basin has depth at least. This class is suæciently large that every point x 0 æ is the unique global minimizer of some cost C F, see Fig.. Let g be gradient descent plus line search, then exactly one iteration is required to ænd the settling point of each basin. Here a mini-max restart distribution is any whose support is the subinterval ë; ë. Deænition 5. If æ is a ænite space, let the volume V of a set refer to its cardinality; if æ is a subset of Euclidean space then take V to be Euclidean volume. Theorem 8. Suppose æ is either ænite or a compact subset of IR n. If the objective class F is such that whenever some set S ç æ of a given volume V is the min-bin of some cost, then every subset of æ of volume V is also the min-bin of some cost, then uniform restart is mini-max optimal. Proof. Let V 0 = inffv : some cost has min-bin of size V g and let M V0 denote a min-bin of size V 0. If V 0 = 0 then the conclusion is true with a zero mini-max value. Otherwise, for the uniform measure ç and any other measure ç, Hence inf C çèm Cè ç çèm V0 è ç çèm V0 è=çèv 0 è: sup inf çèm Cè ç inf çèm Cè=çèV 0 è ç C C showing that ç is the mini-max solution with value çèv 0 è.. IIP parallelization A simple method for parallelizing stochastic search referred to as IIP parallel, meaning independent and identical processes, is studied in èshonkwiler and Van Vleck, 99è. Speciæcally, assume m identical copies of an algorithm such asi R are run independently of each other on m processors with only their random number seeds diæering. The m process expected hitting time, E m,isthe expectation of the ærst process to ænd the min-bin. That is, E m is the expectation of the random variable H = minfh ;H ;:::;H m g where H i is the min-bin hitting time for the ith process. The expectation E m is similarly deæned relative to a global minimizer.

18 Theorem 9. Prèmulti-process not in M by time tè ç sm ç mt as t!. Proof. Since the event H ç t is equivalent to the event H ç t and H ç t and æææand H m ç t; by independence we have the following, PrèH ç tè = èprèh ç tèè m ; t =;;:::: Therefore, from èè, for some integer K. Prèmulti-process not in M by time tè = Remark. As before, since = s mçmt + O è E m = t K ç jç j ç X t=0 ç t! èprèh ç tèè m ; " s çt +hètè ç jç j ç ç t è m the above leads to the following approximation for the expected hitting time for the IIP process E m ç X s m ç mt = s m, ç m : t=0 Remark. It is proved in èshonkwiler and Van Vleck, 99è that m processor IIP speed-up SU m, deæned as the ratio E =E m, satisæes SU m = s m,, çm, ç + Oè, çm è: Recall from Theorem that for I R, s é ; hence one sees that IIP is an especially eæective technique for I R.. Numerical Results In this section the foregoing results are demonstrated on two example problems, one discrete and the other continuous. The ærst is the Traveling Salesman Problem ètspè on 8

19 Basin for Tour è,,,,5,è Basin for Tour è,5,,,,è Fig. cities èkeeping the size small enough so that exact calculations will be possibleè and the other is the minimization of a one variable multi-modal function. For the TSP on cities there are! possible tours so that N = 0 is the number of points in æ. èpairing every tour with its reverse leads to 0 distinct potential solutions but we ignore these pairingsè. The cities, placed in a 0æ0 square, have locations 0:è,è, :è,è, :è,5è, :è8,è, :è,è, 5:è,9è, :è,8è. The space æ of tours is 0 è0,,,,,5,,0è; è0,,,,,,5,0è; è0,,,,5,,,0è; :::; 9 è0,,5,,,,,0è. The minimum tour length is. achieved by Tour è0,,,,5,,,0è and its reverse Tour 8. Tours always begin and end with city 0 so we omit it in the description henceforth. The improvement algorithm g is taken as ësuccessive city swap" deæned by: given a Tour, then èstep è start at the leftmost tour position i =,and èstep è swap the cities in positions i and i +, èstep è if the tour length has not increased, replace Tour by this modiæcation and go to step, otherwise increment i and èstep è if i = return Tour, otherwise go to step. Run Time Estimated Fundamental Polynomial coefficients r (limit=0.0) 0. r0 (limit=0.058) Iterations Fig. Among the basins deæned by g is the depth basin shown in Fig. èaè ending with Tour 9

20 = è; ; ; ; 5; è which is a local minimizer. Successive city swap breaks the symmetry between Tours and their reverse; the basin for Tour, which is the reverse of Tour, has depth and is shown in Fig.èbè. The topology induced by g consists of basins of which are goal basins. The goal basins comprise points so that ç 0 ==0 = 0:08. The length of the longest min-bin path is d m =. The longest non min-bin path has length d = 9 so the structure polynomial is of degree 0. The structure polynomial can be exactly calculated for this problem and is fèçè = 0 èç + 0ç + ç + 8ç +9ç 5 +ç +8ç +ç 8 +ç 9 +ç 0 è, : Its principal root is ç =:05, hence it follows from Theorem that retention ç = ç, = 0:95 and, from equation èè, that acceleration s = :0. As predicted by Theorem, séç. The estimated min-bin hitting time from equation è5è is.9. Run Time Estimate of eta.0 Run Time Estimated Min-bin Hitting Time.05 eta.0 0 E 8 (/s)/(-lambda) Iterations Runtime Estimate of ç Iterations Runtime Estimate of E Fig. Empirical data were acquired by executing the I R algorithm on this problem until a global optimizer was found 00 times. Keeping track of the depth data for each ènon minbinè restart, it is possible to make run time estimates of the coeæcients of the fundamental polynomial as discussed in x. Some of these estimates are shown in Fig.. From the resulting estimated fundamental polynomial, run time estimates of the principal root ç can be calculated; these are shown in Fig. èaè. Then using è5è, min-bin hitting time can be predicted as shown in Fig. èbè. Finally, by the independence of the IIP method, the data were used to simulate parallel processing; the speedup results are shown in Fig. 5. Linear speedup is also shown in the ægure for comparison. 0

21 Empirical Speedup to a Global Optimum 0 8 SU 5 8 Number of processes Fig y Shekel Function f x Fig. Shekel function example Our second problem, shown in Fig., is an example of a Shekel function and is the function f in ètíorn and Zelinskas, 989, pè. The function is given by fèxè =, X0 i= èk i èx, a i èè + c i ; 0 ç x ç 0; where the parameters a i, c i, and k i are as in Table.

22 Shekel Function Parameters a k c Table In this problem the global minimum is -. and occurs at x = 0:95. The min-bin is comprised of the range 0 éxé:8. For the improvement algorithm g in this problem we used Newton's Method for Unconstrained Minimization èsee èdennis and Schnabel, 98èè with the modiæcation that no Newton's step was allowed to exceed a pre-calculated maximum, MAXSTEP. This value was based on the reciprocal of the maximum curvature of the graph and assured that every downhill sequence remained in its restart bin. 0 Log CHD Plot for Test Function f Min-bin Hitting Time Log(chd) Fig. For each trial, the number of iterations required to ænd both the min-bin and the global minimum èwithin 0.è was observed. In order to do run time estimates, numbers of iterations in non goal basins was also observed. From the raw data consisting of 000

23 such trials, the average value of min-bin hitting time E = 85 and global minimum hitting time E = 0. In this example we used the data to determine the complementary hitting time distribution èchdè. Thus for each k, chdèkè is the fraction of runs requiring k or more iterations. Theoretically, the chd is asymptotically geometric, chdèkè ç s çk, : Therefore a logèchdè vs k, plot should tend to a straight line with slope logèçè. This plot, Fig., was indeed observed to be approximately aæne in its central region and least squares was used to calculate its slope. By this method, ç =0:99. Empirical Speedup to the Min-bin Empirical Speedup to a Global Optimum SU SU 5 8 Number of processes 5 8 Number of processes Fig. 8 The chd plot is not suitable for determining the acceleration factor s however as its intercept is too sensitive to the choice of aæne region selected. Instead the acceleration is estimated from speedup data to the min-bin. Speedup data for m parallel processes were obtained by the in-code parallel technique. That is, m separate trial solutions are maintained in each iteration of a single processor algorithm. The ærst of these to reach the miníbin æags a stop to the others. The hitting time is noted, and a new run initiated. The speedup plot is given in Fig. 8; linear speedup is shown for comparison. Since SU ç ms m,, logèsè is the slope of the plot of logèsu=mè versus m, This worked out to be s = :09. Since =ç = :008, as predicted, s é =ç. As in the TSP problem, run time estimations of the fundamental polynomial and its principal root were made. The result is shown in Fig. 9èaè. From that, min-bin hitting time is estimated as shown in Fig. 9.èbè References ëë Dimitri Bertsekas, John Tsitsiklis,Parallel and Distributed Computation, Prentice Hall, Englewood Cliæs è989è

24 Run Time Estimate of eta, Shekel Example eta Iterations Runtime Estimate of eta Run Time Estimate of Min-bin Hitting Time, Shekel Problem E Iterations Runtime Estimate of E Fig. 9 ëë G. Boender, and A. Rinnooy Kan, Bayesian Stopping Rules for Multistart Optimization Methods, Math. Programming, è98è, pp.59í80. ëë J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliæs, è98è ëë J. Galambos, The Asymptotic Theory of Extreme Order Statistics, Wiley, New York, è98è ë5ë J. Pickands, Moment Convergence of Sample Extremes, Ann. Math. Statist., 9è98è, pp ëë F. Schoen, Stochastic Techniques for Global Optimization: A Survey of Recent Advances, J. of Global Optimization, è99è, pp. 0í8. ëë R. Shonkwiler, and E. Van Vleck, Parallel speed-up of Monte Carlo methods for global optimization, to appear in the Journal of Complexity. ë8ë F. Solis, and R. Wets, Minimization by Random Search Techniques, Math. of Operations Research, è98è, pp. 9í0. ë9ë A. Tíorn, and A. Zelinskas, Global Optimization, Springer-Verlag, New York, è989è ë0ë R. Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Cliæs, NJ è9è.