434 CHAP. 8 NUMERICAL OPTIMIZATION. Powell's Method. Powell s Method

Transcription

1 434 CHAP. 8 NUMERICAL OPTIMIZATION Powell's Method Powell s Method Let X be an initial guess at the location of the minimum of the function z = f (x, x 2,...,x N ). Assume that the partial derivatives of the function are not available. An intuitively appealing approach to approximating a minimum of the function f is to generate the next approximation X by proceeding successively to a minimum of f along each of the N standard base vectors. The process generates the sequence of

2 SEC. 8.2 NELDER-MEAD AND POWELL S METHODS 435 y T 9 2 T 7 T T 8 T 6 T 5 T 4 T 2 T 3 T x 2 3 Figure 8. The sequence of triangles {T k } converging to the point (3, 2) for the Nelder-Mead method. points X = P, P, P 2,..., P N = X. Along each standard base vector the function f is a function of one variable. Thus the minimization of f requires the application of either the golden ratio or Fibonacci searches (Section 8.) on an interval over which the function is unimodal. The iteration is then repeated to generate a sequence of points {X k } k=. Unfortunately, the method is, in general, inefficient due to the geometry of multivariable functions. But the step from the point X to the point X is the first step of Powell s method. The essence of Powell s method is to add two steps to the process described in the preceding paragraph. The vector P N P represents, in some sense, the average direction moved during each iteration. Thus the point X is determined to be the point at which the minimum of the function f occurs along the vector P N P. As before, f is a function of one variable along this vector and the minimization requires an application of the golden ratio or Fibonacci searches. Finally, since the vector P N P was such a good direction, it replaces one of the direction vectors for the next iteration. The iteration is then repeated using the new set of direction vectors to generate a sequence of points {X k } k=. The process is outlined below. Let X be an initial guess at the location of the minimum of the function z = f (x, x 2,...,x N ), {E k =[ k ] :k =, 2,...,N} be the set of stan-

3 436 CHAP. 8 NUMERICAL OPTIMIZATION Table 8.6 Function Values at Various Triangles for Example 8.6 k Best point Good point Worst point f (.2,.) = 3.36 f (.,.8) =.6 f (.,.) =. 2 f (.8,.2) = 5.88 f (.2,.) = 3.36 f (.,.8) =.6 3 f (.8,.2) = 5.88 f (3.,.4) = 4.44 f (.2,.) = f (3.6,.6) = 6.24 f (I.8,.2) = 5.88 f (3.,.4) = f (3.6,.6) = 6.24 f (2.4, 2.4) = 6.24 f (.8,.2) = f (2.4,.6) = 6.72 f (3.6,.6) = 6.24 f (2.4, 2.4) = f (3.,.8) = 6.96 f (2.4,.6) = 6.72 f (2.4, 2.4) = f (3.,.8) = 6.96 f (2.55, 2.5) = f (2.4,.6) = f (3.,.8) = 6.96 f (3.5, 2.25) = f (2.55, 2.5) = f (3.,.8) = 6.96 f (2.825, 2.375) = f (3.5, 2.25) = dard base vectors, (9) U =[U U 2 U N ]=[E E 2 E N ], and i =. (i) Set P = X i. (ii) For k =, 2,...,N find the value of γ k that minimizes f (P k + γ k U k ) and set P k = P k + γ k U k. (iii) Set i = i +. (iv) Set U j = U j+ for j =, 2,...,N. Set U N = P N P. (v) Find the value of γ that minimizes f (P + γ U N ).SetX i = P + γ U N (vi) Repeat steps (i) through (v). Example 8.7. Use the process described in the preceding paragraph to find X and X 2 for the function [ f (x, y) ] = cos(x) + sin(y). Use the initial point X = (5.5, 2). Let U = and P = X = (5.5, 2). When i = the function f (P + γ U ) = f ((5.5, 2) + γ (, )) = f (5.5 + γ, 2) = cos(5.5 + γ ) + sin(2)

4 SEC. 8.2 NELDER-MEAD AND POWELL S METHODS 437 has a minimum at γ = Thus P = ( , 2). When i = 2 the function f (P + γ 2 U 2 ) = f (( , 2) + γ 2 (, )) = f ( , 2 + γ 2 ) = cos( ) + sin(2 + γ 2 ) has a minimum at γ 2 = Thus P 2 = ( , ). Set U 2 = (P 2 P ) and [ ] U = The function f (P + γ U 2 ) = f ((5.5, 2) + γ( , )) = f ( γ, γ) = cos( γ)+ sin( γ) has a minimum at γ = Thus X = ( , ). Set P = X. When i = the function f (P + γ U ) = f (( , ) + γ (, )) = f ( , γ ) = cos( ) + sin( γ ) has a minimum at γ = Thus P = ( , ). When i = 2the function f (P + γ 2 U 2 ) = f (( , ) + γ 2 ( , )) = f ( γ 2, γ 2 ) = cos( γ 2 ) + sin( γ 2 ) has a minimum at γ 2 = Thus P 2 = ( , ). Set U 2 = (P 2 P ) and [ ] U = The function f (P + γ U 2 ) = f (( , ) + γ(.85889,.796)) = f ( γ, γ) = cos( γ)+ sin( γ) has a minimum at γ = Thus X 2 = ( , ). The function f (x, y) = cos(x) + sin(y) has a relative minimum at the point P = (π, 3π/2). The graph of f is shown in Figure 8.. Figure 8.2 shows a contour plot of the function f and the relative positions of the points X, X, and X 2.

5 438 CHAP. 8 NUMERICAL OPTIMIZATION y x Figure 8. The graph of f (x, y) = cos(x) + sin(y) x 2 x y x Figure 8.2 The contour graph of f (x, y) = cos(x) + sin(y). x In step (iv) of the previous process the first vector U was discarded and the average direction vector P N P was added to the list of direction vectors. In fact, it would be better to discard the vector U r along which the greatest decrease in f occurred. It seems reasonable that the vector U r is a large component of the average direction vector U N = P N P. Thus, as the number of iterations increase, the set of direction vectors will tend to become linearly dependent. When the set becomes linearly dependent one or more of the directions will be lost and it is likely that the set of points {X} k= will not converge to the point at which the local minimum occurs. Furthermore, in step (iv) it was assumed that the average direction vector represented a good direction in which to continue the search. But that may not be the case. Outline of Powell s Method (i) Set P = X i. (ii) For k =, 2,...,N find the value of γ k that minimizes f (P k + γ k U k ) and set P k = P k + γ k U k. (iii) Set r and U r equal to the maximum decrease in f and the direction of the maximum decrease, respectively, over all the direction vectors in step (ii). (iv) Set i = i +. (v) If f (2P N P ) f (P ) or 2( f (P ) 2 f (P N )+ f (2P N P ))( f (P ) f (P N ) r) 2 r( f (P ) f (2P N P )) 2, then set X i = P N and return to step (i). Otherwise, go to step (vi).

6 SEC. 8.2 NELDER-MEAD AND POWELL S METHODS 439 (vi) Set U r = P N P. (vii) Find the value of γ that minimizes f (P + γ U r ).SetX i = P + γ U r. (viii) Repeat steps (i) through (vii). If the conditions in step (v) are satisfied, then the set of direction vectors is left unchanged. The first inequality in step (v) indicates that there is no further decrease in the value of f in the average direction P N P. The second inequality indicates that the decrease in the function f in the direction of greatest decrease U r was not a major part of the total decrease in f in step (ii). If the conditions in step (v) are not satisfied, then the direction of greatest decrease U r is replaced with the average direction from step (ii); P N P. In step (vii) the function is minimized in this direction. Stopping criteria based on the magnitudes X i X i or f (X i ) are typically found in steps (v) and (vii).

7 Numerical Methods Using Matlab, 4 th Edition, 24 John H. Mathews and Kurtis K. Fink ISBN: Prentice-Hall Inc. Upper Saddle River, New Jersey, USA