A Theory-Based Termination Condition for Convergence in Real- Parameter Evolutionary Algorithms

Size: px

Start display at page:

Download "A Theory-Based Termination Condition for Convergence in Real- Parameter Evolutionary Algorithms"

Ophelia Henderson
6 years ago
Views:

1 A Theory-Based Termination Condition for Convergence in Real- Parameter Evolutionary Algorithms Kalyanmoy Deb Koenig Endowed Chair Professor Department of Electrical and Computer Engineering Michigan State University East Lansing, USA

KKT Proximity Measure (KKTPM) for single and multi-objective optimization!

2 Overview! Most Evolutionary applications in search and optimization! Theoretical Optimality conditions! Karush-Kuhn-Tucker (KKT) conditions! KKT conditions for near-kkt points! Approximate KKT points! KKT Proximity Measure (KKTPM) for single and multi-objective optimization! Results on standard test problems! Termination criterion! Practicalities of implementation! A fast and approximate method! Conclusions 2

Evolutionary Optimization (EO): A Mimicry of Natural Evolution and Genetics begin Solution Representation t := 0; // generation counter Initialization P(t); Evaluation P(t); while not TERMINATION do

3 Evolutionary Optimization (EO): A Mimicry of Natural Evolution and Genetics begin Solution Representation t := 0; // generation counter Initialization P(t); Evaluation P(t); while not TERMINATION do P'(t) := Selection (P(t)); P''(t) := Variation (P'(t)); Evaluation P''(t); P(t+1):= Survivor (P(t),P''(t)); t := t+1; od end Initial population Recombination Mutation Optimum! " Mean approaches optimum! " Variance reduces! " No need of gradients 3

4 Mitsubishi Regional Jet (MRJ) Adopted in Practice!" Nose of Shinkansen (Bullet Train in Japan) N700 Series KONE elevator s control system ( countries/en_mp/documents/brochures/kone%20alta.pdf) 4

5 Termination Condition for EOs Maximum # of generations achieved Pragmatic, but no guarantee of near-optimum Stagnated condition Past τ gen, f improvement is ε Average and best performance is close A target is achieved Pragmatic, but requires knowledge Since none guarantees near-optimum, run EO multiple times Can we take help of Optimization Theory? T o g e t h e r 5

6 Hybrid Methods are Popular EA followed by local search Mutation operator of an EA uses a local search How do we even terminate a local search Optimality conditions satisfied? Are opt. conditions regular near optimum? Neighbor, monotonic along a line? Surprising results follow! optimum 6

7 x2 Unconstrained Single-Objective Problems Problem: Minimize f(x) Optimality condition: x*={x Grad f(x) = 0 & H(x)=+ve definite} Himmelblau Function Norm of Gradient Vector x1 (4 minima) x1 Error(x) has x Error(x) = Grad f(x) many zeros, Cannot be used alone

8 Monotonicity of Error Error reduced to zero locally For unconstrained Problems, norm. can be used a Proximity Measure locally Norm of Gradient Vector at y=2 Optimal point x=3 Norm If +ve def. check on H(x) is made, four minima are found x1 Regular, but not so in the entire space

9 Constrained Optimization Problem! " Decision variables: x = (x 1, x 2,, x n )! " Constraints restrict some solutions to be feasible Min. f(x) s.t. g j (x) # 0 j = 1,2,,J h k (x)=0 k = 1,2,,K x L i $ x i $ x U i i = 1,2,,n! " Equality and inequality constraints! " Minimum of f(x) need not be constrained minimum! " Constraints can be non-linear 9

10 Karush-Kuhn-Tucker (KKT) Conditions Variable bounds are treated as inequality constraints Eqm. cond. A triplet (x, u, v) that sa2sfies above condi2ons is a KKT point, when certain constraint qualifica.on is sa2sfied } Feasibility cond. Complementary Slackness condition Non-negativity of Lagrange multiplier 10

11 Geometrical Significance of " 99:"(<=>"6578%257?" - = - KKT Conditions Inequality constraints only constraint is not satisfied objective value reduces All u j must be +ve or zero "!"$(8,6(8"54@(62-(" 15',257"65=(1"57'A" B$5="%7B(01%4'("15'71>" "C%D('A"6078%80#("B5$" =%7%=,=" both constr. infeasible A KKT point need not be a minimum 11

12 Karush-Kuhn-Tucker (KKT) Necessity Theorem Let f, g, and h are differen.able func.ons and x* be a feasible solu.on. Let I={j g j (x*)=0}, a set of ac.ve constraints. Furthermore, grad g j (x*) for j in I and grad h k (x*) are linearly independent. If x* is an op.mal solu.on to the NLP, then there exists a (u*, v*) such that triplet (x*, u*, v*) solves the Kuhn- Tucker condi.ons (x* is a KKT point) Linear Independence Constraint Qualifica2on (LICQ) condi2on: (strong CQ) Implies certain regularity condi2ons Most prac2cal problems sa2sfy the condi2on 12

13 Main Results KKT necessity theorem can be used to identify points that are not optimal If a feasible point satisfies CQ condition and if it is not a KKT point, it cannot be a minimum If a feasible point satisfies CQ condition and if it is a KKT point, it may or may not be minimum Need more conditions to confirm optimality If a feasible point does not satisfy CQ condition, it may or may not be a KKT point or minimum 13

Approximate KKT Point " x is "-KKT KT point, if given

satisfied " Proven in Dutta et al. (2012) in J.

, Deb, K., Tulshyan, R. and Arora, R. (2013).

14 Approximate KKT Point " x is "-KKT KT point, if given ">0, there exists u i #0 for all constraints, such that Eqm. Cond. not satisfied but within " Complementary slackness satisfied " Proven in Dutta et al. (2012) in J. Global Optimization (Springer Journal) Dutta, J., Deb, K., Tulshyan, R. and Arora, R. (2013). Approximate KKT points and a proximity measure for termination. Journal of Global Optimization, 56(4),

15 KKT Error Metric A feasible iterate x k found by any opt. algorithm Define KKT Error as Extent of violation of Equilibrium Condition for a feasible solution: I(x k ): active constraint set Then define: At KKT point, KKT Error is zero How about at a near-kkt point? Surprising results 15

16 A Quadratic Problem with Linear Constraints Along x=y Acceptable Along x+y=3 NOT Acceptable! 16

17 Modified ε- KKT Point Discontinuity: u i =0 slightly away from boundary, but u i 0 at boundary -> Sudden change x is a Modified ε-kkt point, if given ε>0, there exists u i 0 for all constraints, such that Both Eqm. and Complementary slackness cond. are within bounds Similar theorem proven for smooth and non-smooth problems leading to approximate KKT point Dutta, J., Deb, K., Tulshyan, R. and Arora, R. (2013). Approximate KKT points and a proximity measure for termination. Journal of Global Optimization, 56(4),

18 Proposed KKT Proximity Metric (KKTPM) For a feasible iterate x k, solve # of constraints Compute KKTPM = ε * k By-product: Get u i * for every constraint Corresponding KKTPM for non-smooth problems defined with sub-differentials of f and g. 18

19 A Test Problem KKT Error Metric KKT Error suddenly zero A C Along AC KKT Proximity Metric A Along AC KKTPM smoothly goes to zero C 19

20 G-Series Constrained Test Problems: RGA Results g01 g07 Remarkable, as RGAs do not use any gradient information KKTPM can be used as termination condition g04 g09 20

21 Comparison with KNITRO KNITRO OptErr requires to find u i Similar reduction in performance metrics 21

22 Computed Optima and Lagrange Multipliers G-series problems Settled once and for all 22 Refer to J of Global Optimization paper

23 Multi-Objective Optimization: Handling multiple conflicting objectives! " Bueno, Bonito, Barato? Good, nice, cheap 3B A doomed car " Multiple solutions are optimal " How to apply KKTPM? A Dominated car Keynote Talk at EVOLVE Conference, 23

$such that f(x)-f(x*) " -R M + \{0}$ Domination-based:!

" x (1) is strictly better than x (2) in

24 Which Solutions are Optimal? Mathematical Definition: A solution x* " X is Pareto-optimal if there is no x"x such that f(x)-f(x*) " -R M + \{0} Domination-based:! " x (1) dominates x (2), if! " x (1) is no worse than x (2) in all objectives! " x (1) is strictly better than x (2) in at least one objective! " 1 gets dominated by 3 Keynote Talk at EVOLVE Conference, Min. f2 Objective space Objective space Min. f1 24

25 Pareto-Optimal Solutions! " P =Nondominated(P)! " Solutions which are not dominated by any member of the set P! " O(N log N) algorithms exist! " Pareto-Optimal set = Non-dominated(S)! " A number of solutions are optimal Efficient front Keynote Talk at EVOLVE Conference, 25

26 Evolutionary Multi-Objective Optimization (EMO): Principle Step 1 : Find a set of Pareto-optimal solutions Step 2 : Choose one from the set Keynote Talk at EVOLVE Conference, 26

27 Evolution of EMO! " Early penalty-based approaches! " VEGA (1984)! " Goldberg's (1989) suggestion! MOGA, NSGA, NPGA ( ) used Goldberg's suggestion! Elitist EMO (SPEA, NSGA-II, PAES, MOMGA etc.) ( Present) EMOO Web site (as of Jan 2010) 1,534 journal, 2,266 conference 226 PhD theses VEGA MOGA NPGA NSGA NSGA-II SPEA Keynote Talk at EVOLVE Conference, Iasi, Romania, 2015 MOSES Weighted L p norm 27

28 Elitist Non-dominated Sorting Genetic Algorithm (NSGA-II)! NSGA-II! Modular! No additional parameter! Fast! Commercialization:! MO-Sherpa (RCT)! ModeFrontier (Estico)! isight (Engeneous)! VisualDoc (Vanderplatts) (Deb et al., IEEE TEC 2002) Fast-Breaking Paper in Engineering by ISI Web of Science (Feb 04), Thomson Citation Laureate Award 2006, Current Classic and Most Highly Cited Paper (14,500 GS citations) Keynote Talk at EVOLVE Conference, 28

29 NSGA-II Simulation on an Unconstrained Problem " Parallel search " Multiple solutions in a single run Keynote Talk at EVOLVE Conference, 29

30 NSGA-II on a Constrained Problem Keynote Talk at EVOLVE Conference, 30

31 Many-Objective Optimization One of the Main Thrusts in EMO Today! Multi-objective: {2,3} objectives! Many-objective: >3 objectives! EMO difficulties for many-obj. problems: 1." Large fraction of population gets dominated 2." Maintaining diversity difficult 3." Recombination operator inefficient 4." Representation of PO front requires exponentially more points 5." Performance measures difficult to compute 6." Visualization is difficult Keynote Talk at EVOLVE Conference, 31

32 NSGA-III (IEEE TEC 2014) An EMO for Many-Obj. Optimization " Guided search through supplied reference pts " Similar to NSGA-II selection " Niching through selecting points close to reference lines " Normalization " Association " Niching No additional parameter needed Keynote Talk at EVOLVE Conference, 32

33 Some Results of NSGA-III 3-obj. Problem (IEEE TEC August 2014) 15-obj. Problem Water Problem A few Solutions Constrained Problem Keynote Talk at EVOLVE Conference, Constrained Problem 33

34 NSGA-III on ZDT1 (2 Objectives) Keynote Talk at EVOLVE Conference, 34

35 NSGA-III on DTLZ2 (3 Objectives) Keynote Talk at EVOLVE Conference, 35

36 Land Use Management 315 independently manageable paddocks, each having around 100 choices in any of 10 years ( ) 10 solutions ~10 82 atoms in the universe After 10 Years

37 Variation of 14 Objectives! "sheep substantively lower;! "dairy drastically reduced;! "more and less frequent forest harvesting;! "substantial increase in beef cattle.

38 KKT Proximity in Evolutionary Multi-objective Optimization As points move towards efficient front, KKTPM must reduce KKTPM must have similar values for points equidistant from front 38

39 KKT Optimality Conditions for MO Problems Equilibrium condition x k is supplied Constraint satisf. Compl. slackness Non-neg. of mult. Find λ*,u* for minimum KKT Error: 39

40 KKT Error Metric for Multi-objective Problems An example: Solve: using Matlab s fmincon() 40

41 KKT Error on P1 KKT Error increases towards efficient front KKT Error cannot be a metric 41

42 KKT Proximity Metric for MO Scalarize MO problem using ASF formulation: Choose z and w ASF(KG)=ASF(GH) Minimize ASF finds O Idea borrowed from MCDM literature

43 KKTPM (cont.) z: utopian point w: If F is on efficient front, it is minimum ASF Otherwise, KKT error will not be zero 43

44 KKT Proximity Metric (cont.) Smoother ASF: o Recall for 1- obj: o KKTPM=ε k * 44

45 KKT Proximity Metric (cont.) Treat ASF as single-obj. problem o Solve and find op2mal ε k * 1. Relax compl. slackness cond. 2. Add a penalty o Define KKTPM: Use Matlab s fmincon() to solve it 1 linear and 1 quadratic constraints 45

46 KKT Proximity Metric on P1 Smooth reduction to zero o Contour parallel to PO front 46

47 ZDT1 Test Problem with NSGA-II KKTPM Surface (Zitzler, Deb and Thiele, 2000) NSGA-II Populations N=100 KKTPM parallel to PO front Population converges as a front 47

48 ZDT1 Problem (cont.) KKTPM reduces with closeness to PO front Correlation to distance (R=0.993) 48

49 ZDT2 Test Problem with NSGA-II (Zitzler, Deb and Thiele, 2000) KKTPM Surface NSGA-II Populations N=100 KKTPM parallel to PO front KKTPM is computed for every ND point at every generation of NSGA-II 49

50 ZDT4 Test Problem with NSGA-II (Zitzler, Deb and Thiele, 2000) " ZDT4 has multiple local efficient fronts " Hence more difficult to solve " KKTPM plots demonstrate this aspect " Takes 200 generations to converge KKTPM Measure NSGA-II Populations N=100 Avg g() Smallest 1st Quartile Median 3rd Quartile Largest Avg. g() Average g() Generation Number 50

DTLZ1 Problems with NSGA-III KKTPM Variations

4 3-obj Smallest 1st Quartile Median 3rd

g() 250 200 150 100 Average g() 5-obj 10-obj

51 DTLZ1 Problems with NSGA-III KKTPM Variations (NSGA-III) KKTPM Measure obj Smallest 1st Quartile Median 3rd Quartile Largest Avg. g() Average g() 5-obj 10-obj (Deb et al., 2002) Generation Number

DTLZ2 Problems with NSGA-III KKTPM Variations (NSGA-III) KKTPM Measure 1 0.8 0.6 0.4 0.

52 DTLZ2 Problems with NSGA-III KKTPM Variations (NSGA-III) KKTPM Measure obj Smallest 1st Quartile Median 3rd Quartile Largest Avg. g() Average g() 5-obj 10-obj (Deb et al., 2002) Generation Number

53 Constrained MO Problems TNK (Deb, 2001, Wiley Book) BNH KKTPM Measure Smallest 1st Quartile Median 3rd Quartile Largest Generation Number

54 Problem SRN with NSGA-II NSGA-II convergence is poor (Deb, 2001, Wiley Book) Gen=250 Local search method is being pursued 54

55 Problem OSY with NSGA-II 25% points did not converge until 250 gen. Local search to speed up EMO runs (Deb, 2001, Wiley Book) Local search method is being pursued 55

56 Initial Results on Local Search Based Methods " With and without local search ZDT1 " KKTPM identifies non-po solutions " LS helps find true PO points KKTPM allows differential treatment of ND solutions 56

57 Engineering Design Problems with NSGA-II Welded Beam Design (Deb and Jain, 2014, NSGA-III) Car Side Impact Design 57

58 KKTPM as Termination Criterion Set a threshold Terminate when it is met Consistent results!

Pros and Cons of KKTPM " Advantages: " Guarantees convergence to theoretical optimum " Scientific termination condition " Address troubled areas with local search " Applicable to classical methods

59 Pros and Cons of KKTPM " Advantages: " Guarantees convergence to theoretical optimum " Scientific termination condition " Address troubled areas with local search " Applicable to classical methods as well " Disadvantages: " Computationally demanding (use after every 10 generations or so) " Discuss propose a Direct method next " Applicable to differentiable problems (use numerical gradients) 59

60 A Computationally Faster Method "" EH75$("1(6578"6571#$0%7#" "" T%78"=%7%=,="5B"3$1#"6571#$0%7#")O R." #" Q5',257"5B"0"'%7(0$"1A1#(="5B"(<,02571" "" F01("M?"E8(7260'"#5"O 5&#" "" F01("U?")R%V6,'#." #" 078"O K "(01A"#5"65=&,#(" #" O!8@ W"O K" W"O R" #" O!8@ W"O 5&#" W"O R" "" X12=0#(8"O (1# "N")O!8@ YO K" YO R.Z[ " " 60

61 A Test Case Single-var. problem: ε* =0.0 ε est =0.0 x=0.5 is the optimum x=1.0 is not an optimum ε* = ε est = x=1.0 61

62 Comparison with Direct Methods Median KKTPM Optimal is in between Adjusted and Direct Estimated is closer 1.2 sec versus 62.2 sec (Zitzler, Deb and Thiele, 2000) ZDT1 62

63 More Comparisons on ZDT Problems ZDT2 ZDT4 Estimated is closer to Optimal (Zitzler, Deb and Thiele, 2000) 63

64 More Comparisons on DTLZ Problems DTLZ1_3 DTLZ1_10 (Deb et al., 2002) Estimated is closer to Optimal 64

65 More Comparisons on DTLZ Problems DTLZ2_5 DTLZ5_3 (Deb et al., 2002) Estimated is closer to Optimal 65

66 Constrained Bi-Objective Problems SRN (Deb, 2001, Wiley Book) BNH 66

67 Practical Problems (Deb and Jain, 2014, NSGA-III) CAR WELD 67

68 Single-Objective Constrained Optimization (Direct vs. opt.) True optimum solution added as the final entry g01 RGA Method g07 RGA converges well Stuck True Opt. RGA does not converge True Opt. 68

69 More Single-Objective Constrained Optimization RGA Method g10 g18 g24 Stuck Poor convergence Good convergence Good convergence Good convergence Easy problem g02 g04 Good convergence Stuck g08 Good convergence Stuck

70 Comput. Time " Approx. methods are much faster " Not much loss in accuracy " Good Trade-off S&#>" X12=0#(8" R%$(6#" U\[>[" M>]" M>^" M>^" M>]" M[_>]" M>]" M>^" M>^" M>^" U`aa>b" ]>\" ]>\" ]>M" ]>^" U\>U" \>U" \>U" \>U" \>U" ][>]" \>^" \>^" \>^" \>^" [[>b" \>U" \>U" \>U" \>U" ]U>]" M>U" M>M" M>M" M>M" ^[>_" M>M" M>\" M>\" M>\" ^]>[" M>U" M>M" M>U" M>U" MU]>b" M>]" M>^" M>^" M>]" [\>]" \>U" \>U" \>U" \>U" UMb>U" M>U" M>U" M>U" M>[" a`\>_" b>_" b>m" b>[" b>^" b`_m>`" [\>]" [\>b" [\>a" [M>\" M[^>U" \>_" \>`" \>_" \>_" M\ba>]" [>^" [>^" [>^" [>b" M``>^" M>[" M>[" M>b" M>[" U\[>]" M>M" M>\" M>\" M>\" Ub^>]" M>M" M>\" M>\" M>\" b>b" \>M" \>\" \>\" \>\" MU]M>[" a>]" a>u" a>]" a>]" am>_" \>^" \>b" \>b" \>^" M\ab>U" U>`" U>]" U>]" U>a" UU]>]" U>M" U>M" U>U" U>U"

71 Conclusions! EMO uses stochastic search principles! No convergence proof in finite time! Developed performance metric for convergence based on KKT optimality conditions! KKT proximity metric implemented with ASF scalarization (MCDM) method! Demonstrated to work well on many standard test and engineering problems! Need to couple with a diversity measure! Such studies (theory, MCDM and EMO) should bring respect to EO and EMO fields 71

72 Relevant Papers Deb, K. and Abouhawwash. M. (October, 2014). An Optimality Theory Based Proximity Measure for Set Based Multi- Objective Optimization. COIN Report No (COIN Website, MSU) Deb, K., Abouhawwash, M., and Dutta, J. (2015). A KKT Proximity Measure for Evolutionary Multi-objective and Manyobjective Optimization. Proceedings of Eighth Conference on Evolutionary Multi-Criterion Optimization (EMO-2015). Springer. Deb, K. and Abouhawwash, M. (May, 2015). A Computationally Fast and Approximate Method for Karush- Kuhn-Tucker Proximity Measure. COIN Report No

A Computationally Fast and Approximate Method for Karush-Kuhn-Tucker Proximity Measure

A Computationally Fast and Approximate Method for Karush-Kuhn-Tucer Proximity Measure Kalyanmoy Deb and Mohamed Abouhawwash 1 Department of Electrical and Computer Engineering Computational Optimization