Three Steps toward Tuning the Coordinate Systems in Nature-Inspired Optimization Algorithms

Transcription

1 Three Steps toward Tuning the Coordinate Systems in Nature-Inspired Optimization Algorithms Yong Wang and Zhi-Zhong Liu School of Information Science and Engineering Central South University

2 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 2

3 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 3

4 The Main Paradigms of NIOAs Genetic algorithm (GA) Evolution strategy (ES) Evolutionary programming (EP) Ant colony optimization (ACO) Differential evolution (DE) Particle swarm optimization (PSO) 4

5 The Main Paradigms of NIOAs Genetic algorithm (GA) Evolution strategy (ES) Evolutionary programming (EP) Ant colony optimization (ACO) Differential evolution (DE) Particle swarm optimization (PSO) the most popular paradigms in current studies 4

6 Differential Evolution (1/4) Differential evolution (DE) was proposed by Storn and Price in DE includes three main operators, i.e., mutation, crossover, and selection. R. Storn and K. Price. Differential evolution a simple and efficient adaptive scheme for global optimization over continuous spaces, Berkeley, CA, Tech. Rep. TR , K. Price, R. Storn, and J. Lampinen. Differential Evolution A Practical Approach to Global Optimization. Berlin, Germany: Springer-Verlag,

7 Differential Evolution (2/4) 6

8 Differential Evolution (2/4) 6

9 Differential Evolution (2/4) three main operators 6

10 Differential Evolution (2/4) three main operators mutation + crossover = trial vector generation strategy 6

11 Differential Evolution (3/4) A classic DE version: DE/rand/1/bin base vector differential vector mutation scaling factor 7

12 Differential Evolution (3/4) A classic DE version: DE/rand/1/bin base vector differential vector mutation scaling factor crossover control parameter crossover 7

13 Differential Evolution (3/4) A classic DE version: DE/rand/1/bin base vector differential vector mutation scaling factor crossover control parameter crossover selection 7

14 Differential Evolution (3/4) A classic DE version: DE/rand/1/bin base vector differential vector mutation scaling factor crossover control parameter crossover selection Remark: in the crossover, each variable in DE is updated independently 7

15 Differential Evolution (4/4) Schematic diagram to illustrate DE/rand/1/bin g x i the target vector the mutant vector g g r2 r3 g v i F ( x x ) x g r 2 x g r1 base vector perturbed vectors x g r3 x

16 Differential Evolution (4/4) Schematic diagram to illustrate DE/rand/1/bin the triangle denotes the trial vector g u i g x i the target vector the mutant vector g g r2 r3 g v i F ( x x ) x g r 2 x g r1 base vector perturbed vectors x g r3 x

17 Particle Swarm Optimization (1/3) The movement equations of the classic PSO the personal historical best experience the entire swarm s best experience g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j g+ 1 g g+ 1 i, j = i, j+ i, j x x v the neighborhood s best experience J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

18 Particle Swarm Optimization (1/3) The movement equations of the classic PSO the personal historical best experience the entire swarm s best experience g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j velocity update equation g+ 1 g g+ 1 i, j = i, j+ i, j x x v the neighborhood s best experience J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

19 Particle Swarm Optimization (1/3) The movement equations of the classic PSO the personal historical best experience the entire swarm s best experience g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j velocity update equation g+ 1 g g+ 1 i, j = i, j+ i, j x x v position update equation the neighborhood s best experience J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

20 Particle Swarm Optimization (1/3) The movement equations of the classic PSO g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j velocity update equation g+ 1 g g+ 1 i, j = i, j+ i, j x x v position update equation cognition part J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

21 Particle Swarm Optimization (1/3) The movement equations of the classic PSO g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j velocity update equation g+ 1 g g+ 1 i, j = i, j+ i, j x x v position update equation cognition part social part J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

22 Particle Swarm Optimization (1/3) The movement equations of the classic PSO g + 1 g g g g i, j = i, j + g 1 1, j pbesti, j i, j + 2r2, j j xi, j v v cr ( x ) c ( gbest ) or v + = v + cr ( x ) + c r ( lbest x ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2 2, j i, j i, j velocity update equation g+ 1 g g+ 1 i, j = i, j+ i, j x x v position update equation cognition part social part Remark: in the velocity update, each variable in PSO is updated independently J. Kennedy and R. C. Eberhart. Particle swarm optimization. in Proc. IEEE Int. Conf. Neural Networks, 1995, pp

23 Particle Swarm Optimization (2/3) The principle of the movement equations v + = v + cr ( x ) + c ( gbest ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2r2, j j xi, j g+ 1 g g+ 1 i, j = i, j+ i, j x x v g pbest i, j g 1 x + i, j g 1 v + i, j g gbest j g v i, j g x i, j the global version 10

24 Particle Swarm Optimization (2/3) The principle of the movement equations v + = v + cr ( x ) + c ( gbest ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2r2, j j xi, j g+ 1 g g+ 1 i, j = i, j+ i, j x x v g pbest i, j g 1 x + i, j g 1 v + i, j g gbest j g v i, j g x i, j the global version 10

25 Particle Swarm Optimization (2/3) The principle of the movement equations v + = v + cr ( x ) + c ( gbest ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2r2, j j xi, j g+ 1 g g+ 1 i, j = i, j+ i, j x x v g pbest i, j g 1 x + i, j g 1 v + i, j g gbest j g v i, j g x i, j the global version 10

26 Particle Swarm Optimization (2/3) The principle of the movement equations v + = v + cr ( x ) + c ( gbest ) g 1 g g g g g i, j i, j 1 1, j pbesti, j i, j 2r2, j j xi, j g+ 1 g g+ 1 i, j = i, j+ i, j x x v g pbest i, j g 1 x + i, j g 1 v + i, j g gbest j g v i, j g x i, j the global version 10

27 Particle Swarm Optimization (3/3) The framework of PSO 11

28 Particle Swarm Optimization (3/3) The framework of PSO 11

29 Particle Swarm Optimization (3/3) The framework of PSO 11

30 Particle Swarm Optimization (3/3) The framework of PSO 11

31 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 12

32 What is the Original Coordinate system? 13

33 What is the Original Coordinate system? 13

34 What is the Eigen Coordinate System? The Eigen coordinate system is established by the columns of an orthogonal matrix B, which comes from the Eigen decomposition of the covariance matrix C: C = BD 2 B T 14

35 What is the Eigen Coordinate System? The Eigen coordinate system is established by the columns of an orthogonal matrix B, which comes from the Eigen decomposition of the covariance matrix C: C = BD 2 B T an orthogonal matrix 14

36 What is the Eigen Coordinate System? The Eigen coordinate system is established by the columns of an orthogonal matrix B, which comes from the Eigen decomposition of the covariance matrix C: C = BD 2 B T an orthogonal matrix a diagonal matrix 14

37 What is the Eigen Coordinate System? The Eigen coordinate system is established by the columns of an orthogonal matrix B, which comes from the Eigen decomposition of the covariance matrix C: C = BD 2 B T an orthogonal matrix a diagonal matrix the transposed matrix of B 14

38 What is the Eigen Coordinate System? The Eigen coordinate system is established by the columns of an orthogonal matrix B, which comes from the Eigen decomposition of the covariance matrix C: C = BD 2 B T an orthogonal matrix a diagonal matrix the transposed matrix of B Each column of B is an eigenvector of C, and each diagonal element of D is the square root of an eigenvalue of C. 14

39 How to Establish the Eigen Coordinate System pop x1,1 x1,2 x1, D = xnp,1 xnp,2 x NP, D NP individuals Matlab Code xmean = mean(pop); C = 1/(NP-1)*(pop - xmean(ones(np,1), :))'*(pop - xmean(ones(np,1), :)); C = triu(c) + transpose(triu(c,1)); % enforce symmetry [B,D] = eig(c); 15

40 How to Establish the Eigen Coordinate System pop x1,1 x1,2 x1, D = xnp,1 xnp,2 x NP, D NP individuals Matlab Code xmean = mean(pop); C = 1/(NP-1)*(pop - xmean(ones(np,1), :))'*(pop - xmean(ones(np,1), :)); C = triu(c) + transpose(triu(c,1)); % enforce symmetry [B,D] = eig(c); 15

41 The Advantages of the Covariance Matrix (1/6) Covariance matrices have an appealing geometrical interpretation: they can be uniquely identified with the (hyper-)ellipsoid N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, vol. 9, no. 2, pp ,

42 The Advantages of the Covariance Matrix (2/6) The Eigen decomposition is denoted by C = BD B 2 T 17

43 The Advantages of the Covariance Matrix (2/6) The Eigen decomposition is denoted by C = BD B 2 T D = δ I 2 C= δ I the ellipsoid is isotropic a positive number the identity matrix 17

44 The Advantages of the Covariance Matrix (2/6) The Eigen decomposition is denoted by C = BD B 2 T D = δ I 2 C= δ I the ellipsoid is isotropic a positive number the identity matrix B = I 2 C= D the ellipsoid is axis parallel oriented the identity matrix a diagonal matrix 17

45 The Advantages of the Covariance Matrix (2/6) The Eigen decomposition is denoted by C = BD B 2 T D = δ I 2 C= δ I the ellipsoid is isotropic a positive number the identity matrix B = I 2 C= D the ellipsoid is axis parallel oriented the identity matrix a diagonal matrix B C the ellipsoid can be adapted to suit the contour lines of the objective function the new principal axes of the ellipsoid correspond to the columns of B 17

46 The Advantages of the Covariance Matrix (3/6) the Hessian matrix Setting on is equivalent to optimizing the isotropic function, where with. 18

47 The Advantages of the Covariance Matrix (3/6) the Hessian matrix Setting on is equivalent to optimizing the isotropic function, where with. Remark 1: The optimal covariance matrix equals to the inverse Hessian matrix 18

48 The Advantages of the Covariance Matrix (3/6) the Hessian matrix Setting on is equivalent to optimizing the isotropic function, where with. Remark 1: The optimal covariance matrix equals to the inverse Hessian matrix Remark 2: The objective of covariance matrix adaptation is to approximate the inverse Hessian matrix 18

49 The Advantages of the Covariance Matrix (4/6) minimize : f( x, x ) = x + x, -5 < x < 5, -5 < x < minimize x + minimize x, -5 < x < 5, -5 < x < isotropic function 19

50 The Advantages of the Covariance Matrix (4/6) minimize : f( x, x ) = x + x, -5 < x < 5, -5 < x < minimize x + minimize x, -5 < x < 5, -5 < x < isotropic function 19

51 The Advantages of the Covariance Matrix (5/6) Covariance Matrix Adaptation isotropic function 20

52 The Advantages of the Covariance Matrix (6/6) In general, the covariance matrix C is constructed and adapted according to the feedback information resulting from the search process. 21

53 The Advantages of the Covariance Matrix (6/6) In general, the covariance matrix C is constructed and adapted according to the feedback information resulting from the search process. Therefore, unlike the original coordinate system, the Eigen coordinate system is dynamic throughout the search process, with the aim of suiting the function landscape. 21

54 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 22

55 Motivation The commonly used crossover operators of DE are implemented in the original coordinate system x 2 v ig, x ig, x 1 Y. Wang, H.-X. Li, T. Huang, and L Li. Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Applied Soft Computing, vol. 18, pp , (CoBiDE) 23

56 CoBiDE (1/6) Covariance matrix learning 24

57 CoBiDE (2/6) An explanation x 2 v ig, covariance matrix learning x ig, x 1 x 2 x ig, x 1 v ig, x 2 x 1 25

58 CoBiDE (3/6) the optimal solution 26

59 CoBiDE (4/6) The first question: which individuals should be chosen for computing the covariance matrix 27

60 CoBiDE (4/6) The first question: which individuals should be chosen for computing the covariance matrix Compute the covariance matrix C via the top ps*np individuals in the current population 27

61 CoBiDE (4/6) The first question: which individuals should be chosen for computing the covariance matrix Compute the covariance matrix C via the top ps*np individuals in the current population Utilizing single population distribution information 27

62 CoBiDE (5/6) the λ offspring produced from μ parents the center of the μ best individuals from the λ offspring An issue in the covariance matrix adaptation: the variance will decease significantly N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, vol. 9, no. 2, pp ,

63 CoBiDE (6/6) The second question: how to determine the probability that the crossover is implemented in the Eigen coordinate system 29

64 CoBiDE (6/6) The second question: how to determine the probability that the crossover is implemented in the Eigen coordinate system 29

65 CoBiDE (6/6) The second question: how to determine the probability that the crossover is implemented in the Eigen coordinate system Stochastic tuning 29

66 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 30

67 Motivation Single population fails to contain enough information to reliably estimate the covariance matrix. Y. Wang, H.-X. Li, T. Huang, and L Li. Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Applied Soft Computing, vol. 18, pp , S. Guo and C. Yang. Enhancing differential evolution utilizing Eigenvector-based crossover operator. IEEE Transactions on Evolutionary Computation, vol. 19, no. 1, pp ,

68 Motivation Single population fails to contain enough information to reliably estimate the covariance matrix. Moreover, some extra parameters have been introduced. Y. Wang, H.-X. Li, T. Huang, and L Li. Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Applied Soft Computing, vol. 18, pp , S. Guo and C. Yang. Enhancing differential evolution utilizing Eigenvector-based crossover operator. IEEE Transactions on Evolutionary Computation, vol. 19, no. 1, pp ,

69 CPI-DE (1/3) We make use of the cumulative distribution information of the population to establish an appropriate coordinate system for DE s crossover Y. Wang, Z.-Z. Liu, J. Li, H.-X. Li, and G. G. Yen. Utilizing cumulative population distribution information in differential evolution. Applied Soft Computing, vol. 48, pp , (CPI-DE) 32

70 CPI-DE (1/3) We make use of the cumulative distribution information of the population to establish an appropriate coordinate system for DE s crossover The algorithmic framework Y. Wang, Z.-Z. Liu, J. Li, H.-X. Li, and G. G. Yen. Utilizing cumulative population distribution information in differential evolution. Applied Soft Computing, vol. 48, pp , (CPI-DE) 32

71 CPI-DE (1/3) We make use of the cumulative distribution information of the population to establish an appropriate coordinate system for DE s crossover The algorithmic framework Deterministic tuning Y. Wang, Z.-Z. Liu, J. Li, H.-X. Li, and G. G. Yen. Utilizing cumulative population distribution information in differential evolution. Applied Soft Computing, vol. 48, pp , (CPI-DE) 32

72 CPI-DE (2/3) Rank-NP-update of the covariance matrix in DE NP g+ 1 g+ 1 g g+ 1 g T NP = i ( i:2* NP )( i:2* NP ) i= 1 C w x m x m C = (1 c ) C + c ( σ ) C g+ 1 g g 1 g+ 1 NP NP NP 2 33

73 CPI-DE (2/3) Rank-NP-update of the covariance matrix in DE NP g+ 1 g+ 1 g g+ 1 g T NP = i ( i:2* NP )( i:2* NP ) i= 1 C w x m x m C = (1 c ) C + c ( σ ) C g+ 1 g g 1 g+ 1 NP NP NP 2 33 cumulative population distribution information

74 CPI-DE (3/3) The relationship between rank-np-update in CPI-DE and rank-μ-update in CMA-ES rank-np-update in CPI-DE rank-μ-update in CMA-ES 34

75 CPI-DE (3/3) The relationship between rank-np-update in CPI-DE and rank-μ-update in CMA-ES rank-np-update in CPI-DE rank-μ-update in CMA-ES rank-np-update in CPI-DE is a natural extension of rank-μ-update in CMA-ES 34

76 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 35

77 Motivation It is necessary to strike the balance between the accuracy of estimation and the convergence performance. 36

78 Motivation It is necessary to strike the balance between the accuracy of estimation and the convergence performance. The current methods adjust the original and Eigen coordinate systems in either a random way or a deterministic way. How to exploit the feedback information from the search process to adaptively tune them has not yet been investigated. 36

79 Motivation It is necessary to strike the balance between the accuracy of estimation and the convergence performance. The current methods adjust the original and Eigen coordinate systems in either a random way or a deterministic way. How to exploit the feedback information from the search process to adaptively tune them has not yet been investigated. It is an interesting topic to boost the research on the coordinate systems to other NIOA paradigms. 36

80 A New Point of View (1/4) How to describe some common nature-inspired operators in the original coordinate system Z.-Z. Liu, Y. Wang*, S. Yang, and K. Tang. An adaptive framework to tune the coordinate systems in nature-inspired optimization algorithms. IEEE Transactions on Cybernetics, in press. (ACoS) 37

81 A New Point of View (1/4) How to describe some common nature-inspired operators in the original coordinate system each variable is updated independently in an operator Z.-Z. Liu, Y. Wang*, S. Yang, and K. Tang. An adaptive framework to tune the coordinate systems in nature-inspired optimization algorithms. IEEE Transactions on Cybernetics, in press. (ACoS) 37

82 A New Point of View (2/4) A convenient transformation from a nature-inspired operator in the original coordinate system to the corresponding nature-inspired operator in the Eigen coordinate system 38

83 A New Point of View (2/4) A convenient transformation from a nature-inspired operator in the original coordinate system to the corresponding nature-inspired operator in the Eigen coordinate system 38

84 A New Point of View (2/4) A convenient transformation from a nature-inspired operator in the original coordinate system to the corresponding nature-inspired operator in the Eigen coordinate system 38

85 A New Point of View (2/4) A convenient transformation from a nature-inspired operator in the original coordinate system to the corresponding nature-inspired operator in the Eigen coordinate system 38

86 A New Point of View (3/4) An example: DE 39

87 A New Point of View (3/4) An example: DE the crossover operator in the original coordinate system 39

88 A New Point of View (3/4) An example: DE the crossover operator in the original coordinate system 39

89 A New Point of View (3/4) An example: DE the crossover operator in the original coordinate system the crossover operator in the Eigen coordinate system 39

90 A New Point of View (4/4) Another example: PSO 40

91 A New Point of View (4/4) Another example: PSO the velocity update equation in the original coordinate system 40

92 A New Point of View (4/4) Another example: PSO the velocity update equation in the original coordinate system 40

93 A New Point of View (4/4) Another example: PSO the velocity update equation in the original coordinate system the velocity update equation in the Eigen coordinate system 40

94 ACoS (1/9) 41

95 ACoS (1/9) 41

96 ACoS (2/9) Each individual selects an appropriate coordinate system from the original coordinate system and the Eigen coordinate system by a probability. 42

97 ACoS (3/9) The probability is adaptively updated based on the collected information from the offspring. 43

98 ACoS (4/9) The Eigen coordinate system is coupled with the original coordinate system 44

99 ACoS (5/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy 45

100 ACoS (5/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy An external archive A is used to store the offspring in both the current generation and the past several generations 45

101 ACoS (5/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy An external archive A is used to store the offspring in both the current generation and the past several generations A does not undergo any nature-inspired operators. 45

102 ACoS (5/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy An external archive A is used to store the offspring in both the current generation and the past several generations A does not undergo any nature-inspired operators. Therefore, it can obtain sufficient information to estimate the Eigen coordinate system while never affecting the population size and the generation number. 45

103 ACoS (5/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy An external archive A is used to store the offspring in both the current generation and the past several generations A does not undergo any nature-inspired operators. Therefore, it can obtain sufficient information to estimate the Eigen coordinate system while never affecting the population size and the generation number. strike the balance between the accuracy of estimation and the convergence performance 45

104 ACoS (6/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy 46

105 ACoS (6/9) The Eigen coordinate system is updated by making use of an additional archiving mechanism and the rank-μ-update strategy cumulative population distribution information 46

106 ACoS (7/9) Apply ACoS to DE 47

107 ACoS (7/9) Apply ACoS to DE 47

108 ACoS (7/9) Apply ACoS to DE 47

109 ACoS (7/9) Apply ACoS to DE 47

110 ACoS (7/9) Apply ACoS to DE 47

111 ACoS (8/9) Apply ACoS to PSO 48

112 ACoS (8/9) Apply ACoS to PSO 48

113 ACoS (8/9) Apply ACoS to PSO 48

114 ACoS (8/9) Apply ACoS to PSO 48

115 ACoS (8/9) Apply ACoS to PSO 48

116 ACoS (9/9) ACoS can be used to improve the performance of three famous DE variants: JADE, jde, and SaDE. 49

117 ACoS (9/9) ACoS can be used to improve the performance of three famous DE variants: JADE, jde, and SaDE. ACoS can be used to improve the performance of two of the most popular PSO variants: PSO-w and PSO-cf. 49

118 ACoS (9/9) ACoS can be used to improve the performance of three famous DE variants: JADE, jde, and SaDE. ACoS can be used to improve the performance of two of the most popular PSO variants: PSO-w and PSO-cf. We also apply ACoS to bat algorithm (BA) and teaching learning-based optimization (TLBO). 49

119 ACoS (9/9) ACoS can be used to improve the performance of three famous DE variants: JADE, jde, and SaDE. ACoS can be used to improve the performance of two of the most popular PSO variants: PSO-w and PSO-cf. We also apply ACoS to bat algorithm (BA) and teaching learning-based optimization (TLBO). The structure of ACoS is simple. 49

120 Outline Nature-Inspired Optimization Algorithms (NIOAs) Eigen Coordinate System I: Stochastic Tuning + Single Population Distribution Information II: Deterministic Tuning + Cumulative Population Distribution Information III: Adaptive Tuning + Cumulative Population Distribution Information Conclusion 50

121 Conclusion (1/2) The Eigen coordinate system should be coupled with the original coordinate system in that in the early stage the Eigen coordinate system is inaccurate due to the random distribution of the population. 51

122 Conclusion (1/2) The Eigen coordinate system should be coupled with the original coordinate system in that in the early stage the Eigen coordinate system is inaccurate due to the random distribution of the population. The Eigen coordinate system can significantly accelerate the convergence. 51

123 Conclusion (1/2) The Eigen coordinate system should be coupled with the original coordinate system in that in the early stage the Eigen coordinate system is inaccurate due to the random distribution of the population. The Eigen coordinate system can significantly accelerate the convergence. However, based on our observation and analysis, it is hard for the Eigen coordinate system to change the search pattern of a NIOA, which means that the capability of a NIOA implemented in the Eigen coordinate system jumping from a local optimum is limited if this NIOA tends to converge to a local optimum of an optimization problem. 51

124 Conclusion (2/2) Due to the fact that the theory study of NIOAs is very difficult (in particular, the theoretical analysis of population cooperation is very challenging), tuning the coordinate systems in NIOAs is a very important and promising step toward understanding why NIOAs work or do not work, which deserves in-depth attention in the future. 52

125 Conclusion (2/2) Due to the fact that the theory study of NIOAs is very difficult (in particular, the theoretical analysis of population cooperation is very challenging), tuning the coordinate systems in NIOAs is a very important and promising step toward understanding why NIOAs work or do not work, which deserves in-depth attention in the future. The source codes of CoBiDE, CPI-DE, and ACoS can be downloaded from the following URL: 52

126