Stochastic Velocity Threshold Inspired by Evolutionary Programming

Transcription

1 Stochastic Velocity Threshold Inspired by Evolutionary Programming Zhihua Cui Xingjuan Cai and Jianchao Zeng Complex System and Computational Intelligence Laboratory, Taiyuan University of Science and Technology, Shanxi, P.R.China, Abstract Particle swarm optimization (PSO) is a new robust swarm intelligence technique, which has exhibited good performance on well-known numerical test problems. Though many improvements published aims to increase the computational efficiency, there are still many works need to do. Inspired by evolutionary programming theory, this paper proposes a self-adaptive particle swarm optimization in which the velocity threshold dynamically changes during the course of a simulation, and two further techniques are designed to avoid badly adjusted by the self-adaption. Six benchmark functions are used to testify the new algorithm, and the results show the new adaptive PSO clearly leads to better performance, although the performance improvements were found to be dependent on problems. 1. Introduction Particle swarm optimization[1][2] (PSO) is a populationbased, self-adaptive random search optimization technique based on the simulation of animal social behaviors such as fish schooling, bird flocking, etc. Due to the ease of implementation and fast convergence speed, PSO has been successfully applied with many areas including: data mining [3][4][5][6][7], image compression[8][9],ad Hoc Networks design[10], multi-objective optimization[11] etc. To make the PSO more effective and efficient, many improvements have been proposed and the research can be categorized into four parts: algorithm, topology, parameters, and combination with the other evolutionary computation techniques. There still exists a large room to making algorithm more effectiveness by providing a proportional parameter selection principle. Many published works deal with parameter selection principles, though few are concerned about velocity threshold v max.largev max increases the search region, enhancing the global search capability, as well as small v max decreases the search region, adjusting the search direction of each particle frequently. Since then, a proportional threshold v max selection principle can balance the exploitation and exploration capability of PSO, utilizing more information about search directions. Inspire by the evolutionary programming theory, this paper introduces an adaptive version of PSO modified threshold v max dynamically. Section 2 gives a briefly description of particle swarm optimization and evolutionary programming, and the similarity comparison between these two evolutionary computation techniques are proposed in section3. The details of the adaptive PSO is discussed in section4. In section5, six wellknown benchmark functions are used to test the performance of the new algorithm. Finally, further research aspects are proposed. 2. Particle Swarm Optimization and Evolutionary Programming Like other evolutionary computation techniques, particle swarm optimization maintains one population to evolve. Each individual (called particle),owns two characters: position and velocity, represents a potential solution of the search space. The velocity vector of each particle represents the forthcoming motion tendency information and the update equation of particle j of standard PSO at time t +1 is presented in equation (1): v jk (t +1)=wv jk (t) +c 1 r 1 (p jk (t) x jk (t)) (1) +c 2 r 2 (p gk (t) x jk (t)) and the corresponding position vector is updated by x jk (t +1)=x jk (t) +v jk (t +1) (2) where the k th dimensional variable of velocity vector V j (t+ 1) = (v j1 (t +1);v j2 (t +1); :::; v jn (t +1)) (n denotes the dimension of problem space) is limited by jv jk (t +1)j <v max (3) where v jk (t) and x jk (t) are the k th dimensional variables of velocity and position vectors of particle j at time t, p jk (t) and p gk (t) are the k th dimensional variables of historical positions found by particle j and the whole swarm at time t respectively. w is inertia weight between 0 and 1, accelerator coefficients c 1 and c 2 are known as constants, r 1 and r 2 are two random numbers generated with uniform distribution within (0; 1).

2 The velocity vector of each particle consists of three parts: social part c 1 r 1 (p jk (t) x jk (t)), cognitive part c 2 r 2 (p gk (t) x jk (t)) and momentum part wv jk (t) while the performance of a PSO is determined with balancing among these parts. The momentum part provides the necessary momentum for particles to roam across the search space. The cognitive part represents the personal thinking of each particle while the social part represents the collaborative effect of the particles. Initially, a population of particles is generated with random positions, then random velocities are assigned to each particle. The fitness of each particle is then evaluated according to a predefined objective function. At each generation, the velocity of each particle is calculated according to (1) and (3) and the position for the next function evaluation is updated according to (2). Each time if a particle finds a better position than the previously found best position, its location is stored in memory. If the stop conditions are satisfied, the best position found by the whole swarm is the final solution. The first added parameter is inertia weight introduced by Y.Shi and R.C.Eberhart[12] to balance between the global search and local search. They argued that a large inertia weight facilitates a global search while a small inertia weight facilitates a local search[13][14]. Though Y.L.Zheng etc.[15] proposed a different opinion either global or local search ability associates with a small inertia weight possessing the capability of exploring new space. Meanwhile, a large inertia weight provides the algorithm more chances to be stabilized. To propose a effective strategy of inertia weight, Y.Shi and R.C.Eberhart introduced a technique for adapting the inertia weight dynamically using a fuzzy controller[16][17]. Based on the information about average velocity of a swarm, N.Iwasaki and K.Yasuda finds the success or failure of a search is related to the average of absolute value of velocity. Since then, an adaptive strategy for tuning the inertia weight of the PSO method is provided[18][19]. To ensure convergence, Clerc indicated a version of PSO with constriction factor[20]. The results show that if the constriction factor is correctly chosen, it guarantees the stability of the PSO without the need to bound velocities, while it does not know whether the computational efficiency improved or not. There are still many other improvements aims to computational efficiency and swarm diversity,such as PSO with selection[21], PSO with Gaussian mutation[22], and diversity-guided PSO[23], more details can been found with the corresponding references. Evolutionary programming (EP) was devised by L.J.Fogel in the context of evolving finite state-machines to be used in the prediction of time series [24]. In contrast to genetic algorithm that emphasize crossover, mutation is the main operation in EP. Now, EP has been extended to include more general representations, such as applications involving continuous parameters and combinatorial optimization problems. Since then, EP has been applied successfully for the optimization of real-valued functions. The study results show EP with self-adaptive mutation usually performs better for the test functions[25][26]. Self-adaptive EP for continuous parameter optimization problems is now a widely accepted method in evolutionary computation. It should be noted that EP for continuous parameter optimization has many similarities with evolution strategies, although their development proceeded independently. In conventional EP, each parent generates an offspring via Gaussian mutation and better individuals among parents and offspring are selected as parents of the next generation. The individual of EP is a pair of real-valued vectors (x j ; j ) (j=1,2,...,m) where x j is a position vector while j is a standard deviation vector, and the offspring (x 0 j ; 0 j ) is computed by 0 j (k) = j(k)exp(fi 0 N (0; 1) + fin k (0; 1)) (4) x 0 j (k) =x j(k) + j (k)n k (0; 1) (5) where x j (k) is the k th variable of individual x j (k=1,2,...,n), N k (0; 1) and N (0; 1) are the two random numbers generated with mean zero and standard deviation one while N k (0; 1) is renewed for different p dimension. The factors fi and fi 0 are commonly set to ( 2 p n) 1 and ( p 2n) 1 respectively[27]. In [28], X.Yao introduced a fast EP with Cauchy mutation strategy, and the offspring is computed with x 0 j (k) =x j(k) + j (k)ffi k (t) (6) where ffi k (t) represents a Cauchy random variable with the scale t for each dimension of individual j and the update equation of 0 j (k) is the same as formula (4). Experiments show that Gaussian mutation has a good performance for some unimodal functions and multimodal functions with only a few local optimal points, whereas Cauchy mutation works well on multimodal functions with many local optimal points[28]. 3. The Comparison of Particle Swarm Optimization and Evolutionary Programming Formula (2) implies the k th variable x jk (t+1) of position vector of particle j at time t +1is satisfied with jx jk (t +1) x jk (t)j = jv jk (t +1)j (7) Combined with (3), the above formula means jx jk (t +1) x jk (t)j»v max (8) Formula (8) means x jk (t +1) only falls into the interval [x jk (t) v max ;x jk (t) +v max ] with a constant length of 2v max, no matter the selection of v max is good or not. Since the constant v max is only used to provide a displacement (Fig.1), a proportional schedule of v max can make PSO more effective. Large v max may be better when PSO explores

3 Figure 1. New Position Update with PSO probability densities. It also implies using a dynamic random variable threshold provides a more capabilities no matter exploration and exploitation. Inspired by the above conclusion,one dynamically selfadjusting strategy of velocity threshold v max is needed to improve the performance of PSO. Thanks to the similarity about v max and jk (t), one simple way to choose the value of v max is the self-adaptive method of jk (t) with formula (4) (6). Since then, a new modified version of PSO, called adaptive PSO using EP(APSO,in briefly), is proposed combining the self-adaptive selection principle of EP. It uses position and velocity information, as well as provides more exploration and exploitation capabilities and dynamic adaption of search directions. 4. Adaptive Particle Swarm Optimization Using Evolutionary Programming Figure 2. New Position Update with EP the search space as well as small v max is better if PSO exploits some region. Hence, how to schedule the v max is an interesting problem. In the following part, we compare the PSO and EP firstly, then the problem is solved according to some similarities between these two techniques. In EP, for convenience, we suppose the offspring of parent x jk (t) is x jk (t +1). From formula (5) and (6), the offspring x jk (t +1) falls into the whole axis with some probability density and the length is a random variable (Fig.2). Though the offspring fallen into the interval [x jk (t) jk (t);x jk (t)+ jk (t)] with large probability, as well as other region with small probability. It means the offspring of EP pays more attention to exploration capability if jk (t) larger than v max as well as more exploitation capability on the country. Associated with Fig.1 and Fig.2, the action of coefficient v max is similar with coefficient jk to control the region where new position (offspring) vector fallen with large probability or probability one. Though there are many differences exist: (1) The velocity threshold v max is always a constant as well as the jk is a dynamic random variable changed during the course of a simulation. Obviously, a dynamic random variable offers a more chances adjusting its selection directions to discover the better solutions. (2) The new position vector of PSO is dominated by the interval determined with the current position vector as well as the constant v max, though the offspring vector of EP falls not only within the interval determined with the current individual vector as well as deviation jk,but also extending the search area into whole axis with some The new APSO introduces a different self-adaptive strategy of velocity threshold v max for each dimension of each particle of the swarm, and dynamically adjusts its values using evolutionary programming method. Though the selfadaptive strategy is borrowed from EP, it is a stochastic strategy, and in some cases the selected v max may be bad to provide a balance. Since then, two further strategy to avoid the affection about bad selected v max are also proposed Self-adaptive Strategy of Velocity Threshold Since the velocity threshold for each dimension of each particle is dynamically adjusted, we use (v max ) jk (t) represents the k th dimensional variable of velocity threshold of particle j at time t. Similar with formula (4), the update equation of velocity threshold (v max ) jk (t) is as follows. (v max ) jk (t +1)=(v max ) jk (t)exp(fi 0 N (0; 1) + fin k (0; 1)) (9) where fi and fi 0 are the same as formula (4) Expansion Strategy Since the velocity threshold (v max ) jk (t) is a stochastic process, suppose the velocity threshold (v max ) jk (t +1) at time t +1 is larger than (v max ) jk (t), it means (v max ) jk (t) < (v max ) jk (t +1) (10) Since jv jk (t)j»(v max ) jk (t) (11) We have jv jk (t)j»(v max ) jk (t +1) (12) If the k th dimensional variable of velocity of particle j at time t +1 is satisfied with jv jk (t +1)j»(v max ) jk (t) (13)

4 Then jv jk (t +1)j»(v max ) jk (t +1) (14) is always true no matter which value (v max ) jk (t +1) is. In this case, the self-adaptive strategy of (v max ) jk (t +1) is no affection. Since then, an expansion strategy is used to overcome this shortcoming by expanse the value of v jk (t+1) to larger than (v max ) jk (t) with a probability threshold p E. Though we do not know whether the value of v jk (t +1) is better for small value or large value, the threshold p E does not too small. And this strategy should not be used in the last period to preserve convergence. In this paper, we select the threshold p E decreases linearly from 0.3 to 0.0. This setting only comes from the experimental test, and the expansion strategy is defined as follows. If (random number < p E ):and:(v max ) jk (t) < (v max ) jk (t +1):and:jv jk (t +1)j»(v max ) jk (t) v jk (t +1)=v jk (t +1) (v max) jk (t +1) (v max ) jk (t) 4.3. Hybrid Velocity Threshold Strategy (15) The self-adaptive strategy is a stochastic method, though it enhances the performance greatly (see below). There exist some extreme cases the velocity to be badly adjusted. Since then, one correct mechanism is used to control the velocity addition to the self-adaptive strategy. The mechanism uses the original constant velocity threshold, paralleled with dynamic adaptive velocity threshold to determine the velocity vector, and the better one is chosen as the next velocity vector. In this manner, each particle owns different characters: adaptive velocity (av), original velocity (ov), velocity (v), adaptive position (ap),original position (op),position (p), and adaptive velocity threshold (avt). The APSO is implemented as follows in this study. Step1. Generate the initiate population with m particles, and set the value of (v max ) jk (0) of dimension k of particle j as v 0. The position vector P of each particle is selected within the domain region as well as velocity vector of each particle is chosen within the interval [0; (v max ) jk (0)] uniformly. Step2. Update the position and velocity vectors of each particle with formula (1) and (2). If one random number is less than p E, and the conditions of expansion strategy is satisfied, make k th dimensional variable of adaptive velocity vectors is satisfied jav jk (t +1)j»(v max ) jk (t +1) (16) where (v max ) jk (t +1) represents the k th dimension of velocity threshold of particle j at time t +1, and is updated with formula (9). This strategy gives a dynamic velocity threshold so that the search direction can adaptive changed though the exploration capability not improved. Meanwhile, x jk (t +1) still falls into the interval dominated by x jk (t) and (v max ) jk (t +1) restricting the exploration capability. Since then, an enhanced APSO is used to own a larger global search capability with only adding additional second velocity threshold update equation after formula (9) defined as follows: avt jk (t +1)=(v max ) jk (t +1) P robability Density (17) where P robability Density means some ordinary probability densities such as Gauss and Cauchy. Similarly, the corresponding value of original velocity is determined with jov jk (t +1)j»v max (18) Then the original and adaptive position vectors are computed as follows. ap jk (t +1)=p jk (t) +av jk (t +1) (19) op jk (t +1)=p jk (t) +ov jk (t +1) (20) The velocity and position vectors of particle j are determined according to ρ apjk (t +1); if f (AP (t + 1)) <f(op (t + 1)) ; p jk (t+1) = op jk (t +1), otherwise. (21) ρ avjk (t +1); if f (AP (t + 1)) <f(op (t + 1)) ; v jk (t+1) = ov jk (t +1), otherwise. (22) Step3.Update the historical best position of each particle and the whole swarm. Step4. If the stop criterion is satisfied, output the final best solution of the swarm. Otherwise, goto Step2. 5. Simulation Results The benchmark functions in this section provide a balance of unimodal, multimodal with many local minima and only a few local minima as well as easy and difficult functions. Sphere modal, Schwefel problem 2.22 are the unimodal functions, Schwefel problem 2.26, Ackley and Griewank are multimodal functions with many local minima, while Schefel s foxholes is multimodal functions with only a few local minima. More details can be found in [21]. To give a more detail comparison, six different versions of APSO are designed: APSO with v max computed by formula (9) (ASPO1), APSO1 with additional formula (17) with normal distribution (APSO2), APSO1 with additional formula (17) with uniform distribution (APSO3), APSO1 with additional formula (17) with Cauchy distribution with the scale 1.0,0.5, and 2.0 (APSO4,APSO5,APSO6,respectively). All of the six versions of APSO uses the same coefficients. For each experiment the simulation records the mean (Mean Value), standard deviation (Standard deviation Value),

5 Table 1. Comparison Results of Sphere Model Function SPSO e e e-007 APSO e e e-008 APSO e e e-019 APSO e e e-010 APSO e e e-018 APSO e e e-018 APSO e e e-019 Table 6. Comparison Results of Shekel s Foxholes Function SPSO e e e+000 APSO e e e-001 APSO e e e-001 APSO e e e-001 APSO e e e-001 APSO e e e-001 APSO e e e-001 Table 2. Comparison Results of Schwefel Problem 2.22 Function SPSO e e e-005 APSO e e e-006 APSO e e e-011 APSO e e e-007 APSO e e e-011 APSO e e e-011 APSO e e e-011 Table 3. Comparison Results of Generalized Schwefel Problem 2.26 Function SPSO e e e+003 APSO e e e+004 APSO e e e+004 APSO e e e+004 APSO e e e+004 APSO e e e+004 APSO e e e+004 Table 4. Comparison Results of Ackley s Function SPSO e e e-004 APSO e e e-006 APSO e e e-009 APSO e e e-007 APSO e e e-009 APSO e e e-009 APSO e e e-009 Table 5. Comparison Results of Generalized Griewank Function SPSO e e e-007 APSO e e e-008 APSO e e e+000 APSO e e e-009 APSO e e e+000 APSO e e e+000 APSO e e e+000 best and worst solutions found (Best Solution, Worst Solution, respectively). The coefficients of standard PSO (SPSO) and other versions of APSO are set as follows. The inertia weight w is decreased linearly form 0:9 to 0:4, and two accelerator coefficients are set to 2.0. Total individuals are 100 except Shekel s Foxholes uses only 10, and v max is set to 100% of the upper bound of domain in SPSO as well as the initialized v max set to 3.0 in other APSO. Each experiment the simulation run 30 times while each time the largest evolutionary generation is 1000 except which of Shekel s foxholes is 50. To avoid the velocity threshold falling too low to zero, a low bound 1:0e 5 should be put on v max. The same consideration is given to fi 0 N (0; 1) + fin k (0; 1) to avoid the system overflow, the upper and low bound are set to 1:0e 5 and 1:0e +2,respectively. Table 1 to Table 6 are the comparison results for six benchmark functions. From Table 1 and 2, we found the six versions of APSO are always better than SPSO while APSO1 and APSO3 only surpass the SPSO a little. Though Sphere modal and Schwefel problem 2.22 are the unimodal functions, it implies the APSO2 and APSO4 to APSO6 are commonly fit to solve unimodal functions, and the performance improved significantly. Table 3 to 5 give a set of multimodal functions with many local optima. In table 3,4 and 5, four versions of APSO are surpassed than SPSO except for APSO1 and APSO3. It means the APSO2 and APSO4 to APSO6 can be used to solve the multimodal functions whether epistasis exists or not. Table 6 gives detailed description about multimodal functions with only a few local optima. Though the mean values and the standard deviation implies APSO are always better than SPSO. All in all, APSO2 and APSO4 to APSO6 can work well on unimodal functions, multimodal functions with a few local optima, and multimodal functions with many local optima. 6. Conclusion Inspired by the evolutionary programming, a new version of particle swarm optimization, adaptive PSO using EP, is proposed. New PSO is considered to combing the advantages

6 of PSO and EP using dynamically changed velocity threshold. This character is used provides enhanced exploration capability for increased thresholds as well as exploitation capability for decreased thresholds. New research aspects will include the combing other techniques of EP, and provides some other selection principles of v max. Acknowledgement This paper was supported by the Doctoral Scientific Research Starting Foundation of Taiyuan University of Science and Technology under No , and it was also supported by Shanxi Science Foundation for Young Scientists under Grant References [1] Eberhart, R. C., and Kennedy, J., A new optimizer using particle swarm theory, Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, pp.39-43,1995. [2] Kennedy, J. and Eberhart, R. C., Particle swarm optimization, Proceedings of IEEE International Conference on Neural Networks, IV, pp ,1995. [3] Eberhart, R. C., and Shi, Y., Evolving artificial neural networks, Proceedings of 1998 International Conference on Neural Networks and Brain, Beijing, P.R.China, pp.5-13,1998. [4] Ye B., Sun J., and Xu W.B., Solving the hard Knapsack problem with a binary particle swarm optimization, Lecture Notes in Bioinformatics 4115,Kunming, P.R.China, pp , [5] Hassas,S.,Using swarm intelligence for dynamic web content organizing, Proceedings of 2003 IEEE Swarm Intelligence Symposium, Indianapolis, USA, pp.19-25,2003. [6] Srinivasan,C., Loo,W.H., and Cheu,R.L., Traffic incident detection using particle swarm optimization,proceedings of 2003 IEEE Swarm Intelligence Symposium, Indianapolis, USA, pp ,2003. [11] Zheng Y.L., Ma L.H., Zhang L.Y., and Qian J.X., On the convergence analysis and parameter selection in particle swarm optimization, Proceedings of the Second International Conference on Machine Learning and Cybernetics, Xi an, China,pp ,2003. [12] Shi Y., and Eberhart R.C., Fuzzy adaptive particle swarm optimization, Proceedings of the Congress on Evolutionary Computation, [13] Monson C.K., and Seppi K.D., The Kalman swarm: a new approach to particle motion in swarm optimization,proceedings of the Genetic and Evolutionary Computation Conference,pp , [14] Sun J. etc, Particle swarm optimization with particles having quantum behavior, Proceedings of the IEEE Congress on Evolutionary Computation, pp , [15] Cui Z.H., Zeng J.C., and Cai X.J., A new stochastic particle swarm optimizer,ieee Congress on Evolutionary Computation, pp ,2004. [16] Cui Z.H., Zeng J.C., and Sun G.J.,A fast particle swarm optimization, International Journal of Innovative Computing, Information and Control,2006,2(6): [17] Cui Z.H., Cai X.J., Zeng J.C., and Sun G.J., Predictedvelocity particle swarm optimization using game-theoretic approach, Lecture Notes in Bioinformatics, vol.4115, Kunming, P.R.China, pp , [18] Michalewicz Z., Genetic Algorithm+ Data Structures =Evolution Programs, Springer-Verlag, Berlin,1992. [19] Marcus H., Fitness uniform selection to preserve genetic diversity, Proceedings of IEEE Congress on Evolutionary Computation, pp , [20] Blickle T., and Thiele L., A mathematical analysis of tournament selection, Proceedings of the Sixth International Conference on Genetic Algorithms, San Francisco, California, pp.9-16, [21] Yao X, Liu Y and Lin GM., Evolutionary programming made faster, IEEE Transactions on Evolutionary Computation, 1999, vol.3, no.2, pp [7] Shi, Y. and Eberhart, R. C., A modified particle swarm optimizer, Proceedings of the IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA,pp.69-73,1998. [8] Shi Y., and Eberhart R.C., Parameter selection in particle swarm optimization, Proceedings of the 7 th Annual Conference on Evolutionary Programming, pp ,1998. [9] Shi Y., and Eberhart R.C., Empirical study of particle swarm optimization, Proceedings of the Congress on Evolutionary Computation, pp ,1999. [10] Suganthan P N, Particle swarm optimizer with neighbourhood operator, Proceedings of the Congress on Evolutionary Computation, pp ,1999.