Discrete evaluation and the particle swarm algorithm

Transcription

1 Volume 12 Discrete evaluation and the particle swarm algorithm Tim Hendtlass and Tom Rodgers Centre for Intelligent Systems and Complex Processes Swinburne University of Technology P. O. Box 218 Hawthorn Australia 3122 Abstract. We propose that the optimal performance of the PSO algorithm should differ from that of the real life creatures on which PSO is modelled. If a bird finds a good food source, the likely behaviour for a flock is to congregate there, settle and feed. However, once PSO has found an optimum, while some particles should explore in the immediate vicinity for any better optimum present, the rest of the swarm should set out to explore new areas. The common PSO practice of only evaluating each particle s performance at discrete intervals can, at small computational cost, be used to automatically adjust the PSO behaviour in situations where the swarm is settling so as to encourage part of the swarm to explore further. 1 Introduction The Particle Swarm Optimisation (PSO) algorithm is based on the movement of real creatures, such as birds, and attempts to model the way they balance their behaviour between exploration and exploitation, thus hoping to replicate the good search behaviour found in nature. As an optimisation tool, PSO has proved extremely effective on a wide range of problems. There are, however, two important differences between the artificial PSO algorithm and the real world: Real creatures continuously evaluate their current situation. In PSO we can only evaluate the fitness of the current position at discrete time intervals. It may make sense for real creatures to all settle in one spot if, for example, this spot is especially advantageous in terms of food supply. However, with the PSO, once the fitness of any position visited by any particle is known, it is in fact inefficient for any swarm member to return to that position (at least for episodic problems). In particular, it would be desirable to discourage, at least partially, the tendency for the whole swarm to congregate around any currently good position. Ideally, a part of the swarm should explore in the 1 Copyright 25

2 immediate vicinity, lest there be an even better position close by, while the rest of the swarm should set out to explore new areas. It is possible to use the nature of discrete evaluation to alter the behaviour of PSO when congregating so as to achieve partial settling and partial aggressive further exploration. 2 The Particle Swarm Optimisation (PSO) Algorithm The PSO algorithm uses a number of particles (the swarm) each of which describes a possible solution to the process to be optimised. Each particle moves through problem space so that the set of solutions represented by the particles continuously changes. The movement of each particle is influenced by the current situation of all other particles in the swarm together with some past history of the performance of the swarm as a whole. V Formally, the new velocity of a particle at time T+t is = T + t χ ( MV T X B X S + rand G( ) + rand L( ))) X B t X S t V T + t and is given by: (1) where V T is the velocity of this particle at time T, M is the momentum, X is the current position of the individual, rand returns a random number in the range (, 1). G and L set the relative attention to be placed on the positions B and S respectively. B is the best position found by any individual in the swarm so far, and S a position derived by comparing the relative performance of this individual with the performances of a number of other swarm members. M, G and L are bounded to the range (, 1). The factor χ provides velocity constriction. For more information on traditional PSO see [1,2,3,4]. Traditionally finding the position S involves defining a neighbourhood of the particle and only considering the effect of other particles within that neighbourhood. An alternate way of calculating the position S in an effective (but computationally modest) way that takes account of the performance of all swarm members has been introduced in [5]. All other swarm members are considered to influence the position S for a given particle but the magnitude of the influence decreases with both fitness and distance from the given particle for a minimization problem 1. Equation 1 differs slightly from the conventional version, in that the unit vectors from X to B and from X to S that are used rather than vectors themselves. This provides a less greedy version of the equation compared to the conventional as the magnitude of the attraction to the points B and S no longer depends on the distance from these points. This becomes an important change when, as in this work, particles are required to explore at distances far from already known minima. The parameter t is the time between updates and is required for dimensional consistency. It is usually taken to be one basic time interval and so is often omitted from the equation. This paper is concerned with an effect of the finite value used for t. 1 For a maximization problem it would increase with fitness but decrease with distance. 2 Copyright 25

3 3 Introducing a Short-range Force In order to alter the behaviour of particles when they are settling close together, it is desirable to introduce a short-range interaction between particles. One possibility is to introduce a gravitational style attraction that produces a force of attraction of particle i towards particle j, whose magnitude is inversely proportional to the square of the distance between them. This short-range force will produce a velocity component on one particle towards the other that K can be represented by v ij =, where d is the distance between the particles and K is a 2 ij d ij constant. If evaluation were continuous for PSO this would merely cause close particles to enter stable limit cycles, as opposed to what we wish to achieve. However, for noncontinuous evaluation this short-range force can have a useful effect. 4 The effect of discrete evaluation An effect of discrete evaluation on the traditional PSO algorithm is that particles may move directly over places of interest but not notice them, unless the instant of transit happens to coincide with a fitness evaluation. The actual distance travelled between evaluations depends on the velocity of the particle and it is important that particles slow in regions containing a fitness peak. The exploitation component of basic PSO tends to do this as a consequence of drawing particles that have overflown back towards a good position. Time=t 1 Time=t+ 2 Time=t+ Figure 1. The short-range forces ( ) and the velocities ( ) induced on two approaching particles by a shortrange force. The length of each arrow indicates the The short-range force introduced in section 3 will have little effect while the swarm is dispersed regardless of continuous or discrete evaluation. However, as particles approach each other the magnitude of the short-range force will climb significantly, producing a substantial increase in the velocity of the particles towards each other. For discrete evaluation, by the time of the next evaluation, particles may have passed each other and be at such a distance from each other that the short-range attraction that could bring them back together is far too weak to do this. As a result, the particles will continue to move rapidly apart with almost undiminished velocity, exploring beyond their previous positions. This process is shown diagrammatically in Figure 1. 3 Copyright 25

4 At time t, the separation between the particles is moving into the region in which the magnitude of the short-range attraction (shown by broad arrows) is becoming significant. This augments the effect of their velocities (shown by normal arrows) so that the particles move close together. By time t+1 the particles are close and the short-range effect is large. As a result, the velocity of the particles increases substantially, almost entirely as a consequence of the short-range attraction. By time t+2 when the next evaluation is made the particles will have passed each other, and be so far apart the short-range force is weak. Consequently, the particles continue to diverge, retaining at t+2 much of the velocity obtained as a result of the short-range forces acting at time t+1. The short-range forces will continue to decrease as the particles move apart, leaving only the normal swarm factors to influence their future movement in the absence of other close encounters. If continuous 2 evaluation could be used, as soon as the particles pass each other the direction of the short-range force would reverse and the still high values of short-range attraction would rapidly nullify the effects of the acceleration as the particles converged. As far as the effect of the short-range force is concerned, after the particles have passed and separated they would be first slowed, then come to rest before once again converging. Therefore the net effect of the short-range force under continuous evaluation would be for the particles to eventually enter a stable limit cycle. 1 + ((X-.2)^2 + (Y-.1)^2)^.25-cos(5*pi*sqrt((X-.2)^2 + (Y-.1)^2)) Timbo2 - min.2, ((X-.2)^2 + (Y-.1)^2)^.25-cos(5*pi*sqrt((X-.2)^2 + (Y-.1)^2)) Timbo2 - min.2, Figure 2. The distribution of twenty particles after approximately the same number of evaluations. A single minimum exists at (,). At the left PSO without a short-range force, at the right PSO with a short-range force. The aprticles away from the minimum have moved there after reaching the minimum and being ejected. Under discrete evaluation conditions, we are able to take advantage of an aliasing effect in which conservation of energy appears to have been violated with a convenient spontaneous increase in the net energy of the two particles 3. This increase in energy of the two particles may or may not be a good thing as far as any two particles are concerned. They might just be happening to be passing each other closely in region of poor fitness rather than converging to a fit point. In this case the short-range force may just act as a local perturbation of unpredictable benefit. When a significant part of a swarm is tending to settle on a point of good fitness, the short-range force will have the desirable effect that some of the neighbourhood so engaged will be ejected with significant velocities. As members get ejected 2 Truly continuous evaluation is obviously impossible for any computer based PSO algorithm. Here the practical meaning of the word continuous includes any situation in which the interval between evaluations is sufficiently short so that no particle has moved a significant distance between evaluations. 3 To those with a physics background, such as the authors, this may seem strangely disconcerting. However, if you adapt a real life algorithm by the addition of one or more non-physical features (in this case discrete evaluation) it is unreasonable to expect that the behaviour of the algorithm will continue to match real life behaviour. 4 Copyright 25

5 the number of those left to cluster is decreased with a decrease in the probability of further ejections. Hence the effect of the introduced short-range force on the normal settling behaviour of PSO is self-limiting and some swarm particles are left to continue exploration in the local vicinity. Figure 2 shows the results of using a particle swarm optimisation algorithm to find the minimum of a function. This minimum occurs at the position (.2,.1). In the left hand picture, without a short-range force, all the swarm individuals have converged very close to the minimum. In the right hand picture, with a short-range-force and after approximately the same number of evaluations, the minimum has again been found. This time, while at any moment a number of swarm members remain searching in the immediate vicinity of the minimum, others are being ejected to continue searching elsewhere. In this case, as there is no better minimum to be found, these ejected particles subsequently return to the known minimum, increasing the probability of others being ejected. This production of successive waves of explorer particles continues indefinitely. 5 Experimental results Figure 3. The two-dimensional minimum optimisation test surface used. Minimum B (min B) is slightly better than minimum A (min A). All swarm particles were released in the vicinity of the start point shown so that they all encountered minimum A first. To investigate the effect of a short-range force on the swarm algorithm the two-dimensional two minimum test problem shown in Figure 3 was used. Two local minima, A and B, are separated by a high ridge with poor fitness,. Minimum B (value zero) is marginally better than minimum A (value.1) while on the intervening ridge the value climbs to a value of two. Points off the drawn map were assigned values equal to that of the closest on-map point. All swarm members were initialised around the edge of the drawn map closest to minimum A, specifically in the region in which -15 <X<-95 and 15 < Y < 95. The direction of the initial velocities was randomly chosen with the magnitude being assign a random value between specified minimum and maximum values. A number of tests were each repeated 1 times, each test only differ in the values of some parameters. The maximum velocity that a particle could have was bounded to a value of Max times the maximum initial velocity, where Max is a user chosen parameter. This set the maximum distance that a particle could travel in an iteration, and therefore the maximum 5 Copyright 25

6 distance between fitness evaluations. The parameter values held constant across all tests are shown in Table 1 and the results from a chosen sample of these tests are shown in Table 2. The surface and initial particle release region were specifically designed to make it difficult for a conventional swarm to find the better minimum, minimum B. As expected, without a short-range force, the swarm quickly found and settled on minimum A every time, taking an average of 21.5 iterations with a standard deviation of 8. This result was obtained with a maximum initial speed of 3. Table 1. Fixed test parameters. Swarm size 5 Global weighting G.5 Local weighting L.5 Momentum.9 Initial X position range Initial Y position range Minimum considered found if an evaluation is made within a distance of one unit. A test was terminated if minimum B was not found after 2 iterations The time taken when a short-range force was operational are shown in Table 2. Note that the shortest time occurs with the small value for max but the largest value for K. As the maximum initial speed was increased, the probability that the particle would over fly the minimum in the period between evaluations also increased. As a result particles had to reverse and recross the minimum (possibly several times) before it was observed, thus increasing the detection time. As the probability of a significant short-range force is low until the particles are congregating around the minimum, there was little chance for the speed to be accelerated beyond the initial velocity range. As a result the speed-limiting factor K had little effect. Finding minimum B required that at least one (and probably several) significant shortrange force events take place. Now the maximum distance that a particle can travel in one iteration (max times the maximum initial speed) sets the lowest resolution of the search. The distance between the two minima is just under 13 units. As the maximum distance per iteration rises significantly above this there is an increase in the number of iterations taken to find the minimum as more particles over-fly it. A stronger influence is seen from the factor K that sets the value of the short-range force for a particular inter-particle separation. The average distance travelled per iteration increases with this. Provided that this does not approach the inter-minimum separation, this results in fewer steps before the minimum is found. In the left screen image of Figure 4, we can see that the swarm has already found minimum A at (-4,4) and is tending to congregate in the vicinity of this point. A number of particles can be observed that have been ejected from the main swarm body as a result of short-range forces. One particle is heading towards the minimum B at (5,-5) but has not yet reached it. In the right screen image of Figure 4, this particle has detected a point in the general vicinity of minimum B, which requires an evaluation of a point in the very small region (within one unit) in which the values are less than the best value of the first minimum already found. The actual detection occurred one or two iterations ago, and the value obtained was probably not the absolutely minimum value in this region. The rest of the swarm is now attracted to this 6 Copyright 25

7 new best position and the normal swarm behaviour will ensure that the absolute minimum value is found soon after the swarm body arrives. Figure 5 shows all the points evaluated in 4 iterations with minimum B being found after 32 iterations. Note the curved paths at the top where particles ejected into a region of poor fitness curve round and return to the then best-known minimum (minimum A). Compare this with the paths left by particles diverted from their exploration by an attraction to the just discovered minimum B. The heavy concentration of points between the two minima come from particles that were in the vicinity of minimum A when minimum B was found being attracted to this new, better, minimum. Table 2. Results of the number of times the two minima were found and how long this took. Results are derived from 1 repeats for each set of parameter values for random initial particle positions. The parameter values that result in the minimum time to the second minimum are shaded. Maximum: First minimum: Second minimum: Max Initial speed Distance travelled per iteration K % times found Average iterations SD % times found Average iterations SD Copyright 25

8 2 test3 2 test Figure 4. Two screen images of a particle swarm with shortrange forces solving the test problem described in this paper. Each dot represents the position of a particle Map of all evaluations Figure 5. A map of all the points evaluated. Every point at which a fitness evaluation has been made is indicated by a small dot on the above map. 6 Conclusion The purpose of this work was to test the utility of adding a short-range force to the normal PSO algorithm. The two minimum test problem was deliberately designed to make it hard for a swarm released in the vicinity of minimum A to find the better minimum B. For this problem the short-range force has provided a dramatic improvement in performance in that it allowed the better minimum B to be found reliably while the basic PSO algorithm was not able to find it at all. This has come at the cost of two additional parameters, the values of which do not appear overly critical. The normal swarm algorithm is simultaneously operating with this short-range force. It must be stressed that this short-range force only has a significant scattering effect when 8 Copyright 25

9 particles are very close, most commonly during the settling stage of a swarm. At other times the short-range force only adds what is effectively a small amount of noise to the normal behaviour of the swarm. These experiments have not shown any observable change in the behaviour of the PSO other than this modification of behaviour during the settling phase. Preliminary experiments so far have used the simple formula for the short-range force given in this paper, but the power to which the particle separation distance is raised could be changed so as to make this attraction even more short-ranged. In these experiments, the short-range force between each particle and all others in the swarm is calculated - again in order to negate the need for a user defined neighbourhood - although in practice the number of occasions on which there is more than one significant contribution are few. When applied to the stated problem with multiple local minima, the addition of the shortrange force improves the probability of the swarm finding the global optimum rather than converging and staying on a good but suboptimal position, and there is no reason to believe that this will not be the general result when applied to other multi-optimal problems. As no additional fitness evaluations are required, the additional computational cost of the behaviour modification is relatively small when compared to the overall computation involved, especially for problems with complex fitness functions. 9 References 1 Kennedy J., and Eberhart R.C. Particle Swarm Optimization, Proc. IEEE International Conference on Neural Networks, Perth Australia, IEEE Service Centre, Piscataway NJ USA IV: (1995) 2 Eberhart. R. C., Dobbins P. and Simpson P. Computational Intelligence PC Tools, Academic Press, Boston. (1996) 3 Corne D., Dorigo M., and Glover F. (Editors), The Particle Swarm: Social Adaptation in Information-Processing Systems. Chapter 25 in New Ideas in Optimization. McGraw-Hill Publishing Company, England, ISBN (1999). 4 Eberhart, R. and Shi, Y. Comparing Inertia Weights and Constriction Factors in Particle Swarm Optimisation, Proceedings of the 2 Congress on Evolutionary Computation, pp (2) 5 Hendtlass, T. A Combined Swarm Differential Evolution Algorithm for Optimization Problems. Lecture Notes in Artificial Intelligence, Vol 27, pp , Springer, Berlin. (21) 9 Copyright 25