Adapting the Pheroone Evaporation Rate in Dynaic Routing Probles Michalis Mavrovouniotis and Shengxiang Yang School of Coputer Science and Inforatics, De Montfort University The Gateway, Leicester LE1 9BH, United Kingdo.avrovouniotis@hotail.co and syang@du.ac.uk Abstract. Ant colony optiization (ACO) algoriths have proved to be able to adapt to dynaic optiization probles (DOPs) when stagnation behaviour is avoided. Several approaches have been integrated with ACO to iprove its perforance for DOPs. The adaptation capabilities of ACO rely on the pheroone evaporation echanis, where the rate is usually fixed. Pheroone evaporation ay eliinate pheroone trails that represent bad solutions fro previous environents. In this paper, an adaptive schee is proposed to vary the evaporation rate in different periods of the optiization process. The experiental results show that ACO with an adaptive pheroone evaporation rate achieves proising results, when copared with an ACO with a fixed pheroone evaporation rate, for different DOPs. 1 Introduction Ant colony optiization (ACO) algoriths have shown good perforance when applied to difficult optiization probles under static environents [2]. However, in any real-world applications, we have to deal with dynaic environents, where the optiu changes and needs re-optiization. It is believed that ACO algoriths can adapt to dynaic changes since they are inspired fro nature, which is a continuous adaptation process[7]. In practice, they can adapt by transferring knowledge fro past environents, using the pheroone trails, to speed up re-optiization. The challenge to such algoriths lies in how quickly they can react to dynaic changes in order to aintain the high quality of output instead of early stagnation behaviour, where all ants construct the sae solutions and lose their adaptation capabilities. Developing strategies for ACO algoriths to deal with stagnation behaviour and address dynaic optiization probles (DOPs) has attracted a lot of attention, which includes local and global restart strategies [6], eory-based approaches [5], pheroone anipulation schees to aintain diversity [3], and iigrants schees to increase diversity [8]. The adaptation capabilities of ACO rely on the pheroone evaporation where a constant aount of pheroone is deducted to eliinate pheroone trails that represent bad solutions that ay bias ants to search to the non-proising areas of the search space. In this paper, the ipact of the pheroone evaporation rate
is exained on DOPs, and an adaptive schee is designed for ACO. Adaptive ethods have been successfully applied for different paraeters of ACO, including the evaporation rate [10, 12]. However, these ethods have been investigated on static optiization probles. The rest of the paper is organized as follows. Section 2 describes the generation of dynaic routing DOPs. Section 3 describes an ACO algorith and gives details for its adaptation capabilities in DOPs. Section 4 describes the proposed schee where the evaporation rate in ACO is adapted. Section 5 describes the experients carried out on a series of different DOPs. Finally, Section 6 concludes this paper with directions for future work. 2 Generating Dynaic Routing Environents Routing probles are usually illustrated using weighted graphs. Let G = (V, E) be a weighted graph where V is a set of n nodes and E is a set of links. Each node i has a location defined by (x,y) and each link (i,j) is associated with a non-negative distance d ij. Usually, the distance atrix of a proble instance is defined as D = (d ij ) n n. In order to generate dynaic routing probles, the dynaic benchark generator for perutation-encoded probles (DBGP) [9] is used, which converts any static proble instance to a dynaic environent. In case the optiu of the static proble instance is known, then it will reain known during the environental changes, because DBGP biases algoriths to search to a new location in the fitness landscape, instead of odifying the fitness landscape. Every f iterations a rando vector r(t) is generated that contains all the objects of a proble instance of size n, where T = t/f is the index of the period of change, t is the iteration count of the algorith, and f deterines the frequency of change. The agnitude of change depends on the nuber of swapped locations of objects. More precisely, [0.0, 1.0] defines the degree of change, in which only the first n of r(t) object locations are swapped. Then arandolyre-orderedvectorr (T) isgeneratedthatcontainsonlythe first n objects of r(t). Therefore, exactly n pairwise swaps are perfored using the two rando vectors. 3 ACO in Dynaic Environents 3.1 MAX-MIN Ant Syste The ACO etaheuristic consists of a population of µ ants where they construct solutions and share their inforation aong each other via their pheroone trails. The first ACO algorith developed is the Ant Syste (AS) [1]. Many variations of the AS have been applied to difficult optiization probles [2]. OneofthebestperforingACOalgorithistheMAX-MIN AS(MMAS) [11]. Ants read and write pheroones in order to construct their solutions. Each
ant k uses a probabilistic rule to choose the next city to visit. The decision rule an ant k uses to ove fro city i to city j is defined as follows: p k ij = [τ ij ] α [η ij ] β [τ il ] α β,if j Nk i, (1) [η il ] l N k i where τ ij is the existing pheroone trail between cities i and j, η ij = 1/d ij is the heuristic inforation available a priori, where d ij is the distance between cities i and j. Ni k denotes the neighbourhood of cities for ant k when the current city is i. α and β are the two paraeters which deterine the relative influence of τ and η, respectively. The pheroone trails in MMAS are updated by applying evaporation as follows: τ ij (1 ρ)τ ij, (i,j), (2) where ρ is the evaporation rate which satisfies 0 < ρ 1, and τ ij is the existing pheroone value. After evaporation the best ant deposits pheroone as follows: τ ij τ ij + τ best ij, (i,j) T best, (3) where τij best = 1/C best is the aount of pheroone that the best ant deposits and C best defines the solution quality of tour T best. Since only the best ant deposits pheroone, the algorith will quickly converge towards the best solution of the first iteration. Therefore, pheroone trail liits are iposed in order to avoid this behaviour. 3.2 Response to Dynaic Changes ACO algoriths are able to use knowledge fro previous environents using the pheroone trails generated in the previous iterations. For exaple, when the changing environents are siilar, the pheroone trails of the previous environent ay provide knowledge to speed up the optiization process to the new environent. However, the algorith needs to be flexible enough to accept the knowledge transferred fro the pheroone trails, or eliinate the pheroone trails, in order to adapt well to the new environent. ACO algoriths can be applied directly to DOPs without any odifications due to the pheroone evaporation. Lowering the pheroone values enables the algorith to forget bad decisions ade in previous iterations. When a dynaic change occurs, evaporation eliinates the pheroone trails of the previous environent fro areas that are not visited frequently and ay bias ants not to adapt well to the new environent. The adaptation via pheroone evaporation ay be a sufficient choice when the changing environents are siilar, otherwise a coplete re-initialization of the pheroone trails after a dynaic change occurs ay be a better choice. However, such action is available only in DOPs where the frequency of change is available beforehand or in DOPs where the dynaic changes can be detected. In our case, the dynaic changes can be detect by re-evaluating soe stored solutions, used as detectors, in every iteration [8].
4 ACO with Adaptive Evaporation Rate 4.1 Effect of the Pheroone Evaporation Rate Although ACO has adaptation capabilities due to the pheroone evaporation, the tie required to adapt to the new environent ay depend on the proble size and the agnitude of change. When the environental change is severe then it ay take longer to eliinate unused pheroone trails, therefore a high evaporation rate ay be ore suitable. More precisely, a high evaporation rate will eliinate the high intensity of pheroone trails that are usually concentrated to the optiu of the previous environent that is caused by stagnation behaviour. On the other hand, a high pheroone evaporation rate ay destroy inforation that can be used on further environents, since any bad solution in the current environent ay be good in the next environent. In traditional ACO algoriths the evaporation rate is usually fixed. A low evaporation rate corresponds to slow adaptation, whereas a high evaporation rate corresponds to fast adaptation. However, we believe that a fixed evaporation rate is not the best choice when addressing DOPs, since at different stages of the optiization process for different optiization probles and under different dynaic environents, the ost appropriate evaporation rate varies. 4.2 Detect Stagnation Behaviour In order to adapt the value of the pheroone evaporation during the search process, the exploration of the algorith is easured in order to detect stagnation behaviour. A direct way that can give an indication of exploration is to easure the diversity of the solutions. The easureent for routing probles is usually based on the coon edges between the solutions [8]. Such a easure ay be coputational expensive since there are O(n 2 ) possible pairs to be copared and each single coparison has a coplexity of O(n). A ore efficient easureent is the λ-branching factor [4], which easures the distribution of the pheroone trail values. The idea of λ-branching is described as follows: If for a given object i V, the concentration of pheroone trails on alost all the incident arcs becoes very sall but is large for a few others, then the freedo of exploring other paths fro object i is very liited. Therefore, if this situation arises siultaneously for all objects of graph G, the search space that is searched by ants becoes relatively sall. The average λ(t) branching factor at iteration t is defined as follows: λ(t) = 1 2n n λ i, (4) where n is the nuber of objects in the corresponding graph and λ i is the λ- branching factor for object i, which is defined as follows: λ i = i=1 d L ij (5) j=1
where d is the nuber of available arcs incident to object i and L ij is defined as follows: { 1, if (τin i L ij = +λ(τi ax τi in )) τ ij, (6) 0, otherwise. where λ is a constant paraeter (λ = 0.05 by default [4]), τin i and τi ax are the iniu and axiu pheroone trail values on the arcs incident to object i, respectively. A value of λ(t) close to 1 indicates stagnation behaviour. 4.3 Adapting Pheroone Evaporation Rate Considering the stateents above, if the algorith reaches stagnation behaviour, the evaporation rate needs to be increased in order to eliinate the high intensity of pheroone trails in soe areas and increase exploration. However, very high exploration ay disturb the optiization process because of randoization [8]. According to the behaviour of the algorith in ters of searching, we have the following pheroone evaporation rate update rule: { ρ(t 1) σ, if ρ(t) = λ(t) > 1, (7) ρ(t 1)+σ, otherwise. where λ(t) is defined in Eq. (4) and σ is the step size of varying the evaporation rate ρ at iteration t. A good value of σ was found to be 0.001 because a higher value ay quickly increase ρ to an extree evaporation rate and destroy inforation and a saller value ay not have any effect to the perforance of ACO. 5 Experiental Study 5.1 Experiental Setup In the experients, we copare a MMAS with a global re-initialization of the pheroone trails, denoted as MMAS R, and a MMAS with the best fixed evaporation rate, denoted as MMAS B against the MMAS with the proposed adaptive pheroone evaporation, denoted as MMAS A. For all algoriths, we set α = 1, β = 5, q 0 = 0.0, and µ = 50, except for MMAS R where µ = 50 d T, where d T = 6 is the nuber of detectors. The evaporation rate for MMAS R was set to ρ = 0.4. For MMAS B the best value fro ρ {0.02,0.2,0.4,0.6,0.8} was selected, whereas for MMAS A ρ was adapted by Eq. (7). For each algorith on a DOP, N = 30 independent runs were executed on the sae environental changes. The algoriths were executed for G = 1000 iterations and the overall offline perforance is calculated as follows: P offline = 1 G 1 N P ij (8) G N i=1 j=1
26500 26000 25500 25000 24500 24000 kroa100, f = 10 23500 23000 22500 ρ=0.8 22000 23500 kroa100, f = 100 34000 33000 32000 31000 30000 29000 30000 kroa150, f = 10 kroa150, f = 100 38000 37000 36000 35000 34000 ρ=0.8 33000 34000 kroa200, f = 10 kroa200, f = 100 ρ=0.8 23000 22500 22000 ρ=0.8 21500 29500 29000 28500 28000 27500 ρ=0.8 27000 33500 33000 32500 32000 31500 31000 30500 ρ=0.8 30000 Fig. 1. Ipact of the evaporation rate on the offline perforance of a conventional MMAS on different DOPs. where Pij defines the tour cost of the best ant since the last dynaic change of iteration i of run j [7]. We took three travelling salesan proble (TSP) instances 1 and three vehicle routing proble (VRP) instnaces 2 as the base and used the DBGP described in Section 2 to generate DOPs. The value of f was set to 10 and 100, which indicate fast and slowly changing environents, respectively. The value of was set to 0.1, 0.25, 0.5, and 0.75, which indicate the degree of environental changes fro sall, to ediu, to large, respectively. As a result, eight dynaic environents, i.e., 2 values of f 4 values of, for each proble instance are generated to systeatically analyze the adaptation and searching capability of algoriths on the DOPs. 5.2 Experiental Results and Analysis The experiental results regarding the different ρ values for MMAS B on dynaic TSPs are presented in Fig. 1. Note that the corresponding experiental results for dynaic VRPs show siilar observations and are not presented here. The offline perforance of the different algoriths on dynaic TSPs and dynaic VRPs and the corresponding statistical results of Wilcoxon rank-su test, at the 0.05level ofsignificanceare presentedin Table 1 and Table 2, respectively. Moreover, the dynaic behaviour of the algoriths is presented in Fig. 2. Fro the experiental results, several observations can be ade by coparing the behaviour of the algoriths. 1 http://coopt.ifi.uni-heidelberg.de/software/tsplib95/. 2 http://neo.lcc.ua.es/vrp/.
Table 1. Experiental results of the algoriths regarding the offline perforance. Travelling Salesan Proble Instances f = 10 f = 100 kroa100(optiu=21282) MMAS B 22010 23844 24989 25401 21570 21850 22227 22430 MMAS A 22069 23542 24448 24800 21683 21819 22049 22468 MMAS R 24576 24580 24583 24588 22244 22252 22224 22212 kroa150(optiu=26524) MMAS B 28488 31013 32070 32426 27315 27814 28330 28526 MMAS A 28690 30507 31444 31778 27299 27726 28140 28262 MMAS R 31520 31526 31515 31516 28208 28204 28215 28198 kroa200(optiu=29368) MMAS B 32454 35300 36394 36711 30071 30796 31644 31863 MMAS A 32353 34524 35560 35872 30167 30645 31245 31506 MMAS R 35375 35368 35362 35375 31282 31368 31338 31380 Vehicle Routing Proble Instances f = 10 f = 100 F-n45-k4(Optiu=724) MMAS B 800.42 807.56 816.98 823.20 792.79 796.24 796.33 797.65 MMAS A 800.55 805.22 814.02 820.39 795.63 797.66 797.22 799.04 MMAS R 812.16 812.06 812.34 812.16 799.76 800.46 799.55 799.70 F-n72-k4(Optiu=237) MMAS B 268.69 279.67 285.76 288.36 259.95 263.00 265.82 266.81 MMAS A 270.36 281.13 286.47 289.20 261.17 263.76 267.18 267.44 MMAS R 291.09 291.14 291.14 291.23 270.64 270.91 271.11 270.61 F-n135-k7(Optiu=1162) MMAS B 1298.71 1339.50 1365.33 1375.41 1255.00 1271.66 1286.62 1291.91 MMAS A 1297.46 1335.77 1363.29 1372.89 1255.21 1269.49 1283.98 1288.14 MMAS R 1348.90 1348.65 1348.87 1348.85 1281.08 1283.68 1283.35 1282.20 First,when the evaporationrate is setto ρ = 0.02,whichis the recoended value formmas on static probles[2, p. 71],has the worstresults ondops, as observed fro Fig. 1. Furtherore, a high evaporation rate, i.e., ρ 0.4, often achieves better perforance when f = 10. This is natural because when the environent changes quickly, a fast adaptation is required. When f = 100, the evaporation rate depends on the agnitude of change, e.g., when = 0.1, ρ = 0.2 shows better perforance. However, as the agnitude of change increases, a higher value of ρ achieves better perforance. This validates our clai that the tie required for ACO, in which pheroone evaporation is used, to adapt to the new environent depends on the agnitude of change. Second,MMAS B isoutperforedbymmas R onostdopswith = 0.5 and = 0.75, whereas the forer outperfors the latter on all DOPs with = 0.1 and = 0.25; see the coparisons MMAS B MMAS R in Table 2.
Table 2. Statistical tests of coparing algoriths regarding the offline perforance, where + or eans that the first algorith is significantly better or the second algorith is significantly better, respectively, and eans that the algoriths are not significantly different. Travelling Salesan Proble Instances kroa100 kroa150 kroa200 f = 10, MMAS A MMAS B + + + + + + + + + + MMAS A MMAS R + + + + + + + + MMAS B MMAS R + + + + + + f = 100, MMAS A MMAS B + + + + + + + MMAS A MMAS R + + + + + + + + + MMAS B MMAS R + + + + + + Vehicle Routing Proble Instances F-n45-k4 F-n72-k4 F-n135-k7 f = 10, MMAS A MMAS B + + + + + + MMAS A MMAS R + + + + + + + + MMAS B MMAS R + + + + + + + + f = 100, MMAS A MMAS B MMAS A MMAS R + + + + + + + + + MMAS B MMAS R + + + + + + + + + This is because when the environents are siilar, due to a slight change, the pheroone trails of the previous environent help to start the optiization process fro a proising area in the search space, whereas when the environents are different, due to a severe change, the pheroone trails of the previous environent islead the searching to non-proising areas. This validates our clai that the adaptation of pheroone evaporation is useful when the environents are siilar and useful knowledge can be transferred. Finally, the proposed MMAS A outperfors MMAS B on ost TSP DOPs when = 0.25, = 0.5 and = 0.75, whereas the forer is coparable with the latter when = 0.1; see the coparisons MMAS A MMAS B in Table 2. This is probably because when the evaporation is high, it ay destroy useful knowledge fro the previous environent after a dynaic change. Therefore, a low evaporation rate soeties ay be a better choice, even when the dynaic change is severe, for the first iterations after a dynaic change to obtain knowledge, and a higher evaporation rate ay be a better choice later on to avoid the stagnation behaviour. This can be observed fro Fig. 2 where MMAS A convergesfasterandtoabetteroptiuthanmmas B.Fortunately, even if MMAS A does not achieve the best result copared to MMAS B, e.g., when f = 100 for VRP DOPs, its perforance level is still satisfactory since they are usually not significantly different. This can be expected since the re-
40000 38000 36000 34000 32000 kroa200, f = 100, = 0.1 MMAS R MMAS B MMAS A 42000 40000 38000 36000 34000 32000 kroa200, f = 100, = 0.75 MMAS R MMAS B MMAS A 30000 0 100 200 300 400 500 Iteration F-n135-k7, f = 100, = 0.1 30000 0 100 200 300 400 500 Iteration F-n135-k7, f = 100, = 0.75 1400 MMAS R MMAS B MMAS A 1450 MMAS R MMAS B MMAS A 1350 1300 1400 1350 1300 1250 1250 0 100 200 300 400 500 Iteration 0 100 200 300 400 500 Iteration Fig. 2. Dynaic behaviour of the algoriths with respect to offline perforance against the iterations in slowly changing environents for the first 500 iterations in different DOPs: 1) kroa200 for TSP; and 2) F-n135-k7 for VRP. sults of MMAS B are obtained via fine-tuning the evaporation rate. Moreover, MMAS A outperforsmmas R inostdops,expectwhen = 0.75,because of the sae reasons discussed for MMAS B previously. 6 Conclusions This paper exaines the ipact of the pheroone evaporation on the perforance of ACO algoriths for DOPs. An adaptive evaporation rate is proposed for ACO to deal with DOPs, which is based on the detection of the stagnation behaviour. Experiental studies were perfored on a series of DOPs to investigate the perforance of the proposed approach. Fro the experiental results, several conclusions can be drawn. First, pheroone evaporation is iportant for ACO to address DOPs. Second, the higher the agnitude of the dynaic change, the higher the evaporation rate is needed. Third, the adaptation capabilities of pheroone evaporation perfor well only when the environents are siilar; otherwise, a re-initialization of the pheroone trails is required. Forth, the proposed adaptive evaporation rate prootes the perforance of ACO in any routing DOPs but depends on the dynaics and the type of the DOP.
Finally, copared to the tedious work of fine-tuning the pheroone evaporation rate anually, the proposed adaptive schee is ore convenient and has sufficiently good perforance under different conditions. However, the perforance is slightly decreased in soe cases, for the sake of this convenience. For future work, it will be interesting to consider other ways for adapting the evaporation rate. Moreover, there are evidence that the ore paraeters adapted in ACO, the better the perforance in optiization probles with static environent [10]. Therefore, another future work is to adapt ore ACO paraeters, e.g., α and β, in parallel for DOPs. Acknowledgeent This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of U.K. under Grant EP/K001310/1. References 1. Dorigo, M., Maniezzo, V., Colorni, A.: Ant syste: optiization by a colony of cooperating agents. IEEE Trans. on Syst., Man and Cybern., Part B: Cybern. 26(1), 29 41 (1996) 2. Dorigo, M., Stützle, T.: Ant colony optiization. The MIT Press, London (2004) 3. Eyckelhof, C. J., Snoek, M.: Ant systes for a dynaic TSP: Ants caught in a traffic ja. In: ANTS 2002. LNCS, vol. 2463, pp. 88 99. Springer, Heidelberg (2002) 4. Gabardella, M. L., Dorigo, M.: Ant-Q: A reinforceent learning approach to the traveling salesan proble. In: Proc of the 12th Int. Conf. on Machine Learning, pp. 252-260. Morgan Kaufann (1995) 5. Guntsch, M., Middendorf, M.: Applying population based ACO to dynaic optiization probles. In: ANTS 2002. LNCS, vol. 2463, pp. 111 122. Springer, Heidelberg (2002) 6. Guntsch, M., Middendorf, M.: Pheroone odification strategies for ant algoriths applied to dynaic TSP. In: Boers, E. (eds) EvoApplications 2001. LNCS, vol. 2037, pp. 213-222. Springer, Heidelberg (2001) 7. Jin, Y., Branke, J.: Evolutionary optiization in uncertain environents - a survey. IEEE Trans. on Evol. Coput. 9(3), 303 317 (2005) 8. Mavrovouniotis, M., Yang, S.: Ant colony optiization with eory-based iigrants for the dynaic vehicle routing proble. In: Proc. of the 2012 IEEE Congress on Evol. Coput., pp. 2645-2652. IEEE Press (2012) 9. Mavrovouniotis, M., Yang, S., Yao, X.: A benchark generator for dynaic perutation-encoded probles. In: PPSN XII. LNCS, vol. 7492, pp. 508 517. Springer, Heidelberg (2012) 10. Pellegrini, P., Stützle, T., Birattari, M.: A critical analysis of paraeter adaptation in ant colony optiization. Swar Intelli. 6(1), 23 48 (2012) 11. Stützle, T., Hoos, H.: The MAX-MIN AntSyste and local search for the traveling salesan proble. In: Proc. of the 1997 IEEE Int. Conf. on Evol. Coput., pp. 309 314. IEEE Press (1997) 12. Stützle, T., López-Ibáñez, M., Pellegrini, P., Maur, M., de Oca, M.M., Birattari, M., Dorigo, M.: Paraeter adaptation in ant colony optiization. Autonoous Search, 191 215. Springer, Heidelberg (2012)