Sensitive Ant Model for Combinatorial Optimization

Transcription

1 Sensitive Ant Model for Combinatorial Optimization CAMELIA CHIRA D. DUMITRESCU CAMELIA-MIHAELA PINTEA Abstract: A combinatorial optimization metaheuristic called Sensitive Ant Model (SAM) based on the Ant Colony System (ACS) technique is proposed. ACS agents cooperate indirectly using pheromone trails. SAM improves and extends the ACS approach by enhancing each agent of the model with properties that induce heterogeneity. Agents are endowed with different pheromone sensitivity levels. Highly-sensitive agents are essentially influenced in the decision making process by stigmergic information and thus likely to select strong pheromone-marked moves. Search intensification can be therefore sustained. Agents with low sensitivity are biased towards random search inducing diversity for exploration of the environment. SAM is studied on a dynamic optimization problem. Numerical experiments offer promising results indicating the potential of the proposed SAM metaheuristic. Key Words: Ant systems, Metaheuristics, Stigmergy, Sensitivity, Heterogeneous system LATEX 1 Introduction Complex optimization problems arise in diverse areas including network design, storage and retrieval, sequencing and scheduling, mathematical programming, algebra and number theory, games and puzzles, automata and language theory and program optimization. Many of these combinatorial optimization problems are NP-hard and cannot be solved within polynomial computation times. Metaheuristics are powerful strategies that can detect efficiently high-quality (near optimal) solutions to complex problems within reasonable running time. A new metaheuristic technique called Sensitive Ant Model (SAM) that combines stigmergic communication and heterogeneous agent behavior is proposed. The model of stigmergic communication is borrowed from the Ant Colony System (ACS) technique - a metaheuristic inspired by the behavior of real ant colonies [1]. Agents are endowed with the capability of indirect communication triggered by changes that occur in the environment. Similar with the ACS model, each SAM agent (or ant) deposits a substance called pheromone on a specific path. Stigmergic communication is based on different reactions to virtual pheromone trails produced by agents. It is proposed to induce heterogeneity in the SAM approach via specific system agent properties. These properties can refer to variable pheromone sensitivity, variable agent life time or the particular heuristics engaged by each agent in the search process. In the proposed SAM metaheuristic, agents are endowed with different pheromone sensitivity levels. Highly-sensitive agents are essentially influenced in the decision making process by stigmergic information and thus likely to select strong pheromonemarked moves. Search intensification can be therefore sustained. Agents with low sensitivity are biased towards random search and are able to sustain diversity for exploration of the environment. The proposed SAM approach extends and improves the basic ACS basic model by endowing agents with a certain degree of non-homogeneity. SAM agents are characterized by a specific sensitivity to pheromone traces inducing the idea of heterogeneity in the model. We expect that the search capabilities of agents are improved particularly for complex solution spaces. A heterogeneous agent model has the potential to cope with complex and/or dynamic search spaces. Sensitive agents (or ants) allow many types of reactions to a changing environment facilitating an efficient balance between exploration and exploitation. The introduced SAM metaheuristic is tested for solving a dynamic drilling problem that can be viewed as an instance of the Dynamic Generalized Traveling Salesman Problem (GTSP). Numerical results indicate that the proposed SAM algorithm outperforms the classic ACS model suggesting a promising potential for heterogeneous ant-based models. ISSN: ISBN:

2 2 Stigmergy Stigmergy provides a general mechanism that relates individual and colony level behaviors: individual behavior modifies the environment, which in turn modifies the behavior of other individuals. The environment mediates the communication among individuals. Indirect communication can be facilitated by means of food or liquid exchange, mandibular contact, visual contact and others [2]. Self-organization is hence made possible according to the intensity of the stigmergic interactions among individuals that can adhere to a wide range of interactions [3]. Proposed SAM model involves several agents able to communicate in a stigmergic manner (influenced by pheromone trails) for solving complex search problems. Stigmergic behavior is similar to that of Ant Colony System (ACS) agents [4]. An ant algorithm is essentially a system based on agents that simulate the natural behavior of ants including mechanisms of cooperation and adaptation. The ACS metaheuristic is a particular class of ant algorithms. The behavior of insects is replicated to the search space. The decisions of the ants regarding the path to follow when arriving to a path intersection are influenced by the amount of pheromone on the path. Stronger pheromone trails are preferred and the most promising paths receive a greater pheromone trail after some time. The result of an ACS algorithm is the shortest path found - hopefully corresponding to the optimal solution of the given problem. Let β be a parameter used for tunning the relative importance of edge length in selecting the next node. Let us denote J k i the unvisited successors of node i by ant k and u J k i. q is a random variable uniformly distributed over [0, 1] and q 0 is a parameter similar to the temperature in simulated annealing, 0 q 0 1. The probability p iu of choosing j = u as the next node if q > q 0 (the current node is i) is defined as [4]: p iu (t) = [τ iu (t)][η iu (t)] β Σ o J k i [τ io (t)][η io (t)] β, (1) where τ iu (t) refers to the pheromone trail intensity on edge (i, u) at time t, and η iu (t) represents the visibility of edge (i, u). If q q 0 the next node j is chosen according to the following rule [4]: j = argmax u J k i {τ iu (t)[η iu (t)] β }. (2) Well known and robust algorithms include Ant Colony System [4] and MAX MIN Ant System [5]. Standard ACS rely on a set of homogeneous stigmergic agents. This means that any two agents are equivalent and interchangeable. Homogeneous agents may have some difficulties to cope with very complex search spaces. We propose to endow the agent system with a certain degree of heterogeneity and expect that the agent s search capabilities are improved. Several concepts of heterogeneity may be used for this purpose (for instance, the agent s life time can vary). 3 Sensitivity In the proposed SAM approach, we focus on the ability of agents to detect pheromone traces. Agents are heterogeneous as they are endowed with different levels of sensitivity to pheromone. This variable sensitivity can potentially induce various types of reactions to a changing environment. It is expected that a better balance between search diversification and search exploitation can be achieved by combining stigmergic communication with heterogeneous agent behavior. Within the proposed model each agent is characterized by a pheromone sensitivity level (PSL). The PSL value is expressed by a real number in the unit interval [0, 1]. Extreme situations are: If PSL = 0 the agent completely ignores stigmergic information (the agent is pheromone blind ); If PSL = 1 the agent has maximum pheromone sensitivity (it is a standard ACS ant). Low PSL values indicate that agents can choose very high pheromone-marked moves but are more likely to make a decision independent from the stigmergic information available in the environment (as the agent has reduced pheromone sensitivity). These agents are more independent and can be considered environment explorers. They have the potential to autonomously discover new promising regions of the solution space. Therefore, search diversification can be sustained. Agents with high PSL values are very sensitive to pheromone traces and therefore likely to be influenced by stigmergic information. They can choose any pheromone marked move. Agents of this category are able to intensively exploit the promising search regions already identified. In this case the agent behavior emphasizes search intensification. During their lifetime agents may improve their performance by learning. This process translates to modifications of the pheromone sensitivity. PSL value can increase or decrease according to the search space topology encoded in the agent experience. ISSN: ISBN:

3 4 The Sensitive Ant Model The Sensitive Ant Model uses a set of heterogeneous agents (sensitive ants) able to communicate in a stigmergic manner and make decisions individually influenced by changes in the environment and based on pheromone sensitivity levels specific to each agent. Sensitive ants act exactly like classical ACS ants when PSL value is one. At the opposite side, sensitive ants characterized by a PSL level of zero choose the path in a random manner completely ignoring pheromone trails. For instance, decisions within a system of agents with null sensitivity can be developed using the random walk algorithm. A measure of randomness proportional to the level of PSL is introduced in the rest of the cases, i.e. when 0 < P SL < 1. (3) It is proposed to achieve this by modifying the transition probabilities using the PSL values in a renormalization process. Consider p iu (t, k) the probability for agent k of choosing the next node u from current node i (as given in ACS). Let us denote by sp iu (t, k) the renormalized transition probability for agent k (influenced by PSL) used in the SAM model. In the proposed SAM approach renormalization is accomplished via the following equation: sp iu (t, k) = p iu (t, k) P SL(t, k), (4) where PSL(t, k) represents the PSL value of agent k at time t. It should be noted that if then for each node i we have P SL(t, k) 1 (5) sp iu (t, k) < 1. (6) u In order to associate a standard probability distribution to the system, a virtual state denoted by vs - corresponding to the lost probability - is introduced. The transition probability associated to the virtual state vs is defined as sp i,vs (t, k) = 1 u sp iu (t, k). (7) Therefore, for agent k at moment t we may write sp i,vs (t, k) = 1 P SL(t, k) u and thus p iu (t, k), (8) sp i,vs (t, k) = 1 P SL(t, k). (9) The renormalized probability sp i,vs (t, k) can be correlated to the system heterogeneity at time t. We may interpret sp i,vs (t, k) as the granular heterogeneity of the agent k at iteration t. A measure of the system heterogeneity may be expressed as E = k (sp i,vs (t, k)) 2. (10) i Minimum heterogeneity is associated with normal (ACS) stigmergic agents (maximum sensitivity to pheromone of SAM agents). Maximum heterogeneity corresponds to pheromone-blind agents (SAM agents with zero sensitivity). Quantity E indicates how far a SAM system is from the standard associated ACS system. Alternatively, E may interpreted as measuring the heterogeneity or novelty of a SAM system with respect to the corresponding ACS variant. Several transition mechanisms including virtual state are possible. In our model, next move is selected according to the pseudo-random proportional rule (as given for ACS) in which the renormalized transition probabilities (involved in SAM) are considered. The SAM approach has to specify the action associated with the virtual state introduced. If the selected state is vs then an accessible state is chosen randomly with uniform probability. This mechanism is called virtual state decision rule and it concentrates the essence of the proposed approach. Proposed SAM approach ensures the increasing of randomness in the selection process with the decreasing of the pheromone sensitivity level PSL for an agent. 5 Numerical Experiments The SAM metaheuristic is engaged for solving various instances of the dynamic Generalized Traveling Salesman Problem - a well-known NP-hard optimization problem. The description of the problem, the SAM algorithm and numerical results are presented. 5.1 The Dynamic Generalized Traveling Salesman Problem Let G = (V, E) be an n-node undirected complete graph in which all edges are associated with nonnegative costs. ISSN: ISBN:

4 Let V 1,..., V p be a partition of V into p subsets called clusters. The cost of an edge (i, j) E is c(i, j). The Generalized Traveling Salesman Problem (GTSP) [6], refers to finding a minimum-cost tour H spanning a subset of nodes such that H contains exactly one node from each cluster V i, i {1,..., p}. The problem involves two related decisions: choosing a node subset S V, such that S V k = 1, for all k = 1,..., p and finding a minimum cost cycle (Hamiltonian tour) in S (the subgraph of G induced by S). Let c be the cost function associated to the edges of G. The GTSP is called symmetric if and only if the equality c(i, j) = c(j, i) holds for every i, j V. The Dynamic Generalized Traveling Salesman Problem (Dynamic GTSP) is a variation of the GTSP that addresses dynamic changes in the problem (e.g. blocked edges, cost variation). The Dynamic GTSP can be described as follows: starting from a node within a specific group of nodes (a cluster) an agent has to visit a given number of nodes. The agent goes exactly once through each distinct and available (unblocked) group of nodes. The tour is complete when the agent returns in a node (not necessarily the starting node) from the initial group of nodes. 5.2 SAM Algorithm The SAM algorithm for solving dynamic GTSP uses m agents that are initially randomly placed in the clusters. Each agent is randomly associated to a PSL value. The selection of the next cluster is guided by the renormalized transition probabilities given by (4) and (8). Variable PSL values enable each transition to be more or less biased towards the virtual state i.e. towards a completely random choice of the next move. At each iteration, only the ant generating the best tour is allowed to globally update the pheromone. The global update rule is applied to the edges belonging to the best tour. Let L + be the length of the best tour. Let τ ij (t) be the inverse length of the best tour: τ ij (t) = 1 L +. (11) According to [4] the global pheromone correction rule is: τ ij (t + 1) = (1 ρ)τ ij (t) + ρ τ ij (t). (12) Let nc be the number of clusters used in GTSP (currently defined as the integer part of (n/5)). The SAM algorithm for Dynamic GTSP is outlined below: SAM Algorithm for Dynamic GTSP begin for every edge (i, j) do τ ij (0) = τ 0 place agent k on a specified chosen node from a specified cluster let T + be the shortest tour found let L + be the T + length for t = 1 to N iter do each iteration a random cluster is set visited build tour T k (t) by applying (nc 1) times choose the next node j from an unvisited cluster based on (1),(2), the renormalized probabilities (4), (8) and virtual state decision rule update pheromone trails end for compute the length L k (t) of the detected tour T k (t) if an improved tour is found then update T k (t) and L k (t) for every edge (i, j) T + do update pheromone trails: apply the global rule (12) end for print the shortest tour T + and its length L + end 5.3 Numerical Results Numerical experiments are based on all drilling problem instances that can be found in the TSP library [7]. The drill problems with Euclidean distances have been considered. Parameter setting in the SAM algorithm is: β = 5, τ 0 =0.01, ρ = 0.01 and q 0 = 0.9. The total number of ants considered is 3. The results obtained by the proposed SAM technique have been compared to the standard ACS model. Table 1 shows the numerical results for four problem instances after 50 runs. SAM detects better average solutions for three of the four problems considered. Experiments indicate that ACS is outperformed by the proposed SAM technique for the considered set of dynamic problems. ISSN: ISBN:

5 Table 1: SAM and ACS numerical results for dynamic generalized drilling problem Problem SAM SAM ACS ACS Avg. Min. Avg. Min. U D FL PCB Conclusions and Future Work The proposed Sensitive Ant Model (SAM) combines stigmergic communication and heterogeneous agent behavior. SAM technique involves several agents able to communicate in a stigmergic manner (influenced by pheromone trails) for solving complex search problems. SAM agent system is endowed with a certain degree of heterogeneity by associating agents to different levels of pheromone sensitivity. Variable sensitivity can potentially induce various types of reactions to a changing environment useful in solving dynamic problems by facilitating a better balance between search diversification and exploitation. The proposed SAM metaheuristic proved to be useful for solving the dynamic N P-hard drilling problem (formulated as a dynamic GTSP). Experimental results indicate that SAM obtains better results compared to the standard ACS model for most of the problem instances considered. Future work focuses on testing the proposed SAM metaheuristic for a larger number of dynamic drilling and GTSP instances as well as other complex optimization problems. Another challenging and promising direction refers to developing a learning scheme to adapt the PSL values of agents during the search process. Furthermore, the hybridization between SAM and other techniques for local optimization will be investigated. [4] M. Dorigo and L. Gambardella, Ant colony system: A cooperative learning approach to the traveling salesman problem, IEEE Trans. on Systems, Man,and Cybernetics, 26, 1996, pp [5] T. Stutzle and H. Hoos, Max-min ant system, Future Generation Computer Systems, 16(9), 2000, pp [6] G. Laporte and Y. Nobert, Generalized traveling salesman problem through n sets of nodes: An integer programming approach, Infor., 21(1), 1983, pp [7] comopt/software/tsplib95 References: [1] M. Dorigo and C. Blum, Ant Colony Optimization Theory: A Survey, Theoretical Computer Science, 344, 2-3, pp , [2] E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm intelligence. Oxford Univ. Press, New York, Santa Fe Institute Studies in the Science Complexity, [3] S. Camazine, J. L. Deneubourg, N. R. Franks, J. Sneyd, G. Theraulaz, E. Bonabeau, Self organization in biological systems, Princeton Univ. Press, ISSN: ISBN: