Ant Colony Optimization: an introduction Daniel Chivilikhin 03.04.2013
Outline 1. Biological inspiration of ACO 2. Solving NP-hard combinatorial problems 3. The ACO metaheuristic 4. ACO for the Traveling Salesman Problem 2
Outline 1. Biological inspiration of ACO 2. Solving NP-hard combinatorial problems 3. The ACO metaheuristic 4. ACO for the Traveling Salesman Problem 3
Biological inspiration: from real to artificial ants 4
Ant colonies Distributed systems of social insects Consist of simple individuals Colony intelligence >> Individual intelligence 5
Ant Cooperation Stigmergy indirect communication between individuals (ants) Driven by environment modifications 6
Denebourg s double bridge experiments Studied Argentine ants I. humilis Double bridge from ants to food source 7
Double bridge experiments: equal lengths (1) 8
Double bridge experiments: equal lengths (2) Run for a number of trials Ants choose each branch ~ same number of trials 9
Double bridge experiments: different lengths (2) 10
Double bridge experiments: different lengths (2) The majority of ants follow the short path 11
Outline 1. Biological inspiration of ACO 2. Solving NP-hard combinatorial problems 3. The ACO metaheuristic 4. ACO for the Traveling Salesman Problem 12
Solving NP-hard combinatorial problems 13
Combinatorial optimization Find values of discrete variables Optimizing a given objective function 14
Combinatorial optimization Π = (S, f, Ω) problem instance S set of candidate solutions f objective function Ω set of constraints ~ S S set of feasible solutions (with respect to Ω) Find globally optimal feasible solution s* 15
NP-hard combinatorial problems Cannot be exactly solved in polynomial time Approximate methods generate nearoptimal solutions in reasonable time No formal theoretical guarantees Approximate methods = heuristics 16
Approximate methods Constructive algorithms Local search 17
Constructive algorithms Add components to solution incrementally Example greedy heuristics: add solution component with best heuristic estimate 18
Local search Explore neighborhoods of complete solutions Improve current solution by local changes first improvement best improvement 19
What is a metaheuristic? A set of algorithmic concepts Can be used to define heuristic methods Applicable to a wide set of problems 20
Examples of metaheuristics Simulated annealing Tabu search Iterated local search Evolutionary computation Ant colony optimization Particle swarm optimization etc. 21
Outline 1. Biological inspiration of ACO 2. Solving NP-hard combinatorial problems 3. The ACO metaheuristic 4. ACO for the Traveling Salesman Problem 22
The ACO metaheuristic 23
ACO metaheuristic A colony of artificial ants cooperate in finding good solutions Each ant simple agent Ants communicate indirectly using stigmergy 24
Combinatorial optimization problem mapping (1) Combinatorial problem (S, f, Ω(t)) Ω(t) time-dependent constraints Example dynamic problems Goal find globally optimal feasible solution s* Minimization problem Mapped on another problem 25
Combinatorial optimization problem mapping (2) C = {c 1, c 2,, c Nc } finite set of components States of the problem: X = {x = <c i, c j,, c h, >, x < n < + } Set of candidate solutions: S X 26
Combinatorial optimization problem mapping (3) Set of feasible states: ~ X X ~ We can complete x X into a solution satisfying Ω(t) Non-empty set of optimal solutions: S * ~ X I S 27
Combinatorial optimization problem mapping (4) X states S candidate solutions feasible states X ~ S* optimal solutions S X S* X ~ 28
Combinatorial optimization problem mapping (5) Cost g(s, t) for each In most cases g(s, t) f(s, t) G C = (C, L) completely connected graph C set of components L edges fully connecting the components (connections) G C s S construction graph s ~ S 29
Combinatorial optimization problem mapping (last ) Artificial ants build solutions by performing randomized walks on G C (C, L) 30
Construction graph Each component c i or connection l ij have associated: heuristic information pheromone trail 31
Heuristic information A priori information about the problem Does not depend on the ants On components c i η i On connections l ij η ij Meaning: cost of adding a component to the current solution 32
Pheromone trail Long-term memory about the entire search process On components c i τ i On connections l ij τ ij Updated by the ants 33
Artificial ant (1) Stochastic constructive procedure Builds solutions by moving on G C Has finite memory for: Implementing constraints Ω(t) Evaluating solutions Memorizing its path 34
Artificial ant (2) Has a start state x Has termination conditions e k From state x r moves to a node from the neighborhood N k (x r ) Stops if some e k are satisfied 35
Artificial ant (3) Selects a move with a probabilistic rule depending on: Pheromone trails and heuristic information of neighbor components and connections Memory Constraints Ω 36
Artificial ant (4) Can update pheromone on visited components (nodes) and connections (edges) Ants act: Concurrently Independently 37
The ACO metaheuristic While not dostop(): ConstructAntSolutions() UpdatePheromones() DaemonActions() 38
ConstructAntSolutions A colony of ants build a set of solutions Solutions are evaluated using the objective function 39
UpdatePheromones Two opposite mechanisms: Pheromone deposit Pheromone evaporation 40
UpdatePheromones: pheromone deposit Ants increase pheromone values on visited components and/or connections Increases probability to select visited components later 41
UpdatePheromones: pheromone evaporation Decrease pheromone trails on all components/connections by a same value Forgetting avoid rapid convergence to suboptimal solutions 42
DaemonActions Optional centralized actions, e.g.: Local optimization Ant elitism (details later) 43
ACO applications Traveling salesman Quadratic assignment Graph coloring Multiple knapsack Set covering Maximum clique Bin packing 44
Outline 1. Biological inspiration of ACO 2. Solving NP-hard combinatorial problems 3. The ACO metaheuristic 4. ACO for the Traveling Salesman Problem 45
ACO for the Traveling Salesman Problem 46
Traveling salesman problem N set of nodes (cities), N = n A set of arcs, fully connecting N Weighted graph G = (N, A) Each arc has a weight d ij distance Problem: Find minimum length Hamiltonian circuit 47
TSP: construction graph Identical to the problem graph C = N L = A states = set of all possible partial tours 48
TSP: constraints All cities have to be visited Each city only once Enforcing allow ants only to go to new nodes 49
TSP: pheromone trails Desirability of visiting city j directly after i 50
TSP: heuristic information η ij = 1 / d ij Used in most ACO for TSP 51
TSP: solution construction Select random start city Add unvisited cities iteratively Until a complete tour is built 52
ACO algorithms for TSP Ant System Elitist Ant System Rank-based Ant System Ant Colony System MAX-MIN Ant System 53
Ant System: Pheromone initialization Pheromone initialization where: m number of ants Τ ij = m / C nn, C nn path length of nearest-neighbor algorithm 54
Ant System: Tour construction Ant k is located in city i N k i is the neighborhood of city i k Probability to go to city : j N i p k ij = l τ k N i α ij τ α il η β ij η β il 55
Tour construction: comprehension k pij α = 0 greedy algorithm β = 0 only pheromone is at work l η η quickly leads to stagnation = τ k N i α ij τ α il β ij β il 56
Ant System: update pheromone trails evaporation Evaporation for all connections (i, j) L: τ ij (1 ρ) τ ij, ρ [0, 1] evaporation rate Prevents convergence to suboptimal solutions 57
Ant System: update pheromone trails deposit T k path of ant k C k length of path T k Ants deposit pheromone on visited arcs: τ ij τ ij + m k = 1 τ k ij, ( i, j) L k τ ij = 1/ 0, C k, ( i, j) otherwise T k 58
Elitist Ant System Best-so-far ant deposits pheromone on each iteration: τ ij τ ij + m k =1 τ k ij + e τ best ij τ best ij = 1/ 0, C best, ( i, j) otherwise T best 59
Rank-based Ant System Rank all ants Each ant deposits amounts of pheromone proportional to its rank 60
MAX-MIN Ant System 1. Only iteration-best or best-so-far ant deposits pheromone 2. Pheromone trails are limited to the interval [τ min, τ max ] 61
Ant Colony System Differs from Ant System in three points: More aggressive tour construction rule Only best ant evaporates and deposits pheromone Local pheromone update 62
Ant Colony System 1.Tour Construction j = { β } τ η arg max, if q q k l N il il 0 i like Ant System, otherwise 2. Local pheromone update: τ ij (1 ξ)τ ij + ξτ 0, 63
Comparing Ant System variants 64
State of the art in TSP CONCORDE http://www.tsp.gatech.edu/concorde.html Solved an instance of 85900 cities Computation took 286-2719 CPU days! 65
Current ACO research activity New applications Theoretical proofs 66
Further reading M. Dorigo, T. Stützle. Ant Colony Optimization. MIT Press, 2004. http://iridia.ulb.ac.be/~mdorigo/aco/ 67
Next time Some proofs of ACO convergence 68
Thank you! Any questions? This presentation is available at: http://rain.ifmo.ru/~chivdan/presentations/ Daniel Chivilikhin [mailto: chivdan@gmail.com] 69
Used resources http://teamaltman.com/wp-content/uploads/2011/06/uncertainty-ant-apple- 1024x1024.jpg http://myrealestatecoach.files.wordpress.com/2012/04/ant.jpg http://ars.els-cdn.com/content/image/1-s2.0-s1568494613000264-gr3.jpg http://www.theorie.physik.uni-goettingen.de/forschung/ha/talks/stuetzle.pdf http://www.buyingandsellingwebsites.com/wp-content/uploads/2011/12/ant-150.jpg http://moodle2.gilbertschools.net/moodle/file.php/1040/event- 100_Days/Ant_Hormiga.gif http://4.bp.blogspot.com/_spae2p8y-kg/tk3hq8bumti/aaaaaaaaaha/p75k- GcT_oo/s1600/ant+vision.GIF http://1.bp.blogspot.com/- txkjq4nsqoy/uqz3vabiy_i/aaaaaaaaaja/ik8jtlhelqk/s1600/16590492-illustrationof-an-ant-on-a-white-background.jpg 70