Convergence of Ant Colony Optimization on First-Order Deceptive Systems

Transcription

1 Convergence of Ant Colony Optimization on First-Order Deceptive Systems Yixin Chen Washington University in St. Louis Department of Computer Science & Engineering St. Louis, MO 6330, USA Haiying Sun Yangzhou University Department of Computer Science Yangzhou, 5009, China Abstract Deceptive problems have been considered difficult for ant colony optimization ACO) and it was believed that ACO will fail to converge to global optima of deceptive problems. This paper presents a convergence analysis of ACO on deceptive systems. This paper proves, for the first time, that ACO can achieve reachability convergence but not asymptotic convergence for a class of first order deceptive systems FODS) without assuming a minimum pheromone at each iteration. Experimental results confirm the analysis. Introduction Ant colony optimization ACO) is a popular method for hard discrete optimization problems 6, 5,, 9]. Although there are a huge amount of experimentations and variants of ACO, its theoretical foundation is still in its early development 5]. Recently, there has been an increased effort to deepen the understanding of the convergence behavior of ACO. Guijahr 7, 8] proves the convergence of a particular implementation of ACO called the graph-based ant system GBAS). However, GBAS is quite different from common ACO implementations and its practical performance is unknown. Dorigo et al. shows the convergence of another class of ACO 0, 6], in which there is a lower bound τ min to all pheromone values. Such a method is denoted as ACO τmin. Typically, there are two types of convergence of a stochastic optimization algorithm: Asymptotic convergence. An algorithm has asymptotic convergence if t P s t) =, where P s t) is the probability that the algorithm generates an optimal solution in the t th iteration. Reachability convergence. An algorithm has reachability convergence if t P r t) =, where P r t) is the probability that the algorithm generates an optimal solution at least once in the st to t th iterations. Dorigo et al. 0, 6] show that ACO τmin achieves reachability convergence under certain assumptions. Dorigo et al. 0, 6] also show the asymptotic convergence of ACO τmint), in which the pheromone lower bound τ min t) changes over time, under the assumption that τ min t) = d/lnt +), where d is a constant. The main itation of studying ACO τmin and ACO τmint) is that they do not allow for an exponentially fast decrement of the pheromone trails resulted by using a constant evaporation factor, which is used by most ACO implementations. In this paper, we consider an ACO algorithm that uses the exponentially fast decrement of the pheromone and prove its reachability convergence. Our results are established for a particular class of optimization problem, called the n-bit trap problem. The n-bit trap problem is deemed a difficult problem because it is a first-order deceptive system FODS). FODS are characterized by local optimal fix-points with large basins of attractions. It is known that both genetic algorithms GA) and ACO may fall into the local optimal traps and fail to find the global optimal solution, 3, 4]. We present an ACO algorithm that follows closely to the most common ACO implementations and prove that our algorithm achieves reachability convergence for the n-bit trap problem. We also show that the algorithm does not achieve asymptotic convergence. We prove that, the standard ACO algorithm with an exponential decrease of pheromone, cannot converge to optimal solutions of the n-bit trap problems. This result is meaningful as it provides a first explanation of the lack of asymptotic convergence of ACO on deceptive problems observed by many others, 3, 4].

2 Algorithm : Ant colony optimization ACO) initialize pheromone values τ j i for each i =,,n and j =,, D i ; foreach iteration t =,, do foreach ant k =,,,mdo s ConstructSolutionτ); if s is a feasible solution then if fs) <fs b ) or s b =null then s b s; Ψ t Ψ t {s}; UpdatePheromoneτ,Ψ t,s b ); Ant Colony Optimization and Deceptive Problems In this section, we introduce the framework of the ACO algorithm we will use and review the concept of deceptive problems. ACO are designed for solving constrained optimization CO) problems defined as follows 5]. Definition A constrained optimization problem is defined by a model P =S, Ω,f), where: ) S is a search space defined over a finite set of discrete variables; ) Ω is a set of constraints on the variables; and 3) f : S R is the objective function to be maximized. In the search space S, there are n decision variables X i,i =,,n, where X i can take values from the set D i = {c i,,c Di i }. A variable assignment is written as X i = c j i. A complete assignment to all X i gives a solution instantiation. S is the set of all such complete assignments. We also denote the set of all solution components as R = {c j i i =,,n, j =,, D i }. A solution s Sis called a feasible solution if it satisfies all the constraints in Ω. A feasible solution s is a global optimum if fs ) fs) for all s S. The set of all global optima is denoted by S S. The general ACO algorithm considered in this paper is shown in Algorithm. In the ACO algorithm, we assign a value τ j i, called the pheromone, to each of the solution component c j i. The vector of all pheromone is denoted by τ. The central component of ACO is the ConstructSolutionτ) procedure that each ant uses to construct a solution. This construction procedure assembles solutions as sequences of elements from the finite set of solution components S. Let s p be the partial solution constructed by an ant. Initially s p = {} is empty. The ant at each construction step adds a feasible solution component from the set Rs p ), where Rs p ) R \{s p } is the set of feasible solution components that satisfy the constraints in Ω given the partial solution s p. When selecting the component for variable X i, the probability that the ant chooses c j i is, j =,, D i, P c j i sp )= τ j i ]α ηc j i c )]β ki Rsp ) τ i k]α ηc k. ) i )]β Here, ηc j i ) is a heuristic function for the component c j i. α>0and β>0are two parameters controlling the relative importance of the pheromone and heuristic information. In iteration t, after all ants have constructed solutions, ACO calls UpdatePheromoneτ,Ψ t,s b ) to update the pheromone vector τ. Here, Ψ t is the set of solutions constructed by the ants in iteration t and s b is the incumbent best solution. The pheromone update rule is, i =,,n, j =,, D i, τ j i = ρτ j i + S j i F s), ) s S j i where ρ 0, ) is an evaporation factor and S j i is the set of solutions in Ψ t that have c j i as a component, and F : S R + is a quality function such that, for any s, s S, if fs) >fs ), then F s) F s ). The expected iteration quality is denoted as W F τ t) where t is the iteration number. W F τ t) is defined as: W F τ t) = s S F s)p s τ), 3) where P s τ) is the probability that the solution s is generated by an ant given the pheromone vector τ. Definition Given a constrained optimization problem P, an ACO algorithm is a local optimizer for P if for any initial pheromone values, the expected iteration quality satisfies: W F τ t +) W F τ t), t 0. 4) We can now introduce the notion of deception for local optimizers 3]. Definition 3 Given a constrained optimization problem P, it is called a first-order deceptive system FODS) for an ACO algorithm, if the ACO algorithm is a local optimizer, and there exists an initial setting of pheromone values such that the algorithm does not in expectation converge to a global optimum. 3 Convergence Analysis of ACO on FODS Currently, whether a problem is a FODS is mostly proved by empirical studies, and lacks theoretical analysis. For example, experiments have been run to prove that the n-bit trap and job-shop scheduling problems are deceptive

3 s bit bit bit bit fs) Table. Fitness function value of the 4-bit trap problem. systems by showing that the ACO does not converge to the global optimal solutions 3]. However, there is no proof as to why the ACO cannot achieve asymptotic convergence for such problems. Also, it is yet to be determined if the ACO can achieve reachability convergence on deceptive systems. Further, understanding the time complexity for solving deceptive problems is important because that it sheds insights into the worst-case performance of ACO, as the deceptive problems are harder problems for ACO. In this paper, we study these issues on the n-bit trap problem, a well-known example of FODS that has also been studied for evolutionary algorithms. 3. The n-bit trap problem The n-bit trap problem is to find among the n binary numbers, from 0 to n, the one with the highest fitness. The fitness of a binary number s is defined as: { hs) s 0 fs) = n + s =0, 5) where hs) is the Hamming distance between s and 0. Obviously, the global optimum is s =0. To solve the n-bit trap problem, we design the following ACO algorithm which fits into the general ACO framework in Algorithm. We use m ants. In each iteration, an ant sequentially fixes the j th bit of the binary number in order of j =,,,n. For the j th bit, the ant has two choices c 0 j and c j, corresponding to setting the j th bit to 0 and, respectively. The pheromone of c 0 j and c j are τ j 0 and τ j, respectively. Define the set G k j = {b b b n ) S b j = k}, where k {0, } and b j is the value of the j th bit. Hence, G k j is the set of binary numbers whose j th bit is k. We define the function F G k j ) as the average fitness of all the numbers in G k j : F G k j )= G k j fs). 6) s G k j For example, when n =4,wehaveF G 0 j )=.5 and F G j )=.5for all j =,, 3, 4. We initialize pheromone τj k as τ j k0) = F Gk j ), for all j =,,nand k {0, }. For the j th bit, the probability an ant selects value k, k =0or, is: P c k j,t)= τj kt) τj 0t)+τ j, 7) t) which is derived from ) by setting α =and β =0.In each iteration, the pheromone is updated as: τj k t +)=ρτ j k t)+ fs), 8) s S k j where S k j is the set of solutions generated at the tth iteration that have c k j as the jth bit. 3. Asymptotic convergence of ACO on the n-bit trap problem Lemma For the n-bit trap problem and F G k j ) function defined in 6), we have: F G n + j )= and F G 0 j )=n Proof. We have from the definition in 6) that: F G j ) = F G 0 j ) = n n k= C k n k = n + n n n +)+ = n k= + n +. 9) n C k n k) 0) + n + n ) Lemma Let β>0 be a constant, we have t= β + Proof. We consider four cases. ) In case β =,wehave t= β + β t ) = β )=0 ) t 3 3 T 4 T = =0. 3) T ) In case β >, we have T t= β + β t ) < T t= β + t ). Let β t= t )=r, wehave: r t= β + β T t )= ). 4) t t=

4 T Again, because t= t ) = 0, we have r T t= β + β t ) = 0 which means T t= β + β t )=0. 3) In case β<, and β = q where q is an integer larger than. Let T = p q and c = q t= t ), then we have: T t ) = c t ) t= Therefore, t=q p q = c p jq + i ) >c jq )q. j= i=0 j= T t= t ) >c p p j= jq ] )q q p = c p j= jq ) =0 5) Since we know T t= t ) = 0 and and ] q c p p j= jq ) 0, combined with 5) ] q we get c p p j= jq ) = 0, which implies p p j= jq )=0, which is equivalent to: t= qt ) = t= β t )=0. 4) In case β <, and β = p r where p and r are both integers larger than. We can always find β = q where q is an integer larger than such that β <β. Since T t= β t ) > T t= β t ) and T β t= t )=0from case 3), we have T t= β t )=0. We consider the ACO algorithm described in Algorithm using 7) and 8) as the selection and update functions, respectively. We name this algorithm ACO n bit. For the ACO n bit algorithm, define: α j t) = τ j 0t) τj j =,,,n,t=0,,,. 6) t), Namely, α j t) is the ratio of the pheromone on value 0 to the pheromone on value for bit j in iteration t. Theorem The ACO n bit algorithm cannot achieve asymptotic convergence on the n-bit trap problem when n>3. Proof. For the n-bit trap problem, since the global optimal solution is s = {0, 0,, 0), we show that j =,,,n, τ 0 ] Eα j t) jt)] = E t t τj t) =0. 7) Suppose we have m ants. We first prove by induction that Eα j t)] <, j =,,,n,t=0,,,. 8) When t =0, τj 00) is initialized to F G0 j ) and τ j 0) is initialized to F G j ). From Lemma, while n > 3, F G j ) > FG0 j ). For a fixed n, we set F G0 j ) = α 0 F G j ), where α 0 is a constant and 0 <α 0 <. Hence, we have our base case: Eα j 0)] = τ j 00) τj 0) = F G0 j ) F G j ) = α 0 <. To prove 8) by induction, we assume that Eα j t)] <. According to 8), we have that in the t +) th iteration, the expected value of pheromone satisfies ] Eτj 0 t +)] ρeτj 0 mαj t) t)] + E +α j t) F G0 j ), ] Eτj t +)] ρeτ j t)] + E m +α j t) F G j ). Given 6), we have Eα j t +)]= Eτ 0 j t+)] Eτ j t+)] ρeτ 0 j t)]+ meαj t)] +Eα j t)] F G0 j ) ρeτj t)]+ m +Eα j t)] F G j ) = ρ+eαj t)])eαjt)]eτ j t)]+eαjt)]mf G0 j ) ρ+eα jt)])eτ j t)]+mf G j ) Hence, we have Eα j t +)] α 0 )mf G j = Eα j t)] ) ) ρ + Eα j t)])eτj t)] + mf G j ) < Eα j t)]. Therefore, we get, for any j =,, n, > α 0 = Eα j 0)] >Eα j )] > > Eα j t)] >Eα j t +)]> > 0. 9) Thus, we have proved 8) by induction. Further, since Eτj t)] = ρeτj t )] + +Eα j t)] mf G j ) < Eτj t )] + mf G j), 0) recursively applying 0) leads to Eτj t)] <Eτ j 0)] + tmf G j ) < t +)mf G j ). ) Substituting ) into 9) yields ) α 0 ) Eα j t +)]<Eα j t)] ρ + Eα j t)])t +)+

5 From 9), we know +Eα j t)] <. Therefore, ) Eα j t +)] <Eα j t)] α0) ρt+)+ ) <Eα j t)] α0) ρ+3)t Let β = α0 ρ+3,wehave Eα j t +)]<Eα j t)] β ) < t t i= β ) α 0 i From Lemma, we get t Eα j t)] = 0. Therefore, the ACO n bit algorithm will not converge to the optimal solution s = {0, 0,, 0}. Theorem gives an explanation why the ACO algorithms have difficulties in solving deceptive problems such as the n-bit trap problem. 3.3 Reachability convergence of ACO on the n-bit trap problem Although ACO n bit cannot achieve asymptotic convergence, we will show that it can achieve reachability convergence. We need some preinary results first. Lemma 3 The pheromone in the t th iteration of ACO n bit satisfies τj 0 ρt+ t) ρ ϕf G0 j ), where ϕ is a constant, 0 <ϕ<, for j =,,,nand t =,,. Proof. From the pheromone update rule in 8), we have τj k t) =ρτ j k t ) + fs), ) s S k j } Let θj {fs) s 0 =min =b b b n ),s S,b j =0 and θ 0 j = ϕf G0 j ). Obviously, 0 <ϕ<. Since τ 0 j t) ρτ 0 j t ) + θ0 j,wehaveτ 0 j t) ρτ 0 j t ) + ϕf G0 j ). Therefore, we get τj 0 t) ρ t τj 0 0) + ρ t + ρ t + +)ϕf G 0 j ) = ρ t + ρ t + ρ t + +)ϕf G 0 j ) = ρt+ ρ ϕf G0 j ). Lemma 4 For any positive integer n and a real number ρ 0, ), there exists a positive integer t 0 such that ρt+ ρ > t n for any t>t 0. Proof. We see that fact that t t n ρ t+ ρ ρ = t ρ t+ ) n t =0. Thus, there exists an integer t 0 > 0 such that ρt+ ρ >t n for t>t 0. From Lemma 3 and Lemma 4, we see that when t is large enough, τj 0t) t n ϕf G 0 j ). We set τ min 0 j, t) = t n ϕf G 0 j ) and τ min 0 t) =min j n τmin 0 j, t). If we set θ min =min j n ϕf G 0 j ), then from Lemma 3 and Lemma 4, τmin 0 t) t n θmin, t >t 0. 3) Lemma 5 The pheromone in the t th iteration of ACO n bit satisfies τj kt) ρ F Gk j ), for k =0,, j =,,,n and t =,,. Proof. From the pheromone update rule in 8), we have τj k t) =ρτj k t ) + fs), 4) s S k j } Let λ k j {fs) s =max =b b b n ),s S,b j = k and λ k j = φf Gk j ) where φ>is a constant for a given n. Since τj kt) ρτ j kt ) + mλk j,wehaveτ j kt) ρτj kt ) + φmf Gk j ). Therefore, we get τj k t) ρt τj k 0) + ρt + ρ t + +)φmf G k j ) ρ φmf Gk j ). We denote τmaxj) k = ρ φmf Gk j ) and τ max = max j n,k=0, τmax k j). Theorem The ACO n bit algorithm achieves reachability convergence on the n-bit trap problem. Proof. Although there are multiple ants, we only need to prove that one ant can achieve reachability convergence in order to establish the result. From Lemma 4 and Lemma 5, we know that for t>t 0, where t 0 is defined in Lemma 4, P c 0 j,t)= τj 0t) τj 0t)+τ j t) τ min 0 t). 5) Hence, let P t) be the probability that the ant can generate the optimal solution s =0, 0,, 0) in iteration t, we have ] τ 0 n P t) min t) 6) Let P succ T ) and P fail T ), respectively, be the probability that the ant does and does not find s in the first T iterations. From 3) and Lemma 5, we know ) τ 0 n ] P fail T ) = P t)) min t) t= t=t 0 θmin t=t 0 ) n ) n ] t n.

6 Figure. Average solution pheromone. Figure. Average incumbent solution. Denote θmin ) n = γ, then P fail T ) T t=t 0 γ t ]. From Lemma, we have that P fail T ) = 0 and thus P succ T ) = P fail T )=. 4 Experimental Results We have implemented an ACO on the n-bit trap problem. We set n =, evaporation rate ρ =0.95, and use 0 ants. First, to test the asymptotic convergence, we make 00 runs and record Q, the solution pheromone at each iteration t. Figure shows the average pheromone at each iteration of the 00 runs. We see that Q decreases as t increases, indicating that ACO does not have asymptotic convergence on the n-bit trap problem. Next, to show reachability convergence, we plot in Figure the average fitness of the incumbent solution f versus the iteration number t. We see that the average f approaches the optimal value as t increases, showing the reachability convergence of ACO. 5 Conclusions In this paper, we have theoretically analyzed ACO algorithms on deceptive problems, a class of problems that are considered difficult for ACO. We have presented a first attempt towards a convergence analysis of ACO on deceptive systems. Using the n-trap problem as an example of firstorder deceptive problems, we have proved that that ACO can achieve reachability convergence but not asymptotic convergence for some first order deceptive systems without assuming a minimum pheromone at each iteration. We have also presented experimental results that confirm the analysis. References ] C. Blum. Theoretical and practical aspects of ant colony optimization. Dissertations in Artificial Intelligence, 463, 004. ] C. Blum. Ant colony optimization: Introduction and recent trends. Physics of Life Reviews, 4): , December ] C. Blum and M. Dorigo. Deception in ant colony optimization. In Proceedings of ANTS004, Lecture Notes in Computer Science, pages 9 30, ] C. Blum and M. Dorigo. Search bias in ant colony optimization: on the role of competition-balanced systems. IEEE Transactions on Evolutionary Computation, 9:59 74, ] M. Dorigo and C. Blum. Ant colony optimization theory: A survey. Theoretical Computer Science, 344:43 78, ] M. Dorigo and T. Stützle. Ant Colony Optimization. MIT Press, ] W. Gutjahr. A Graph-based Ant System and its convergence. Applied Soft Computing, 6: , ] W. Gutjahr. ACO algorithms with guaranteed convergence to the optimal solution. Info. Processing Lett., 8:45 53, 00. 9] S. Shtovba. Ant algorithms: Theory and applications. Programming and Computer Software, 3:67 78, ] T. Stützle and M. Dorigo. A Short Convergence Proof for a Class of Ant Colony Optimization Algorithms. IEEE Transactions on Evolutionary Computation, 6: , 00.