THE PROBLEM OF locating all the optima within a fitness

Transcription

1 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST Where Are the Niches? Dynamic Fitness Sharing Antonio Della Cioppa, Member, IEEE, Claudio De Stefano, and Angelo Marcelli, Member, IEEE Abstract The problem of locating all the optima within a multimodal fitness landscape has been widely addressed in evolutionary computation, and many solutions, based on a large variety of different techniques, have been proposed in the literature. Among them, fitness sharing (FS) is probably the best known and the most widely used. The main criticisms to FS concern both the lack of an explicit mechanism for identifying or providing any information about the location of the peaks in the fitness landscape, and the definition of species implicitly assumed by FS. We present a mechanism of FS, i.e., dynamic fitness sharing, which has been devised in order to overcome these limitations. The proposed method allows an explicit, dynamic identification of the species discovered at each generation, their localization on the fitness landscape, the application of the sharing mechanism to each species separately, and a species elitist strategy. The proposed method has been tested on a set of standard functions largely adopted in the literature to assess the performance of evolutionary algorithms on multimodal functions. Experimental results confirm that our method performs significantly better than FS and other methods proposed in the literature without requiring any further assumption on the fitness landscape than those assumed by the FS itself. Index Terms Evolutionary algorithms (EAs), fitness sharing (FS), niching, speciation. I. INTRODUCTION THE PROBLEM OF locating all the optima within a fitness landscape has been widely studied and for its solution, many algorithms based on a large variety of different techniques have been proposed in the literature [1] [7]. In this context, evolutionary algorithms (EAs) have proved their ability to explore very large problem spaces and to efficiently approximate the desired solution. Nonetheless, EAs have also shown an intrinsic drawback when dealing with multimodal functions, in that they have the property of converging to a population containing just one solution. Such a behavior is the result of the combined effects of both the genetic drift [2], [8], i.e., the tendency of the selection mechanism to converge over time toward a uniform distribution of mutants of the fittest individual, and the evaluation mechanism, which computes the fitness of each individual in the population independently of the fitness of the others. Moving from these considerations, a relevant research activity has been devoted for counterbalancing those effects in order to make EAs able to deal with multimodal fitness landscapes [3]. The basic idea most of the methods are based upon, drawn from an analogy with natural ecosystems, is that of preserving genetic diversity Manuscript received July 18, 2005; revised March 24, A. Della Cioppa and A. Marcelli are with the Department of Electrical and Information Engineering, University of Salerno, I Fisciano (SA), Italy ( adellacioppa@unisa.it). Claudio De Stefano is with Dipartimento di Automazione, Elettromagnetismo, Ingegneria dell Informazione e Matematica Industriale, Università di Cassino, Cassino (FR), Italy. Digital Object Identifier /TEVC by encouraging the formation of species or niches, each representing one of the possible solutions [1] [7], [9] [11]. In Nature, an ecosystem is typically composed by different physical spaces (niches) that exhibit different features and allow both the formation and the maintenance of different types of life (species). It is assumed that a species is formed by individuals with similar biological features capable of interbreeding among themselves, but unable to breed with individuals of other species [12]. As a species adapts to the specific features of the niche in which it lives, natural selection favors the emergence of specialized properties within the species that allow its individuals to effectively exploit the niche resources. The fitness of an individual, then, measures its ability to exploit environmental resources to generate offspring. As a consequence, a natural ecosystem can be characterized by the following properties: it is capable of supporting the presence of different species, each occupying a different physical niche; the physical resources of a niche are finite and are shared only among the individuals populating that niche; the density of individuals populating a given niche (niche carrying capacity) depends on both the amount of resources for that niche, and the ability of the species to exploit them. By analogy, in artificial systems, a niche corresponds to a peak of the fitness landscape, while a species to a subpopulation of individuals that, in terms of a given metric, exhibit similar features. Following this analogy, niching or speciation methods have been introduced in evolutionary artificial systems to promote the formation and the maintenance of stable subpopulations. Among niching methods, fitness sharing (FS) and implicit fitness sharing are the best known and the most widely used [3] [5], [7], [10], [11], [13] [17]. In the former, the fitness represents the resource for which the individuals belonging to the same niche compete [3], while in the latter [4], [10], the sharing effects are achieved by means of a sample-and-match procedure which resembles the bidding mechanism commonly used in classifier systems [18]. In FS, subpopulations are formed and maintained by reducing the probability that the number of individuals populating a peak becomes larger and larger, so as to cause the disappearance of the individuals populating other peaks. This is achieved by reducing the fitness of similar individuals as their number grows, with the implicit assumption that similar individuals belong to the same species and populate the same niche. To this purpose, the definition of both a similarity metric on the search space and a threshold (niche radius), representing the maximal distance among individuals to be considered similar and therefore belonging to the same niche, is required. It has been proved that, when the number of individuals within the population is large enough and the niche radius is properly set, FS provides as many species in the population as the number of peaks in the fitness landscape, thus populating all the niches [19], [20] X/$ IEEE

2 454 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 In the implicit fitness sharing [4] each individual s fitness is functionally dependent on the rest of the population The functional dependence is introduced through the use of a simplified bidding mechanism similar to that of classifier systems the use of the simple bidding procedure combined with a traditional genetic algorithm is sufficient for the population to discover and maintain independent subpopulations. More specifically, sharing is accomplished by inducing competition for limited and explicit resources. For each environmental resource, a set of individuals is randomly selected from the population and each individual is matched against the resource. The individual with the highest score is then rewarded. Such a procedure is repeated a given number of times, through which the score of individuals is updated. At each generation, the discovered niches are obtained by selecting the minimal set of individuals needed to match all the resources. Therefore, niching is implicit in that the number of peaks is determined dynamically and there is no specific limitation on the distance between peaks. As a consequence, the method avoids the difficulty of appropriately choosing the niche radius, and it is able to deal with problems in which the peaks are not equally spaced [4], [10], [11]. So, one of the most important limitations of FS seems to be removed. Actually, the implicit fitness sharing introduces other parameters to be set, e.g., the size of the sample of individuals that compete, the number of competition cycles and the definition of a matching procedure. Moreover, the limitation on the population size mentioned above still takes place. Finally, the method can be applied, at best, on problems in which explicit and finite resources are available [4], [10], [11], e.g., a training set in a pattern recognition task. Our main criticism to the FS method, in addition to those reported in the literature regarding the difficulty of setting effective values for the niche radius and the population size [21], as well as the lack of an explicit mechanism for identifying or providing any information about the location of the peaks in the fitness landscape [22], concerns the definition of species implicitly assumed by the FS method. As discussed above, in order to ensure that subpopulations are steadily formed and maintained, only the individuals belonging to the same niche should share the resources of the niche. This assumption is not generally true for the FS method [23], because each individual in the population shares its fitness with all the individuals located at a distance smaller than the niche radius, no matter for the actual peak, i.e., for the niche, to which they belong. As a consequence, individuals belonging to different peaks may share their fitness, while they should not. Only when the distance between the borders of neighboring peaks is at least equal to the niche radius, is it possible to achieve the perfect separation among niches. Overall, the nonperfect discrimination between peaks results in a higher probability of niche loss, that can be counterbalanced by an increase of the population size required to solve a multimodal problem. This, in turn, results in a computational time that makes FS impractical as the number of peaks becomes large. In order to overcome these drawbacks, we have proposed a method for estimating the optimal values for the population size and the niche radius without any a priori information on the fitness landscape based on an explicit identification of the peaks in the fitness landscape [24]. Here, we propose a sharing method, called dynamic fitness sharing (DFS), whose foundations were presented in [25], aimed at discovering and populating all the niches in the environment according to their carrying capacity. The proposed method is based upon a dynamic, explicit identification of the species discovered at each generation and their localization in the fitness landscape. The application of the FS mechanism, thus, is restricted to individuals belonging to the same species. Eventually, an elitist strategy is applied on the species, by copying the species masters, i.e., the fittest individuals of the species discovered at each generation in the new population. DFS is independent of the EA actually used to search the solution space, of the values of its internal parameters and, finally, of the particular encoding. The idea of improving the ability of FS to promote both the preservation of good individuals through the generations and the maintenance of genetic diversity by adopting an explicit identification of niches has been investigated during the last decade. In fact, since the publication of the first paper on niching by Goldberg and Richardson in 1987, a number of methods have been proposed in literature [22], [26] [29]. The first attempt to dynamically identify the niche in the population during the evolution was made by Miller and Shaw in [22]. Their dynamic niche sharing (DNS) adopts a dynamic species 1 identification and partitions the population by assuming that the number of niches of the fitness landscape is a priori known. The individuals not belonging to any of the previously identified species are grouped into a unique nonspecies class. Moreover, the fitness of the individuals is modified according to two different sharing mechanisms. The shared fitness of an individual belonging to a species is computed by dividing its raw fitness by the occupation number of the niche, i.e., the number of individuals populating that niche. However, the standard FS formula is used for individuals belonging to the nonspecies class. The authors motivate this choice in terms of computational cost, in that the occupation number is computed only once for all the individuals of a species, while the niche count has to be computed for each individual in the population. The role played by each feature (species identification and sharing mechanisms) on the performance is unclear. It is also unclear how DNS performs on complex, deceptive fitness landscape. Moreover, the algorithm does not use an elitist strategy on the species, and therefore it does not tackle the problem of reducing the population size needed to solve the problem. The main drawback of this proposal, however, is that in many applications the number of niches is not known a priori, and this makes it very hard to apply the DNS successfully. The method recently proposed by Stanley et al. [26], [27] is interesting in that it uses FS to promote species discovery and maintenance by combining explicit species identification with elitism, but in a quite different context. In particular, the NEAT algorithm deals with problems whose solution requires searching multiple spaces, because different species in NEAT have a different number of parameters (i.e., dimensions). In such a context, it would not be realistic to search for all the peaks, 1 Note that the authors use the term niche for referring to both the subpopulations and the peaks of the fitness landscape. For the sake of coherence with the terminology used in this paper, we change the term niche into species when appropriate, while describing their work.

3 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 455 because as new dimensions are added the algorithm moves into different spaces where new peaks are located, and to exploit those new peaks it may have to give up some of the older ones. In other words, NEAT assumes that species will continually die out and be replaced and uses FS in order to protect innovative species in their own niches. To achieve this goal, NEAT adopts the following strategies: each species has a representative individual that is randomly chosen among the individuals belonging the species; if an individual in the population does not belong to any species, then a new species is created with the individual as its representative; when a species reproduces, its representative is reproduced if the species is above a minimum size; before reproduction, NEAT eliminates the lowest performing individuals from the population. Interestingly, the above strategies are similar to those of DNS and DFS, but NEAT uses FS for different purposes, namely, protecting innovative solutions, while searching multiple spaces. Therefore, it is not directly comparable with any other method using FS while searching for all the peaks in a given space. Species conserving genetic algorithm (SCGA) by Balazs et al. [28] and opt-ainet by de Castro et al. [29], on the other hand, are particularly interesting in that they do not consider any sharing mechanism. Once a new species is discovered, its fittest individual is retained in the next generations until a fitter individual for that species is generated. Both the algorithms upgrade the set of best individuals with the new ones discovered in the current population, thus realizing a sort of elitism with incremental memory. This idea represents the major novelty of these approaches, allowing to preserve genetic diversity and to maintain species independently on the fitness of their fittest individuals. Unfortunately, such a behavior implies that each species populating a region of the fitness landscape survives during the entire evolution, whether or not it corresponds to an actual niche. In addition, the number of individuals forming a species is not related to the niche carrying capacity. In particular: SCGA is prone to stagnation, as it follows from the authors comments to the experimental results reported in Section IV of their paper. As they noticed, the presence in the final population of species not populating the niches of the fitness landscape, requires a method for selecting the desired solution among the species. This last aim can be achieved only when some a priori knowledge about either the location of the actual niches or their relative fitness is available. As regards opt-ainet, the above drawback is tackled by introducing suppression and genetic diversity mechanisms that allow a dynamic population size. This feature makes opt-ainet very different from all the other niching methods, but it is difficult to estimate the effectiveness of those mechanisms on complex problems with a very large number of niches. Moreover, in our opinion, it is hard to determine the influence of such mechanisms on opt-ainet performance. DFS overcomes these drawbacks, because of the following properties: all the species present in the population at each generation are automatically and explicitly identified; its elitist strategy avoids both the premature convergence and the stagnation usually associated with elitism; the adoption of a more biologically plausible definition of species in combination with the suggested elitist strategy ensures the convergence of the algorithm towards a population containing as many species as the number of niches to be populated; the sharing mechanism ensures that each niche will be populated according to its carrying capacity; the population size required to solve the problem is much smaller than that required by FS. The proposed method has been tested on a set of standard multimodal functions largely adopted in the literature, and the experimental results were compared with those provided by FS. The performance of the DFS has also been compared with that of DNS and SCGA, in that those methods introduce a smaller number of parameters, and the role of such parameters on the system performance is clear. The remainder of this paper is organized as follows: Section II describes the background of the FS method, while Section III presents DFS. The experimental results are illustrated and discussed in Section IV, and our conclusions are eventually left to Section V. II. FITNESS SHARING (FS) In artificial evolutionary systems, a population of individuals evolves according to probabilistic transition operators, i.e., selection, mutation, and eventually crossover. At each generation, the individuals are selected according to their fitness and manipulated by the genetic operators, thus creating offspring. Then, the parents are usually replaced by their offspring in order to obtain a new population. In a simple EA, the number of individuals whose fitness value is above the average increases during the evolution, and an increase in their number shifts the average fitness value towards even greater values until a suboptimum is reached, which is equal to the highest fitness among all existing individuals. Then, the system will tend to assume a state of selective equilibrium, which is unstable. It will be upset when, as a result of genetic changes, a new individual appears with a higher fitness value. In such a way, a new state of equilibrium arises, and so on. At the end of such evolutionary process, the population consists of an optimal individual, i.e., the master, with its mutant distribution. The selection mechanism is responsible for the adaptation, allowing the EA to concentrate its efforts in the most promising area of the search space and driving the population toward a single distribution constituted by the fittest individual and its close mutants. So, at the end of such evolutionary process, the population consists of a single fittest individual, representing the best solution found by the algorithm, and its cloud of mutants. There are many cases, however, when the desired solution is not necessarily the best one, but rather a collection of best, as is the case, for instance, in multimodal function optimization. In order to deal with this class of problem, it is necessary to prevent the species with highest fitness value replacing the competing

4 456 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 species by inducing some kind of restorative pressure to counterbalance the convergence pressure of selection. As with natural ecosystems, niching has been suggested as a viable mean to simultaneously evolve subpopulations exploiting different ecological niches by some kind of sharing. In the FS [3], the peaks in the fitness landscape are considered as environmental niches with a given amount of environmental resources proportional to the peak s height and all the individuals populating a given peak have to share the resource of that peak. When a peak is overcrowded, the resources of the niche are overused and the selective pressure is increased by derating the fitness of the individuals populating that niche. On the other hand, less crowded peaks have the fitness of their individuals less derated, thus increasing selective pressure. The modification of the raw fitness of an individual is accomplished according to the presence of similar individuals within the population, with the implicit assumption that similar individuals populate the same niche. The concept of similarity between two individuals is implemented by defining a metric on either the genotypic or the phenotypic space and by setting a threshold value which represents the maximal distance between individuals to be considered similar, i.e., belonging to the same species, and hence populating the same niche. The shared fitness of an individual at generation is given by where is the raw fitness of the individual and is the niche count which depends on the number and the relative positions of the individuals within the population with whom the raw fitness is shared. The niche count is given by where is the sharing function which measures the similarity between two individuals. The most commonly adopted form of is the following: where is the defined distance function, and is the radius of the niches. In general, is computed by using a metric in the phenotypic space [15], while the choice of the values for the parameters and depends on the fitness landscape. 2 Other definitions of the sharing function can be found in [6]. Let us point out that the basic assumption underlying the method is that there exists a value of which allows is to associate one and only one niche to each peak of the fitness landscape (perfect discrimination) [3]. Moreover, as it follows from formula (3), such a value is the same for all the peaks. Under this assumption, FS allows the formation and the maintenance of quasi-stable subpopulations, in that it reduces the effects of the 2 In the large majority of studies on the FS, the value of has been set to 1, yielding to a triangular form for the sharing function. Therefore, in the following, we shall consider only the case =1. (1) (2) (3) genetic drift because the shared fitness of similar individuals is reduced as far as their number increases. Thus, FS allows other individuals, with lower raw fitness or belonging to less populated niches, to compete for selection and reproduction. A state of dynamic equilibrium is eventually reached when the niches are populated according to their relative fitness. Although it has been proved that FS provides as many subpopulations as the number of peaks in a multimodal fitness landscape [20], it suffers from two main drawbacks. There is no explicit identification of the niches [6]. According to the (2), FS considers each individual in the population as the master of a species. Therefore, a given individual may belong to many species, namely, the species of whom it is the master and the species with at least one individual within its niche radius. As a consequence, the perfect discrimination hypothesis is not satisfied, and therefore it is not guaranteed that FS would provide as many subpopulations as the number of peaks. To accurately decide when two individuals belong to the same niche and should therefore have their fitness shared, we need to know where the niches are. The population size required to populate a given number of niches increases very rapidly with the number of niches, leading to an unbearable computational cost in many applications. III. DYNAMIC FITNESS SHARING (DFS) Starting from the considerations discussed in the previous sections, we propose a mechanism of FS, which adopts a more biological plausible definition of species based on the following criteria: a species consists of a subpopulation of at least two individuals; each individual in the population belongs to one and only one species; a species is represented by its species master, i.e., the individual of the species with the highest raw fitness value. DFS is based upon explicitly finding the species at each generation, using FS to ensure that each niche will be populated proportionally to the fitness of its peak, and preserving species with few individuals from extinction by elitism. For this purpose, DFS embodies an explicit, dynamic identification and localization on the fitness landscape of the species discovered at each generation, the application of the FS mechanism to each species separately, and a species elitist strategy. Dynamic species identification is accomplished by assuming only that effective values for the niche radius and for the population size have been selected in order to achieve the perfect discrimination among the peaks of the fitness landscape and to maintain the actual number of niches. It should be remarked here that the above assumptions are equivalent to the assumption that we know a priori the number of perfectly distinguishable niches to be populated and maintained. Obviously, DFS suffers the same drawbacks as FS when wrong values for either the niche radius or the population size are selected. Deb and Goldberg proposed a criterion for estimating the niche radius given the heights of the peaks and their distances [15], while Mahfoud suggested a lower bound for the population size

5 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 457 given the number of niches to be maintained [20]. Nonetheless, since in most of the real applications there is very little a priori knowledge about the fitness landscape, it is generally agreed that the setting of those parameters remains the main problem when using FS [21]. In [24] and [25], we proposed a method that can effectively use dynamic species identification mechanism for estimating the optimal values for the population size and the niche radius without any a priori information on width, number, height, and position of the peaks in the fitness landscape under examination. The mechanism for dynamically identifying the species in the population at each generation is based on the definition of species and exploits the only information available in the population, i.e., the raw fitness of all the individuals and the similarity metric defined on either the genotypic or the phenotypic space. The skeleton of the mechanism is outlined in Algorithm 1. Algorithm 1: Dynamic Species Identification begin sort the current population according to the raw fitness; ; (number of niches at generation t) (Dynamic Species Set) for to do if the th individual is not marked then ; (number of individuals in the current niche) for to do if ( ( th individual is not marked)) then mark the th individuals as belonging to the th species; ; end if end for if then mark the th individual as the species master of the th species; insert the pair ( th individual, ) in DSS; end if end if end for end Dynamic identification of the species masters is obtained by ordering the population at generation according to the raw fitness of the individuals. Then, the best individual is chosen as species master candidate. Once a species master candidate has ; been selected, its species can be defined as a subset of individuals in the population which have a distance from the species master less than the niche radius and do not belong to other species. If such individuals are found in the current population, the candidate is assumed as the actual species master and the individuals belonging to its species marked, otherwise, the candidate is considered an isolated individual. The process is iterated by choosing the first not marked individual of as a master candidate for a new species and terminates when all the individuals in the current population have been analyzed. In such a way, it is possible to partition the population at the time into a number of species, say, and in a number of isolated individuals where represents the set of all the isolated individuals. The output provided by this algorithm is then used to implement the species sharing mechanism: for each species, the shared fitness is computed according to the (1), but assuming that only the individuals belonging to the current species contribute to the niche count. In fact, if the individual, its niche count is computed according to the following formula: while is computed according to (3). On the contrary, the fitness of isolated individuals is not modified, nor they contribute to modify the fitness of any other individual in the population, with the aim to improve the chance for those individuals to generate a new species. Summarizing, DFS provides, at each generation, the total number of species, the set of species masters, the distribution of the species in the fitness landscape, and the shared fitness value for each individual in the population. Note that, since the population has been ordered according to the raw fitness of the individuals, the species masters are ordered according to decreasing values of their raw fitness. Finally, the species elitist strategy is implemented by copying the species masters found at each generation in the next population. The introduction of this elitist strategy offers two main advantages. The preservation in the next generation of the species masters of all the discovered species avoids the risk that some of them, due to the application of genetic operators, may disappear. This strategy also allows for a significant improvement of the performance of our method in terms of both the number of generations and the population size needed for discovering and maintaining the species populating the actual niches of the fitness landscape. Moreover, in case of applications for which the actual number of niches to be populated can be estimated, a -elitist strategy can be implemented by simply copying the first species masters, if present, in the next population. Although it may be argued that, in general, the use of an elitist strategy favors exploitation rather then exploration, in our case this effect is mitigated by the following properties of DFS: (4)

6 458 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 TABLE I TEST PROBLEMS the genetic diversity is preserved because only the species masters are copied in the next population; in the worst case of a population in which each species master has only one individual in its neighborhood, the number of species masters copied in the next population is equal to, thus allowing that at most 50% of the population be replaced by new individuals at each generation. Finally, the complexity of the proposed algorithm is in the worst case (all the individuals in the population have mutual distance greater than ) and in the best case (all the individuals belong to the same niches), compared with the complexity of each iteration of the FS. The skeleton of an EA with DFS is outlined in the Algorithm 2. Algorithm 2: EA with Dynamic Fitness Sharing (DFS) begin ; randomly initialize a population of individuals; while (not reached the maximum number of generations) do evaluate the raw fitness of each individual; apply the Dynamic Species Identification algorithm; for to do apply the FS among the individuals belonging to the th species; end for copy the species masters in the new population; apply the selection mechanism; apply the crossover operator; perform mutation on the offsprings; end while end ; problems are widely studied and used in literature [3], so as to allow a reasonable experimental framework. Table I reports the main features of the test problems employed in terms of number of niches, extraneous peaks, deception, difficulty, and metric. For the sake of comparison, we have used the same metrics adopted in literature [20]. The EA used for all the experiments is the standard genetic algorithm described in [30]. For all the problems faced, the entire evolution of generations, i.e., the run, has been repeated times, with different initial populations in order to reduce the well-known effects of randomness embedded in the EAs. The performance of our method, as well as those of FS, DNS, and SCGA, has been measured in terms of the number of niches discovered and maintained during the evolution with respect to the actual number of peaks in the fitness landscape. Such a performance measure follows the assumption that FS aims at discovering and maintaining stable subpopulations, each populating a niche of the landscape, and that this is ensured only when a proper niche radius is selected, so as to have the perfect discrimination among peaks. If this is not true, as we have shown in [24], the dynamics of the system could be very unstable in terms of the number and the location of the niches discovered and maintained. In order to evaluate the performance, for each evolution we compute at each generation the number of niches (the population size and niche radius being fixed) and store it in the element of a niche matrix made of rows (one for each evolution) and columns (one for each generation). At the end of each experiment consisting in runs, we compute the average number of niches discovered at each generation by averaging the values in all the columns Then, the values represent the average behavior of the algorithm for the assigned values of and. Finally, we compute the standard errors (5) IV. EXPERIMENTAL FRAMEWORK In order to validate the proposed method, we have taken into account the classical functions for testing niching methods, often referred to in literature as, and. Such of. (6)

7 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 459 Fig. 1. (a) M1 and (b) M2 functions. The specific parameter settings for each test function employed are reported in the following subsections. A. and Functions The functions and have been originally proposed by Goldberg and Richardson [3] and are the simplest among the ones proposed in the literature for studying the behavior of a niching method. The aim in using such functions is particularly devoted to analyze the ability of the method to maintain the discovered niches rather than its searching ability. is defined as it follows: with. It exhibits five peaks for and, whose height is equal to 1.0. is the same as, but the values corresponding to the peaks are 1.0, 0.917, 0.707, 0.459, and 0.250, respectively. Its analytical form is the following: Fig. 1 shows the plots of and. As regards the experiments, in order to compare our experimental findings with those obtained by Mahfoud [20], for both functions we have used the same parameter setting, i.e., the genotype has been encoded by using 30 bits, the crossover and the mutation rates have been set to 0.9 and 0.01, respectively, has been set equal to 0.1, the FS has been accomplished by phenotypic comparison, the selection mechanism chosen has been the roulette wheel, and the number of generations has been set equal to 200. Finally, according to Mahfoud [20], we have computed for both functions the lower bound for the population size able to maintain the desired number of niches at least for generations with a given probability in case of peaks of identical fitness (7) (8) (9) and in case of peaks of arbitrary fitness (10) where is the ratio between the maximum and the minimum peak fitness values. To maintain five niches for generations with probability, it follows from (9) that should be 42 for, while according to (10), it should be 94 for. As regards the function, Fig. 2 reports the plots of the average number of niches discovered at each generation as computed by means of (5), along with the respective standard error, for FS, DNS, DFS, and SCGA, respectively. It should be noted that, in comparison to the other considered approaches, FS does not provide any explicit species identification. For this reason, the number of species discovered at each generation has been computed by using our dynamic species identification algorithm. The same plots are reported in Fig. 3 for the function. Our algorithm outperforms both FS and DNS in terms of number of niches maintained. In addition, DFS shows better performance than SCGA in that the latter, maintains a greater number of niches than the actual one, while DFS does not suffer from this drawback. The reason is that, depending on the distribution of the individuals on the landscape at a given time, the species identification procedure of SCGA could select a greater number of species masters than the actual number of species, and then the elitist strategy can only upgrade the species set. Our algorithm, instead, requires at least two individuals for identifying a species and its elitist strategy does not take memory of the past species set. In fact, as it can be simply noted from the figures, the standard error computed for SCGA is constant and greater than zero during the entire evolution, while it should be constant only when in all the runs the algorithm discovers the same number of niches, as with the case of our algorithm. Moreover, it should be noted that both the algorithms require a smaller population size with respect to the lower bound found by Mahfoud [20] for discovering and maintaining the actual number of niches. In fact, both the algorithms succeed in discovering and maintaining all

8 460 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 Fig. 2. M1: average number of niches and its standard error by (a) FS, (b) DNS, (c) SCGA, and (d) DFS. In all the experiments N =42. the niches in the landscape with a population size at least of 30 and 50 individuals for and, respectively. Finally, by computing the distance between the species masters and the known maxima, we can verify that the masters are correctly centered on the peaks. B. Function The third function used in our experiment was originally introduced by DeJong [2] and it is known as Shekel s Foxholes.It is a two-dimensional function, whose landscape exhibits 25 equidistant peaks emerging from a flat surface. The range of variations for the independent variables is the same, namely,, and its peaks are located in correspondence of the coordinates, where and are integer variables ranging in the interval. The peaks heights range in the interval and the highest peak is located at. The analytical form of the function is the following: (11) where and.a plot of the function is shown in Fig. 4. As the figure shows, this function is far more complex than the previous ones, but it is not deceptive, in that low order schemata still lead to near-optimal solutions. The parameter settings for the experimental findings are the same as those used by Mahfoud in [20]: the genotype has been encoded by using 17 bits, the crossover and the mutation rates have been set to 1.0 and 0.0, respectively, has been set equal to 11.0, the FS has been accomplished by phenotypic comparison, the selection mechanism chosen has been the roulette wheel, and the number of generations has been set equal to 200. As for the previous functions, following Mahfoud [20], we have computed also for the lower bound for the population size able to maintain the desired number of species. To maintain 25 niches for at least generations with probability, it follows from (10) that is equal to 266 individuals. As regards the experimental findings, Fig. 5 reports the plots of the average number of niches discovered at each generation as computed by means of (5), along with the respective standard error for FS, DNS, DFS, and SCGA, respectively. Also, in this case, it is evident that the performance of DFS is better than all the others. With regards to FS and DNS, DFS exhibits similar performance in discovering the niches at the beginning of the evolution, but it exhibits better performance in maintaining the discovered niches throughout the remaining evolution. In case of SCGA, DFS is better in both discovering and maintaining the niches, in that the discovery of the actual

9 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 461 Fig. 3. M2: average number of niches and its standard error by (a) FS, (b) DNS, (c) SCGA, and (d) DFS. In all the experiments N =94. niches takes less time and it maintains the actual number of niches, while SCGA maintains a greater number of them. It should be noted that, in case of, this behavior is more evident. In fact, while our algorithm discovers and maintains 25 species that correspond exactly to the actual niches in the landscape, SCGA maintains in the flat area a given number of seeds that do not correspond to any niche in the landscape. Finally, also for function, experiments have shown that both DFS and SCGA require a population size smaller than the lower bound estimated by Mahfoud. C. Function The function was introduced by Deb et al. in [31] in order to study the scaling problem in the FS. It is a function of 30 binary variables, massively multimodal and deceptive. Its expression is composed by the sum of the values of five unitation bimodal and deceptive subfunctions (Fig. 6). Such unitation functions are defined on binary strings and their values depend on the number of bits 1 in the strings. For example, if is a string of ten bits and is the number of bits 1 in the string, is an unitation function. The analytical form of is the following: (12) with and is a scale factor. The deception order, i.e., the number of bits that can be changed without any change in the fitness values is 6 bit. Each unitation function has 22 optima corresponding to and whose height is 1.0, and local optima corresponding to strings containing three 1, whose height is Thus, exhibits a total of optima, of whom 32 are optima with height equal to 5.0, and for the remaining ones the heights range in the interval. The difficulty in solving this kind of problem is that it requires a very efficient genetic search, a high genetic diversity in the population, and a scaling of the fitness function. With regard to the latter, is usually scaled by using equal to 15 [9], [21], while the genetic search can be improved by using either special genetic operators or larger mutation rates. Moreover, the lower bound for the population size able to maintain the desired number of species (32 optima and about 5,000,000 nondesirable local optima) cannot be used in this case, by setting the probability equal to 0.99 it gives a population size of 405. As expected, this population size is largely insufficient to solve the function. In fact, Goldberg et al. in [9] used a population size of 5000 individuals to face and solve the problem, while in [17] the coevolutionary sharing is introduced. It requires two populations, i.e., a population of 35 businessmen and a population of 2000 customers to solve the problem, where the locations of the businessmen correspond to niche locations

10 462 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 Fig. 4. M6 function. Fig. 5. M6: average number of niches and its standard error by (a) FS, (b) DNS, (c) SCGA, and (d) DFS. In all the experiments N = 266. and the locations of customers correspond to solutions. Experiments not reported here have shown that DNS requires a population size similar to that used by Goldberg et al. in [9] to solve the problem. Keeping these considerations in mind for the problem at hand, in the following we present only the results of DFS and SCGA, in that both FS and DNS require a much larger population size. The parameter settings for both the algorithms are the following:

11 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 463 Fig. 6. M7 unitation function. has been set to 6.0, the FS has been accomplished by genotypic comparison, has been set to 15, the selection mechanism chosen has been the roulette wheel, the number of generations has been set to 200, and the crossover rate has been set to 1.0. Moreover, to improve the genetic search efficiency, we have used as mutation the operator [32] with mutation rate of 1.0. Such an operator has proved to be more efficient that canonical bit-flip mutation in a wide range of applications, and it is based on simultaneous mutation of small groups of neighboring bits. This choice does not affect the comparison among different experiments, because it mainly affects the search ability of the algorithm, not its ability to maintain the discovered niches. Finally, a population size of 600 individuals has been used. Fig. 7 reports the plots of the average number of niches discovered at each generation as computed by means of (5), along with the respective standard error for DFS and SCGA, respectively. It should be noted that, while our algorithm is able to discover and maintain all the 32 desirable species, none of them is discovered by SCGA. Such a poor performance is due to both the absence of sharing and to the elitist mechanism of SCGA that drastically reduces the search ability. In fact, about 81% of the population is preserved by elitism and only 19% is used for searching new solutions, while in our algorithm only about 38% of the population is preserved at the end of evolution. Moreover, DFS makes use of the sharing mechanism for populating the discovered niches, while SCGA does not. In fact, in the experiment performed, no explicit upper limit on the number of species masters to be saved has been imposed. As can be simply noted from Fig. 7(b), the total number of species masters saved in the next generation does not reach in any case the upper limit of imposed by our method. If an explicit limit is imposed, the performance of DFS gets an improvement in terms of both the convergence ability, i.e., the number of generations needed to discover all the desirable species, and the population size needed to maintain those species. In fact, Fig. 8 reports the same plot as the previous one for DFS with an upper limit on the number of species masters to be saved equal to 100 and with a population size equal to 400 individuals. Such values have been chosen according to preliminary experimental tuning. As expected, our algorithm is able to discover and maintain all the 32 desired species, thus drastically reducing the drawback in using the elitist strategy. Fig. 7. M7: average number of niches and its standard error by (a) SCGA and (b) DFS. In all the experiments N =600. Fig. 8. M7: average number of niches and its standard error by DFS with an upper limit on the number of species to be saved equal to 100 and N = 400. Finally, it should be noted that, the approach of both the algorithms can be adopted only when there is a priori knowledge about the fitness landscape, i.e., the number of peaks and their locations are known, so as to select the desired species from all the niches maintained. When such information is not available, the number of niches is very large and the landscape is very complex as in the case of, something else is needed that

12 464 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 11, NO. 4, AUGUST 2007 Fig. 9. M7: average number of niches and its standard error by DFS with N = 400 and the threshold on the minimum fitness value set to 4.7. can help the algorithm to discriminate the desirable niches from the undesirable ones. Such a discrimination can be accomplished by introducing in our dynamic species identification algorithm a threshold value for the raw fitness. Only the individuals whose raw fitness is greater than the threshold are then considered species masters. This allows us to identify only the most relevant and perfectly discriminable peaks, if any. Obviously, such a method avoids setting an upper limit, other than, on the number of species masters to be preserved by the elitist strategy. Fig. 9 reports the results of DFS with no upper limit on the number of species masters to be saved and with the threshold on the minimum fitness value set to 4.7. Such a value has been chosen according to a preliminary experimental tuning, performed as discussed in [24] and [25]. As expected, our algorithm is able to discover and maintain all the 32 niches. It should be noted that only the niche masters of desired niches have been saved by the elitist strategy, thus drastically reducing the drawback in using such a technique. V. CONCLUSION In this paper, we have proposed a mechanism for FS, i.e., DFS, which has been devised in order to overcome the limitations exhibited by the FS. The proposed method allows: an explicit dynamic identification of the species discovered at each generation and their localization on the fitness landscape; the application of the sharing mechanism to each species separately; a straightforward implementation of a species elitist strategy, by copying in the new population the most representative individuals of the species, i.e., the species masters, discovered at each generation. Dynamic species identification is accomplished without assuming any a priori knowledge on the number of niches to be populated. The only assumption of our method, as well as of the FS and of all the methods employing a single niche radius to describe all the niches within the search domain, is the perfect discrimination hypothesis, according to which the peaks of the fitness landscape are fully distinguishable by means of a proper setting of the niche radius. Our method is based upon the idea that an explicit identification of the species at each generation can help both to ensure that each niche will be populated proportionally to the fitness of its peak and to preserve species with few individuals from extinction by allowing a species elitist mechanism. As a consequence, the overall performance of any EA adopting such a strategy should improve in terms of both maintenance and discovery. The experimental results obtained on a standard set of multimodal functions have shown that our method outperforms all other methods proposed in the literature and considered here. They have also shown that DFS is effective, because it provides all the actual species masters when effective values for the niche radius and for the population size are selected. It should also be noted that the elitist strategy implemented by our method allows to steadily maintain the actual number of peaks also in case of a population size smaller than the lower bound computed by Mahfoud. In our opinion, possible improvements for DFS could be the following. The definition of a mechanism that allows the use of a variable niche radius. In fact, real-world problems rarely have distinct peaks and a single value of the niche radius should not be adequate enough to achieve optimal niching. A mechanism that allows a dynamic population size similar to that introduced by opt-ainet in order to improve the discover and maintenance ability of our method. The estimate of the minimum population size needed by our method for steadily maintaining the actual number of peaks in the landscape. ACKNOWLEDGMENT The authors gratefully acknowledge the anonymous reviewers for their insight comments and useful suggestions. REFERENCES [1] D. Cavicchio, Adaptive search using simulated evolution, Ph.D. dissertation, Univ. Michigan, Ann Arbor, MI, [2] K. A. De Jong, An analysis of the behavior of a class of genetic adaptive systems, Ph.D. dissertation, Univ. Michigan, Ann Arbor, MI, 1975, Abstracts International 36(10), 5140B; Univ. Michigan [3] D. E. Goldberg and J. Richardson, Genetic algorithms with sharing for multimodal function optimization, in Genetic Algorithms and Their Applications, J. J. Grefenstette, Ed. Hillsdale, NJ: Lawrence Erlbaum, 1987, pp [4] R. E. Smith, S. Forrest, and A. S. Perelson, Searching for diverse, cooperative populations with genetic algorithms, Evol. Comput., vol. 1, no. 2, pp , [5] S. W. Mahfoud, Crowding and preselection revisited, in Lecture Notes in Computer Science, B. M. R. Manner, Ed. : Elsevier Science, 1992, Parallel Problem Solving from Nature 2, pp , No. IlliGAL Rep [6] D. Beasley, D. R. Bull, and R. R. Martin, A sequential niche technique for multimodal function optimization, Evol. Comput., vol. 1, no. 2, pp , [7] J. Horn, D. E. Goldberg, and K. Deb, Implicit niching in a learning classifier system: Nature s way, Evol. Comput., vol. 2, no. 1, pp , [8] H. Asoh and H. Mühlenbein, On the mean convergence time of evolutionary algorithms without selection and mutation, in Parallel Problem Solving From Nature: PPSN III, Y. Davidor, H.-P. Schwefel, and R. Männer, Eds. Berlin, Germany: Springer-Verlag, 1994, pp [9] C. K. Oei, D. E. Goldberg, and S.-J. Chang, Tournament selection, niching, and the preservation of diversity Univ. Illinois at Urbana Champaign, Urbana, IL, IlliGAL Rep , [10] S. Forrest, R. E. Smith, B. Javornik, and A. S. Perelson, Using genetic algorithms to explore pattern recognition in the immune system, Evol. Comput., vol. 1, no. 3, pp , 1993.

13 DELLA CIOPPA et al.: WHERE ARE THE NICHES? DYNAMIC FITNESS SHARING 465 [11] P. Darwen and X. Yao, Every niche method has its niche: Fitness sharing and implicit sharing compared, in Lecture Notes in Computer Science, H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, Eds. Berlin, Germany: Springer-Verlag, 1996, vol. 1141, Parallel Problem Solving From Nature-PPSN IV, pp [12] E. Mayr, Systematics and the Origin of Species From the Viewpoint of a Zoologist. New York: Columbia Univ. Press, [13] J. Horn, Finite Markov chain analysis of genetic algorithms with niching, in Proc. 5th Int. Conf. Genetic Algorithms, S. Forrest, Ed., 1993, pp [14], GAs (With Sharing) in Search, Optimization and Machine Learning. San Mateo, CA: Morgan Kaufmann, [15] K. Deb and D. E. Goldberg, An investigation of niche and species-formation in genetic function optimization, in Proc. 3rd Int. Conf. Genetic Algorithms, J. D. Schaffer, Ed., 1989, pp [16] J. Horn and D. E. Goldberg, Natural niching for evolving cooperative classifiers, in Proc. 1st Annu. Conf. Genetic Program., D.B.F.J.R. Koza, D. E. Goldberg, and R. L. Riolo, Eds., 1996, pp [17] D. E. Goldberg and L. Wang, Adaptive niching via coevolutionary sharing, in Genetic Algorithms and Evolution Strategies in Engineering and Computer Science. New York: Wiley, 1997, pp [18] L. B. Booker, D. E. Goldberg, and J. H. Holland, Classifier systems and genetic algorithms, Artif. Intell., vol. 40, pp , [19] S. Mahfoud, Genetic drift in sharing methods, in Proc. 1st IEEE Conf. Evol. Comput., 1994, pp [20] S. W. Mahfoud, Population size and genetic drift in fitness sharing, in Proc. Foundations Genetic Algorithms, L. D. Whitley and M. D. Vose, Eds., 1995, pp [21] B. Sareni and L. Krähenbühl, Fitness sharing and niching methods revisited, IEEE Trans. Evol. Comput., vol. 2, no. 3, pp , Sep [22] B. L. Miller and M. J. Shaw, Genetic algorithms with dynamic niche sharing for multimodal function optimization, in Proc IEEE Int. Conf. Evol. Comput., 1996, pp [23] W. M. Spears, Simple subpopulation schemes, in Proc. 4th Annu. Conf. Evol. Program., A. V. Sebald and L. J. Fogel, Eds., 1994, pp [24] A. Della Cioppa, C. De Stefano, and A. Marcelli, On the role of population size and niche radius in fitness sharing, IEEE Trans. Evol. Comput., vol. 8, no. 6, pp , Dec [25] A. Della Cioppa, Competizione e Cooperazione Negli Algoritmi Evolutivi: Il Problema Della Speciazione e i Metodi di Niching, (in Italian) Ph.D. dissertation, Univ. Naples Federico II, Naples, Italy, [26] K. O. Stanley and R. Mikkulainen, Evolving neural networks through augmenting topologies, Evol. Comput., vol. 10, no. 2, pp , [27], Competitive coevolution through evolutionary complexification, J. Artif. Intell. Res., vol. 21, pp , [28] M. E. Balazs, L. Jianping, G. T. Parks, and P. J. Clarkson, The effect of distance measure in a GA with species conservation, IEEE Trans. Evol. Comput., vol. 10, no. 3, pp , [29] L. N. de Castro and J. Timmins, An artificial immune network for multimodal function optimization, in Proc. IEEE Congr. Evol. Comput., IEEE World Congr. Comput. Intell., 2002, vol. 1, pp [30] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, [31] D. E. Goldberg, K. Deb, and J. Horn, Massive multimodality, deception, and genetic algorithms, in Lecture Notes in Computer Science,. Berlin, Germany: Springer-Verlag, 1992, Proc. 2nd Workshop, Parallel Problem Solving from Nature PPSN 2, pp [32] I. De Falco, A. Della Cioppa, A. Iazzetta, and E. Tarantino, The effectiveness of co-mutation in evolutionary algorithms: The M operator, in Proc. IEEE Int. Conf. Evol. Comput. World Congr. Comput. Intell., Anchorage, AK, 1998, pp Antonio Della Cioppa (M 05) was born in Bellona, Italy, on June 13, He received the Laurea degree in physics and the Ph.D. degree in computer science, both from University of Naples Federico II, Naples, Italy, in 1993 and 1999, respectively. From 1999 to 2003, he was a Postdoctoral Fellow at the Department of Computer Science and Electrical Engineering, University of Salerno, Salerno, Italy. In 2004, he joined the Department of Electrical and Information Engineering, University of Salerno, where he is currently an Assistant Professor of Computer Science and Artificial Intelligence. He has been active in the fields of Artificial Intelligence and Cybernetics. His current research interests are in the fields of theoretical and computational physics (complexity, statistical mechanics of equilibrium and nonequilibrium phenomena, theory of dynamical systems, chaos), prebiotic evolution, Darwinian dynamics and speciation, evolutionary computation, and artificial life. Dr. Della Cioppa is a member of the Association for Computing Machinery (ACM), the IEEE Computer Society, the IEEE Computational Intelligence Society, the European Network of Excellence in Evolutionary Computing (EvoNet), the Machine Learning Network (MLNet), the Knowledge Discovery Network of Excellence (KDNet), and the AIIA (ECCAI-Italian Chapter). He serves as Program Committee member of many international conferences such as the Genetic and Evolutionary Computation Conference and Conference on Evolutionary Computation. Claudio De Stefano was born in Naples, Italy, on October 4, He received the Laurea degree (Hon.) in electronic engineering and the Ph.D. degree in electronic and computer engineering from the University of Naples Federico II, Naples, Italy, in 1990 in 1994, respectively From 1994 to 1996, he was an Assistant Professor of Computer Science at the Department of Computer Science and Systems, University of Naples Federico II. In 1996, he joined the Faculty of Computer Engineering, University of Sannio, Benevento, where he has been an Assistant Professor of Computer Science. In 2001, he joined the Faculty of Engineering, University of Cassino, where he is currently an Associate Professor of Computer Science and Artificial Intelligence. He has authored over 70 research papers in international journals and conference proceedings. He has been active in the fields of pattern recognition, image analysis, machine learning, and parallel computing. His current research interests include online and offline handwriting recognition, cursive script segmentation, neural networks, and evolutionary learning systems. Dr. De Stefano is a member of the International Association for Pattern Recognition (IAPR). Angelo Marcelli (M 87) received the M.Sc. degree in electronic engineering (cum laude) and the Ph.D. degree in electronic and computer engineering from the University of Napoli Federico II, Naples, Italy, in 1983 and 1987, respectively. From 1987 to 1989, he was Chief Researcher of the Computer Vision and Artificial Intelligence Laboratory, CRIAI, Napoli, Italy, where he also founded and directed the Italy Russian Laboratory for Image Analysis and Processing. From 1989 to 1992, he has held a Researcher position at the Department of Computer and System Engineering, School of Engineering, University of Napoli Federico II. From 1992 to 1997, he was appointed a Senior Researcher and Assistant Professor of Computer Engineering at the same department. Since 1998, he has been with the Department of Electrical and Information Engineering, University of Salerno, where he is currently a Professor of Computer Engineering. He has been Visiting Scholar of many institutions, such as the Institute of Engineering Cybernetics, Minsk (BELARUS), the Image Analysis Laboratory, State University of New York, Stony Brook, Document Analysis Laboratory, Rensselaer Polytechnic Institute, Troy, NY. His current research interest include handwriting recognition, theory and application of evolutionary algorithms, active vision model, and natural computation. Dr. Marcelli is a member of the International Association for Pattern Recognition (IAPR) and is the President-Elect of the International Graphonomics Society. He serves as Area Editor for the International Journal of Document Analysis and Recognition, as Reviewer for many international journals, such as the IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, the IEEE TRANSACTIONS ON CIRCUIT AND SYSTEMS FOR VIDEO TECHNOLOGY, the IEEE TRANSACTIONS ON IMAGE PROCESSING, Pattern Recognition, and Pattern Recognition Letters.