arxiv: v1 [cs.ai] 20 Nov 2013

Transcription

1 Aalyzig Evolutioary Optimizatio i Noisy Eviromets Chao Qia, Yag Yu, Zhi-Hua Zhou Natioal Key Laboratory for Novel Software Techology Najig Uiversity, Najig 20023, Chia arxiv:3.4987v [cs.ai] 20 Nov 203 Abstract May optimizatio tasks have to be hadled i oisy eviromets, where we caot obtai the exact evaluatio of a solutio but oly a oisy oe. For oisy optimizatio tasks, evolutioary algorithms (EAs), a kid of stochastic metaheuristic search algorithm, have bee widely ad successfully applied. Previous work maily focuses o empirical studyig ad desigig EAs for oisy optimizatio, while, the theoretical couterpart has bee little ivestigated. I this paper, we ivestigate a largely igored questio, i.e., whether a optimizatio problem will always become harder for EAs i a oisy eviromet. We prove that the aswer is egative, with respect to the measuremet of the expected ruig time. The result implies that, for optimizatio tasks that have already bee quite hard to solve, the oise may ot have a egative effect, ad the easier a task the more egatively affected by the oise. O a represetative problem where the oise has a strog egative effect, we examie two commoly employed mechaisms i EAs dealig with oise, the re-evaluatio ad the threshold selectio strategies. The aalysis discloses that the two strategies, however, both are ot effective, i.e., they do ot make the EA more oise tolerat. We the fid that a small modificatio of the threshold selectio allows it to be prove as a effective strategy for dealig with the oise i the problem. Key words: Noisy optimizatio, evolutioary algorithms, re-evaluatio, threshold selectio, ruig time, computatioal complexity Correspodig author addresses: qiac@lamda.ju.edu.c (Chao Qia), yuy@ju.edu.c (Yag Yu), zhouzh@ju.edu.c (Zhi-Hua Zhou) Preprit submitted for review October 8, 208

2 . Itroductio Optimizatio tasks ofte ecouter oisy eviromets. For example, i airplae desig, every prototype is evaluated by simulatios so that the evaluatio result may ot be perfect due to the simulatio error; ad i machie learig, a predictio model is evaluated oly o a limited amout of data so that the estimated performace is shifted from the true performace. Noisy eviromets could chage the property of a optimizatio problem, thus traditioal optimizatio techiques may have low efficacy. While, evolutioary algorithms (EAs) [4] have bee widely ad successfully adopted for oisy optimizatio tasks [5, 22, 9, 8]. EAs are a kid of radomized metaheuristic optimizatio algorithms, ispired by atural pheomea icludig evolutio of species, swarm cooperatio, immue system, etc. EAs typically ivolve a cycle of three stages: reproductio stage produces ew solutios based o the curretly maitaied solutios; evaluatio stage evaluates the ewly geerated solutios; selectio stage wipes out bad solutios. A ispiratio of usig EAs for oisy optimizatio is that the correspodig atural pheomea have bee processed successfully i oisy eviromets, ad hece the algorithmic simulatios are also likely to be able to hadle oise. Besides, improved mechaisms have bee iveted for better hadlig oise. Two represetative strategies are re-evaluatio ad threshold selectio: by the re-evaluatio strategy [2, 6], wheever the fitess (also called cost or objective value) of a solutio is required, EAs make a idepedet evaluatio of the solutio despite of whether the solutio has bee evaluated before, such that the fitess is smoothed; by the threshold selectio strategy [23, 6, 5], i the selectio stage EAs accept a ewly geerated solutio oly if its fitess is larger tha the fitess of the old solutio by at least a threshold, such that the risk of acceptig a bad solutio due to oise is reduced. A assumptio implied by usig a oise hadlig mechaism i EAs is that the oise makes the optimizatio harder, so that a better hadlig mechaism ca reduce the egative effect by the oise [3, 7, 26, 3]. This paper firstly ivestigates if this assumptio is true. We start by presetig a experimetal evidece usig (+)-EA optimizig the hardest case i the pseudo-boolea fuctio class [25]. Experimet results idicate that the oise, however, makes the optimizatio easier rather tha harder, uder the measuremet of expected ruig time. Followig the experimet evidece, we the derive sufficiet theoretical coditios, uder which the oise will make the optimizatio easier or harder. By fillig the coditios, we preset proofs that, for the (+λ)-ea (a class of EAs employig offsprig populatio size λ), the oise will make the optimizatio easier o the hardest case i the pseudo-boolea fuctio class, while harder o the easiest case. The proofs imply that we eed to take care of the oise oly whe the optimizatio is moderately or less complex, ad igore this issue whe the optimizatio task itself is quite hard. 2

3 For the situatios where the oise eeds to be cared, this paper examies the re-evaluatio ad the threshold selectio strategies for their polyomial oise tolerace (PNT). For a kid of oise, the PNT of a EA is the maximum oise level such that the expected ruig time of the algorithm is polyomial. The closer the PNT is to, the better the oise tolerace is. Takig the easiest pseudo-boolea fuctio case as the represetative problem, we aalyze the PNT for differet cofiguratios of the (+)-EA with respect to the oe-bit oise, whose level is characterized by the oise probability. For the (+)-EA (without ay oise hadlig strategy), we prove that the PNT has a lower boud Ω(poly()) ad a upper boud O(2 poly()). Sice the (+)-EA with re-evaluatio has the PNT Θ( log ) [0], it is surprisigly that the re-evaluatio makes the PNT much worse. We further prove that for the (+)-EA with re-evaluatio usig threshold selectio, whe the threshold is, the PNT is ot less tha 2e, ad whe the threshold is 2, the PNT has a lower boud Ω(poly()) ad a upper boud O(2 poly()). The PNT bouds idicate that threshold selectio improves the re-evaluatio strategy, however, o improvemets from the (+)-EA are foud. We the itroduce a small modificatio ito the threshold selectio strategy to tur the origial hard threshold to be a smooth threshold. We prove that with the smooth threshold selectio strategy the PNT is, i.e., the (+)-EA is always a polyomial algorithm disregard the probability of oe-bit oise o the problem. The rest of this paper is orgaized as follows. Sectio 2 itroduces some backgroud. Sectio 3 shows that the oise may ot always be bad, ad presets a sufficiet coditio for that. Sectio 4 aalyzes oise hadlig strategies. Sectio 5 cocludes. 2. Backgroud 2.. Noisy Optimizatio A geeral optimizatio problem ca be represeted as arg max x f(x), where the objective f is also called fitess i the cotext of evolutioary computatio. I real-world optimizatio tasks, the fitess evaluatio for a solutio is usually disturbed by oise, ad cosequetly we ca ot obtai the exact fitess value but oly a oisy oe. I this paper, we will ivolve the followig kids of oise, ad we will always deote f N (x) ad f(x) as the oisy ad true fitess of a solutio x, respectively. additive oise f N (x) = f(x) + δ, where δ is uiformly selected from [δ, δ 2 ] at radom. multiplicative oise f N (x) = f(x) δ, where δ is uiformly selected from [δ, δ 2 ] at radom. 3

4 oe-bit oise f N (x) = f(x) with probability ( p ) (0 p ); otherwise, f N (x) = f(x ), where x is geerated by flippig a uiformly radomly chose bit of x {0, }. This oise is for problems where solutios are represeted i biary strigs. Additive ad multiplicative oise has bee ofte used for aalyzig the effect of oise [7, 2]. Oebit oise is specifically for optimizig pseudo-boolea problems over {0, }, ad also the ivestigated oise i the oly previous work for aalyzig ruig time of EAs i oisy optimizatio [0]. For oe-bit oise, p cotrols the oise level. I this paper we assume that the parameters of the eviromet (i.e., p, δ ad δ 2 ) do ot chage over time. It is possible that a large oise could make a optimizatio problem extremely hard for particular algorithms. We are iterested i the oise level, uder which a algorithm could be tolerat to have polyomial ruig time. We defie the polyomial oise tolerace (PNT) as Defiitio, which characterizes the maximum oise level for allowig a polyomial expected ruig time. Note that, the oise level ca be measured by the adjustig parameter, e.g., δ, δ 2 for the additive ad multiplicative oise, ad p for the oe-bit oise. We will study the PNT of EAs for aalyzig the effectiveess of oise hadlig strategies. Defiitio (Polyomial Noise Tolerace (PNT)) The polyomial oise tolerace of a algorithm o a problem, with respect to a kid of oise, is the maximum oise level such that the algorithm has expected ruig time polyomial to the problem size Evolutioary Algorithms Evolutioary algorithms (EAs) [4] are a kid of populatio-based metaheuristic optimizatio algorithms. Although there exist may variats, the commo procedure of EAs ca be described as follows:. Geerate a iitial set of solutios (called populatio); 2. Reproduce ew solutios from the curret populatio; 3. Evaluate the ewly geerated solutios; 4. Update the populatio by removig bad solutios; 5. Repeat steps 2-5 util some criterio is met. The (+)-EA, as i Algorithm, is a simple EA for maximizig pseudo-boolea problems over {0, }, which reflects the commo structure of EAs. It maitais oly oe solutio, ad repeatedly improves the curret solutio by usig bit-wise mutatio (i.e., the 3rd step of Algorithm ). It has bee widely used for the ruig time aalysis of EAs, e.g., [7, 2]. 4

5 Algorithm ((+)-EA) Give pseudo-boolea fuctio f with solutio legth, it cosists of the followig steps:. x := radomly selected from {0, }. 2. Repeat util the termiatio coditio is met 3. x := flip each bit of x with probability p. 4. if f(x ) f(x) 5. x := x. where p (0, 0.5) is the mutatio probability. The (+λ)-ea, as i Algorithm 2, applies a offsprig populatio size λ. I each iteratio, it first geerates λ offsprig solutios by idepedetly mutatig the curret solutio λ times, ad the selects the best solutio from the curret solutio ad the offsprig solutios as the ext solutio. It has bee used to disclose the effect of offsprig populatio size by ruig time aalysis [20, 24]. Note that, (+)-EA is a special case of (+λ)-ea with λ =. Algorithm 2 ((+λ)-ea) Give pseudo-boolea fuctio f with solutio legth, it cosists of the followig steps:. x := radomly selected from {0, }. 2. Repeat util the termiatio coditio is met 3. i :=. 4. Repeat util i > λ. 5. x i := flip each bit of x with probability p. 6. i := i x = arg max x {x,x,...,x λ } f(x ). where p (0, 0.5) is the mutatio probability. The ruig time of EAs is usually defied as the umber of fitess evaluatios (i.e., computig f( )) util a optimal solutio is foud for the first time, sice the fitess evaluatio is the computatioal process with the highest cost of the algorithm [7, 28] Markov Chai Modelig We will aalyze EAs by modelig them as Markov chais i this paper. Here, we first give some prelimiaries. EAs geerate solutios oly based o their curretly maitaied solutios, thus, they ca be modeled ad aalyzed as Markov chais, e.g., [7, 28]. A Markov chai {ξ t } + t=0 modelig a EA is costructed by takig the EA s populatio space X as the chai s state space, i.e. ξ t X. Let X X deote the set of all optimal populatios, which cotais at least oe optimal solutio. The goal 5

6 of the EA is to reach X from a iitial populatio. Thus, the process of a EA seekig X ca be aalyzed by studyig the correspodig Markov chai. A Markov chai {ξ t } + t=0 (ξ t X ) is a radom process, where t 0, ξ t+ depeds oly o ξ t. A Markov chai {ξ t } + t=0 is said to be homogeeous, if t 0, x, y X : P (ξ t+ = y ξ t = x) = P (ξ = y ξ 0 = x). () I this paper, we always deote X ad X as the state space ad the optimal state space of a Markov chai, respectively. Give a Markov chai {ξ t } + t=0 ad ξˆt = x, we defie the first hittig time (FHT) of the chai as a radom variable τ such that τ = mi{t ξˆt+t X, t 0}. That is, τ is the umber of steps eeded to reach the optimal state space for the first time startig from ξˆt = x. The mathematical expectatio of τ, E[[τ ξˆt = x]] = ip (τ = i), is called the expected first hittig time (EFHT) of this chai startig from ξˆt = x. If ξ 0 is draw from a distributio π 0, E[[τ ξ 0 π 0 ]] = x X π 0(x)E[[τ ξ 0 = x]] is called the expected first hittig time of the Markov chai over the iitial distributio π 0. For the correspodig EA, the ruig time is the umbers of calls to the fitess fuctio util meetig a optimal solutio for the first time. Thus, the expected ruig time startig from ξ 0 ad that startig from ξ 0 π 0 are respectively equal to N + N 2 E[[τ ξ 0 ]] ad N + N 2 E[[τ ξ 0 π 0 ]], (2) where N ad N 2 are the umber of fitess evaluatios for the iitial populatio ad each iteratio, respectively. For example, for (+)-EA, N = ad N 2 = ; for (+λ)-ea, N = ad N 2 = λ. Note that, whe ivolvig the expected ruig time of a EA o a problem i this paper, if the iitial populatio is ot specified, it is the expected ruig time startig from a uiform iitial distributio π u, i.e., N + N 2 E[[τ ξ 0 π u ]] = N + N 2 x X X E[[τ ξ 0 = x]]. The followig two lemmas o the EFHT of Markov chais [4] will be used i this paper. Lemma Give a Markov chai {ξ t } + t=0, we have Lemma 2 x X : E[[τ ξ t = x]] = 0; x / X : E[[τ ξ t = x]] = + y X P (ξ t+ = y ξ t = x)e[[τ ξ t+ = y]]. Give a homogeeous Markov chai {ξ t } + t=0, it holds t, t 2 0, x X : E[[τ ξ t = x]] = E[[τ ξ t2 = x]]. 6

7 For aalyzig the EFHT of Markov chais, drift aalysis [7, 8] is a commoly used tool, which will also be used i this paper. To use drift aalysis, it eeds to costruct a fuctio V (x) (x X ) to measure the distace of a state x to the optimal state space X. The distace fuctio V (x) satisfies that V (x X ) = 0 ad V (x / X ) > 0. The, by ivestigatig the progress o the distace to X i each step, i.e., E[[V (ξ t ) V (ξ t+ ) ξ t ]], a upper (lower) boud of the EFHT ca be derived through dividig the iitial distace by a lower (upper) boud of the progress. Lemma 3 (Drift Aalysis [7, 8]) Give a Markov chai {ξ t } + t=0 ad a distace fuctio V (x), if it satisfies that for ay t 0 ad ay ξ t with V (ξ t ) > 0, the the EFHT of this chai satisfies that where c l, c u are costats. 0 < c l E[[V (ξ t ) V (ξ t+ ) ξ t ]] c u, V (ξ 0 )/c u E[[τ ξ 0 ]] V (ξ 0 )/c l, 2.4. Pseudo-Boolea Fuctios The pseudo-boolea fuctio class i Defiitio 2 is a large fuctio class which oly requires the solutio space to be {0, } ad the objective space to be R. May well-kow NP-hard problems (e.g., the vertex cover problem ad the 0- kapsack problem) belog to this class. Diverse pseudo- Boolea problems with differet structures ad difficulties have bee used for aalyzig the ruig time of EAs, ad the to disclose properties of EAs, e.g., [, 7, 2]. Note that, we cosider oly maximizatio problems i this paper sice miimizig f is equivalet to maximizig f. Defiitio 2 (Pseudo-Boolea Fuctio) A fuctio i the pseudo-boolea fuctio class has the form: f : {0, } R. I hardest (or called Trap) problem i Defiitio 3 is a special istace i this class, which is to maximize the umber of 0 bits of a solutio except the global optimum... (briefly deoted as ). Its optimal fuctio value is 2, ad the fuctio value for ay o-optimal solutio is ot larger tha 0. It has bee widely used i the theoretical aalysis of EAs, ad the expected ruig time of (+)-EA with mutatio probability has bee proved to be Θ( ) [2]. It has also bee recogized as the hardest istace i the pseudo-boolea fuctio class with a uique global optimum for the (+)-EA [25]. Defiitio 3 (I hardest Problem) I hardest Problem of size is to fid a bits biary strig x such that x ( = arg max x {0,} f(x) = 3 x i 7 x i ),

8 where x i is the i-th bit of a solutio x {0, }. I easiest (or called OeMax) problem i Defiitio 4 is to maximize the umber of bits of a solutio. The optimal solutio is, which has the maximal fuctio value. The ruig time of EAs has bee well studied o this problem [7, 2, 27]. Particularly, the expected ruig time of (+)-EA with mutatio probability o it has bee proved to be Θ( log ) [2]. It has also bee recogized as the easiest istace i the pseudo-boolea fuctio class with a uique global optimum for the (+)-EA [25]. Defiitio 4 (I easiest Problem) I easiest Problem of size is to fid a bits biary strig x such that x ( = arg max x {0,} f(x) = where x i is the i-th bit of a solutio x {0, }. x i ), 3. Noise is Not Always Bad 3.. Empirical Evidece It has bee observed that oisy fitess evaluatio ca make a optimizatio harder for EAs, sice it may make a bad solutio have a better fitess, ad the mislead the search directio of EAs. Droste [0] proved that the ruig time of (+)-EA ca icrease from polyomial to expoetial due to the presece of oise. However, whe studyig the ruig time of (+)-EA solvig the hardest case I hardest i the pseudo-boolea fuctio class, we have observed oppositely that oise ca also make a optimizatio easier for EAs, which meas that the presece of the oise decreases the ruig time of EAs for fidig the optimal solutio. For I hardest problem over {0, }, there are 2 possible solutios, which are deoted by their correspodig iteger values 0,,..., 2, respectively. The, we estimate the expected ruig time of (+)-EA maximizig I hardest whe startig from every solutio. For each iitial solutio, we repeat idepedet rus for 000 times, ad the the average ruig time is recorded as a estimatio of the expected ruig time (briefly called as ERT). We ru (+)-EA without oise, with additive oise ad with multiplicative oise, respectively. For the mutatio probability of (+)-EA, we use the commo settig p =. For additive oise, δ = ad δ 2 =, ad for multiplicative oise, δ = 0. ad δ 2 = 0. The results for = 3, 4, 5 are plotted i Figure. We ca observe that the curves by these two kids of oise are always uder the curve without oise, which shows that I hardest problem becomes easier for (+)-EA i a oisy eviromet. Note that, the three curves meet at the last poit, sice the iitial solutio 2 is the optimal solutio ad the ERT =. 8

9 Estimated ERT without oise 5 additive multiplicative Iitial solutio Estimated ERT without oise 50 additive multiplicative Iitial solutio Estimated ERT without oise 500 additive multiplicative Iitial solutio (a) = 3 (b) = 4 (c) = 5 Figure : Estimated ERT compariso for (+)-EA solvig I hardest problem with or without oise A Sufficiet Coditio I this sectio, by comparig the expected ruig time of EAs with ad without oise, we derive a sufficiet coditio uder which the oise will make a optimizatio easier for EAs. Most practical EAs employ time-ivariat operators, thus we ca model a EA without oise by a homogeeous Markov chai. While for a EA with oise, sice oise may chage over time, we ca just model it by a Markov chai. Note that, the two EAs with ad without oise are differet oly o whether the fitess evaluatio is disturbed by oise, thus, they must have the same values o N ad N 2 for their ruig time Eq.2. The, comparig their expected ruig time is equivalet to comparig the EFHT of their correspodig Markov chais. We first defie a partitio of the state space of a homogeeous Markov chai based o the EFHT, ad the defie a jumpig probability of a Markov chai from oe state to oe state space i oe step. It is easy to see that X 0 i Defiitio 5 is just X, sice E[[τ ξ 0 X ]] = 0. Defiitio 5 (EFHT-Partitio) For a homogeeous Markov chai {ξ t } + t=0, the EFHT-Partitio is a partitio of X ito o-empty subspaces {X 0, X,..., X m } such that () x, y X i, E[[τ ξ 0 = x]] = E[[τ ξ 0 = y]]; (2) E[[τ ξ 0 X 0 ]] < E[[τ ξ 0 X ]] <... < E[[τ ξ 0 X m ]]. Defiitio 6 For a Markov chai {ξ t } + t=0, P t ξ (x, X ) = y X P (ξ t+ = y ξ t = x) is the probability of jumpig from state x to state space X X i oe step at time t. Theorem Give a EA A ad a problem f, let a Markov chai {ξ t } + t=0 ad a homogeeous Markov chai {ξ t} + t=0 model A ruig o f with oise ad without oise respectively, ad deote {X 0, X,..., X m } as the EFHT-Partitio of {ξ t} + t=0, if for all t 0, x X X 0, ad for all itegers i [0, m ], i j=0 P t ξ (x, X j ) i j=0 P t ξ (x, X j), (3) 9

10 the oise makes f easier for A, i.e., for all x X, E[[τ ξ 0 = x]] E[[τ ξ 0 = x]]. The coditio of this theorem (i.e., Eq.3) ituitively meas that the presece of oise leads to a larger probability of jumpig ito good states (i.e., X j with small j values), startig from which the EA eeds less time for fidig the optimal solutio. For the proof, we eed the followig lemma, which is proved i the appedix. Lemma 4 Let m (m ) be a iteger. If it satisfies that () 0 i m, P i, Q i 0, ad m P i = m Q i = ; (2) 0 E 0 < E <... < E m ; (3) 0 k m, k P i k Q i, the it holds that m P i E i m Q i E i. Proof of Theorem. holds. We use Lemma 3 to derive a boud o E[[τ ξ 0 ]], based o which this theorem For usig Lemma 3 to aalyze E[[τ ξ 0 ]], we first costruct a distace fuctio V (x) as x X, V (x) = E[[τ ξ 0 = x]], (4) which satisfies that V (x X ) = 0 ad V (x / X ) > 0 by Lemma. The, we ivestigate E[[V (ξ t ) V (ξ t+ ) ξ t = x]] for ay x with V (x) > 0 (i.e., x / X ). E[[V (ξ t ) V (ξ t+ ) ξ t = x]] = V (x) E[[V (ξ t+ ) ξ t = x]] = V (x) y X P (ξ t+ = y ξ t = x)v (y) = E[[τ ξ 0 = x]] y X P (ξ t+ = y ξ t = x)e[[τ ξ 0 = y]] (by Eq.4) = + y X P (ξ = y ξ 0 = x)e[[τ ξ = y]] y X P (ξ t+ = y ξ t = x)e[[τ ξ 0 = y]] (by Lemma ) = + y X P (ξ t+ = y ξ t = x)e[[τ ξ 0 = y]] y X P (ξ t+ = y ξ t = x)e[[τ ξ 0 = y]] (by Eq. ad Lemma 2, sice {ξ t} + t=0 is homogeeous.) = + m j=0 (P t ξ (x, X j) P t ξ (x, X j ))E[[τ ξ 0 X j ]]. (by Defiitios 5 ad 6) 0

11 Sice m j=0 P t ξ (x, X j) = m j=0 P t ξ (x, X j) =, E[[τ ξ 0 X j ]] icreases with j ad Eq.3 holds, by Lemma 4, we have m Thus, we have, for all t 0, all x / X, j=0 P t ξ (x, X j)e[[τ ξ 0 X j ]] m j=0 P t ξ (x, X j )E[[τ ξ 0 X j ]]. E[[V (ξ t ) V (ξ t+ ) ξ t = x]]. Thus, by Lemma 3, we get for all x X, E[[τ ξ 0 = x]] V (x) = E[[τ ξ 0 = x]], (the = is by Eq.4) which implies that oise leads to less time for fidig the optimal solutio, i.e., oise makes optimizatio easier. We prove below that the experimetal example satisfies this sufficiet coditio. We cosider (+λ)- EA, which covers (+)-EA ad is much more geeral. Let {ξ t } + t=0 ad {ξ t} + t=0 model (+λ)-ea with ad without oise for maximizig I hardest problem, respectively. For I hardest problem, it is to maximize the umber of 0 bits except the optimal solutio. It is ot hard to see that the EFHT E[[τ ξ 0 = x]] oly depeds o x 0 (i.e., the umber of 0 bits). We deote E (j) as E[[τ ξ 0 = x]] with x 0 = j. The order of E (j) is showed i Lemma 5, the proof of which is i the Appedix. Lemma 5 For ay mutatio probability 0 < p < 0.5, it holds that E (0) < E () < E (2) <... < E (). Theorem 2 Either additive oise with δ 2 δ 2 or multiplicative oise with δ 2 > δ > 0 makes I hardest problem easier for (+λ)-ea with mutatio probability less tha 0.5. Proof. The proof is by showig that the coditio of Theorem (i.e., Eq.3) holds here. By Lemma 5, the EFHT-Partitio of {ξ t} + t=0 is X i = {x {0, } x 0 = i} (0 i ) ad m i Theorem equals to here. Let f N (x) ad f(x) deote the oisy ad true fitess, respectively. For ay x X k (k ), we deote P (0) ad P (j) ( j ) as the probability that for the λ offsprig solutios x,..., x λ geerated by bit-wise mutatio o x, mi{ x 0,..., x λ 0 } = 0 (i.e., the least umber of 0 bits is 0), ad mi{ x 0,..., x λ 0 } > 0 max{ x 0,..., x λ 0 } = j (i.e., the largest umber of 0 bits is j while the least umber of 0 bits is larger tha 0), respectively. The, we aalyze oe-step trasitio probabilities from x for both {ξ t} + t=0 (i.e., without oise) ad {ξ t} + t=0 (i.e., with oise).

12 For {ξ t} + t=0, because oly the optimal solutio or the solutio with the largest umber of 0 bit amog the paret solutio ad λ offsprig solutios will be accepted, we have P t ξ (x, X 0) = P (0); j k : P t ξ (x, X j) = 0; P t ξ (x, X k) = k j= P (j); k + j : P t ξ (x, X j) = P (j). (5) For {ξ t } + t=0 with additive oise, sice δ 2 δ 2, we have f N ( ) f( ) + δ 2 + δ 2 2 = δ 2 ; y, f N (y) f(y) + δ 2 δ 2. For multiplicative oise, sice δ 2 > δ > 0, the f N ( ) > 0; y, f N (y) 0. Thus, for these two oises, we have y, f N ( ) f N (y), which implies that if the optimal solutio is geerated, it will always be accepted. Thus, we have, ote that X 0 = { }, P t ξ (x, X 0 ) = P (0). (6) Due to the fitess evaluatio disturbed by oise, the solutio with the largest umber of 0 bit amog the paret solutio ad λ offsprig solutios may be rejected. Thus, we have k + i : Pξ t (x, X j ) P (j). (7) j=i j=i By combiig Eq.5, Eq.6 ad Eq.7, we have i : Pξ t (x, X j ) j=i Pξ t (x, X j). j=i Sice j=0 P t ξ (x, X j) = j=0 P t ξ (x, X j) =, the above iequality is equivalet to 0 i : i Pξ t (x, X j ) j=0 i Pξ t (x, X j), j=0 which implies that the coditio Eq.3 of Theorem holds. Thus, we ca get that I hardest problem becomes easier for (+λ)-ea uder these two kids of oise. Theorem gives a sufficiet coditio for that oise makes optimizatio easier. If its coditio Eq.3 chages the iequality directio, which implies that oise leads to a smaller probability of jumpig to good states, it obviously becomes a sufficiet coditio for that oise makes optimizatio harder. We show it i Theorem 3, the proof of which is as similar as that of Theorem, except that the iequality directio eeds to be chaged. 2

13 Theorem 3 Give a EA A ad a problem f, let a Markov chai {ξ t } + t=0 ad a homogeeous Markov chai {ξ t} + t=0 model A ruig o f with oise ad without oise respectively, ad deote {X 0, X,..., X m } as the EFHT-Partitio of {ξ t} + t=0, if for all t 0, x X X 0, ad for all itegers i [0, m ], i j=0 P t ξ (x, X j ) i j=0 P t ξ (x, X j), (8) the oise makes f harder for A, i.e., for all x X, E[[τ ξ 0 = x]] E[[τ ξ 0 = x]]. The we apply this coditio to the case that (+λ)-ea is used for optimizig the easiest case I easiest i the pseudo-boolea fuctio class. Let {ξ t } + t=0 ad {ξ t} + t=0 model (+λ)-ea with ad without oise for maximizig I easiest problem, respectively. It is ot hard to see that the EFHT E[[τ ξ 0 = x]] oly depeds o x 0. We deote E 2 (j) as E[[τ ξ 0 = x]] with x 0 = j. The order of E 2 (j) is showed i Lemma 6, the proof of which is i the Appedix. Lemma 6 For ay mutatio probability 0 < p < 0.5, it holds that E 2 (0) < E 2 () < E 2 (2) <... < E 2 (). Theorem 4 Ay oise makes I easiest problem harder for (+λ)-ea with mutatio probability less tha 0.5. Proof. We use Theorem 3 to prove it. By Lemma 6, the EFHT-Partitio of {ξ t} + t=0 is X i = {x {0, } x 0 = i} (0 i ). For ay o-optimal solutio x X k (k > 0), we deote P (j) (0 j ) as the probability that the least umber of 0 bits for the λ offsprig solutios geerated by bit-wise mutatio o x is j. For {ξ t} + t=0, because the solutio with the least umber of 0 bits amog the paret solutio ad λ offsprig solutios will be accepted, we have 0 j k : P t ξ (x, X j) = P (j); P t ξ (x, X k) = j=k P (j); k + j : P t ξ (x, X j) = 0. For {ξ t } + t=0, due to the fitess evaluatio disturbed by oise, the solutio with the least umber of 0 bits amog the paret solutio ad λ offsprig solutios may be rejected. Thus, we have i i 0 i k : Pξ t (x, X j ) P (j). The, we ca get 0 i : j=0 j=0 i Pξ t (x, X j ) j=0 i Pξ t (x, X j). j=0 This implies that the coditio Eq.8 of Theorem 3 holds. Thus, by Theorem 3, we ca get that oise makes I easiest problem harder for (+λ)-ea. 3

14 3.3. Discussio We have show that oise makes I hardest ad I easiest problems easier ad harder, respectively, for (+λ)-ea. These two problems are kow to be the hardest ad the easiest istace respectively i the pseudo-boolea fuctio class with a uique global optimum for the (+)-EA [25]. We ca ituitively iterpret the discovered effect of oise for EAs o these two problems. For I hardest problem, the EA searches alog the deceptive directio while oise ca add some radomess to make the EA have some possibility to ru alog the right directio; for I easiest problem, the EA searches alog the right directio while oise ca oly harm the optimizatio process. We thus hypothesize that we eed to take care of the oise oly whe the optimizatio problem is moderately or less complex. To further verify our hypothesis, we employ the Jump m, problem, which is a problem with adjustable difficulty ad ca be cofigured as I eaisest whe m = ad I hardest whe m =. Defiitio 7 (Jump m, Problem) Jump m, Problem of size with m is to fid a bits biary strig x such that ( m + x = arg max x {0,} Jump m, (x) = x i if x i m or x i = x i otherwise where x i is the i-th bit of a solutio x {0, }. We test (+)-EA with mutatio probability o Jump m,. It is kow that the expected ruig time of the (+)-EA o Jump m, is Θ( m + log ) [2], which implies that Jump m, with larger value of m is harder. I the experimet, we set = 5, ad for oise, we use the additive oise with δ = 0.5 δ 2 = 0.5, the multiplicative oise with δ = δ 2 = 2, ad the oe-bit oise with p = 0.5, respectively. We record the expected ruig time gap startig from each iitial solutio gap = (E[[τ]] E[[τ ]])/E[[τ ]], where E[[τ]] ad E[[τ ]] deote the expected ruig time of the EA optimizig the problem with ad without oise, respectively. The larger the gap meas that the oise has a more egative effect, while the smaller the gap meas that the oise has a less egative effect. For each iitial solutio ad each cofiguratio of oise, we repeat the ruig of the (+)-EA 000 times, ad estimate the expected ruig time by the average ruig time, ad thus estimate the gap. The results are plotted i Figure 2. We ca observe that the gaps for larger m are lower (i.e., the egative effect by oise decreases as the problem hardess icreases), ad the gaps for large m ted to be 0 or egative values (i.e., oise ca have o or positive effect whe the optimizatio is quite hard). These empirical observatios give support to our hypothesis that the oise should be hadled carefully oly whe the optimizatio is moderately or less complex. ), 4

15 Estimated gap m= m=2 m=3 m=4 m=5 Estimated gap m= m=2 m=3 m=4 m=5 Estimated gap m= m=2 m=3 m=4 m= Iitial solutio Iitial solutio Iitial solutio (a) additive oise (b) multiplicative oise (c) oe-bit oise Figure 2: Estimated ERT gap for (+)-EA solvig Jump m,5 problem with or without oise. 4. O the Usefuless of Noise Hadlig Strategies 4.. Re-evaluatio There are aturally two fitess evaluatio optios for EAs [2, 2, 6, 9]: sigle-evaluatio we evaluate a solutio oce, ad use the evaluated fitess for this solutio i the future. re-evaluatio every time we access the fitess of a solutio by evaluatio. For example, for (+)-EA i Algorithm, if usig re-evaluatio, both f(x ) ad f(x) will be calculated ad recalculated i each iteratio; if usig sigle-evaluatio, oly f(x ) will be calculated ad the previous obtaied fitess f(x) will be reused. Ituitively, re-evaluatio ca smooth oise ad thus could be better for oisy optimizatios, but it also icreases the fitess evaluatio cost ad thus icreases the ruig time. Its usefuless was ot yet clear. Note that, the aalysis i the previous sectio assumes sigle-evaluatio. I this sectio, we take the I easiest problem, where oise has bee proved to have a strog egative effect i the previous sectio, as the represetative problem, ad compare these two optios for (+)-EA with mutatio probability solvig this problem uder oe-bit oise to show whether reevaluatio is useful. Note that for oe-bit oise, p cotrols the oise level, that is, oise becomes stroger as p gets larger, ad it is also the variable of the PNT. Theorem 5 The PNT of (+)-EA usig sigle-evaluatio with mutatio probability o I easiest problem is lower bouded by /Ω(poly()) ad upper bouded by /O(2 poly()), where poly() idicates ay polyomial of, with respect to oe-bit oise. The theorem is straightforwardly derived from the followig lemma. 5

16 Lemma 7 For (+)-EA usig sigle-evaluatio with mutatio probability o I easiest problem uder oe-bit oise, the expected ruig time is O( 2 + /( p )) ad Ω(p /(2 ( p ))). Proof. Let L deote the oisy fitess value f N (x) of the curret solutio x. Because (+)-EA does ot accept a solutio with a smaller fitess (i.e., the 4th step of Algorithm ) ad it does t re-evaluate the fitess of the curret solutio x, L (0 L ) will ever decrease. We first aalyze the expected steps util L icreases whe startig from L = i (deoted by E[[i]]), ad the sum up them to get a upper boud E[[i]] for the expected steps util L reaches the maximum value. For E[[i]], we aalyze the probability P that L icreases i two steps whe L = i, the E[[i]] = 2 P. Note that, oe-bit oise ca make L be x, x or x +, where x = x i is the umber of bits. Whe aalyzig the oisy fitess f N (x ) of the offsprig x i each step, we eed to first cosider bit-wise mutatio o x ad the oe radom bit flip for oise. Whe 0 < L <, x = L, L or L +. () For x = L, P L+ ( )( ) p L + L+ ( )( ) ( p ) L ( )( ) ( p ), sice it is sufficiet to flip oe 0 bit for mutatio ad oe 0 bit for oise i the first step, or flip oe 0 bit for mutatio ad o bit for oise i the first step ad flip oe 0 bit for mutatio ad o bit for oise i the secod step. (2) For x = L, P ( ) p L + L ( ) ( p ), sice it is sufficiet to flip o bit for mutatio ad oe 0 bit for oise, or flip oe 0 bit for mutatio ad o bit for oise i the first step. (3) For x = L +, P ( ) L ( p + p ), sice it is sufficiet to flip o bit for mutatio ad o bit or oe 0 bit for oise i the first step. Thus, for these three cases, we have P p ( L L )( ) + ( )2( ) ( p ) 2 L (p + ( p ) 2 ( L)( L ) 2 3( L)( L ) ) e 2 2 4e 2 2, where the is by ( ) e ad the 2 is by 0 p. L Whe L = 0, x = 0 or. By cosiderig case (2) ad (3), we ca get the same lower boud for P. Whe L = ad the optimal solutio has ot bee foud, x = 2 or. By cosiderig case () ad (2), we ca get P 3/(2e 2 2 ). Based o the above aalysis, we ca get that the expected steps util L = is at most 2 E[[i]] 2 ( L=0 4e 2 2 3( L)( L ) + 2e2 2 ), i.e., O( 2 ). 3 6

17 Whe L =, x = or (i.e., the optimal solutio has bee foud). If x =, the optimal solutio will be geerated ad accepted i oe step with probability ( ) ( p ) ( p) e, because it eeds to flip the uique 0 bit for mutatio ad o bit for oise. This implies that the expected steps for fidig the optimal solutio is at most Thus, we ca get the upper boud O( 2 + e ( p ). p ) for the expected ruig time of the whole process. The, we are to aalyze the lower boud. Assume that the iitial solutio x iit has umber of bits, i.e., x iit =. If the fitess of x iit is evaluated as, which happes with probability p, before fidig the optimal solutio, the solutio will always have umber of bits ad its fitess will always be. From the above aalysis, we kow that i such a situatio, the probability of geeratig ad acceptig the optimal solutio i oe step is ( ) ( p ) ( p). Thus, the expected ruig time for fidig the optimal solutio whe startig from x iit = is at least p ( p ) = p ( p. Because the iitial solutio is uiformly distributed over {0, ) }, the probability that the algorithm starts from x iit = is /2. Thus, we ca get the lower boud Ω( p 2 ( p ) Theorem 6 ) for the expected ruig time of the whole process. The PNT of (+)-EA usig re-evaluatio with mutatio probability o I easiest problem is Θ( log() ), with respect to oe-bit oise. The theorem is straightforwardly derived from the followig lemma. Lemma 8 ([0]) For (+)-EA usig re-evaluatio with mutatio probability o I easiest problem uder oe-bit oise, the expected ruig time is polyomial whe p O(log()/), ad the ruig time is polyomial with super-polyomially small probability whe p ω(log()/) Threshold Selectio Durig the process of evolutioary optimizatio, most of the improvemets i oe geeratio are small. Whe usig re-evaluatio, due to oisy fitess evaluatio, a cosiderable portio of these improvemets are ot real, where a worse solutio appears to have a better fitess ad the survives to replace the true better solutio which has a worse fitess. This may mislead the search directio of EAs, ad the slow dow the efficiecy of EAs or make EAs get trapped i the local optimal solutio, as observed i Sectio 4.. To deal with this problem, a selectio strategy for EAs hadlig oise was proposed [23]. threshold selectio a offsprig solutio will be accepted oly if its fitess is larger tha the paret solutio by at least a predefied threshold τ 0. 7

18 For example, for (+)-EA with threshold selectio as i Algorithm 3, its 4th step chages to be if f(x ) f(x) + τ rather tha if f(x ) f(x) i Algorithm. Such a strategy ca reduce the risk of acceptig a bad solutio due to oise. Although the good local performace (i.e., the progress of oe step) of EAs with threshold selectio has bee show o some problems [23, 6, 5], its usefuless for the global performace (i.e., the ruig time util fidig the optimal solutio) of EAs uder oise is ot yet clear. Algorithm 3 ((+)-EA with threshold selectio) Give pseudo-boolea fuctio f with solutio legth, ad a predefied threshold τ 0, it cosists of the followig steps:. x := radomly selected from {0, }. 2. Repeat util the termiatio coditio is met 3. x := flip each bit of x with probability p. 4. if f(x ) f(x) + τ 5. x := x. where p (0, 0.5) is the mutatio probability. I this sectio, we compare the ruig time of (+)-EA with ad without threshold selectio solvig I easiest problem uder oe-bit oise to show whether threshold selectio will be useful. Note that, the aalysis here assumes re-evaluatio. Algorithm 4 shows a radom walk o a graph. Lemma 9 gives a upper boud o the expected steps for a radom walk to visit each vertex of a graph at least oce, which will be used i the followig aalysis. Algorithm 4 (Radom Walk) Give a udirected coected graph G = (V, E) with vertex set V ad edge set E, it cosists of the followig steps:. start at a vertex v V. 2. Repeat util the termiatio coditio is met 3. choose a eighbor u of v i G uiformly at radom. 4. set v := u. Lemma 9 ([]) Give a udirected coected graph G = (V, E), the expected cover time of a radom walk o G is upper bouded by 2 E ( V ), where the cover time of a radom walk o G is the umber of steps util each vertex v V has bee visited at least oce. Theorem 7 The PNT of (+)-EA usig re-evaluatio with threshold selectio τ = ad mutatio probability o I easiest problem is ot less tha 2e, with respect to oe-bit oise. 8

19 The theorem ca be directly derived from the followig lemma. Lemma 0 For (+)-EA usig re-evaluatio with threshold selectio τ = ad mutatio probability o I easiest problem uder oe-bit oise, the expected ruig time is O( 3 ) whe p 2e. Proof. We deote the umber of oe bits of the curret solutio x by L (0 L ). Let P d deote the probability that the offsprig solutio x by bit-wise mutatio o x has L + d ( L d L) umber of oe bits, ad let P d deote the probability that the ext solutio after bit-wise mutatio ad selectio has L + d umber of oe bits. The, we aalyze P d. We cosider 0 L. Note that oe-bit oise ca chage the true fitess of a solutio by at most, i.e., f N (x) f(x). () Whe d 2, f N (x ) L+d+ L f N (x). Because a offsprig solutio will be accepted oly if f N (x ) f N (x) +, the offsprig solutio x will be discarded i this case, which implies that d 2 : P d = 0. (2) Whe d =, the offsprig solutio x will be accepted oly if f N (x ) = L f N (x) = L, the probability of which is p L+ Thus, P = P (p L p L+ ). p L, sice it eeds to flip oe 0 bit of x ad flip oe bit of x. (3) Whe d =, if f N (x) = L, the probability of which is p L, the offsprig solutio x will be accepted, sice f N (x ) L + = L > f N (x); if f N (x) = L f N (x ) L +, the probability of L which is ( p ) ( p + p ), x will be accepted; if f N (x) = L + f N (x ) = L + 2, the probability of which is p L p L, x will be accepted; otherwise, x will be discarded. Thus, P L = P (p + ( p L L )( p + p ) + p p L ). (4) Whe d 2, it is easy to see that P d > 0. Because we are to get the upper boud of the expected ruig time for fidig the optimal solutio for the first time, we pessimistically assume that d 2 : P d = 0. The, we compare P with P. P L P p L ( L ) p p L( L) e 2, where the secod iequality is by P L ( ) sice it is sufficiet to flip just oe 0 bit, ad the last iequality is by ( ) e. P = P (p L p L + ) L (p L p L + L L( L + ) L( L) ) p p e2 2 e 2, where the first iequality is by P L sice it is ecessary to flip at least oe bit, the secod iequality is by p L( L+) 2e, ad the last iequality is by 2 L. Thus, we have for all 0 L, P P. Because we are to get the upper boud of the expected ruig time for fidig, we ca pessimistically assume that P = P. The, we ca view the 9

20 evolutioary process as a radom walk o the path {0,, 2,..., }. We call a step that jumps to the eighbor state a relevat step. Thus, by Lemma 9, it eeds at most 2 2 expected relevat steps to fid. Because the probability of a relevat step is at least P P ( p ) 2 L ( ) ( 2e )2 ( 2e )2 /e, the expected ruig time for a relevat step is O(). Thus, the expected ruig time of (+)-EA with τ = o I easiest problem with p 2e is upper bouded by O(3 ). Theorem 8 The PNT of (+)-EA usig re-evaluatio with threshold selectio τ = 2 ad mutatio probability o I easiest problem is lower bouded by /Ω(poly()) ad upper bouded by /O(2 poly()), where poly() idicates ay polyomial of, with respect to oe-bit oise. The theorem ca be directly derived from the followig lemma. Lemma For (+)-EA usig re-evaluatio with threshold selectio τ = 2 ad mutatio probability o I easiest problem uder oe-bit oise, the expected ruig time is O( log /(p ( p ))) ad Ω( 2 /(2 p ( p ))). Proof. Let L (0 L ) deote the umber of oe bits of the curret solutio x. Here, a offsprig solutio x will be accepted oly if f N (x ) f N (x) 2. As i the proof of Lemma 0, we ca derive d : P d = 0; P ( L = P p (( p L L ) + p ) + ( p )(p ) ) ; d 2 : P d > 0. Thus, L will ever decrease i the evolutio process, ad it ca icrease i oe step with probability P d>0 > P L ( )( ) (( p )p ( ) + L( L ) p2 2 ) 2e ( p L )p. The, we ca get that the expected steps util L = (i.e., the optimal solutio is foud) is at most L=0 2e ( p )p ( L), i.e., O( log p ( p ) ). The, we are to aalyze the lower boud. Assume that the iitial solutio x iit has umber of bits. Before fidig the optimal solutio, the solutio x i the populatio will always satisfy x = because d : P d = 0. The optimal solutio (i.e., x = ) will be foud i oe step with probability P = P p ( p )( ) = ( )( ) p ( p )( ) p( p) e. Thus, the expected steps for fidig the optimal solutio whe startig from x iit = is at least e p ( p ). By the uiform distributio of the iitial solutio, the probability that x iit = is /2. Thus, we ca get the lower boud Ω( 2 2 p ( p ) ) for the expected ruig time of the whole process. 20

21 4.3. Smooth Threshold Selectio We propose the smooth threshold selectio as i Defiitio 8, which modifies the origial threshold selectio by chagig the hard threshold value to a smooth oe. We are to show that, by such a small modificatio, the PNT of (+)-EA o I easiest problem is improved to, which meas that the expected ruig time of (+)-EA is always polyomial disregard the oe-bit oise level. Defiitio 8 (Smooth Threshold Selectio) Let δ be the gap betwee the fitess of the offsprig solutio x ad the paret solutio x, i.e., δ = f(x ) f(x). The, the selectio process will behave as follows: () if δ 0, x will be rejected; (2) if δ =, x will be accepted with probability 5 ; (3) if δ >, x will be accepted. Theorem 9 The PNT of (+)-EA usig re-evaluatio with smooth threshold selectio ad mutatio probability o I easiest problem is, with respect to oe-bit oise. Proof. We first aalyze P d as that aalyzed i the proof of Lemma 0. The oly differece is that whe the fitess gap betwee the offsprig ad the paret solutio is, the offsprig solutio will be accepted with probability 5 here, while it will be always accepted i the proof of Lemma 0. Thus, for smooth threshold selectio, we ca similarly derive d 2 : P d = 0; P L = P (p p L + P ( L = P p (p L + L + p p d 2 : P d > 0. L ) 5 ; 5 + ( p L ) + p ) + ( p )(( p ) ) ; p L ) Note that L (0 L ) deotes the umber of oe bits of the curret solutio x. Our goal is to reach L =. If startig from L =, L will reach i oe step with probability P L P (p p L + L ( L ) (p p 5e 2 ( 5e ( p )( p ) L + 5 ) 5 + ( p )( p ) 5 ) p2 + ( p ) 2 ) (by L = ad ( ) e ) 2 Ω( 2 ). (by 0 p ) Thus, for reachig L =, we eed to reach L = for O( 2 ) times i expectatio. 2

22 The, we aalyze the expected ruig time util L =. I this process, we ca pessimistically assume that L = will ever be reached, because our fial goal is to get the upper boud o the expected ruig time for reachig L =. For 0 L 2, we have P P L P (p p L ) L P (p p L+ ) 5 5( L)( L ) el( L + ) L = 5( L ) e( + 2 L ) >. ( ) (p L p L ) L (p L p L+ ) 5 Agai, we ca pessimistically assume that P = P ad d 2, P d = 0, because we are to get the upper boud o the expected ruig time util L =. The, we ca view the evolutioary process for reachig L = as a radom walk o the path {0,, 2,..., }. We call a step that jumps to the eighbor state a relevat step. Thus, by Lemma 9, it eeds at most 2( ) 2 expected relevat steps to reach L =. Because the probability of a relevat step is at least P P (( p )( p ) 5 + p L L 5e 2 (( p ) 2 + p 2 ( L)( L ) 2 ) 2 5e 2 (( p ) p2 ) 2 5e 2 p L , the expected ruig time for a relevat step is O( 4 ). The, the expected ruig time for reachig L = is O( 6 ). 5 ) Thus, the expected ruig time of the whole optimizatio process is O( 8 ) for ay p [0, ], ad the this theorem holds. We draw a ituitive uderstadig from the proof of Theorem 9 that why the smooth threshold selectio ca be better tha the origial threshold selectios. By chagig the hard threshold to be a smooth threshold, it ca ot oly make the probability of acceptig a false better solutio i oe step small eough, i.e. P P, but also make the probability of producig progress i oe step large eough, i.e., P is ot small. 5. Discussios ad Coclusios This paper studies theoretical issues of oisy optimizatio by evolutioary algorithms. First, we discover that a optimizatio problem may become easier istead of harder i a oisy eviromet. We the derive a sufficiet coditio uder which oise makes optimizatio easier or harder. By fillig this coditio, we have show that for (+λ)-ea, oise makes the optimizatio 22

23 o the hardest ad the easiest case i the pseudo-boolea fuctio class easier ad harder, respectively. We also hypothesize that we eed to take care of oise oly whe the optimizatio problem is moderately or less complex. Experimets o the Jump m, problem, which has a adjustable difficulty parameter, supported our hypothesis. I problems where the oise has a egative effect, we the study the usefuless of two commoly employed oise-hadlig strategies, re-evaluatio ad threshold selectio. The study takes the easiest case i the pseudo-boolea fuctio class as the represetative problem, where the oise sigificatly harms the expected ruig time of the (+)-EA. We use the polyomial oise tolerace (PNT) level as the performace measure, ad aalyzed the PNT of each EA. The re-evaluatio strategy seems to be a reasoable method for reducig radom oise. However, we derive that the (+)-EA with sigle-evaluatio has a PNT lower boud /Ω(poly()) from Theorem 5 which is close to, whilst the (+)-EA with re-evaluatio has the PNT Θ(log()/) which ca be quite close to zero as is large. It is surprise to see that the re-evaluatio strategy leads to a much worse oise tolerace tha that without ay oise hadlig method. The re-evaluatio with threshold selectio strategy has a better PNT comparig with the re-evaluatio aloe. Whe the threshold is, we derive a PNT lower boud 2e from Theorem 7, ad whe the threshold is 2, we obtai /Ω(poly()) from Theorem 8. The improvemet from re-evaluatio aloe could be explaied as that the threshold selectio filters out fake progresses that caused by the oise. However, it still showed o improvemets from the (+)-EA without ay oise hadlig method. We the proposed the smooth threshold selectio, which acts like the threshold selectio with threshold 2 but accepts progresses with a probability. We proved that the (+)-EA with the smooth threshold selectio has the PNT from Theorem 9, which exceeds that of (+)-EA without ay oise hadlig method. Our explaatio is that, like the origial threshold selectio, the proposed oe filters out fake progresses, while it also keep some chaces to accept real progresses. Although the ivestigated EAs ad problems i this paper are simple ad specifically used for the theoretical aalysis of EAs, the aalysis still disclosed couter-ituitive results ad, particularly, demostrated that theoretical ivestigatio is essetial i desigig better oise hadlig strategies. We are optimistic that our fidigs may be helpful for practical uses of EAs, which will be studied i the future. 6. Ackowledgemets to be added... 23

24 Refereces [] R. Aleliuas, R. Karp, R. Lipto, L. Lovasz, ad C. Rackoff. Radom walks, uiversal traversal sequeces, ad the complexity of maze problems. I Proceedigs of the 20th Aual Symposium o Foudatios of Computer Sciece (FOCS 79), pages , Sa Jua, Puerto Rico, 979. [2] D. V. Arold ad H.-G. Beyer. Local performace of the (+)-ES i a oisy eviromet. IEEE Trasactios o Evolutioary Computatio, 6():30 4, [3] D. V. Arold ad H.-G. Beyer. A compariso of evolutio strategies with other direct search methods i the presece of oise. Computatioal Optimizatio ad Applicatios, 24():35 59, [4] T. Bäck. Evolutioary Algorithms i Theory ad Practice: Evolutio Strategies, Evolutioary Programmig, Geetic Algorithms. Oxford Uiversity Press, Oxford, UK, 996. [5] T. Bartz-Beielstei. New experimetalism applied to evolutioary computatio. PhD thesis, Uiversity of Dortmud, [6] T. Beielstei ad S. Marko. Threshold selectio, hypothesis tests, ad DOE methods. I Proceedigs of the IEEE Cogress o Evolutioary Computatio (CEC 02), pages , Hoolulu, HI, [7] H.-G. Beyer. Evolutioary algorithms i oisy eviromets: theoretical issues ad guidelies for practice. Computer Methods i Applied Mechaics ad Egieerig, 86(2): , [8] S.-J. Chag, H.-S. Hou, ad Y.-K. Su. Automated passive filter sythesis usig a ovel tree represetatio ad geetic programmig. IEEE Trasactios o Evolutioary Computatio, 0(): 93 00, [9] Y. Chag ad S. Che. A ew query reweightig method for documet retrieval based o geetic algorithms. IEEE Trasactios o Evolutioary Computatio, 0(5):67 622, [0] S. Droste. Aalysis of the (+) EA for a oisy OeMax. I Proceedigs of the 6th ACM Aual Coferece o Geetic ad Evolutioary Computatio (GECCO 04), pages , Seattle, WA, [] S. Droste, T. Jase, ad I. Wegeer. A rigorous complexity aalysis of the (+) evolutioary algorithm for liear fuctios with Boolea iputs. Evolutioary Computatio, 6(2):85 96, 998. [2] S. Droste, T. Jase, ad I. Wegeer. O the aalysis of the (+) evolutioary algorithm. Theoretical Computer Sciece, 276(-2):5 8,