Chapter Introduction

Transcription

1 Unversty of Pretora etd Wle, D N (5) Chapter Objectvty of the PSOA. Introducton In scence, most physcal phenomena are nvarant. It s fundamental that mathematcal representaton of these phenomena reflects ths nvarance. Ths fundamental requrement s nown as objectvty, frame-ndfference or observer ndependence, and s well nown n classcal mechancs [5]. For objectvty, the descrpton of some quantty has to be nvarant under pure translatons, as well as pure rotatons. Objectvty or observer ndependence s also hghly desrable (almost essental) n optmzaton procedures, to reflect the nvarance of the physcal processes that are optmzed. Robust optmzaton procedures and algorthms should defntely be nvarant. In classcal gradent based optmzaton, the gradent vector (or some conjugate drecton to the gradent), ndcates some drecton of mprovement, even f ths drecton s not optmal. Ths accounts for the reference frame; classcal optmzaton s (usually) frame nvarant. In modern (stochastc) optmzaton procedures, the requrement of observer ndependence s equally essental. These algorthms nclude genetc programmng [7], genetc algorthms [8], evolutonary strateges [9], dfferental evoluton [] and the partcle swarm optmzaton algorthm (PSOA) [, ]. For the genetc algorthm (GA), Salomon [5, 55] demonstrated the lac of rotatonal nvarance of the algorthm. He showed that the GA s performance at low mutaton rates s sgnfcantly nfluenced by the frame of reference used to pose a problem. The PSOA was ntroduced by Kennedy and Eberhart [] as a gradent free stochastc optmzaton algorthm. The fundamental prncple behnd the PSOA s the evolutonary advantages that the sharng of nformaton offers. Ths s often nown as collaboratve searchng. The PSOA s qute smple: At frst, a swarm of p partcles s randomly deployed n an n-dmensonal desgn doman. The partcles then update ther postons n the desgn doman over unt tme ncrements usng a smple stochastc rule, nown as the velocty rule. The qualty of each partcle s poston at each teraton s then evaluated usng the objectve or cost functon. Each partcle s cogntve memory allows t to remember t s own best cost functon 7

2 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 8 value, wth assocated poston, over tme. Importantly, the socal nteracton and awareness of the partcles allow them to also remember the best cost functon value the swarm tself found over tme. In Chapter t was shown that mplementaton subtletes due to ambguous notaton have resulted n two dstnctly dfferent mplementatons of the PSOA. Whle ths does not repute negatvely on the ngenuty of the dea of Kennedy and Eberhart, dscernng between these two mplementatons s of crucal mportance. The behavor of the respectve mplementatons s maredly dfferent, although they only dffer n the formulaton of the velocty updatng rule. In fact, the dfferences are merely due to subtle dfferences n the ntroducton of randomness nto the algorthm. In ths chapter, the objectvty of the PSOA s nvestgated. It s shown that the frst formulaton PSOAF s objectve, combned wth the dsadvantage that the partcle trajectores collapse to lne searches. It s then show that the second formulaton PSOAF s not objectve, although t has the advantage that the partcle trajectores are n-dmensonal space fllng. A novel formulaton that s both objectve and dverse,.e. the algorthm generates partcle trajectores that are space fllng, s then presented.. Notes on the nvestgaton The nvestgaton nto the objectvty of the PSOA s started by defnng the nstantaneous search doman of a partcle, vz. the doman to whch the search of a partcle at teraton s restrcted as dscussed n Chapter. From Eqs. (.) and (.), t s observed that the nstantaneous search doman s composed from a determnstc contrbuton gven by (x + wv ), and a stochastc contrbuton due to ν. The stochastc doman s bounded, and has an assocated probablty dstrbuton, due to the random scalng wth fnte scalars. In order to nvestgate the objectvty of the stochastc contrbuton ν of the nstantaneous search doman, Monte Carlo smulatons [56] are used. These are conducted for dfferent values of p, pg and x. Scatter plots are constructed to defne the doman of possble stochastc vectors ν by generatng nstances of ν. In all nvestgatons c = c =.. Formulaton (PSOAF) For PSOAF, the stochastc vector ν s gven by ν = c r (p x ) + c r (pg x ), (.) where r and r represent two unform real random scalars between and, whch are updated at every teraton, and for each partcle n the swarm. The random numbers r and r ndependently scale only the magntudes of the cogntve and socal vectors, respectvely gven by c (p x ) and c (p g x ). The cogntve vector c (p x ) and the socal vector c (p g x ) can be anythng from normal to parallel w.r.t. each other. When the cogntve and socal vectors are not parallel, Eq. (.) may be nterpreted as the vector equaton of a bounded plane P n n-dmensonal space. The bounded plane s then translated n

3 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 9 x + () P (p x ) (p g x ) x + wv x + () Fgure.: Parttonng the poston vector x + nto a determnstc contrbuton (x + wv ), and a stochastc contrbuton (ν P ), for c = c =. x + () L (p x ) (p g x ) x + wv x + () Fgure.: Parttonng the poston vector x + nto a determnstc contrbuton (x + wv ), and a stochastc contrbuton (ν L ), for c = c =. n-dmensonal space by the addton of x and wv, as depcted n Fgure.. Whenever the cogntve and socal vectors c (p x ) and c (p g x ) are parallel, Eq. (.) may be nterpreted as the vector equaton of a bounded lne L n n-dmensonal space. Agan, the bounded lne s translated n n-dmensonal space by the addton of x and wv, as depcted n Fgure.. The ntrnsc propertes of a vector are ts magntude and drecton; these exst ndependent of a reference frame [57]. In PSOAF, only the vector magntudes (whch are nvarant) are randomly scaled. Also, snce the vectors c (p x ) and c (p g x ) are constructed through the subtracton of two vectors, they are also translatonally nvarant. Both crtera for observer ndependence are met, hence PSOAF s objectve... PSOAF: Investgaton of the nstantaneous search doman Objectvty of PSOAF s now llustrated by conductng Monte Carlo smulatons. Smlar smulatons wll be conducted for the algorthmc formulatons n sectons to come.

4 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA () ν ν+().5 (pg x).5 (pg x ).5.5 (p x) ν+().5 (p x) ν+().5.5 Fgure.: PSOAF: Scatter plot of stochastc vectors ν, generated usng Monte possble Carlo smulatons, wth a) p x = [ ] and pg x = [ ] and b) p x = [ ] and pg x = [ ]. Each pont represents the end pont of a stochastc vector wth orgn at [ ]. Frst, a study s conducted for non-parallel cogntve and socal vectors c (p x ) and c (pg x ). In Fgure., the vectors p x and pg x are respectvely gven by [ ] and [ ]. A scatter plot yelds the plane P, wth c and c merely scalng P. A scatter plot s then constructed after rotatngthe vectors pg x and p x 5 clocwse, as depcted n Fgure.. Hence p x and pg x are respectvely gven by [ ] and [ ]. From Fgure., t s clear that the doman remans a bounded plane P, whch s merely rotated 5 clocwse. It also follows from random varable theory that the probablty dstrbuton over the doman P s unform [5], as llustrated n Fgures. and.. Secondly, a smlar study s conducted for parallel cogntve and socal vectors c (p x ) and c (pg x ), as depcted n Fgure.. In Fgure., the parallel vectors p x and pg x are respectvely gven by [ ] and [ ]. The doman s a bounded lne L wth c and c merely scalng the length of L. Agan a scatter plot s constructed after rotatng pg x and p x 5 clocwse, as depcted n Fgure.. Now, p x and pg x are respectvely gven by [ ] and [ ]. As shown n Fgure., the bounded lne L s merely rotated. It follows from random varable theory that the probablty dstrbuton over the bounded lne s tr-lnear [5], for dfferent vector lengths p x and pg x. As dscussed earler and graphcally demonstrated here, PSOAF s objectve. A rotaton of the vectors p, pg and x merely results n a rotaton of the stochastc domans, P and L. Ths

5 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA ν+().5 () ν.5 (pg x ).5.5 (p x ).5 (p x ) (pg x ) ν+() ν+() 5 6 Fgure.: PSOAF: Scatter plot of possble stochastc vectors ν, generated usng Monte g Carlo smulatons, wth a) p x = [ ] and p x = [ ] and b) p x = [ ] and pg x = [ ]. follows snce only the magntude of the cogntve and socal vectors are scaled n PSOAF.. Formulaton (PSOAF) The stochastc vector ν of PSOAF s gven by ν = c r (p x ) + c r (pg x ), (.) where the operator ndcates component by component multplcaton between two vectors. Hence the random vectors r m are gven by r m = (ρ, ρ,, ρn ), m =,, (.) wth ρl, l =,,, n unform random numbers between and. Eq. (.) s no longer a vector equaton of a bounded plane P, snce every non-zero component of (p x ) and (pg x ) s ndependently scaled. As a result, the doman of possble stochastc vectors s generalzed to n-dmensonal space S. However, snce the components of a vector are gven wth respect to a specfc reference frame, PSOAF s rotatonally varant. (Although PSOAF s of course translatonally nvarant.) Nevertheless, PSOAF s observer dependent, snce only one of the two crtera of objectvty s met... PSOAF: Investgaton of the nstantaneous search doman The observer dependence of PSOAF s now quantfed, usng Monte Carlo smulatons, smlar to those n Secton...

6 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA g (p x) ().5 ν ν+() ν+().5.5 (p x).5 (p x).5 (pg x ) ν+().5.5 Fgure.5: PSOAF: Scatter plot of possble stochastc vectors ν, generated usng Monte Carlo smulatons wth a) p x = [ ] and pg x = [ ] and b) p x = [ ] and pg x = [ ]. As before, the study s conducted for non-parallel cogntve and socal vectors c (p x ) and c (pg x ). Fgure.5 depcts that the doman s an n-dmensonal space S (wth n = n ths case), wth c and c merely scalng S. It s also clear that the probablty dstrbuton over S s non-unform. The scatter plot after rotatng the vectors pg x and p x 5 clocwse, s depcted n Fgure.5. It s clear that the doman changes after rotaton of the vectors. However, the doman remans an n-dmensonal space S, but the sze of, and the probablty dstrbuton over, the doman depends on the orentaton w.r.t. the Cartesan coordnate axs. The study s repeated for parallel cogntve and socal vectors c (p x ) and c (pg x ), as depcted n Fgure.6. The doman s stll generalzed to n-dmensonal space S wth c and c merely scalng the sze of S. The scatter plot after rotatng pg x and p x 5 clocwse, s depcted n Fgure.6. It s clear that the doman changes sgnfcantly after rotaton of the vectors. In fact, the doman collapses to a bounded lne L, snce both vectors are parallel to one of the Cartesan bass vectors. As dscussed earler and graphcally demonstrated here, PSOAF s observer dependent. A rotaton of the vectors (p x ) and (pg x ) results n the sze of, and the probablty dstrbuton over, the stochastc doman to change. Ths follows snce PSOAF scales the components of the cogntve and socal vectors. Snce the components of a vector are observer dependent, PSOAF s also observer dependent. However, the advantage of PSOAF s that the partcle trajectores reman space fllng n ndmensonal space as shown n Chapter. The result s that dversty n partcle trajectores are mantaned.

7 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA ν+().5 () ν.5 g (p x).5.5 (p x ).5 ν+() 5 (p x ) (pg x ) 6.5 ν+() 5 6 Fgure.6: PSOAF: Scatter plot of possble stochastc vectors ν, generated usng Monte g Carlo smulatons wth a) p x = [ ] and p x = [ ] and b) p x = [ ] and pg x = [ ]..5 Novel Formulaton: PSOAF* As dscussed n Secton., PSOAF s objectve, although the partcle trajectores collapse to lnes. (The advantage of dverse (n-dmensonal) partcle search trajectores are quantfed n Chapter.) On the other hand, PSOAF allows for partcles to have dverse search trajectores, but unfortunately ths comes at the cost of sacrfcng objectvty. An mplementaton of the PSOA s now presented that allows for dverse partcle search trajectores, whle retanng objectvty. Based on PSOAF, the novel, dverse mplementaton, s denoted PSOAF*. In PSOAF*, the vector magntudes are scaled, and the vector drectons of (p x ) and (pg x ) perturbed, by multplyng each of the above vectors wth an ndependent random rotaton matrx. The random rotaton matrces are constructed anew for each partcle and for every teraton, hence ν = c r Q (p x ) + c r Q (pg x ), (.) wth each Ql, l =,, a random rotaton matrx of dmenson n n. Q s a proper orthogonal matrx (wth determnant ). Numerous methods are avalable to construct rotaton matrces (e.g. see the approach of Salomon [5]. Constructng n n matrces usng Salomon s routne s however computatonally expensve, snce (n )(n ) matrx-matrx multplcatons are requred.) As a computatonally vable alternatve, the exponental map s used [58]. There are agan numerous ways to construct exponental maps. The smple seres method s selected [58]. The general

8 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA seres expanson of an exponental map s gven by Q = I + W + W W + 6 W W W +, (.5) where I s the dentty matrx and W s a sew matrx. The random sew matrx W s constructed as follows: W = απ 8 (A AT ), (.6) wth A an n n random matrx wth each entry a unform random number between.5 and.5. α s a real scalng factor and superscrpt T denotes the matrx transpose. The author selects to construct the exponental map Q for small perturbatons, usng only the frst two terms of a truncated seres method,.e. Q = I + W. (.7) Ths s the lnear approxmaton to a rotaton matrx, and s vald for small perturbatons, snce the entres of the hgher order terms are close to zero. The advantage of the smplfcaton s that the number of matrx-matrx multplcatons s zero. (It s mportant to note that the varable bounds defnng D should be normalzed, such that the boundary ranges are equal.).5. PSOAF*: Investgaton of the nstantaneous search doman As before, the objectvty of PSOAF* s quantfed usng Monte Carlo smulatons. In dmensons, α = 5 s selected. (Although ths s not small, ths serves to clearly llustrate the proposed concept). Agan, the study for non-parallel cogntve and socal vectors s conducted, as depcted n Fgure.7. The doman generalzes to n-dmensonal space S, wth c and c scalng S. The scatter plot after rotatng the vectors ( p g ( ) x and p x) 5 clocwse s depcted n Fgure.7. Clearly, the doman remans generalzed to n-dmensonal space S, rotated 5 clocwse. The probablty dstrbuton over the doman S s non-unform. Secondly, the study for parallel cogntve and socal vectors c (p x ) and c (p g x ) s conducted. Agan the doman generalzes to n-dmensonal space S, wth c and c merely scalng the doman. A scatter plot after rotatng ( p g x ) and ( p x ) 5 clocwse s constructed, as depcted n Fgure.8. Evdently, the n-dmensonal space S s merely rotated, and the probablty dstrbuton over the doman s non-unform. As dscussed earler and graphcally demonstrated here, PSOAF* s an objectve formulaton. A rotaton of the vectors (p x ) and (pg x ) merely results n a rotaton of the stochastc doman S. The drawbac of PSOAF s overcome n PSOAF*, where n addton to scalng the vector magntudes, the vectors are drectonally perturbed. The magntudes and drectons of (p x ) and

9 Unversty of Pretora etd Wle, D N (5) ν+().5 ν+() CHAPTER. OBJECTIVITY OF THE PSOA.5.5 g ( p x) ( pg x).5 ( p x).5 ν+() ( p x) ν () () ν ν+() Fgure.7: PSOAF*: Scatter plot of possble stochastc vectors ν, generated usng Monte g Carlo smulatons, wth a) p x = [ ] and p x = [ ] and b) p x = [ ] and pg x = [ ]..5.5 ( p.5 ( p x) ( pg x ) ( pg x).5 x) ν+() 5 ν+() 5 generated usng Fgure.8: PSOAF*: Scatter plot of nstances of the stochastc vectors ν, g Monte Carlo smulatons, wth a) p x = [ ] and p x = [ ] and b) p x = [ ] and pg x = [ ].

10 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 6 (p g x ) are used to only ndcate potental mprovement, thereby placng only some fath n both drecton and step sze. Ths s n contrast to PSOAF, where absolute fath s placed n the drectons prescrbed by (p x ) and (pg x ), whle only some fath s placed on step sze. (Incdentally, termnaton occurs when p, pg and x converge on the same pont n n-dmensonal space, combned wth wv.).6 Numercal experments An emprcal study to quantfy the (lac of) objectvty of the three dscussed mplementatons of the PSOA s now performed. A synchronous updatng method s used []. Real varables are mplemented usng double-precson floatng-pont arthmetc. For ths study the algorthm parameters are c = c =, the swarm sze s p = partcles and the computatons are performed for varous constant nerta factors w. Intal veloctes are assumed to equal. In PSOAF*, α = s smply selected. (The author does not see an optmal value for α, but merely wshes to llustrate the effects of perturbng the vector drectons.) Furthermore, no boundary or velocty restrctons are mplemented. Each run conssts of functon evaluatons ( teratons). All results presented are averaged over runs. In the study the followng fve test functons are used: ) The Rosenbroc functon (unmodal, f ): f (x) = n ( ( ) x x ( ) ) + x. = ) The Quadrc functon (unmodal, f ): f (x) = ) The Acley functon (multmodal, f ): f (x) = exp ( exp ( ( n ) x j. = j= ). n n = x ) n n = cos(πx ) + + e. v) The generalzed Rastrgn functon (multmodal, f ): n ( ) f (x) = x cos(πx ) +. v) Fnally, the generalzed Grewan functon (multmodal, f ): f (x) = n n ( x x ) cos +. = = =

11 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 7 Table.: Test functon parameters Functon n doman f ±.8 f ±. f ±. f ± 5. f ± 6. The parameters used n the study are gven n Table.. The doman column represents the range of each dmenson of the desgn varables; the test functon domans are symmetrcal about. n all dmensons. The multmodal functons f and f are decomposable [5], vz. the desgn varables are uncoupled. Ths mples that once an optmal value for a gven desgn varable s obtaned, t remans optmal, ndependent of the other desgn varables. Ths s smlar to optmzng n -dmensonal optmzaton problems, nstead of n-dmensonal coupled optmzaton problem. The test set s therefore studed n the unrotated or decomposable reference frame f(x), as well as n an arbtrary rotated reference frame f(rx), n whch the desgn varables are coupled [55]. Here, R s a random, proper orthogonal transformaton matrx, constructed as n [5]. The transformaton matrx results n a pure rotaton of each test functon. For each of the ndependent runs, a new random rotaton matrx R s constructed, to ensure that there s no bas toward any partcular reference frame..7 Dscusson of Results Depcted n Fgures.9,.,.,. and. are the mean objectve functon values after 5 functon evaluatons (or teratons) averaged over runs for both the unrotated and rotated functons. The rotatonal nvarance of PSOAF and PSOAF* are evdent from Fgures.9,.,.,. and.. The poor performance of PSOAF drectly results from the partcle trajectores collapsng to lnes as shown n Chapter. There s a sgnfcant mproved performance for all the test functons wth PSOAF*, due to the scalng of the vector magntudes and perturbaton of the vector drectons. The rotatonal varance of PSOAF s evdent from Fgures.9,.,.,. and.. There s a severe performance loss for some of the rotated functons compared to the unrotated functons. Two functons result n smlar performance for the rotated and unrotated functons, namely the Quadrc functon f, and the Grewan functon, f. The Quadrc and Grewan functons are almost nsenstve to rotaton. (The Grewan functon s a sphercal functon on whch snusodal nose s mposed. Hence ths functon s artfcally ndfferent to rotaton, snce many local mnma appear, rrespectve of rotaton.)

12 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 8 Average objectve functon value PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Average objectve functon value PSOAF unrotated PSOAF rotated Inerta constant ω Inerta constant ω Fgure.9: Average functon value obtaned wth a) PSOAF and PSOAF*, and b) PSOAF after 5 functon evaluatons ( teratons) averaged over runs on the rotated and unrotated Rosenbroc test functon f. Average objectve functon value PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Average objectve functon value PSOAF unrotated PSOAF rotated Inerta constant ω Inerta constant ω Fgure.: Average functon value obtaned wth a) PSOAF and PSOAF*, and b) PSOAF after 5 functon evaluatons ( teratons) averaged over runs on the rotated and unrotated Quadrc test functon f.

13 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA 9 Average objectve functon value PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Inerta constant ω Average objectve functon value PSOAF unrotated PSOAF rotated Inerta constant ω Fgure.: Average functon value obtaned wth a) PSOAF and PSOAF*, and b) PSOAF after 5 functon evaluatons ( teratons) averaged over runs on the rotated and unrotated Acley test functon f. Average objectve functon value PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Average objectve functon value PSOAF unrotated PSOAF rotated Inerta constant ω Inerta constant ω Fgure.: Average functon value obtaned wth a) PSOAF and PSOAF*, and b) PSOAF after 5 functon evaluatons ( teratons) averaged over runs on the rotated and unrotated Rastrgn test functon f.

14 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA Average objectve functon value PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Average objectve functon value PSOAF unrotated PSOAF rotated Inerta constant ω Inerta constant ω Fgure.: Average functon value obtaned wth a) PSOAF and PSOAF*, and b) PSOAF after 5 functon evaluatons ( teratons) averaged over runs on the rotated and unrotated Grewan test functon f. Inadvertently, ths also suggest that non-sphercal unmodal test functons should be used to evaluate objectvty. The two unmodal functons, namely the Rosenbroc functon f and the Quadrc functon f, are of some nterest, snce they ndcate the ablty of an algorthm to search wthn a local basn. The performance of PSOAF* s sgnfcantly better than PSOAF for both functons, for both the rotated and unrotated test functons. PSOAF demonstrates a severe performance loss for the Rosenbroc functon, for the rotated functon compared to the unrotated functon. (Note the scale of the graphs n Fgure.9.) The performance dfference between PSOAF and PSOAF* for the unmodal Quadrc test functon f, s depcted n Fgure.. Fgure. depcts the mean functon value convergence hstory of PSOAF (wth w =.8), PSOAF (wth w =.) and PSOAF* (wth w =.5 and α = ) over 5 teratons. The values for w are optmal for each algorthm, but no attempt was made to optmze α. Of the three formulatons, t s clear that PSOAF* s computatonally the most effectve on the Quadrc test functon. For the multmodal functons, PSOAF demonstrates notable performance loss. See for example the Acley functon f, and the Rastrgn functon f. In contrast, the performance of PSOAF* s comparable to the best obtaned wth PSOAF, wth no performance loss due to rotaton. For the sae of clarty, an overvew of the performances of PSOAF, PSOAF and PSOAF* s gven n Table.. The table summarzes the best functon values obtaned, together wth the nerta factor at whch the best functon value s obtaned after 5 functon evaluatons ( teratons). The results for both the unrotated and rotated test functons are gven.

15 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA Table.: Constant nerta factor at whch the best average objectve functon value s obtaned for the unrotated test functons. The accompanyng average objectve functon value for rotated test functons s also presented. PSOAF w f unrotated f best avg f rotated f best avg f f f f f w PSOAF f unrotated f best avg f rotated f best avg f f f f f.6.5. w PSOAF* (α = ) f unrotated f best avg f rotated f best avg f f.5..8 f f f.6..

16 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA Average objectve functon value 5 5 PSOAF unrotated PSOAF rotated PSOAF unrotated PSOAF rotated PSOAF* (α=) unrotated PSOAF* (α=) rotated Number of teratons Fgure.: Mean functon value hstory plot averaged over runs on the rotated and unrotated Quadrc test functon f wth PSOAF (wth w =.8), PSOAF (wth w =.) and PSOAF* (wth w =.5 and α = )..8 Comments on PSOAF*.8. On nvarance It s now shown that PSOAF* s not strctly rotatonally nvarant, but only n a stochastc sense. Consder an arbtrary vector, expressed n two dfferent reference frames, by respectvely y and y. The two reference frames are related by a pure rotaton M, hence where Orth + ndcates the space of proper orthogonal matrces. y = My, M Orth +, (.8) Now apply two ndependent drectonal perturbatons (rotatons) Q Orth + and Q Orth + to y and y respectvely. The vectors ŷ and ŷ then obtaned, are respectvely gven by and ŷ = Qy, (.9) ŷ = Q y. (.) Strct determnstc rotatonal nvarance requres a one-to-one mappng of the perturbed vectors n ether reference frame. Hence ŷ = Mŷ, M Orth + (.) By substtutng Eqs. (.8), (.9) and (.) nto Eq. (.), the followng s obtaned Eq. (.) s rewrtten as Q My = MQy, M Orth +. (.) (Q M MQ)y =, M Orth +. (.)

17 Unversty of Pretora etd Wle, D N (5) CHAPTER. OBJECTIVITY OF THE PSOA Snce Eq. (.) has to hold for any arbtrary vector y, t follows that Q M = MQ, M Orth +. (.) The unque soluton to Eq. (.) s that both Q and Q are the second-order sotropc tensor,.e. Q = Q = I. (.5) The foregong mples that a strct enforcement of rotatonal nvarance results n Q l = I, l =,. In other words, PSOAF* reduces to PSOAF. However, snce the PSOA s a stochastc algorthm, t s adequate to satsfy Eq. (.5) n an average sense only. In order to satsfy Q = Q = I n a stochastc sense, t s suffcent to requre that mean(q ) = mean(q) = I, f the probablty dstrbutons of Q and Q are chosen equal over dentcal domans..8. Implementatonal ssues of PSOAF* Further to the mplementaton n Secton.5, numerous strateges exst to acheve ndependent drectonal perturbaton. An obvous, computatonally nexpensve possblty s to randomly perturb each component of the unt vectors (p x )/ (p x ) and (pg x )/ (pg x ) ; the vectors (p x ) and (p g x ) are then reconstructed from the normalzaton of the perturbed vectors. (Although ths maes a rgorous mathematcal analyss of the algorthm dffcult.) In the mplementaton, n updatng Q, strateges to lmt the computatonal expense assocated wth matrx multplcatons and the generaton of random numbers may also be mplemented. For example, multplyng the sum of c (p x ) and c (p g x ) by a sngle random rotaton matrx, reduces the number of matrx multplcatons by half. However, depcted n Fgure.5 s the dfference n nstantaneous search doman that results when ndependent rotaton matrces Q Q are used, as opposed to dentcal rotaton matrces Q = Q. To reduce the computatonal effort even further, the vectors c (p x ) and c (p g x ) of all the partcles can be drectonally perturbed by the same ndependent rotaton matrces, vz. Q l = Q l, l =, and =,,, p..8. Alternatves to PSOAF* Fnally, there are of course numerous methods to ntroduce dversty nto PSOAF, as opposed to the proposed opton of ndependent drectonal perturbaton. Only a sngle alternatve s mentoned here, namely an ncrease n the socal awareness of the partcles. In turn, ths may for example be effected by ncreasng the number of partcles that contrbute to Eq. (.) [5, ]. (One may of course acheve n-dmensonal searches, f the number of partcles p n, unless the partcle trajectores are parallel.) Addtonal nformaton about the objectve functon s then also used n the searches of any partcle.

18 Unversty of Pretora etd Wle, D N (5) ().5 + ν ν + () CHAPTER. OBJECTIVITY OF THE PSOA ( p x ) g ( p x).5 ( p x) g ( p x).5.5 ν+() ν+() Fgure.5: Scatter plot of possble stochastc vectors ν, generated usng Monte Carlo smulatons, wth (p x ) = [ ] and (p x ) = [ ] usng a) dentcal rotaton matrces Q = Q and b) ndependent rotaton matrces Q 6= Q..9 Closure It s shown that PSOAF s objectve, but t demonstrates an overall poor performance, due to the partcle trajectores collapsng to lnes. Ths s a drect result of only scalng the magntude of the cogntve and socal vectors c (p x ) and c (pg x ). In turn, PSOAF s not objectve, whch results n severe performance loss for rotated functons. Nevertheless, PSOAF stll outperforms PSOAF for both rotated and unrotated test functons, snce the algorthm s dverse,.e. the partcle trajectores do not collapse to lnes. A novel mplementaton denoted PSOAF* s proposed, whch s both objectve and dverse. In PSOAF*, the magntudes are scaled, and the drectons perturbed ndependently, of both the cogntve and socal vectors c (p x ) and c (pg x ). (Ths however comes at the cost of an addtonal scalng factor.) PSOAF* outperforms PSOAF for the unmodal functons used, for both the rotated and unrotated test functons. In addton, ts performance s comparable to PSOAF for the multmodal functons, wth the added advantage of beng ndependent of the reference frame n whch the objectve functon s formulated.