Particle Swarm Optimization Algorithm for Agent-Based Artificial Markets. Tong Zhang and B. Wade Brorsen

Transcription

1 Paricle Swarm Opimizaion Algorihm for Agen-Based Arificial Markes Tong Zhang and B. Wade Brorsen Absrac Paricle swarm opimizaion (PSO) is adaped o simulae dynamic economic games. The robusness and speed of he PSO algorihm is compared o a geneic algorihm (GA) in a Courno oligopsony marke. Arificial agens wih he PSO learning algorihm find he opimal sraegies ha are prediced by heory. PSO is simpler and more robus o changes in algorihm parameers han GA. PSO also converges faser and gives more precise answers han he GA mehod which was used by some previous economic sudies. Key words algorihm Simulaion Agen-based marke Paricle swarm opimizaion Geneic Conac Auhor: Tong Zhang Research Insiue of Economics and Managemen Souhwesern Universiy of Finance and Economics P.R.China Phone: Fax: ongzhang@swufe.edu.cn B. Wade Brorsen Regens Professor 44 Ag Hall Deparmen of Agriculural Economics Oklahoma Sae Universiy

2 A Paricle Swarm Opimizaion Algorihm for Agen-Based Arificial Markes Inroducion Agen-based models are increasingly used o sudy economic phenomena and are especially suiable for simulaing economic games in which agens inerac wih each oher wih bounded raionaliy and adapive learning rules. Work o dae has shown ha such models can obain he same resuls as heoreical models (Arifovic 994; Alkemade, Poure, and Amman 2006). Agen-based models offer considerable poenial o sudy aucions and marke mechanism designs as well as more radiional indusrial organizaional opics. These models have he poenial o sudy much more complex economic problems han can be analyzed heoreically, such as markes conaining heerogeneous agens, or agens using combinaorial sraegies. They can also have a much lower cos han experimenal markes wih human subjecs. To dae, however, he complexiy of he agen-based models has been limied. One limiaion of hese models is he ime i akes o find an opimum, while oher limiaions are he algorihm complexiy and he robusness o algorihm parameers. Previous research using agen-based models in economics have used eiher a geneic algorihm (GA) (Arifovic 994 and 996; Axelrod 987; Bullard and Duffy 999; Riechmann 200; Virend 2000) or reinforcemen learning (RL) (Erev and Roh 998; Kuschinski, Uhmann, and Polani 2003). Wih GA, researchers have o be very careful o choose parameers and mehods for each problem or premaure convergence may occur. The large populaion size required also makes GA slow o find equilibrium. RL is a subarea of machine learning, and he environmen is ypically formulaed as a finie-sae

3 Markov decision process in which an agen increases he probabiliy of choosing successful sraegies under he possible sraegy spaces of is rivals. When he possible sraegy space is large or coninuous, he compuaional cos increases exponenially. To avoid he problems of GA and RL, we use a paricle swarm opimizaion (PSO) o model he learning behavior of agens. PSO is a sochasic opimizaion echnique developed by Eberhar and Kennedy (995). The idea of PSO came from waching he way flocks of birds, fish, or oher animals adap o avoid predaors and find food by sharing informaion. In PSO, a se of randomly generaed soluions moves owards he opimal soluion over a number of ieraions by assimilaing and sharing informaion among all members of he swarm. PSO has been shown o have he same abiliy o find a global opimum as geneic algorihms, bu is able o find opimums faser han geneic algorihms (Panda and Padhy 2007; Mouser and Dunn 2005; Hassan e al. 2005). Exising PSO mehods, however, canno be direcly applied o solving agen-based models. Wih an agen-based model, all agens solve heir own opimizaion problems under a dynamic economic environmen since an agen s profi depends on he acions of oher agens. The objecive of his essay is o adap PSO o solve an agen-based model under a dynamic environmen wih non-cooperaive agens. We also compare he proposed PSO algorihm o a geneic algorihm for finding equilibrium in he Courno oligopsony marke. 2

4 2 Theoreical Model of Oligopsony Marke The Courno oligopsony marke describes a siuaion where a few buyers compee in a marke and each of hem can influence he marke price. In his siuaion, buyers mus make sraegic decisions, while aking ino accoun he decisions of heir rivals. Consider a homogeneous produc marke wih M buyers and N sellers. The number of buyers is much less han he number of sellers ( M << N ). Assume ha buyers process producs ha will be sold in he reail marke and he marginal cos for processing is consan for all processors. The marginal value equals he selling price minus he marginal processing cos. To focus our research on he games beween buyers and sellers in his marke, we assume he final produc price P and he marginal processing cos mc are consan. Thus he value of he produc before processing R= P mc is also consan. Each firm uses he processing raio as is choice variable, where xi is he processing raio; x d = q /( R N), () i i d qi is he processing quaniy of he firm, and i also defines he amoun of procuremen; R is he marginal revenue of produc and also he supply level of sellers under he perfec compeiion price level; and N is he oal number of sellers. For example, if under perfec compeiion, all sellers will provide 0,000 producs, and he processing quaniy of processor i is 3,000, is processing raio M d xi equals 0.3. Then he oal demand can be wrien as D= i = q i. A he ning of each processing period, buyers make procuremen sraegies x simulaneously. We assume all sellers are homogeneous and have supply funcion q s j = p, so he oal supply is S = Np. The oal demand of buyers will deermine 3

5 he marke price ogeher wih he aggregae supply of sellers and he marke price is p= D N. In he simulaion marke, wih 4 buyers and 00 sellers, R equals $00. According o heory, under perfec compeiion, he marke price is $00, he aggregae supply is 0,000, and he processing raio is 25% for each firm; if he marke reaches Courno-Nash equilibrium, he marke price equals $80 and he processing raio is 20% for each firm. 3 Paricle Swarm Opimizaion Algorihm This research adjuss PSO for a non-cooperaive game by consrucing muliple parallel markes and leing each agen have is own clones in every marke. This means each agen has a separae flock of birds ha does no share informaion wih he flocks of oher agens. The asynchronous bes sraegies of one agen in every parallel srucure are called local bes soluions and he bes fi sraegy among all parallel srucures a he curren simulaion ieraion is called he global bes soluion. Each firm has is clones in every parallel marke, and hese clones rade independenly and simulaneously in all markes. We can look a his seing as firms separaing he sellers ino groups or separaing a longer ime ino muliple periods and rying differen sraegies wihin each group or period. This kind of markeing sraegy can be observed in many real markes. For example in he fed cale markes, packing firms send many agens o purchase cale from feedlos, and each of hem visis feeders in a cerain area. Agens bid differenly, bu hey will share informaion a he end of each period and adjus heir sraegies o increase profi. In real world markes, he dynamics of marke prices are mosly pah dependen, which means he marke prices only change a 4

6 small value each ime. So he adapive feaure of PSO is similar o acual learning behavior. Considering ha agens are coninuously changing heir sraegies, he pas local bes soluions may no be he bes for he curren period. Therefore, we adjus PSO by reesing he hisorical bes locals of each agen under he curren marke environmen and choose he bes fi sraegy as he curren bes local. Every agen coninuously uses is own PSO algorihm o search for beer soluions in each parallel marke guided by heir own bes local and global soluions. 3. PSO Algorihm Descripion We se up K parallel markes, and le he M buyers each have heir own clones in every marke. Alhough having he same behavior rules, one agen and is K clones may ake differen marke sraegies since he iniialized random values are differen. In he simulaion, buyers dynamically change heir markeing sraegies wih he PSO algorihm, bu sellers are price akers and simply sell heir producs o he curren highes bidders. The clone of firm i in he h k parallel marke has a quaniy raio value x i, k [0,] as a sraegy parameer, and each iniial sraegy parameer is randomly seleced from a U (0,) disribuion a he ning of he simulaion. Each clone has an evoluionary velociy, v [, ], which deermines he change of is sraegy. The i, k + changes of he clones sraegies are influenced by he locaion of he bes soluions l achieved by his clone iself, pi, k [0,] for he k h clone, and by he whole populaion, g p i [0,]. The superscrips l and g indicae local and global, he subscrips k and 5

7 i indicae h k parallel marke and i h firm respecively. Profi funcion π ) is used o value he performance of each sraegy x,. In every simulaion sep, he sraegy of he i h firm in he updaed by he following equaions: i k k ( x i, k h k parallel marke is x i, k, = xi, k, + vi, k, +, and (2) v l g + = w vi, k, + cu ( pi, k, xi, k, ) + c2u2 ( pi, k, xi, k, ), (3) i, k, where x i k,, is he procuremen raio in period, v i, k, is he velociy vecor, u j [0,], j =, 2 are uniformly disribued random numbers, c and c2 are learning parameers and are called self confidence facor and swarm confidence facor respecively, and w is an ineria weigh facor. The following equaions indicae how o choose l p i k,, and p g i,. In economic games, he payoff of one agen s sraegy is also deermined by he sraegies of is rivals, and he changing of is rivals behaviors forms he dynamic economic environmen of his agen. This may cause he agen s previous bes local sraegies o no perform well in he curren period. Thus, we reevaluae an agen s bes sraegy by using is L previous bes locals o rade versus oher agens curren period sraegies and compare heir payoffs wih ha of is curren sraegy, and hen choose he bes among hem as he bes local of he curren period. This procedure can be wrien as: l l { ( p ),, π ( p ), ( x x } = π K π, (4) l p i, k, arg max k i, k, k i, k, L k i, k, ) i ' i, k, where i' indicaes firm i s rivals. The bes global is seleced from he bes local parameers, 6

8 p g i, l l l { π ( p ), π ( p ), K, π ( p )} = arg max i,, 2 i,2, K i, K,, (5) where k =,2,..., K and K is he oal number of parallel markes. Chaerjee and Siarry (2006) sae ha he ineria weigh w in (3) is criical for he PSO s convergence behavior. A large ineria weigh provides a larger exploraion, bu a smaller one is needed o fine-une he curren search area. Therefore, a compromise is worhwhile, such as saring w wih a higher iniial weigh a he ning and hen decreasing i wih ieraions w ( max ) max = β β, (6) w w 0 + / where boh w β0 andβ are consans, max is he maximum number of ieraions in one w simulaion round, and indicaes he curren ieraion. The self confidence facor c and swarm confidence facor c 2 in equaion (3) are se as in he following equaion: c c where boh β andβ are consans. 0 c ( max ) max c, 2, 0 c c / = = + β 3.2 Summary of Simulaion Procedure wih PSO β, (7) In he Courno oligopsony game, buyers selec independenly and simulaneously he quaniy hey produce. The oal supplies plus he demand curve deermine he reail price. We se up K parallel markes, and each marke conains he same agens. Each firm uses he bes finess curren sraegy in each marke as is bes local and is own bes performance in all parallel markes as is bes global. The bes global differs by firm. Table 5 presens he pseudo-code of he Paricle Swarm Opimizaion Algorihm. (i) For he firs L ning ieraions, randomly iniialize sraegies for all buyers in 7

9 every parallel marke. We choose he quaniy raio x i k [0,] and he,, U movemen velociies v 0 for i =,..., M, k =,..., K, and =,..., L. i, k, = (ii) Buyers updae heir sraegies wih equaions (2) and (3). (iii) Afer he firs L ieraions, each buyer reess he pas L bes locals under he curren economic environmen and compares heir performance wih ha of he curren sraegy, he bes among hem is chosen as he new bes local, as equaion (4) shows. (iv) (v) Following equaion (5), he bes fi among all bes locals is he bes global. If he marke does no reach equilibrium, go o sep (ii). 4 Geneic Algorihm GA is a general-purpose opimizaion mehod based loosely on Darwinian principles of biological evoluion, reproducion, and he survival of he fies (Goldberg 989). GA mainains a pool of candidae soluions called a populaion and repeaedly modifies hem. A each sep, he GA selecs candidaes from he curren populaion o be parens and uses hem o produce children for he nex generaion. Over successive generaions, he populaion evolves oward an opimal soluion. The GA is well suied o and has been exensively applied o solve complex design opimizaion problems because i can handle boh discree and coninuous variables, as well as nonlinear objecive and consrain funcions. Alhough shown o be oversaed by laer work, he GA sudy by Alkemade, Poue and Amman (2006) indicaes ha o avoid premaure convergence of he evoluionary algorihm, each agen should have a large populaion of sraegies from which agens can choose. 8

10 4. Geneic Algorihm Operaors and Parameers In GA, a sraegy of each firm can be represened wih a chromosome ha conains informaion abou his sraegy. The mos used way of encoding is a binary sring. We use B bi binary srings o encode sraegies and he bis can be viewed as genes. Each firm has a populaion of K chromosomes and represens a collecion of is sraegies a ime period. The h k sraegy of firm i in period can be saed wih a sring of lengh B as b where { 0,} i, k, a a K a, (8) B B i, k,, i, k,,, i, k, a is aken a he b h posiion in he sring, b {,2,..., B}, and can be decoded ino a decimal ineger using B b b, s, = i, k, 2 ) b= d i ( a. (9) B b The maximum value is d max = 2. Afer choosing one acive sraegy d i, k,, he firm s b= procuremen raio can be calculaed wih xi, k, = di, k, / d max. For example, if a sring conains 4 bis, a binary code 00 can be decoded o decimal value d = = 5, and he maximum binary code of his sring i can be decoded o value d max = 5. Thus a firm using 00 as is sraegy will have a procuremen raio of x = d d = 5 /5 30%. Since he larger he sring lengh, i, k, i, k, / max = he more accurae he procuremen raio, we use 5 bis as he sring lengh in he simulaion. Buyers decision rules are updaed using four geneic operaors, eliism, reproducion, crossover, and muaion. Eliism can very rapidly increase performance of 9

11 GA, because i prevens losing he bes found soluion o dae (De Jong 975). In our Courno game, he profi difference beween sraegies could be large. To avoid one highprofi sraegy dominaing he nex generaion wih profi proporional selecion, ranking selecion is used as he reproducion mehod. Eliism copies a few of he bes sraegies from he curren K sraegies o he new populaion wih an eliism raeε. If ε = 0% and K = 00, his means he 0 bes sraegies are copied from he old populaion o he new one. The res are chosen wih linear ranking selecion. Reproducion chooses chromosomes as parens from he old sraegy populaion. The ranking selecion mehod ranks an agen s sraegies in is populaion o K from wors o bes according o profi (K = populaion size). If more han one sraegy has he same profi, he sraegies are randomly ranked. The selecion probabiliies of he sraegies x k (k =,, K) are given by ( rmax rmin )( rank( xk ) ) p( xk ) = ( rmin + ), (0) K K where p is he probabiliy of sraegies being chosen as new ones, r + r 2, and r max 2. We choose r max =. and min max min = r = 0.9, so ha p x ) equals. ( k Crossover selecs genes from wo paren chromosomes and creaes a new sraegy. All new sraegies seleced wih eliism and ranking selecion mehods are randomly mached as a group of paren chromosomes. For each pair of parens, he crossover is performed wih a probabiliyχ. The crossover randomly chooses an across poin of he chromosome sring, and bis before and afer his poin are 0

12 exchanged for boh chromosomes o generae new ones. Crossover can look like he daa shown in Figure Muaion akes place afer a crossover is performed wih a probabiliyµ. This muaion is o preven falling ino a local opimum. In he binary encoding mehod, muaion changes he bis of he new sraegy from o 0 or 0 o wih he muaion raeµ. Like PSO, we can also use niching mehods in GA: ε = β, () ε ε 0 + β / max where ε, χ and ( max ) max χ = β, and (2) χ χ 0 + β / ( max ) max µ = β, (3) µ µ 0 + β / µ indicae eliism, crossover, and muaion rae, β s are consan, max is he number of maximum ieraions of he one simulaion round. 4.2 GA Simulaion Procedure for Courno Marke A he ning of he simulaion, buyers, sraegies are randomly generaed. Reurns are calculaed for each sraegy. A new populaion is generaed from he curren one wih he following procedure. Firs, he ε S highes-reurn sraegies are copied o he new populaion as elies. Then, ( ε ) S sraegies are chosen wih ranking selecion mehods from he whole populaion of he old generaion and are randomly mached and crossed over. Muaion is performed for he new sraegies excep he elies. Figure 2 gives an ouline of he program.

13 5 Comparison of Algorihm Performance A Courno game, which has a known Nash equilibrium, is used o evaluae he performance of PSO and GA. Twelve parameer seings for PSO and GA respecively in hree caegories, fixed algorihm parameers, changing algorihm parameers, and differen algorihm srucures. One simulaion round conains muliple ieraions and agens rade wih each oher repeaedly. Wihin hese periods, agens use PSO or GA o updae heir sraegies based on heir rivals sraegies. Considering he randomness of he learning pah, under each seing, we run he Courno game for 20 rounds wih differen random iniialized sraegies, and he game is repeaed for 400 ieraions per round. 5. Algorihm Convergence Crieria Zero diversiy in he populaion's sraegy values signals he sopping poin for GA and PSO. For every agen, if he variance of he sraegies in he populaion is less han 0.0% and he variance of he mean value of he sraegies for 0 generaions is less han 0.0%, we say he algorihm reaches equilibrium. Considering he GA muaion feaure, we delee 5% of he sraegies ha have he larges difference from he mean when calculaing he mean and variance for GA. 2

14 5.2 Robusness Analysis Robusness o small variaions in he echnical parameer seings (ha have no clear economic meaning) is paricularly imporan in agen-based models. Also imporan is ha resuls are valid for a wide range of parameer seings. PSO and GA are nondeerminisic and are no guaraneed o reurn o he same soluion in each run. The speed and accuracy of he algorihms can vary depending on he chosen parameers. We es robusness of he conclusions by comparing he performance under alernaive algorihm parameers, populaion size, and rees imes, which are lised in Table. 6 Simulaion Resuls Hamm, Brorsen, and Hagan (2007) recommend using muliple ses of saring values when using geneic algorihms. Thus, for each se of algorihm parameers, we run 20 imes wih differen randomly generaed iniial sraegies and hen calculae he mean and sandard deviaion of he marke price and he players procuremen raios a 20 equilibrium poins. 6. Robusness Analysis We compare he performance of he PSO and GA wih hree caegories: fixed algorihm parameers; changing algorihm parameers; and differen algorihm srucure parameers. In each seing in he fixed algorihm caegory, we use consan values for he algorihm parameers ( w, c, and c 2 for PSO, ε,χ,andµ for GA). In each seing of he changing algorihm caegory, he algorihm parameers are changing wih ime as shown in 3

15 equaions (6), (7), and () o (3). Under our design of he marke, heoreically he Nash equilibrium of marke price is $80 and he procuremen raio for each buyer is 20%. These values are used o evaluae he performance of he algorihm. Table 2 gives he simulaion resuls wih he fixed algorihm seings. For PSO, all seings give near Nash equilibrium soluions wih low variances. For GA, all four seings give a near Nash equilibrium marke price, bu buyers may have heerogeneous sraegies a he marke equilibrium. Table 3 presens he resuls of he changing algorihm parameer seings. PSO gives near Nash equilibrium resuls for all four seings wih a small sandard deviaion. The seings of GA do no show much difference from he fixed algorihm parameer seings. Table 4 shows he resuls wih differen algorihm srucures. For boh PSO and GA, he large populaion size gives a beer performance, bu PSO is no as sensiive as GA o he algorihm srucure changes. PSO seings wih boh large and small populaion size give near Nash equilibrium resuls for marke price and buyers sraegies wih small differences. GA seings wih small populaion size show bigger differences in buyers sraegies. For boh PSO and GA wih changing parameers, programs need less machine ime and ieraions o reach equilibrium han wih fixed algorihm parameers. GA generally uses around 5 o 80 imes as much machine ime as PSO. The speed of he GA algorihm used is slow because GA codes and decodes binary sring bis, and addiional evaluaion and calculaion is needed for he ranking and roulee selecion. From he above analysis, he overall performance of PSO is considerably faser and more precise han GA and less sensiive o he value of parameers. 4

16 6.2 Individual Runs Afer analyzing he overall resuls, we also choose he bes performance parameer se of PSO and GA ou of all of he seings in boh fixed and changing parameer caegory and draw he figure of one individual simulaion run for each of hem o illusrae how he buyers find he bes response sraegies during he dynamic environmen and how he markes reach equilibrium. The bes performance parameer ses from fixed and changing caegories for each algorihm are ses 2 and 7 in Table 2 and and 4 in Table 3. The evoluion of he marke price level is shown in Figure 3. In his example, boh GA and PSO reach equilibrium a abou he same ime. The GA coninues o muae as he ieraions proceed, bu he muaions are quickly discarded. Wih he PSO algorihm, agens end o have he same sraegy under marke equilibrium, which is prediced by heory, while wih GA individuals have differen sraegies, as Figures 3 and 4 show. The difference can be explained by he learning mehods used by he wo algorihms. In GA, if he sraegy populaion of one firm converges faser han ohers and ohers coninue o adjus heir sraegies, he marke resuls in equilibrium wih heerogeneous sraegies. The global bes in he parallel markes is aken ino accoun in PSO, so one firm has lile probabiliy o ake a larger marke share han ohers. Then he equilibrium of PSO usually conains homogenous sraegies of agens. 5

17 7 Summary and Conclusions Paricle swarm opimizaion is adaped o simulae agen-based models by allowing each agen o have is own parallel srucures and learn from hem. The proposed PSO algorihm is compared o a geneic algorihm for finding equilibrium in he Courno oligopsony marke. Wih rial and error, arificial agens using a PSO learning algorihm can learn o play he bes response sraegies ha heory predics. PSO needs few parameers, and he simulaion resuls are more robus o changing parameers han GA. The parameers of GA need o be carefully chosen o sui he specific simulaion problem. I also requires parameer uning for good performance and can someimes be compuaionally expensive. The comparison is underaken under a relaively simple economic marke design. The reader is cauioned ha he generalizabiliy of he resuls is no known. There are housands of variaions on geneic algorihms ha have been suggesed. The performance of boh PSO and GA can depend on he parameers used. Bu, for he problem considered here, he adapaion of PSO was shown o work well in solving an agen-based model. References Alkemade, F., H. La Poure, and H. Amman. (2006). Robus evoluionary algorihm design for socio-economic simulaion. Compuaional Economics 28: Arifovic, J. (994). Geneic algorihm learning and he cobweb-model. Journal of Economic Dynamics and Conrol 8:3-28. Arifovic, J. (996). The behavior of he exchange rae in he geneic algorihm and experimenal economies. Journal of Poliical Economy 04:

18 Arifovic, J., and M. Maschek. (2006). Revisiing individual evoluionary learning in he cobweb model An illusraion of he virual spie-effec. Compuaional Economics 28: Axelrod, R. (987). The evoluion of sraegies in he ieraed prisoner s dilemma. In L. Davis (Ed.), Geneic algorihms and simulaed annealing (pp. 32-4). London: Piman. Bullard, J., and J. Duffy. (999). Using geneic algorihms o model he evoluion of heerogeneous beliefs. Compuaional Economics 3:4-60. Chaerjee, A., and P. Siarry. (2006). Nonlinear ineria weigh variaion for dynamic adapaion in paricle swarm opimizaion. Compuers & Operaions Research 33: Dawid, H. (999). On he convergence of geneic learning in a double aucion marke. Journal of Economic Dynamics and Conrol 23: De Jong, K. A. (975). An analysis of he behavior of a class of geneic adapive sysems. Ph.D. Disseraion, Universiy of Michigan, Ann Arbor, Mich. Eberhar, R.C., and J. Kennedy. (995). A New opimizer using paricle swarm heory. Proceedings of he Sixh Inernaional Symposium on Micromachine and Human Science, Nagoya, Japan. pp Erev, I., and A. Roh. (998). Predicing how people play games: reinforcemen learning in experimenal games wih unique mixed sraegy equilibria. American Economic Review 88: Goldberg, D.E. (989). Geneic algorihms in search, opimizaion, and machine learning. Massachuses: Addison-Wesley. Hamm, L., B.W. Brorsen, and M.T. Hagan. (2007). Comparison of sochasic global opimizaion mehods o esimae neural nework weighs. Neural Processing Leers 26: Hassan, R., B. Cohanim, O. De Weck, and G. Vener. (2005). A comparison of paricle swarm opimizaion and he geneic algorihm. Proceedings of he 46h AIAA/ASME/ASCE/AHS/ASC Srucures, Srucural Dynamics and Maerials Conference, Ausin, Texas, AIAA Kuschinski, E., T. Uhmann, and D.Polani. (2003). Learning compeiive pricing sraegies by muli-agen reinforcemen learning. Journal of Economic Dynamics & Conrol 27:

19 Panda, S. and N.P. Padhy. (2007). Comparison of paricle swarm opimizaion and geneic algorihm for csc-based conroller design. Inernaional Journal of Compuer Science and Engineering :-49. Mouser, C.R., and S.A. Dunn. (2005). Comparing geneic algorihms and paricle swarm opimisaion for an inverse problem exercise. Anizam Journal 46:C89-C0. Riechmann, T. (200). Geneic algorihm learning and evoluionary games. Journal of Economic Dynamics & Conrol 25: Vriend, J.N. (2000). An illusraion of he essenial difference beween individual and social learning, and is consequences for compuaional analyses. Journal of Economic Dynamics & Conrol 24:-9. 8

20 Chromosome Chromosome New Chromosome New Chromosome Fig. Crossover Daa. ( is he crossover poin) 9

21 Sep Sep 2 Sep 3 Sep 4 Iniializaion { Every agen iniializes he saring sraegy populaion pool by randomly drawing S sraegies; Assign monopoly profi as relaive payoff. } For each generaion do { For each agen {Randomly choose is acive sraegy from he populaion} Play he Courno game; Calculae and sore payoff for he curren acive sraegy. } unil all sraegies are played. Updae algorihm operaors or parameers if needed; Generae a new populaion from he old one { Chooseε S sraegies wih highes profi as elies; Choose ( ε ) S sraegies wih ranking selecion mehods; Randomly mach all he seleced ones as parens, apply single crossover o hem wih rae χ, unil ge ( ε ) S number sraegies, and apply muaion o hem; These ( ε ) S sraegies combine wih elies o form he new populaion. } If no converged, go back o sep 2; End; Fig. 2 Ouline of he geneic algorihm (GA) for Courno game 20

22 (a) Wih fixed algorihm parameers Fig. 3 Marke price level (b) Wih changing algorihm parameers Noe:. In (a), he parameer se for PSO is [w, c, c 2 ] = [ ε χ parameer se for GA is [ε,χ, µ ] = [ β 0, β 0, in equaions () o (3) are zeros. w c c2 2. In (b), he parameer se for PSO is [ β, β β ε χ is [ β, β, µ β ε χ µ [ β 0, β 0, β 0 ] = [0, 0, 0]., c c2 β, β, β ] = [0.4,, ]; w µ β 0 ] = [0%.76%, 0.33%]; and slopes ] = [0.5,, ]; parameer se for GA ] = [20%, 66%, %]; and inerceps in equaions () o (3) are zeros, 2

23 (a) GA (b) PSO Fig. 4 Quaniy level wih fixed algorihm parameers w c c2 Noe: The parameer se for PSO is [w, c, c 2 ] = [ β 0, β 0, β 0 ] = [0.4,,]; he parameer ε χ µ se for GA is [ε,χ, µ ] = [ β 0, β 0, β 0 ] = [0%.76%, 0.33%]; and slopes in equaions () o (3) are zeros. 22

24 (a) GA (b) PSO Fig. 5 Quaniy level wih changing algorihm parameers w c c2 Noe: The parameer se for PSO is [ β, β β ] = [0.5,, ]; he parameer se for GA ε χ is [ β, β, µ β ε χ µ [ β 0, β 0, β 0 ] = [0, 0, 0]., ] = [20%.76%, %]; and inerceps in equaions () o (3) are zeros, 23

25 Table Parameers for PSO and GA in he Courno Oligopsony Simulaions PSO parameers GA parameers Number of parallel markes: K Sraegy populaion size: K Number of rees local bes parameers: L Loop per ieraion: L w w Ineria weigh: w = β 0 + β ( max ) / max Sring bi: B c Local confidence facor: 0 ( max ) c c ε = + β max ε = β + β ε β Eliism rae: 0 / max = β 0 + β / c 2 Global confidence facor: 2 0 ( max ) 2 c c χ χ = β + β max Crossover rae: ( max ) max χ µ µ Muaion rae: µ = β 0 + β ( max ) / max Ranking selecion parameer: r max =., r min =0.9.,, 24

26 Table 2 PSO and GA Simulaion Resuls wih Fixed Algorihm Parameers Capaciy Raio Se Parameers Saisic Marke Price($) Buyer Buyer 2 Buyer 3 Buyer 4 Machine Time PSO w c c Mean % 2.54% 20.30% 9.86% 294 N/A SD % 0.53% 0.43% 0.53% Mean % 20.0% 9.99% 20.00% SD % 0.0% 0.00% 0.00% Mean % 2.32% 9.5% 9.0% SD % 0.93% 0.67%.0% Mean % 20.09% 9.98% 9.95% SD % 0.8% 0.06% 0.08% 6 0 GA ε χ µ Equilibrium Ieraion % 80.00%.00% Mean % 25.53% 9.33% 8.98% 4,780 N/A SD % 3.35% 3.9% 3.93% % 80.00% 0.33% Mean % 20.77% 25.33% 2.3% 5,40 N/A SD % 2.68% 4.57% 2.93% 2, % 76.00% 0.33% Mean % 2.92% 20.23% 8.70% 5, SD % 3.03% 2.60% 3.40%, % 56.00% 0.33% Mean % 25.0% 8.75% 2.9% 4, SD % 4.25% 4.42% 3.47%,46 04 Noe: For PSO, he parallel marke size for PSO is 20. For GA, he populaion size is 40; he bi lengh is 5. 25

27 Table 3 PSO and GA Simulaion Resuls wih Changing Algorihm Parameers Capaciy Raio Se Parameers Saisic Marke Price($) Buyer Buyer 2 Buyer 3 Buyer 4 Machine Time c β β Mean % 20.00% 20.00% 20.04% PSO w Equilibrium Ieraion SD % 0.02% 0.03% 0.02% Mean % 20.00% 20.02% 20.00% SD % 0.02% 0.02% 0.02% Mean % 20.00% 20.00% 20.00% SD % 0.0% 0.03% 0.0% Mean % 9.99% 9.85% 9.87% SD % 0.27% 0.28% 0.24% GA ε β χ µ β β % 86.00% 0.33% Mean % 8.55% 25.02% 8.76% 3, SD % 3.2% 3.38% 3.79% % 66.00%.00% Mean % 8.75% 9.53% 2.88% 4, SD % 2.28% 2.43% 2.56% % 76.00%.00% Mean % 8.55% 7.06% 25.08% 4, SD % 3.75% 3.97% 2.90% % 66.00% 0.33% Mean % 20.32% 9.93% 9.48% 4, SD % 2.26% 2.65%.2% Noe:. For PSO, he parallel marke size for PSO is 20. For GA, he populaion size is 40, and he bi lengh is For PSO, he inerceps 3. For GA, he inerceps w β and 0 ε β 0, c β in equaions (6) and (7) are chosen as consan value 0.5 and respecively. 0 χ β 0 and β µ 0 in equaions (), (2), and (3) are zero. 26

28 Table 4 PSO and GA Simulaions Resuls under Differen Algorihm Srucure Capaciy Raio Se P L Saisic Marke Price($) Buyer Buyer 2 Buyer 3 Buyer 4 Machine Time Equilibrium Ieraion PSO Mean % 9.99% 20.03% 9.98% SD % 0.04% 0.0% 0.0% Mean % 9.97% 20.04% 20.02% SD % 0.38% 0.98% 0.23% Mean % 20.22% 9.0% 20.22% SD % 5.04% 4.88%.93% 5 45 GA Mean % 20.32% 8.74% 20.3% 35, SD %.40%.4%.86% 2, Mean % 20.3% 25.00% 2.49% 4,3 50 SD % 2.84% 3.43% 2.35% Mean % 2.49% 2.97% 22.20%,838 2 SD % 3.67% 3.62% 4.2% Noe: P indicaes parallel markes number and populaion size for PSO and GA respecively, L indicaes number of rees local bes of PSO and loop number per generaion of GA respecively. 27

29 Table 5 Pseudo-code of Paricle Swarm Opimizaion Algorihm program MAIN; for each seing do {sar main loop} for each evoluionary ieraion do if use changing parameer seing hen updae PSO_w, PSO_c and PSO_c2; for each parallel marke do for each firm do Sraegy:=CALCULATE_STRATEGY; {see funcion below} calculae oupu level wih sraegy; deermine marke price; for each firm do {repor oucomes} for each firm do CHOOSE_LEARNING_PRAMETERS; TEST_EQUILIBRIUM; funcion CALCULATE_STRATEGY; profi:=(marke price)*(oupu level)-coss; wih sraegy do finess:=profi; if evoluionary ieraion number < L hen for each firm do for each parallel marke do make random sraegy; finess:=0; else {applicaion dynamic PSO algorihm} {see procedure below} {see procedure below} {iniialize sraegies and finess values} {generae new sraegy} updae PSO_vecor; Sraegy:= New_PSO_vecor+Previous_Sraegy; 28

30 Table 5 (Coninued) procedure CHOOSE_LEARNING_PRAMETERS; {choose Bes Local and Bes Global parameers wih dynamic PSO} for each parallel marke do {selec he Bes Local} for each of he previous L hisory Bes Locals do sraegy:=hisory_bes_local; calculae oupu level wih sraegy; deermine marke price; for his firm do profi:=(marke price)*(oupu level)-coss; wih sraegy do finess:=profi; choose he bes performance sraegy from he pervious Bes Locals of previous L ieraions and he curren sraegy under he curren economic environmen; Bes Local of he curren ieraion:=he chosen bes performance sraegy; Bes Global Parameer:= Bes Local wih highes finess value among all parallel markes; TEST_EQUILIBRIUM; for each firm do calculae he mean of sraegies in all parallel markes; for each firm do for each parallel marke do if he variance of he mean value of sraegies of pas L ieraions >0.0% hen reurn; if he variance of sraegies of all parallel markes >0.0% hen reurn; evoluion reaches equilibrium; 29

31 Table 5 (Coninued) The main program. The main loop of he program. If PSO algorihm parameer w, c and c2 decrease wih ime, updae hem wih equaion (6) and (7). Each firm use clone agens o rade in each of he parallel markes simulaneously. Each firm calculae a new sraegy in his parallel marke, he sraegy is he raio of he oal demand of all firms in he marke under perfec compeiion. Oupu level equals sraegy muliple he possible oal demands. The oal demand of buyers will deermine he marke price ogeher wih he aggregae supply of sellers. Repor he marke oucomes. The finess value equals he profi generaed by ha sraegy. Wih he dynamic PSO algorihm, he procedure CHOOSE_LEARNING_PRAMETERS is applied o each firm separaely. Tes if evoluion reaches equilibrium. Wih new Bes Local and Bes Global parameers, generae new sraegy. For he firs L ieraions, randomly make sraegies for each firm. Calculae new sraegy for each firm wih he Bes Local and Bes Global parameers. Updae PSO learning vecor by equaion (3). Calculae new sraegy by equaion (2) For each firm, applied dynamic PSO algorihm. Using dynamic PSO learning algorihm o selec Bes Local in each parallel marke. In each parallel marke, assign one of he neares previous L hisory Bes Local as he firm s sraegy. Deermine he marke price by assuming his firm uses assigned sraegy while oher firms use curren sraegies. Each firm compares he performances of hisory Bes Locals wih ha of is curren sraegy, and chooses he bes among hem as he Bes Local. Choose he Bes Global parameer for each firm. Deermine he evoluion reaches equilibrium or no. If he variance of he sraegies in he populaion is less han 0.0% and he variance of he mean value of he sraegies for L generaions is less han 0.0%, we say he algorihm reaches equilibrium. 30