A New Algorithm Using Hopfield Neural Network with CHN for N-Queens Problem

Transcription

1 36 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl 009 A ew Algorthm Usng Hopfeld eural etwork wth CH for -Queens Problem We Zhang and Zheng Tang, Facult of Engneerng, Toama Unverst, Toama-Sh, JAPA Summar A model of neurons wth CH (Contnuous Hsteress eurons) for the Hopfeld neural networks s studed We prove theoretcall that the emergent collectve propertes of the orgnal Hopfeld neural networks also are present n the Hopfeld neural networks wth contnuous hsteress neurons The network archtecture s appled to the -Queens problem and results of computer smulatons are presented and used to llustrate the computaton power of the network archtecture The smulaton results show that the Hopfeld neural network wth CH s much better than other algorthms for -Queens problem n terms of both the computaton tme and the soluton qualt Ke words: Hopfeld neural network, hsteress, collectve propertes, - Queen problem Introducton The optmzaton problems are encountered n varous stuaton There s a problem whch has dscrete determned varables s called Combnatoral Optmzaton Problems Ths problem s complcated more than a lnear programmng, and t s called P- Hard -Queens problem s one of the P-Hard combnatoral optmzaton problems -Queens problem s that chess queens must be placed on a square chessboard composed of rows and columns, n such a wa that the do not attack each other for 8 drectons The auto assocatve memor model proposed b Hopfeld [, ] has attracted consderable nterest both as a content address memor (CAM) and, more nterestngl, as a method of solvng dffcult optmzaton problems [3-5] The Hopfeld neural networks contan hghl nterconnected nonlnear processng elements ( neurons ) wth two-state threshold neurons [] or graded response neurons [] Takefu and Lee proposed a two-state (bnar) hsteretc neuron model to suppress the oscllator behavors of neural dnamcs [4] However, Tatesh and Tamura showed Takefu and Lee s model dd not alwas guarantee the descent of energ functon [7], Wang also explaned wh the model ma lead to naccurate results and oscllator behavors n the convergence process [8] Snce ther report, several modfcatons on the hsteretc functon, for example Galán and Muñoz s bnar [9] and Bhartkar and Mendel s multvalued [0] hsteretc functons In ths paper, we propose a new Hopfeld neural network algorthm for effcent solvng -Queens problem Dfferent to the orgnal Hopfeld neural network, our archtecture uses contnuous hsteress neurons We prove theoretcall that the emergent collectve propertes of the orgnal Hopfeld neural network also are present n the Hopfeld network wth contnuous hsteress neurons Smulatons of randoml generated neural networks are performed on both networks and show that the Hopfeld neural networks wth CH have the collectve computatonal propertes lke the orgnal Hopfeld neural networks What a more, t converges faster than the orgnal Hopfeld neural networks do In order to see how well the archtecture neurons do for solvng practcal combnatoral optmzaton problems, a large number of computer smulatons are carred out for the -Queens problem Hopfeld etwork wth Contnuous Hsteress eurons Orgnal Hopfeld eural etworks For the orgnal Hopfeld neural networks, let the output 0 varable for neuron have the range and be a contnuous and monotone-ncreasng functon of the nstantaneous nput x to neuron The tpcal nput-output relaton g (x ) = /( e) shown n Fg(a) s sgmod 0 wth asmptotes and as, r x = g( x ) = /( e ( θ ) Where r s the gan factor andθ s the threshold parameter Contnuous Hsteress eurous If In bologcal sstem, x wll lag behnd the nstantaneous outputs of the other cells because of the nput capactance C of the cell membranes, the transmenmbrane resstance, and the fnte mpedance w between the ) () Manuscrpt receved Aprl 5, 009 Manuscrpt revsed Aprl 0, 009

2 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl output and the cell bod of cell Thus there s a resstance-capactance (C) chargng equaton that determnes the rate of change of x dx x C ( ) = w I =,, L, = x = g ( ) () dx C ( ) = w = x = g ( ) x I θ ( x ) =,, L, (6) Where C s the total nput capactance of the amplfer and ts assocated nput lead w represents the electrcal current nput to cell due to the present potental of cell, and w s thus the snapse effcac I s an other (fxed) nput current to neuron In terms of electrcal crcuts, g (x ) represents the nput-output characterstc of a nonlnear amplfer wth neglgble response tme It s convenent also to defne the nverse output-nput relaton, g ( ) In order to mprove the soluton qualt of -Queens problem, we proposed a new neural network method for effcentl solvng the -Queens problem In ths method, an contnuous hsteress neuron s appled to the Hopfeld neural network The contnuous hsteress neurons change the value of ther output or leave them fxed accordng to a hsteretc threshold rule (Fg (b)) Mathematcall, the hsteretc neuron functon s descrbed as: where and ( x ( t) x ) = g x ( t) θ( x ) γ α, γ ( x ( t δt)) = γ β, x 0 x 0 α, x 0 θ ( x ) = β, x 0 β > α, and ( γ, γ ) > 0, and dx x Δ Thus, there s a resstance-capactance (C) chargng equaton that determnes the rate of change of x α β (3) (4) (5) Consder the energ: E = = = w x ( x x ) d = = Its tme dervatve for a smmetrc W s: de = I 0 g d x ( w I = = (7) θ( x( t δt)) ) (8) The parenthess s the rght-hand sde of Eq6, so de = = = = d dx C ( )( ) C g ' d ( )( ) Snce g ( ) s a monotone ncreasng functon and C s postve, each term n ths sum s nonnegatve Therefore: de de d 0, = 0 = 0 for all (9) (0) Together wth the boundedness of E, Eq6 shows that the tme evoluton of the sstem s a moton n state space that seeks out mnma n E and comes to a stop at such ponts E s a Lapunov functon for the sstem

3 38 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl 009 where A and B are coeffcents, the output = represents that a queen s placed at -th or -th column on the chessboard, and output =0 represents no placement there The frst term becomes zero f one queen s placed n ever row The second term becomes zero f one queen s placed n ever column and the thrd term becomes zero f no more than one queen s placed on an dagonal lne We can get the total nput (x ) of neuron b usng the partal dervaton term of the energ functon Thus, the total nput (x ) of neuron s gven: x = A B k, k k = k k, k k 0 A k = k, k, k k 0 k k () Fg Hsteress functons 3 Applcaton to -Queens Problem -Queens problem s classc of dffcult optmzaton The task s gven a standard chessboard and chess queens, to place them on the board so that no queen s on the lne of attack of an other queen The problem can be solved b constructng an approprate energ functon and mnmzng the energ functon to zero (E=0) usng an two-dmensonal Hopfeld neural networks [~4] The obectve energ functon of the -Queens problem s gven b: A E = B = = k = k= = k= k, k k, k k 0 k k, k 0 k, k k () Usng Eq, networks wth contnuous hsteress neurons for a total 0 chessboard sze nstances from 0 to 300 queens problem were smulated on a dgtal computer 00 smulaton runs wth dfferent ntal states were performed n each of these nstances In the smulatons, Eq6 was used as the nput/output functons of neurons The neuron s actvaton functon has four parameters assocated wth t We set α = 0 5 β = 0 5 and γ α = γ β = 00 for all neurons The parameters A and B were set to The maxmum teraton steps were set to 000 Usng the same conon, the orgnal Hopfeld neural networks was also executed for comparson The smulaton results are shown n Table In Table, where the convergence rates and the average numbers of teraton steps requred for the convergence were summarzed The smulaton results show that the networks wth contnuous hsteress neurons can almost fnd optmum soluton to all -Queens problems wthn short computaton tmes; whle the orgnal Hopfeld neural networks can hardl fnd an optmum soluton to the -Queens problems We also compared our results wth that found b Takefu s neural network [] and the maxmum neural network [3] Table shows the results b the four dfferent networks, where the convergence rates and the average numbers of teraton steps requred for the convergence are summarzed From Table we can see that the Hopfeld neural networks wth contnuous hsteress neurons was ver effectve, and was better than other extng neural networks n terms of the computaton tme and the soluton qualt for the -Queens problem Further, the average numbers of teraton steps ndcated that the problem sze dd not strongl reflect the global

4 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl mnmum convergence rate and number of teraton steps From the smulaton results we can summarze that the Hopfeld neural networks wth contnuous hsteress neurons s ver effectve for solvng the -Queens problem Table : Smulaton result Queens Proposed network Hopfeld Maxmum Takefu Convergence Step Convergence Step Convergence Step Convergence Step Smulaton esults Experments were frst performed to show the convergence of the Hopfeld neural networks wth contnuous hsteress neurons In the smulatons, a 00-neuron Hopfeld neural network wth contnuous hsteress neurons ( =,,, 00) was chosen Intal parameters of the network, connecton weghts and thresholds were randoml generated unforml between 0 and 0 Smulatons on a randoml generated 00-neuron Hopfeld network wth contnuous hsteress neurons were carred out We dd 00 smulatons wth dfferent randoml-generated weghts and thresholds Fg shows the statstcall compared convergence charactorstcs of both networks wth the average energ over 00 smulatons From ths fgure we can see that both the Hopfeld neural networks wth contnuous hsteress neurons ( α = 0 5, β = 0 5) and the orgnal Hopfeld neural networks converged to stable states that dd not further change wth tme It s worth to note that the Hopfeld neural network wth contnuous hsteress neurons ( α = 0 5, β = 0 5 ) seek out a smaller mnmum at E = -638 than the orgnal Hopfeld neural network at E = Fg The average convergence characterstc of a 00- neuron Hopfeld network wth and wthout the contnuous hsteress neurons In general, the performance of optmzaton problem usng neural network depends on parameters In the proposed algorthm, α and β are mportant parameters whch nfluence the performance of the network To stud the approprate range of α and β, we smulated two graphs usng dfferent α and β We expermented wth the followng values of α = 0, β = 0, 05,, L,3, respectvel Fg3 shows the smulaton results From the smulaton results we found that the networks wth contnuous hsteress neurons ( β 5, when α = 0 ), had the best performance But, when β was larger than 30 ( α = 0 ), the contnuous hsteress neurons tended to degrade the performance of the network Usng the sameα and β set, we also smulated the average number of the teratons b the Hopfeld neural networks wth contnuous hsteress neurons Fg4 shows the smulaton results Fg3 The percentage of optmal soluton of the 0 Queens problem b the Hopfeld neural networks wth contnuous hsteress neurons (β - α = 0, 05,,, 3) From these fgures, we fnd that the approprate range of α and β s almost as same as that n the frst nstance

5 40 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl 009 From these smulaton results, we ma conclude that the proposed archtecture could mprove the performance of the Hopfeld neural network b selectng an approprate value of contnuous hsteress neurons Fg4 The percentage of optmal soluton of the 0 Queens problem b the Hopfeld neural networks wth contnuous hsteress neurons (β - α = 0, 05,,, 3) In order to wdel verf the proposed algorthm, we have also tested t wth a few number of randoml generated queens defned n terms of two parameters, A and B Fg5 shows the smulaton results From ths we can fnd that the when A= and B=, t has the best performance for the problem Fg5 The relatonshp of the parameters A, B and wth the contnuous hsteress neurons 5 Conclusons In ths paper, we have presented theoretcal and expermental evdence showng that the Hopfeld neural networks wth contnuous hsteress neurons has the same collectve computatonal propertes as the orgnal Hopfeld neural networks We have also compared the network wth the orgnal Hopfeld neural networks As theoretcall predcted, t was found expermentall that the Hopfeld neural networks wth contnuous hsteress neurons converged faster than the orgnal Hopfeld neural networks dd In order to confrm the practcal worth of the Hopfeld neural networks wth contnuous hsteress neurons, t was also appled to -Queens problem A large number of computer smulaton have been carred out for -Queens problem to verf the effectveness of ths network n combnatoral optmzaton problems The smulaton results showed that the Hopfeld neural networks wth contnuous hsteress neurons were better than the orgnal Hopfeld neural networks method and other exstng neural network methods for solvng - Queens problem n terms of the computaton tme and the soluton qualt eferences [] JJ Hopfeld, eural network and phscal sstems wth emergent collectve computatonal abltes Proc atl Acad Sc USA Vol79 pp , 98 [] JJ Hopfeld, eurons wth graded response have collectve computatonal propertes lke those of two-state neurons Proc atl Acad Sc USA Vol8 pp , 984 [3] JJ Hopfeld and DWTank, eural computaton of decsons n optmzaton problems Bol Cbern, Vol5 pp4-5, 985 [4] JJ Hopfeld and DWTank, Computng wth neural crcuts: a model Scence, no33, pp65-633,986 [5] DW Tank and JJ Hopfeld, Smple neural optmzaton network: An A/D converter, sgnal decson crcut, and lnear programmng crcut IEEE Trans, Crcuts & Sstems, volcas-33, no5, pp533-54,986 [6] Y Takefu and KC Lee, An hsterss bnar neuron: A model suppressng the oscllator behavor of neural dnamcs, Bol Cbern, Vol64 pp , 99 [7] M Tatesh and S Tamura, "Comments on `Artfcal neural networks for four-colorng map problems and K-colorablt problems'," IEEE Trans Crc Sst-I: Fundamental Theor and Applcatons, vol 4, no 3, pp 48-49, March, 994 [8] L Wang, Dscrete-tme convergence theor and updatng rules for neural networks wth energ functons, IEEE Trans eural etworks, vol8, no5, pp , 997 [9] G Galán-Marín, J Muñoz-Pérez, A new nput-output functon for bnar Hopfeld neural networks, Proceedngs of the Internatonal Work-Conference on Artfcal and atural eural etworks: Foundatons and Tools for eural Modelng, pp3-30, June 0-04, 999

6 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL9 o4, Aprl [0] S Bhartkar, J M Mendel, The hsteretc Hopfeld neural network, IEEE Trans eural etworks, vol, no4, pp , 000 [] Y Takefu, eural etwork Parallel Computng, Kluwer Academc Publshers, 99 [] Y Takenaka, Funabk and S shkawa, Maxmum neural network algorthms for -Queens problem J IPSJ, vol37, no0, pp78-788, 996 [3] Y Takenaka, Funabk and S shkawa, A proposal of competton resoluton methods on the maxmum neuron model through -Queens problem J IPSJ, vol38, no, pp4-48, 997 [4] Y Takenaka, Funabk and T Hgashno, A proposal of neuron flter: A constrant resoluton scheme of neural networks for combnatoral optmzaton problems, IEICE Trans Fundamentals, vole83-a, no9, pp85-83, 000 We Zhang receved a BS degree from Zheang ormal Unverst, Zheang, Chna n 004 She receved an MS degree from Toama Unverst, Toama, Japan, n 007 Form 007 she s workng toward thede degree at Toama Unverst, Japan Her man research nterests are neural networks and optmzaton problems Zheng Tang receved a BS degree from Zheang Unverst, Zheang, Chna n 98 and an MS degree and a DE degree from Tsnghua Unverst, Beng, Chna n 984 and 988, respectvel From 988 to 989, he was an Instructor n the Insttute of Mcroelectroncs at Tsnghua Unverst From 990 to 999, he was an Assocate Professor n the Department of Electrcal and Electronc Engneerng, Mazak Unverst, Mazak, Japan In 000, he oned Toama Unverst, Toama, Japan, where he s currentl a Professor n the Department of Intellectual Informaton Sstems Hs current research nterests nclude ntellectual nformaton technolog, neural networks, and optmzatons