Crossbar Switch Problem Solver by Hysteresis Neural Networks

Transcription

1 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar Crossbar Swtch Problem Solver b Hsteress eural etworks We Zhang and Zheng Tang, Facult of Engneerng, Toama Unverst, Toama-Sh, JAPA Summar In ths paper, we propose a contnuous hsteress neurons (CH) Hopfeld neural network archtecture for effcentl solvng crossbar swtch problems. A Hopfeld neural network archtecture wth contnuous hsteress and ts collectve computatonal propertes are studed. It s proved theoretcall and confrmed b smulatng the randoml generated Hopfeld neural network wth CH. The network archtecture s appled to a crossbar swtch problem and results of computer smulatons are presented and used to llustrate the computaton power of the network archtecture. The smulaton results show that the Hopfeld neural network archtecture wth CH s much better than the bnar hsteress Hopfeld neural network archtecture for crossbar swtch problem n terms of both the computaton tme and the soluton qualt. Ke words: etwork archtecture, crossbar swtch problem, contnuous hsteress, Hopfeld neural network. Introducton The auto assocatve memor model proposed b Hopfeld [,] has attracted consderable nterest both as a content address memor (CAM) and, more nterestngl, as a method of solvng dffcult optmzaton problems [3-5]. The Hopfeld neural network contan hghl nterconnected nonlnear processng elements ( neurons ) wth two-state threshold neurons [] or graded response neurons []. Takefu and Lee proposed a two-state (bnar) hsteretc neuron model to suppress the oscllator behavors of neural dnamcs [6]. However, Tatesh and Tamura showed Takefu and Lee s model dd not alwas guarantee the descent of energ functon [7], Wang also explaned wh the model ma lead to naccurate results and oscllator behavors n the convergence process [8]. Snce ther report, several modfcatons on the hsteretc functon, for example Galán and Muñoz s bnar [9] and Bhartkar and Mendel s multvalued [0] hsteretc functons. Xa proposed a new algorthm that the Hopfeld neural network wth the bnar hsteress and clamed that the soluton qualt found b ther algorthm was superor to that of the best exstng parallel algorthm []. Snce Hopfeld and Tank s works, there has been a lot of nterest n the Hopfeld neural networks because ts advantage over other approaches for solvng optmzaton problems. The advantages nclude massve parallelsm, convenent hardware mplementaton of the neural network archtecture, and a common approach for solvng varous optmzaton problems []. The Hopfeld neural network archtecture has also been appled to real-tme control of a crossbar swtch used for swtchng hgh-speed packets at maxmum throughput [3-9], and shown to be capable of achevng ver good performance especall for small sze crossbar swtch problems. In ths paper, we ntroduce a new Hopfeld neural network algorthm for effcent solvng crossbar swtch problem. Dfferent to the bnar hsteress Hopfeld neural network, our archtecture uses contnuous hsteress neurons (CH). We prove theoretcall that the emergent collectve propertes of the orgnal Hopfeld neural network also are present n the Hopfeld network wth contnuous hsteress neurons (CH). Smulatons of randoml generated neural networks are performed on both networks and show that the Hopfeld neural networks wth CH have the collectve computatonal propertes lke the orgnal Hopfeld neural networks. What a more, the Hopfeld neural networks wthch converges faster than the Hopfeld neural networks wth the bnar hsteress neurons do. In order to see how well the Hopfeld neural networks wth CH do for solvng practcal combnatoral optmzaton problems, a large number of computer smulatons are carred out for the crossbar swtch problem. The smulaton results show that the Hopfeld neural network archtecture wth CH s much better than the prevous works ncludng the Hopfeld neural network wth hsteress bnar neurons for the crossbar swtch problem n terms of both the computaton tme and the soluton qualt.. Bnar Hsteress eurons Hopfeld etwork Lke the orgnal Hopfeld neural networks, the total nput to neuron of the Hopfeld neural networks wth hsteress bnar neuron s: x = w + h where x s the total nput of neuron, s the output of neuron. The element w s the smmetrc nterconnecton strength from neuron to neuron, and h s the offset bas of neuron. () Manuscrpt receved Januar 5, 00 Manuscrpt revsed Januar 0, 00

2 86 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar 00 Each neuron samples ts nput at random tmes. But, unlke the Hopfeld neural network s two-state threshold neurons (Fg. (a)), the hsteress bnar neurons change the value of ther output or leave them fxed accordng to a hsteretc threshold rule (Fg. (b)) = no change 0 f x > a f x < b f b x a where a s the upper trp pont (UTP), and b s the lower trp pont (LTP). As shown n Fg., f x >a, then = and f x <b, then =0. When b x a, keeps unchanged,.e., = f the last was and =0 f the last was 0. Consder the ereng: E = w h () (3) 3. Hopfeld etwork wth Contnuous Hsteress eurons(ch) For the orgnal Hopfeld neural networks, let the 0 output varable for neuron have the range and be an contnuous and monotone-ncreasng functon of the nstantaneous nput x to neuron. The tpcal nputoutput relaton g (x ) = /( + e) shown n Fg.(a) s 0 sgmod wth asmptotes and. as, r ( x θ ) = g( x ) = /( + e ) (4) Where r s the gan factor andθ s the threshold parameter. In bologcal sstem, x wll lag behnd the nstantaneous outputs of the other cells because of the nput capactance C of the cell membranes, the transmenmbrane resstance R, and the fnte mpedance w between the output and the cell bod of cell. Thus there s a resstance-capactance(rc) chargng equaton that determnes the rate of change of x. dx x C ( ) = w + I =,, L, R = x = g ( ) (5) where C s the total nput capactance of the amplfer and ts assocated nput lead. w represents the electrcal current nput to cell due to the present potental of cell, and w s thus the snapse effcac. I s an other (fxed) nput current to neuron. In terms of electrcal crcuts, g (x ) represents the nput-output characterstc of a nonlnear amplfer wth neglgble response tme. It s convenent also to defne the nverse output-nput relaton, g ( ). In order to mprove the soluton qualt of crossbar swtch problem, we proposed a new neural network method for effcentl solvng the crossbar swtch problem. In ths method, a contnuous hsteress neuron(ch) s appled to the Hopfeld neural network. The hsteress contnuous neurons change the value of ther output or leave them fxed accordng to a hsteretc threshold rule (Fg. (b)). Mathematcall, hsteretc neuron functon s descrbed as: where ( x ( t) x ( t δ t)) = g x ( t) θ ( x ( t t)) δ (6) Fg. Hsteress bnar functons

3 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar γ α, γ ( x ( t δt)) = γ β, x ( t δt) 0 x ( t δt) 0 (7) E s a Lapunov functon for the sstem. And α, x ( t δt) 0 θ ( x ( t δt)) = (8) β, x ( t δt) 0 dx ( t δt) β > α, and ( γ α, γ β ) > 0,and x ( t δt) Δ. Thus, there s a resstance-capactance(rc) chargng equaton that determnes the rate of change of x. dx C ( ) = w = x = g ( ) Consder the energ: x + I R. θ ( x( t δt)) + R =,, L, (9) E = = = w x + ( x x ) d + = R I = 0 Its tme dervatve for a smmetrc W s: g (0) Fg.. Hsteress functons. de d x θ ( x( t δt)) = ( w + I + ) () = = R R The parenthess s the rght-hand sde of Eq.9, so de = = = = d dx C ( )( ) d C g ' ( )( ) () Snce g ( ) s a monotone ncreasng functon and C s postve, each term n ths sum s nonnegatve. Therefore: de de d 0, = 0 = 0 for all (3) Together wth the boundedness of E, Eq.(9) shows that the tme evoluton of the sstem s a moton n state space that seeks out mnma n E and comes to a stop at such ponts. 4. Comparson of two Hsteress eurons The contnuous hsteress neurons (CH) Hopfeld neural network and the bnar hsteress neurons Hopfeld neural network are two knds of the hsteress network. The two algorthms dffer from other was: t ) s multvalued; ) has memor; and 3) s adaptve. γ ( x θ ) In the Eq.5, f the value of e get to be nfnte,the r ( x θ ) output varable = ( ) = /( + g x e ) wll be close γ ( x θ ) to "0", and f the value of e get to the nfntesmal, t wll be close to "". In ths wa, we can consder the bnar hsteress neuron s a specal case of contnuous hsteress neuron (Fg.3). The algorthms that how to update the network can be dscussed wth the applcaton to the crossbar swtch problem.

4 88 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar 00 Fg.3. hsteretc actvaton functon 5. Applcaton to Crossbar Swtch Problem The problem of maxmzng the throughput of packets through a crossbar swtch s best descrbed b referrng to Fg., whch shows how requests to swtch packets through an crossbar swtch can be represented b an bnar request matrx R [][3]. Rows and columns of the matrx R are assocated wth nputs and outputs, respectvel, of the crossbar swtch. A matrx element r = ndcates that there s a request for swtchng at least one packet from nput lne to output lne of the swtch; otherwse r = 0. If we consder the crossbar swtch for pont-to-pont connectons, then at most one crosspont ma be closed on an row or column of the swtch durng packed transmsson. The state of the swtch can be represented b an bnar confguraton matrx C, where c = ndcates that nput lne s connected to output lne b the closed crosspont ( ). c = 0 ndcates that crosspont ( ) s open. For proper operaton of the swtch, there should be at most one closed crosspont n each row and each column. The throughput of the swtch s optmal when the matrx C, whch s a subset of the matrx R (.e., c r for ever (, )), contans at most a n each row/column, and has maxmum overlap wth R. Examples of optmal matrces are shown n Fg.4 for a 4 4 crossbar. Fg.4. Schematc archtecture of 4 4 crossbar control wth an example of nput request matrx and ts optmal confguraton matrces Each swtch nlet has a queue manager. When an nlet queue manager receves a packet, t examnes the packet s destnaton address and determnes ts swtch outlet. It then updates the row request vector for that nlet b settng to, the bt correspondng to the swtch outlet, and places the packet on the nlet queue. The crossbar swtch s controlled b a neural network that has one neuron n correspondence to each swtch crosspont. Row request vectors from all the nlets are suppled to the neural network, whch uses them to compute an optmal confguraton matrx for the swtch. The resultng row confguraton vectors are then returned to the correspondng queue manager, whle the crossbar swtch crossponts selected b the computed confguraton matrx are closed. Each queue manager presents to ts nlet a sngle packet destned to the outlet selected b the row vector returned b the neural network, whch thus gets routed through the closed crosspont to ts proper outlet. The queue manager also updates ts row request vector b clearng the selected column bt, provded that no packets reman queued for that output. Ths process s cclcal: new packets are receved whle queued packets are beng transmtted. If all packets were of a constant length, then t

5 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar would be possble to receve new packets, transmt selected packets, and compute the next confguraton n parallel. The computaton of an optmal confguraton matrx should be completed n a few mcroseconds, whch s less than t takes to transmt a packet n a hgh-speed fber optc based communcaton sstem [7][8]. The crossbar swtch problem can be solved b constructng an approprate energ functon and mnmzng the energ functon to zero (E=0) usng an two-dmensonal Hopfeld neural network,.e., a matrx V = [ ]. The obectve energ functon of the crossbar swtch problem s gven b [8]: A E = B + k = k = = k = k (4) where A and B are coeffcents, k s the output value of neuron k and k s the output value of neuron k. The frst term wll be zero f each row contans no more than one, wth all the other values beng zero. Smlarl, the second term s zero f each column contans no more than one. We can get the total nput ( x ) of neuron b usng the partal dervaton term of the energ functon. Thus, the total nput ( x ) of neuron s gven: convergence of the Hopfeld neural networks wth contnuous hsteress neurons. In the smulatons, a 00- neuron Hopfeld neural network wth contnuous hsteress neurons ( =,,, 00) was chosen. Intal parameters of the network, connecton weghts and thresholds were randoml generated unforml between.0 and.0. Smulatons on a randoml generated 00- neuron Hopfeld network wth dfferent value of α and β ( α = β = 0 andα = 0.6, β = 0. 6 for =,, 00) were also carred out. Fg.5 shows the convergence charactorstcs of both networks. From ths fgure we can see that both the Hopfeld neural networks wth contnuous hsteress neurons ( α = 0.6, β = 0. 6 ) converged to stable states that dd not further change wth tme. It s worth to note that the Hopfeld neural network wth contnuous hsteress neurons ( α = 0.6, β = 0. 6 ) seek out a smaller mnmum at seek out E = than the Hopfeld neural network wthout hsteress neurons at E = ( α = β = 0 ). x = A k B k (5) k = k = For a gven matrx R,.e., an matrx V, updatng the matrx element b Eqs. (5)(6)(7) and (5) can produce a new matrx V, whch s ether an optmal or a local mnmal matrx,.e., a subset of the matrx V or R b the convergence characterstcs of the Hopfeld networks wth contnuous hsteress neurons. 5.Smulaton Results Snce our archtecture s vald for an Hopfeld neural networks as llustrated n the prevous secton, we used the archtecture for some randoml generated problems and a large number of real crossbar swtch problems up to swtches. Experments were frst performed to show the Fg. 5 The convergence characterstc of a 00-neuron Hopfeld network wth and wthout the contnuous hsteress neurons. The Hopfeld neural network wth contnuous hsteress neurons were also appled to crossbar swtch problem and smulatons were also carred out. The proposed algorthm was mplemented n C++ on PC Staton (PentumIIII 3GHz). A sgmod functon was used as ntput/output functon and the temperature parameter γ = γ α β was set to We now present smulaton results of the archtecture when appled to real crossbar swtch problem. The parameters A and B were set to A=, B=0.5. The weghts and external nput currents were all the same as the orgnal Hopfeld neural network. Contnuous hsteress neurons was set to α = 0.6, β = The network archtectures for 4 knds of nstances from 4 4 to

6 90 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar crossbar swtches were smulated on a dgtal computer. In smulatons, 00 smulaton runs wth dfferent randoml generated ntal states were performed on each of these nstances. Eq.(5)(6)(7) were used as the nput/output functon. Usng Eq.(5), all neurons were computed exactl once n one teraton step. The maxmum teraton step was set to 000. When the teraton steps exceeded the maxmum teraton step, the teraton was termnated. In Table., the column labeled optmal s the global convergence tmes among 00 smulatons and the column labeled steps s the average number of teraton steps requred for the convergence n the 00 smulatons. The smulaton results show that the archtectures wth contnuous hsteress neurons (CH) could almost fnd optmum soluton to all crossbar swtch problems wthn short computaton tmes. Crossbar Swtches Table. Computatonal results. Hsteress Bnar Hopfeld Proposed Archtecture Optmal(%) Steps Optmal(%) Steps We compared our results wth that found b the bnar hsteress Hopfeld neural network []. Table shows the results b the two dfferent networks: the bnar hsteress Hopfeld neural network and our network archtecture, where the convergence rates and the average number of teraton steps requred for the convergence are summarzed. From Table., we can see that the Hopfeld neural network archtecture wth contnuous hsteress neurons (CH) was ver effectve, and was better than the bnar hsteress Hopfeld neural network n terms of the computaton tme and the soluton qualt for crossbar swtch problem. Further, the number of teraton steps of our archtecture was almost ndependent on the problem sze, whle that the bnar hsteress Hopfeld neural network was three steps more than our network archtecture when we reach optmum soluton to the large sze. 6.Conclusons We have proposed a contnuous hsteress Hopfeld neural network archtecture for the crossbar swtch problem, and showed ts effectveness b smulaton experments. The proposed archtecture was based on a modfed Hopfeld neural network n whch the contnuous hsteress neurons(ch) were added to mprove soluton qualt. We proved theoretcall that the emergent collectve propertes of the orgnal Hopfeld neural network also were present n the Hopfeld neural network archtectures wth contnuous hsteress neurons (CH). In order to verf the performance of the proposed archtecture, we have tested the archtectures wth a large number of randoml generated examples and crossbar swtch problems up to swtches. The smulaton results showed that the Hopfeld neural network archtecture wth contnuous hsteress neurons(ch) was much better than the prevous works ncludng the Hopfeld neural network wth hsteress bnar neurons for crossbar swtch problem n terms of both the computaton tme and the soluton qualt. References [] J.J. Hopfeld, eural network and phscal sstems wth emergent collectve computatonal abltes, Proc. atl. Acad. Sc. USA. Vol.79 pp , 98. [] J.J. Hopfeld, eurons wth graded response have collectve computatonal propertes lke those of two-state neurons, Proc. atl. Acad. Sc. USA. Vol.8 pp , 984. [3] J.J. Hopfeld and D.W.Tank, eural computaton of decsons n optmzaton problems, Bol. Cbern., Vol.5 pp.4-5, 985. [4] J.J. Hopfeld and D.W.Tank, Computng wth neural crcuts: a model, Scence, no.33, pp ,986. [5] D.W. Tank and J.J. Hopfeld, Smple neural optmzaton network: An A/D converter, sgnal decson crcut, and lnear programmng crcut, IEEE Trans, Crcuts & Sstems, vol.cas-33, no.5, pp ,986. [6] Y. Takefu and K.C. Lee, An hsterss bnar neuron: A model suppressng the oscllator behavor of neural dnamcs, Bol. Cbern., Vol.64 pp , 99. [7] M. Tatesh and S. Tamura, Comments on `Artfcal neural networks for four-colorng map problems and K-colorablt problems', IEEE Trans. Crc. Sst.-I: Fundamental Theor and Applcatons, vol. 4, no. 3, pp , March, 994. [8] L. Wang, Dscrete-tme convergence theor and updatng rules for neural networks wth energ functons, IEEE Trans. eural etworks, vol.8, no.5, pp , 997. [9] G. Galán-Marín, J. Muñoz-Pérez, A new nput-output functon for bnar Hopfeld neural networks, Proceedngs of the Internatonal Work-Conference on Artfcal and

7 IJCSS Internatonal Journal of Computer Scence and etwork Securt, VOL.0 o., Januar 00 9 atural eural etworks: Foundatons and Tools for eural Modelng, pp.3-30, June 0-04, 999. [0] S. Bhartkar, J. M. Mendel, The hsteretc Hopfeld neural network, IEEE Trans. eural etworks, vol., no.4, pp , 000. []Guangpu Xa, Zheng Tang, Ronglong Wang and Yong L, Hopfeld eural etwork wth Hsteress for maxmum Cut Problem, eural Informaton Processng - Letters and Revews, Vol.4, o., pp. 9-6,004. [] X.C.Zeng and T. Martnez, A new relaxaton procedure n the Hopfeld neural networks for solveng Optmzaton Problems, euron Processng Letters, 0, pp.-,999. [3] S. Matsuda, Capabltes of hopfeld neural networks for searchng the optmal soluton to n-rook problem and crossbar swtch problem (n Japanese), Trans. of Inst. Of Electroncs, Informaton and Communcaton Engneers, J80- D-II, o. 9, pp ,997. [4] S. Matsuda, Guaranteeng effcenc and safet of a hopfeld neural network for crossbar (n Japanese), Trans. Of Inst. Of Electroncs, Informaton and Communcaton Engneers, J80-D-II, o., pp ,997. [5] M.M. Al and H.T. guen, A neural network mplementaton of an nput access scheme n a hgh-speed packet swtch, IEEE Global Telecommuncatons Conference and Expo, pp. 9-96,989. [6] R.J.T. Morrs and B. Samad, eural network control of communcatons sstems, IEEE Trans. on eural etworks, 5, o. 4, pp ,994. [7] A. Marrakch and T. Troudet, A neural net arbtrator for large crossbar packet-swtches, IEEE Trans. On Crcuts and Sstems, 36, o. 7, pp ,989. [8] T.P. Troudet and S.M. Walters, eural network archtecture for crossbar swtch control, IEEE Trans. On Crcuts and Sstems, 38, o., pp. 4-56,99. [9] Y. Takefu, eural etwork Parallel Computng, Kluwer Academc Publshers,99. We Zhang receved a B.S. degree from Zheang ormal Unverst, Zheang, Chna n 004. She receved an M.S. degree from Toama Unverst, Toama, Japan, n 007. Form 007 she s workng toward the.d.e. degree at Toama Unverst, Japan. Her man research nterests are neural networks and optmzaton problems. Zheng Tang receved a B.S. degree from Zheang Unverst, Zheang, Chna n 98 and an M.S. degree and a D.E. degree from Tsnghua Unverst, Beng, Chna n 984 and 988, respectvel. From 988 to 989, he was an Instructor n the Insttute of Mcroelectroncs at Tsnghua Unverst. From 990 to 999, he was an Assocate Professor n the Department of Electrcal and Electronc Engneerng, Mazak Unverst, Mazak, Japan. In 000, he oned Toama Unverst, Toama, Japan, where he s currentl a Professor n the Department of Intellectual Informaton Sstems. Hs current research nterests nclude ntellectual nformaton technolog, neural networks, and optmzatons.