ELUCIDATION OF TSP WITH SAMAN-NET

Transcription

1 ELUCIDATION OF WITH SAMAN-NET Naveen Kumar Sharma 1 IIMT College of Engneerng, Greater Noda Abstract: arous problems of combnatoral optmzaton and permutaton can be solved wth neural network optmzaton. Travelng salesman problem s an mportant example of ths category, whch s characterzed by ts large number of teratng degree of freedom. There are varous solutons of ths problem has been gven. These solutons are exact and heurstc methods, but all the exact approaches may be consdered for theoretcal nterest. In ths paper, we propose the Smulated Annealng of mean feld approxmaton for choosng the possble mnmum dstance path that satsfes all the necessary constrants. A energy functon wth mean feld approxmaton s proposed. And the annealng schedule s also defned dynamcally whch changes wth the dstance on each teraton of the process. The algorthm shows that ths approach generates optmal soluton for the aforesad problem. Keywords:, MFA, SA, and Optmzaton. I. INTRODUCTION Most of the tradtonal problems of combnatoral permutaton can be solved wth the help of ANN[1], as t s well known that ANN conssts of varous non-lnear processng unts []. These processng unts may be nterconnected through varous topologes [3]. One form of the topology s, feed back manner. In ths form, a set of processng unts, connected to each processng unt except to tself. The output of each unt s feed as nput to all other unts. Wth each lnk connectng between two unts, a weght s assocated, whch determnes the amount of output, a unt output provded as an nput to the other unts. The functon of a feedback network wth nonlnear unts can be descrbed n terms of the trajectory of the state of a network wth tme. By assocatng an energy functon wth each state, the trajectory descrbes a traversal along the energy landscape. The mnma of the energy landscape correspond to the stable states, whch can be used to store the gven nput patterns. The numbers of patterns that can be stored n the network depends upon the number of unts and the strength of the connectng lnks. The state of the network at successve nstants of tme.e. the trajectory of the states s determned by the actvaton dynamcs [4], for the network. Any pattern can be stored and recalled from such type of network [5]. Durng the process of the recallng the pattern, the network reaches to an equlbrum state [6], wth the actvaton and synaptc dynamcs. Assocated wth each output state s an energy [7], whch depends on the network parameters lke the weghts and bas, besdes the state of the network. The energy as a functon of the state of the network corresponds to an energy landscape. One of the most prevalent uses of Neural Network s neural optmzaton whch s a technque for solvng a problem by castng t nto a mathematcal equaton that, when ether mzed or mnmzed, solves the problem wthout gong nto detaled dynamcs of the concerned physcal system. In other words, one of the most successful applcatons of the neural network prncples s n solvng optmzaton problem [8, 9]. There are many stuatons where a problem may be formulated as Mnmzaton or Maxmzaton of some cost functon or objectve functon subject to constrants. It s possble to map such problem onto a feedback network, where the unts and connecton strengths are dentfed by comparng the cost functon of the problem wth the energy functon of the network expressed n terms of the states values of the unts and the connecton strength. It has been demonstrated [10] that how hghly All rghts Reserved 81

2 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: network of smple analog processor can collectvely compute good solutons to dfferent optmzaton problems. One of the most studed problems n the context of optmzaton usng the method of neural networks s the Travelng Salesman Problem, where the objectve s to fnd out the shortest route connectng all the ctes to be vsted by a salesman [11]. There are varous soluton of ths problem has been gven [1-0], there are varous attempts have been made for fndng the approprate soluton of Travelng Salesman Problem(, n whch randomzed mprovement heurstcs s a popular one, was proposed by Junger, Renelt and Rnald[1]. Some other approaches to solve extremely large s (havng tens to thousands or mllons of varables were proposed by Johnson, and Junger[13], Renelt and Rnald[1], genetc algorthmc and neural net approaches had been proposed by Potvn[14,15], smulated annealng approach had been proposed by Aarts, et al[16], and tabu search approach had been proposed by Fechter[17]. Performance guarantees for heurstcs had been gven by Johnson and Papadmtrou [18], the probablty analyss of heurstcs are gven by Karp and Steele [19] and the development emprcal testng of heurstcs s reported by Golden and Stewart [0]. Another method for solvng the problem was proposed by Behzad Kamgar-Pars and Behrooz Kamgar-Pars s based on analyzng dynamcal stablty of vald solutons, whch yelds relatonshps among search space for parameter values can t be arbtrary, thus the search space for parameter values becomes greatly restrcted and therefore fndng optmal values becomes much less tedous [1]. These solutons are exact and heurstc methods, but all the exact approaches are of consderable theoretcal nterest. The tradtonal method to solve such problems s gradent decent approaches of hll clmbng and a stochastc smulated annealng. Hopfeld and Tank proposed the soluton of, based on nherently parallel heurstc [] and the Mean Feld Annealng (MFA algorthm [, 3]. In ths paper, we propose the Smulated Annealng of Mean feld approxmaton method for descrbng the possble mnmum path that satsfyng all the necessary constrants on the problem. Energy functon for any Hopfeld type Feedback Neural Network can be represented that wll satsfy all the mposed constrants of the problem. A global constrant n the form of dstance of the traveled path (that s beng selected randomly can be selected for the Annealng schedule. The constrants can be reduced as per the schedule and correspondngly the energy functon s beng estmated. It can be seen that the possble mnmum energy functon wll represent the mnmum dstance path of the travelng salesman. II. SIMULATED ANNEALING OF MEAN FIELD APPROIMATION FOR The travelng sales man problem s an mportant example of the combnatoral optmzaton problem, whch s characterzed by ts large number of nteractng degrees of freedom. For a gven number of ctes (N and ther ntercty dstances, the objectve s to determne a closed loop of the tour of ctes, such that the total dstance s mnmzed subjected to the constrants, that each cty s vsted only once and all the ctes are covered n the tour. The Hopfeld memory can be used to solve ths problem. In ths process the characterstc of nterest s the rapd mnmzaton of the energy functon. To use the Hopfeld memory for the applcaton, we map the problem onto the Hopfeld type network archtecture. The frst term s to develop a representaton of the problem of the soluton that fts an archtecture havng a sngle array of the processng elements (PE. We develop t by allowng a set of N PEs to represent the N possble postons for a gven cty n the sequence of the tour. The weght matrx format can be found from the cty poston. The output wll be labeled as x, where the, subscrpt refers to the cty and the, subscrpt refers to the poston on the tour. To formulate the connecton weght matrx, the energy functon must be constructed that satsfyng the followng crtera: a. Energy mnma must favor states that have each cty only once on the tour. b. Energy mnma must favor states that have each poston on the tour only once. c. Energy mnma must favor states that nclude all N All rghts Reserved 8

3 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: d. Energy mnma must favor states wth the shortest total dstance. Devotng the state of a processng unt of a Hopfeld network as x = 1 ndcates that the cty s to be vsted at the th stage of the tour, the energy functon [4] can be wrtten as E A B Y 1 1 j 1, j 1 j 1 1 Y 1, Y C N N ( 1 1 n D d xy (, 1, 1 Y Y 1 Y 1, Y 1 (.1 The Mean feld Annealng algorthm [, 3] also proposed a soluton for ths problem and wth the MFA, the followng energy functon can be proposed [3] as, E d j j 1 d Y Y ( Y, 1 Y, 1 (. Where d s real constant, whch s slghtly larger then the largest dstance between the ctes n the gven nstance. So, n the equaton (., there s only two terms. The frst term s regardng feasblty, whch nhbts two ctes from beng n the same tour poston. The second summaton term s used for the mnmzaton of the tour length. The term d s used for balancng of the summaton terms. The output x of a neuron (, s nterpreted as the probablty of fndng cty n tour poston. The mean feld for a neuron (, s defned accordng to the energy functon gven n equaton (. as, E N N x d d Y Y ( Y Y, 1, 1 Y (.3 Intally all the neurons are arranged to the average value and the annealng schedule T wth the d. The weghts of the nterconnectons are ntalzed wth the small random numbers. As per the Hopfeld model the states of the any th neuron can be defne as, ( t 1 f [ W j j ( t] (.4 j As the dynamcs gven n equaton (.4 the network searches the stable state. Ths stable state may lead to a state correspondng to a local mnmum of the energy functon. In order to reach to the global mnmum,.e. the possble mnmum path traveled by the salesman, by passng the local mnma, we use the concept of stochastc updaton of the unt n the actvaton dynamcs of the network. In stochastc updaton, the state of a unt s updated usng the probablstc updaton, whch s controlled by the annealng schedule constrant parameter (T = d. At the lower value of the constrant parameter, the stochastc update approaches the determnstc update, whch s drected by the output functon of the unt. The probablty dstrbutons of the states at thermal equlbrum [14] can be wrtten as, Where Z s the partton functon. E / T 1 P( e (.5 All rghts Reserved 83

4 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: Intally the arbtrary cty s selected and the energy functon s constructed as gven n equaton (.. The Annealng schedule assgned wth the mum dstance of the path.e. to ts hgher value. So at hgher T, many states are lkely to be vsted, rrespectve of the energes of those states. Therefore for as per the Smulated Annealng Schedule, the value of T s gradually reduced, the output value of the states perturbs. Ths perturbaton contnues untl the network settles to an stable state or equlbrum state. So, the network estmates the energy functon for ths state and compares t wth the prevous energy functon by computng ΔE. If ΔE 0, we accept the soluton wth the hghest probablty.e. 1, otherwse we accept t wth the probablty gven n the equaton (.5. The state probabltes are computed by collectng the dstrbuton of the states for a large number of cycles of updates of the states of the network at a gven T. The cycles are repeated untl the probabltes of the states do not change substantally for the dfferent sets of cycles. On each teraton of ths process, the soluton s accepted wth the probablty 1, the Annealng Schedule parameter T s beng assgned wth the d of the ly constructed energy functon. Then every tme on the acceptable soluton wth hgher probablty the constrants parameter changes wth the ly found mum dstance. Ths ly found mum dstance would be less then the prevously found mum dstance. So the Smulated Annealng process wll contnue wth the value of the constrant parameter. Ths process s contnung tll the fnal value of the schedule. At ths state the unts of the network represents the state of equlbrum, whch wll represents the mnmum energy functon for the network. Thus the mnmum energy functon wll represent the possble shortest path for the travelng salesman problem. In order to speed up the process of the Smulated Annealng the Mean feld approxmaton s used [5], n whch the stochastc update of the bnary unts s replaced by determnstc analog states [6]. Thus the fluctuatng actvaton value of each unt s replaced wth ts average value. The equaton (.5 can be express wth ths method as, ( t 1 f [ W j j ( t ] f W j j ( t (.6 j j And from the stochastc updaton wth thermal equlbrum we have, 1 N ( t 1 tanh[ W j j ( t ] (.7 T j Ths equaton s solved teratvely startng wth some arbtrary values < (0> ntally. Once the steady equlbrum values of < > have been obtaned, the value of T s lowered. The next set of average states at thermal equlbrum s determned usng the average state values at the prevous thermal equlbrum condton of the ntal value < (0> n the equaton (.6 for teratve soluton. The set equaton of the Mean feld approxmaton s a result of mnmzaton of an effectve energy defned as the functon of T [7] as, 1 E( tanh[ ] (.8 T Where the effectve energy E (< > s the expresson for energy of the Hopfeld model usng averages for the state varables. Now the constrant parameter T decreases as per the Annealng schedule of Mean feld approxmaton, the state of the network perturbs wth the stochastc asynchronous updaton of the processng elements. So, as the output value perturbs, the neurons produces the updated states. The states updaton contnues untl the equaton (.6 All rghts Reserved 84

5 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: Hence, for the stablty condtons the energy functon on each teraton of annealng schedule of Mean feld approxmaton can be expressed as, E d j E Hence, the energy dfference P s mp 1 d ( j, 1, 1 Y Y Y Y (.9 can be computed as, old d N N E E E 1 old j + [(d d j Y Y Y ( old Y ][(, 1 Y, 1 old ( Y, 1 Y, 1 ] (.10 If the energy dfference ΔE 0, the probablty of acceptng the soluton becomes 1 otherwse t e E / T On each teraton wth the hghest probablty of acceptng the soluton, constrant. parameter of Annealng Schedule s defned as, T d (.11 On each teraton the constrant parameter s beng changed to the mnmum value wth respect to the prevous value. Hence, each tme the network determnes the stable condton wth the value of energy functon E and constrants parameter T and when E 0. The neurons produce the stable states that represent the poston of the cty wth mnmum dstance wth respect to the prevous poston. So, at the stablty wth E 0 the state of the neurons can be defned from the equaton (.8 as, d 1 [ x x j d j Y Y ( Y Y ] 1, 1, 1 x tanh d (.1 The Mean feld for the neuron (,, Where output< > can be nterpreted as the probablty of fndng cty n the th tour poston as defned from the equaton (.3 as, E N N d d Y ( Y Y Y, 1, 1 Y (.13 Thus, the entre process contnues untl fxed pont s found for the every value of constrant parameter T. In ths process of selectng a fxed pont, change n the energy s computed and the probablty of acceptng the pont n the mnmum dstance path can be found. The feld for the selected ponts wll also be computed. Ths entre process wll contnue for every schedule of the constrants parameter T, as t s changng wth the value of d, untl the T reaches to the fnal value. ALGORITHM The algorthm of the entre process can be proposed as: 1. Intalze all neurons on the stable state as; ( t 1 f [ W ( t] And ntalze the weghts and bas wth the small random number. The value of T s ntalzed wth any large random number. j All rghts Reserved 85

6 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: Do untl the fxed pont s found..1 Randomly select a cty say.. Compute the energy functon as; d 1 E j d Y ( Y Y, 1, 1 j Y Compute the state of the unt at equlbrum; 1 E x tanh[ d Also calculate the E usng equaton (.7.3 f E 0 accept wth P < (accept > =1 and set T= d else P < ] (accept > = exp (- E Compute the Mean feld as N Ex d d Y ( Y Y, 1 Y, 1 (The average of the output values of accepted fxed pont.e. neurons 3. If T reaches to the fnal value stop the processng otherwse decrease the T accordng to the annealng schedule and repeat the step. III. CONCLUSIONS In ths paper, we presented the method of mean feld approxmaton for determnng the possble mnmum length path for a travelng salesman. Ths mnmum length path wll satsfy all the constrants mposed on the route. The followng observaton can be made from the soluton: 1. The soluton of the problem s obtaned by determnng the stable state.e. the average functon of the neurons of the network usng stochastc asynchronous relaton procedure wth an annealng schedule.. It s also true that ths neural network approach does not yeld mnmum cost functon some as the tme. 3. For the large number of ctes, the optmum soluton for ths problem depends on the chan of the parameter used for the constrant terms and for mplementng the annealng process. 4. The energy functon s beng used that s dfferent from the old energy functon proposed by Hopfeld and Tank. Ths functon of Mean feld approxmaton nvolvng the less term wth respect to old energy functon. 5. The constrant parameter T of the Annealng Schedule wll be change on each teraton of the process wth the d dstance, whch s slghtly larger dstance between ctes that have selected for the path. Thus energy teraton wll be schedule wth the optmze value of T. Although the algorthm and the energy functon can successfully apply to the problem and almost optmal soluton could be found wth 30 ctes problem. The more experments and analytcal nvestgaton are stll requred for ncrease the effcency and speed of the soluton. REFERENCES [1] C.Peterson and B.Soderberg; Int.J.Neural systems 1 ( [] J.J.Hopfeld and D.W.Tank; Bologcal Cyber 5 ( [3] P.K.Smpson;Artfcal Neural Systems, Elmsford, Ny: pergamon press, 1990 [4] B.Kasko, Neural network for Sgnal Processng, Prentce hall, Englewood Clffs, N.J, All rghts Reserved 86

7 Internatonal Journal of Modern Trends n Engneerng and Research (IJMTER olume 03, Issue 03, [March 016] ISSN (Onlne: ; ISSN (Prnt: [5] J.A.Freeman and D.M.Skapura; Neural Networks, MA: Addsen Weslly, [6] M.A. Cohen and S.Grossberg; IEEE Trans. Systems, Mans Cybernetcs,13 ( [7] J.J.Hopfeld; proc.natonal Acad.sc.5(198 [8] Y..Andreyev, Y.L.Belsky, A.S.Durtren and D.A.Kumnov; IEEE Tr Neural Net 7( [9] C.Peaterson and B.Soderberg; Int.J.Neural System3 ( [10] J.J.Hopfeld and D.W.Tank; Bologcal Cyber 5 ( [11] C.Peterson; Neural Computaton ( [1] M. Jünger, G. Renelt and G. Rnald (1994. "The Travelng Salesman Problem," n Ball, Magnant, Monma and Nemhauser (eds., Handbook on Operatons Research and the Management Scences North Holland Press, [13] D. S. Johnson (1990. "Local Optmzaton and the Travelng Salesman Problem," Proc. 17th Colloquum on Automata, Languages and Programmng, Sprnger erlag, [14] J.. Potvn (1996 "Genetc Algorthms for the Travelng Salesman Problem", Annals of Operatons Research 63, [15] J.. Potvn (1993, "The Travelng Salesman Problem: A Neural Network Perspectve", INFORMS Journal on Computng [16] E.H.L. Aarts, J.H.M. Korst and P.J.M. Laarhoven (1988 "A Quanttatve Analyss of the Smulated Annealng Algorthm: A Case Study for the Travelng Salesman Problem", J. Stats. Phys. 50, [17] C.N. Fechter (1990 "A Parallel Tabu Search Algorthm for Large Scale Travelng Salesman Problems" Workng Paper 90/1 Department of Mathematcs, Ecole Polytechnque Federale de Lausanne, Swtzerland. [18] D. S. Johnson and C.H. Papadmtrou (1985. "Performance Guarantees for Heurstcs," n The Travelng Salesman Problem, Lawler, Lenstra, Rnooy Kan and Shmoys, eds., John Wley, [19] R. Karp and J.M. Steele (1985. "Probablstc Analyss of Heurstcs," n The Travelng Salesman Problem, Lawler, Lenstra, Rnnooy Kan and Shmoys, eds., John Wley, [0] B.L. Golden and W.R. Stewart (1985. "Emprcal Analyss of Heurstcs," n The Travelng Salesman Problem, Lawler, Lenstra, Rnooy Kan and Shmoys, eds., John Wley, [1] Behzad Kamgar-Pars and Behrooz Kamgar-Pars, an effcent model of neural networks for optmzaton Proc. IEEE Frst Internatonal Conference on Neural Networks (San Dego, 1987, vol.3, pp [] Bout, D.E. an den, and Mller III,T.K.Proceedngs of the ICNN, IEEE.( [3] G.Blbro, R.Mann, T.Mller, W.Snyder, D. E. an den Bout and M.Whte. In Advances n Neural Informaton Processng System1 ( [4] B.Muller and J.Renhardt; Neural Networks, New York. Sprnger erlog ( [5] C.Petrson and J.R.Anderson; Complex System, 1( [6] R.J.Glauber; J. Math.Phys.4 ( [7] S.Haykn; Neural Networks: A Comprehensve Foundaton, New York: Macmllan College Publshng Company Inc., All rghts Reserved 87