CS407 Neural Computation

Transcription

1 CS407 Neural Computaton Lecture 8: Neural Netorks for Constraned Optmzaton. Lecturer: A/Prof. M. Bennamoun

2 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng

3 Introducton There are nets that are desgned for constraned optmzaton problems (such as the Travelng Salesman Problem, TSP). These nets have fed eghts that ncorporate nformaton concernng the constrants and the quantty to be optmzed. The nets terate to fnd a pattern of o/p sgnals that represents a soluton to the problem. E.g of such nets are the Boltzmann machne (thout learnng), the contnuous Hopfeld net, and several varatons (Gaussan and Cauchy nets). Other optmzaton problems to hch ths type of NNs can be appled to are: ob shop schedulng, space allocaton, 3

4 Travelng Salesman Problem (TSP) The am of the TSP s to fnd a tour of a gven set of ctes that s of mnmum length. A tour conssts of vstng each cty eactly once and returnng to the startng cty. The tour of mnmum dstance s desred. The dffculty of fndng a soluton ncreases rapdly as the number of ctes ncreases. Many approaches other than NNs to solve ths problem are etensvely reported n the lterature. 4

5 Introducton NN approach to constraned optmzaton Each unt represents a hypothess, th the unt on f the hypothess s true, off f the hypothess s false. The eghts are fed to represent both the constrants of the problem and the functon to be optmzed. The soluton of the problem corresponds to the mnmum of an energy functon or the mamum of a consensus functon for the net. NNs have several potental advantages over tradtonal technques for certan types of optmzaton problems. They can fnd near optmal solutons quckly for large problems. 5

6 Introducton NN approach to constraned optmzaton They can also handle stuatons n hch some constrants are eak (desrable but not absolutely requred). For e.g. n the TSP, t s physcally mpossble to vst ctes smultaneously, but t may be desrable to vst each cty only once. The dfference n these types of constrants could be reflected by makng the penalty for havng unts n the same column on smultaneously larger than the penalty for havng unts n the same ro on smultaneously. If t s more mportant to vst some ctes than others, these ctes can be gven larger self-connecton eghts. 6

7 Introducton NN archtecture for the TSP For n ctes, e use n unts, arranged n a square array. A vald tour s represented by eactly one unt beng on n each ro and n each column. To unts beng on n a ro ndcates that the correspondng cty as vsted tce; To unts beng on n a column shos that the salesman as n to ctes at the same tme. The unts n each ro are fully nterconnected; smlarly The unts n each column are fully nterconnected. The eghts are set so that unts thn the same ro (or the same column) ll tend not to be on at the same tme. In addton, there are connectons (see later). beteen unts n adacent columns and beteen unts n the frst and last columns, correspondng to the dstances beteen ctes 7

8 8 J,0 J,9 J,8 J,7 J,6 J,5 J,4 J,3 J, J, I,0 I,9 I,8 I,7 I,6 I,5 I,4 I,3 I, I, H,0 H,9 H,8 H,7 H,6 H,5 H,4 H,3 H, H, G,0 G,9 G,8 G,7 G,6 G,5 G,4 G,3 G, G, F,0 F,9 F,8 F,7 F,6 F,5 F,4 F,3 F, F, E,0 E,9 E,8 E,7 E,6 E,5 E,4 E,3 E, E, D,0 D,9 D,8 D,7 D,6 D,5 D,4 D,3 D, D, C,0 C,9 C,8 C,7 C,6 C,5 C,4 C,3 C, C, B,0 B,9 B,8 B,7 B,6 B,5 B,4 B,3 B, B, A,0 A,9 A,8 A,7 A,6 A,5 A,4 A,3 A, A, Cty J I H G F E D C B A Introducton NN archtecture for the TSP b b,,,n,,,n n, n, n,n b b b b b b b

9 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 9

10 Boltzmann machne The states of the unts of a Boltzmann machne NNs are bnary valued, th probablstc state transtons. The confguraton of the net s the vector of the states of the unts. The Boltzmann machne descrbed n ths lecture has fed eght, hch epress the degree of desrablty that unts X and X both be on. In applyng Boltzmann machne to constraned optmzaton problems, the eghts represent the constrants of the problem and the quantty to be optmzed. Note that the descrpton presented here s based on the mamzaton of a consensus functon (rather than the mnmzaton of a cost functon). The archtecture of a Boltzmann machne s qute general, consstng of a set of unts (X and X are representatve unts) a set of b-drectonal connectons beteen pars of unts. If unts X and X are connected,! 0. The b-drectonal nature of the connecton s often represented as 0

11 Boltzmann machne A unt may also have a self-connecton (or equvalently, there may be a bas unt, hch s alays on and connected to every other unt; n ths nterpretaton, the self-connecton eght ould be replaced by the bas eght). The state of unt X s ether ( on ) or 0 ( off ). The obectve of the NN s to mamze the consensus functon: C The sum runs over all unts of the net. The net fnds ths mamum (or at least a local mamum) by lettng each unt attempt to change ts state (from on to off or vce versa). The attempts may be made ether sequentally (one unt at a tme) or n parallel (several unts smultaneously). Only the sequental Boltzmann machne ll be dscussed here.

12 Boltzmann machne C If unt s on, If unt s off, 0

13 Boltzmann machne The change n consensus f unt X ere to change ts state (from to 0 or from 0 to ) s Contrbuton from all nodes hch are on and connected to thru C( ) [ ] + here s the current state of unt X. The coeffcent [ ] ll be + f unt X s currently off and f unt X s currently on X 0 X NOTE: that f unt X ere to change ts actvaton, the resultng change n consensus can be computed from nformaton that s local to unt X,.e. from eghts on connectons and actvatons of unts to hch unt X s connected (th 0 If unt X s not connected to unt X ). 3

14 Boltzmann machne Hoever, unt X does not necessarly change ts state, even f dong so ould ncrease the consensus of the net. The probablty of the net acceptng a change n state for unt X s A(, T) + ep C( ) T The control parameter T (called temperature) s gradually reduced as the net searches for a mamal consensus Loer values of T make t more lkely that the net ll accept a change of state that ncreases ts consensus and less lkely that t ll accept a change that reduces ts consensus. C T 0 ep 0 A(, T) (assumng C > 0) T 4

15 Boltzmann machne The use of a probablstc update procedure for the actvatons, th the control parameter decreasng as the net searches for the optmal soluton to the problem represented by ts eghts, reduces the chances of the net gettng stuck n a local mamum. Ths process of gradually reducng T s called smulated annealng. It s analogous to the physcal annealng process used to produce a strong metal (th a regular crystallne structure). Durng annealng a molten metal s cooled gradually n order to avod mperfectons n the crystallne structure of the metal due to freezng. 5

16 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 6

17 Boltzmann machne Archtecture Here s the archtecture of a Boltzmann machne for unts n a D array. The unts thn each ro are fully nterconnected. Smlarly, the unts thn each column are also fully nterconnected. The eghts on each of the connectons s p (here p>0). Each unt has a self-connecton, th eght b>0 A typcal unt s labelled, b b b,,,n b b,,,n b n, n, n,n b b b 7

18 Boltzmann machne Settng the eghts: Algorthm The eghts for a Boltzmann machne are fed so that the net ll tend to make state transtons toard a mamum of the consensus functon defned above. If e sh the net (shon n prevous slde) to have eactly one unt on n each ro and n each column, e must choose the values of the eghts p and b so that mprovng the confguraton corresponds to ncreasng the consensus. Each unt s connected to every other unt n the same ro th eght p (p > 0) Smlarly, each unt s connected to every other unt n the same column th eght p. The eghts are penaltes for volatng the condton at most one unt be on n each ro and each column. In addton, each unt has a self-connecton, of eght b>0. 8

19 Boltzmann machne Algorthm The self-connecton eght s an ncentve (bonus) to encourage a unt to turn on f t can do so thout causng more than one unt to be on n a ro or column. If p > b, the net ll functon as desred: If unt s off (u 0) and none of the unts connected to s on, changng the status of to on ll ncrease the consensus of the net by the amount b (hch s a desrable change). On the other hand, f one of the unts n ro or n column (say,, + s already on ), attemptng to turn unt on ould result n a change of consensus by the amount b. Thus, for b < 0 (.e. p>b), the effect ould be to decrease the consensus (the net ll tend to reect ths unfavorable change). Bonus and penalty connectons, th p>b, ll be used n the net for the TSP to represent the constrants for a vald tour. 9

20 Boltzmann machne Algorthm Applcaton procedure: The eght beteen unt and IJ s denoted (, ; I,J) (, (, ; I, J) p ;, ) b f I The applcaton procedure s as follos: or J (but not both); Step 0 Intalze eghts to represent the constrants of the problem Intalze the control parameter (temperature) T Intalze actvatons of unts (random bnary values). Step Whle stoppng condton s false, do steps -8 Step Step 3 Step 4 Do steps 3-6 n tmes (ths consttutes an epoch) Choose ntegers I and J at random beteen and n (unt IJ s the current canddate to change ts state) Compute the change n consensus that ould result: C( ) [ u ] ( I, J; I, J) + (, ; I, J) IJ u, I, J 0

21 Boltzmann machne Step 5 Step 6 Algorthm Compute the probablty of acceptance of the change: A( T) C( ) + ep T Determne hether or not to accept the change Let R be a random number beteen 0 and. If R<A, accept the change: u u (Ths changesthe state of unt ) I, J I, J I, J Step 7 If R> A, reect the proposed change. Reduce the control parameter T ( ne) 0.95T( old) Step 8 Test stoppng condton: If there has been no change of state for a specfed number of epochs, or f the temperature has reached a specfed value, stop; otherse contnue.

22 Boltzmann machne Intal Temperature: Algorthm The ntal temperature should be taken large enough so that the probablty of acceptng a change of state s appromately 0.5, regardless of hether the change s benefcal or detrmental. T ep C T A(, T) 0.5 Hoever, snce a hgh startng temperature ncreases the requred computaton tme sgnfcantly, a loer ntal temperature may be more practcal n some applcatons.

23 Boltzmann machne Coolng schedule Algorthm Theoretcal results sho that the temperature should be cooled sloly accordng to the logarthmc formula: here k s an epoch. T0 T B ( k) log( + k) Eponental coolng schedule can be used: T( ne) α T( old) here the temperature s reduced after each epoch. A larger α (such as α 0.98) allos for feer epochs at each temperature A smaller α (such as α 0.9) may requre more epochs at each temperature. 3

24 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 4

25 Boltzmann machne Applcaton TSP Nomenclature: n number of ctes n the tour (there are n unt n the net) nde desgnatng a cty; n nde desgnatng poston n tour, mod n;.e. n +, 0 n, unt representng the hypothess that the th cty s vsted at the th step of the tour u, actvaton of unt, ; u, f the hypothess s true, u, 0 f the hypothess s false d,k dstance beteen cty and cty k, k d mamum dstance beteen ctes. 5

26 Boltzmann machne Applcaton TSP Archtecture: For ths applcaton t s convenent to arrange the unts of the NN n a grd (Fg. belo). The ros of the grd represent ctes to be vsted The columns the poston of a cty n the tour. Cty A B C D E F G H I J Poston A, B, C, D, E, F, G, H, I, J, A, B, C, D, E, F, G, H, I, J, 3 A,3 B,3 C,3 D,3 E,3 F,3 G,3 H,3 I,3 J,3 4 A,4 B,4 C,4 D,4 E,4 F,4 G,4 H,4 I,4 J,4 5 A,5 B,5 C,5 D,5 E,5 F,5 G,5 H,5 I,5 J,5 6 A,6 B,6 C,6 D,6 E,6 F,6 G,6 H,6 I,6 J,6 7 A,7 B,7 C,7 D,7 E,7 F,7 G,7 H,7 I,7 J,7 8 A,8 B,8 C,8 D,8 E,8 F,8 G,8 H,8 I,8 J,8 9 A,9 B,9 C,9 D,9 E,9 F,9 G,9 H,9 I,9 J,9 0 A,0 B,0 C,0 D,0 E,0 F,0 G,0 H,0 I,0 J,0 6

27 Boltzmann machne Applcaton TSP, has a self-connecton of eght b; ths represents the desrablty of vstng cty, at stage., s connected to all other unts n ro th penalty eght p; ths represents the constrants that the same cty s not to be vsted tce., s connected toall other unts n column th penalty eght p; ths represents the constrant that ctes cannot be vsted smultaneously., s connected to k,+ for k n, k, th eght d, Ths represents the dstance traveled n makng the transton from cty at stage to cty k at stage +, s connected to k,- for k n, k, th eght d, Ths represents the dstance traveled n makng the transton from cty k at stage - to cty at stage k k 7

28 Boltzmann machne Applcaton TSP Settng the eghts: The desred net ll be constructed n steps Frst, a NN ll be formed for hch the mamum consensus occurs henever the constrants of the problem are satsfed..e. hen eactly one unt s on n each ro and n each column. Second, e ll add eghted connectons to represent the dstances beteen the ctes. In order to treat the problem as a mamum consensus problem, the eghts representng dstances ll be negatve. A Boltzmann machne th eghts representng the constrants (but not the dstances) for the TSP s shon belo. If p>b, the net ll functon as desred (as eplaned earler). To complete the formulaton of a Boltzmann NN for the TSP, eghted connectons representng dstances must be ncluded. For ths purpose, a typcal unt, s connected to the unts k,- and k,+ (for all k ) by eghts that represent the dstances beteen cty and cty k 8

29 Boltzmann machne Applcaton TSP b b b,,,n b b,,,n b n, n, n,n b b b 9

30 Boltzmann machne Applcaton TSP The dstance eghts are shon on the fgure belo for the typcal unt,,- -d,, -d,,+ k,- k,+ -d k, -d k,,-,,+ -d n, -d n, n,- n, n,+ Boltzmann NN for the TSP; eghts represent the dstances for unt, 30

31 Boltzmann machne Applcaton TSP NOTE: nts n the last column are connected to unts n the frst column by connectons representng the approprate dstances. Hoever, unts n a partcular column are not connected to unts n columns other than those mmedately adacent to the sad column. We no consder the relaton beteen the constrant eght b and the dstance eghts. Let d denote the mamum dstance beteen any ctes n the tour. Assume that no cty s vsted n the th poston of the tour and that no cty s vsted tce. In ths case, some cty, say s not vsted at all;.e. no unt s on n column or n ro. Snce allong, to turn on should be encouraged, the eghts should be set so that the consensus ll be ncreased f t turns on. The change n consensus ll be b d, k d, k here k ndcates the cty vsted at stage - of the tour k denotes the cty vsted at stage + (and cty s vsted at stage ). Ths change > b-d (ths change should be > even for mamum dstance beteen ctes, d) 3

32 Boltzmann machne Applcaton TSP Hoever equalty ll occur only f the ctes vsted n postons - and + are both the mamum dstance d, aay from cty In general, requrng the change n consensus to be postve ll suffce, so e take b>d. Thus, e see that f p>b, the consensus functon has a hgher value for a feasble soluton (one that satsfes the constrants) than for a non-feasble soluton If b>d the consensus ll be hgher for a short feasble soluton than for a longer tour. In summary: p>b >d 3

33 Boltzmann machne Analyss The TSP s a nce model for a varety of constraned optmzaton problems. It s hoever a dffcult problem for the Boltzmann machne, because n order to go from one vald tour to another, several nvald tours must be accepted. By contrast, the transton from vald solutons to vald soluton may not be as dffcult n other constraned optmzaton problems. Equlbrum: The net s n thermal equlbrum (at a partcular temperature) hen the probs P α and P β of confguratons of the net, α and β, obey the Boltzmann dstrbuton P P α β E ep β E T α E E α β energy of energy of confgα confgβ 33

34 Boltzmann machne Analyss At hgher temperatures, the probs of dfferent confgs are more nearly equal T ep(.) P ~ α P β At loer temperatures, there s a stronger bas toard confs th loer energy. Startng at a suffcently hgh temperature ensures that the net ll have appromately equal probs of acceptng or reectng any proposed state transton. If the temp s reduced sloly, the net ll reman n equlbrum at loer temps. It s not practcal to verfy drectly the equlbrum condton at each temp, as there are too many possble confguratons. 34

35 Boltzmann machne Energy functon: The energy of a confguraton can be defned as: E < + Analyss Where θ s a threshold and self-connectons (or bases) are not used. The dfference n energy beteen a confg th unt X k off and one th X k on (and the state of all other unts remanng unchanged) s E( k) θ k If the unts change ther actvatons randomly and asynchronously and the net alays moves to a loer energy (rather than movng to a loer energy th a probablty that s less than ), the dscrete Hopfeld net results. + θ k 35

36 Boltzmann machne Analyss To smplfy notaton, one may nclude a unt n the net that s connected to every other unt and s alays on. Ths allos the threshold to be treated as any other eght, so that E < The energy gap beteen the conf th unt X k off and that th unt X k on s E ( k) k 36

37 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 37

38 Contnuous Hopfeld net A modfcaton of the dscrete Hopfeld net, th contnuousvalued output functons, can be used ether for assocatve memory problems (as th the dscrete form) or constraned optmzaton problems such as the TSP. As th the dscrete Hopfeld net, the connectons beteen unts are bdrectonal. So that the eght matr s symmetrc;.e. the connecton from unt to unt (th eght ) s the same as the connecton from to (th eght ). For the contnuous Hopfeld net, e denote the nternal actvty of a neuron as u ; Its output sgnal s v g u ) If e defne an energy functon ( E n n 0.5 v v + θ v n 38

39 Contnuous Hopfeld net E 0.5 n n n v v + n θ v for E E ( θ v + θ v ) ( v v + v v + ) + ( θ v + θ v ) v v + v v + for θ θ 39

40 Contnuous Hopfeld net Then the net ll converge to a stable confguraton that s a mnmum of the energy functon as long as n n n de 0 But E 0.5 dt vv + θv de dt E v dv du du dt E v g '( u net v ) > dv du + θ 0, E v du dt net (chan rule) E s a Lyapounov Energy functon Hence de dt g '( u )( net ) 0 40

41 Contnuous Hopfeld net For ths form of the energy functon, the net ll converge f the actvty of each neuron changes th tme accordng to the dfferental equaton du dt E v n v θ In the orgnal presentaton of the contnuous Hopfeld net, the energy functon as: E 0.5 n n v v n θ v + τ n v 0 g ( v) dv Tme constant 4

42 4 Contnuous Hopfeld net If the actvty of each neuron changes th tme accordng to the dfferental equaton the net ll converge. In the Hopfeld-Tank soluton of the TSP, each unt has ndces. The frst nde, y, etc. denotes the cty, The second,, etc, denotes the poston n the tour. The Hopfeld-Tank energy functon for the TSP s: + + n v u dt du θ τ y y y y y y v v v d D v N C v v B v v A E ) (,,,,,,,,,

43 43 Contnuous Hopfeld net The dfferental equaton for the actvty of unt X,I s The o/p sgnal s gven by applyng the sgmod functon (th range beteen 0 and ), hch Hopfeld and Tank epressed as X y I y I y y X I X y I y XJ I X I X v v d D v N C v B v A u dt du ( ),,,,,,, τ [ ] u u g v α tanh( 0.5 ) ( +

44 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 44

45 Contnuous Hopfeld net Approach Formulate the problem n terms of a Hopfeld energy of the form: n n E 0.5 v v + θ v n Problem Formulaton by Hopfeld Energy Energy Mnmzaton By Hopfeld Soluton State 45

46 46 Contnuous Hopfeld net Archtecture for TSP The unts used to solve the 0-cty TSP are arranged as shon J,0 J,9 J,8 J,7 J,6 J,5 J,4 J,3 J, J, I,0 I,9 I,8 I,7 I,6 I,5 I,4 I,3 I, I, H,0 H,9 H,8 H,7 H,6 H,5 H,4 H,3 H, H, G,0 G,9 G,8 G,7 G,6 G,5 G,4 G,3 G, G, F,0 F,9 F,8 F,7 F,6 F,5 F,4 F,3 F, F, E,0 E,9 E,8 E,7 E,6 E,5 E,4 E,3 E, E, D,0 D,9 D,8 D,7 D,6 D,5 D,4 D,3 D, D, C,0 C,9 C,8 C,7 C,6 C,5 C,4 C,3 C, C, B,0 B,9 B,8 B,7 B,6 B,5 B,4 B,3 B, B, A,0 A,9 A,8 A,7 A,6 A,5 A,4 A,3 A, A, Cty J I H G F E D C B A Poston

47 Contnuous Hopfeld net Archtecture for TSP The connecton eghts are fed and are usually not shon or even eplctly stated. The eghts for nter-ro connectons correspond to the parameter A n the energy equaton; There s a contrbuton to the energy f unts n the same ro are on. Smlarly, the nter-columnar connectons have eghts B; The dstance connectons appear n the fourth term of the energy equaton. More eplctly, the eghts beteen unts and y are (, : y, ) Aδ ( ) ( ) + ( + y δ Bδ δ y C Ddy δ, + δ, f δ s the DracDelta 0 otherse In addton each unt receves an eternal nput sgnal I +CN The parameter N s usually taken to be somehat larger than the number of ctes n 47

48 Contnuous Hopfeld net Algorthm for TSP Step 0 Intalze actvatons of all unts Intalze t to a small value Step Whle stoppng condton s false, do steps -6 Step Perform steps 3-5 n tmes (n s the number of ctes) Step 3 choose a unt at random Step 4 change actvty on selected unt: ( ne) u, ( old) + t u, ( old) A v B vy, + C N v D d y y u,, y( vy, + y, + v Step 5 apply output functon [ + u ] v, 0.5 tanh( α, Step 6 check stoppng condton 48

49 Contnuous Hopfeld net Algorthm for TSP Hopfeld and Tank used the follong parameter values n ther soluton of the problem: A B 500, C 00, D 500, N 5, α 50 The large value of α gves a very steep sgmod functon, hch appromates a step functon. The large coeffcents and a correspondngly small t result n very lttle contrbuton from the decay term ( ( old) t ) u, The ntal actvty levels (u, ) ere chosen so that v, 0 (the desred total actvaton for a vald tour). Hoever, some nose as ncluded so that not all unts started th the same actvty (or o/p sgnal). 49

50 Neural Nets for Constraned Optmzaton. Introducton Boltzmann machne Introducton Archtecture and Algorthm Boltzmann machne: applcaton to to the TSP Contnuous Hopfeld nets Contnuous Hopfeld nets: applcaton to to the TSP References and suggested readng 50

51 Suggested Readng. L., Fundamentals of Neural Netorks, Prentce-Hall, 994, Chapter 7. 5