DETERMINATION OF PARAMETERS IN RELAXATION- SEARCH NEURAL NETWORKS FOR OPTIMIZATION PROBLEMS

Transcription

1 DETERMIATIO OF PARAMETERS I RELAXATIO- SEARCH EURAL ETWORKS FOR OPTIMIZATIO PROBLEMS Gursel Serpen Electrical Engineering & Computer Science Department The University of Toledo, Toledo OH 0 USA David L. Livingston Director of Engineering Division VWC College Ranoke, VA Azadeh Parvin The Civil Engineering Department The University of Toledo, Toledo OH 0 USA IC 97 - Houston, TX

2 HIGHLIGHTS A procedure Defines constraint weight parameters of the Hopfield network in order to establish the solutions of a given optimization problem as stable equilibrium points in the state space of the network dynamics Demonstration Application of the methodology is demonstrated on a well known benchmark problem, the Traveling Salesman Problem. Simulation Results Indicate that the proposed bounds on the constraint weight parameters establish the solutions as stable points and consequently, the Hopfield network consistently converges to a solution after each relaxation.

3 ITRODUCTIO Single-layer relaxation-type recurrent neural networks: Hopfield, Mean-Field Annealing, and Boltzmann Machine Uncertainty in determining values for the parameters of the network Convergence to IVALID solutions A procedure to determine values for the parameters of a single-layer recurrent neural network: speficially discretedynamics Hopfield network. It establishes solutions of a given optimization problem as stable equilibrium points in the state space of the network dynamics.

4 THE DISCRETE HOPFIELD ETWORK, for i =,,, Let s i represent a node output where s i = 0 and is the number of network nodes. Then, the equation given by E = wijsi sj bi si + Θ s i= j= i= i= i i, i j is the Liapunov function whose local minima are the final states of the network with node dynamics defined by s + = 0 if net < Θ, i k i k i s + = if net > Θ, and i k i k i i k i k i k i s + = s if net= Θ, i =,,,, where k is a discrete time index, and net = w s + b i k j= ij j k i with i j and Θ i is the threshold of node s i. THE DISCRETE HOPFIELD ETWORK

5 (cont d) The weight term is defined by Z ij = ϕ ϕ ϕ ij ij, ϕ= w g δ d where Z is the number of constraints. Given the set of constraints {,,, } C ϕ C C C Z, gϕ R if the hypotheses nodes si and s j represent for C ϕ are mutually supporting and gϕ R if the same hypotheses are mutually conflicting. The term δ ϕ ij is equal to if the two hypotheses represented by nodes si and s j are related under C ϕ and is equal to 0 otherwise. ϕ The d ij term is equal to for all i and j under a hard constraint and is a pre-defined cost for a soft constraint, which is typically associated with a cost term in optimization problems.

6 TRAVELLIG SALESPERSO PROBLEM Consider the following energy function proposed by Hopfield to map this problem to the network topology, E s gr ij r ( ) = δ sis j gcδ ij c sis j i= j= i= j= g η γ si M gηdij ijsis j δ, i= i= j= where δ δ r ij c ij = if row( i) = row( j) or δ = 0, otherwise; = if col( i) = col( j) or δ = 0, otherwise; { ( ) ( ) [ ( ) ( )]} δ η ij = if col i col j = row i row j δ η ij = 0, otherwise; r ij c ij or i j; g, g, gη R, d ij is the distance between cities row(i) r c and row(j), and superscripts/subscripts r, c, γ and η stand for row, column, global and distance inhibitions, respectively. Functions row(i) and row(j) return the row and column location of nodes i and j, respectively.

7 TRAVELLIG SALESPERSO PROBLEM (cont d) The row and column inhibition energy terms are minimum if, at most, one node is active for each row and column. The energy term associated with the constraint weight parameter g γ is minimum if exactly M nodes are active within the overall network topology. The energy term for the distance constraint is minimum if the solution has the minimum total distance. Comparison of this energy function with the generic energy function yields the following weight matrix entries, external bias terms, and thresholds: ij r ij r c ij c η η ij ij w = g δ + g δ + g δ d + g γ, b i ( ) = M g - γ and Θ i = 0 for i, j =,,, ; i j

8 Column Interaction Topology Row Interaction Topology Distance Interaction Topology Global Interaction Topology Consider the input to an inactive node within a solution ( n e t i < Θ i ). An inactive node receives inputs from two active nodes under the row and the column inhibition constraints. A total of M active nodes collectively contribute Mg γ to the input of an inactive node due to global inhibitory interaction. An inactive node receives inputs from two active nodes in two neighbooring columns under the inhibitory distance constraint. ote also that Θ i = 0 and b i = 0. ( M ) g γ for i =,,,. As a result, the inequality for an inactive node to be stable takes the form of where d min d i for i =,,,. g r + gc + gη d + gγ 0 min.

9 Column Interaction Topology Row Interaction Topology Distance Interaction Topology Global Interaction Topology An active node: net i > Θ i, for i =,,,. o nodes contribute to the input of the active node under the column and row inhibition constraints. other active nodes contribute ( M ) g γ. Two active nodes within the neighbooring columns under inhibitory distance constraint contribute. We also have Θ i = 0 and b i = 0. ( M) g γ for i =,,,. Then the inequality for the active node becomes g η d g γ max

10 BOUDS O COSTRAIT WEIGHT PARAMETERS The inequality for an inactive node to be stable takes the form of g r + gc + g η d min + g γ 0 where d min d i for i =,,,. This inequality is satisfied for any values of the constraint weight parameter magnitudes. The inequality for the active node in a solution to be stable g η d g γ max This inequality is the only one which needs to be satisfied for solutions to be the stable points of the network dynamics.

11 SIMULATIO RESULTS The TSP was mapped to discrete network dynamics for the problem sizes of 0, 0, 0, 0, and 0 cities and for three distinct operating points. The distances between cities are random variables uniform in the interval [0, ]. A total of 00 relaxations were realized for each problem size and operating point pair. The operating points employed in the experiments were chosen at the limiting points of the admissible subspace of the constraint weight parameter space. Specifically, small and large values for the difference given by g γ - g η d m ax were used as the test values for the operating points.

12 SIMULATIO RESULTS (cont d) Operating g r g c g η g γ Points Table. Operating Point Definitions. Operating Points Cities OP OP OP 0 00% 00% 00% 0 00% 00% 00% 0 00% 00% 00% 0 00% 00% 00% 0 00% 00% 00% Table. Convergence Rate vs. Operating Point TSP Size. The results show the network converged to a solution after each relaxation in all test cases.

13 SIMULATIO RESULTS (cont d) The quality of the solutions, the measure of which is the ormalized Total Distance (TD), was observed for various problem sizes. The TD is computed by dividing the total distance of a solution by the number of cities. The problem sizes used in the tests are 0, 0, and 0 cities. A total of 00 relaxations were performed for each problem size. Distance Distribution umber of Solutions Cities 0 Cities 0 Cities 0 [0.,0.) [0.,0.) [0.,0.) [0.,0.) [0.,0.7] ormalized Total Distance Figure. Quality of Solutions vs. Problem Size.

14 COCLUSIOS Simulation study conducted on TSP for up to 0 cities indicated that the proposed bounds on constraint weight parameters led the Hopfield network towards a solution after each relaxation. The procedure essentially made the solution point set equal to stable point set for the TSP. The same procedure generally establishes the solution point set as a subset of the stable point set: some non-solution points become stable and the network might converge to a nonsolution in that case.