The Minimum Lel Spnning Tree Prolem: Illustrting the Utility of Genetic Algorithms Yupei Xiong, Univ. of Mrylnd Bruce Golden, Univ. of Mrylnd Edwrd Wsil, Americn Univ. Presented t BAE Systems Distinguished Speker Series, Mrch 2006
Outline of Lecture 10 - Minute Introduction to Grph Theory nd Complexity Introduction to the MLST Prolem A GA for the MLST Prolem Four Modified Versions of the Benchmrk Heuristic A Modified Genetic Algorithm Results nd Conclusions 2
Defining Trees A grph with no cycles is cyclic A tree is connected cyclic grph Some exmples of trees A spnning tree of grph G contins ll the nodes of G 3
Spnning Trees 2 2 2 1 4 6 3 5 1 4 6 5 3 1 4 6 3 5 7 7 7 Grph G A spnning tree of G Another spnning tree of G 4
Miniml Spnning Trees A network prolem for which there is simple solution method is the selection of minimum spnning tree from n undirected network over n cities The cost of instlling communiction link etween cities i nd j is c ij = c ji 0 Ech city must e connected, directly or indirectly, to ll others, nd this is to e done t minimum totl cost Attention cn e confined to trees, ecuse if the network contins cycle, removing one link of the cycle leves the network connected nd reduces cost 5
A Miniml Spnning Tree 4 6 6 Originl Network 1 4 3 5 7 2 1 9 7 2 2 5 3 1 4 6 6 1 4 3 5 7 2 1 9 7 5 3 2 1 2 Minimum Spnning Tree 6
The Trveling Slesmn Prolem Imgine suurn college cmpus with 140 seprte uildings scttered over 800 cres of lnd To promote sfety, security gurd must inspect ech uilding every evening The gol is to sequence the 140 uildings so tht the totl time (trvel time plus inspection time) is minimized This is n exmple of the well-known TSP Originl prolem Possile solution 7
Definitions Anlysis of Algorithms Algorithm- method for solving clss of prolems on computer Optiml lgorithm verifile optiml solution Heuristic lgorithm fesile solution Performnce Mesures Numer of sic computtions / Running time Computtionl effort --- Prolem size --- Plyer one --- Plyer two 8
Computtionl Effort s Function of Prolem Size Computtionl effort 10,000,000 1,000,000 2 n n nlog2n n2 n3 2^n n 3 100,000 10,000 1,000 100 10 n 2 nlog 2 n n 0 20 40 60 80 100 120 n Prolem size 9
Terminology Good vs. Bd Algorithms Reserchers hve emphsized the importnce of finding polynomil time lgorithms, y referring to ll such polynomil lgorithms s inherently good Algorithms tht re not polynomilly ounded, re leled inherently d Good Optiml Algorithms Exist for these Prolems Trnsporttion prolem Miniml spnning tree prolem Shortest pth prolem Liner progrmming 10
High Qulity Heuristic Algorithms Good Optiml Algorithms Don t Exist for these Prolems Trveling slesmn prolem (TSP) Minimum lel spnning tree prolem (MLST) Why Focus on Heuristic Algorithms? For the ove prolems, optiml lgorithms re not prcticl Efficient, ner optiml heuristics re needed to solve rel-world prolems The key is to find fst, high-qulity heuristic lgorithms 11
One More Concept from Grph Theory A disconnected grph consists of two or more connected grphs Ech of these connected sugrphs is clled component A disconnected grph with two components 12
Introduction The Minimum Lel Spnning Tree (MLST) Prolem Communictions network design Edges my e of different types or medi (e.g., fier optics, cle, microwve, telephone lines, etc.) Ech edge type is denoted y unique letter or color Construct spnning tree tht minimizes the numer of colors 13
A Smll Exmple Introduction Input 1 c e 6 e d Solution 1 6 e e 2 5 d 3 4 2 5 3 4 14
Where did we strt? Literture Review Proposed y Chng & Leu (1997) The MLST Prolem is NP-hrd Severl heuristics hd een proposed One of these, MVCA (mximum vertex covering lgorithm), ws very fst nd effective Worst-cse ounds for MVCA hd een otined 15
Literture Review An optiml lgorithm (using cktrck serch) hd een proposed On smll prolems, MVCA consistently otined nerly optiml solutions A description of MVCA follows 16
Description of MVCA 0. Input: G (V, E, L). 1. Let C { } e the set of used lels. 2. repet 3. Let H e the sugrph of G restricted to V nd edges with lels from C. 4. for ll i L C do 5. Determine the numer of connected components when inserting ll edges with lel i in H. 6. end for 7. Choose lel i with the smllest resulting numer of components nd do: C C {i}. 8. Until H is connected. 17
How MVCA Works Input 1 c e 6 e d 2 5 d 3 4 Intermedite Solution 1 6 2 5 3 4 Solution 1 6 e e 2 5 3 4 18
Worst-Cse Results 1. Krumke, Wirth (1998): MVCA OPT 1+ 2ln n 2. Wn, Chen, Xu (2002): MVCA OPT 1+ ln ( n 1) 3. Xiong, Golden, Wsil (2005): MVCA OPT H = 1 < 1+ ln i i= 1 where = mx lel frequency, nd H = th hrmonic numer 19
Some Oservtions The Xiong, Golden, Wsil worst-cse ound is tight Unlike the MST, where we focus on the edges, here it mkes sense to focus on the lels or colors Next, we present genetic lgorithm (GA) for the MLST prolem 20
Genetic Algorithm: Overview Rndomly choose p solutions to serve s the initil popultion Suppose s [0], s [1],, s [p 1] re the individuls (solutions) in genertion 0 Build genertion k from genertion k 1 s elow For ech j etween 0 nd p 1, do: End For t [ j ] = crossover { s [ j ], s [ (j + k) mod p ] } t [ j ] = muttion { t [ j ] } s [ j ] = the etter solution of s [ j ] nd t [ j ] Run until genertion p 1 nd output the est solution from the finl 21 genertion
Crossover Schemtic (p = 4) Genertion 0 S[0] S[1] S[2] S[3] Genertion 1 S[0] S[1] S[2] S[3] Genertion 2 S[0] S[1] S[2] S[3] Genertion 3 S[0] S[1] S[2] S[3] 22
Crossover Given two solutions s [ 1 ] nd s [ 2 ], find the child T = crossover { s [ 1 ], s [ 2 ] } Define ech solution y its lels or colors Description of Crossover. Let S = s [ 1 ] s [ 2 ] nd T e the empty set. Sort S in decresing order of the frequency of lels in G c. Add lels of S, from the first to the lst, to T until T represents fesile solution d. Output T 23
An Exmple of Crossover s [ 1 ] = {,, d } s [ 2 ] = {, c, d } d d d d c c c T = { } S = {,, c, d } Ordering:,, c, d 24
An Exmple of Crossover T = { } T = {, } T = {,, c } c c c 25
Muttion Given solution S, find muttion T Description of Muttion. Rndomly select c not in S nd let T = S c. Sort T in decresing order of the frequency of the lels in G c. From the lst lel on the ove list to the first, try to remove one lel from T nd keep T s fesile solution d. Repet the ove step until no lels cn e removed e. Output T 26
An Exmple of Muttion S = {,, c } S = {,, c, d } d d c c c Add { d } c c c Ordering:,, c, d 27
An Exmple of Muttion Remove { d } S = {,, c } Remove { } S = {, c } c c c c c c T = {, c } 28
Three Modified Versions of MVCA Voss et l. (2005) implement MVCA using their pilot method The results were quite time-consuming We dded prmeter ( % ) to improve the results Three modified versions of MVCA MVCA1 uses % = 100 MVCA2 uses % = 10 MVCA3 uses % = 30 29
MVCA1 We try ech lel in L (% = 100) s the first or pilot lel Run MVCA to determine the remining lels We output the est solution of the l solutions otined For lrge l, we expect MVCA1 to e very slow 30
MVCA2 (nd MVCA3) We sort ll lels y their frequencies in G, from highest to lowest We select ech of the top 10% (% = 10) of the lels to serve s the pilot lel Run MVCA to determine the remining lels We output the est solution of the l/10 solutions otined MVCA2 will e fster thn MVCA1, ut not s effective MVCA3 selects the top 30% (% = 30) nd exmines 3l/10 solutions MVCA3 is compromise pproch 31
A Rndomized Version of MVCA (RMVCA) We follow MVCA in spirit At ech step, we consider the three most promising lels s cndidtes We select one of the three lels The est lel is selected with pro. = 0.4 The second est lel is selected with pro. = 0.3 The third est lel is selected with pro. = 0.3 We run RMVCA 50 times for ech instnce nd output the est solution 32
A Modified Genetic Algorithm (MGA) We modify the crossover opertion descried erlier We tke the union of the prents (i.e., S = S 1^S 2 ) s efore Next, pply MVCA to the sugrph of G with lel set S (Sf L), node set V, nd the edge set E ' (E ' f E) ssocited with S The new crossover opertion is more time-consuming thn the old one The muttion opertion remins s efore 33
Computtionl Results 48 comintions: n = 50 to 200 / l = 12 to 250 / density = 0.2, 0.5, 0.8 20 smple grphs for ech comintion The verge numer of lels is compred 34
Performnce Comprison MVCA GA MGA MVCA1 MVCA2 MVCA3 RMVCA Row Totl MVCA - 3 0 0 0 0 0 3 GA 30-0 1 9 4 2 46 MGA 33 30-10 20 16 16 125 MVCA1 35 30 10-24 20 18 137 MVCA2 31 20 5 0-0 6 62 MVCA3 34 27 8 0 23-11 103 RMVCA 35 30 7 3 20 10-105 Summry of computtionl results with respect to ccurcy for seven heuristics on 48 cses. The entry (i, j) represents the numer of cses heuristic i genertes solution tht is etter thn the solution generted y heuristic j. 35
Running Times MVCA GA MGA MVCA1 MVCA2 MVCA3 RMVCA n = 100, l = 125, d = 0.2 0.05 1.80 7.50 8.25 0.80 2.30 3.85 n = 150, l = 150, d = 0.5 0.10 1.85 4.90 11.85 1.15 3.45 4.75 n = 150, l = 150, d = 0.2 0.15 3.45 13.55 21.95 2.15 6.35 8.45 n = 150, l = 187, d = 0.5 0.15 2.20 6.70 21.70 2.00 6.15 7.50 n = 150, l = 187, d = 0.2 0.20 3.95 17.55 39.35 3.60 11.20 11.90 n = 200, l = 100, d = 0.2 0.15 3.75 11.40 11.25 1.15 3.35 6.75 n = 200, l = 200, d = 0.8 0.25 2.45 5.80 26.70 2.70 8.00 8.65 n = 200, l = 200, d = 0.5 0.25 3.45 10.15 38.65 3.90 10.15 12.00 n = 200, l = 200, d = 0.2 0.35 6.20 26.65 68.25 6.85 20.35 20.55 n = 200, l = 250, d = 0.8 0.30 3.05 7.55 52.25 5.25 15.35 12.95 n = 200, l = 250, d = 0.5 0.30 3.95 12.60 69.90 6.80 20.35 16.70 n = 200, l = 250, d = 0.2 0.50 6.90 33.15 124.35 12.10 35.80 28.80 Averge running time 0.23 3.58 13.13 41.20 4.04 11.90 11.90 Running times for 12 demnding cses (in seconds). 36
One Finl Experiment for Smll Grphs 240 instnces for n = 20 to 50 re solved y the seven heuristics Bcktrck serch solves ech instnce to optimlity The seven heuristics re compred sed on how often ech otins n optiml solution Procedure OPT MVCA GA MGA MVCA1 MVCA2 MVCA3 RMVCA % optiml 100.00 75.42 96.67 99.58 95.42 87.08 93.75 97.50 37
Conclusions We presented three modified (deterministic) versions of MVCA, rndomized version of MVCA, nd modified GA All five of the modified procedures generted etter results thn MVCA nd GA, ut were more time-consuming With respect to running time nd performnce, MGA seems to e the est 38
Relted Work The Lel-Constrined Minimum Spnning Tree (LCMST) Prolem We show the LCMST prolem is NP-hrd We introduce two locl serch methods We present n effective genetic lgorithm We formulte the LCMST s MIP nd solve for smll cses We introduce dul prolem 39