CS261: A Seon Course in Algorithms Leture #5: Minimum-Cost Biprtite Mthing Tim Roughgren Jnury 19, 2016 1 Preliminries Figure 1: Exmple of iprtite grph. The eges {, } n {, } onstitute mthing. Lst leture introue the mximum-rinlity iprtite mthing prolem. Rell tht iprtite grph G = (V W, E) is one whose verties re split into two sets suh tht every ege hs one enpoint in eh set (no eges internl to V or W llowe). Rell tht mthing is suset M E of eges with no shre enpoints (e.g., Figure 1). Lst leture, we skethe simple reution from this prolem to the mximum flow prolem. Moreover, we eue from this reution n the mx-flow/min-ut theorem fmous optimlity onition for iprtite mthings. A speil se is Hll s theorem, whih sttes tht iprtite grph with V W hs perfet mthing if n only if for every suset S V of the left-hn sie, the numer N(S) of S on the right-hn sie is t lest S. See Prolem Set #2 for quite goo running time ouns for the prolem. But wht if iprtite grph hs mny perfet mthings? In pplitions, there re often resons to prefer one over nother. For exmple, when ssigning jos to works, perhps there re mny workers who n perform prtiulr jo, ut some of them re etter t 2016, Tim Roughgren. Deprtment of Computer Siene, Stnfor University, 474 Gtes Builing, 353 Serr Mll, Stnfor, CA 94305. Emil: tim@s.stnfor.eu. 1
it thn others. The simplest wy to moel suh preferenes is tth ost e to eh ege e E of the input iprtite grph G = (V W, E). We lso mke three ssumptions. These re for onveniene, n re not ruil for ny of our results. 1. The sets V n W hve the sme size, ll it n. This ssumption is esily enfore y ing ummy verties (with no inient eges) to the smller sie. 2. The grph G hs t lest one perfet mthing. This is esily enfore y ing ummy eges tht hve very high ost (e.g., one suh ege from the ith vertex of V to the ith vertex of W, for eh i). 3. Ege osts re nonnegtive. This n e enfore in the ovious wy: if the most negtive ege ost is M, just M to the ost of every ege. This s the sme numer (nm) to every perfet mthing, n thus oes not hnge the prolem. The gol in the minimum-ost perfet iprtite mthing prolem is to ompute the perfet mthing M tht minimizes e M e. The fesile solutions to the prolem re the perfet mthings of G. An equivlent prolem is the mximum-weight perfet iprtite mthing prolem (just multiply ll weights y 1 to trnsform them into osts). When every ege hs the sme ost n we only re out rinlity, the prolem reues to the mximum flow prolem (Leture #4). With generl osts, there oes not seem to e nturl reution to the mximum flow prolem. It s true tht eges in flow network ome with tthe numers (their pities), ut there is type mismth: ege pities ffet the set of fesile solutions ut not their ojetive funtion vlues, while ege osts o the opposite. Thus, the minimum-ost perfet iprtite mthing prolem seems like new prolem, for whih we hve to esign n lgorithm from srth. We ll follow the sme kin of isipline pproh tht serve us so well in the mximum flow prolem. First, we ientify optimlity onitions, whih tell us when given perfet mthing is in ft minimum-ost. This step is struturl, not lgorithmi, n is nlogous to our result in Leture #2 tht flow is mximum if n only if there is no s-t pth in the resiul network. Then, we esign n lgorithm tht n only terminte with the fesiility n optimlity onitions stisfie. For mximum flow, we h one lgorithmi prigm tht mintine fesiility n worke towr the optimlity onitions (ugmenting pth lgorithms), n seon prigm tht mintin the optimlity onitions n worke towr fesiility (push-relel). Here, we follow the seon pproh. We ll ientify invrints tht imply the optimlity onition, n esign n lgorithm tht keeps them stisfie t ll times n works towr fesile solution (i.e., perfet mthing). 2 Optimlity Conitions How o we know if given perfet mthing hs the minimum-possile ost? Optimlity onitions re ifferent for ifferent prolems, ut for the prolems stuie in CS261 they re 2
ll quite nturl in hinsight. We first nee n nlog of resiul network. This requires some efinitions (see lso Figure 2). 7 5 6 2 Figure 2: If our mthing ontins {, } n {, }, then is oth n M-lternting yle n negtive yle. Definition 2.1 (Negtive Cyle) Let M e mthing in the iprtite grph G = (V W, E). () A yle C of G is M-lternting if every other ege of C elongs to M (Figure 2). 1 () An M-lternting yle is negtive if the eges in the mthing hve higher ost thn those outsie the mthing: e > e. Otherwise, it is nonnegtive. e C M e C\M One interesting thing out lternting yles is tht y toggling the eges of C with respet to M tht is, removing the eges of C M n plugging in the eges of C \ M yiels new mthing M tht mthes extly the sme set of verties. (Verties outsie of C re lerly unffete; verties insie C remin mthe to preisely one other vertex of C, just ifferent one thn efore.) Suppose M is perfet mthing, n we toggle the eges of n M-lternting yle to get nother (perfet) mthing M. Dropping the eges from C M sves us ost of e C M e, while ing the eges of C \ M ost us e C\M e. Then M hs smller ost thn M if n only if C is negtive yle. The point of negtive yle is tht it offers quik n onvining proof tht perfet mthing is not minimum-ost (sine toggling the eges of the yle yiels heper mthing). But wht out the onverse? If perfet mthing is not minimum-ost, re we gurntee suh short n onvining proof of this ft? Or re there ostles to optimlity eyon the ovious ones of negtive yles? 1 Sine G is iprtite, C is neessrily n even yle. One ertinly n t hve more thn every other ege of C ontine in the mthing M. 3
Theorem 2.2 (Optimlity Conitions for Min-Cost Biprtite Mthing) A perfet mthing of iprtite grph hs minimum-ost if n only if there is no negtive M- lternting yle. Proof: We hve lrey rgue the only if iretion. For the hrer if iretion, suppose tht M is perfet mthing n tht there is no negtive M-lternting yle. Let M e ny other perfet mthing; we wnt to show tht the ost of M is t lest tht of M. Consier M M, mening the symmetri ifferene of M, M (if you wnt to think of them s sets) or their XOR (if you wnt to think of them s 0/1 vetors). See Figure 3 for two exmples. e e e f f = f e e e f f = f Figure 3: Two exmples tht show wht hppens when we XOR two mthings (the she eges). In generl, M M is union of (vertex-)isjoint yles. The reson is tht, sine every vertex hs egree 1 in oth M n M, every vertex of v hs egree either 0 (if it is mthe to the sme vertex in oth M n M ) or 2 (otherwise). A grph with ll egrees either 0 or 2 must e the union of isjoint yles. Sine tking the symmetri ifferene/xor with the sme set two times in row reovers the initil set, (M M ) M = M. Sine M M is isjoint union of yles, tking the symmetri ifferent/xor with M M just mens toggling the eges in eh of its yles (sine they re isjoint, they on t interfere n the toggling n e one in prllel). Eh of these yles is M-lternting, n y ssumption eh is nonnegtive. Thus toggling the eges of the yles n only proue more expensive perfet mthing M. Sine M ws n ritrry perfet mthing, M must e minimum-ost perfet mthing. 4
3 Reue Costs n Invrints Now tht we know when we re one, we work towr lgorithms tht terminte with the optimlity onitions stisfie. Following the push-relel pproh (Leture #3), we next ientify invrints tht will imply the optimlity onitions t ll times. Our lgorithm will mintin these s it works towr fesile solution (i.e., perfet mthing). Continuing the nlogy with the push-relel prigm, we mintin extr numer p v for eh vertex v V W, lle prie (nlogous to the heights in Leture #3). Pries re llowe to e positive or negtive. We use pries to fore us to eges to our urrent mthing only in isipline wy, somewht nlogous to how we only pushe flow ownhill in Leture #3. Formlly, for prie vetor p (inexe y verties), we efine the reue ost of n ege e = (v, w) y p e = e p v p w. (1) Here re our invrints, whih re respet to urrent mthing M n urrent vetor p of pries. Invrints 1. Every ege of G hs nonnegtive reue ost. 2. Every ege of M is tight, mening it hs zero reue ost. v(7) 7 w(0) v(5) 7 w(2) 5 5 6 6 x(2) 2 y(0) x(2) 2 y(0) Figure 4: For the given (perfet) mthing (she eges), () violtes invrint 1, while () stisfies ll invrints. For exmple, onsier the (perfet) mthing in Figure 4. Is it possile to efine pries so tht the invrints hol? To stisfy the seon invrint, we nee to mke the eges (v, w) n (x, y) tight. We oul try setting the prie of w n y to 0, whih then ittes setting p v = 7 n p x = 2 (Figure 4()). This violtes the first invrint, however, sine the reue ost of ege (v, y) is -1. We n stisfy oth invrints y resetting p v = 5 n p w = 2; then oth eges in the mthing re tight n the other two eges hve reue ost 1 (Figure 4()). 5
The mthing in Figure 4 is min-ost perfet mthing. This is no oiniene. Lemm 3.1 (Invrints Imply Optimlity Conition) If M is perfet mthing n oth invrints hol, then M is minimum-ost perfet mthing. Proof: Let M e perfet mthing suh tht oth invrints hol. By our optimlity onition (Theorem 2.2), we just nee to hek tht there is no negtive yle. So onsier ny M-lternting yle C (rememer negtive yle must e M-lternting, y efinition). We wnt to show tht the eges of C tht re in M hve ost t most tht of the eges of C not in M. Aing n sutrting v C p v n using the ft tht every vertex of C is the enpoint of extly one ege of C M n of C \ M (e.g., Figure 5), we n write e = p e + p v (2) v C n e C M e C\M e = e C M e C\M p e + v C p v. (3) (We re using nottion n using C oth to enote the verties in the yle n the eges in the yle; hopefully the mening is lwys ler from ontext.) Clerly, the thir terms in (2) n (3) re the sme. By the seon invrint (eges of M re tight), the seon term in (2) is 0. By the first invrint (ll eges hve nonnegtive reue ost), the seon term in (3) is t lest 0. We onlue tht the left-hn sie of (2) is t most tht of (3), whih proves tht C is not negtive yle. Sine C ws ritrry M-lternting yle, the proof is omplete. f e Figure 5: In the exmple M-lternting yle n mthing shown ove, every vertex is n enpoint of extly one ege in M n one ege not in M. 4 The Hungrin Algorithm Lemm 3.1 reues the prolem of esigning orret lgorithm for the minimum-ost perfet iprtite mthing prolem to tht of esigning n lgorithm tht mintins the two invrints n omputes n ritrry perfet mthing. This setion presents suh n lgorithm. 6
4.1 Bkstory The lgorithm we present goes y vrious nmes, the two most ommon eing the Hungrin lgorithm n the Kuhn-Munkres lgorithm. You might therefore fin it weir tht Kuhn n Munkres re Amerin. Here s the story. In the erly/mi-1950s, Kuhn relly wnte n lgorithm for solving the minimum-ost iprtite mthing prolem. So he ws reing grph theory ook y Kőnig. This ws tully the first grph theory ook ever written in the 1930s, n ville in the U.S. only in 1950 (even then, only in Germn). Kuhn ws intrigue y n offhn ittion in the ook, to pper of Egerváry. Kuhn trke own the pper, whih ws written in Hungrin. This ws wy efore Google Trnslte, so he ought ig English-Hungrin itionry n trnslte the whole thing. An inee, Egerváry s pper h the key ies neessry for goo lgorithm. Kőnig n Egerváry were oth Hungrin, so Kuhn lle his lgorithm the Hungrin lgorithm. Kuhn only prove termintion of his lgorithm, n soon therefter Munkres oserve polynomil time oun (silly the oun prove in this leture). Hene, lso lle the Kuhn-Munkres lgorithm. In (finl?) twist to the story, in 2006 it ws isovere tht Joi, the fmous mthemtiin (you ve stuie multiple onepts nme fter him in your mth lsses), me up with n equivlent lgorithm in the 1840s! (Pulishe only posthumously, in 1890.) Kuhn, then in his 80s, ws goo sport out it, giving tlks with the title The Hungrin Algorithm n How Joi Bet Me By 100 Yers. 4.2 The Algorithm: High-Level Struture The Hungrin lgorithm mintins oth mthing M n pries p. The initiliztion is strightforwr. set M = set p v = 0 for ll v V W Initiliztion The seon invrint hols vuously. The first invrint hols euse we re ssuming tht ll ege osts (n hene initil reue osts) re nonnegtive. Informlly (n wy unerspeifie), the min loop works s follows. The terms ugment, goo pth, n goo set will e efine shortly. Min Loop (High-Level) while M is not perfet mthing o if there is goo pth P then ugment M y P else fin goo set S; upte pries oringly 7
4.3 Goo Pths We now strt filling in the etils. Fix the urrent mthing M n urrent pries p. Cll pth P from v to w goo if: 1. oth enpoints v, w re unmthe in M, with v V n w W (hene P hs o length); 2. it lterntes eges out of M with eges in M (sine v, w re unmthe, the first n lst eges re not in M); 3. every ege of P is tight (i.e., hs zero reue ost n hene eligile to e inlue in the urrent mthing). Figure 6 epits simple exmple of goo pth. (2) 4 (2) 4 4 (2) 4 (2) Figure 6: Dshe eges enote eges in the mthing n re eges enote goo pth. The reson we re out goo pths is tht suh pth llows us to inrese the rinlity of M without reking either invrint. Speifilly, onsier repling M y M = M P. This n e thought of s toggling whih eges of P re in the urrent mthing. By efinition, goo pth is M-lternting, with first n lst hops not in M; thus, P M = P \ M 1, n the size of M is one more thn M. (E.g., if P is 9-hop pth, this toggling removes 4 eges from M ut tht s in 5 other eges.) No reue osts hve hnge, so ertinly the first invrint still hols. All eges of P re tight e efinition, so the seon invrint lso ontinues to hol. Augmenttion Step given goo pth P, reple M y M P Fining goo pth is efinitely progress fter n suh ugmenttions, the urrent mthing M must e perfet n (sine the invrints hol) we re one. How n we effiiently fin suh pth? An wht o we o if there s no suh pth? 8
To effiiently serh for suh pth, let s just follow our nose. It turns out tht rethfirst serh (BFS), with twist to enfore M-lterntion, is ll we nee. 6 1 3 8 5 2 4 7 Figure 7: Dshe eges re the eges in the mthing. Only tight eges re shown. The lgorithm will e ler from n exmple. Consier the grph in Figure 7; only the tight eges re shown. Note tht the grph oes not ontin goo pth (if it i, we oul use it to ugment the urrent mthing to otin perfet mthing, ut vertex #4 is isolte so there is no perfet mthing.). 2 So we know in vne tht our serh will fil. But it s useful to see wht hppens when it fils. 2 1 5 6 3 7 8 Figure 8: BFS spnning tree if we strt BFS trvel from noe 3. Note tht the ege {2, 6} is not use. We strt grph serh from n unmthe vertex of V (the first suh vertex, sy); see lso Figure 8. In the exmple, this is vertex #3. Lyer 0 of our serh tree is {3}. We otin lyer 1 from lyer 0 y BFS; thus, lyer 1 is {2, 7}. Note tht if either 2 or 7 is unmthe, then we hve foun (one-hop) goo pth n we n stop the serh. Both 2 n 7 re lrey mthe in the exmple, however. Here is the twist to BFS: t the next lyer 2 we put only the verties to whih 2 n 7 re mthe, nmely 1 n 8. Conspiuous in its sene is vertex #6; in regulr BFS it woul e inlue in lyer 2, ut here we omit it euse it is not mthe to vertex of lyer 1. The reson for this twist is tht we wnt every pth in our serh tree to e M-lternting (sine goo pths nee to e M-lternting). 2 Rememer we ssume only tht G ontins perfet mthing; the sugrph of tight eges t ny given time will generlly not ontin perfet mthing. 9
We then swith k to BFS. At vertex #8 we re stuk (we ve lrey seen its only neighor, #7). At vertex #1, we ve lrey seen its neighor 2 ut hve not yet seen vertex #5, so the thir lyer is {5}. Note tht if 5 were unmthe, we woul hve foun goo pth, from 5 k to the root 3. (All eges in the tree re tight y efinition; the pth is lternting n of o length, joining two unmthe verties of V n W.) But 5 is lrey mthe to 6, so lyer 4 of the serh tree is {6}. We ve lrey seen oth of 6 s neighors efore, so t this point we re stuk n the serh termintes. In generl, here is the serh proeure for fining goo pth (given urrent mthing M n pries p). Serhing for Goo Pth level 0 = the first unmthe vertex r of V while not stuk n no other unmthe vertex foun o if next level i is o then efine level i from level i 1 vi BFS // i.e., neighors of level i 1 not lrey seen else if next level i is even then efine level i s the verties mthe in M to verties t level i 1 if foun nother unmthe vertex w then return the serh tree pth etween the root r n w else return stuk To unerstn this suroutine, onsier n ege (v, w) M, n suppose tht v is rehe first, t level i. Importntly, it is not possile tht w is lso rehe t level i. This is where we use the ssumption tht G is iprtite: if v, w re rehe in the sme level, then psting together the pths from r to v n from r to w (whih hve the sme length) with the ege (v, w) exhiits n o yle, ontriting iprtiteness. Seon, we lim tht i must e o (f., Figure 8). The reson is just tht, y onstrution, every vertex t n even level (other thn 0) is the seon enpoint rehe of some mthe ege (n hene nnot e the enpoint of ny other mthe ege). We onlue tht: (*) if either enpoint of n ege of M is rehe in the serh tree, then oth enpoints re rehe, n they pper t onseutive levels i, i + 1 with i o. Suppose the serh tree rehes n unmthe vertex w other thn the root r. Sine every vertex t n even level (fter 0) is mthe to vertex t the previous level, w must e t n o level (n hene in W ). By onstrution, every ege of the serh tree is tight, n every pth in the tree is M-lternting. Thus the r-w pth in the serh tree is goo pth, llowing us to inrese the size of M y 1. 10
4.4 Goo Sets Suppose the serh gets stuk, s in our exmple. How o we mke progress, n in wht sense? In this se, we keep the mthing the sme ut upte the pries. Define S V s the verties t even levels. Define N(S) W s the neighors of S vi tight eges, i.e., N(S) = {w : v S with (v, w) tight}. (4) We lim tht N(S) is preisely the verties tht pper in the o levels of the serh tree. In proof, first note tht every vertex t n o level is (y onstrution/bfs) jent vi tight ege to vertex t the previous (even) level. For the onverse, every vertex w N(S) must e rehe in the serh, euse (y si properties of grph serh) the serh n only stuk if there re no unexplore eges out of ny even vertex. The set S is goo set, in tht is stisfies: 1. S ontins n unmthe vertex; 2. every vertex of N(S) is mthe in M to vertex of S (sine the serh file, every vertex in n o level is mthe to some vertex t the next (even) level). See lso Figure 9. 1 2 3 4 5 6 7 Figure 9: S = {1, 2, 3, 4} is exmple of goo set, with N(S) = {5, 6}. Only lk eges re tight eges (i.e. (4, 7) is not tight). The mthing eges re she. Hving foun suh goo set S, the Hungrin lgorithm uptes pries s follows. Prie Upte Step given goo set S, with neighors vi tight eges N(S) for ll v S o inrese p v y for ll w N(S) o erese p v y // is s lrge s possile, sujet to invrints 11
Pries in S (on the left-hn sie) re inrese, while pries in N(S) (on the right-hn sie) re erese y the sme mount. How oes this ffet the reue ost of eh ege of G (Figure 9)? 1. for n ege (v, w) with v / S n w / N(S), the pries of v, w re unhnge so p vw is unhnge; 2. for n ege (v, w) with v S n w N(S), the sum of the pries of v, w is unhnge (one inrese y, the other erese y ) so p vw is unhnge; 3. for n ege (v, w) with v / S n w N(S), p v stys the sme while p w goes own y, so p vw goes up y ; 4. for n ege (v, w) with v S n w / N(S), p w stys the sme while p v goes up y, so p vw goes own y. So wht hppens with the invrints? Relling (*) from Setion 4.3, we see tht eges of M re in either the first or seon tegory. Thus they sty tight, n the seon invrint remins stisfie. The first invrint is enngere y eges in the fourth tegory, whose reue osts re ropping with. 3 By the efinition of N(S), eges in this tegory re not tight. So we inrese to the lrgest-possile vlue sujet to the first invrint the first point t whih the reue ost of some ege in the fourth tegory is zeroe out. 4 Every prie upte mkes progress, in the sense tht it stritly inreses the size of serh tree. To see this, suppose prie upte uses the ege (v, w) to eome tight (with v S, w / N(S)). Wht hppens in the next itertion, when we serh from the sme vertex r for goo pth? All eges in the previous serh tree fll in the seon tegory, n hene re gin tight in the next itertion. Thus, the serh proeure will regrow extly the sme serh tree s efore, will gin reh the vertex v, n now will lso explore long the newly tight ege (v, w), whih s the itionl vertex w W to the tree. This n only hppen n times in row efore fining goo pth, sine there re only n verties in W. 3 Eges in thir tegory might go from tight to non-tight, ut these eges re not in M (every vertex of N(S) is mthe to vertex of S) n so no invrint is violte. 4 A etil: how o we know tht suh n ege exists? If not, then ll neighors of S in G (vi tight eges or not) elong to N(S). The two properties of goo sets imply tht N(S) < S. But this violtes Hll s onition for perfet mthings (Leture #4), ontriting our stning ssumption tht G hs t lest one perfet mthing. 12
4.5 The Hungrin Algorithm (All in One Ple) The Hungrin Algorithm set M = set p v = 0 for ll v V W while M is not perfet mthing o level 0 of serh tree T = the first unmthe vertex r of V while not stuk n no other unmthe vertex foun o if next level i is o then efine level i of T from level i 1 vi BFS // i.e., neighors of level i 1 not lrey seen else if next level i is even then efine level i of T s the verties mthe in M to verties t level i 1 if T ontins n unmthe vertex w W then let P enote the r-w pth in T reple M y M P else let S enote the verties of T in even levels let N(S) enote the verties of T in o levels for ll v S o inrese p v y for ll w N(S) o erese p v y // is s lrge s possile, sujet to invrints return M 4.6 Running Time Sine M n only ontin n eges, there n only e n itertions tht fin goo pth. Sine the serh tree n only ontin n verties of W, there n only e n pries uptes etween itertions tht fin goo pths. Computing the serh tree (n hene P or S n N(S)) n (if neessry) n e one in O(m) time. This gives running time oun of O(mn 2 ). See Prolem Set #2 for n implementtion with running time O(mn log n). 4.7 Exmple We reinfore the lgorithm vi n exmple. Consier the grph in Figure 10. 13
v(0) 5 2 3 w(0) x(0) 7 y(0) Figure 10: Exmple grph. Initilly, ll pries re 0. We initilize ll pries to 0 n the urrent mthing to the empty set. Initilly, there re no tight eges, so there is ertinly no goo pth. The serh for suh pth gets stuk where it strts, t vertex v. So S = {v} n N(S) =. We exeute prie upte step, rising the prie of v to 2, t whih point the ege (v, w) eomes tight. Next itertion, the serh strts t v, explores the tight ege (v, w), n enounters vertex w, whih is unmthe. Thus this ege is e to the urrent mthing. Next itertion, new serh strts from the only remining unmthe vertex on the left (x). It hs no tight inient eges, so the serh gets stuk immeitely, with S = {x} n N(S) =. We thus o prie upte step, with = 5, t whih point the ege (x, w) eomes newly tight. Note tht the eges (v, y) n (x, y) hve reue osts 1 n 2, respetively, so neither is tight. Next itertion, the serh from x explores the inient tight ege (x, w). If w were unmthe, we oul stop the serh n the ege (x, w). But w is lrey mthe, to v, so w n v re ple t levels 1 n 2 of the serh tree. v hs no tight inient eges other thn to w, so the serh gets stuk here, with S = {x, v} n N(S) = {w}. So we o nother prie upte step, inresing the prie of x n v y n eresing the prie of w y. With = 1, the reue ost of ege (v, y) gets zeroe out. The finl itertion isovers the goo pth x w v y. Augmenting on this pth yiels the minimum-ost perfet mthing {(v, y), (x, w)}. 14