Lecture 16. Min Cut and Karger s Algorithm

Transcription

1 Lecture 16 Mi Cut ad Karger s Algorithm

2 Aoucemets HW 7 due Friday HW 8 released Friday Psych! There is o HW8. FINAL EXAM: Wedesday December 13 3:30 6:30pm

3 Advertisemet! CS83 with Omer Reigold: Witer quarter Fri1:30 PM -4:20 PM

4 Last time Miimum Spaig Trees! Prim s Algorithm Kruskal s Algorithm

5 Today Miimum Cuts! Karger s algorithm Karger-Stei algorithm Back to radomized algorithms!

6 Recall: cuts i graphs *For today, all graphs are udirected ad uweighted. A cut is a partitio of the vertices ito two oempty parts.

7 Recall: cuts i graphs *For today, all graphs are udirected ad uweighted. A cut is a partitio of the vertices ito two oempty parts. Part 1 Part 2

8 This is ot a cut

9 This is a cut

10 This is a cut These edges cross the cut. They go from oe part to the other.

11 A (global) miimum cut is a cut that has the fewest edges possible crossig it.

12 A (global) miimum cut is a cut that has the fewest edges possible crossig it.

13 Why global? Next time we ll talk about mi s-t cuts s Miimum cut which separates a specified vertex s from t t Today, there are o special vertices, so the miimum cut is global.

14 A (global) miimum cut is a cut that has the fewest edges possible crossig it.

15 Why might we care about global miimum cuts? Oe example is image segmetatio:

16 Why might we care about global miimum cuts? Oe example is image segmetatio: big edge weights* betwee similar pixels. We ll see more applicatios for other sorts of mi-cuts ext week *For the rest of today edges are t weighted; but the algorithm ca be adapted to deal with edge weights.

17 Karger salgorithm Fids globalmiimum cuts i udirected graphs Radomized algorithm Karger s algorithm might be wrog. Compare to QuickSort, which just might be slow. Why would we wat a algorithm that might be wrog? With high probability it wo t be wrog. Maybe the stakes are low ad the cost of a determiistic algorithm is high.

18 Differet sorts of gamblig QuickSortis a Las Vegas radomized algorithm It is always correct. It might be slow. Yes, this is a techical term. Formally: For all iputs A, QuickSort(A) returs a sorted array. For all iputs A, with high probability over the choice of pivots, QuickSort(A) rus quickly.

19 Differet sorts of gamblig Karger salgorithm is a Mote Carlo radomized algorithm It is always fast. It might be wrog. Formally: For all iputs G, with probability at least over the radomess i Karger salgorithm, Karger(G) returs a miimum cut. For all iputs G, with probability 1 Karger salgorithm rus i time o more tha.

20 Karger salgorithm Pick a radom edge. Cotractit. Repeat util you oly have two vertices left. a b a b New ode! Why is this a good idea? We ll see shortly.

21 Karger salgorithm a b f g e c d h

22 Karger salgorithm a radom edge! b f g e c d h

23 Karger salgorithm Create a superode! f a b g Create a superedge! Create a superedge! e c d h

24 Karger salgorithm a,b Create a superode! f g Create a superedge! Create a superedge! e c d h

25 Karger salgorithm f a,b g e c d radom edge! h

26 Karger salgorithm f a,b Create a superedge! g c d e h Create a superode! Create a superedge!

27 Karger salgorithm a,b Create a superedge! f g e,h Create a superedge! c d Create a superode!

28 Karger salgorithm f a,b g e,h c radom edge! d

29 Karger salgorithm f a,b g e,h c,d

30 Karger salgorithm f a,b g radom edge! e,h c,d

31 Karger salgorithm f a,b,c,d g e,h

32 Karger salgorithm f a,b,c,d g radom edge! e,h

33 Karger salgorithm f a,b,c,d e,h,g

34 Karger salgorithm f a,b,c,d radom edge! e,h,g

35 Karger salgorithm Now stop! There are oly two odes left. a,b,c,d The miimum cut is give by the remaiig super-odes: {a,b,c,d} ad {e,h,f,g} e,h,g,f

36 Karger salgorithm The miimum cut is give by the remaiig super-odes: {a,b,c,d} ad {e,h,f,g} a b f g e c d h

37 Karger salgorithm Does it work? Is it fast?

38 How do we implemet this? See Lecture 16 IPythoNotebook for oe way This maitais a secodary supergraph which keeps track of supernodesad superedges There s a hidde slide with pseudocode Ruig time? We cotract at most -2 edges Each time we cotract a edge we get rid of a vertex, ad we get rid of at most 2 vertices total. Naively each cotractio takes time O() Maybe there are about odes i the supernodesthat we are mergig. So total ruig time O( 2 ). We ca do O(m α )with a uio-fid data structure, but O( ( )is good eough for today.

39 Pseudocode Let u. deote the SuperNodei Γcotaiig u Say E u.,v. is the SuperEdgebetwee u.,v.. Karger( G=(V,E) ): Γ= { SuperNode(v) : v i V } E u.,v. = {(u,v)} for (u,v) i E E u.,v. = {} for (u,v) ot i E. F = copy of E This slide skipped i class while Γ > 2: (u,v) uiformly radom edge i F merge( u, v ) // merge the SuperNode cotaiig u with the SuperNode cotaiig v. F F E u.,v. // remove all the edges i the SuperEdgebetwee those SuperNodes. returthe cut give by the remaiig two supernodes. // oe superodefor each vertex // oe superedgefor each edge // we ll choose radomly from F Thewhile loop rus -2 times mergetakes time O() aively merge( u, v ): // merge also kows about Γ ad the E u.,v. s x.= SuperNode( u. v.) // create a ew superode for each wi Γ {u.,v.}: E x.,w. =E u.,w. E v.,w. Remove u. ad v.from Γad add x.. total rutime O( 2 ) We ca do a bit better with facy data structures, but let s go with this for ow.

40 Karger salgorithm Does it work? No? Is it fast? O( 2 )

41 Why did that work? We got really lucky! This could have goe wrog i so may ways.

42 Karger salgorithm Say we had chose this edge a b radom edge! f g e c d h

43 Karger salgorithm Say we had chose this edge Now there is o way we could retur a cut that separates b ad e. a b,e f g c d h

44 Eve worse If the algorithm EVERchooses either of these edges, it will be wrog. a b f g e c d h

45 a b f g How likely is that? e c d For this particular graph, I did it 10,000 times: h The algorithm is oly correct about 20% of the time!

46 That does t soud good Too see why it s good after all, we ll do a case study of this graph. Let s compare Karger salgorithm to the algorithm: Choose a completely radom cut ad hope that it s a miimum cut. c a b d e f h g The pla: See that 20% chace of correctess is actually otrivial. Use repetitio to boost a algorithm that s correct 20% of the time to a algorithm that s correct 99% of the time.

47 Radom cuts Suppose that we chose cuts uiformly at radom. That is, pick a radom way to split the vertices ito 2 parts. etc

48 Radom cuts Suppose that we chose cuts uiformly at radom. That is, pick a radom way to split the vertices ito 2 parts. The probability of choosig the miimum cut is* umber ofmicuts i that graph umber of ways to split 8 vertices i 2 parts = 2 2 O Aka, we get a miimum cut 0.8% of the time. *For this example i particular

49 Kargeris better tha completely radom! a b f g c d e h Karger salg. is correct about 20% of the time Completely radom is correct about 0.8% of the time

50 What s goig o? Which is more likely? Thig 1: It s ulikely that Kargerwill hit the mi cut sice it s so small! Lucky the lackadaisical lemur a b e f g B: The algorithm ever chooses ay of the edges i this big cut. c d h a b f g A: The algorithm ever chooses either of the edges i the miimum cut. c d e h Neither A or B are very likely, but A is more likely tha B.

51 What s goig o? Thig 2: By oly cotractig edges we are igorig certai really-ot-miimal cuts. Lucky the lackadaisical lemur c a d b e f h g B: This cut ca t be retured by Karger salgorithm! (Because how would a ad g ed up i the same super-ode?) a b f g A: This cut ca be retured by Karger s algorithm. c d e h This cut actually separates the graph ito three pieces, so it s ot miimal either half of it is a smaller cut.

52 Why does that help? Okay, so it s better tha radom We re still wrog about 80% of the time. The mai idea: repeat! If I m wrog 20% of the time, the if I repeat it a few times I ll evetually get it right. The pla: See that 20% chace of correctess is actually otrivial. Use repetitio to boost a algorithm that s correct 20% of the time to a algorithm that s correct 99% of the time.

53 Thought experimet from pre-lecture exercise Suppose you have a magic butto that produces oe of 5 umbers, {a,b,c,d,e}, uiformly at radom whe you push it. Q: What is the miimum of a,b,c,d,e? How may times do you have to push the butto before you see the miimum value? What is the probability that you have to push it more tha 5 times? 10 times? [O board]

54 [This is approximately what s o the board] This is the same calculatio we ve doe a buch of times: E[ we push the butto ] = 1/(0.20) = 5 Pr[ Number of times util we get the miimum value This oe we ve doe less frequetly: We push the butto t times ad do t ever get the mi We push the butto ] = (1 0.2) U Pr[ 5 times ad do t ] = (1 0.2) V 0.33 ever get the mi We push the butto Pr[ 10 times ad do t ] = (1 0.2) XY 0.1 ever get the mi Slide skipped i class

55 I this cotext Ru Karger s! The cut size is 6! Ru Karger s! The cut size is 3! Ru Karger s! The cut size is 3! Ru Karger s! The cut size is 2! Correct! Ru Karger s! The cut size is 5! If the success probability is about 20%, the if you ru Karger salgorithm 5times ad take the best aswer you get, that will likely be correct!

56 For this particular graph Repeat Karger salgorithm about 5 times, ad we will get a mi cut with decet probability. I cotrast, we d have to choose a radom cut about 1/0.008 = 125 times! c a b d e f h g Hag o! This 20% figure just came from ruig experimets o this particular graph. What about geeral graphs? Ca we prove this? Also, we should be a bit more precise about this about 5 times statemet. Plucky the pedatic pegui The pla: See that 20% chace of correctess is actually otrivial. Use repetitio to boost a algorithm that s correct 20% of the time to a algorithm that s correct 99% of the time.

57 Questios To geeralize this approach to all graphs 1. What is the probability that Karger salgorithm returs a miimum cut? 2. How may times should we ru Karger s algorithm to probably succeed? Say, with probability 0.99? Or more geerally, probability 1 δ?

58 Aswer to Questio 1 Claim: The probability that Karger salgorithm returs a miimum cut is at least X _ ] ^ I this case, X _ ` =0.036, so we are ^ guarateed to wi at least 3.6% of the time.

59 Aswers 1. What is the probability that Karger salgorithm returs a miimum cut? Accordig to the claim, at most 2. How may times should we ru Karger s algorithm to probably succeed? Say, with probability 0.99? Or more geerally, probability 1 δ? X ] ^

60 Before we prove the Claim 2. How may times should we ru Karger s algorithm to succeed with probability 1 δ? a b f g e c d h

61 A computatio Puchlie: If we repeat T = 2 l(1/δ)times, we wi with probability at least 1 δ. Suppose : the probability of successfully returig a miimum cut is p 0,1, we wat failure probability at most δ 0,1. Pr [ do t retur amicut i T trials ]= 1 p m So p = 1/ ( by the Claim. Let s choose T = ( l(1/δ). Pr [ do t retur a mi cut i T trials ] = 1 p m e rs m =e rsm =e rtu v w =δ Idepedet 1 p e rs 1 p e rs

62 Theorem Assumig the claim about 1/ ( Suppose G has vertices. Cosider the followig algorithm: bestcut= Noe for t=1,, ( l X : cadidatecut Karger(G) if cadidatecut is smaller tha bestcut: bestcut cadidatecut retur bestcut The Pr[ this does } t retur a micut ] δ. How may repetitios would you eed if istead of Kargerwe just chose a uiformly radom cut?

63 Aswers 1. What is the probability that Karger salgorithm returs a miimum cut? Accordig to the claim, at most 2. How may times should we ru Karger s algorithm to probably succeed? Say, with probability 0.99? Or more geerally, probability 1 δ? ( log X times. X ] ^

64 What s the ruig time? ( l X repetitios, ad O(2 ) per repetitio. So, O ( ( l X =O Treatig δas costat. Agai we ca do better with a uio-fid data structure. Write pseudocode for or better yet, implemet a fast versio of Karger salgorithm! How fast ca you make the asymptotic ruig time? Ollie the over-achievig ostrich

65 Theorem Assumig the claim about 1/ ( Suppose G has vertices. The [repeatig Karger s algorithm] fids a mi cut i G with probability at least 0.99 i time O( 4 ). Now let s prove the claim

66 Claim The probability that Karger salgorithm returs a miimum cut is at least X _ ] ^

67 Now let s prove that claim Say that S* is a miimum cut. Suppose the edges that we choose are e 1,e 2,, e -2 PR[ retur S* ] = PR[ oe of the e i cross S* ] = PR[ e 1 does t cross S* ] PR[ e 2 does t cross S* e 1 does t cross S* ] PR[ e -2 does t cross S* e 1,,e -3 do t cross S* ] a b f g e c d S* h

68 Focus i o: PR[ e j does t cross S* e 1,,e j-1 do t cross S* ] Suppose: After j-1 iteratios, we have t messed up yet! What s the probability of messig up ow? a,b These two edges have t bee chose for cotractio! f g e,h c d

69 Focus i o: PR[ e j does t cross S* e 1,,e j-1 do t cross S* ] Suppose: After j-1 iteratios, we have t messed up yet! What s the probability of messig up ow? Say there are k edges that cross S* Every remaiig ode has degree at least k. Otherwise we d have a smaller cut. Thus, there are at least (-j+1)k/2 edges total. b/c there are -j + 1 odes left, each with degree at least k. Recall: the degree of the vertex is the umber of edges comig out of it. So the probability that we choose oe of the k edges crossig S* at step j is at most: k ƒj 1 k 2 = 2 rj 1 c a,b d f e,h g

70 Focus i o: PR[ e j does t cross S* e 1,,e j-1 do t cross S* ] So the probability that we choose oe of the k edges crossig S* at step j is at most: ]ƒˆ v ^ = ( rš X The probability we do t choose oe of the k edges is at least: 1 ( = ršrx rš X rš X a,b f g c d e,h

71 Now let s prove that claim Say that S* is a miimum cut. Suppose the edges that we choose are e 1,e 2,, e -2 PR[ retur S* ] = PR[ oe of the e i cross S* ] = PR[ e 1 does t cross S* ] PR[ e 2 does t cross S* e 1 does t cross S* ] PR[ e -2 does t cross S* e 1,,e -3 do t cross S* ] a b f g e c d S* h

72 Now let s prove that claim Say that S* is a miimum cut. Suppose the edges that we choose are e 1,e 2,, e -2 PR[ retur S* ] = PR[ oe of the e i cross S* ] = r( r rx r r( rv r rœ r Œ V ( X a b f g e c d S* h

73 Now let s prove that claim Say that S* is a miimum cut. Suppose the edges that we choose are e 1,e 2,, e -2 PR[ retur S* ] = PR[ oe of the e i cross S* ] = r( ( = = X ] ^ rx r rx r r( rv r rœ r Œ V ( X a b f g c d S* e h

74 Theorem Assumig the claim about 1/ ( Suppose G has vertices. The [repeatig Karger s algorithm] fids a mi cut i G with probability at least 0.99 i time O( 4 ). That proves this Theorem!

75 What have we leared? If we radomly cotract edges: It s ulikely that we ll ed up with a mi cut. But it s ot TOO ulikely By repeatig, we likely will fid a mi cut. Here I chose δ=0.01 just for cocreteess. Repeatig this process: Fids a global mi cut i time O( 4 ), with probability We ca ru a bit faster if we use a uio-fiddata structure. *Note, i the lecture otes, we take δ= X,which makes the ruig time O( 4 log()). It depeds o how sure you wat to be!

76 More geerally Wheever we have a Mote-Carlo algorithm with a small success probability, we ca boostthe success probability by repeatig it a buch ad takig the best solutio.

77 Ca we do better? Repeatig O( 2 ) times is pretty expesive. O( 4 ) total rutime to get success probability The Karger-Stei Algorithmwill do better! The trick is that we ll do the repetitios i a clever way. O( 2 log 2 () ) rutime for the same success probability. Warig! This is a tricky algorithm! We ll sketch the approach here: the importat part is the high-level idea, ot the details of the computatios. To see how we might save o repetitios, let s ru through Karger s algorithm agai.

78 Karger salgorithm a b f g e c d h

79 Karger salgorithm Probability that we did t mess up: 12/14 There are 14 edges, 12 of which are good to cotract. a radom edge! b f g e c d h

80 Karger salgorithm Create a superode! f a b g Create a superedge! Create a superedge! e c d h

81 Karger salgorithm a,b Create a superode! f g Create a superedge! Create a superedge! e c d h

82 Karger salgorithm Probability that we did t mess up: 11/13 Now there are oly 13 edges, sice the edge betwee a ad b disappeared. f a,b g e c d radom edge! h

83 Karger salgorithm f a,b Create a superedge! g c d e h Create a superode! Create a superedge!

84 Karger salgorithm a,b Create a superedge! f g e,h Create a superedge! c d Create a superode!

85 Karger salgorithm Probability that we did t mess up: 10/12 Now there are oly 12 edges, sice the edge betwee e ad h disappeared. f a,b g e,h c radom edge! d

86 Karger salgorithm f a,b g e,h c,d

87 Karger salgorithm Probability that we did t mess up: 9/11 f a,b g radom edge! (We pick at radom from the origial edges). e,h c,d

88 Karger salgorithm f a,b,c,d g e,h

89 Karger salgorithm Probability that we did t mess up: 5/7 f a,b,c,d g radom edge! e,h

90 Karger salgorithm f a,b,c,d e,h,g

91 Karger salgorithm Probability that we did t mess up: 3/5 f a,b,c,d radom edge! e,h,g

92 Karger salgorithm a,b,c,d e,h,g,f

93 Karger salgorithm Now stop! There are oly two odes left. a,b,c,d e,h,g,f

94 Probability of ot messig up At the begiig, it s pretty likely we ll be fie. The probability that we mess up gets worse ad worse over time. 12/14 11/13 10/12 9/11 Moral: Repeatig the stuff from the begiig of the algorithm is wasteful! probability of success 5/7 3/5 iteratio

95 Istead a c a,b d b e f f g h g Cotract! c d e h This brach made a bad choice. a,b c d e f,g h Cotract! But it s okay sice this brach made a good choice. a,b c a,b,e d e f,g f,g h Cotract! FORK! a,b c Cotract! a,b d e f,g f,g,h h etc c d h c d e etc

96 I words Ru Karger salgorithm o G for a bit. Util there are u ( superodesleft. The split ito two idepedet copies, G 1 ad G 2 Ru Karger salgorithm o each of those for a bit. Util there are ^ ( = ( superodesleft i each. The split each of those ito two idepedet copies

97 I pseudocode KargerStei(G = (V,E)): V if < 4: fid a mi-cut by brute force Ru Karger salgorithm o G with idepedet repetitios util odes remai. ( G 1, G 2 copies of what s left of G S 1 = KargerStei(G 1 ) S 2 = KargerStei(G 2 ) returwhichever of S 1, S 2 is the smaller cut. \\time O(1)

98 Recursio tree odes ( odes Cotract a buch of edges odes Make 2 ( copies ( odes Cotract a buch of edges Cotract a buch of edges odes odes odes Make 2 copies odes odes Make 2 copies odes 8 odes O odes O odes O odes O odes

99 Recursio tree depth is log ( = t =2log () t ( () umber of leaves is 2 2log() = 2 odes This couts as oe level for this aalysis ( odes Cotract a buch of edges This couts as oe level for this aalysis Cotract a buch of edges odes ( odes Make 2 copies ( odes odes Cotract a buch of edges

100 Two questios Does this work? Is it fast?

101 At the j th level Cotract a buch of edges (ˆ/^ odes ( (ˆ v)/^ odes The amout of work per level is the amout of work eeded to reduce the umber of odes by a factor of 2. That s at most O( 2 ). sice that s the time it takes to ru Karger salgorithm oce, cuttig dow the umber of superodes to two. ( (ˆ v)/^ odes Make 2 copies ( (ˆ v)/^ odes Our recurrece relatio is T() = 2T(/ 2) + O( 2 ) The Master Theorem says T() = O( 2 log()) Jedi Master Yoda

102 Two questios Does this work? Is it fast? Yes, O( 2 log()).

103 Why / 2? Suppose we cotract t edges, util there are t superodes remaiig. Suppose the first -t edges that we choose are e 1,e 2,, e -t PR[ oe of the e i cross S* (up to the -t th) ] = PR[ e 1 does t cross S* ] PR[ e 2 does t cross S* e 1 does t cross S* ] PR[ e -t does t cross S* e 1,,e -t-1 do t cross S* ]

104 Why / 2? Suppose we cotract t edges, util there are t superodes remaiig. Suppose the first -t edges that we choose are e 1,e 2,, e -t PR[ oe of the e i cross S* (up to the -t th) ] = r( r rx = U (UrX) (rx) ] ] ^ ^rx = (rx) 1 2 r r( Choose t=/ 2 rv r whe is large rœ U X r U U U ( UrX U X

105 Recursio tree odes ( odes Cotract a buch of edges Pr[ failure ] = 1/2 Cotract a buch of edges Pr[ failure ] = 1/2 odes odes Make 2 ( copies ( odes odes Cotract a buch of edges Pr[ failure ] = 1/2 odes O odes Make 2 copies Pr[ failure ] = 1/2 odes O odes odes O odes Make 2 copies Pr[ failure ] = 1/2 etc. odes O odes

106 Probability of success Is the probability that there s a path from the root to a leaf with o failures. odes ( odes odes Make 2 ( copies ( odes or Each with probability 1/2 odes odes odes Make 2 copies odes odes Make 2 copies odes O odes O odes O odes O odes

107 The problem we eed to aalyze Let T be biary tree of depth 2log() Each ode of T succeeds or fails idepedetly with probability 1/2 What is the probability that there s a path from the root to ay leaf that s etirely successful?

108 Aalysis Say the tree has height d. Let p d be the probability that there s a path from the root to a leaf that does t fail. p = X ( Pr = X ( Pr Pr at least oe subtree has a successful path wis +Pr both wi wis Cotract a buch of edges 2 ( rx)/( odes 2 /( odes 2 ( rx)/( odes Make 2 copies 2 ( rx)/( odes = X p ( ( rx+p rx p rx = p rx X p ( ( rx

109 It s a recurrece relatio! p = p rx X p ( ( rx p Y =1 We are real good at those. I this case, the aswer is: Claim: for all d, p X X Prove this! (Or see hidde slide for a proof). Siggi the Studious Stork

110 Recurrece relatio p = p rx X p ( ( rx p Y =1 Claim: for all d, p X X Proof: iductio o d. Base case: 1 1. YEP. Iductive step: say d > 0. Suppose that p rx X. p = p rx X p ( ( rx X X ( X ^ X X X = X X This slide skipped i class

111 What does that mea for Karger-Stei? For d = 2log() that is, d = the height of the tree: Claim: for all d, p X X p (t () 1 2log ()+1 aka, Pr[ Karger-Stei is successful ] = Ω X t

112 Altogether ow We ca do the same trick as before to amplify the success probability. Ru Karger-Stei O log log X success probability 1 δ. Each iteratio takes time O ( log That s what we proved before. times to achieve Choosig δ=0.01as before, the total rutime is O ( log log = O ( log ( Much better tha O( 4 )!

113 What have we leared? Just repeatig Karger salgorithm is t the best use of repetitio. We re probably goig to be correct ear the begiig. Istead, Karger-Stei repeats whe it couts. If we wait util there are odes left, the probability ( that we fail is close to ½. This lets us fid a global miimum cut i a udirected graph i time O( 2 log 2 () ). Notice that we ca t do better tha 2 i a dese graph (we eed to look at all the edges), so this is pretty good.

114 Recap Some algorithms: Karger s algorithm for global mi-cut Improvemet: Karger-Stei Some cocepts: Mote Carlo algorithms: Might be wrog, are always fast. We ca boost their success probability with repetitio. Sometimes we ca do this repetitio very cleverly.

115 Next time Aother sort of mi-cut: s-t mi-cut also max-flow! Beforeext time Pre-lecture exercise: examples of cuts ad flows.