CS 331 Design and Analysis of Algorithms. -- Divide and Conquer. Dr. Daisy Tang

Size: px

Start display at page:

Download "CS 331 Design and Analysis of Algorithms. -- Divide and Conquer. Dr. Daisy Tang"

Eugene Benson
5 years ago
Views:

1 CS 33 Desig d Alysis of Algorithms -- Divide d Coquer Dr. Disy Tg

2 Divide-Ad-Coquer Geerl ide: Divide problem ito subproblems of the sme id; solve subproblems usig the sme pproh, d ombie prtil solutios, if eessry.

3 Problem : Fid the Mx d Mi Give uordered list of elemets, fid mx d mi The strightforwrd lgorithm: mx mi A[0]; for i to - do if A[i] > mx mx A[i]; if A[i] < mi mi A[i]; # of ey omprisos: 3

4 Divide-d-Coquer Approh List List List elemets / elemets / elemets mi, mx mi, mx mi, mx mi = MIN mi, mi mx = MAX mx, mx 4

5 The Divide-d-Coquer lgorithm: proedure Rmxmi i, j, fmx, fmi; // i, j re idex #, fmx, { // fmi re output prmeters se: i = j: fmx fmi A[i]; i = j : if A[i] < A[j] the fmx A[j]; fmi A[i]; else fmx A[i]; fmi A[j]; else: mid i j ; } ed se ll Rmxmi i, mid, gmx, gmi; ll Rmxmi mid+, j, hmx, hmi; fmx MAX gmx, hmx; fmi MIN gmi, hmi; 5

6 Exmple: fid mx d mi i the followig rry:, 3, -5, -8, 5, 60, 7, 3, 47 = 9 Idex: Arry: Rmxmi0, 8, 60, , 4,, -8 5, 8, 60, ,,, -5 3, 4, 5, -8 5,6, 60, 7 7, 8, 47, ,,, 3,,-5, -5

7 Alysis of Reursio 7 For lgorithms with reursive lls, we defie reurree reltio to fid its omplexity Let T be the umber of ey omprisos eeded to Rmxmi Reurree reltio otherwise T T 0

8 T T T T T T T Reursio rehes bse ses whe / =, thus = + d = log -, reple bove, we got.5 /... log log log O T

9 I Clss Exerise # Defie the followig reurree reltio d solve it to fid the time omplexity it Qit { if retur ; else retur Q/5+Q/5; } 9

10 Theorem 0 Theorem: Clim: where, b, re ostts They do t hve to be the sme ostt. We me them the sme here for simpliity. It will ot ffet the overll result. O Log O O T Log b T b T

11 Proof: i i b b b b b b b b T b T b b b T b b T b b T T Stops whe = for some iteger b T T

12 if <, i0 i is ostt T = b ostt T = O if =, i i0 T b i0 b Log b O Log

13 3 if >, 0 Log Log Log i i O b b b b b b T 0 i i

14 Problem : Iteger Multiplitio The problem: Multiply two lrge itegers, eh of digits Exmple: 3x4 The trditiol wy: Eh of the digits of the first umber is multiplied by eh of the digits of the seod umber Use two loops, it tes O opertios 4

15 The Divide-d-Coquer Wy Suppose x d y re lrge itegers, divide x ito two prts d b, d divide y ito d d We trsform the problem of multiplyig two -digit itegers ito four sub-problems of multiplyig two / digit itegers x y x y b x 0 b d y 0 d 0 0 b 0 bd 0 d d b 5

16 Alysis So the reurree reltio is: No improvemet over trditiol method Isted, if we write: x y 0 0 bd 0 bd 0 d b Now we hve three subproblems T 3T b Log 3.58 O O T 4T b Log 4 O O b d bd 6

17 The lgorithm by Divide-d-Coquer: proedure multiplitio x,y { if x = 0 or y = 0, retur 0; = MAX # of digits i x, # of digits i y; if =, retur x y i the usul wy; else m = ; = x divide 0 m ; b = x mod 0 m ; = y divide 0 m ; d = y mod 0 m ; p= multiplitio, ; p= multiplitio b, d; p3 = multiplitio +b, +d; m m retur p 0 p 0 p3 p p ; } 7

18 Exmple: x = 43, y = 56 P x = 43 y = 56 = 3 m = = 4 b = 3 = 5 d = 6 P = P P = 8 P 3 = P P = P x = 4 y = 5 = m = = b = 4 = d = 5 P = P = 0 P 3 = 35 P = 350 P x = 7 y = 3 = m = = b = 7 = 3 d = P = 3 P = 7 P 3 = 3 P = 57 8

19 Your Exerise: x = 37 d y =3 P x = 37 y = 3 = 3 m = = b = = d = P = P P = P 3 = P P = P x = y = = m = = b = = d = P = P = P 3 = P = P x = y = = m = = b = = d = P = P = P 3 = P P = P x = y = = m = = b = = d = P = P = P 3 = P = 9

20 Problem #3: MergeSort Give list of umbers, sort them i o-desedig order Ide: split ito two sublists of similr size, sort idividul sublist d merge the sorted sublists The lgorithm: proedure MergeSort low, high { } if the low high mid low high ; ll MergeSort low, mid; ll MergeSort mid+, high; ll Merge low, mid, high; 0

21 MergeSort Exmple A = {7, 0,, 0, 5, 3, 5, }

22 proedure Merge low, mid, high { i low, j mid, low; while i mid d j high if A[i] A[j], the } else U[] A[j]; j++; U[] A[i]; i++; ++; if i > mid move A[j]..A[high] to U[]..U[high]; else move A[i]..A[mid] to U[]..U[high]; for p low to high A[p] U[p];

23 Alysis Worst se for MergeSort T T b O Log Best se? Averge se? I-ple sort? Does ot use y extr spe beyod tht eeded to store the iput 3

24 Problem 4: QuiSort Give list of umbers, sort them i odesedig order Ide: Divide-d-Coquer Split bsed o Pivot elemet Merge is ot eessry Compred with MergeSort 4

25 Prtitio Prtitio splits the set of elemets ito subsets A d B bsed o pivot p Suh tht xa < p < yb Choie of pivot is importt Strightforwrd pproh to prtitio? 5

26 [0] [] [] [3] [4] [5] [6] [7] [8] [9] [0] toobigidex Quisort toosmllidex Strtig t the begiig of the rry, we loo for the first elemet tht is greter th the pivot. toobigidex Strtig from the other ed we loo for the first vlue tht is less th or euql to the pivot. toosmllidex After fidig the two out-of-ple elemets, exhge them. toobigidex++ toosmllidex - - Stop whe two idies ross eh other 6

27 [0] [] [] [3] [4] [5] [6] [7] [8] [9] [0] toobigidex toosmllidex toobigidex toosmllidex toobigidex toosmllidex 7

28 toosmllidex toobigidex Pivot 40 is i its fil positio 8

29 The lgorithm: proedure QuiSort p, q { if p < q, the ll Prtitio p, q, pivotpositio; ll QuiSort p, pivotpositio ; ll QuiSort pivotpositio+, q; } proedure Prtitio first, lst, pp // pp is retur vlue { } pivot = A[first]; tb = first + ; loop { } ts = lst; while tb <= lst && A[tb] < pivot { tb = tb+; } while ts > first &&A[ts] > pivot { ts = ts-; } if tb<ts the swp A[tb] d A[ts] else exit; A[first] = A[ts]; A[ts] = pivot; retur ts; //pivot positio 9

30 I-Clss Exerises # Sort E, X, A, M, P, L, E i lphbetil order by Mergesort d by Quisort. Show steps. Sort 65, 60, 60, 60 3 i oderesig order by Mergesort d by Quisort. Show steps. A sortig lgorithm is lled stble if it preserves the reltive order of y two equl elemets i its iput. Is Mergesort stble? Is Quisort stble? 30

31 Alysis Worst se Pivot is either mi or mx O, how to prove? Worse th MergeSort Best se Split the list evely Olog 3

32 Averge Cse Assume pivot is seleted rdomly, so the probbility of pivot beig the th elemet is equl prob pivot, Let C = # of ey ompriso for items C C Averge it over ll C A represets the vg. omplexity for sortig umbers by QuiSort C A C A C A 3

33 33 Multiply both side by Reple by - Subtrt from 0 C C C C A A A A 0 C C C A A A C C C C C A A A A A C C C A A A

34 34 divide + for both sides Best-Cse = Averge-Cse = OLog : 3 3 Log O Log Log C Log Log d C boudry se C Note C C C C e e A e e A A A A A A

35 Summry for QuiSort Bsed o experimetl results The verge performe of QuiSort is better th tht of the MergeSort Whe 6, isertio sort is fst We hge QuiSort little bit: do t prtitio the list whe 6, it speeds up QuiSort 35

36 I-Clss Exerises #3 Desig vrit biry serh reursive lgorithm whih splits the set ot ito sets of equl sizes, but ito sets of sizes oe third d two thirds. How does this lgorithm ompre with the origil biry serh i both the best se d the worst se? 36

37 Problem 5: Seletio The Problem: give list of elemets, fid the th smllest oe Speil ses: =, mi; =, mx Strtegies: Method : Sort the list d retur the th smllest oe, Olog Method : Sortig is ot required. Prtitio repetedly util we fid the th elemet Te dvtge of the Prtitio lgorithm i Quisort 37

38 Method If pivot_positio =, doe If pivot_positio >, fid the th elemet i the st sublist If pivot_positio <, fid the - pivot_positioth elemet i the d sublist 38

39 proedure Selet A,, // o-reursive lgorithm { m ; j ; loop ll Prtitio m, j, pivotpositio; se: = pivotpositio: retur A; < pivotpositio: j pivotpositio ; else: m pivotpositio + ; ed loop; } 39

40 Exmple A[] = {65, 70, 75, 80, 85, 60, 55, 50, 45} = 5 = 7 40

41 Alysis Best se O whe pivot is the th smllest oe Worst se O Prtitio will be lled O times d eh tes O steps 4

42 Method 3 By hoosig pivot more refully, we obti seletio lgorithm with worst se omplexity O How? Use mm medi of medis rule to hoose pivot elemets re divided ito /r groups of r elemets eh r is smll ostt, for iste, r = 5 or 7. The remiig r/r elemets re ot used. The medi m i of eh of these /r groups is foud. The the medi mm of m i s is foud. Now, we hoose mm s pivot so tht t lest some frtio of the elemets will be smller th pivot d t lest some other frtio of elemets will be greter th pivot. 4

43 Exmple 36 elemets divide ito 7 /r subsets of size 5 r Elemets mm Noderesig Order ll olums mm medi of medis The middle row is i o-deresig order Elemets mm 43

44 proedure Selet A,, { if r, the sort A d retur the th elemet; divide A ito r subset of size r eh, igore exess elemets d let M m, m,, be the set of medis of the subsets; m r V = Selet ; r } use Prtitio to prtitio A usig v s the pivot; // ow ssume v is t pivotpositio se: = pivotpositio: retur v; < pivotpositio: let S be the set of elemets A.. pivotpositio, retur Selet S, pivotpositio, ; else: let R be the set of elemet Apivotpositio+.., retur Selet R, pivotpositio, pivotpositio; ed se; 44

45 Exmple SeletA,, 7 Where A = {5, 30, 65, 70, 75, 80, 85, 60, 55, 50, 45} 45

46 Alysis: How my M i s or mm? At lest How my elemets or mm? At lest sie the medi of r elemets is the smllest elemet How my elemets > mm or < mm? At most Assume r = 5 So, R d S re t most 0.75 whe 4. 46

47 I-Clss Exerise #4 Write equivlet reursive lgorithm for SeletA,, give o slide 39 47

48 Problem 6: Mtrix Multiplitio Problem: Multiply two mtries A d B, eh of size x 48

49 Trditiol Wy Use three for loops 49

50 Divide-d-Coquer Wy Trsform the problem of multiplyig A d B, eh of size by, ito 8 subproblems, eh of size / by / I other words, C be omputed by 8 multiplitios d 4 dditios of / by / mtries 50

51 The imge prt with reltioship ID rid5 ws ot foud i the file. Exmple: Use Divide-d-Coquer to solve it s follows. First, fill 0 olums d rows to me = j for some iteger j. 5

52 Time Complexity of New Algorithm? where is the ost for dditio So, it is o improvemet ompred with the trditiol wy 5

53 Strsse s Mtrix Multiplitio Disovered wy to ompute the C ij s usig 7 multiplitios d 8 dditios or subtrtios 53

55 Alysis 55

56 Alysis, Cotiue 56

57 I-Clss Exerise #5 Apply Strsse s lgorithm to ompute the followig. Show the steps. 57

58 Whe NOT to use Divide-d-Coquer A problem of size is divided to or more subproblems, eh lmost of size A problem of size is divided ito lmost subproblems of size /, where is ostt 58

Merge Sort. Outline and Reading. Divide-and-Conquer. Divide-and-conquer paradigm ( 4.1.1) Merge-sort ( 4.1.1)

Merge Sort. Outline and Reading. Divide-and-Conquer. Divide-and-conquer paradigm ( 4.1.1) Merge-sort ( 4.1.1) Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort versio 1.3 1 Outlie d Redig Divide-d-coquer prdigm ( 4.1.1 Merge-sort ( 4.1.1 Algorithm Mergig two sorted sequeces Merge-sort tree