) 2 2(2. Assume that

Size: px

Start display at page:

Download ") 2 2(2. Assume that"

Bruno Stafford
5 years ago
Views:

1 lg lg W W W W W W W Assume tht lg log. 0. with for, W W W

2 For ot power of, W 0 W W W for hrd to lyze this cse exctly ecuse of the floors d ceiligs. usig iductio rgumet lie the oe i Ex B.5 i Appedix B, it c e show W is odecresig. Therefore, W lg. As i iry serch, we c ssume tht = d lyze the time complexity get the sme result. For similr recursio reltios, we ssume tht =, from ow o.

3 dditiol spce 4+4 A B - eeded spce fter A d B re doe. -reuse spce for A d B + + totl dditiol spce 4 Fig. The steps doe y hum whe sortig with Mergesort

4 Spce Complexity Alysis i-plce sort sortig lgorithm tht does ot use y extr spce eyod tht eeded to store the iput. mergesort is ot i-plce sort ecuse it uses the rrys U d V esides the iput rry S. ew rrys U d V will e creted ech time mergesort is clled. At the top level, the sum of the umers of items i these two rrys is. I the top-level recursive cll, the sum of the umers of items i the two rrys is out /, i the recursive cll t the ext level, the sum of the umer of items i the two rrys is out /4. Therefore, dditiol spce is, 4 possile to reduce the mout of extr spce to oly oe rry cotiig items. ut, still it is ot i-plce sort. 4

5 Mergesort with improved spce complexity prolem: Sort eys i odecresig sequece. iputs: positive iteger, S[..] outputs: S[..] cotiig the eys i odecresig order lgorithm: void mergesortidex low, idex high { idex mid; if low < high { mid = low + high / ; mergesortlow, mid; mergesortmid+, high; mergelow, mid, high; } } mergesort, ;... 5

6 merge prolem: merge the two sorted surrys of S creted i mergesort. iputs: low, mid, high surrys S[low..high], where S[low..mid] d S[mid+..high] re lredy sorted. outputs: S[low..high] sorted i odecresig order. 6

7 lgorithm: void mergeidex low, idex mid, idex high { idex i, j, ; eytype U[low..high]; // locl vrile i = low; j = mid + ; = low; while i <= mid && j <= high { if S[i] < S[j] { U[] = S[i]; i++; } else { U[] = S[j]; j++; } ++; } if i > mid copy S[j] through S[high] to U[] through U[high]; else copy S[i] through S[mid] to U[] through U[high]; copy U[low] through U[high] to S[low] through S[high]; } 7

8 dditiol spce copy it lter copy it lter reuse A A B 4icludig A C 8icludig C totl 8 mergesort steps. Additiol spce is. 8

9 Quicsort Developed y C.A.R. Hore British Computer Scietist i 96. Quicsort my e misuderstood sice it is ot the fstest sortig lgorithm. Prtitio exchge sort is more suitle me. ex:

10 Fig. The steps doe y hum whe sortig with quicsort. The surrys re eclosed i rectgles wheres the pivots re free. 0

11 Quicsort prolem : Sort eys i odecresig order. iputs : positive iteger > 0, S[..] outputs : S[..] sorted i odecresig order lgorithm: void quicsort idex low, idex high { idex pivotpoit; if high > low { prtitiolow,high,pivotpoit; quicsortlow,pivotpoit-; quicsortpivotpoit+,high; } }

12 Prtitio prolem : Prtitio the rry S for quicsort. iputs : low, high the surry of S idexed from low to high. outputs : pivotpoit, the pivot poit for the surry idexed from low to high. void prtitio idex low, idex high, idex& pivotpoit { idex i, j; eytype pivotitem; pivotitem = S[low]; //select the st item s pivotitem j = low; fori = low + ; i <= high; i++ if S[i] < pivotitem { j++; exchge S[i] d S[j]; } pivotpoit = j; exchge S[low] d S[pivotpoit];// put pivotitem ito pivotpoit } j: the right most positio of items less th pivotitem

13 Fig. A exmple of procedure prtitio pivot poit

14 Alysis of Prtitio Every-Cse Time complexity sic opertio: the compriso of S[i] with pivotitem iput size: = high - low +, the umer of items i the surry. lysis: Becuse every item except the first is compred, T = -. 4

15 Alysis of Quicsort Worst-Cse Time Complexity sic opertio : the compriso of S[i] with pivotitem i prtitio. iput size :, the umer of items i the rry S. The worst cse occurs if the rry is lredy sorted i odecresig order. No items re less th the first item i the rry, which is the pivot item. Whe prtitio is clled t the top level, o items re plced to the left of the pivot item, d the vlue of pivotitem ssiged y prtitio is. Similrly, i ech recursive cll, pivotpoit receives the vlue of low. Therefore, T = T0 +T - +. Sice T0 = 0, T = T - + -, for > 0 T0 = 0 5

16 This recurrece is solved i Ex. B.6 i Appedix B. T = T T - = T T - = T T = T + T = T0 + 0 T0 = 0 T We hve estlished tht the worst cse is t lest -/. Although ituitively it my ow seem tht this is s d s thigs c get, we still eed to show this. We will ccomplish this y usig iductio to show tht, for ll, W 6

17 7 Prove tht for ll,. Proof: mthemticl iductio iductio se: For = 0, iductio hypothesis: Assume tht, for 0 <, iductio step: let p e the vlue of pivotpoit retured y prtitio Whe p= or, hs mx vlue. Therefore, W 00 0 W W W p p p p p p p p p p p p W p W W y iductio hypothesis p p p mx p p p p W W

18 8 Averge-Cse Time Complexityquicsort sic opertio : the compriso of S[i] with pivotitem i prtitio. iput size :, the umer of items i the rry S. Let A e the verge sortig time for items. The proility tht pivotpoit is p is /. Sice Averge time to sort surrys whe pivotpoit is p is [Ap - + A - p] d time to prtitio is -, p p p A A A A A A A A A p A p A A 0 0 ] [

19 Multiplyig y we hve, A Applyig to -, A p p A p A p Sutrctig from, A A A A A Dividig oth sides y A A 9

20 If we let A We hve the recurrece 0 0 for 0 Hece,,,...,, 0 0 0

21 Therefore, i i i i i i i i i We c igore the d term i the lst equlity. Sice l = log e, i i l We hve l. Ad lg = l / l, A l l lg.8 lg l lg 0.69 Quicsort is good lgorithm with O lg.

22 Mtrix Multiplictio prolem : Determie the product of two mtrices. iputs : positive iteger, A[..][..] d B[..][..]. outputs : C[..][..]. void mtrixmult it, cost umer A[][], cost umer B[][], umer C[][] { idex i, j, ; for i = ; i <= ; i++ for j = ; j <= ; j++ { C[i][j] = 0; for = ; <= ; ++ C[i][j] = C[i][j] + A[i][] * B[][j]; } }

23 Time Complexity Alysis I: sic opertio : oe elemetry multiplictio iput size : Every-Cse Time Complexity: Time Complexity Alysis IImodified. for exercise sic opertio : oe elemetry dditio iput size : Every-Cse Time Complexity: T T

24 Multiplictio of two Mtrices simple method prolem: the product C of two Mtrices c c c c We eed 8 multiplictios d 4 dditios/sutrctios. 4

25 Strsse s Algorithm prolem: the product C of two Mtrices c c c c Strsse s method: m m4 m5 m7 m C m m4 m m where m m m m 4 m5 m6 m lysis: Strsse s method requires 7 multiplictios d 8 dditios/sutrctios. Seems to e ot very impressive. But, for lrge mtrices, Strsse s method shows good performce. 7 m m 5 m 6 5

26 Product of two Mtrices: Strsse s Algorithm prolem: Determie the product of two mtrices where is power of. We divide the mtrices A d B, ito 4 sumtrices. C C A A B C C A A B Strsse s Algorithm : M M 4 M 5 M 7 M C M M 4 M M where M A A B B M A A B M A B B M 4 A B B M5 A A B M6 A A B B M A A B 7 B B B M M 5 M 6 6

27 Strsse s Algorithm prolem: Determie the product of two mtrices where is power of. iput: = power of, two mtrices A d B. output: the product C of A d B. void strsse it, *_mtrix A, *_mtrix B, *_mtrix& C { } if <= threshold compute C = A*B usig the stdrd lgorithm; else { } prtitio A ito 4 sumtrices A, A, A, A ; prtitio B ito 4 sumtrices B, B, B, B ; compute C = A*B usig Strsse s Algorithm; // smple recursive cll // strsse/, A +A, B +B,M threshold: the poit t which we feel it is more efficiet to use the stdrd lgorithm th it would e to cll procedure strsse recursively. 7

28 Alysis Every-Cse Time Complexity of Numer of MultiplictiosSimple Method T: The time required to multiply mtrices A d B. sic opertio: oe elemetry multiplictio. iput size:, the umer of rows d colums i the mtrices. Every-Cse Time Complexity : Let threshold =. the threshold does ot me y effect o the order The recurrece is T 8T, T We expd the recurrece d get, 8 8 for, T 888 lg lg 8 times 8

29 Every-Cse Time Complexity of Numer of Multiplictios Strsse s Algorithm I sic opertio: oe elemetry multiplictio. iput size:, the umer of rows d colums i the mtrices. Every-Cse Time Complexity : Let threshold =. the threshold does ot me y effect o the order The recurrece is T 7T, T We expd the recurrece d get, T lg lg 7.8 for,.8 times For mig mtrix size power of, isert 0s i the mtrix. 9

30 Every-Cse Time Complexity of Numer of Additios/Sutrctios Strsse s Algorithm II sic opertio: oe elemetry dditio/sutrctio. iput size:, the umer of rows d colums i the mtrices. Every-Cse Time Complexity : Let threshold =. The recurrece is T 7T 8 for, T 0 Usig Mster s TheoremEx B.0 i Appedix B, the solutio is T 6 6 lg

31 multiplictios dditios/sutrctios Stdrd Algorithm Strsse s Algorithm Tle. A compriso of two lgorithms tht multiply mtrices

32 Discussio Whether mtrix multiplictio c e doe i qudrtic time remis ope questio. No oe hs ever creted qudrtic time lgorithm for mtrix multiplictio, d o oe hs prove tht it is ot possile to crete such lgorithm. Shumel Wiogrd: developed vrit of Strsse s lgorithm with 5 dditios/sutrctios.8 T 5 5 Coppersmith d Wiogrd987: sic opertio: multiplictio T 5 The costt is so lrge tht Strsse s lgorithm is usully more efficiet..8

33 Arithmetic with Lrge Itegers Cosider rithmetic opertios o lrge itegers. - stroomy Use rry of itegers to represet lrge iteger. ex 54, S[6] S[5] S[4] S[] S[] S[] : the umer of digits i the lrge itegers - simple multiplictio of itegers requires elemetry multiplictios. Additio/Sutrctio requires lier time. - lier time: u 0 m, u divide 0 m, u rem 0 m 567,8 = , 9,4,7 = u = x 0 m + y digits / digits / digits m u = x 0 m + y, v = w 0 m + z uv = x 0 m + yw 0 m + z = xw 0 m + xz+wy 0 m + yz

34 Lrge Iteger Multiplictio prolem : Multiply lrge itegers, u d v. iputs: lrge itegers, u d v. outputs: prod the product of u d v. lrge_iteger prodlrge_iteger u, lrge_iteger v{ lrge_iteger x, y, w, z; it, m; } = mximumumer of digits i u, umer of digits i v; ifu == 0 v == 0 retur 0; else if <= threshold retur u v otied i the usul wy; else{ m = / ; x = u divide 0 m ; y = u rem 0 m ; w = v divide 0 m ; z = v rem 0 m ; retur prodx, w 0 m + prodx, z+prodw, y 0 m + prody, z; } 4

35 Worst-Cse Time ComplexityLrge Itegers Multiplictio: sic opertio: The mipultio of oe deciml digit i lrge iteger whe ddig, sutrctig, or doig divide 0 m, rem 0 m, 0 m. iput size:, the umer of digits i ech of the two itegers. Assume tht it is power of. Additio, sutrctio, divide 0 m, rem 0 m, 0 m ll hve lier-time complexities i terms of, c. W W s 4W 0 c, for s, is power of W lg4 Appedix Theorem B.5 The Mster Theorem 5

36 Improved Method: I Alg..9, xw, xz+yw, yz re computed. 4 multiplictios improved method r = x+yw+z=xw+xz+yw+yz xz+yw = r xw yz - We eed oly multiplictios though the umer of dditios/sutrctios re icresed for computig x+y, w+z, r=xz+yw tht tes lier time. 6

37 multiplictio of Lrge Iteger Multiplictio prolem : Multiply lrge itegers, u d v. iputs: lrge itegers, u d v. outputs: prod the product of u d v. lrge_iteger prodlrge_iteger u, lrge_iteger v{ lrge_iteger x, y, w, z, r, p, q; it, m; = mximumumer of digits i u, umer of digits i v; ifu == 0 v == 0 retur 0; else if <= threshold retur u v otied i the usul wy; else{ m = / ; x = u divide 0 m ; y = u rem 0 m ; w = v divide 0 m ; z = v rem 0 m ; r = prodx+y,w+z; p = prodx, w; q = prody, z; retur p 0 m + r p-q 0 m + q; } } 7

38 Worst-Cse Time Complexity of prod: prodx+y, w+z / iput size /+ prodx, w / prody,z / W c W s 0 W W c, for s, is power of W lg.58 8

39 Determiig Thresholds Determie for which vlues of it is t lest s fst to cll ltertive lgorithm s it is to divide the istce further. these vlues deped o the divide-d-coquer lgorithm, the ltertive lgorithm, d the computer. threshold: A istce size such tht for y smller istce it would e t lest s fst to cll the other lgorithm s it would e to divide the istce further. 9

40 Time Complexity of mergesort ssume tht mergesort tes to divide d recomie istce of size is μs ruig time i the origil recurrece it is - timeslogicl. By simplifyig, W W W s, W 0 s W W / s, W 0 s W lg s, is power of Exchge sorts tes exctly W s 40

41 Mye elieve tht the optiml poit where mergesort should cll Exchge sort c ow e foud y solvig the iequlity The solutio is <59. s lg s - Wrog!! W lg s is the complexity of mergesort if we eep dividig util =. [ex.7] exchge sort is clled whe t. s W W W t t W W t t t t t s t t t t t t t Sice t is threshold, for =t/, use tt-/. t is eve; t =8 odd t =8.008 [result] optiml t =8 4

42 Whe Not to Use Divide-d-Coquer A istce of size is divided ito two or more istces ech lmost of size. time complexity : expoetil time. ex T T A istce of size is divided ito lmost istces of size /c, where c is costt. time complexity : lg. ex T T / 4

43 The Mster Theorem For >, >, fuctio f, oegtive iteger, T is give y The, T hs the followig symptotic oud. log. If for costt > 0, the T log log If f, the T log. log. If f, for costt > 0, c < d for ll lrge, f c f, the T f, where c e or. f T T log f,. 4

44 Exmples of the Mster s Theorem 9T T log log 9 = 9, =, f =,, log if =, 9 f, ccordig to the Mster s Theorem, T 9 log T T log log 0 =, =, f =,,. Accordig to the Mster s Theorem, f T lg 44

45 Exmples of the Mster s Theorem T T 4 lg log =, = 4, f = lg, log4 0.79, log If =, 4 f. I order to use the Mster s Theorem, we hve to show tht for ll lrge, there exists c< stisfyig f 4 c f. c If, for ll lrge. Hece, 4 4 lg 4 4 T T lg lg T lg log log =, =, f = lg,, f. I order to use the Mster s Theorem, we hve to show tht for ll lrge, there exists c< stisfyig f. But, there does ot exist such c stisfyig c f lg lg c lg for lrge sice. log c lg Therefore, we c ot use the Mster s Theorem d the followig Corollry c e used i this cse. 45

46 Corollry f T T For some 0, f = log lg, T log Sice, lg T T lg. f lg T lg 46

47 47 d T c T T of power is, lg log if if if T ex 5 4 / 8 T T T 4 8, T ex 7 5 / 9 T T T 9, 9 log T ex 00 5 / 8 T T T 8, log T Theorem B.5 i Appedix B.

48 48 of power is, c T T lg log if O if O if O T Also, of power is, c T T lg log if if if T

49 Ch. Dymic Progrmmig Textoo: Foudtios of Algorithms Usig C++ Pseudocode y R. Nepolit d K. Nimipour This is prepred oly for the clss.

50 Dymic Progrmmig Divide-d-coquer pproch is top-dow pproch. efficiet for prolems i which suistces re urelted. Suistces i Fiocci lgorithmrecursive re relted. duplicte clcultio of fi iefficiet divide-d-coquer pproch is ot suitle to th Fiocci Term Prolem. Dymic progrmmig pproch is ottom-up pproch. As i divide-d-coquer pproch, divide istce of prolem ito suitces. Solve them d store the results, d lter, we loo them up. Avoid duplicte computtios y usig idices effectively. Developmet procedure Estlish recursive property. Solve istce of the prolem i ottom-up fshio y solvig smller istces first. 50

51 5 The Biomil Coefficiet Biomil Coefficiet formul For vlues of d tht re ot smll, we c ot compute! or!00!?. We estlish tht C 0 for!!! or 0 if 0 if

52 Biomil Coefficiet Algorithmdivided-coquer pproch prolem: Compute the iomil coefficiet. iputs: itegers d, outputs: i, lgorithm: it iit, it { if == 0 == retur ; else retur i-,- + i-, } 5

53 Time Complexity Alysisdivide-d-coquer esy to desig, ut ot efficiet. resos : the sme istces re solved i ech recursive cll. ex i-,- d i-, oth eed the result of i-,-, d this istce is solved i ech recursive cll. To determie C, C - terms re computed. 5

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Divide-and-Conquer

Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, Divide-and-Conquer Presettio for use with the textook, Algorithm Desig d Applictios, y M. T. Goodrich d R. Tmssi, Wiley, 25 Divide-d-Coquer Divide-d-Coquer Divide-d coquer is geerl lgorithm desig prdigm: Divide: divide the