Data Structures Lecture 9 - PDF Free Download

Fall 2017 Fag Yu Software Security Lab. Dept. Maagemet Iformatio Systems, Natioal Chegchi Uiversity Data Structures Lecture 9

Midterm o Dec. 7 (9:10-12:00am, 106) Lec 1-9, TextBook Ch1-8, 11,12 How to prepare your midterm: Uderstad ALL the materials metioed i the slides Discuss with me, your TAs, or classmates Read the text book to help you uderstad the materials You are allowed to brig a A4 size ote Prepare your ow ote; write whatever you thik that may help you get better scores i the midterm

7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Fudametal Algorithms Divide ad Coquer: Merge-sort, Quick-sort, ad Recurrece Aalysis

Divide-ad-Coquer A geeral algorithm desig paradigm Divide: divide the iput data S i two or more disjoit subsets S 1, S 2, Recursio: solve the sub problems recursively Coquer: combie the solutios for S 1, S 2,, ito a solutio for S The base case for the recursio are subproblems of a costat size Aalysis ca be doe usig recurrece equatios

Merge-sort Merge-sort is a sortig algorithm based o the divide-adcoquer paradigm Like heap-sort It uses a comparator It has O( ) ruig time Ulike heap-sort It does ot use a auxiliary priority queue It accesses data i a sequetial maer (suitable to sort data o a disk)

Merge-sort Merge-sort o a iput sequece S with elemets cosists of three steps: Divide: partitio S ito two sequeces S 1 ad S 2 of about /2 elemets each Recur: recursively sort S 1 ad S 2 Coquer: merge S 1 ad S 2 ito a uique sorted sequece Algorithm mergesort(s, C) Iput sequece S with elemets, comparator C Output sequece S sorted accordig to C if S.size() > 1 (S 1, S 2 ) partitio(s, /2) mergesort(s 1, C) mergesort(s 2, C) S merge(s 1, S 2 )

Mergig Two Sorted Sequeces The coquer step of mergesort cosists of mergig two sorted sequeces A ad B ito a sorted sequece S cotaiig the uio of the elemets of A ad B Mergig two sorted sequeces, each with /2 elemets ad implemeted by meas of a doubly liked list, takes O() time Algorithm merge(a, B) Iput sequeces A ad B with /2 elemets each Output sorted sequece of A B S empty sequece while A.isEmpty() B.isEmpty() if A.first().elemet() < B.first().elemet() S.addLast(A.remove(A.first())) else S.addLast(B.remove(B.first())) while A.isEmpty() S.addLast(A.remove(A.first())) while B.isEmpty() S.addLast(B.remove(B.first())) retur S

Merge-Sort Tree 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 A executio of merge-sort is depicted by a biary tree each ode represets a recursive call of merge-sort ad stores usorted sequece before the executio ad its partitio sorted sequece at the ed of the executio the root is the iitial call the leaves are calls o subsequeces of size 0 or 1

A executio example 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Partitio 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Recur: base case 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Recur: Base case 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Merge 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Recursive call,, merge 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Merge 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Recursive call,, merge, merge 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 6 8 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Merge 7 2 9 4 3 8 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 6 8 7 2 2 7 9 4 4 9 3 8 3 8 6 1 1 6 7 7 2 2 9 9 4 4 3 3 8 8 6 6 1 1

Aalysis of Merge-sort The height h of the merge-sort tree is O( ) at each recursive call we divide i half the sequece, The overall amout or work doe at the odes of depth i is O() we partitio ad merge 2 i sequeces of size /2 i we make 2 i+1 recursive calls Thus, the total ruig time of merge-sort is O( ) depth #seqs size 0 1 1 2 /2 i 2 i /2 i

Quick-sort A radomized sortig algorithm based o the divide-ad-coquer paradigm: x Divide: pick a radom elemet x (called pivot) ad partitio S ito L elemets less tha x E elemets equal x x G elemets greater tha x Recur: sort L ad G L E G Coquer: joi L, E ad G x

Partitio We partitio a iput sequece as follows: We remove, i tur, each elemet y from S ad We isert y ito L, E or G, depedig o the result of the compariso with the pivot x Each isertio ad removal is at the begiig or at the ed of a sequece, ad hece takes O(1) time Thus, the partitio step of quick-sort takes O() time Algorithm partitio(s, p) Iput sequece S, positio p of pivot Output subsequeces L, E, G of the elemets of S less tha, equal to, or greater tha the pivot, resp. L, E, G empty sequeces x S.remove(p) while S.isEmpty() y S.remove(S.first()) if y < x L.addLast(y) else if y = x E.addLast(y) else { y > x } G.addLast(y) retur L, E, G

Quick-Sort Tree 7 4 9 6 2 2 4 6 7 9 4 2 2 4 7 9 7 9 2 2 9 9 A executio of quick-sort is depicted by a biary tree Each ode represets a recursive call of quick-sort ad stores Usorted sequece before the executio ad its pivot Sorted sequece at the ed of the executio The root is the iitial call The leaves are calls o subsequeces of size 0 or 1

Executio Example Pivot selectio 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 7 2 9 4 2 4 7 9 3 8 6 1 1 3 8 6 2 2 9 4 4 9 3 3 8 8 9 9 4 4

24 Partitio, recursive call, pivot selectio 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 2 4 3 1 2 4 7 9 3 8 6 1 1 3 8 6 2 2 9 4 4 9 3 3 8 8 9 9 4 4 Quick-Sort

25 Partitio, recursive call, base case 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 2 4 3 1 2 4 7 3 8 6 1 1 3 8 6 1 1 9 4 4 9 3 3 8 8 9 9 4 4 Quick-Sort

26 Recursive call,, base case, joi 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 2 4 3 1 1 2 3 4 3 8 6 1 1 3 8 6 1 1 4 3 3 4 3 3 8 8 9 9 4 4 Quick-Sort

27 Recursive call, pivot selectio 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 2 4 3 1 1 2 3 4 7 9 7 1 1 3 8 6 1 1 4 3 3 4 8 8 9 9 9 9 4 4 Quick-Sort

28 Partitio,, recursive call, base case 7 2 9 4 3 7 6 1 1 2 3 4 6 7 8 9 2 4 3 1 1 2 3 4 7 9 7 1 1 3 8 6 1 1 4 3 3 4 8 8 9 9 9 9 4 4 Quick-Sort

29 Joi, joi 7 2 9 4 3 7 6 1 1 2 3 4 6 7 7 9 2 4 3 1 1 2 3 4 7 9 7 17 7 9 1 1 4 3 3 4 8 8 9 9 9 9 4 4 Quick-Sort

I-place Quick-sort Quick-sort ca be implemeted to ru iplace I the partitio step, we use replace operatios to rearrage the elemets The recursive calls cosider elemets with rak less tha h elemets with rak greater tha k Algorithm iplacequicksort(s, l, r) Iput sequece S, raks l ad r Output sequece S with the elemets of rak betwee l ad r rearraged i icreasig order if l r retur i a radom iteger betwee l ad r x S.elemAtRak(i) (h, k) iplacepartitio(x) iplacequicksort(s, l, h - 1) iplacequicksort(s, k + 1, r)

I-Place Quick-Sort Perform the partitio usig two idices to split S ito L, E, G Algorithm Quicksort(leftBoud, rightboud, S) If(leftBoud>=rightBoud) retur; Set rightboud as the pivot (x = S[righBoud]) Set j = leftboud; k = rightboud-1; Whe j<k: Sca j to the right (j++) util j >= k or the elemet S[j] > x. Sca k to the left (k--) util j>=k or the elemet S[k]<=x. Swap elemets if j < k Swap pivot with j Quicksort(leftBoud, j-1, S); Quicksort(j+1, rightboud, S)

I-Place Quick-Sort j k 3 2 5 1 0 7 3 5 9 2 7 9 8 9 7 9 6 (pivot = 6) j k 3 2 5 1 0 7 3 5 9 2 7 9 8 9 7 9 6

Summary of Sortig Algorithms Algorithm Time Notes selectio-sort O( 2 ) i-place slow (good for small iputs) isertio-sort O( 2 ) quick-sort O( ) expected heap-sort O( ) merge-sort O( ) i-place slow (good for small iputs) i-place, radomized fastest (good for large iputs) i-place fast (good for large iputs) sequetial data access fast (good for huge iputs)

Recurrece Equatio Aalysis The coquer step of merge-sort cosists of mergig two sorted sequeces, each with /2 elemets ad implemeted by meas of a doubly liked list, takes at most b steps, for some costat b. Likewise, the basis case ( < 2) will take at b most steps. Therefore, if we let T() deote the ruig time of merge-sort: T ( ) = 2T ( / b 2) + b if if < 2 2

Recurrece Equatio Aalysis We ca therefore aalyze the ruig time of merge-sort by fidig a closed form solutio to the above equatio. That is, a solutio that has T() oly o the left-had side. We ca achieve this by iterative substitutio: I the iterative substitutio, or plug-ad-chug, techique, we iteratively apply the recurrece equatio to itself ad see if we ca fid a patter

Iterative Substitutio T ( ) = 2T ( / 2) + b 2 = 2(2T ( / 2 )) + b( / 2)) + b 2 2 = 2 T ( / 2 ) + 2b 3 3 = 2 T ( / 2 ) + 3b 4 4 = 2 T ( / 2 ) + 4b =... i i = 2 T ( / 2 ) + ib Note that base, T()=b, case occurs whe 2 i =. That is, i =. So, T( ) = b + b Thus, T() is O( ).

The Recursio Tree Draw the recursio tree for the recurrece relatio ad look for a patter: depth T s size 0 1 1 2 /2 i 2 i /2 i b T ( ) = 2T ( / 2) + b if < 2 if 2 time b b b Total time = b + b (last level plus all previous levels)

Guess-ad-Test Method I the guess-ad-test method, we guess a closed form solutio ad the try to prove it is true by iductio: For example: b T ( ) = 2T ( / 2) + b Guess: T() < c if < 2 if 2

Guess-ad-Test Method T() = 2T( /2) +b < 2(c( /2)( /2)) + b = c( 2) + b = c c + b < c (?) Wrog! We caot make this last lie be less tha c

Guess-ad-Test Method, (cot.) Recall the recurrece equatio: T ( ) b = 2T ( / 2) + b if < 2 if 2 Guess #2: T() < c 2. T() = 2T( /2) +b = 2(c( /2) 2 ( /2)) + b = c( 2) 2 + b = c 2 2c + c + b c 2 (if c>b) So, T() is O( 2 ). I geeral, to use this method, you eed to have a good guess ad you eed to be good at iductio proofs.

Master Method May divide-ad-coquer recurrece equatios have the form: T ( ) = at ( c / b) + f ( ) if if < d d

Master Method The Master Theorem: 1. for some ) ( ) / ( provided )), ( ( is ) ( the ), ( is ) ( if 3. ) ( is ) ( the ), ( is ) ( if 2. ) ( is ) ( the ), ( is ) ( if 1. 1 < Θ Ω Θ Θ Θ + + δ δ ε ε f b af f T f T f T O f a k a k a a a b b b b b

Master Method, Example 1 T ( ) = 4T ( / 2) + The form: T ( ) c = at ( / b) + f ( ) if if < d d The Master Theorem: b 1. if f ( ) is O( 2. 3. if if provided Solutio: f ( ) is Θ( f ( ) is Ω( a ε a+ ε ), the T( ) is Θ( af ( / b) δf ( ) ), the T( ) is Θ( f a = 4, b =2, f() is b a=2, so case 1 says T() is O( 2 ) b b a k ), the T( ) is Θ( ( )), for someδ < 1. b a ) b a k+ 1 )

Master Method, Example 2 T( ) = 2T ( / 2) + The form: T ( ) c = at ( / b) + f ( ) if if < d d The Master Theorem: b 1. if f ( ) is O( 2. 3. if if provided Solutio: f ( ) is Θ( f ( ) is Ω( b b a ε a a+ ε ), the T( ) is Θ( af ( / b) δf ( ) ), the T( ) is Θ( ), the T( ) is Θ( f ( )), for someδ < 1. a = 2, b =2 Solutio: b a=1, so case 2 says T() is O( 2 ). k b a ) b a k+ 1 )

Master Method, Example 3 T( ) = T( / 3) + The form: T ( ) c = at ( / b) + f ( ) if if < d d The Master Theorem: b 1. if f ( ) is O( 2. if f 3. if f ( ) is Θ( ( ) is Ω( b b a ε a a+ ε ), the T( ) is Θ( ), the T( ) is Θ( ), the T( ) is Θ( f ( )), provided af ( / b) δf ( ) for someδ < 1. Solutio: a = 1, b =3 b a=0, so case 3 says T() is O( ). k b a ) b a k+ 1 )

Master Method, Example 4 T ( ) = 8T ( / 2) + The form: T ( ) 2 c = at ( / b) + f ( ) if if < d d The Master Theorem: b 1. if f ( ) is O( 2. if f 3. if f ( ) is Θ( ( ) is Ω( b b a ε a a+ ε ), the T( ) is Θ( ), the T( ) is Θ( ), the T( ) is Θ( f ( )), provided af ( / b) δf ( ) for someδ < 1. Solutio: a = 8, b =2 b a=3, so case 1 says T() is O( 3 ). k b a ) b a k+ 1 )

HW8 (Due o Dec. 14) Quick sort keywords! Implemet a quick sort algorithm for keywords Add each keyword ito a array/liked list iorder Sort the keywords upo request Output all the keywords

Operatios Give a sequece of operatios i a txt file, parse the txt file ad execute each operatio accordigly operatios add(keyword k) sort() output() descriptio Isert a keyword k to a array Sort the keywords usig quick sort Output all keywords i the array

A iput file Similar to HW7, 1. You eed to read the sequece of operatios from a txt file 2. The format is firm 3. Raise a exceptio if the iput does ot match the format add Fag 3 add Yu 5 add NCCU 2 add UCSB 1 output add MIS 4 Sort output [Fag, 3][Yu, 5][NCCU, 2][UCSB, 1] [UCSB, 1][NCCU, 2][Fag, 3][MIS, 4] [Yu, 5]