Desig ad Aalysis of Algorithms CSE 53 Lecture 9 Media ad Order Statistics Juzhou Huag, Ph.D. Departmet of Computer Sciece ad Egieerig Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms
Medias ad Order Statistics The ith order statistic of elemets S{a, a,, a } : ith smallest elemets Also called selectio problem Miimum ad maximum Media, lower media, upper media Selectio i expected/average liear time Selectio i worst-case liear time Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms
Order Statistics The ith order statistic i a set of elemets is the ith smallest elemet The miimum is thus the st order statistic The maximum is (duh) the th order statistic The media is the / order statistic If is eve, there are medias How ca we calculate order statistics? What is the ruig time? Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 3
Order Statistics How may comparisos are eeded to fid the miimum elemet i a set? The maximum? Ca we fid the miimum ad maximum with less tha twice the cost? Yes: Walk through elemets by pairs Compare each elemet i pair to the other Compare the largest to maximum, smallest to miimum Total cost: 3 comparisos per elemets O(3/) Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 4
O(lg ) Algorithm Suppose elemets are sorted by a O(lg ) algorithm, e.g., MERGE-SORT Miimum: the first elemet Maximum: the last elemet The ith order statistic: the ith elemet. Media: If is odd, the ((+)/)th elemet. If is eve, the ( (+)/ )th elemet, lower media the ( (+)/ )th elemet, upper media All selectios ca be doe i O(), so total: O(lg ). Ca we do better? Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 5
Selectio i Expected Liear Time O() Select ith elemet A divide-ad-coquer algorithm RANDOMIZED- SELECT Similar to quicksort, partitio the iput array recursively Ulike quicksort, which works o both sides of the partitio, just work o oe side of the partitio. Called prue-ad-search, prue oe side, just search the other side). Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 6
Fidig Order Statistics: The Selectio Problem A more iterestig problem is selectio: fidig the ith smallest elemet of a set We will show: A practical radomized algorithm with O() expected ruig time A cool algorithm of theoretical iterest oly with O() worst-case ruig time Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 7
Radomized Selectio Key idea: use partitio() from quicksort But, oly eed to examie oe subarray This savigs shows up i ruig time: O() We will agai use a slightly differet partitio tha the book: q RadomizedPartitio(A, p, r) A[q] A[q] p q r Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 8
Radomized Selectio RadomizedSelect(A, p, r, i) if (p r) the retur A[p]; q RadomizedPartitio(A, p, r) k q - p + ; if (i k) the retur A[q]; if (i < k) the retur RadomizedSelect(A, p, q-, i); else retur RadomizedSelect(A, q+, r, i-k); k A[q] A[q] p q r Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 9
Radomized Selectio Aalyzig RadomizedSelect() Worst case: partitio always 0:- T() T(-) + O()??? O( ) (arithmetic series) No better tha sortig! Best case: suppose a 9: partitio T() T(9/0) + O()??? O() (Master Theorem, case 3) Better tha sortig! What if this had bee a 99: split? Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 0
Radomized Selectio Average case For upper boud, assume ith elemet always falls i larger side of partitio: T ( ) T ( max( k, k) ) + Θ( ) k 0 T ( k) + Θ( ) k / Let s show that T() O() by substitutio What happeed here? Max(k-, -k)k- if k> / Max(k-, -k)-k if k< / Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms
CSE53 Desig ad Aalysis of Algorithms Dept. CSE, UT Arligto What happeed he Split the recurrece What happeed he What happeed here? What happeed here? Radomized Selectio Assume T() c for sufficietly large c: ( ) ( ) ( ) ( ) ( ) ( ) ( ) c c c k k c ck k T T k k k k + Θ + Θ + Θ + Θ + Θ ) ( ) ( / / The recurrece we started with Substitute T() c for T(k) Expad arithmetic series Multiply it out
Radomized Selectio Assume T() c for sufficietly large c: T ( ) c c c c c c + + Θ( ) 4 c c c + Θ( ) 4 c c c + Θ( ) 4 c (if c is big eough) ( ) + Θ( ) The recurrece so far What Multiply happeed it out here? What Subtract happeed c/ here? What Rearrage happeed the arithmetic here? What we set out to What happeed here? prove Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 3
Worst-Case Liear-Time Selectio Radomized algorithm works well i practice What follows is a worst-case liear time algorithm, really of theoretical iterest oly Basic idea: Geerate a good partitioig elemet Call this elemet x Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 4
Worst-Case Liear-Time Selectio The algorithm i words:. Divide elemets ito groups of 5. Fid media of each group (How? How log?) 3. Use Select() recursively to fid media x of the /5 medias 4. Partitio the elemets aroud x. Let k rak(x) 5. if (i k) the retur x if (i < k) the use Select() recursively to fid ith smallest elemet i first partitio else (i > k) use Select() recursively to fid (i-k)th smallest elemet i last partitio Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 5
Worst-Case Liear-Time Selectio (Sketch situatio o the board) How may of the 5-elemet medias are x? At least / of the medias /5 / /0 How may elemets are x? At least 3 /0 elemets For large, 3 /0 /4 (How large?) So at least /4 elemets x Similarly: at least /4 elemets x Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 6
Worst-Case Liear-Time Selectio Thus after partitioig aroud x, step 5 will call Select() o at most 3/4 elemets The recurrece is therefore: T ( ) Dept. CSE, UT Arligto T T ( 5 ) + T ( 3 4) + Θ( ) ( 5) + T ( 3 4) + Θ( ) c 5 + 3c 4 + Θ( ) 9c 0 + Θ( ) c ( c 0 Θ( ) ) c if c is big eough??? /5 /5??? Substitute T() c??? Combie fractios??? Express i desired form??? What we set out to prove CSE53 Desig ad Aalysis of Algorithms 7
Worst-Case Liear-Time Selectio Ituitively: Work at each level is a costat fractio (9/0) smaller Geometric progressio! Thus the O() work at the root domiates Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 8
Liear-Time Media Selectio Give a black box O() media algorithm, what ca we do? ith order statistic: Fid media x Partitio iput aroud x if (i (+)/) recursively fid ith elemet of first half else fid (i - (+)/)th elemet i secod half T() T(/) + O() O() Ca you thik of a applicatio to sortig? Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 9
Liear-Time Media Selectio Worst-case O( lg ) quicksort Fid media x ad partitio aroud it Recursively quicksort two halves T() T(/) + O() O( lg ) Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms 0
Summary The ith order statistic of elemets S{a, a,, a } : ith smallest elemets: Miimum ad maximum. Media, lower media, upper media Selectio i expected/average liear time Worst case ruig time Prue-ad-search Selectio i worst-case liear time: Dept. CSE, UT Arligto CSE53 Desig ad Aalysis of Algorithms