5. DIVIDE AND CONQUER I

Transcription

1 5. DIVIDE AND CONQUER I mergesort counting inversions randomized quicksort median and selection closest pair of points Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley Last updated on 3/12/18 9:21 AM

2 Divide-and-conquer paradigm Divide-and-conquer. Divide up problem into several subproblems (of the same kind). Solve (conquer) each subproblem recursively. Combine solutions to subproblems into overall solution. Most common usage. Divide problem of size n into two subproblems of size n / 2. Solve (conquer) two subproblems recursively. Combine two solutions into overall solution. Consequence. Brute force: Θ(n 2 ). Divide-and-conquer: O(n log n). O(n) time O(n) time attributed to Julius Caesar 2

3 5. DIVIDE AND CONQUER mergesort counting inversions randomized quicksort median and selection closest pair of points SECTIONS

4 Sorting problem Problem. Given a list L of n elements from a totally ordered universe, rearrange them in ascending order. 4

5 Sorting applications Obvious applications. Organize an MP3 library. Display Google PageRank results. List RSS news items in reverse chronological order. Some problems become easier once elements are sorted. Identify statistical outliers. Binary search in a database. Remove duplicates in a mailing list. Non-obvious applications. Convex hull. Closest pair of points. Interval scheduling / interval partitioning. Scheduling to minimize maximum lateness. Minimum spanning trees (Kruskal s algorithm).... 5

6 Mergesort Recursively sort left half. Recursively sort right half. Merge two halves to make sorted whole. input A L G O R I T H M S sort left half A G L O R I T H M S sort right half A G L O R H I M S T merge results A G H I L M O R S T 6

7 Merging Goal. Combine two sorted lists A and B into a sorted whole C. Scan A and B from left to right. Compare ai and bj. If ai bj, append ai to C (no larger than any remaining element in B). If ai > bj, append bj to C (smaller than every remaining element in A). sorted list A sorted list B ai bj merge to form sorted list C

8 Mergesort implementation Input. List L of n elements from a totally ordered universe. Output. The n elements in ascending order. MERGE-SORT(L) IF (list L has one element) RETURN L. Divide the list into two halves A and B. A MERGE-SORT(A). B MERGE-SORT(B). L MERGE(A, B). RETURN L. T(n / 2) T(n / 2) Θ(n) 8

9 <latexit sha1_base64="g9biiiqvnrc2nlak0qbomus1fe8=">aaacy3icbvfdb9mwfhuypkr56syjl1d0ockkljmq2mkctekfjzsklk2qo8pxbjprjhpzdmojfesp8k/4nzhzjtgo++kjc67p/upkkywnwz+ev3fv/oohvuf9x0+epns+2d/4zopkc5zxqhb6mmegpva4s8jkvcw1sjyrejfcf2r0i++ojsju1k5ljho2vcitnflhlqa/pyp1fugpuinn06cjlowqufm0mz49deenuisrw4pi4fdbb4goyqouzsngjhnscqyjkjnzfbru0tfq3elg9z1zbesoqnzaa28ltev98da7t1glxrolwtamwjbglog6mcrdnc/2vqoafrzkuvkumthzkcxtxdntbzfopqomloxfsyxohvqsrxpx7ti38noxkwrumkxqflr23x81y41z54nlzjm9mrtaq/5pm1c2m8s1ugvlufgbqlklwrbqxazsozfbuxaacs1cr8cvmgbcuvttvwm9s+rbk9srsglepljdsruymjvbjhz3dhfmjootipr6fng26dbziy/jkziiermtm/kznjmz4v7pc7yxn/g/+nb/4f+8sfw97s8lshx+r7/ftnwb</latexit> <latexit sha1_base64="g9biiiqvnrc2nlak0qbomus1fe8=">aaacy3icbvfdb9mwfhuypkr56syjl1d0ockkljmq2mkctekfjzsklk2qo8pxbjprjhpzdmojfesp8k/4nzhzjtgo++kjc67p/upkkywnwz+ev3fv/oohvuf9x0+epns+2d/4zopkc5zxqhb6mmegpva4s8jkvcw1sjyrejfcf2r0i++ojsju1k5ljho2vcitnflhlqa/pyp1fugpuinn06cjlowqufm0mz49deenuisrw4pi4fdbb4goyqouzsngjhnscqyjkjnzfbru0tfq3elg9z1zbesoqnzaa28ltev98da7t1glxrolwtamwjbglog6mcrdnc/2vqoafrzkuvkumthzkcxtxdntbzfopqomloxfsyxohvqsrxpx7ti38noxkwrumkxqflr23x81y41z54nlzjm9mrtaq/5pm1c2m8s1ugvlufgbqlklwrbqxazsozfbuxaacs1cr8cvmgbcuvttvwm9s+rbk9srsglepljdsruymjvbjhz3dhfmjootipr6fng26dbziy/jkziiermtm/kznjmz4v7pc7yxn/g/+nb/4f+8sfw97s8lshx+r7/ftnwb</latexit> <latexit sha1_base64="g9biiiqvnrc2nlak0qbomus1fe8=">aaacy3icbvfdb9mwfhuypkr56syjl1d0ockkljmq2mkctekfjzsklk2qo8pxbjprjhpzdmojfesp8k/4nzhzjtgo++kjc67p/upkkywnwz+ev3fv/oohvuf9x0+epns+2d/4zopkc5zxqhb6mmegpva4s8jkvcw1sjyrejfcf2r0i++ojsju1k5ljho2vcitnflhlqa/pyp1fugpuinn06cjlowqufm0mz49deenuisrw4pi4fdbb4goyqouzsngjhnscqyjkjnzfbru0tfq3elg9z1zbesoqnzaa28ltev98da7t1glxrolwtamwjbglog6mcrdnc/2vqoafrzkuvkumthzkcxtxdntbzfopqomloxfsyxohvqsrxpx7ti38noxkwrumkxqflr23x81y41z54nlzjm9mrtaq/5pm1c2m8s1ugvlufgbqlklwrbqxazsozfbuxaacs1cr8cvmgbcuvttvwm9s+rbk9srsglepljdsruymjvbjhz3dhfmjootipr6fng26dbziy/jkziiermtm/kznjmz4v7pc7yxn/g/+nb/4f+8sfw97s8lshx+r7/ftnwb</latexit> <latexit sha1_base64="g9biiiqvnrc2nlak0qbomus1fe8=">aaacy3icbvfdb9mwfhuypkr56syjl1d0ockkljmq2mkctekfjzsklk2qo8pxbjprjhpzdmojfesp8k/4nzhzjtgo++kjc67p/upkkywnwz+ev3fv/oohvuf9x0+epns+2d/4zopkc5zxqhb6mmegpva4s8jkvcw1sjyrejfcf2r0i++ojsju1k5ljho2vcitnflhlqa/pyp1fugpuinn06cjlowqufm0mz49deenuisrw4pi4fdbb4goyqouzsngjhnscqyjkjnzfbru0tfq3elg9z1zbesoqnzaa28ltev98da7t1glxrolwtamwjbglog6mcrdnc/2vqoafrzkuvkumthzkcxtxdntbzfopqomloxfsyxohvqsrxpx7ti38noxkwrumkxqflr23x81y41z54nlzjm9mrtaq/5pm1c2m8s1ugvlufgbqlklwrbqxazsozfbuxaacs1cr8cvmgbcuvttvwm9s+rbk9srsglepljdsruymjvbjhz3dhfmjootipr6fng26dbziy/jkziiermtm/kznjmz4v7pc7yxn/g/+nb/4f+8sfw97s8lshx+r7/ftnwb</latexit> A useful recurrence relation Def. T (n) = max number of compares to mergesort a list of length n. Recurrence. 0 n =1 T (n) T ( n/2 )+T( n/2 ) + n n>1 Solution. T (n) is O(n log 2 n). between n / 2 and n 1 compares Assorted proofs. We describe several ways to solve this recurrence. Initially we assume n is a power of 2 and replace with = in the recurrence. 9

10 <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> Divide-and-conquer recurrence: recursion tree Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n. T (n) = 0 n =1 2 T (n/2) + n n>1 assuming n is a power of 2 T (n) n = n T (n / 2) T (n / 2) 2 (n/2) = n log 2 n T (n / 4) T (n / 4) T (n / 4) T (n / 4) 4 (n/4) = n T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) 8 (n/8) = n T(n) = n log 2 n 10

11 <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> <latexit sha1_base64="z3k0jp42wp//6lnqi9e2pn3k2fw=">aaaco3icbvfna9taef0pbzq4h3hsyy5dnyaufkcyhaseleavpfsqfrsjwmksvinnywoldkfbrvap7s0/psthhdrpwc6pnznvzt8mpzkwguc35288ebr5bgu78/zfy1c73d29n7aojmcrkfrhrhnuuumni5kk8lo0ypne4vvy+6xjx92hsblqq5qxgod8qmumbsdhtbq/hkf6hurncn5cnsjbqds1cip20ynoajieihbgncgmdrqrcw9gave0dvonmmeuzgdrb3ayx4ng6l2t0etdn/92dsluass/6facframeazcfvryg5etxw8vsgtr5ahjkg7toaxkimtusaqfbt/kysnflz/i2ehnc7rxvtrpaw8dk0jwghc0wzl9t6pmubxzphgvoacbu55ryp/lxhvlp3etdvkravewkksuuagn45bkg4lu3aeujhs7grjhhgty/7iyzaldolh5st2rtbrfimusohkz3rgyrnv2giwg/u/98pvh3svpa+cw22dv2bel2qm7yf/zjrsxwe69tw/h6/qh/jf/hz98kpw9tuc1wwk//gpfucbd</latexit> Proof by induction Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n. T (n) = 0 n =1 2 T (n/2) + n n>1 assuming n is a power of 2 Pf. [ by induction on n ] Base case: when n = 1, T(1) = 0 = n log2 n. Inductive hypothesis: assume T(n) = n log 2 n. Goal: show that T(2n) = 2n log 2 (2n). recurrence T(2n) = 2 T(n) + 2n inductive hypothesis = 2 n log2 n + 2n = 2 n (log2 (2n) 1) + 2n = 2 n log2 (2n). 11

12 <latexit sha1_base64="vraymk/lbrqg3uogoqdaffrdfhc=">aaacxxicbvfda9swfjw9ry77srvhvvywbqsmzxyytcnsfpbsxw6strczimvxqagsgukucsbsj+yp7d9mdl1y0t2nwzn389y0lmk6kpotha8epnr8zo9p79nzfy9f9fcpflpdgy4zrqu2lymzkixcmrno4mvpkbwpxiv0+nujx9ygsukrqvuxmbrsququohoewvr/t4fqcl5cj6a4fkrmvpfd9ogkgvdaha5cdskhq+vz4kpyakxzahsmrejzpkmqc6m1afvpdns0+ajo5aodqcnyflltgngnqy/xbvdvd917ffxwrbhod6jr1abcb3ehbqsl88v+ceazzascleoswtupo9ilntnocin+rspiyfg1w+lcq8uktend2ribd57jipfn5fo5anl/k2pwwlsuup9zmhdld7wg/j82r1x+ktrclzvdxw8h5zuep6h5cwtcihdy7qhjrvhdgv8xw7jzn9ua0vyukw9duq8qjbjocievbuuma1ymdz27d2bj0zdr/opz4psks3opvcfvyzde5jickjnytmaeb2ewdojghj6fknthzw1qghq1r8lwhl/+aje6018=</latexit> <latexit sha1_base64="vraymk/lbrqg3uogoqdaffrdfhc=">aaacxxicbvfda9swfjw9ry77srvhvvywbqsmzxyytcnsfpbsxw6strczimvxqagsgukucsbsj+yp7d9mdl1y0t2nwzn389y0lmk6kpotha8epnr8zo9p79nzfy9f9fcpflpdgy4zrqu2lymzkixcmrno4mvpkbwpxiv0+nujx9ygsukrqvuxmbrsququohoewvr/t4fqcl5cj6a4fkrmvpfd9ogkgvdaha5cdskhq+vz4kpyakxzahsmrejzpkmqc6m1afvpdns0+ajo5aodqcnyflltgngnqy/xbvdvd917ffxwrbhod6jr1abcb3ehbqsl88v+ceazzascleoswtupo9ilntnocin+rspiyfg1w+lcq8uktend2ribd57jipfn5fo5anl/k2pwwlsuup9zmhdld7wg/j82r1x+ktrclzvdxw8h5zuep6h5cwtcihdy7qhjrvhdgv8xw7jzn9ua0vyukw9duq8qjbjocievbuuma1ymdz27d2bj0zdr/opz4psks3opvcfvyzde5jickjnytmaeb2ewdojghj6fknthzw1qghq1r8lwhl/+aje6018=</latexit> <latexit sha1_base64="vraymk/lbrqg3uogoqdaffrdfhc=">aaacxxicbvfda9swfjw9ry77srvhvvywbqsmzxyytcnsfpbsxw6strczimvxqagsgukucsbsj+yp7d9mdl1y0t2nwzn389y0lmk6kpotha8epnr8zo9p79nzfy9f9fcpflpdgy4zrqu2lymzkixcmrno4mvpkbwpxiv0+nujx9ygsukrqvuxmbrsququohoewvr/t4fqcl5cj6a4fkrmvpfd9ogkgvdaha5cdskhq+vz4kpyakxzahsmrejzpkmqc6m1afvpdns0+ajo5aodqcnyflltgngnqy/xbvdvd917ffxwrbhod6jr1abcb3ehbqsl88v+ceazzascleoswtupo9ilntnocin+rspiyfg1w+lcq8uktend2ribd57jipfn5fo5anl/k2pwwlsuup9zmhdld7wg/j82r1x+ktrclzvdxw8h5zuep6h5cwtcihdy7qhjrvhdgv8xw7jzn9ua0vyukw9duq8qjbjocievbuuma1ymdz27d2bj0zdr/opz4psks3opvcfvyzde5jickjnytmaeb2ewdojghj6fknthzw1qghq1r8lwhl/+aje6018=</latexit> <latexit sha1_base64="vraymk/lbrqg3uogoqdaffrdfhc=">aaacxxicbvfda9swfjw9ry77srvhvvywbqsmzxyytcnsfpbsxw6strczimvxqagsgukucsbsj+yp7d9mdl1y0t2nwzn389y0lmk6kpotha8epnr8zo9p79nzfy9f9fcpflpdgy4zrqu2lymzkixcmrno4mvpkbwpxiv0+nujx9ygsukrqvuxmbrsququohoewvr/t4fqcl5cj6a4fkrmvpfd9ogkgvdaha5cdskhq+vz4kpyakxzahsmrejzpkmqc6m1afvpdns0+ajo5aodqcnyflltgngnqy/xbvdvd917ffxwrbhod6jr1abcb3ehbqsl88v+ceazzascleoswtupo9ilntnocin+rspiyfg1w+lcq8uktend2ribd57jipfn5fo5anl/k2pwwlsuup9zmhdld7wg/j82r1x+ktrclzvdxw8h5zuep6h5cwtcihdy7qhjrvhdgv8xw7jzn9ua0vyukw9duq8qjbjocievbuuma1ymdz27d2bj0zdr/opz4psks3opvcfvyzde5jickjnytmaeb2ewdojghj6fknthzw1qghq1r8lwhl/+aje6018=</latexit> <latexit sha1_base64="oidllbbcsvzr990sj0ljhkzac4e=">aaacwxicbvdlsgmxfe3hv62vvpdugkvwucqmccoicg5cvrc20kldjr2tozlkso6izehv+dvu9r8ep8b0sbdvc0lozrk3n/feqrqwff+r4k2srq1vfddlw9s7u3vlyv6j1znh0oraatoomqupfdrroir2aoalsyrwplyd6k0xmfzo9ycjflojgyjrf5yho6kyh17ra+q20gzjla+vg/gtometrnfqchdshxoqndihdc3khpwrft2fbv0lgjmoknk0okphp+xpniwgketmbsfwu+zmzkdgesalmloqmj5ka+g4qfgctptprxvty8f0af8btxtskfu7imejtamkdpkjw2e7re3i/7rohv2lbi5umieopmvuzyrftsc+0z4wwfgohgdccpdxyp+zyrydmwtdpm+nwbcmyv8zjbjuwrir8rungzsxg2xp/olmaf2y7t+fvw8u5nywyse5iickiofkhtyrbmkstt7io/kgn4vvz/okxmmw6hxmnqdkibydh4oetjc=</latexit> Divide-and-conquer: quiz 1 Which is the exact solution of the following recurrence? T (n) = 0 n =1 T ( n/2 ) + T ( n/2 )+n 1 n>1 no longer assuming n is a power of 2 A. T(n) = n log2 n B. T(n) = n log2 n C. T(n) = n log2 n + 2 log 2 n 1 D. T(n) = n log2 n 2 log 2 n + 1 = n log 2 k E. Not even Knuth knows. k=1 12

13 <latexit sha1_base64="plaap4pyckuiob8guiy/wl4ixim=">aaacynicbvfdb9mwfhuyykn8deorlys6ubgijboiqxnoei+8ia2pzzpqqhkcm86ay0e2m7wk+syf4wfxb3cytkid98vh51yf+5wwulgxrx+ccofe/qe7ew97jx4/efqsv3/w0+rkcjxylbw5sjlfkrronxasl0qdreglnqdxxxv9/bqnfvpn3krepgaljxlbmfpuvp97mlrvgj4aldg8pzriqqiae0+77tgtcf4ddbh0nygcdhv8hvgq1kdpbdtgivepkygvudtaghp/bns0udf96w0bkaoqndbaw0ltw3+5te5rvfnxw7w/iezrg3axxb0ykc7o5vvbac00rwpujktm7syospfuzdjbjfqhkosl41dsgtmpfsvqjnw7yzw88kwgur8m18pby/77o2aftasi9zkfc5d2w2vi/2mzyuxhss1uwtlu/kzqxklwgprdqcymcidxhjbuho8v+cuzjdt/vo0qrxejfgoselkpwxwgw6x0s2dys8v4e2d3wfro9gku//gwod3u1rlhxpcxzehimian5bs5i1pcg93gxfaxgiffqxuuwvomnqy6p8/jros//gilc9vh</latexit> <latexit sha1_base64="plaap4pyckuiob8guiy/wl4ixim=">aaacynicbvfdb9mwfhuyykn8deorlys6ubgijboiqxnoei+8ia2pzzpqqhkcm86ay0e2m7wk+syf4wfxb3cytkid98vh51yf+5wwulgxrx+ccofe/qe7ew97jx4/efqsv3/w0+rkcjxylbw5sjlfkrronxasl0qdreglnqdxxxv9/bqnfvpn3krepgaljxlbmfpuvp97mlrvgj4aldg8pzriqqiae0+77tgtcf4ddbh0nygcdhv8hvgq1kdpbdtgivepkygvudtaghp/bns0udf96w0bkaoqndbaw0ltw3+5te5rvfnxw7w/iezrg3axxb0ykc7o5vvbac00rwpujktm7syospfuzdjbjfqhkosl41dsgtmpfsvqjnw7yzw88kwgur8m18pby/77o2aftasi9zkfc5d2w2vi/2mzyuxhss1uwtlu/kzqxklwgprdqcymcidxhjbuho8v+cuzjdt/vo0qrxejfgoselkpwxwgw6x0s2dys8v4e2d3wfro9gku//gwod3u1rlhxpcxzehimian5bs5i1pcg93gxfaxgiffqxuuwvomnqy6p8/jros//gilc9vh</latexit> <latexit sha1_base64="plaap4pyckuiob8guiy/wl4ixim=">aaacynicbvfdb9mwfhuyykn8deorlys6ubgijboiqxnoei+8ia2pzzpqqhkcm86ay0e2m7wk+syf4wfxb3cytkid98vh51yf+5wwulgxrx+ccofe/qe7ew97jx4/efqsv3/w0+rkcjxylbw5sjlfkrronxasl0qdreglnqdxxxv9/bqnfvpn3krepgaljxlbmfpuvp97mlrvgj4aldg8pzriqqiae0+77tgtcf4ddbh0nygcdhv8hvgq1kdpbdtgivepkygvudtaghp/bns0udf96w0bkaoqndbaw0ltw3+5te5rvfnxw7w/iezrg3axxb0ykc7o5vvbac00rwpujktm7syospfuzdjbjfqhkosl41dsgtmpfsvqjnw7yzw88kwgur8m18pby/77o2aftasi9zkfc5d2w2vi/2mzyuxhss1uwtlu/kzqxklwgprdqcymcidxhjbuho8v+cuzjdt/vo0qrxejfgoselkpwxwgw6x0s2dys8v4e2d3wfro9gku//gwod3u1rlhxpcxzehimian5bs5i1pcg93gxfaxgiffqxuuwvomnqy6p8/jros//gilc9vh</latexit> <latexit sha1_base64="plaap4pyckuiob8guiy/wl4ixim=">aaacynicbvfdb9mwfhuyykn8deorlys6ubgijboiqxnoei+8ia2pzzpqqhkcm86ay0e2m7wk+syf4wfxb3cytkid98vh51yf+5wwulgxrx+ccofe/qe7ew97jx4/efqsv3/w0+rkcjxylbw5sjlfkrronxasl0qdreglnqdxxxv9/bqnfvpn3krepgaljxlbmfpuvp97mlrvgj4aldg8pzriqqiae0+77tgtcf4ddbh0nygcdhv8hvgq1kdpbdtgivepkygvudtaghp/bns0udf96w0bkaoqndbaw0ltw3+5te5rvfnxw7w/iezrg3axxb0ykc7o5vvbac00rwpujktm7syospfuzdjbjfqhkosl41dsgtmpfsvqjnw7yzw88kwgur8m18pby/77o2aftasi9zkfc5d2w2vi/2mzyuxhss1uwtlu/kzqxklwgprdqcymcidxhjbuho8v+cuzjdt/vo0qrxejfgoselkpwxwgw6x0s2dys8v4e2d3wfro9gku//gwod3u1rlhxpcxzehimian5bs5i1pcg93gxfaxgiffqxuuwvomnqy6p8/jros//gilc9vh</latexit> Analysis of mergesort recurrence Proposition. If T(n) satisfies the following recurrence, then T(n) n log2 n. T (n) Pf. [ by strong induction on n ] Base case: n = 1. 0 n =1 T ( n/2 )+T ( n/2 ) + n n>1 Define n 1 = n / 2 and n 2 = n / 2 and note that n = n 1 + n 2. Induction step: assume true for 1, 2,..., n 1. no longer assuming n is a power of 2 inductive hypothesis T(n) T(n1) + T(n2) + n n1 log2 n1 + n2 log2 n2 + n n1 log2 n2 + n2 log2 n2 + n = n log2 n2 + n n ( log2 n 1) + n = n log2 n. n 2 = dn/2e d e l m apple 2 d dlog 2 e l ne / 2 m =2 dlog 2 ne / 2 d eappled e log 2 n 2 appledlog 2 ne 1 d eappled e an integer e

14 Digression: sorting lower bound Challenge. How to prove a lower bound for all conceivable algorithms? Model of computation. Comparison trees. Can access the elements only through pairwise comparisons. All other operations (control, data movement, etc.) are free. Cost model. Number of compares. Q. Realistic model? A1. Yes. Java, Python, C++, A2. Yes. Mergesort, insertion sort, quicksort, heapsort, A3. No. Bucket sort, radix sorts, Comparable[] a =...;... Arrays.sort(a); can access elements only via calls to compareto() 14

15 Comparison tree (for 3 distinct keys a, b, and c) yes a < b no code between compares (e.g., sequence of exchanges) height of pruned tree = worst-case number of compares b < c a < c yes no yes no a b c a < c b a c b < c yes no yes no a c b c a b b c a c b a each reachable leaf corresponds to one (and only one) ordering; exactly one reachable leaf for each possible ordering 15

16 Sorting lower bound Theorem. Any deterministic compare-based sorting algorithm must make Ω(n log n) compares in the worst-case. Pf. [ information theoretic ] Assume array consists of n distinct values a1 through an. Worst-case number of compares = height h of pruned comparison tree. Binary tree of height h has 2 h leaves. n! different orderings n! reachable leaves. h n! leaves 2 h leaves 16

17 Sorting lower bound Theorem. Any deterministic compare-based sorting algorithm must make Ω(n log n) compares in the worst-case. Pf. [ information theoretic ] Assume array consists of n distinct values a1 through an. Worst-case number of compares = height h of pruned comparison tree. Binary tree of height h has 2 h leaves. n! different orderings n! reachable leaves. 2 h # leaves n! h log2 (n!) n log2 n n / ln 2 Stirling s formula Note. Lower bound can be extended to include randomized algorithms. 17

18 SHUFFLING A LINKED LIST Problem. Given a singly linked list, rearrange its nodes uniformly at random. Assumption. Access to a perfect random-number generator. Performance. O(n log n) time, O(log n) extra space. all n! permutations equally likely input L null shuffled L null 18

19 5. DIVIDE AND CONQUER mergesort counting inversions randomized quicksort median and selection closest pair of points SECTION 5.3

20 Counting inversions Music site tries to match your song preferences with others. You rank n songs. Music site consults database to find people with similar tastes. Similarity metric: number of inversions between two rankings. My rank: 1, 2,, n. Your rank: a 1, a 2,, a n. Songs i and j are inverted if i < j, but a i > a j. A B C D E me you inversions: 3-2, 4-2 Brute force: check all Θ(n 2 ) pairs. 20

21 Counting inversions: applications Voting theory. Collaborative filtering. Measuring the sortedness of an array. Sensitivity analysis of Google s ranking function. Rank aggregation for meta-searching on the Web. Nonparametric statistics (e.g., Kendall s tau distance). Rank Aggregation Methods for the Web Cynthia Dwork Ravi Kumar Moni Naor Rank Aggregation Methods for the Web Cynthia Dwork Ravi Kumar Moni Naor D. Sivakumar D. Sivakumar ABSTRACT ABSTRACT 1.1 Motivation 1. INTRODUCTION 1.1 Motivation 21

22 Counting inversions: divide-and-conquer Divide: separate list into two halves A and B. Conquer: recursively count inversions in each list. Combine: count inversions (a, b) with a A and b B. Return sum of three counts. input count inversions in left half A count inversions in right half B count inversions (a, b) with a A and b B output = 17 22

23 Counting inversions: how to combine two subproblems? Q. How to count inversions (a, b) with a A and b B? A. Easy if A and B are sorted! Warmup algorithm. Sort A and B. For each element b B, - binary search in A to find how elements in A are greater than b. list A list B sort A sort B binary search to count inversions (a, b) with a A and b B

24 Counting inversions: how to combine two subproblems? Count inversions (a, b) with a A and b B, assuming A and B are sorted. Scan A and B from left to right. Compare ai and bj. If ai < bj, then ai is not inverted with any element left in B. If ai > bj, then bj is inverted with every element left in A. Append smaller element to sorted list C. count inversions (a, b) with a A and b B ai bj merge to form sorted list C

25 Counting inversions: divide-and-conquer algorithm implementation Input. List L. Output. Number of inversions in L and L in sorted order. SORT-AND-COUNT(L) IF (list L has one element) RETURN (0, L). Divide the list into two halves A and B. (ra, A) SORT-AND-COUNT(A). (rb, B) SORT-AND-COUNT(B). (rab, L) MERGE-AND-COUNT(A, B). T(n / 2) T(n / 2) Θ(n) RETURN (ra + rb + rab, L). 25

26 <latexit sha1_base64="3pxfe56q7vvcgldq8zgmwt26i7u=">aaac2hicbvhlattafb0pfatuy0npqpuhtotdwzfciqmmjdawukzbbgiayubjk3viacrmroqn8kkr0m1/pn/tv+liuac2exfd4zw799xhuihpmqj+ep7onbv37u8+6dx89pjj0+7e/lebl0bawoqqn1cjt6ckhjfkvhbvgobzouayuf5q65ffwfiz6xeuc4gzptmylykjoybd36o+pqrssn/vt4clmjo6eq6ixxxyki3mglwfhtlxlcesskiypqfapychdeuziwynkmuuddrnklv5bqg+oqbmnlgu/cyvrkuburvadw+l1kjvwby/tegw0no2p0m3fwycjug2cfvqi21ctpa8ftbnrzmbrqg4tveyfbhx3kauctyqpywci2s+g8hbztowcdvsdkvfowzkuzdummukdfvvj4pn1i6zxgvmhod2u6vj/2lrielpxeldlaha3bilpaky0/pmdconcfrlb7gw0vvkxzwbltadc82lqv2awjukwprainwkg6zcbrpebzhc3nk2gb8pzgbhl7e989n2nbvkbxlj+iqkj+scfcyxzeye99wbeh+9t37kf/d/+d9vun2v/fomrix/6y+lidpe</latexit> <latexit sha1_base64="3pxfe56q7vvcgldq8zgmwt26i7u=">aaac2hicbvhlattafb0pfatuy0npqpuhtotdwzfciqmmjdawukzbbgiayubjk3viacrmroqn8kkr0m1/pn/tv+liuac2exfd4zw799xhuihpmqj+ep7onbv37u8+6dx89pjj0+7e/lebl0bawoqqn1cjt6ckhjfkvhbvgobzouayuf5q65ffwfiz6xeuc4gzptmylykjoybd36o+pqrssn/vt4clmjo6eq6ixxxyki3mglwfhtlxlcesskiypqfapychdeuziwynkmuuddrnklv5bqg+oqbmnlgu/cyvrkuburvadw+l1kjvwby/tegw0no2p0m3fwycjug2cfvqi21ctpa8ftbnrzmbrqg4tveyfbhx3kauctyqpywci2s+g8hbztowcdvsdkvfowzkuzdummukdfvvj4pn1i6zxgvmhod2u6vj/2lrielpxeldlaha3bilpaky0/pmdconcfrlb7gw0vvkxzwbltadc82lqv2awjukwprainwkg6zcbrpebzhc3nk2gb8pzgbhl7e989n2nbvkbxlj+iqkj+scfcyxzeye99wbeh+9t37kf/d/+d9vun2v/fomrix/6y+lidpe</latexit> <latexit sha1_base64="3pxfe56q7vvcgldq8zgmwt26i7u=">aaac2hicbvhlattafb0pfatuy0npqpuhtotdwzfciqmmjdawukzbbgiayubjk3viacrmroqn8kkr0m1/pn/tv+liuac2exfd4zw799xhuihpmqj+ep7onbv37u8+6dx89pjj0+7e/lebl0bawoqqn1cjt6ckhjfkvhbvgobzouayuf5q65ffwfiz6xeuc4gzptmylykjoybd36o+pqrssn/vt4clmjo6eq6ixxxyki3mglwfhtlxlcesskiypqfapychdeuziwynkmuuddrnklv5bqg+oqbmnlgu/cyvrkuburvadw+l1kjvwby/tegw0no2p0m3fwycjug2cfvqi21ctpa8ftbnrzmbrqg4tveyfbhx3kauctyqpywci2s+g8hbztowcdvsdkvfowzkuzdummukdfvvj4pn1i6zxgvmhod2u6vj/2lrielpxeldlaha3bilpaky0/pmdconcfrlb7gw0vvkxzwbltadc82lqv2awjukwprainwkg6zcbrpebzhc3nk2gb8pzgbhl7e989n2nbvkbxlj+iqkj+scfcyxzeye99wbeh+9t37kf/d/+d9vun2v/fomrix/6y+lidpe</latexit> <latexit sha1_base64="3pxfe56q7vvcgldq8zgmwt26i7u=">aaac2hicbvhlattafb0pfatuy0npqpuhtotdwzfciqmmjdawukzbbgiayubjk3viacrmroqn8kkr0m1/pn/tv+liuac2exfd4zw799xhuihpmqj+ep7onbv37u8+6dx89pjj0+7e/lebl0bawoqqn1cjt6ckhjfkvhbvgobzouayuf5q65ffwfiz6xeuc4gzptmylykjoybd36o+pqrssn/vt4clmjo6eq6ixxxyki3mglwfhtlxlcesskiypqfapychdeuziwynkmuuddrnklv5bqg+oqbmnlgu/cyvrkuburvadw+l1kjvwby/tegw0no2p0m3fwycjug2cfvqi21ctpa8ftbnrzmbrqg4tveyfbhx3kauctyqpywci2s+g8hbztowcdvsdkvfowzkuzdummukdfvvj4pn1i6zxgvmhod2u6vj/2lrielpxeldlaha3bilpaky0/pmdconcfrlb7gw0vvkxzwbltadc82lqv2awjukwprainwkg6zcbrpebzhc3nk2gb8pzgbhl7e989n2nbvkbxlj+iqkj+scfcyxzeye99wbeh+9t37kf/d/+d9vun2v/fomrix/6y+lidpe</latexit> Counting inversions: divide-and-conquer algorithm analysis Proposition. The sort-and-count algorithm counts the number of inversions in a permutation of size n in O(n log n) time. Pf. The worst-case running time T(n) satisfies the recurrence: T (n) = (1) n =1 T ( n/2 ) + T ( n/2 ) + (n) n>1 26

27 5. DIVIDE AND CONQUER mergesort counting inversions randomized quicksort median and selection closest pair of points SECTION

28 3-WAY PARTITIONING Goal. Given an array A and pivot element p, partition array so that: Smaller elements in left subarray L. Equal elements in middle subarray M. Larger elements in right subarray R. Challenge. O(n) time and O(1) space. the array A p the partitioned array A L M R 28

29 Randomized quicksort Pick a random pivot element p A. 3-way partition the array into L, M, and R. Recursively sort both L and R. the array A p partition A sort L sort R the sorted array A

30 Randomized quicksort Pick a random pivot element p A. 3-way partition the array into L, M, and R. Recursively sort both L and R. RANDOMIZED-QUICKSORT(A) IF (array A has zero or one element) RETURN. Pick pivot p A uniformly at random. (L, M, R) PARTITION-3-WAY(A, p). RANDOMIZED-QUICKSORT(L). RANDOMIZED-QUICKSORT(R). T(i) Θ(n) T(n i 1) new analysis required (i is a random variable depends on p) 30

31 Analysis of randomized quicksort Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < < an is O(n log n). Pf. Consider BST representation of pivot elements. the original array of elements A a 7 a 6 a 12 a 3 a 11 a 8 a 9 a 1 a 4 a 10 a 2 a 13 a 5 first pivot in left subarray first pivot (chosen uniformly at random) a9 a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 31

32 Analysis of randomized quicksort Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < < an is O(n log n). Pf. Consider BST representation of pivot elements. ai and aj are compared once iff one is an ancestor of the other. first pivot in left subarray first pivot (chosen uniformly at random) a9 a3 and a6 are compared (when a3 is pivot) a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 32

33 Analysis of randomized quicksort Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < < an is O(n log n). Pf. Consider BST representation of pivot elements. ai and aj are compared once iff one is an ancestor of the other. first pivot in left subarray first pivot (chosen uniformly at random) a9 a2 and a8 are not compared (because a3 partitions them) a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 33

34 Divide-and-conquer: quiz 2 Given an array of n 8 distinct elements a1 < a2 < < an, what is the probability that a7 and a8 are compared during randomized quicksort? A. 0 B. 1 / n C. 2 / n D. 1 if two elements are adjacent, one must be an ancestor of the other a9 a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 34

35 Divide-and-conquer: quiz 3 Given an array of n 2 distinct elements a1 < a2 < < an, what is the probability that a1 and an are compared during randomized quicksort? A. 0 B. 1 / n C. 2 / n a1 and an are compared iff first pivot is either a1 or an D. 1 a9 a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 35

36 Analysis of randomized quicksort Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < < an is O(n log n). Pf. Consider BST representation of pivot elements. ai and aj are compared once iff one is an ancestor of the other. Pr [ ai and aj are compared ] = 2 / ( j - i + 1), where i < j. Pr[a2 and a8 compared] = 2/7 compared iff either a2 or a8 is chosen as pivot before any of { a3, a4, a5, a6, a7} a9 a3 a11 a1 a8 a10 a13 a2 a4 a12 a6 a5 a7 36

37 Analysis of randomized quicksort Proposition. The expected number of compares to quicksort an array of n distinct elements a1 < a2 < < an is O(n log n). Pf. Consider BST representation of pivot elements. ai and aj are compared once iff one is an ancestor of the other. Pr [ ai and aj are compared ] = 2 / ( j - i + 1), where i < j. Expected number of compares = Remark. Number of compares only decreases if equal elements. n i=1 all pairs i and j n j=i+1 2 j i +1 = 2 2n n i=1 harmonic sum n j=1 n n i+1 j=2 1 j 1 j 2n (ln n + 1) 37

38 Tony Hoare Invented quicksort to translate Russian into English. [ but couldn t explain his algorithm or implement it! ] Learned Algol 60 (and recursion). Implemented quicksort. Tony Hoare 1980 Turing Award 4 ALGORITHM 64 QUICKSORT C. A. R. HOARE Elliott Brothers Ltd., Borehamwood, Hertfordshire, Eng. procedure quicksort (A,M,N); value M,N; array A; integer M,N; comment Quicksort is a very fast and convenient method of sorting an array in the random-access store of a computer. The entire contents of the store may be sorted, since no extra space is required. The average number of comparisons made is 2(M--N) In (N--M), and the average nmnber of exchanges is one sixth this amount. Suitable refinements of this method will be desirable for its implementation on any actual computer; begin integer 1,J ; if M < N then begin partition (A,M,N,I,J); quicksort (A,M,J) ; quicksort (A, I, N) end end quieksort p Communications of the ACM (July 1961) 38

39 NUTS AND BOLTS Problem. A disorganized carpenter has a mixed pile of n nuts and n bolts. The goal is to find the corresponding pairs of nuts and bolts. Each nut fits exactly one bolt and each bolt fits exactly one nut. By fitting a nut and a bolt together, the carpenter can see which one is bigger (but cannot directly compare either two nuts or two bolts). Brute-force solution. Compare each bolt to each nut Θ (n 2) compares. Challenge. Design an algorithm that makes O(n log n) compares. 39

40 5. DIVIDE AND CONQUER mergesort counting inversions randomized quicksort median and selection closest pair of points SECTION 9.3

41 Median and selection problems Selection. Given n elements from a totally ordered universe, find k th smallest. Minimum: k = 1; maximum: k = n. Median: k = (n + 1) / 2. O(n) compares for min or max. O(n log n) compares by sorting. O(n log k) compares with a binary heap. max heap with k smallest Applications. Order statistics; find the top k ; bottleneck paths, Q. Can we do it with O(n) compares? A. Yes! Selection is easier than sorting. 43

42 Randomized quickselect Pick a random pivot element p A. 3-way partition the array into L, M, and R. Recur in one subarray the one containing the k th smallest element. QUICK-SELECT(A, k) Pick pivot p A uniformly at random. (L, M, R) PARTITION-3-WAY(A, p). Θ(n) IF (k L ) RETURN QUICK-SELECT(L, k). T(i) ELSE IF (k > L + M ) RETURN QUICK-SELECT(R, k L M ) T(n i 1) ELSE IF (k = L ) RETURN p. 44

43 Randomized quickselect analysis Intuition. Split candy bar uniformly expected size of larger piece is ¾. T(n) T(3 n / 4) + n T(n) 4 n not rigorous: can t assume E[T(i)] T(E[i]) Def. T(n, k) = expected # compares to select k th smallest in array of length n. Def. T(n) = maxk T(n, k). Proposition. T(n) 4 n. Pf. [ by strong induction on n ] Assume true for 1, 2,, n 1. T(n) satisfies the following recurrence: can assume we always recur of larger of two subarrays since T(n) is monotone non-decreasing T(n) n + 1 / n [ 2T(n / 2) + + 2T(n 3) + 2T(n 2) + 2T(n 1) ] n + 1 / n [ 8(n / 2) + + 8(n 3) + 8(n 2) + 8(n 1) ] n + 1 / n (3n 2 ) inductive hypothesis = 4 n. tiny cheat: sum should start at T( n/2 ) 45

44 Selection in worst-case linear time Goal. Find pivot element p that divides list of n elements into two pieces so that each piece is guaranteed to have 7/10 n elements. Q. How to find approximate median in linear time? A. Recursively compute median of sample of 2/10 n elements. T(n) = Θ(1) if n = 1 T (7/10 n) + T (2/10 n) + Θ(n) otherwise two subproblems of different sizes! T(n) = Θ(n) we ll need to show this 46

45 Choosing the pivot element Divide n elements into n / 5 groups of 5 elements each (plus extra) n = 54 47

46 Choosing the pivot element Divide n elements into n / 5 groups of 5 elements each (plus extra). Find median of each group (except extra). medians n = 54 48

47 Choosing the pivot element Divide n elements into n / 5 groups of 5 elements each (plus extra). Find median of each group (except extra). Find median of n / 5 medians recursively. Use median-of-medians as pivot element. medians median of medians n = 54 49

48 Median-of-medians selection algorithm MOM-SELECT(A, k) n A. IF (n < 50) RETURN k th smallest of element of A via mergesort. Group A into n / 5 groups of 5 elements each (ignore leftovers). B median of each group of 5. p MOM-SELECT(B, n / 10 ) (L, M, R) PARTITION-3-WAY(A, p). IF (k L ) RETURN MOM-SELECT(L, k). ELSE IF (k > L + M ) RETURN MOM-SELECT(R, k L M ) ELSE RETURN p. median of medians 50

49 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. medians median of medians p n = 54 51

50 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. At least n / 5 / 2 = n / 10 medians p. medians p median of medians p n = 54 52

51 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. At least n / 5 / 2 = n / 10 medians p. At least 3 n / 10 elements p. medians p median of medians p n = 54 53

52 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. medians median of medians p n = 54 54

53 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. At least n / 5 / 2 = n / 10 medians p. medians p median of medians p n = 54 55

54 Analysis of median-of-medians selection algorithm At least half of 5-element medians p. At least n / 5 / 2 = n / 10 medians p. At least 3 n / 10 elements p. medians p median of medians p n = 54 56

55 Median-of-medians selection algorithm recurrence Median-of-medians selection algorithm recurrence. Select called recursively with n / 5 elements to compute MOM p. At least 3 n / 10 elements p. At least 3 n / 10 elements p. Select called recursively with at most n 3 n / 10 elements. Def. C(n) = max # compares on any array of n elements. C(n) C ( n/5 ) + C (n 3 n/10 ) n median of medians recursive select computing median of 5 ( 6 compares per group) partitioning ( n compares) Intuition. C(n) is going to be at least linear in n C(n) is super-additive. Ignoring floors, this implies that C(n) C(n / 5 + n 3n / 10) + 11/5 n = C(9n / 10) + 11/5 n C(n) 22n. 57

56 Median-of-medians selection algorithm recurrence Median-of-medians selection algorithm recurrence. Select called recursively with n / 5 elements to compute MOM p. At least 3 n / 10 elements p. At least 3 n / 10 elements p. Select called recursively with at most n 3 n / 10 elements. Def. C(n) = max # compares on any array of n elements. C(n) C ( n/5 ) + C (n 3 n/10 ) n median of medians recursive select computing median of 5 ( 6 compares per group) partitioning ( n compares) Now, let s solve given recurrence. Assume n is both a power of 5 and a power of 10? Prove that C(n) is monotone non-decreasing. 58

57 <latexit sha1_base64="y96hacevn4aord6hbv5tphcoy0y=">aaac0nicbvhlbtnafb2bvwmppmxj5ooilaorbccikkjuyozlktctllgi8fg6hxu8tmbgkjgvbwllj/bj/a1j1xjnwl0dnxof5yalfmygwr/pv3x7zt17e/d7dx4+erzfpzj8zopkc4x4iqt9ktcduiimrlasl0qnle8knidx00y//47aiej9tass45wtlmgez9zr8/7v6vadaz18pjmetxahvm1do7pu0ukal4baxfqoqwswbgvuiora6szephyp0ygvmswkder1gkhu8rg8bncwxsfb+cehwu2dzprxogzx9bjpuzpopb3to6jsbqf5fxcmgjzgf4qdgjauzuyh3ifnc17lqcyxzjhzgjq2rpm2gkt0f1ygs8av2ajndiqwo4nr1tm1phdmcplbpsuuhza9wvgz3jhvnrjmnnlls6015p+0wwwz47gwqqwskn49kksk2akab0eqnhirvw4wroxbffglc25z98ankw3vevngjfwyuoixkw6x0i6tzo2l4bznuyb6m/owcr+8g5wed3bukafkgrmsklwnp+qzosmr4d6+n/zove9+5nf+d//ndarvdtvpyeb4v/4ck0dzrw==</latexit> <latexit sha1_base64="y96hacevn4aord6hbv5tphcoy0y=">aaac0nicbvhlbtnafb2bvwmppmxj5ooilaorbccikkjuyozlktctllgi8fg6hxu8tmbgkjgvbwllj/bj/a1j1xjnwl0dnxof5yalfmygwr/pv3x7zt17e/d7dx4+erzfpzj8zopkc4x4iqt9ktcduiimrlasl0qnle8knidx00y//47aiej9tass45wtlmgez9zr8/7v6vadaz18pjmetxahvm1do7pu0ukal4baxfqoqwswbgvuiora6szephyp0ygvmswkder1gkhu8rg8bncwxsfb+cehwu2dzprxogzx9bjpuzpopb3to6jsbqf5fxcmgjzgf4qdgjauzuyh3ifnc17lqcyxzjhzgjq2rpm2gkt0f1ygs8av2ajndiqwo4nr1tm1phdmcplbpsuuhza9wvgz3jhvnrjmnnlls6015p+0wwwz47gwqqwskn49kksk2akab0eqnhirvw4wroxbffglc25z98ankw3vevngjfwyuoixkw6x0i6tzo2l4bznuyb6m/owcr+8g5wed3bukafkgrmsklwnp+qzosmr4d6+n/zove9+5nf+d//ndarvdtvpyeb4v/4ck0dzrw==</latexit> <latexit sha1_base64="y96hacevn4aord6hbv5tphcoy0y=">aaac0nicbvhlbtnafb2bvwmppmxj5ooilaorbccikkjuyozlktctllgi8fg6hxu8tmbgkjgvbwllj/bj/a1j1xjnwl0dnxof5yalfmygwr/pv3x7zt17e/d7dx4+erzfpzj8zopkc4x4iqt9ktcduiimrlasl0qnle8knidx00y//47aiej9tass45wtlmgez9zr8/7v6vadaz18pjmetxahvm1do7pu0ukal4baxfqoqwswbgvuiora6szephyp0ygvmswkder1gkhu8rg8bncwxsfb+cehwu2dzprxogzx9bjpuzpopb3to6jsbqf5fxcmgjzgf4qdgjauzuyh3ifnc17lqcyxzjhzgjq2rpm2gkt0f1ygs8av2ajndiqwo4nr1tm1phdmcplbpsuuhza9wvgz3jhvnrjmnnlls6015p+0wwwz47gwqqwskn49kksk2akab0eqnhirvw4wroxbffglc25z98ankw3vevngjfwyuoixkw6x0i6tzo2l4bznuyb6m/owcr+8g5wed3bukafkgrmsklwnp+qzosmr4d6+n/zove9+5nf+d//ndarvdtvpyeb4v/4ck0dzrw==</latexit> <latexit sha1_base64="y96hacevn4aord6hbv5tphcoy0y=">aaac0nicbvhlbtnafb2bvwmppmxj5ooilaorbccikkjuyozlktctllgi8fg6hxu8tmbgkjgvbwllj/bj/a1j1xjnwl0dnxof5yalfmygwr/pv3x7zt17e/d7dx4+erzfpzj8zopkc4x4iqt9ktcduiimrlasl0qnle8knidx00y//47aiej9tass45wtlmgez9zr8/7v6vadaz18pjmetxahvm1do7pu0ukal4baxfqoqwswbgvuiora6szephyp0ygvmswkder1gkhu8rg8bncwxsfb+cehwu2dzprxogzx9bjpuzpopb3to6jsbqf5fxcmgjzgf4qdgjauzuyh3ifnc17lqcyxzjhzgjq2rpm2gkt0f1ygs8av2ajndiqwo4nr1tm1phdmcplbpsuuhza9wvgz3jhvnrjmnnlls6015p+0wwwz47gwqqwskn49kksk2akab0eqnhirvw4wroxbffglc25z98ankw3vevngjfwyuoixkw6x0i6tzo2l4bznuyb6m/owcr+8g5wed3bukafkgrmsklwnp+qzosmr4d6+n/zove9+5nf+d//ndarvdtvpyeb4v/4ck0dzrw==</latexit> Divide-and-conquer: quiz 4 Consider the following recurrence C(n) = Is C(n) monotone non-decreasing? 0 n 1 C( n/5 )+C(n 3 n/10 ) n n>1 A. Yes, obviously. B. Yes, but proof is tedious. C. Yes, but proof is hard. D. No. C(19) = 165 C(20) =

58 <latexit sha1_base64="ucntptmzk5k2uvff1msqyvcmsiy=">aaac9hicbvfnj9mwehxc11k+usury4gk1aq2xebhutmsxixjirv0pboqjjvpwus4ke1aq6g/hrpiyh/hf/bvcnjkbfvm9pteeobnc5wrav0u/qnca9dv3lx1cltx5+69+w+ah0efbvyyguorqcycx9yikhqhtjqf57lbnsykr/hlh0offuvjzayhbpnjjovzlrmpupputpl70nydyh2mkpublma51kxwa+2qwfpvndwf5ndhoaszwao0videbiynkayt3wis5qtgpz8bftyx7tz3cnbmklfzzkc/6aezne7as6ofjuev/jnpdfvniegiphrv9vyyyhmw1b4jnse1jqzdpdsynjzbuteqc/yb3yaw2dtz9da4yrnmfclqjxs3dkyj3e1kbpwucn0chcwci0s+x7ghmqdoj2wd+gqeegygit8iybsdmr36ousptcs09p0pdxd2v6vi/2njwiunk1lqvhcoxxpruihwgvrfcdnpudi19iali71xebfc5+b8r29tqwfnklyukreflikb4q6r3mizxqvidzpbb8ox3xdd+ul16/rke+cbeuqekzah5c05jr/jgrksedbgfhwjepgt/b7+ch+uw8ng8+yh2arw118pf+rq</latexit> <latexit sha1_base64="ucntptmzk5k2uvff1msqyvcmsiy=">aaac9hicbvfnj9mwehxc11k+usury4gk1aq2xebhutmsxixjirv0pboqjjvpwus4ke1aq6g/hrpiyh/hf/bvcnjkbfvm9pteeobnc5wrav0u/qnca9dv3lx1cltx5+69+w+ah0efbvyyguorqcycx9yikhqhtjqf57lbnsykr/hlh0offuvjzayhbpnjjovzlrmpupputpl70nydyh2mkpublma51kxwa+2qwfpvndwf5ndhoaszwao0videbiynkayt3wis5qtgpz8bftyx7tz3cnbmklfzzkc/6aezne7as6ofjuev/jnpdfvniegiphrv9vyyyhmw1b4jnse1jqzdpdsynjzbuteqc/yb3yaw2dtz9da4yrnmfclqjxs3dkyj3e1kbpwucn0chcwci0s+x7ghmqdoj2wd+gqeegygit8iybsdmr36ousptcs09p0pdxd2v6vi/2njwiunk1lqvhcoxxpruihwgvrfcdnpudi19iali71xebfc5+b8r29tqwfnklyukreflikb4q6r3mizxqvidzpbb8ox3xdd+ul16/rke+cbeuqekzah5c05jr/jgrksedbgfhwjepgt/b7+ch+uw8ng8+yh2arw118pf+rq</latexit> <latexit sha1_base64="ucntptmzk5k2uvff1msqyvcmsiy=">aaac9hicbvfnj9mwehxc11k+usury4gk1aq2xebhutmsxixjirv0pboqjjvpwus4ke1aq6g/hrpiyh/hf/bvcnjkbfvm9pteeobnc5wrav0u/qnca9dv3lx1cltx5+69+w+ah0efbvyyguorqcycx9yikhqhtjqf57lbnsykr/hlh0offuvjzayhbpnjjovzlrmpupputpl70nydyh2mkpublma51kxwa+2qwfpvndwf5ndhoaszwao0videbiynkayt3wis5qtgpz8bftyx7tz3cnbmklfzzkc/6aezne7as6ofjuev/jnpdfvniegiphrv9vyyyhmw1b4jnse1jqzdpdsynjzbuteqc/yb3yaw2dtz9da4yrnmfclqjxs3dkyj3e1kbpwucn0chcwci0s+x7ghmqdoj2wd+gqeegygit8iybsdmr36ousptcs09p0pdxd2v6vi/2njwiunk1lqvhcoxxpruihwgvrfcdnpudi19iali71xebfc5+b8r29tqwfnklyukreflikb4q6r3mizxqvidzpbb8ox3xdd+ul16/rke+cbeuqekzah5c05jr/jgrksedbgfhwjepgt/b7+ch+uw8ng8+yh2arw118pf+rq</latexit> <latexit sha1_base64="ucntptmzk5k2uvff1msqyvcmsiy=">aaac9hicbvfnj9mwehxc11k+usury4gk1aq2xebhutmsxixjirv0pboqjjvpwus4ke1aq6g/hrpiyh/hf/bvcnjkbfvm9pteeobnc5wrav0u/qnca9dv3lx1cltx5+69+w+ah0efbvyyguorqcycx9yikhqhtjqf57lbnsykr/hlh0offuvjzayhbpnjjovzlrmpupputpl70nydyh2mkpublma51kxwa+2qwfpvndwf5ndhoaszwao0videbiynkayt3wis5qtgpz8bftyx7tz3cnbmklfzzkc/6aezne7as6ofjuev/jnpdfvniegiphrv9vyyyhmw1b4jnse1jqzdpdsynjzbuteqc/yb3yaw2dtz9da4yrnmfclqjxs3dkyj3e1kbpwucn0chcwci0s+x7ghmqdoj2wd+gqeegygit8iybsdmr36ousptcs09p0pdxd2v6vi/2njwiunk1lqvhcoxxpruihwgvrfcdnpudi19iali71xebfc5+b8r29tqwfnklyukreflikb4q6r3mizxqvidzpbb8ox3xdd+ul16/rke+cbeuqekzah5c05jr/jgrksedbgfhwjepgt/b7+ch+uw8ng8+yh2arw118pf+rq</latexit> Median-of-medians selection algorithm recurrence Analysis of selection algorithm recurrence. T(n) = max # compares on any array of n elements. T(n) is monotone non-decreasing, but C(n) is not! T (n) 6n n<50 max{ T (n 1), T( n/5 )+T (n 3 n/10 ) n) } n 50 Claim. T(n) 44 n. Pf. [ by strong induction ] Base case: T(n) 6 n for n < 50 (mergesort). Inductive hypothesis: assume true for 1, 2,, n 1. Induction step: for n 50, we have either T(n) T(n 1) 44 n or T(n) T( n / 5 ) + T(n 3 n / 10 ) + 11/5 n inductive hypothesis 44 ( n / 5 ) + 44 (n 3 n / 10 ) + 11/5 n 44 (n / 5) + 44 n 44 (n / 4) + 11/5 n for n 50, 3 n / 10 n / 4 = 44 n. 60

59 Divide-and-conquer: quiz 5 Suppose that we divide n elements into n / r groups of r elements each, and use the median-of-medians of these n / r groups as the pivot. For which r is the worst-case running time of select O(n)? A. r = 3 B. r = 7 T(n) = T(n / 3) + T(n 2n / 6) + Θ(n) T(n) = Θ(n log n) T(n) = T(n / 7) + T(n 4n / 14) + Θ(n) T(n) = Θ(n) C. Both A and B. D. Neither A nor B. 61

60 Linear-time selection retrospective Proposition. [Blum Floyd Pratt Rivest Tarjan 1973] There exists a compare-based selection algorithm whose worst-case running time is O(n). Time Bounds for Selection* MANUEL BLUM, ROBERT W. FLOYD, VAUGHAN PRATT, RONALD L. RIVEST, AND ROBERT E. TARJAN Department of Computer Science, Stanford University, Stanford, California Received November 14, 1972 The number of comparisons required to select the i-th smallest of n numbers is shown to be at most a linear function of n by analysis of a new selection algorithm--pick. Specifically, no more than n comparisons are ever required. This bound is improved for extreme values of i, and a new lower bound on the requisite number of comparisons is also proved. Theory. Optimized version of BFPRT: n compares. Upper bound: [Dor Zwick 1995] 2.95 n compares. Lower bound: [Dor Zwick 1999] ( ) n compares. Practice. Constants too large to be useful. 62

61 5. DIVIDE AND CONQUER mergesort counting inversions randomized quicksort median and selection closest pair of points SECTION 5.4

62 Closest pair of points Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer vision, geographic information systems, molecular modeling, air traffic control. Special case of nearest neighbor, Euclidean MST, Voronoi. fast closest pair inspired fast algorithms for these problems 64

63 Closest pair of points Closest pair problem. Given n points in the plane, find a pair of points with the smallest Euclidean distance between them. Brute force. Check all pairs with Θ(n 2 ) distance calculations. 1D version. Easy O(n log n) algorithm if points are on a line. Non-degeneracy assumption. No two points have the same x-coordinate. 65

64 Closest pair of points: first attempt Sorting solution. Sort by x-coordinate and consider nearby points. Sort by y-coordinate and consider nearby points. 66

65 Closest pair of points: first attempt Sorting solution. Sort by x-coordinate and consider nearby points. Sort by y-coordinate and consider nearby points. 8 67

66 Closest pair of points: second attempt Divide. Subdivide region into 4 quadrants. 68

67 Closest pair of points: second attempt Divide. Subdivide region into 4 quadrants. Obstacle. Impossible to ensure n / 4 points in each piece. 69

68 Closest pair of points: divide-and-conquer algorithm Divide: draw vertical line L so that n / 2 points on each side. Conquer: find closest pair in each side recursively. Combine: find closest pair with one point in each side. Return best of 3 solutions. seems like Θ(n 2 ) L

69 How to find closest pair with one point in each side? Find closest pair with one point in each side, assuming that distance < δ. Observation: suffices to consider only those points within δ of line L. L δ = min(12, 21) δ 71

70 How to find closest pair with one point in each side? Find closest pair with one point in each side, assuming that distance < δ. Observation: suffices to consider only those points within δ of line L. Sort points in 2 δ-strip by their y-coordinate. Check distances of only those points within 7 positions in sorted list! why? L δ = min(12, 21) 2 1 δ 72

71 How to find closest pair with one point in each side? Def. Let s i be the point in the 2 δ-strip, with the i th smallest y-coordinate. Claim. If j i > 7, then the distance between s i and s j is at least δ. Pf. Consider the 2δ-by-δ rectangle R in strip whose min y-coordinate is y-coordinate of s i. Distance between s i and any point s j above R is δ. Subdivide R into 8 squares. At most 1 point per square. diameter is / 2 < At most 7 other points can be in R. δ R si L sj ½δ ½δ constant can be improved with more refined geometric packing argument 2δ 73

72 Closest pair of points: divide-and-conquer algorithm CLOSEST-PAIR(p1, p2,, pn) Compute vertical line L such that half the points are on each side of the line. δ1 CLOSEST-PAIR(points in left half). δ2 CLOSEST-PAIR(points in right half). δ min { δ1, δ2 }. Delete all points further than δ from line L. Sort remaining points by y-coordinate. Scan points in y-order and compare distance between each point and next 7 neighbors. If any of these distances is less than δ, update δ. O(n) T(n / 2) T(n / 2) O(n) O(n log n) O(n) RETURN δ. 74

73 <latexit sha1_base64="dpqsxns9emydq5iq+l5by9kuo1i=">aaac2nicbvfnixnbeo0zv9b4lv0phrwuzpueic4swq6elqvbpa6quavpifr0apjme7qh7h5jghlyjf79i/4c/4092vkwixv6vfdvrz7sqgrrouhpen66fefuvb37rqcphz1+0t4/+gp1atioujbaxkbmohqkr044izefqzanei/sqw+1fvenjrvadd2ywcrnmyuywznz1kt9e9hvptiffk1xjltffs+7atebhc7rsw7cg1dahs5cbskdq+vz40nyaaxjqh+meejzh10qm6m1afxmckhz4x7qwws6gfrkkgsj1fbgajwuuklnohas3t9ytsiqatpbpn2j+te6ybfedeiqjs4n+8ebnwpe5qgcl8zacrwvlqmycyjl9muwfgvgr9gmxx4qlqnnqvvtv/dsm1pi/g6zvg7w7l8vfcutxeapz8yzm9ttrsb/p41ll50klvbf6vdxa6osloa01i+cqtdinvx6wlgrflbgc2yyd/6dgy7r3gxyju2qrake11pcyqvbompqk8bbn9sfo6p+u3785w3n7kq55x55tl6qlonjmtkjn8k5greepatog4/bpzajv4c/wp/xqwhq1dwlgxh++gt4httm</latexit> <latexit sha1_base64="dpqsxns9emydq5iq+l5by9kuo1i=">aaac2nicbvfnixnbeo0zv9b4lv0phrwuzpueic4swq6elqvbpa6quavpifr0apjme7qh7h5jghlyjf79i/4c/4092vkwixv6vfdvrz7sqgrrouhpen66fefuvb37rqcphz1+0t4/+gp1atioujbaxkbmohqkr044izefqzanei/sqw+1fvenjrvadd2ywcrnmyuywznz1kt9e9hvptiffk1xjltffs+7atebhc7rsw7cg1dahs5cbskdq+vz40nyaaxjqh+meejzh10qm6m1afxmckhz4x7qwws6gfrkkgsj1fbgajwuuklnohas3t9ytsiqatpbpn2j+te6ybfedeiqjs4n+8ebnwpe5qgcl8zacrwvlqmycyjl9muwfgvgr9gmxx4qlqnnqvvtv/dsm1pi/g6zvg7w7l8vfcutxeapz8yzm9ttrsb/p41ll50klvbf6vdxa6osloa01i+cqtdinvx6wlgrflbgc2yyd/6dgy7r3gxyju2qrake11pcyqvbompqk8bbn9sfo6p+u3785w3n7kq55x55tl6qlonjmtkjn8k5greepatog4/bpzajv4c/wp/xqwhq1dwlgxh++gt4httm</latexit> <latexit sha1_base64="dpqsxns9emydq5iq+l5by9kuo1i=">aaac2nicbvfnixnbeo0zv9b4lv0phrwuzpueic4swq6elqvbpa6quavpifr0apjme7qh7h5jghlyjf79i/4c/4092vkwixv6vfdvrz7sqgrrouhpen66fefuvb37rqcphz1+0t4/+gp1atioujbaxkbmohqkr044izefqzanei/sqw+1fvenjrvadd2ywcrnmyuywznz1kt9e9hvptiffk1xjltffs+7atebhc7rsw7cg1dahs5cbskdq+vz40nyaaxjqh+meejzh10qm6m1afxmckhz4x7qwws6gfrkkgsj1fbgajwuuklnohas3t9ytsiqatpbpn2j+te6ybfedeiqjs4n+8ebnwpe5qgcl8zacrwvlqmycyjl9muwfgvgr9gmxx4qlqnnqvvtv/dsm1pi/g6zvg7w7l8vfcutxeapz8yzm9ttrsb/p41ll50klvbf6vdxa6osloa01i+cqtdinvx6wlgrflbgc2yyd/6dgy7r3gxyju2qrake11pcyqvbompqk8bbn9sfo6p+u3785w3n7kq55x55tl6qlonjmtkjn8k5greepatog4/bpzajv4c/wp/xqwhq1dwlgxh++gt4httm</latexit> <latexit sha1_base64="dpqsxns9emydq5iq+l5by9kuo1i=">aaac2nicbvfnixnbeo0zv9b4lv0phrwuzpueic4swq6elqvbpa6quavpifr0apjme7qh7h5jghlyjf79i/4c/4092vkwixv6vfdvrz7sqgrrouhpen66fefuvb37rqcphz1+0t4/+gp1atioujbaxkbmohqkr044izefqzanei/sqw+1fvenjrvadd2ywcrnmyuywznz1kt9e9hvptiffk1xjltffs+7atebhc7rsw7cg1dahs5cbskdq+vz40nyaaxjqh+meejzh10qm6m1afxmckhz4x7qwws6gfrkkgsj1fbgajwuuklnohas3t9ytsiqatpbpn2j+te6ybfedeiqjs4n+8ebnwpe5qgcl8zacrwvlqmycyjl9muwfgvgr9gmxx4qlqnnqvvtv/dsm1pi/g6zvg7w7l8vfcutxeapz8yzm9ttrsb/p41ll50klvbf6vdxa6osloa01i+cqtdinvx6wlgrflbgc2yyd/6dgy7r3gxyju2qrake11pcyqvbompqk8bbn9sfo6p+u3785w3n7kq55x55tl6qlonjmtkjn8k5greepatog4/bpzajv4c/wp/xqwhq1dwlgxh++gt4httm</latexit> Divide-and-conquer: quiz 6 What is the solution to the following recurrence? (1) n =1 T (n) = T ( n/2 )+T( n/2 ) + (n log n) n>1 A. T(n) = Θ(n). B. T(n) = Θ(n log n). C. T(n) = Θ(n log 2 n). D. T(n) = Θ(n 2 ). 75

74 Refined version of closest-pair algorithm Q. How to improve to O(n log n)? A. Don t sort points in strip from scratch each time. Each recursive call returns two lists: all points sorted by x-coordinate, and all points sorted by y-coordinate. Sort by merging two pre-sorted lists. Theorem. [Shamos 1975] The divide-and-conquer algorithm for finding a closest pair of points in the plane can be implemented in O(n log n) time. Pf. (1) n=1 T (n) = T ( n/2 ) + T ( n/2 ) + (n) n>1 <latexit sha1_base64="3pqz3or04schqniwe5a09tyf+wa=">aaac03icbvfnb9naef27fjtwlzyjlxepkbfsaleifkvajc4ci5s0lbjwtn6mk1xxa2t3xsuyuscu/bh+ef+gtetkjgfot+/nzjupojfc2cd44/l79+4/elj/qpx4ydnnz9shhxcmkzthmc9kpq9izlakhwmrrmsrxcnly4mx8fwxsr+8qw1epkz2lwousrksiedmomra/j3qqh58hbancs5uyv0vs27rar0t0lju2im3qc0ubqkigsplcsmjwaolk6b/imnkckddkhozzrru8qlqxeme0mfboobk5chko1xwtmo81i7hpzupfku1a4aatjtbp6gddkhyga5p4nx64b3swcalfjxlkhkzcypcrixtvncjbsvcym74nzvjxehfujrrwr91da8dm4pelzvkykln/ltrstsyvrq7zjtzhdnwkvj/2qswyvlucpuxfhw/nuokctad6kmwexq5lsshgnfczqp8wttj1v1xw6xunspf2krcfkrwbizbrlrlq1l1xxd7zrtgfnl/0a+/vesmz5pz7pox5bxpkpcckih5ss7jmhcv7b33pntd/8l/7v/wf96m+l5t84jshp/rl+pw2nq=</latexit> 76

75 Divide-and-conquer: quiz 7 What is the complexity of the 2D closest pair problem? A. Θ(n). B. Θ(n log * n). C. Θ(n log log n). D. Θ(n log n). E. Not even Tarjan knows. 77

76 Computational complexity of closest-pair problem Theorem. [Ben-Or 1983, Yao 1989] In quadratic decision tree model, any algorithm for closest pair (even in 1D) requires Ω(n log n) quadratic tests. (x 1 - x 2 ) 2 + (y 1 - y 2 ) 2 Theorem. [Rabin 1976] There exists an algorithm to find the closest pair of points in the plane whose expected running time is O(n). A NOTE ON RABIN S NEAREST-NEIGHBOR ALGORITHM * Steve FORTUNE and John HOPCROFT Department of Computer Science, Cornell University, Ithaca, NY, U.S.A. not subject to Ω(n log n) lower bound because it uses the floor function Received 20 July 1978, revised version received 21 August 1978 Probabiktic algorithms, nearest neighbor, hashing 78

77 Digression: computational geometry Ingenious divide-and-conquer algorithms for core geometric problems. problem brute clever closest pair O(n 2 ) O(n log n) farthest pair O(n 2 ) O(n log n) convex hull O(n 2 ) O(n log n) Delaunay/Voronoi O(n 4 ) O(n log n) Euclidean MST O(n 2 ) O(n log n) running time to solve a 2D problem with n points Note. 3D and higher dimensions test limits of our ingenuity. 79

78 Convex hull The convex hull of a set of n points is the smallest perimeter fence enclosing the points. Equivalent definitions. Smallest area convex polygon enclosing the points. Intersection of all convex set containing all the points. 80

79 Farthest pair Given n points in the plane, find a pair of points with the largest Euclidean distance between them. Fact. Points in farthest pair are extreme points on convex hull. 81

80 Delaunay triangulation The Delaunay triangulation is a triangulation of n points in the plane such that no point is inside the circumcircle of any triangle. point inside circumcircle of 3 points Some useful properties. No edges cross. Delaunay triangulation of 19 points Among all triangulations, it maximizes the minimum angle. Contains an edge between each point and its nearest neighbor. no point in the set is inside the circumcircle 82

81 Euclidean MST Given n points in the plane, find MST connecting them. [distances between point pairs are Euclidean distances] Fact. Euclidean MST is subgraph of Delaunay triangulation. Implication. Can compute Euclidean MST in O(n log n) time. Compute Delaunay triangulation. Compute MST of Delaunay triangulation. it s planar ( 3n edges) 83

82 Computational geometry applications Applications. Robotics. VLSI design. Data mining. Medical imaging. Computer vision. Scientific computing. Finite-element meshing. Astronomical simulation. Models of physical world. Geographic information systems. Computer graphics (movies, games, virtual reality). airflow around an aircraft wing 84