COL 730: Parallel Programming
|
|
- Douglas Montgomery
- 5 years ago
- Views:
Transcription
1 COL 730: Parallel Programming
2 PARALLEL SORTING
3 Bitonic Merge and Sort Bitonic sequence: {a 0, a 1,, a n-1 }: A sequence with a monotonically increasing part and a monotonically decreasing part For some i, {a 0 <= <=a i } and {a i+1 >= >= a n-1 } Or, a cyclic-shift of indices makes it bitonic
4 Split Bitonic Sequence Subsequence 2 Subsequence 1 Say {a 0 <= <=a n/2 >= a n/2+1 >= >= a n-1 } Subsequence 1: {min(a 0, a n/2+1 ), min(a 1, a n/2+2 ), } Subsequence 2: {max(a 0, a n/2+1 ), max(a 1, a n/2+2 ), } Recursively sort each bitonic subsequence Subsequence 1 <= Subsequence 2
5 Split Bitonic Sequence Subsequence 2 Subsequence 1 Say {a 0 <= <=a n/2 >= a n/2+1 >= >= a n-1 } Subsequence 1: {min(a 0, a n/2+1 ), min(a 1, a n/2+2 ), } Subsequence 2: {max(a 0, a n/2+1 ), max(a 1, a n/2+2 ), } Recursively sort each bitonic subsequence Subsequence 1 <= Subsequence 2 5
6 Merge Bitonic Sequence Sequence 1 Sequence 2 Sort the first sequence in increasing order Sort the second in decreasing order
7 Bitonic Sort Sort each pair alternately in increasing and decreasing orders Every sequence of length four is now bitonic Sort recursively Again alternate increasing and decreasing orders Forming bitonic sequences of length eight now And so on.. 7
8 Bitonic Network log n stages log 2 n Dme nlog 2 n work Compare- Exchange Stage 1 Stage 2 Stage 3 t n = t n/2 + log n No beker than nlog 2 n/p Dme on p procs
9 Sorting if n>p Simulate n/p virtual processors per processor like PRAM simulation earlier Or, divide element into p blocks and locally sort n/p elements (of each block) Then perform bitonic sort of blocks compare exchange becomes compare-split O(n/p log n/p + n/p log 2 p) time local sorting log 2 p comp-splits
10 Odd-Even Exchange Repeat n/2 Dmes O(n 2 ) work Divide into p blocks Sort each block locally Repeat p/2 times odd and even phase compare-split each done on p blocks by p processors O(n/p log n/p) + O(n) time each processor sequentially splits 2n/p sized block local sort p comp-split
11 Batcher s Odd-Even Merge
12 Batcher s Odd-Even Merge Two way merge Given two sorted subsequences: Merge only even elements of each: e0, e1, e2, e3,.. Merge only odd elements of each: o0, o1, o2, o3,.. Perform a pair-wise exchange W(2n) = 2W(n) + O(n) T(2n) = T(n) + O(1) n log n work log n Dme e0, min(e1, o0), max(e1, o0), min(e2, o1), max(e2, o1),.. on-1
13 Parallel Bucket Sort Divide the range [a,b] of numbers into p equal sub-ranges or, buckets Divide input into p blocks pi Sends elements in its block to ith bucket For uniformly distributed input, expected bucket size is uniform Locally sort each bucket
14 Parallel Bucket Sort Decide buckets e.g., ranges of key values Parallel for i: Put element i in bucket b May keep each bucket locally at processor Later Merge buckets from all processors Sort each bucket separately For uniformly distributed input expected bucket size is uniform But, real risk of load imbalance Sample sort: Choose a sample of size s Sort the samples Choose B-1 evenly spaced element from the sorted list These splitters provide ranges for B buckets O(n/p log n/p + p log p)?
15 Parallel SpliKer SelecDon Divide n elements equally into B blocks (Quick)Sort each block (n/b log n/b) For each sorted block: Choose B-1 evenly spaced splitters Use the B*(B-1) elements as samples Sort the samples (B 2 log B) Choose B-1 Splitters Arrange elements by bucket in output array Count the number of elements in each bucket Perform prefix Sum of counts; Reserve space per bucket In-place No bucket contains more than 2*n/B elements (n/b + B log B)
16 Radix Sort Least significant digit first b-bits per round Entire list sorted per round (of bucket sort) Subdivide into p equal subsets Most significant digit first Separate into buckets Sort only within bucket per round load balance? Enough buckets => Local sort per processor Parallel round Local bucketing Bucket merging 16
17 Sort n/p elements, then Merge Merge 2n/p-elements pairs P 0,P 1, P 2,P 3 p-processors merging P 0,P Merge n/p-elements 1 P 2,P 3 pairs P 0 P 1 P p-1 Sort first n/p elements Sort second n/p elements HOW EFFICIENTLY CAN YOU MERGE?
18 Merge Sort Divide into P groups Locally sort each group N/P log (N/P) = O((N log N)/P) Parallel merge P groups Binary tree: log(p) stages P/2 pairs using P processors At leaf (level 1), 2 processors merge two N/P sized lists each time = O(N/P) Level i: 2 i+1 processors merge two 2 i N/P sized lists each time = O(N/P) At the root: P processor merge two N/2 sized list time = O(N/P) O(N/P) log P using P processors to perform each merge 18
19 Merging Options Optimal merge O(n) work n-list merge Sort the smallest elements of each list: Ls Output the smallest element of Ls Advance pointer of the corresponding list Parallel Insert the next element into Ls Extension of Batcher s algorithm simpler to implement and schedule 19
20 Batcher s Odd-Even Merge Two way merge Given two sorted subsequences: Merge only even elements of each: e0, e1, e2, e3,.. Merge only odd elements of each: o0, o1, o2, o3,.. Perform a pair-wise exchange W(2n) = 2W(n) + O(n) T(2n) = T(n) + O(1) n log n work log n Dme e0, min(e1, o0), max(e1, o0), min(e2, o1), max(e2, o1),.. on-1
21 Optimal Multi-way Merge N=P 2 1.Divide L1 and L2, respectively, into P sublists each ith sublist contains elements at positions i, P+i, 2P+i.. N-P+i 2.Pi merges the ith sublist from L1 and the ith sublist from L2 Result go back to the positions originally occupied by the two sublists Each element is at most P off from its final position 3.Divide list in blocks of length P 4.Pi merges pairs of blocks block 2i and 2i+1 (Results are put in-place.) 5.Pi merges blocks 2i+1 and 2i+2 Results in-place 21
22 Multi-way Merge: Proof A Let L1i and L2i be the sublists assigned to Pi in step 1 Consider X, nth element in L2i: there are three cases X lies between the mth element of L1i, A, and (m+1)st, B Or, X is <all elements of L1i, or it is >all L1i Case 1: At least [mp+i+1]+[np+i] = [(m+n+1)p+i]+(i-p+1) elements <X mp+i+1 elements are from L1 (<= A) and np+ i are from L2 (<X) At most [(m+1)p+i] + [np+i] = [(m+n+1)p+i] + i elements < X (m+1)p+i elements are from L1 (<B) and np+ i are from L2 (<X) Rank of X in all elements is between [(m+n+1)p + i] + i-p+1 and [(m+n+1)p + i] + i The array-position of X after merger is [(m+n+1)p + i] i+mp i+(m+1)p i+np P P A B X I J, L1 L2 L2 i + (m+n+1)p X 22
23 Multi-way Merge: Proof B Claim: Each block is sorted Proof: Consider P elements of block i: Block i XY j j+1 j j j+1 X, the j th element (0 <= j < P) is stored by Pj This j th element is the i th element for Pj Prove that j th element (X) is smaller than the j+1 st element (Y) for all j Every element assigned to Pj+1 in Step 1 > the element to its left which is assigned to Pj Y is the ith element from Pj+1 after Step 2 each element assigned to Pj+1 is greater than an element assigned to Pj => Y is greater than at least i elements from Pj But X is the ith (sorted) element of Pj. Hence X < Y 23
24 Claim: Multi-way Merge: Proof C After Step 2, every element in block i is < all elements in block j, j>i+1 i.e., the largest element in block i, X, is < the smallest, Y, in block i+2 X is the last element in the block i and Y is the first element in block i+2 Note: X is assigned to PP-1 and Y is assigned to P0 in Step 1 P-2 of the elements assigned to P0 are > the resp. elements on their left which, in turn, are all assigned to PP-1 The two exceptions are the first elements from L1 and L2, resp Since Y is the (i+2)nd element from P0 after Step 2, Y is greater than at least i elements assigned to PP-1, But X is the ith element from Pp-1. Hence X < Y O(N/P) per merge, 3 merges Block i X PP-1 Block i+2 Y P0 24
25 Multi-way Merge N>P 2 1.Divide L1 and L2, respectively, into P sublists each ith sublist contains elements at positions i, P+i, 2P+i.. N-P+i 2.Pi merges the ith sublist from L1 and the ith sublist from L2 Result go back to the positions originally occupied by the two sublists Each element is at most P+1 off from its final position 3.Divide list in blocks of length P 4.Pi merges pairs of blocks block 2i and 2i+1 Results are put in-place 5.Pi merges blocks 2i+1 and 2i+2 Results in-place & P 2 +2i and P 2 +2i+1, etc. & P 2 +2i and P 2 +2i+1, etc. SDll O(N/P) 25
26 MulD-way Merge Sort Total Time = O(N/P) log P 2N/P-element pairs P 0,P 1, P 2,P 3 P/4 pairs, 4 procs each Time at each level = O(N/P) N/P-element P/2 pairs, pairs P 0,P 1 2 procs each P 2,P 3 Time = (N/P)/2 Sort first N/P elements Sort second N/P elements P lists, sort each Sorting Time = O(N/P) log (N/P) = O(N/P log N)
27 Practical Sorting Multi-way suitable for CUDA? Too many elements per processor (Shared Mem size) Slow network Sample sort Shared memory Bitonic, Odd-even merge External sort Merge 27
28 Example CUDA Sort 1.Divide input into p equal parts size t each 2.Sort each part in a block Batcher s odd-even mergesort 3.Merge p sorted lists Merge pairs of subsequences per block? the number of pairs decreases quickly Use multiple blocks per merge for larger lists subdivide into smaller tasks 28
29 Single Block Merge Lists with, say, t ~ 256 elements each, merge using a single t-thread block For an element ai A, Rank(ai, C ) = i + Rank(ai, B) Thread i computes Rank(ai, B) using binary search B in shared memory Rank elements of B similarly Thread i writes ai to the final Rank position in shared memory 29
30 Multiple Block Merge Divide large lists into sequences of size t Select splitters SA and SB Every t-th element, partitions lists into chunks of size t Merge SA and SB into S recursively select splitters if SA or SB are too large Compute Rank(s, B), s SA Compute Rank(s, SB) = Rank(s, S) Rank(s, SA) This provides the t-element chunk of B within which s must lie Binary search per thread Similarly compute Rank(s, A), s SB Split A and B, each with S Merge pairs, one per block A B t t S A S B 30
31 LOG N TIME SORTING
32 Examples of Fast Sort Rank sort? Given Array A, For each i, find rank A[i] A[rank[i]] = A[i] How fast can you find Rank(A:A)? If you had n 2 processors How fast can you merge sorted sublists? If you had n 2 processors?
33 c-cover Merging Consider sequence A and B X: a c-cover of A and B Two consecutive elements of X have at most c elements of A between them and at most c elements of B Given Rank(X:A) and Rank(X:B) Compute Rank(A:B) and Rank(B:A) In O(1) time, with O(n) work If X is a c-cover of B, and we know Rank(A:X) and Rank(X:B) Compute Rank(A:B) in O(1)
34 c-cover Merging Rank(X:A);Rank(X:B) => Rank(A:B);Rank(B:A) a1.. ar 1 ar ar 2.. a ar i ar i.. A i x1 x2.. xi-1 xi.. xs b1.. bt 1 bt bt 2.. bt i bt i.. Rank(a:B) = Rank(x i-1 :B) + Rank(a:B i ) But Bi <= c, as X is a c-cover of B Rank(a:B) can be found in O(1) B i 34
35 c-cover Merging Rank(A:X);Rank(X:B) => Rank(A:B) a ai.... a... x1 x2.. xi-1 xi.. xs b1.. bt 1 bt bt 2.. bt i bt i.. Rank(a:B) = Rank(Rank(a, X), B) + Rank(a:B i ) But Bi <= c, as X is a c-cover of B Rank(a:B) can be found in O(1) B i 35
36 OpDmal O(log n)-dme Merge Sort Algorithm runs in stages on a binary tree Node i maintains sorted list L[i] s th stage generates list L s [i] Initially: L 0 [i] = null for internal nodes L 0 [i] = value at leaf node i Procedure works for any proper binary tree Not necessarily balanced
37 Fast OpDmal Merge Sort: DefiniDons Algorithm proceeds up the tree, one stage at a time At stage s, node n is active if height(n) <= s <= 3*height(n) height(n) = height(tree) path-length from root to n At each stage a node is active, It merges a sample of the lists of its children
38 Algorithm parallel for n = active nodes: L s+1 [n] = Merge(Sample s (left), Sample s (right])) Sample s (n) = SUB 4 (L s [n]), if s <= 3*height(n) SUB 2 (L s [n]), if s = 3*height(n) + 1 SUB 1 (L s [n]) = L s [n], if s >= 3*height(n) + 2 where, SUB i (l 0, l 1, l 2 ) = (l i-1, l 2i-1, l 3i-1 )
39 Pipelined Merges A node: activates 1 step after child active for 3 extra steps height On activation, a node merges its children's every 4th element On children s deactivation, a node takes children s alternate elements then every child element 39
40 Brief Analysis 1.Node becomes full at stage = 3*height(node) Induction on height 2.Number of elements at a node doubles: L s+1 [n] <= 2 L s [n] + 4 Induction on height 3.Number of elements in active nodes = O(n) Levels active are s/3 to s Level s/3 is full = O(n) Levels above = O(n/2) and O(n/4).. Total = O(n) 4.Sample s (n) is a 4-cover for Sample s+1 (n) i.e., No more than 4 items of L s+1 [n] between two consecutive items of L s [n] 5.For each stage s > height(n), L s [n] is a 4-cover for Sample s (left) and Sample s (right)
41 Samples(n) is a 4-cover for Samples+1(n) Def: Interval [a,b] k-intersects [-,Samples, ] if {x, s.t. x Samples & a x b} = k Claim: [a,b] k-intersects Samples-1 => [a,b] 2k-intersects Samples Induction: [a,b] k-intersects Samples-1 [a,b] 2k-intersects Sample s Show [a,b] 4k-intersects Sample s+1 SUB 4 (L s+1 ): 4k L s+1 = Merge(SUB 4 (L s ),): 16k-2 SUB 4 (L s ) 2k a L s 8k-3 b SUB 4 (L s-1 ) p 1 a p 2 a 1 b 1 2 b 2 SUB 4 ( L s ): lee & right p 1 +p 2 8k-1 2p 1 +2p 2 2(8k-1) 41
42 Algorithm Details Merge(Sample s (left), Sample s (right)) => compute: Rank(Sample s (left), Sample s (right)) Rank(Sample s (right), Sample s (left)) Use Rank(L s,sample s (left)), Rank(L s,sample s (right)) O(1), since Ls is 4-cover of each Samples Compute Rank(L s :Sample s (left): Rank(L s :Sample s-1 (left)) Rank(Sample s-1 (left):sample s (left)) 4-cover Similarly compute Rank(L s :Sample s (right)) 42
43 Algorithm Details cover L s L s+1 Sample s Sample s For s >= 2, Merge => compute: Rank(Sample s (left), Sample s (right)) Rank(Sample s (right), Sample s (left)) Use Rank(L s :Sample s (left)), Rank(L s :Sample s (right)) O(1), since Ls is 4-cover of each Samples Compute Rank(L s :Sample s (left): Rank(L s :Sample s-1 (left)) Rank(Sample s-1 (left):sample s (left)) 4-cover Similarly compute Rank(L s :Sample s (right)) 43
Algorithms PART II: Partitioning and Divide & Conquer. HPC Fall 2007 Prof. Robert van Engelen
Algorithms PART II: Partitioning and Divide & Conquer HPC Fall 2007 Prof. Robert van Engelen Overview Partitioning strategies Divide and conquer strategies Further reading HPC Fall 2007 2 Partitioning
More informationData Structures and Algorithms
Data Structures and Algorithms Spring 2017-2018 Outline 1 Sorting Algorithms (contd.) Outline Sorting Algorithms (contd.) 1 Sorting Algorithms (contd.) Analysis of Quicksort Time to sort array of length
More informationSorting algorithms. Sorting algorithms
Properties of sorting algorithms A sorting algorithm is Comparison based If it works by pairwise key comparisons. In place If only a constant number of elements of the input array are ever stored outside
More informationLecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018
CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of
More informationFast Sorting and Selection. A Lower Bound for Worst Case
Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 0 Fast Sorting and Selection USGS NEIC. Public domain government image. A Lower Bound
More informationInteger Sorting on the word-ram
Integer Sorting on the word-rm Uri Zwick Tel viv University May 2015 Last updated: June 30, 2015 Integer sorting Memory is composed of w-bit words. rithmetical, logical and shift operations on w-bit words
More informationLecture 6 September 21, 2016
ICS 643: Advanced Parallel Algorithms Fall 2016 Lecture 6 September 21, 2016 Prof. Nodari Sitchinava Scribe: Tiffany Eulalio 1 Overview In the last lecture, we wrote a non-recursive summation program and
More informationDictionary: an abstract data type
2-3 Trees 1 Dictionary: an abstract data type A container that maps keys to values Dictionary operations Insert Search Delete Several possible implementations Balanced search trees Hash tables 2 2-3 trees
More informationAlgorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count
Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth
More informationEE/CSCI 451: Parallel and Distributed Computation
EE/CSCI 451: Parallel and Distributed Computation Lecture #19 3/28/2017 Xuehai Qian Xuehai.qian@usc.edu http://alchem.usc.edu/portal/xuehaiq.html University of Southern California 1 From last class PRAM
More informationR ij = 2. Using all of these facts together, you can solve problem number 9.
Help for Homework Problem #9 Let G(V,E) be any undirected graph We want to calculate the travel time across the graph. Think of each edge as one resistor of 1 Ohm. Say we have two nodes: i and j Let the
More informationLinear Time Selection
Linear Time Selection Given (x,y coordinates of N houses, where should you build road These lecture slides are adapted from CLRS.. Princeton University COS Theory of Algorithms Spring 0 Kevin Wayne Given
More informationSorting. Chapter 11. CSE 2011 Prof. J. Elder Last Updated: :11 AM
Sorting Chapter 11-1 - Sorting Ø We have seen the advantage of sorted data representations for a number of applications q Sparse vectors q Maps q Dictionaries Ø Here we consider the problem of how to efficiently
More informationAlgorithms and Data Structures 2016 Week 5 solutions (Tues 9th - Fri 12th February)
Algorithms and Data Structures 016 Week 5 solutions (Tues 9th - Fri 1th February) 1. Draw the decision tree (under the assumption of all-distinct inputs) Quicksort for n = 3. answer: (of course you should
More informationAlgorithms And Programming I. Lecture 5 Quicksort
Algorithms And Programming I Lecture 5 Quicksort Quick Sort Partition set into two using randomly chosen pivot 88 31 25 52 14 98 62 30 23 79 14 31 2530 23 52 88 62 98 79 Quick Sort 14 31 2530 23 52 88
More informationDictionary: an abstract data type
2-3 Trees 1 Dictionary: an abstract data type A container that maps keys to values Dictionary operations Insert Search Delete Several possible implementations Balanced search trees Hash tables 2 2-3 trees
More informationMA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3
MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents
More informationSorting Algorithms. We have already seen: Selection-sort Insertion-sort Heap-sort. We will see: Bubble-sort Merge-sort Quick-sort
Sorting Algorithms We have already seen: Selection-sort Insertion-sort Heap-sort We will see: Bubble-sort Merge-sort Quick-sort We will show that: O(n log n) is optimal for comparison based sorting. Bubble-Sort
More informationDivide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14
Divide and Conquer Algorithms CSE 101: Design and Analysis of Algorithms Lecture 14 CSE 101: Design and analysis of algorithms Divide and conquer algorithms Reading: Sections 2.3 and 2.4 Homework 6 will
More informationCMPT 307 : Divide-and-Conqer (Study Guide) Should be read in conjunction with the text June 2, 2015
CMPT 307 : Divide-and-Conqer (Study Guide) Should be read in conjunction with the text June 2, 2015 1 Introduction The divide-and-conquer strategy is a general paradigm for algorithm design. This strategy
More informationJeffrey D. Ullman Stanford University
Jeffrey D. Ullman Stanford University 3 We are given a set of training examples, consisting of input-output pairs (x,y), where: 1. x is an item of the type we want to evaluate. 2. y is the value of some
More informationAdvanced Analysis of Algorithms - Midterm (Solutions)
Advanced Analysis of Algorithms - Midterm (Solutions) K. Subramani LCSEE, West Virginia University, Morgantown, WV {ksmani@csee.wvu.edu} 1 Problems 1. Solve the following recurrence using substitution:
More informationCS60007 Algorithm Design and Analysis 2018 Assignment 1
CS60007 Algorithm Design and Analysis 2018 Assignment 1 Palash Dey and Swagato Sanyal Indian Institute of Technology, Kharagpur Please submit the solutions of the problems 6, 11, 12 and 13 (written in
More informationRecap: Prefix Sums. Given A: set of n integers Find B: prefix sums 1 / 86
Recap: Prefix Sums Given : set of n integers Find B: prefix sums : 3 1 1 7 2 5 9 2 4 3 3 B: 3 4 5 12 14 19 28 30 34 37 40 1 / 86 Recap: Parallel Prefix Sums Recursive algorithm Recursively computes sums
More information4.8 Huffman Codes. These lecture slides are supplied by Mathijs de Weerd
4.8 Huffman Codes These lecture slides are supplied by Mathijs de Weerd Data Compression Q. Given a text that uses 32 symbols (26 different letters, space, and some punctuation characters), how can we
More informationCPS 616 DIVIDE-AND-CONQUER 6-1
CPS 616 DIVIDE-AND-CONQUER 6-1 DIVIDE-AND-CONQUER Approach 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger)
More informationQuick Sort Notes , Spring 2010
Quick Sort Notes 18.310, Spring 2010 0.1 Randomized Median Finding In a previous lecture, we discussed the problem of finding the median of a list of m elements, or more generally the element of rank m.
More informationData Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 8 Analyzing Quick Sort Sofya Raskhodnikova and Adam Smith Reminder: QuickSort Quicksort an n-element array: 1. Divide: Partition the array around a pivot
More informationReview Of Topics. Review: Induction
Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case
More informationAdvanced Implementations of Tables: Balanced Search Trees and Hashing
Advanced Implementations of Tables: Balanced Search Trees and Hashing Balanced Search Trees Binary search tree operations such as insert, delete, retrieve, etc. depend on the length of the path to the
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points randomized quicksort median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationAnalysis of Algorithms CMPSC 565
Analysis of Algorithms CMPSC 565 LECTURES 38-39 Randomized Algorithms II Quickselect Quicksort Running time Adam Smith L1.1 Types of randomized analysis Average-case analysis : Assume data is distributed
More informationBin Sort. Sorting integers in Range [1,...,n] Add all elements to table and then
Sorting1 Bin Sort Sorting integers in Range [1,...,n] Add all elements to table and then Retrieve in order 1,2,3,...,n Stable Sorting Method (repeated elements will end up in their original order) Numbers
More informationdata structures and algorithms lecture 2
data structures and algorithms 2018 09 06 lecture 2 recall: insertion sort Algorithm insertionsort(a, n): for j := 2 to n do key := A[j] i := j 1 while i 1 and A[i] > key do A[i + 1] := A[i] i := i 1 A[i
More informationParallel programming using MPI. Analysis and optimization. Bhupender Thakur, Jim Lupo, Le Yan, Alex Pacheco
Parallel programming using MPI Analysis and optimization Bhupender Thakur, Jim Lupo, Le Yan, Alex Pacheco Outline l Parallel programming: Basic definitions l Choosing right algorithms: Optimal serial and
More informationFundamental Algorithms
Fundamental Algorithms Chapter 5: Searching Michael Bader Winter 2014/15 Chapter 5: Searching, Winter 2014/15 1 Searching Definition (Search Problem) Input: a sequence or set A of n elements (objects)
More informationBDD Based Upon Shannon Expansion
Boolean Function Manipulation OBDD and more BDD Based Upon Shannon Expansion Notations f(x, x 2,, x n ) - n-input function, x i = or f xi=b (x,, x n ) = f(x,,x i-,b,x i+,,x n ), b= or Shannon Expansion
More informationLinear Selection and Linear Sorting
Analysis of Algorithms Linear Selection and Linear Sorting If more than one question appears correct, choose the more specific answer, unless otherwise instructed. Concept: linear selection 1. Suppose
More informationFundamental Algorithms
Chapter 2: Sorting, Winter 2018/19 1 Fundamental Algorithms Chapter 2: Sorting Jan Křetínský Winter 2018/19 Chapter 2: Sorting, Winter 2018/19 2 Part I Simple Sorts Chapter 2: Sorting, Winter 2018/19 3
More informationFundamental Algorithms
Fundamental Algorithms Chapter 2: Sorting Harald Räcke Winter 2015/16 Chapter 2: Sorting, Winter 2015/16 1 Part I Simple Sorts Chapter 2: Sorting, Winter 2015/16 2 The Sorting Problem Definition Sorting
More informationModule 1: Analyzing the Efficiency of Algorithms
Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?
More informationWeek 5: Quicksort, Lower bound, Greedy
Week 5: Quicksort, Lower bound, Greedy Agenda: Quicksort: Average case Lower bound for sorting Greedy method 1 Week 5: Quicksort Recall Quicksort: The ideas: Pick one key Compare to others: partition into
More informationSearching. Sorting. Lambdas
.. s Babes-Bolyai University arthur@cs.ubbcluj.ro Overview 1 2 3 Feedback for the course You can write feedback at academicinfo.ubbcluj.ro It is both important as well as anonymous Write both what you
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 9 Divide and Conquer Merge sort Counting Inversions Binary Search Exponentiation Solving Recurrences Recursion Tree Method Master Theorem Sofya Raskhodnikova S. Raskhodnikova;
More informationBucket-Sort. Have seen lower bound of Ω(nlog n) for comparisonbased. Some cheating algorithms achieve O(n), given certain assumptions re input
Bucket-Sort Have seen lower bound of Ω(nlog n) for comparisonbased sorting algs Some cheating algorithms achieve O(n), given certain assumptions re input One example: bucket sort Assumption: input numbers
More informationQuiz 2. Due November 26th, CS525 - Advanced Database Organization Solutions
Name CWID Quiz 2 Due November 26th, 2015 CS525 - Advanced Database Organization s Please leave this empty! 1 2 3 4 5 6 7 Sum Instructions Multiple choice questions are graded in the following way: You
More informationCSCE 750, Spring 2001 Notes 2 Page 1 4 Chapter 4 Sorting ffl Reasons for studying sorting is a big deal pedagogically useful Λ the application itself
CSCE 750, Spring 2001 Notes 2 Page 1 4 Chapter 4 Sorting ffl Reasons for studying sorting is a big deal pedagogically useful Λ the application itself is easy to understand Λ a complete analysis can often
More informationDivide-and-conquer. Curs 2015
Divide-and-conquer Curs 2015 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers. Known Examples: Binary
More informationRandomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th
CSE 3500 Algorithms and Complexity Fall 2016 Lecture 10: September 29, 2016 Quick sort: Average Run Time In the last lecture we started analyzing the expected run time of quick sort. Let X = k 1, k 2,...,
More informationLecture 26 December 1, 2015
CS 229r: Algorithms for Big Data Fall 2015 Prof. Jelani Nelson Lecture 26 December 1, 2015 Scribe: Zezhou Liu 1 Overview Final project presentations on Thursday. Jelani has sent out emails about the logistics:
More informationSorting DS 2017/2018
Sorting DS 2017/2018 Content Sorting based on comparisons Bubble sort Insertion sort Selection sort Merge sort Quick sort Counting sort Distribution sort FII, UAIC Lecture 8 DS 2017/2018 2 / 44 The sorting
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)
More informationDivide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,
Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b
More information! Break up problem into several parts. ! Solve each part recursively. ! Combine solutions to sub-problems into overall solution.
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer.! Break up problem into several parts.! Solve each part recursively.! Combine solutions to sub-problems into overall solution. Most common
More informationA Linear Time Algorithm for Ordered Partition
A Linear Time Algorithm for Ordered Partition Yijie Han School of Computing and Engineering University of Missouri at Kansas City Kansas City, Missouri 64 hanyij@umkc.edu Abstract. We present a deterministic
More informationCopyright 2000, Kevin Wayne 1
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common
More informationCentral Algorithmic Techniques. Iterative Algorithms
Central Algorithmic Techniques Iterative Algorithms Code Representation of an Algorithm class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length;
More informationPartition and Select
Divide-Conquer-Glue Algorithms Quicksort, Quickselect and the Master Theorem Quickselect algorithm Tyler Moore CSE 3353, SMU, Dallas, TX Lecture 11 Selection Problem: find the kth smallest number of an
More informationDivide-and-conquer: Order Statistics. Curs: Fall 2017
Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.
More informationACO Comprehensive Exam March 20 and 21, Computability, Complexity and Algorithms
1. Computability, Complexity and Algorithms Part a: You are given a graph G = (V,E) with edge weights w(e) > 0 for e E. You are also given a minimum cost spanning tree (MST) T. For one particular edge
More informationLecture 2 September 4, 2014
CS 224: Advanced Algorithms Fall 2014 Prof. Jelani Nelson Lecture 2 September 4, 2014 Scribe: David Liu 1 Overview In the last lecture we introduced the word RAM model and covered veb trees to solve the
More informationSelection and Adversary Arguments. COMP 215 Lecture 19
Selection and Adversary Arguments COMP 215 Lecture 19 Selection Problems We want to find the k'th largest entry in an unsorted array. Could be the largest, smallest, median, etc. Ideas for an n lg n algorithm?
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationCS361 Homework #3 Solutions
CS6 Homework # Solutions. Suppose I have a hash table with 5 locations. I would like to know how many items I can store in it before it becomes fairly likely that I have a collision, i.e., that two items
More informationKartsuba s Algorithm and Linear Time Selection
CS 374: Algorithms & Models of Computation, Fall 2015 Kartsuba s Algorithm and Linear Time Selection Lecture 09 September 22, 2015 Chandra & Manoj (UIUC) CS374 1 Fall 2015 1 / 32 Part I Fast Multiplication
More information1 Terminology and setup
15-451/651: Design & Analysis of Algorithms August 31, 2017 Lecture #2 last changed: August 29, 2017 In this lecture, we will examine some simple, concrete models of computation, each with a precise definition
More informationMethods for solving recurrences
Methods for solving recurrences Analyzing the complexity of mergesort The merge function Consider the following implementation: 1 int merge ( int v1, int n1, int v, int n ) { 3 int r = malloc ( ( n1+n
More informationQuiz 1 Solutions. (a) f 1 (n) = 8 n, f 2 (n) = , f 3 (n) = ( 3) lg n. f 2 (n), f 1 (n), f 3 (n) Solution: (b)
Introduction to Algorithms October 14, 2009 Massachusetts Institute of Technology 6.006 Spring 2009 Professors Srini Devadas and Constantinos (Costis) Daskalakis Quiz 1 Solutions Quiz 1 Solutions Problem
More informationData selection. Lower complexity bound for sorting
Data selection. Lower complexity bound for sorting Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 12 1 Data selection: Quickselect 2 Lower complexity bound for sorting 3 The
More informationBiased Quantiles. Flip Korn Graham Cormode S. Muthukrishnan
Biased Quantiles Graham Cormode cormode@bell-labs.com S. Muthukrishnan muthu@cs.rutgers.edu Flip Korn flip@research.att.com Divesh Srivastava divesh@research.att.com Quantiles Quantiles summarize data
More informationAlgorithms. Quicksort. Slide credit: David Luebke (Virginia)
1 Algorithms Quicksort Slide credit: David Luebke (Virginia) Sorting revisited We have seen algorithms for sorting: INSERTION-SORT, MERGESORT More generally: given a sequence of items Each item has a characteristic
More informationProblem. Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26
Binary Search Introduction Problem Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26 Strategy 1: Random Search Randomly select a page until the page containing
More informationCOMP 633: Parallel Computing Fall 2018 Written Assignment 1: Sample Solutions
COMP 633: Parallel Computing Fall 2018 Written Assignment 1: Sample Solutions September 12, 2018 I. The Work-Time W-T presentation of EREW sequence reduction Algorithm 2 in the PRAM handout has work complexity
More information0-1 Knapsack Problem in parallel Progetto del corso di Calcolo Parallelo AA
0-1 Knapsack Problem in parallel Progetto del corso di Calcolo Parallelo AA 2008-09 Salvatore Orlando 1 0-1 Knapsack problem N objects, j=1,..,n Each kind of item j has a value p j and a weight w j (single
More informationAdvanced Counting Techniques. Chapter 8
Advanced Counting Techniques Chapter 8 Chapter Summary Applications of Recurrence Relations Solving Linear Recurrence Relations Homogeneous Recurrence Relations Nonhomogeneous Recurrence Relations Divide-and-Conquer
More informationGraph-theoretic Problems
Graph-theoretic Problems Parallel algorithms for fundamental graph-theoretic problems: We already used a parallelization of dynamic programming to solve the all-pairs-shortest-path problem. Here we are
More informationChapter 5. Divide and Conquer. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each
More informationSorting and Searching. Tony Wong
Tony Wong 2017-03-04 Sorting Sorting is the reordering of array elements into a specific order (usually ascending) For example, array a[0..8] consists of the final exam scores of the n = 9 students in
More informationDiscrete Applied Mathematics
Discrete Applied Mathematics 159 (011) 1398 1417 Contents lists available at ScienceDirect Discrete Applied Mathematics journal homepage: www.elsevier.com/locate/dam Recursive merge sort with erroneous
More informationLecture 9. Greedy Algorithm
Lecture 9. Greedy Algorithm T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018
More informationCS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationAnalysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort
Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map
More informationData Structures in Java
Data Structures in Java Lecture 20: Algorithm Design Techniques 12/2/2015 Daniel Bauer 1 Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways of
More informationSorting. CMPS 2200 Fall Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 2200 Fall 2017 Sorting Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 11/17/17 CMPS 2200 Intro. to Algorithms 1 How fast can we sort? All the sorting algorithms
More informationAdministrivia. COMP9020 Lecture 7 Session 2, 2017 Induction and Recursion. Lecture 6 recap. Lecture 6 recap
Administrivia COMP9020 Lecture 7 Session 2, 2017 Induction and Recursion Guidelines for good mathematical writing Assignment 1 Solutions now available; marks available soon Assignment 2 available on Saturday,
More informationCPSC 320 Sample Final Examination December 2013
CPSC 320 Sample Final Examination December 2013 [10] 1. Answer each of the following questions with true or false. Give a short justification for each of your answers. [5] a. 6 n O(5 n ) lim n + This is
More informationAdvanced Counting Techniques
. All rights reserved. Authorized only for instructor use in the classroom. No reproduction or further distribution permitted without the prior written consent of McGraw-Hill Education. Advanced Counting
More information6.854 Advanced Algorithms
6.854 Advanced Algorithms Homework Solutions Hashing Bashing. Solution:. O(log U ) for the first level and for each of the O(n) second level functions, giving a total of O(n log U ) 2. Suppose we are using
More informationCSE613: Parallel Programming, Spring 2012 Date: May 11. Final Exam. ( 11:15 AM 1:45 PM : 150 Minutes )
CSE613: Parallel Programming, Spring 2012 Date: May 11 Final Exam ( 11:15 AM 1:45 PM : 150 Minutes ) This exam will account for either 10% or 20% of your overall grade depending on your relative performance
More informationLecture 1: Asymptotics, Recurrences, Elementary Sorting
Lecture 1: Asymptotics, Recurrences, Elementary Sorting Instructor: Outline 1 Introduction to Asymptotic Analysis Rate of growth of functions Comparing and bounding functions: O, Θ, Ω Specifying running
More informationRadix Sorting With No Extra Space
Radix Sorting With No Extra Space Gianni Franceschini Univ. of Pisa francesc@di.unipi.it S. Muthukrishnan Google Inc., NY muthu@google.com June 27, 2007 Mihai Pǎtraşcu MIT mip@mit.edu Abstract It is well
More informationCS 2210 Discrete Structures Advanced Counting. Fall 2017 Sukumar Ghosh
CS 2210 Discrete Structures Advanced Counting Fall 2017 Sukumar Ghosh Compound Interest A person deposits $10,000 in a savings account that yields 10% interest annually. How much will be there in the account
More informationReductions, Recursion and Divide and Conquer
Chapter 5 Reductions, Recursion and Divide and Conquer CS 473: Fundamental Algorithms, Fall 2011 September 13, 2011 5.1 Reductions and Recursion 5.1.0.1 Reduction Reducing problem A to problem B: (A) Algorithm
More informationCPSC 121: Models of Computation. Module 9: Proof Techniques (part 2) Mathematical Induction
CPSC 121: Models of Computation Module 9: Proof Techniques (part 2) Mathematical Induction Module 9: Announcements Midterm #2: th Monday November 14, 2016 at 17:00 Modules 5 (from multiple quantifiers
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the
More informationCS 161 Summer 2009 Homework #2 Sample Solutions
CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than
More informationAlgorithms Test 1. Question 1. (10 points) for (i = 1; i <= n; i++) { j = 1; while (j < n) {
Question 1. (10 points) for (i = 1; i
More informationCSE 421 Algorithms: Divide and Conquer
CSE 42 Algorithms: Divide and Conquer Larry Ruzzo Thanks to Richard Anderson, Paul Beame, Kevin Wayne for some slides Outline: General Idea algorithm design paradigms: divide and conquer Review of Merge
More informationHomework 7 Solutions, Math 55
Homework 7 Solutions, Math 55 5..36. (a) Since a is a positive integer, a = a 1 + b 0 is a positive integer of the form as + bt for some integers s and t, so a S. Thus S is nonempty. (b) Since S is nonempty,
More information1 Probability Review. CS 124 Section #8 Hashing, Skip Lists 3/20/17. Expectation (weighted average): the expectation of a random quantity X is:
CS 24 Section #8 Hashing, Skip Lists 3/20/7 Probability Review Expectation (weighted average): the expectation of a random quantity X is: x= x P (X = x) For each value x that X can take on, we look at
More information