Design and Analysis of Algorithms PART II-- Median & Runtime Analysis Recorded by Chandrasekar Vijayarenu and Shridharan Muthu.

Transcription

1 Design and Analysis of Algorithms PART II-- Median & Runtime Analysis Recorded by Chandrasekar Vijayarenu and Shridharan Muthu. Medians and Order Statistics Minimum and maximum How many comparisons are necessary to determine the minimum of a set of n elements? We can easily obtain an upper bound of n 1 comparisons: examine each element of the set in turn and keep track of the smallest element seen so far. In the following procedure, we assume that the set resides in array A, where length[a] = n. MINIMUM(A) 1 min A[1] 2 for i 2 to length[a] 3 do if min > A[i ] 4 then min A[i ] 5 return min Date Jan To get the minimum or maximum the operation is linear and hence the order for min and max is O(n). Mean Mean is the average of all the numbers.

2 µ = avg(a) Median Median is exactly the middle off all numbers. This function is more stable and robust since there is no square factor compared to mean Algorithms for finding the median. 1. Simple a. Sort A[] Order for sort is O(n log n) b. Median = A[ size / 2] 2. Randamized algorithm a 1, a 2. a n a. step 1: Order (form groups) b. step 2: Find median in each group c. step 3: find median of the median (m 1, m 2, m 3,..m 5 ) = M d. step 4: Do partition of A by M L R If L = n/2 1 then k = n/2 and the middle term is the median. If L <= n/2 1 then median (R, k L ) If L >= n/2 then median (L, k) Example:

3 Consider A[] = { } Form a group of 5 numbers Find the median for each row Find the median of the medians Median of = 19 Therefore M = 19 Do partition of A[] by M

4 Since M = 19 is the middle term, the median is 19. This example shows that if there are k elements=pivot, these k elements do not necessarily locate together in the final results.

5 Date Feb 4 th 2008 Run time analysis of Quick Sort: T(n) = T(n/10) + T(9/10n) + cn = O(n log 2 n) Average Case: The split up of sub arrays in the average case would be 1/10n 9/10n Best Case: (LUCKY) For the best case, the array will be split exactly in the middle i.e. pivot element will be exactly in the middle of the array. Runtime Analysis of Median Find: Let s take an example Here the entire array is being split into data chunks of size 5 i.e. n/

6 After sorting each and every column, we will get In the above figure, the middle row which is circled is the median of the corresponding columns. Now we need to find the median of medians. So the middle row needs to be sorted. After sorting the middle row, the middle element will be the median of medians is the median of medians. In the above figure the column 3 and column 4 are swapped. Generally the partition will be like L 18 R 3n/10 <= L 3n/10 <= R < 7n/10 n/ The actual array is split into 10 half lines which are shown in the boxes (above diag). There are exactly 3n/10 elements in both the boxes. The black box has all the elements which are definitely lesser than the median of medians. All the elements which are in the brown box have all the elements which are greater than median of medians. T (n) <= T (n/5) + T (7n/10) + cn We are going to prove two recurrence relations in this class 1. For Quick sort

7 T (n) = T (1/10n) + T (9/10n) + cn We need to prove this is of order O (n logn) 2. For Median finding T (n) = T (1/5n) + T (7/10n) + cn We need to prove this is of order O (n) Recurrence 1(Quick Sort) T (n) = T (1/10n) + T (9/10n) + cn = T ((1/10) 2 n) + T (9/10 * 1/10 n) + cn/10 + T (1/10 * 9/10 n) + T ((9/10) 2 n) + 9cn/10 + cn = T ((1/10) 2 n) + 2 T (9/10 * 1/10 n) + T ((9/10) 2 n) + 2cn Generalizing the above equation we get T((1/10) k n) + ( K 1) T((1/10) k 1 9/10 n) + ( K 2) T((1/10) k 2 (9/10) 2 n)+..+ ( K 1) T((1/10) 2 ( 9/10) k 1 n) + ( K 0) T( (9/10) k n) + kcn The above circled factor is the largest factor in the above equation because of the following reasons 1. ( K 0) will give the largest number. 2. (9/10) k will be the largest factor in the above equation. The smallest factor is (1/10) k. We need to deduce the above equation to Closed Form i.e. to T(1) Therefore (9/10) k n = 1 n = (10/9) k log n = k log(10/9) k = (log n)/log (10/9) substituting value of k and taking the largest factor outside in the above equation, we get T (n) <= T ((9/10) k n) [ 1+ ( K 1) + ( K 2)+ + ( K 1)+1 ] + kcn <= T ((9/10) k n) [ 2 k ] + kcn (using Binomial Theorem) <= T (1) 2 (logn/log(10/9)) + cn logn/log(10/9) <= T (1)[2 logn ] 1/log(10/9) + cn logn/log(10/9) <= T (1) n 1/log(10/9) + + cn logn/log(10/9)

8 = O (nlog 2 n) Hence T (n) = O (nlog 2 n) Recurrence 2 (Median Finding) T (n) = T (1/5n) + T (7/10 n) + cn = T ((1/5) 2 n) + T (1/5 * 7/10 n) + cn/5 + T (7/10 * 1/5 n) + T ((7/10) 2 n) + 7cn/10 + cn = T ((1/5) 2 n) + 2 T (1/5 * 7/10 n) + T ((7/10) 2 n) + (9/10+1) cn = T((1/5) k n) + ( K 1) T((1/5) k 1 7/10 n) + ( K 2) T((1/5) k 2 (7/10) 2 n)+..+ ( K 1) T((1/5) 2 ( 7/10) k 1 n) + ( K 0) T( (7/10) k n) + [(9/10) k 1 + (9/10) k ]cn K terms = [ (1 (9/10) k+1 )/(1 (9/10))] cn To deduce the above equation to Closed Form, we set (7/10) k n = 1 n = (10/7) k Taking log on both sides log n = k log(10/7) k = log n/ log(10/7) Substituting the value of k in the above equation we get T (n) <= T (1) n 1/log(10/7) + (1 (9/10) (logn/log(10/7) + 1) / (1 (9/10)) * cn 9/10 is less than 1 and when the value of logn/log(10/7) grows higher and higher, the total value grows lesser and lesser. So it can be neglected. Therefore T (n) = O (1/(1 (9/10) * n) = O (10 n) = O (n)

9 Normally we partition the array by 5 i.e. n/5 What if we partition the array by 3 i.e. n/3 (sorting will be easier for 3 elements) n/3 X X X X M X X X X There are totally 6 half lines in the above diagram. 2/6 n > guaranteed lower elements than Y. The other half is 1 2/6n = 4/6n = 2/3n So the run time analysis T (n) <= T (n/3) + T (2n/3) + O (n) Adding the constants n/3 + 2n/3 = n i.e. non decreasing function Theoretically for the group of 3, the order is O (n logn) What is we partition by 7 i.e. n/7 n/7 X X X M X X X There are totally 14 half lines and out of which, 4 are guaranteed. 4/14 n = 1/7 n The other half is 1 4/14 n = 10/14 n = 5/7 n T(n) <= T (n/7) + T (5/7n) + cn

10 Theoretically n/7 + 5/7n = 6/7n which is < n. So it is a decreasing function and the order is O (n) So the recurrence relations are T g=7 (n) <= T (1) n 1/log (7/5) + Cn/(1 (6/7)) <= T (1) n 1/log (7/5) + 7Cn T g=5 (n) <= T (1) n 1/log (10/7) + Cn/(1 (9/10)) <= T (1) n 1/log (10/7) + 10Cn