Lecture 12: Algorithm Analysis

Transcription

1 Lecture 12: Algorithm Analysis CS6507 Python Programming and Data Science Applications Dr Kieran T. Herley Department of Computer Science University College Cork KH (26/02/18) Lecture 12: Algorithm Analysis / 21

2 Summary Insert suitable summary here KH (26/02/18) Lecture 12: Algorithm Analysis / 21

3 Comparing Algorithms Capturing algorithm performance 1 Can evaluate algorithms experimentally But designing experiments can be hard 1 Sources: (L) stopwatch.onlineclock.net, (R) wikipedia.org KH (26/02/18) Lecture 12: Algorithm Analysis / 21

4 Comparing Algorithms Capturing algorithm performance cont d Execution times for IS for sample of problem instances in range 10 to 100. Average execution times for IS and MS for problem instance sizes in range 100 to Precise running time depends on specific problem instance. 2 Running time generally increases with problem size. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

5 Comparing Algorithms Observations 1 Note parabolic upwards tilt to IS curve 2 Insertionsort has an intinsic n 2 aspect to perf. on lists of length n: 1 n 1 iterations, one per list position 2 Iteration i involves walkback through lst[0..i 1] seeking suitable home for element lst[i] potentially work proportional to n KH (26/02/18) Lecture 12: Algorithm Analysis / 21

6 Comparing Algorithms Growth rates Graph shows growth rates of some functions commonly seen in algorithm performance. Note logarithmic axes. For n sufficiently large, 1 log n n n log n n 2 n 3 2 n So n log n performance (MS) is superior to n 2 performance (IS). Growth rate of alg. performance captures how well algorithm scales to larger problem instances. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

7 Comparing Algorithms Drawbacks of experimental evaluation Needs substantial software effort: skilful algorithm implementation experimental apparatus (e.g. to gather timing info.) Need to ensure level playing field across experiments: identical hardware, system load etc. Need to choose test cases with great care. Which to choose and why? How do we know they tell the whole story? KH (26/02/18) Lecture 12: Algorithm Analysis / 21

8 Algorithm analysis An alternative approach: analytical evaluation Evaluate algorithm analytically Count primitive operations Desk-based, paper-and-pencil analysis Seek function e.g. 2n 2 3n + 5 to characterise alg. performance in terms of problem size Focus analysis on certain operations within alg. Analysis as bean-counting exercise: estimate the number of operations entailed in alg. execution. Estimate gives proxy for total volume of computational work during execution. Focus on worst-case performance For each problem size, estimate the greatest number of ops. that might be required. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

9 Algorithm analysis Counting primitive operations Our primitive ops: Assigning an identifier to an object Performing an arithmetic or logical operation Comparing two numbers Accessing an single list element by index Calling a function (excl. operations executed within function) Returning from a function Note that each addition or comparison may be executed many times; also need to capture number of executions (repetitions). KH (26/02/18) Lecture 12: Algorithm Analysis / 21

10 Algorithm analysis Simple Example def find max(data): biggest = data[0] for val in data: if val > biggest: biggest = val return biggest KH (26/02/18) Lecture 12: Algorithm Analysis / 21

11 Algorithm analysis Simple Example cont d def find max(data): biggest = data[0] for val in data: if val > biggest: biggest = val return biggest line #po # reps subtotal n n 5 1 n n n + 3 Upshot: find max has a worst-case performance of 2n + 3 steps. Performance is proportional to list length as you would expect. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

12 Insertion-Sort Revisited Example 2 Insertion-Sort def insertion sort ( lst ): for i in range(1, len( lst )): cur = lst [ i ] j = i 1 while j >= 0 and lst[j] > cur: lst [ j+1] = lst[ j ] j = j 1 lst [ j+1] = cur KH (26/02/18) Lecture 12: Algorithm Analysis / 21

13 Insertion-Sort Revisited Example 2 Insertion-Sort def insertion sort ( lst ): for i in range(1, len( lst )): cur = lst [ i ] j = i 1 while j >= 0 and lst[j] > cur: lst [ j+1] = lst[ j ] j = j 1 lst [ j+1] = cur Nested loops complicate task of estimating no. of execution of assignment on Line 6, say. However, note that for a given i: Focus on control variable j. Inner loop (lines 4 7) has at most i iterations. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

14 Insertion-Sort Revisited Example 2 cont d def insertion sort ( lst ): for i in range(1, len( lst )): cur = lst [ i ] j = i 1 while j >= 0 and lst[j ] > cur: lst [ j+1] = lst[ j ] j = j 1 lst [ j+1] = cur line #po # reps subtotal n 1 2(n 1) 4 2 n 1 2(n 1) 5 4 i i 7 2 i 8 3 n 1 3(n 1) (9) KH (26/02/18) Lecture 12: Algorithm Analysis / 21

15 Insertion-Sort Revisited Example 2 cont d def insertion sort ( lst ): for i in range(1, len( lst )): cur = lst [ i ] j = i 1 while j >= 0 and lst[j ] > cur: lst [ j+1] = lst[ j ] j = j 1 lst [ j+1] = cur line #po # reps subtotal n 1 2(n 1) 4 2 n 1 2(n 1) 5 4 i + 1 n 1 i=1 4(i + 1) 6 4 i n 1 i=1 4i i=1 2i 7 2 i n n 1 3(n 1) (9) KH (26/02/18) Lecture 12: Algorithm Analysis / 21

16 Insertion-Sort Revisited Example 2 cont d def insertion sort ( lst ): for i in range(1, len( lst )): cur = lst [ i ] j = i 1 while j >= 0 and lst[j ] > cur: lst [ j+1] = lst[ j ] j = j 1 lst [ j+1] = cur line #po # reps subtotal n 1 2(n 1) 4 2 n 1 2(n 1) 5 4 i + 1 n 1 i=1 4(i + 1) 6 4 i n 1 i=1 4i i=1 2i 7 2 i n n 1 3(n 1) (9) n 2 6n 10 Upshot: insertion sort has a worst-case performance of 5n 2 6n 10. Observation: the n 2 -ness is the most important feature, the rest is clutter. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

17 Big-Oh notation Big-Oh notation c g(n) f (n) n 0 n Definition Let f (n) and g(n) be functions mapping nonnegative integers to real numbers. We say that f (n) is O(g(n)) if there is a real constant c > 0 and an integer constant n 0 1 such that f (n) cg(n), for n n 0. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

18 Big-Oh notation Example Claim 5n 2 6n 10 O(n 2 ) Proof Here f (n) = 5n 2 6n 10 and g(n) = n 2. Notice 5n 2 6n 10 5n 2 and so condition holds for c = 5 and n 0 = 1. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

19 Big-Oh notation Example Claim 5n 2 6n 10 O(n 2 ) Proof Here f (n) = 5n 2 6n 10 and g(n) = n 2. Notice 5n 2 6n 10 5n 2 and so condition holds for c = 5 and n 0 = 1. Rule of thumb To obtain O() category, identify dominant term delete lower-order additive terms (here 6n and 10) delete leading multiplicative constants (here 5) KH (26/02/18) Lecture 12: Algorithm Analysis / 21

20 Big-Oh notation Observations Set O(n 2 ) contains n 2 itself functions dwarfed by it, e.g. 1, log n, n, n, n log n (plus others) constant multiples of above e.g. 10n log n additive combination of above Relationship among O() sets: O(1) O(log n) O(n) O(n log n) O(n 2 ) O(n 3 ) O(2 n ) in decreasing order of desirability left to right. KH (26/02/18) Lecture 12: Algorithm Analysis / 21

21 Analysis of Merge-Sort Merge-Sort again def merge(s1, s2, s ): i = j = 0 while i + j < len(s ): if j == len(s2) or\\ ( i < len(s1) and s1[ i ] < s2[j ]): s[ i+j] = s1[i ] i = i + 1 else : s[ i+j] = s2[j ] j = j + 1 def merge sort(s ): n = len(s) if n < 2: return mid = n // 2 s1 = s[0:mid] s2 = s[mid:n] merge sort(s1) merge sort(s2) merge(s1, s2, s) Fact: merge sort has a worst-case performance of O(n log n) (compared with O(n 2 ) for IS). KH (26/02/18) Lecture 12: Algorithm Analysis / 21

22 Analysis of Merge-Sort Analysis of MS def merge(s1, s2, s ): i = j = 0 while i + j < len(s): if j == len(s2) or\\ ( i < len(s1) and s1[ i ] < s2[j ]): s[ i+j] = s1[i ] i = i + 1 else : s[ i+j] = s2[j ] j = j + 1 At most s 1 + s 2 iterations (number of s1, s2 elements) At most O(1) steps per iteration Execution time is O( s 1 + s 2 ). KH (26/02/18) Lecture 12: Algorithm Analysis / 21

23 Analysis of Merge-Sort Analysis cont d def merge sort(s ): n = len(s) if n < 2: return mid = n // 2 s1 = s[0:mid] s2 = s[mid:n] merge sort(s1) merge sort(s2) merge(s1, s2, s) Each execution entails: O( s ) steps for splitting (assume) O( s ) steps for merge (steps inside recusive calls accounted for separately) Recursion involves 1 call (level zero), 2 calls (level one), 4 calls (level two),... size of list s halves at each level (n, n/2, n/2 2, ) KH (26/02/18) Lecture 12: Algorithm Analysis / 21

24 Analysis of Merge-Sort Analysis cont d n 2 2 n 2 n + n n n n n n 2 k + 2 k k + 2 k = O k=lg n (n) ( k ) ( k ) = O 2 i n = O(n lg n) 2 i O n = O(k n) i=0 i=0 n 2 = n 2 2 = = O 0 (n) + O 1 (n) + O 2 (n) + KH (26/02/18) Lecture 12: Algorithm Analysis / 21

25 Back Material Notes and Acknowledgements Reading Code Acknowledgements The material in this lecture is based largely on the treatment of GTG Chapter 3. The Merge-Sort tree is based on one taken from KH (26/02/18) Lecture 12: Algorithm Analysis / 21