CS Algorithms and Complexity

Transcription

1 CS Algorithms and Complexity Recurrence Relation and Mathematical Analysis of Algorithms Sean Anderson 1/16/18 Portland State University

2 Table of contents 1. Non-Recursive Algorithms (Cont.) 2. Recursive Algorithms 3. Intermission 4. Recurrence Relations 5. Reduce-and-Conquer 1

3 Non-Recursive Algorithms (Cont.)

4 Analyzing Non-Recursive Algorithms 1. Measure input size (bit? ints?) 2. Identify basic operation 3. Consider instance classes (need cases?) 4. Make summation or recurrence relation of how often basic operation happens 5. Find closed form or establish order 2

5 While Loops For loops are easy to build summations for (unless we do something weird.) For while loops, we need different methods. Sane for loops terminate; while might not Proof of termination: Loop (in)variants Proof by induction Gödel s Incompleteness Theorem, Halting Problem 3

6 Binary Size Problem: Binary Size Instance: A positive integer n Problem: Find the number of bits in n s binary representation 4

7 Binary Size Solution Binary(n) count 1 while n 1 do: count count + 1 n n / 2 done return count 5

8 Loop Variants We have some tools to help prove termination: Loop Invariant - a logical predicate I that holds after each loop Loop Variant: Let C be the loop condition Let V = z at the beginning of the loop block Prove that, at the end of the block, V < z given all invariants Prove that C V 0 6

9 Binary Loop (In)variant C = n > 0 Obvious invariant: I = C, i.e. n is always positive. Variant 1: V = n; n/2 is always < n if n > 0 When n 0, C is violated, so loop terminates Variant 2: V = n ; when n is halved, n reduced by 1 When n = 0, n = 0, C is violated 7

10 Binary Efficiency n 1 = n = lg n i=1 8

11 Logs Some notes on logarithms: lg = log 2 length of binary representation log = log 10 common log ln = log e natural log 9

12 Decimal Length Imagine an algorithm like Binary(n), but dividing by 10 instead of 2. Is its order smaller? Larger? The same? In other words, is lg n O(log n), Ω(log n), or both? 10

13 Log Conversions Converting bases from a to b: Recall that log b x = log a x log a b 1 log a b is a constant So log b x Θ(log a x) In CS, log is often generic log 11

14 Recursive Algorithms

15 Recursion: the property of an algorithm that uses recursion Like while loops, recursion can get pretty crazy. In fact, any loop can be implemented as recursion and vice versa. 12

16 Binary(n) - Recursive Binary(n) if n = 0: return 0 else: return 1 + Binary(n/2) 13

17 Anatomy of a Recursive Algorithm How does recursion relate to our concept of (in)variants? Base case: solution to simplest case(s) solved by hand General case: solution in terms of simpler case(s) Invariant: prove that if recursive calls are valid solutions, our return is a valid solution Variant: only make recursive calls to smaller/simpler problem instances When might we break these rules? What will happen? 14

18 Collatz Conjecture Conjecture: this algorithm terminates for all n Collatz(n) if even(n): return 1 + Collatz(n/2) else if n > 1: return 1 + Collatz(3*n+1) else: return 0 15

19 Tower of Hanoi Toy Problem: Tower of Hanoi Instance: n disks of unique size stacked on one of 3 posts Problem: moving only one disk at a time, and maintaining the constraint that disks only stack on top of larger disks, move all disks to post 3. 16

20 Animation Tower of Hanoi Animation 17

21 Tower of Hanoi Idea Basic concept: Move the n 1 stack to spare post Move the bottom disk to goal post Move the n 1 stack to goal post 18

22 Intermission

23 Intermission Algorithms Bogobogo Sort To sort a list of n integers: B A[1] for i 2..n: B.append(A[i]) B random_permutation(b) if not sorted(b): A random_permutation(a) restart return B 19

24 Recurrence Relations

25 Core Idea How best to define the behavior of a recursive algorithm? Recursively! T(n) - how often our basic operation happens in an instance of size n, in terms of T(n ) where n < n T(0) or T(1), etc. - how often it happens in the base case 20

26 Examples T(n) = T(n 1) + n T(0) = 0 2T(n 1) + 1 T(1) = 1 T(n) = T(n 1) + T(n 2) T(0) = 1 T(1) = 1 21

27 Forward Substitution T(n) = T(n 1) + n T(0) = 0 22

28 Forward Substitution T(n) = T(n 1) + n T(0) = 0 T(1) =

29 Forward Substitution T(n) = T(n 1) + n T(0) = 0 T(1) = T(2) =

30 Forward Substitution T(n) = T(n 1) + n T(0) = 0 T(1) = T(2) = T(n) = n i O(n 2 ) i=0 25

31 Backward Substitution Expand T(n 1), T(n 2), etc. until a pattern emerges. 26

32 Tower of Hanoi Recurrence T(n) = T(n 1) T(n 1) T(n) = 2T(n 1)

33 Tower of Hanoi Recurrence T(n) = 2(2T(n 2) + 1) + 1 = 4T(n 2) + 3 = 8T(n 3)

34 Tower of Hanoi Recurrence T(n) = 2 i T(n i) + 2 i 1 29

35 Tower of Hanoi Complexity Omitted step: proof by induction. Tower of Hanoi is O(2 n ). Is there a faster way to do this? 30

36 Reduce-and-Conquer

37 Reduce-and-Conquer Also called decrease-and-conquer: a strategy of exploiting the relationship between a problem and smaller instances of the same problem. Every example in this lecture has been an example of reduce-and-conquer. 31

38 Subcategories R-&-C problems come in a few flavors: Reducing by a constant - e.g. Binary (measured by bits), Tower of Hanoi Reducing by a constant factor - e.g. Binary Search Reducing by a variable amount - e.g. Dijkstra s GCD 32

39 Next lecture: sorting with R-&-C (insertion sort) compared to exhaustive (selection sort). 33

40 Master Theorem

41 References i Tower of Hanoi gif By Trixx (I designed this using [GFDL ( or CC BY-SA 3.0 ( via Wikimedia Commons