Why do we need math in a data structures course?

Transcription

1 Math Review 1

2 Why do we need math in a data structures course? To nalyze data structures and algorithms Deriving formulae for time and memory requirements Will the solution scale? Quantify the results Proving algorithm correctness 2

3 Example Consider lgorithm1 that divides the input array in half and calls lgorithm2 on each half lgorithm1 (,n) // is an integer array of size n x floor(n/2) lgorithm2 (,1,x) lgorithm2 (,x+1,n) ssume lgorithm2(,i,j) s running time is (j-i+1) What is the running time of lgorithm1? 3

4 Floors and Ceilings floor(x), denoted x, is the greatest integer x ceiling(x), denoted x, is the smallest integer x Normally used to divide input into integral parts N N N 4

5 Exponents ) ( N N N N N N N B B B B B B X X X X X X X X X X X X

6 Logarithms log log log log log log lg ln X B X B log log B < B log log log log B log X 2 for all ; ; e, B, C B log + log B; B e C C X B X B > "logarithm of > 0, 1, B > 0 B base X" "natural logarithm" Our convention for the course: lg n log 2 n log n log 10 n ln n log e n PS: In Weiss book, log n log 2 n 6

7 What is the meaning of the log function? For example, lg N

8 Example How many times to halve an array of length n until its length is 1? Halve (n) i 0 while n 1 i i + 1 n floor(n/2) return i What will be the value of i? 8

9 Factorials n! n! < n n! 1 if n 0 n ( n 1)! if n > 0 n n 2π n ( n / e) (1 +θ (1/ n)) Stirling's approximation PermutationSort (,n) // is an integer array of length n while is not in order Permute () 9

10 Modular rithmetic mod N ( mod N) " is E.g., 81 If Then + C and D N congruent to B modulo N" 61 1(mod10) B (mod N) / N BD (mod N) ( B mod N) B + C (mod N) B (mod N) Basis of most encryption schemes: (Message mod Key) 10

11 Series General N f ( i) f (0) + f (1) i 0 f ( N) Linearity N ( cf ( i) + g( i)) c i 0 i 0 N f ( i) + N i 0 g( i) rithmetic series N i N( N + 1) i

12 Series Geometric series N i 0 N i 0 i i N i 0 i 1 ; 1 if 0 < < 1 Example How many nodes in a complete binary tree of depth D?

13 Proofs What do we want to prove? Properties of a data structure always hold for all operations lgorithm s running time / memory will never exceed some threshold lgorithm will always be correct lgorithm will always terminate Techniques Proof by induction Proof by counterexample Proof by contradiction 13

14 Proof by Induction Goal: Prove some hypothesis is true Three-step process Variation: Ind/hyp: ll values < k, Ind/step: show for valuek 1. Base case: Show hypothesis is true for some initial conditions 2. Inductive hypothesis: ssume hypothesis is true for all values k 3. Inductive step: Show hypothesis is true for next larger value (typically k+1) 14

15 Inductive Proof: Example Prove arithmetic series N i N( N + 1) i 1 2 Base case: Show true for N1 1 1(1 + 1) i 1 i

16 Example (cont.) Variation: ssume hyp for N<k, and Check for Nk Ind/Hyp: ssume true for all N<k Ind/Step: Now see if it is true for k + 1 k Nk+1 i 1 i ( k + 1) + i i 1 k( k + 1) ( k + 1) + 2 2( k + 1) + k( k + 1) 2 ( k + 1)( k + 2) 2 16

17 More Examples Prove the geometric series N i 0 i N Prove that the number of nodes N in a complete binary tree of depth D is 2 D

18 Proof by Counterexample Prove hypothesis is not true by giving an example that doesn t work Example: 2 N > N 2? Proof: N2 Proof by example? Proof by lots of examples? Proof by all possible examples? Empirical proof Hard when input size and contents can vary arbitrarily 18

19 nother Example of a Counterexample proof 100 D Given N cities and costs for traveling between each pair of cities, find the least-cost tour to visit each city exactly once Hypothesis B C 10 Given a least-cost tour for N cities, the same tour will be least-cost for (N-1) cities E.g., if B C D is the least-cost tour for cities {,B,C,D}, then B C will be the least-cost tour for cities {,B,C} 10 Is this hypothesis true? 19

20 nother Example (cont.) Counterexample Cost ( B C D) 40 (optimal) Cost ( B C) 30 Cost ( C B) 20 Not the least cost tour for {,B,C} B 10 Least cost tour for {,B,C} D C Conclusion: Least cost tours don t necessarily contain smaller least cost tours 20

21 Proof by Contradiction Start by assuming that the hypothesis is false Show this assumption leads to a contradiction (i.e., some know property is violated) Can t use special cases or specific examples Therefore, hypothesis must be true 21

22 Example for proof by contradiction Single pair shortest path problem 100 D Given N cities and costs for traveling between each pair of cities, find the least-cost path to go from city X to city Y Hypothesis B C least-cost path from X to Y contains least-cost paths from X to every city on the path E.g., if X C1 C2 C3 Y is the least-cost path from X to Y, then 10 X C1 C2 C3 is the least-cost path from X to C3 X C1 C2 is the least-cost path from X to C2 X C1 is the least-cost path from X to C1 Conclusion: Least cost paths should contain smaller least cost paths 22

23 P Proof by contradiction.. X C Y P ssume hypothesis is false > Given a least-cost path P from X to Y that goes through C, there is a better path P from X to C than the one in P > But we could replace the subpath from X to C in P with this lesser-cost path P > The path cost from C to Y is the same > Thus we now have a better path from X to Y > But this violates the assumption that P is the least-cost path from X to Y (hence a contradiction!) Therefore, the original hypothesis must be true 23

24 Recursion recursive function is defined in terms of itself 1 if n 0 Example: n! n ( n 1)! if n > 0 Factorial (n) if n 0 then return 1 else return (n * Factorial (n-1)) 24

25 Basic Rules of Recursion Base cases Must always have some base cases, which can be solved without recursion Making progress Recursive calls must always make progress toward a base case Design rule ssume all recursive calls work Compound interest rule Try not to duplicate work by solving the same instance of a problem in separate recursive calls

26 Example Fibonacci numbers F(0) 1 F(1) 1 F(n) F(n-1) + F(n-2) Called a recurrence Fibonacci (n) if (n 1) then return 1 else return (Fibonacci (n-1) + Fibonacci (n-2)) 26

27 Example (cont.) Computation tree for: Fibonacci (5) F(5) F(4) F(3) F(3) F(2) F(2) F(1) F(2) F(1) F(1) F(0) F(1) F(0) F(1) F(0) 27

28 Running time for Fibonacci(n)? Show that the running time T(n) of Fibonacci(n) is exponential in n Use mathematical induction Show T(n) < (5/3) n for n>1 Use induction ctually, only proved that T(n) is no more than exponential Need to also prove T(n) is at least exponential 28

29 Solving recurrences Example: lgo1(,1,n) // is an integer array of size n x floor(n/2) lgo1(,1,x) lgo1(,x+1,n) How much time does lgo1 take? Express time as a function of n (input size) Let T(n) be the time taken by lgo1 on an input size n Then, T(n) 1 + T(n/2) + T(n/2) > T(n) 2T(n/2) + 1

30 Solving recurrences Recurrence: T(n) 2T(n/2) + 1 T(1) 1 Solution: T(n) 2T(n/2) + 1 2[2T(n/2 2 ) + 1] T(n/2 2 ) T(n/2 3 ) + 3 (k steps) 2 k T(n/2 k ) + k For termination, n/2 k 1 klg n T(n) 2 lg n T(1) + lg n n + lg n (base case)

31 Ponder this 1. Do constants matter for asymptotic analysis? 2. Recurrence vs. Recursion - recurrence need not always be implemented using recursion - How?

32 Recursive Function Calls Code structure:? Call tree: M() { () () B() C() } () { } B() { B() } C() { D() E() } D() { } E() { F() E() } F() { } M B C B D E B F E M time Call sequence B B B C D E F E

33 Tail Recursion Iteration time (n) { (n-1) } M Tail recursive code (n) { for(in;i>0;i--) { } } M Iteration Last recursive call replaced with while() or for() loop Iterative code is more desirable than tail recursive code. Why?

34 Tower of Hanoi Goal: Move all disks from peg to peg B using peg C Rules: 1. Move one disk at a time 2. Larger disks cannot be placed above smaller disks B C Invented by a French Mathematician Edouard Lucas, 1883 Question: What is the minimum number of moves necessary to solve the problem?

35 Tower of Hanoi: lgorithm Recursive lgorithm: 1. First, move the top n-1 disks, recursively, from to C (using B) 2. Move n th disk (ie., largest & bottom-most in ) from to B 3. Then, move all the n-1 disks, recursively, from C to B (using ) 2) move(1,,b) 1) move(n-1,,c) B C 3) move(n-1,c,b) Recursive solution

36 Tower of Hanoi: nalysis Let T(n) minimum number of moves required to solve the problem nalysis: T(1)1 Base case T(n) 2.T(n-1)+1 recurrence Solving this yields T(n)2 n -1 (how?) In the original Tower of Hanoi problem, n8 & so T(n)255 (which is fine!) For Tower of Brahma, n moves ssuming each move takes 1 microsecond, this would take 5,000 centuries to complete So lots of time before the world ends!

37 Recursive lgorithm for Tower of Hanoi (pseudocode) src Move (n: disk,, B, C) PRE: n disks on ; B and C unaffected POST: n disks on B; and C unaffected BEGIN IF n0 THEN RETURN Move (n-1,,c,b) Move n th disk from to B directly Move (n-1,c,b,) END Tail Recursion dst temp

38 Summary Floors, ceilings, exponents, logarithms, series, and modular arithmetic Proofs by mathematical induction, counterexample and contradiction Recursion Solving recurrences Tools to help us analyze the performance of our data structures and algorithms 38