Fall Lecture 5

Transcription

1 Fall 2018 Lecture 5

2 Today Work sequential runtime Recurrences exact and approximate solutions Improving efficiency program recurrence work

3 asymptotic Want the runtime of evaluating f(n), for large n independent of architecture basic ops take constant time We will give a big-o classification f(n) is O(g(n)) if there are N and c such that n N, f(n) c g(n)

4

5 motivation Why take exponential time when we can take quadratic time?

6 asymptotic Ignore additive constants n is O(n 5 ) Absorb multiplicative constants n 5 is O(n 5 ) Be as accurate as you can O(n 2 ) O(n 3 ) O(n 4 ) Common terminology logarithmic, linear, polynomial, exponential

7 work W(e), the work of e, is the time to evaluate e sequentially, on a single processor each basic operation is constant-time work = total number of operations Often we have a function f and a notion of size for argument values, and want Wf(n), the work of f(v) when v has size n May want either exact or asymptotic estimate

8 work and evaluation Evaluation steps e => e represent basic ops, so the work for e is the number of steps If e => (k) v then W(e) = k (2+2)+(2+2) => 4+(2+2) => 4+4 => 8 W((2+2) + (2+2)) = 3 W(e1+e2) = W(e1) + W(e2) + 1

9 work and application If f is a function value and e => (k) v then W(f e) = k + W(f v) (fn x => x+x) (2+2) => (fn x => x+x) 4 W((fn x => x+x) (2+2)) = 1 + W((fn x => x+x) 4) => => = W(4+4) = 3

10 recurrences Given a recursive function f and a non-negative size that decreases in every recursive call Extract a recurrence relation for the applicative work of f Wf (n) = work of f v on values v of size n Idea: express Wf(n) in terms of Wf(m), 0 m < n Q: When can this method succeed? A: If the work of f v depends only on the size of v (!)

11 recurrences fun f(x) = if x=0 then 1 else x + f(x-1) if x 0, argument value is non-negative, decreases Wf(n) = if n=0 then k1 else k2 + Wf(n-1) Wf(0) = k1 Wf(n) = k2 + Wf(n-1) for n>0 where k1, k2 are constants Wf(n) = k1 + n k2 for all n 0

12 example fun Fib(0) = 1 Fib(1) = 1 Fib(n) = Fib(n-1) + Fib(n-2) A recurrence for the work to evaluate Fib(n) WFib(0) = c0 WFib(1) = c0 WFib(n) = WFib(n-1) + WFib(n-2) + c1 for some constants c0, c1

13 finding solutions Try to find a closed form solution for W(n) (usually, by guessing and induction) OR Code the recurrence in ML, test for small n, look for a common pattern OR Find solution to a simplified recurrence with the same asymptotic properties OR Appeal to table of standard recurrences

14 exp fun exp (n:int):int = if n=0 then 1 else 2 * exp (n-1) Let M be (fn n => if n=0 then 1 else 2 * exp(n-1)) exp 4 => (1) M 4 => (5) 2 * (M 3) M 4 => if 4=0 then => if false then => 2 * exp (4-1) => 2 * M (4-1) => 2 * M 3 => (5) 2 * (2 * (M 2)) => (5) 2 * (2 * (2 * (M 1))) M 3 => (5) 2 * M 2 => (5) 2 * (2 * (2 * (2 * (M 0))) => (3) 2 * (2 * (2 * (2 * 1))) => (4) 16 exp 4 => (28) 16

15 exp It s not hard to prove that for all n 0, exp n => (5n+8) k, where k is the numeral for 2 n But do we need to be so accurate? And does 5n+8 tell us about actual runtime in milliseconds? No! But it does tell us runtime is linear.

16 big-o It s useful to classify runtimes asymptotically This abstracts away from additive and multiplicative constants (which may be machine-dependent, so not very significant) And ignores runtime on small inputs (which may be special-cased in the code, so don t imply much) For f, g : int -> int we say that f is O(g) if there is a constant c and an integer N such that for all n N, f(n) c * g(n). (usually f(n) and g(n) are always positive so we omit the - symbols)

17 exp fun exp (n:int):int = if n=0 then 1 else 2 * exp (n-1) Let Wexp(n) be the runtime for exp(n) Wexp(0) = c0 Wexp(n) = Wexp(n-1) + c1 for n>0 for some constants c0 and c1 c0 cost of n=0 c1 cost of n=0, n-1, mult by 2

18 solution Can prove by induction on n that Wexp(n) = c0 + n c1 for n 0 the work of exp(n) is linear in n

19 classification Wexp(n) = c0 + n c1 Wexp(n) is O(n) O-class for Wexp(n) is independent of c0, c1 Let N=42, c = max(c0,c1)+1. For all n N, Wexp(n) c n (would also work with N=1) (would also work with an even bigger c)

20 summary We ve shown that for n 0, exp n computes the value of 2 n in O(n) steps This fact is independent of machine details (provided basic operations are constant time) Can we do better?

21 faster exp? The definition of exp relies on the fact that 2 n = 2 (2 n-1 ) Everybody knows that 2 n = (2 n div 2 ) 2 if n is even

22 fastexp fun square(x:int):int = x * x fun fastexp (n:int):int = if n=0 then 1 else if n mod 2 = 0 then square(fastexp (n div 2)) else 2 * fastexp(n-1) fastexp 4 = square(fastexp 2) = square(square (fastexp 1)) = square(square (2 * fastexp 0)) = square(square (2 * 1)) = square 4 =16

23 is it faster? fun fastexp (n:int):int = if n=0 then 1 else if n mod 2 = 0 then square(fastexp (n div 2)) else 2 * fastexp(n-1) Let Wfastexp(n) be the work for fastexp(n) Wfastexp(0) = k0 Wfastexp(n) = Wfastexp(n div 2) + k1 for n>0, even Wfastexp(n) = Wfastexp(n-1) + k2 for n>0, odd for some constants k0, k1, k2

24 is it faster? fun fastexp (n:int):int = if n=0 then 1 else if n mod 2 = 0 then square(fastexp (n div 2)) else 2 * fastexp(n-1) Let Wfastexp(n) be the work for fastexp(n) Wfastexp(0) = c0 Wfastexp(1) = c1 Wfastexp(n) = Wfastexp(n div 2) + c2 Wfastexp(n) = Wfastexp(n div 2) + c3 for n>1, even for n>1, odd for some constants c0, c1, c2, c3

25 solution? Not so obvious how to solve for Wfastexp(n) A closed form would involve c0, c1, c2, c3 But we only care about asymptotic behavior So we can work with a simpler recurrence that has the same asymptotic properties simplification: choose each constant to be 1

26 Let Tfastexp(n) be given by simplified recurrence Tfastexp(0) = 1 Tfastexp(1) = 1 Tfastexp(n) = Tfastexp(n div 2) + 1 for n>1 Wfastexp(n) and Tfastexp(n) are asymptotically equivalent (belong to the same big-o class)

27 solution For n>1, Tfastexp(n) is defined like log(n) fun log n = if n=1 then 0 else log(n div 2) + 1 We know that log(n) = log2(n) for all n>0 Can show that there is a constant c such that Tfastexp(n) c log2(n) for all large enough n

28 classification Tfastexp(n) is O(log2 n) Wfastexp(n) depends on c0, c1, c2, c3 We can find constants clow and chigh such that clow Tfastexp(n) Wfastexp(n) chigh Tfastexp(n) and this implies that Wfastexp(n) is also O(log2(n))

29 really, faster Work of exp(n) is O(n) Work of fastexp(n) is O(log n) O(log n) is a proper subset of O(n) fastexp is asymptotically faster than exp

30 even faster? The definition of fastexp relies on 2 n = (2 n div 2 ) 2 2 n = 2 (2 n-1 ) if n is even if n is odd A moment s thought tells us that 2 n = 2 (2 (n div 2) ) 2 if n is odd

31 pow fun pow (n:int):int = case n of 0 => 1 1 => 2 _ => let val k = pow(n div 2) in if n mod 2 = 0 then k*k else 2*k*k end

32 work of pow(n) Wpow(0) = c0 Wpow(1) = c1 Wpow(n) = c2 + Wpow(n div 2) for n>1 Same recurrence as Wfastexp Same asymptotic behavior pow(n) is O(log n)

33 comparison fastexp(n) and pow(n) have O(log n) work. For n 0, fastexp(n) = pow(n). For n<0, fastexp(n) and pow(n) fail to terminate. So fastexp and pow are extensionally equivalent and have the same asymptotic work classification.

34 badpow fun badpow (n:int):int = case n of 0 => 1 1 => 2 _ => let val k2 = badpow(n div 2)*badpow(n div 2) in if n mod 2 = 0 then k2 else 2*k2 end

35 work of badpow(n) Wbadpow(0) = c0 Wbadpow(1) = c1 Wbadpow(n) = c2 + 2 Wbadpow(n div 2) for n>1 Same asymptotic class as Tbadpow(0) = 1 Tbadpow(1) = 1 Tbadpow(n) = Tbadpow(n div 2) for n>1

36 examples Tbadpow(2 0 ) = 1 Tbadpow(2 1 ) = 1 + 2*Tbadpow(2 0 ) = 1 + 2*1 = 3 Tbadpow(2 2 ) = 1 + 2*Tbadpow(2 1 ) = 1 + 2*3 = 7 Tbadpow(2 m ) = 2 m+1-1

37 analysis Tbadpow(2 m ) is O(2 m ) Wbadpow(2 m ) is O(2 m ) This implies that Wbadpow(n) is O(n) Wpow(n) is O(log n) O(log n) O(n) pow is asymptotically faster than badpow

38 list reversal fun rev [ ] = [ ] rev (x::l) = (rev [x] For list values A and B, W@(A, B) is linear in the length of A For all L, length(rev L)= length(l) Runtime of rev(l) depends only on length of L

39 work of rev fun rev [ ] = [ ] rev (x::l) = (rev [x] Wrev(n) = work to reverse a list of length n Wrev(0) = 1 Wrev(n) = Wrev(n-1) + (n-1) + 1

40 solution Wrev(n) = n + Wrev(n-1) = n + (n-1) + Wrev(n-2) = n + (n-1) Wrev(0) = 1/2 n(n+1) + 1 Wrev(n) is O(n 2 ) quadratic runtime

41 faster rev Use an extra argument to accumulate the reversed list revver : int list * int list -> int list Instead of append after the recursive call, do a cons before the recursive call fun revver([ ], A) = A revver(x::l, A) = revver(l, x::a)

42 faster rev fun revver([ ], A) = A revver(x::l, A) = revver(l, x::a) fun Rev L = revver(l, [ ]) For all L, A, revver(l, A) = (rev A For all L, Rev L = rev L

43 analysis Explain why Wrevver(n) is O(n)

44 standard results T(n) = c + T(n-1) O(n) T(n) = c + n + T(n-1) O(n 2 ) T(n) = c + T(n div 2) O(log n) T(n) = c + 2 T(n div 2) O(n) T(n) = c + k T(n-1) O(k n )