Introduction to Algorithmic Complexity. D. Thiebaut CSC212 Fall 2014

Transcription

1 Introduction to Algorithmic Complexity D. Thiebaut CSC212 Fall 2014

2 Case Study: Fibonacci

3 public class RecurseFib {! private static long computefibrecursively( int n ) { if ( n<= 1 ) return 1; return computefibrecursively( n-1 ) + computefibrecursively( n-2 ); }! private static long computefibiteratively( int n ) { if ( n<=1 ) return 1; long[] fibs = new long[ n+1 ]; fibs[0] = fibs[1] = 1; for ( int i=2; i<=n; i++ ) fibs[i] = fibs[i-1]+fibs[i-2]; return fibs[ fibs.length - 1 ]; } public static void main(string[] args) { } } for ( int N = 0; N < 100; N++ ) { long start = System.currentTimeMillis(); long Fib = computefibrecursively( N ); //long Fib = computefibiteratively( N ); long end = System.currentTimeMillis(); System.out.println( String.format( "Fib(%d) = %d, %1.2f msec", N, Fib, 1.0*(end-start) ) ); }

4 Fib(0) = 1, 0.00 msec Fib(1) = 1, 0.00 msec Fib(2) = 2, 0.00 msec Fib(3) = 3, 0.00 msec Fib(4) = 5, 0.00 msec Fib(5) = 8, 0.00 msec Fib(6) = 13, 0.00 msec Fib(7) = 21, 0.00 msec Fib(8) = 34, 0.00 msec Fib(9) = 55, 0.00 msec Fib(10) = 89, 0.00 msec Fib(11) = 144, 0.00 msec Fib(12) = 233, 0.00 msec Fib(13) = 377, 0.00 msec Fib(14) = 610, 0.00 msec Fib(15) = 987, 0.00 msec Fib(16) = 1597, 1.00 msec Fib(17) = 2584, 0.00 msec Fib(18) = 4181, 1.00 msec Fib(19) = 6765, 1.00 msec Fib(20) = 10946, 1.00 msec Fib(21) = 17711, 0.00 msec Fib(22) = 28657, 0.00 msec Fib(23) = 46368, 0.00 msec Fib(24) = 75025, 0.00 msec Fib(25) = , 0.00 msec Fib(26) = , 0.00 msec Fib(27) = , 1.00 msec Fib(28) = , 1.00 msec Fib(29) = , 2.00 msec Fib(30) = , 3.00 msec Fib(31) = , 5.00 msec Fib(32) = , 8.00 msec Fib(33) = , msec Fib(34) = , msec Fib(35) = , msec Fib(36) = , msec Fib(37) = , msec Fib(38) = , msec Fib(39) = , msec Fib(40) = , msec Fib(41) = , msec Fib(42) = , msec Fib(43) = , msec Fib(44) = , msec Fib(45) = , msec computefibrecursively()

5 time (ms) computefibrecursively() Nth Fibonacci Term

6 Fib(0) = 1, 0.00 msec Fib(1) = 1, 0.00 msec Fib(2) = 2, 0.00 msec Fib(3) = 3, 0.00 msec Fib(4) = 5, 0.00 msec Fib(5) = 8, 0.00 msec Fib(6) = 13, 0.00 msec Fib(7) = 21, 0.00 msec Fib(8) = 34, 0.00 msec Fib(9) = 55, 0.00 msec Fib(10) = 89, 0.00 msec Fib(11) = 144, 0.00 msec Fib(12) = 233, 0.00 msec Fib(13) = 377, 0.00 msec Fib(14) = 610, 0.00 msec Fib(15) = 987, 0.00 msec Fib(16) = 1597, 0.00 msec Fib(17) = 2584, 0.00 msec Fib(18) = 4181, 0.00 msec Fib(19) = 6765, 0.00 msec Fib(20) = 10946, 0.00 msec Fib(21) = 17711, 0.00 msec Fib(22) = 28657, 0.00 msec Fib(23) = 46368, 0.00 msec Fib(24) = 75025, 0.00 msec Fib(25) = , 0.00 msec Fib(26) = , 0.00 msec Fib(27) = , 0.00 msec Fib(28) = , 0.00 msec Fib(29) = , 0.00 msec Fib(30) = , 0.00 msec Fib(31) = , 0.00 msec Fib(32) = , 0.00 msec Fib(33) = , 0.00 msec Fib(34) = , 0.00 msec Fib(35) = , 0.00 msec Fib(36) = , 0.00 msec Fib(37) = , 0.00 msec Fib(38) = , 0.00 msec Fib(39) = , 0.00 msec Fib(40) = , 0.00 msec Fib(41) = , 0.00 msec Fib(42) = , 0.00 msec Fib(43) = , 0.00 msec Fib(44) = , 0.00 msec Fib(45) = , 0.00 msec Fib(46) = , 0.00 msec Fib(47) = , 0.00 msec Fib(48) = , 0.00 msec Fib(49) = , 0.00 msec Fib(50) = , 0.00 msec computefibiteratively() (Using System.currentTimeMillis() to measure elapsed time) Fib(90) = , 0.00 msec Fib(91) = , 0.00 msec Fib(92) = , 0.00 msec Fib(93) = , 0.00 msec Fib(94) = , 0.00 msec

7 Fib(0) = 1, 0.00 msec Fib(1) = 1, 0.00 msec Fib(2) = 2, 0.00 msec Fib(3) = 3, 0.00 msec Fib(4) = 5, 0.00 msec Fib(5) = 8, 0.00 msec Fib(6) = 13, 0.00 msec Fib(7) = 21, 0.00 msec Fib(8) = 34, 0.00 msec Fib(9) = 55, 0.00 msec Fib(10) = 89, 0.00 msec Fib(11) = 144, 0.00 msec Fib(12) = 233, 0.00 msec Fib(13) = 377, 0.00 msec Fib(14) = 610, 0.00 msec Fib(15) = 987, 0.00 msec Fib(16) = 1597, 0.00 msec Fib(17) = 2584, 0.00 msec Fib(18) = 4181, 0.00 msec Fib(19) = 6765, 0.00 msec Fib(20) = 10946, 0.00 msec Fib(21) = 17711, 0.00 msec Fib(22) = 28657, 0.00 msec Fib(23) = 46368, 0.00 msec Fib(24) = 75025, 0.00 msec Fib(25) = , 0.00 msec Fib(26) = , 0.00 msec Fib(27) = , 0.00 msec Fib(28) = , 0.00 msec Fib(29) = , 0.00 msec Fib(30) = , 0.00 msec Fib(31) = , 0.00 msec Fib(32) = , 0.00 msec Fib(33) = , 0.00 msec Fib(34) = , 0.00 msec Fib(35) = , 0.00 msec Fib(36) = , 0.00 msec Fib(37) = , 0.00 msec Fib(38) = , 0.00 msec Fib(39) = , 0.00 msec Fib(40) = , 0.00 msec Fib(41) = , 0.00 msec Fib(42) = , 0.00 msec Fib(43) = , 0.00 msec Fib(44) = , 0.00 msec Fib(45) = , 0.00 msec Fib(46) = , 0.00 msec Fib(47) = , 0.00 msec Fib(48) = , 0.00 msec Fib(49) = , 0.00 msec Fib(50) = , 0.00 msec computefibiteratively() (Using System.currentTimeMillis() to measure elapsed time) Fib(90) = , 0.00 msec Fib(91) = , 0.00 msec Fib(92) = , 0.00 msec Fib(93) = , 0.00 msec Fib(94) = , 0.00 msec

8 time (us) computefibiteratively() Nth Fibonacci Term Fib(1) = 1, usec Fib(2) = 2, usec Fib(4) = 5, usec Fib(8) = 34, usec Fib(16) = 1597, usec Fib(32) = , usec Fib(64) = , usec Fib(128) = , usec Fib(256) = , usec Fib(512) = , usec Fib(1024) = , usec Fib(2048) = , usec Fib(4096) = , usec Fib(8192) = , usec Fib(16384) = , usec Fib(32768) = , usec Fib(65536) = , usec computefibiteratively() (Using System.nanoTime() to measure elapsed time)

9 import java.math.biginteger;! public class RecurseFibBigInteger {! private static BigInteger computefibiteratively( int n ) { if ( n<=1 ) return BigInteger.valueOf( 1 );!! } BigInteger[] fibs = new BigInteger[ n+1 ]; fibs[0] = fibs[1] = BigInteger.valueOf( 1 ); for ( int i=2; i<=n; i++ ) fibs[i] = fibs[i-1].add( fibs[i-2] ); return fibs[ fibs.length - 1 ]; private static String format( BigInteger x ) { String s = x.tostring(); if ( s.length() > 40 ) { return s.substring(0, 10) + String.format( "...(%d digits)...", ( s.length()-20 ) ) + s.substring( s.length()-10, s.length()-1 ); } else return s; } public static void main(string[] args) { If You Are Curious } } for ( int N = 1; N < ; N = N*2 ) { long start = System.nanoTime(); //long Fib = computefibrecursively( N ); BigInteger Fib = computefibiteratively( N ); long end = System.nanoTime(); System.out.println( String.format( "Fib(%d) = %s, %1.3f usec", N, format( Fib ), (end-start)/1000.0f ) ); }

10

11

12 Moral of the Story:! "All Algorithms are Equal! But Some are more Equal! than Others."

13 We need to calculate the number of operations performed by our programs

14 Statement Examples Cost (time) Declaration int i; constant Assignment Arithmetic i = 3; list = new ; i + j i++ constant constant Test if ( a < 10 ) constant Loop for (int i=0; i<n; i++ ) { // body } Tdecl = N*T N*T Function Call a = myfunc( ) constant + T

15 Statement Examples Cost (time) Passing Parameters myfunc( table, stack); constant Instantiating! Object Stack s = new Stack(); It depends on the constructor Declaring! an Array int[] a = new int[1000]; constant Accessing! ith element! of array x = Fib[i-1]; constant

16 An Example

17 Program 1: private static long computefibiteratively( int n ) { 2: if ( n<=1 ) 3: return 1; 4: long[] fibs = new long[ n+1 ]; 5: fibs[0] = fibs[1] = 1; 6: for ( int i=2; i<=n; i++ ) 7: fibs[i] = fibs[i-1]+fibs[i-2]; 8: return fibs[ fibs.length - 1 ]; 9:} Time 1: c1 2: c2 3: c3 4: c4+c5 5: c6+c7+c8+c9 6: c10+(n-2)*(c11) 7: (n-2)*(c12) 8: c13 9: 0

18 Program 1: private static long computefibiteratively( int n ) { 2: if ( n<=1 ) 3: return 1; 4: long[] fibs = new long[ n+1 ]; 5: fibs[0] = fibs[1] = 1; 6: for ( int i=2; i<=n; i++ ) 7: fibs[i] = fibs[i-1]+fibs[i-2]; 8: return fibs[ fibs.length - 1 ]; 9:} Time 1: c1 2: c2 3: c3 4: c4+c5 5: c6+c7+c8+c9 6: c10+(n-2)*(c11) 7: (n-2)*(c12) 8: c13 9: 0 Total Time = Σ(ci) + (n-2)*σ(cj) = C1 * n + C2

19 time (us) Execution Time proportional to n! We knew that! computefibiteratively() Nth Fibonacci Term

20 Dropping the Constants

21 Assume Algorithm with Following Time Complexity T(n) = 3 n^ n

22 Dropping the Constants T(n) = 3 n^ n T(n) = 3 n^ n

23 Linear Scale

24 Log Scale } }

25 Why is 2^n Growth Rate Bad?

26 T(n) = Something that grows like n^2

27 Introducing BIG-O

28 Definition f(n) is O(g(n)) if there exist positive numbers c and N such that! f(n) <= cg(n) for all n >= N

29 Is T(n)=3n^ n + 1E6 O(n^2)?

30 But 3n^ n + 1E6 is O(n^3) too!

31 f(n) = O( n^2) y=c.n^2 Range of Function Growth if algorithm has O(n^2) time complexity, it will not take! more than c * n^2 time units to execute when the problem is big enough.

32 Introducing BIG-Ω (big Omega)

33 Definition f(n) is Ω(g(n)) if there exist positive numbers c and N such that! f(n) >= cg(n) for all n >= N

34 Is T(n)=3n^ n + 1E6 Ω(n^2)?

35 But 3n^ n + 1E6 is also Ω(n)

36 f(n) = Ω( n^2) y=c.n^2 Range of Function Growth if algorithm has Ω(n^2) time complexity, it will not take! less than c * n^2 time units to execute when the problem is big enough.

37 Ω+O = Θ

38 Definition f(n) is Θ(g(n)) if there exists positive numbers c1, c2, and N such that! c1g(n) >= f(n) >= c2g(n) for all n >= N

39

40 Conclusion: T(n)=3n^ n + 1E6 is O(n^2), is Ω(n^2), and therefor, is Θ(n^2)

41 Worst, Average,and Best Cases

42 List Question 1: What are the time complexities of index() in the Best, Average, and Worst Cases?! Question 2: For Successful Searches?! Question 3: For Unsuccessful Searches?

43 Time Complexity Vector LinkedList Doubly! LinkedList ArrayList! (based on Vector) AddFront AddTail RemoveFront RemoveTail Index InsertAt