Algorithm. Executing the Max algorithm. Algorithm and Growth of Functions Benchaporn Jantarakongkul. (algorithm) ก ก. : ก {a i }=a 1,,a n a i N,

Transcription

1 Algorithm and Growth of Functions Benchaporn Jantarakongkul 1 Algorithm (algorithm) ก ก ก ก ก : ก {a i }=a 1,,a n a i N, ก ก : 1. ก v ( v ก ก ก ก ) ก ก a 1 2. ก a i 3. a i >v, ก v ก a i ก ก ก v ก ก 2 Executing the Max algorithm ก ก ก ก ก {a i }=7,12,3,15,8 ก ก v = a 1 = 7 ก : a 2 = 12 a 2 >v?, ก v 12 ก : a 3 = 3 3>12?, v. 15>12?, ก v=15 ก 3 1

2 Algorithm Characteristics Some important general features of algorithms: Input. Information or data that comes in. Output. Information or data that goes out. Definiteness. Algorithm is precisely defined. Correctness. Outputs correctly relate to inputs. Finiteness. Won t take forever to describe or run. Effectiveness. Individual steps are all do-able. Generality. Works for many possible inputs. Efficiency. Takes little time & memory to run. 4 Pseudocode Language procedure name(argument: type) variable:= expression {informal statement} begin statements end {comment} if condition then statement [else statement] for variable:= initial value to final value statement while condition statement return expression 5 Max procedure in pseudocode procedure max(a 1, a 2,, a n : integers) v:= a 1 {ก v ก ก } for i:= 2 to n { ก } if a i > v then v:= a i { กก v?} { ก v ก ก } return v 6 2

3 Example task ก ก (List) ก ก L ก n ก ก ก ก x ก x ก ก x ก (index) x ก ก ก 7 Search alg. #1: Linear Search procedure linear search (x: integer, a 1, a 2,, a n : distinct integers) i:= 1 { ก ก } while (i n x a i ) { } i:= i + 1 { } if i n then location:= i { ก } else location:= 0 { ก } return location { ก ก 0} 8 Search alg. #2: Binary Search : ก ก ก ก ก ก ก กก ( ก ก ) <x <x <x >x 9 3

4 Search alg. #2: Binary Search procedure binary search (x:integer, a 1, a 2,, a n : distinct integers) i:= 1 {ก ก } j:= n {ก ก } while i<j begin { ก ก >1 } m:= (i+j)/2 { ก } if x>a m then i := m+1 else j := m end if x = a i then location:= i else location:= 0 return location 10 Algorithm Examples ก ก k ก a c d f g h k l m o p r s u v x z ก ก ก 11 Algorithm Examples ก ก k ก a c d f g h k l m o p r s u v x z ก ก ก 12 4

5 Algorithm Examples ก ก k ก a c d f g h k l m o p r s u v x z ก ก ก 13 Algorithm Examples ก ก k ก a c d f g h k l m o p r s u v x z ก ก ก 14 Algorithm Examples ก ก k ก a c d f g h k l m o p r s u v x z ก ก ก k ก! 15 5

6 Time Complexity ก (Algorithm) ก: 1. (time) ก 2. (memory space) กก ก (Time complexity) 16 Time Complexity ก กก ก ก (number of operations) ก n ก ก ก ก Assignment(:=) Comparison(=,, <, >,, ) Arithmetic operation(+, -,, ) Logical operations (and, or, not) ก f(n) ก ก n ก f(n) ก (time complexity) 17 Orders of Growth ก กก (worst case) ก ก ก ก n ก ก ก ก ก ก กก ก ก ก ก ( ก ก ) ก ก ก ก ก f(n) ก g(n), f(n) กก g(n) ( n ก ) ก ก ก ก ก ก ก ก ( ก ก ก ) ก ก 18 6

7 Orders of Growth - Motivation ก A f A (n)=10n+12 ก n ก B f B (n)=n 2 +1 ก n ก ก ก ก A 19 Orders of Growth ก : time complexity ก A ก B n , ,000 ก A 10n ,012 10,012 1,000,012 ก B n ,001 1,000,001 10,000,000, Visualizing Orders of Growth กก n ก ก ก กก ก ก Value of function f B (n)=n 2 +1 f A (n)=10n+12 Increasing n 21 7

8 Big-O example, graphically ก 10n+12 ก n n>0 ก 11n ก n n ก ก 12 ก 11n cn =11n 10n+12 n n กก 12 n k=12 Value of function Increasing n 10n+12 O(n) 22 Concept of order of growth ก f A (n)=10n+ n+12 ก n (at most order n), n O(n) f B (n)=n 2 +1 ก n 2, O(n 2 ) ก O(n 2 ) ก ก ก O(n) ก ก n 2 ก กก 23 Example ก A B time complexities f g ก f(n) = 3 n 2-3n + 1 g(n)=n 2 ก ก Solution ก n f(n)=g(n) 3 n 2-3n + 1 = n 2 2 n 2-3n + 1 = 0 (2n - 1)(n - 1) = 0 n = 1/2 n =

9 Example n กก n > 1, n = 2 f(2) = 3(2) 2-3(2) + 1 = 7 g(2) = 2 2 = 4 g(n) < f(n) ก n > 1 g(n) ก ก ก B ก ( ก ) n 25 The Growth of Functions ก ก ก big-o O notation ก f g ก ก ก ก R R ก f(n) O(g(n g(n)) c k f(n) c g(n) ก n k ก ก f ก (at most order) g, f O(g), f = O(g) f O( O(g) big-o O notation ก ก ก ก f(n) n ก ก g(n) ก ก f(n) 2n+5 O(n) ก ก f(n) g(n) ก ก big-o f(n) c g(n) ก n k 26 The Growth of Functions ก f(n) O(g(n)), ก c k ก c, k กก c / k ก big-o ก Value of function c g (n) f (n) f(n) c*g(n), n k n k f (n) = O ( g ( n )) 27 9

10 n n O(n 2 ) c,k: n>k: n 2 +10n cn 2 n 2 +10n n 2 +n 2 n 10 n n 2n 2 ก n 10 c = 2, k = Value of function k =10 n n n 2 n 2 28 Does 5n+2 O(n)? Yes Proof: ก Big Oh, c>0 k>0 5n+2 cn ก n k ก ก c k ก k=1 5(1)+2 c = 7 ก c=6 5n+2 6n n 2 k=2 ก c>0 k>0 5n+2 cn ก n k Big Oh 5n+2 O(n) 29 Does n 2 O(n)? No (Prove by contradiction) n 2 O(n) ก Big Oh, c>0 k>0 0 n 2 cn ก n k n 2 cn n กก max{c, k}, ก ก ก c>0 n 2 cn ก n k, n 2 O(n) 30 10

11 Example f(n) ) = 7n7 2 O(n 3 ) n 7 : 7n 2 n 3 c = 1 k = 7: f(n) cn 3 n k f(n) O(n 3 ) ก : f(n) O(n 2 ) f(n) O(n 3 ) ก ก n 3 ก n 2, n 3 ก f(n) 31 Useful Rules for Big-O c>0, O(cf cf)=o( O(f+c)= )=O( O(f - c)=o( O(f) Example: 23*log n O(log n) f 1 (n) O(g 1 (n)) f 2 (n) O(g 2 (n)), (f 1 + f 2 )(n) O(max(g 1 (n), g 2 (n))) Example: (order) n 2 +n? n 2 O(n 2 ), n O(n) n 2 +n O(max(n 2,n)) n 2 +n O(n 2 ) ก ก (polynomial) f(n) ) = a k n k + a k-1 n k a 0, a 0, a 1,, a k, ก f(n) O(n k ) Example: (order) 4n 3 + 2n 2 + n + 5 O(n 3 ) 32 Useful Rules for Big-O f 1 (n) O(g(n)) f 2 (n) is O(g(n)), (f 1 + f 2 )(n) O(g(n)) Example: (order) n 2 + 3n 2 +1? n 2 O(n 2 ) 3n O(n 2 ) n 2 + 3n 2 +1 O(n 2 ) f 1 (n) O(g 1 (n)) f 2 (n) is O(g 2 (n)), (f 1 f 2 )(n) O(g 1 (n) g 2 (n)) Example: (order) (3n+1)*(2n+log n)? 3n+1 O(n) 2n+log n O(n) (3n+1)*(2n+log n) O(n*n)=O(n 2 ) 33 11

12 Complexity Examples ก : : 1. for i = 1 to n do 1.1 for j = 1 to n do A(i,j) := x 1.1 n ก 1. n ก Time Complexity f(n) = n 2 ก f(n) ก O(n 2 ) f(n) O(n 2 ) 34 Complexity Examples ก : 1. i := 1 2. p := 1 3. for j = 1 to n do 3.1 p := p i 3.2 i := i + 1 ก ก (assignment statements) ก 3. n ก ก 2 ก (arithmetic operations) Time complexity f(n) = 4n + 2 ก f(n) ก O(n) f(n) O(n) 35 Complexity Examples ก ก procedure max_diff1(a 1, a 2,,, a n : integers) m := 0 for i := 1 to n-1n for j := i + 1 to n if a i a j > m then m := a i a j {m ก } Time complexity f(n) : n n n = (n( 1)n/2 = 0.5n 2 0.5n ก f(n) ก O(n 2 ) f(n) O(n 2 ) 36 12

13 Complexity Examples ก ก ก ก ก ก ก ก : procedure max_diff2(a 1, a 2,,, a n : integers) min := a1 max := a1 for i := 2 to n if a i < min then min := a i else if a i > max then max := a i m := max - min Time complexity f(n) : 2 +2( 2(n 1)+2 = 2n+22 ก f(n) ก O(n) f(n) O(n) 37 Efficiency of search algorithms Linear search: (sequential search) : Best case : First element is the required element: O(1) Worst case: Last element or element not present : O(n) Average case: N/2 : After dropping the multiplicative constant (1/2) : O(n) 38 Binary search algorithm Search requires the following steps: 1. Inspect the middle item of an array of size N. 2. Inspect the middle of an array of size N/2 3. Inspect the middle item of an array of size N/power(2,2) and so on until N/power(2,k) = 1. This implies k = log2n k is the number of partitions. Best case : O(1) Worst case : O(log2N) Average Case : O(log2N)/2 = O(log2N) 39 13

14 Growth-rate Functions ก g(n) ก ก n : O(1) constant time, the time is independent of n, e.g. array lookup O(log n) logarithmic time, usually the log is base 2, e.g. binary search O(n) linear time, Time requirement increases directly with the size of the problem. e.g. linear search O(n*log n) Algorithms that divide the problems into subproblems and solve them.e.g. efficient sorting algorithms e.g. merge sort O(n 2 ) quadratic time, Algorithms that use two nested loops are examples. e.g. selection sort, bubble sort O(n k ) polynomial (where k is some constant) O(2 n ) exponential time, very slow! Order of growth of some common functions O(1) < O(log n) < O(n) < O(n * log n) < O(n 2 ) < O(n 3 ) < O(2 n ) 40 A comparison of growth-rate functions 41 14