Asyptotic Notatio 3 Chapter Asyptotic Notatio Goal : To siplify the aalysis of ruig tie by gettig rid of details which ay be affected by specific ipleetatio ad hardware. [1] The Big Oh (O-Notatio) : It gives us the upper boud (Worst case behaviour) of the ruig tie. If f() ad g() are two icreasig fuctios o o-egative ubers the. f() = O(g()) if there exists costat c ad 0 where c ad 0 are two positive costats ad c > 0 & 0 1 f c g. such that 0 Ruig tie c.g() f() Exaple: If f() = 60 + 5 + 11 the f() < 60 + 5 + 11 76 where c = 76 f 1 0 O f < 76. 0 Iput size Siple Rule: Pop lower order ters ad costat factors. e.g. 5 6 0 is O ad 5 log 7 is O log []. Oega Notatio ( ) : It gives the tight boud (Lower boud) o ruig tie. If f() ad g() are two icreasig fuctios over o-egative ubers. The f g if there exists a costat c ad 0 ad c 0 & 0 1 such that f c g 0
4 Asyptotic Notatio Ruig tie f() c g() 0 Iput size For exaple: f 60 5 11 60 1 Therefore, f where c 60 ad > 1 [3]. Theta Notatio : It gives the tight boud (Average case behaviour) o ruig tie. If f() ad g() are two icreasig fuctio defied over o-egative ubers the f g if there exists costats c 1, c ad 0 such that c g f c g 1 0 f() is sadwidthed betwee c g ad c g 1 0 If f O g ad f g The f g ad vice-versa. As we have, f 60 5 11 ad f O ad f Therefore, f [4]. Little O Notatio (o) : f o g g is strictly (do ot cosider equality) larger tha f g 0 Graphical represetatio reai sae. f li 0 g f
Asyptotic Notatio 5 [5] Little Oega : f g g is strictly saller tha 0 f g f Graphical represetatio reai sae. f li g Types of tie coplexitites: (i) costat tie coplexity O1 (ii) logarithic tie coplexity Olog (iii) liear tie coplexity O (iv) quadratic O 3 (v) cubic O k (vi) polyoial O k 0 (vii) expoetial O c c 1 (1) 1 log log log log () log (3) (4) log (5)! (6)! (7)! (8) 100 log (9) log (10) log log log log (11) a b a log b Note :! O correct!=o exactly correct Properties of Asyptotic Notatio: (1) Relfexive properties: (i) f O f
6 (ii) f f (iii) f f (iv) f o f (v) f f () Syetric properties: (i) if the f O g g O f (ii) if the f g g f (iii) if (3) Trasitive property: the f g g f (i) If f O g & & g O h the f O h (ii) If & & the f g g h f h (4) If f O g the h f O h g where, h is positive fuctio. (5) If f O g d O h the (i) ax f d g h O g h (ii) f d O g h Proble : Cosider the followig z fuctios. & & f O g g O f & & g O h h O g True/False (a) f g O h (b) f O h (c) f g O h h (d) h O f Fid the Tie coplexity? ai ( ) x y z; for (i = 1; i ; i ) ( tie exeuctio) x y z; for i 1; i ; i Asyptotic Notatio
Asyptotic Notatio 7 for j 1; j ; j x x z; so, O( ). Fid coplexity. ai ( ) x y z for i 1; i ; i for j 1; j i; j tie excutio x y z ( 1 + + 3 +... +) = so, O Differece betwee Relative Aalysis ad Absolute Aalysis Relative Aalysis : (1) It is software ad hardware depedet aalysis. () Exact aswers (3) Aswer is chagig fro syste to syste. Absolute aalysis: (1) It is software ad hardware idepedet aalysis. () Approxiate aswers (3) Aswer will be sae i all coputer systes. Note : Absolute aalysis is the stadard aalysis i practice. Absolute aalysis : It is a deteriatio of order of agitude of a stateet Nuber of ties a stateet is executig. Exaple : ai ( ) x y z; 1 tie executio O(1) or order of 1. 1
8 SOLVED PROBLEMS Asyptotic Notatio 1. Solve the recurrece equatios T() = T( 1) + T(1) = 1 Sol. By usig substitutio ethod we get followig series: ( 1) ( ) ( 3)... 3 1 Which is su of atural ubers. ( 1) O( ). What is the geeratig fuctio G(z) for the sequece of Fiboacci ubers? z Sol. The geeratig fuctio for the Fiboacci ubers G(z) is G(z). 1 z z 3. As. Sol. 1 k O(), where O() stads for order is [GATE-1987 : Marks] [GATE-1987 : Marks] (a) O() (b) O( ) (c) O( 3 ) (d) O(3 ) [GATE-1993 : Marks] (b) 1 k ( 1) O() O(1) O() O(3)... O() O O( ) 4. Cosider the followig two fuctios: As. Sol. 3 for 0 10, 000 g () for 10, 000 Which of the followig is true? (a) g 1() is O(g ()) 1 3 for 0 100 g () for 100 (b) (c) g () is O(g 1 ()) (d) (a) 3 g 1() is O( ) g () is O() [GATE-1994 : Marks] Therefore; 3 for N 10000 g 1() O(g ()) Correct optio is (a).
Asyptotic Notatio 5. Which of the followig is false? (a) log 100 log O 100 (b) log O(log log ) x y (c) If 0 x y the O( ) (d) O(k) As. (b, d) Sol. (a) We kow that f() = O(g()) i.e., f() k.g() for k, soe positive itegers ad > 0 log 100 log 10000 100 for k = 10000 log 100 log = O 100 9 [GATE-1996 : 1 Mark] (b) log 1 log log log O(log log ) x y (c) as 0 x y x y O( ) (d) k for k O(k) 6. The cocateatio of two lists is to be perfored i O(1) tie. Which of the followig ipleetatios of a list should be used? (a) sigly liked list (b) doubly liked list (c) circular doubly liked list (d) array ipleetatio of list [GATE-1997 : 1 Mark] As. (c) Sol. As list cocateatio requires traversig at least oe list to the ed. So sigly liked list ad doubly liked requires O() tie coplexity whereas circular doubly liked list required O(1) tie. 7. Let f() = log ad g() = (log ) 10 be two positive fuctios of. Which of the followig stateets is correct? (a) f () O(g()) ad g() O(f ()) (b) g() O(f ()) ad f () O(g()) (c) f () O(g()) ad g() O(f ()) (d) f () O(g()) ad g() O(f ()) [GATE-001 : 1 Mark] As. (b) Sol. f () log (log ) 10 g() (log ) Whereas f () log 10 g() O(f ()) O(g()) because 10 log (log )
10 Asyptotic Notatio 8. I the worst case, the uber of coparisos eeded to search sigly liked list of legth for a give eleets is (a) log (b) / (c) log 1 (d) [GATE-00 : 1 Mark] As. Sol. (d) Worst case of searchig sigly liked list is whe give eleet does t preset at all i the sigly liked list. Usig liear search the require coparisos i worst case. 9. Cosider the followig fuctios As. log f () 3 g() h()! Which of the followig is true? (a) h() is O(f()) (b) h() is O(g()) (c) g() is ot O(f()) (d) f() is O(g()) [GATE-00 : Marks] (d) log Sol. f () 3, g(), h() =! = O( ) log log b log a g() [a b ] 10. Cosider the followig algorith for searchig for a give uber x i a isorted array A[1...] havig distict values: 1. Choose a i uiforly at rado fro 1...;. If A[i] = x the Stop else Goto 1; Assuig that x is preset o A, What is the expected uber of coparisos ade by the algorith before it teriates? (a) (b) 1 (c) (d) / [GATE-00 : Marks] As. (a) Sol. Let expected uber of coparisos be E. Value of E is su of followig expressio for all the possible cases: Case-I If A[i] is foud i the first attept, Nuber of coparisos = 1 Probability = 1/. Case-II If A[i] if foud i the secod attept, Nuber of coparisos = ( 1) 1 Probability Case-III If A[i] is foud i the third attept, Nuber of coparisos = 3 ( 1) ( 1) 1 Probability There are actually ifiite such cases. So, we have followig ifiite series for E. 1 1 1 ( 1) ( 1) 1 E 3......(i) After ultiplyig equatio (i) with ( 1), we get ( 1) 1 1 1 1 E......(ii)
Asyptotic Notatio Subtractig (ii) fro (i), we get E 1 1 1 1 1 1... The expressio o right side is a G.P. with ifiite eleets. a So apply the su forula 1 r 11 E 1 ( 1) 1 E Correct optio is (a). 11. The ruig tie of the followig algorith Procedure A(). If < = retur (1) else retur A As. Sol. best described by (a) O() (b) O(log ) (c) O(log log ) (d) O(1) (c) Recursive relatio for procedure A() is T() T c1if Let T() T( ) T( ) S() T() S() / T T S / T() T c1if S() S c1 O(log ) O(log log ) [ log ] T() S() O(log log ) Correct optio is (c). ; is [GATE-00 : Marks] 1. Cosider the followig three clais 1. ( k) ( ), where k ad are costat. 1 O( ) 3. 1 O( ) Which of these clais are correct? (a) 1 ad (b) 1 ad 3 (c) ad 3 (d) 1, ad 3 [GATE-003 : 1 Mark] As. (a) Sol. Cosider each stateet separately
1 I. II. III. f () ( k) so, f () (1 ) f () 1 C C C f () O( ) 1 f () 1 f () f () f () O( ) 1 f () (Assue k = 1 is costat) 1 1 f () f () f () O( ) Therefore I ad II are correct. Asyptotic Notatio 13. Cosider the followig C fuctio. float f (float x, it y) float p, s; it i; for (s = 1, p = 1, i = 1; i < y; i++) p* = x/i; s + = p; retur s; For large values of y, the retur value of the fuctio f best approxiates (a) x y (b) e x (c) l (1 + x) (d) x x [GATE-003 : 1 Mark] As. Sol. (b) The give fuctio f is ot recursive, so cosider the followig iteratio ethod. i p s x p p* s s p i Iitialize 1 1 1 1 p x s 1 x x x p x s 1 x 3 x x x x 3 p s 1 x 4 6 3 3 4 x x x x x 4 p s 1 x 6 4 6 4 4 3 4 5 x x x x x x 5 p s 1 x 4 5 6 4 10 For large value of y assue y the i also teds to ifiite it eas icreet of for loop ay teds to ifiite. I the give fuctio we choose y as a large iteger but ot ifiite. The retur value of the fuctio f is s.
Asyptotic Notatio 13 3 4 5 x x x x s 1 x 6 4 10 3 4 5 x x x x s 1 x!!3!4!5 s e x 14. The tighest lower boud o the uber of coparisos, i the worst case, for coparisos-based sortig is of the order of (a) (b) (c) log (d) log [GATE-004 : 1 Mark] As. (c) Sol. Ay decisio tree that sorts distict eleets has height at least log. So the tighest lower boud o the uber of copariso based sortig is log but fro starlig s approxiatio. (/e) Takig log both sides log log (/e) log log (/e) log (log log e) log (log 1.44) log log 1.44 So log O( log ) 15. What does the followig algorith approxiate? (Assue > 1, > 0). x = ; y = 1; while (x y > ) x = (x + y)/; y = /x; prit (x); (a) log (b) (c) 1/ (d) 1/3 [GATE-004 : Marks] As. Sol. (c) Let x = = 9. The loop will be teriated whe x y = 0 or x y < 0. Cosider the followig iteratio for x 9, y 1 x y > 0 x ( x y)/, y / x 9 1 8 x 5.0, y 9/5.0 1.8 5.0 1.8 3. x 3.4, y 9/3.4.6 3.4.6.80 x 3.0, y 9/3.0 3.0 x y = 3.0 3.0 = 0, loop teriated So, = 9 the x = 3 x = () 1/ = (9) 1/ x = 3 So the algorith coputes x = 1/.
14 Asyptotic Notatio 16. Let A[1,..., ] be a array storig a bit (1 or 0) at each locatio, ad f() is a fuctio whose tie coplexity is (). Cosider the followig progra fraget writte i a C like laguage: couter = 0; for (i = 1; i < ; i++) if (A[i] = = 1) couter ++; else f (couter); couter = 0; The coplexity of this progra fraget is (a) ( ) (b) ( log ) ad O( ) (c) () (d) O( log ) [GATE-004 : Marks] As. Sol. (c) The give code is 1. Couter = 0;. for (i = 1; i < = ; i++) 3. if (A[i] = = 1) couter ++; 4. else f(couter); couter = 0; 5. The tie coplexity of the progra fraget depeds o the frequecy (Nuber of steps) of lie 3 d 4. I lie 4 the frequecy depeds o the variable couter ad there is o icreet i the couter variable which is iitialize to 0, so f(0) the couter = 0 it eas there is o cell i a array which havig a bit 0, so all cells i the array cotais 1. Cosider the lie 3 if (A[i] = = 1) couter ++; the value of i will be icreases upto so the value of couter will be. Sice is the frequecy of the lie 3 ad the frequecy of lie 4 is 0. So the tie coplexity of lie 3 is O() o average ad f(0) = O(1) is the tie coplexity of lie 4. So the tie coplexity of the progra fraget is axiu of lie 3 ad 4 which is O() o average. 17. The tie coplexity of the followig C fuctio is (assue > 0) it recursive (it ) if ( = = 1) retur (1); else retur (recursive ( 1) + recursive ( 1)); (a) O() (b) O( log ) (c) O( ) (d) O( ) [GATE-004 : Marks] As. Sol. (d) The give C fuctio is recursive. The best way to fid the tie coplexity of recursive fuctio is that covert the code (algorith) ito recursio equatio ad solutio of the recursio equatio is the tie coplexity of give algorith. 1. it recursive (it ). if ( = = 1) retur (1); 3. else 4. retur (recursive ( 1) + recursive ( 1)); 5. Let recursive() = T()
Asyptotic Notatio Accordig to lie 4 the recursio equatio is T() = T( 1) + T( 1), > 1. So the coplete recursio equatios is T() = 1, = 1 T() = T( 1) + T( 1), > 1 or T() = T( 1), > 1 T(1) = 1 = T() = T(1).1 = 1 T(3) = T() =. = T(4) = T(3) =. = 3 T() = 1 or T() = 1/ So, T() = O( ) 18. The recurrece equatio T(1) = 1 T() = T( 1) +, evaluates to (a) 1 (b) (c) 1 (d) [GATE-004 : Marks] As. (a) Sol. T(1) = 1 T() = T( 1) + T() = T(1) + =.1 + = 4 T(3) = T() + 3 =.4 + 3 = 11 T(4) = T(3) + 4 =.11 + 4 = 6 T( 1) = T( ) + = ( 1) So T() = + 1 19. Let f(), g() ad h() be fuctios defied for positive itegers such that f() = O(g(), g()) O(f()), g() = O(h()), ad h() = O(g()). Which oe of the followig stateets is FALSE? (a) f() + g() = O(h()) + h()) (b) f() = O(h()) (c) h() O(f()) (d) f()h() O(g()h()) [GATE-004 : Marks] As. (d) Sol. We ca verify as: f f BUT g f. Therefore, f < g, Also g = h as g = O(h) ad h = O(g). Therefore f < g ad g = h (a) f () g() O(h()) h()) is true. (b) f () O(h()) is true. f g f h h h f h (c) h() O(f ()) is true. (d) f ()h() O(g()h()) is false. h f is correct. f h g h iplies fh O(gh) 0. The tie coplexity of coputig the trasitive closure of a biary relatio o a set of eleets is kow to be (a) O() (b) O( log ) (c) O( 3/ ) (d) O( 3 ) [GATE-005 : 1 Mark] 15