CS161 Desig ad Aalysis of Algorithms Da Boeh 1 Admiistrative Lecture 1, April 3, 1 Web page http://theory.staford.edu/~dabo/cs161» Hadouts» Aoucemets» Late breakig ews Gradig ad course requiremets» Midterm/fial/hw» Project» Late HW policy» Importace of readable HW» Collaboratio Probability - Ch. 6., pp 14-115 READ NOW! 1
Why Study Algorithms? (why cs161?) Bag of tricks» Sortig» Data structures: queues/stacks/heaps/trees» Search Methodology - how to desig algorithms» Divide & coquer» Recursive algorithms» Radomized algorithms» Dyamic programmig Useful abstractios.» Schedulig classes graphs.» Job assigmet balls ad boxes. Higher-level way of approachig problems 3 How to compare algorithms? Code ad ru - experimet» Iputs?» Parameters?» Bad implemetatios? Average case» what is average iput?? Worst case» Asymptotics» rough idea o performace» aalytical depedece betwee parameters 4
Example from Ch. Isertio sort: for j = to key = A(j) i=j-1 while i > ad A(i) > key A(i+1) = A(i) A(i) = key i-- ed ed Example: 7 3 5 8 1 3 7 3 5 7 3 5 7 8 1 3 5 7 8 1 3 5 7 8 5 About Pseudo-Code Not really a program, just a outlie Eough details to establish the ruig time ad correctess. No error-hadlig mechaisms. Eve pseudo-code is too complicated! Note that for a trivial algorithms it obscures what is really goig o... The i-place part is a optimizatio. We could start by a simpler descriptio:» Go over the umbers oe-by-oe, startig from the first, copy to ew array.» Each time copy to the correct place i the ew array.» I order to create empty space, shift the umbers that are larger tha the curretly cosidered umber oe cell to the right. 6 3
Aalysis Correctess ad termiatio. Ruig time:» Depeds o iput size» iput properties Wat a upper boud o:» Worst case: max T(), ay iput.» Expected: E[T()], iput take from a distributio. which?? example: sortig arrivig TCP/IP packets they are mostly sorted already.» Best case: Ca be used to argue that the algorithm is really bad. (ay algorithm ca be rewritte to have a excellet best case performace) 7 Back to isertio sort Isertio sort: for j = to key = A(j) i=j-1 while i > ad A(i) > key A(i+1) = A(i) A(i) = key i-- ed ed Simplified algorithm:» Go over the umbers oe-by-oe, startig from the first, copy to ew array.» Each time copy to the correct place i the ew array.» I order to create empty space, shift the umbers that are larger tha the curretly cosidered umber oe cell to the right. -1... Â( t j -1) j= times t j each 8 4
Aalysis Best ruig time: Outer loop always executed, Ier loop - ot executed if iput already sorted. Assume each operatio takes 1 time uit - approximatio. + ( - 1) + ( - 1) + t + ( t - 1) + ( - 1) t j worst case ª j  fi t = j j= Would like to formalize this statemet!  ( + 1) -1 14 43 This domiates!  j j j= j= Do we really eed to pay close attetio to all the idices i the summatios? Maybe some or them are ot really importat?? 9 Formalizatio ( +1) How to formalize that was the mai issue?? The aswer is asymptotic aalysis:» Igore machie-depedet costats.» Look at growth of T() as Ituitio: drop low-order terms eg: 54+ 1-3+ =Θ( 4) Idea: as, Θ( ) becomes better (faster) tha Θ( 4) 1 5
Back to isertio sort aalysis Ier loop was Q(j) T () Θ ( t ) ( ) ( j =Θ tj =Θ ) Is this formal? NO! Example, usig the same logic: Θ (1) +Θ (1) =Θ(1) seems to imply that Θ (1) =Θ(1) Icorrect! i= 1 We eed formalizatio!?? Aother example: log ~ 1/1 11 Asymptotics big-oh otatio: f ( ) = O( g( )) cost c, s.t. : f ( ) cg( ) Example: = O( 6 ) but ot vice versa!! = is ot equality but membership i a set. Set otatio is cumbersome: O( g( )) = { f ( ) cost c, s.t. : f ( ) cg( )} What do we mea by f ( ) = O( ) + h( ) = O( ), f ( ) = h( ) + We are too lazy to specify h() exactly! 1 6
Asymptotics Small-oh otatio: f ( ) = o( g( )) cost c, s.t. : f ( ) < cg( ) Differeces from big-oh Prove that = o( ): Give c, lets take = / c for, c c c ( c ) = > QED g() f() Factor f() g() f=o(g) i both cases! 13 Omega otatio Big-Omega: f ( ) = Ω( g( )) cost c, s.t. : cg( ) f ( ) Small-omega: f ( ) = ω( g( )) cost c, s.t. : cg( ) < f ( ) O: o: < W: : > 14 7
Trasitivity etc. Most rules apply: Example: trasitivity a b, b c a c f = Og ( ), g= Oh () f = Oh () Proof: f = Og ( ) cost c1, 1 s.t. 1 : f() cg 1 () g= Oh () cost c, s.t. : g () ch () Take 3= max( 1, ), c3= cc 1 The: 3: f() cg 1 () cch 1 () = ch 3 () f() = Og ( ( )) QED Not all rules apply! f, g s.t. f O( g) ad g O( f ) example: f =, g = 1 + si 15 Theta otatio Theta: f ( ) = Θ( g( )) cost c, c, s.t. : c g( ) f ( ) c g( ) 1 1 Ofte cofused with Big-Oh otatio! Example: / = Θ( ) Proof: take = 8, the for : / / 4+ 8 / 4 = / 4 O the other had, we have: / < / Thus: / 4 / / i.e. c = 1/ 4, c = 1/. 1 Claim: Low order terms do ot matter. Needs a proof! (HW?) 16 8
Simple Theorem Claim f = O g ad g = O f f = Θ g ( ) ( ( )) ( ) ( ( )) ( ) ( ( )) Proof: 1, c1 s.t. 1: f ( ) c1 g( ), c s.t. : g( ) c f ( ) max( 1, ): 1 c g ( ) f ( ) c 1 g ( ) QED 17 Summary Remember the defiitios. Formally prove from defiitios. Use ituitio from the properties of,, etc. 18 9