Sums, products ad sequeces How to write log sums, e.g., 1+2+ (-1)+ cocisely? i=1 Sum otatio ( sum from 1 to ): i 3 = 1 + 2 + + If =3, i=1 i = 1+2+3=6. The ame ii does ot matter. Could use aother letter ot yet i use. I geeral, let f: Z R, m, Z, m. i=m f(i) = f m + f m + 1 + + f If m=, i=m f(i) =f(m)=f(). If =m+1, i=m f(i) = f(m)+f(m+1) If >m, 1 i=m f(i) = ( i=m f(i) ) + f() Example: f x = x 2. 2 2 + 3 2 + 4 2 = 4 i=2 i 2 = 29 Similarly for product otatio (product from m to ): 1 Π i=m f i = f m f m + 1 f = (Π i=m f i ) f() 4 For f x = x, 2 3 4 = Π i=2 i = 24 1 2 = Π i=1 i =! ( factorial)
Recurreces ad sequeces To defie a sequece (of thigs), describe the process which geerates that sequece. Sequece: eumeratio of objects s 1, s 2, s 3,, s,, Sometimes use otatio s for the sequece (i.e., set of elemets formig a sequece) Sometimes start with s 0 rather tha s 1 Basis (iitial coditios): what are the first (few) elemet(s) i the sequece. 0 m i=0 i = 0. i=m i = m. 0! = 1. 1!=1. A 0 = Recurrece (recursio step, iductive defiitio): a rule to make a ext elemet from already costructed oes. +1 i=m i = i=m i (+1)! =! (+1) A +1 = P(A ) Resultig sequeces: m, 2m+1, 3m+3, 1, 2,6, 24, 120, + + 1. Here, assume that m,,,,,,,,,
Special sequeces Arithmetic progressio: Sequece: c, c + d, c + 2d, c + 3d,, c +, Recursive defiitio: Basis: s 0 = c, for some c R Recurrece: s +1 = s + d, where d R is a fixed umber. Closed form: s = c + Closed forms are very useful for aalysis of recursive programs, etc. Geometric progressio: Sequece: c, cc, cr 2, cr 3,, cr, Recursive defiitio: Basis: s 0 = c, for some c R Recurrece: s +1 = s r, where r R is a fixed umber. Closed form: s = c r
Fiboacci sequece Imagie that a ship leaves a pair of rabbits o a islad (with a lot of food). After a pair of rabbits reaches 2 moths of age, they produce aother pair of rabbits, which i tur starts reproducig whe reachig 2 moths of age How may pairs rabbits will be o the islad i moths, assumig o rabbits die? Basis: F 1 = 1, F 2 = 1 Recurrece: F = F 1 + F 2 Sequece: 1,1,2,3,5,8,13 Closed form: F = ((1+ 5)/2) ( 1 5)/2 Golde ratio: a+b = a = 1+ 5 a b 2 5
Partial sums Properties of a sum: i=m f i + g i = i=m f(i) + i=m g(i) i=m c f(i) = c i=m f(i) Sum of arithmetic progressio: s = c + for some c, d R Sequece: c, c + d, c + 2d, c + 3d,, c +, Partial sum: i=0 s = i=0 (c + ii ) = i=0 c + ii c( + 1) + d +1 i = c( + 1) + d i=0 Sum of geometric progressio: s = c r for some c, r R Sequece: c, cc, cr 2, cr 3,, cr, Partial sum: If r=1, the i=0 s = c + 1 If r 1, the s i=0 = cr+1 c r 1 2 i=0 =
Tower of Haoi game Rules of the game: Start with all disks o the first peg. At ay step, ca move a disk to aother peg, as log as it is ot placed o top of a smaller disk. Goal: move the whole tower oto the secod peg. Questio: how may steps are eeded to move the tower of 8 disks? How about disks?
Tower of Haoi game Rules of the game: Start with all disks o the first peg. At ay step, ca move a disk to aother peg, as log as it is ot placed o top of a smaller disk. Goal: move the whole tower oto the secod peg. Questio: how may steps are eeded to move the tower of 8 disks? How about disks? Let us call the umber of moves eeded to trasfer disks H(). Names of pegs do ot matter: from ay peg i to ay peg j i would take the same umber of steps. Basis: oly oe disk ca be trasferred i oe step. So H(1) = 1 Recursive step: suppose we have -1 disks. To trasfer them all to peg 2, eed H( 1) umber of steps. To trasfer the remaiig disk to peg 3, 1 step. To trasfer -1 disks from peg 2 to peg 3 eed H(-1) steps agai. So H() = 2H(-1)+1 (recurrece). Closed form: H() = 2 1.
Recurrece relatios Recurrece: a equatio that defies a tt elemet i a sequece i terms of oe or more of previous terms. Thik of F = s for some sequece {s } H() = 2H(-1)+1 F() = F(-1)+F(-2) A closed form of a recurrece relatio is a expressio that defies a tt elemet i a sequece i terms of directly. Ofte use recurrece relatios ad their closed forms to describe performace of (especially recursive) algorithms.
Closed forms of some sequeces Arithmetic progressio: Sequece: c, c + d, c + 2d, c + 3d,, c +, Recursive defiitio: Basis: s 0 = c, for some c R Recurrece: s +1 = s + d, where d R is a fixed umber. Closed form: s = c + Closed forms are very useful for aalysis of recursive programs, etc. Geometric progressio: Sequece: c, cc, cr 2, cr 3,, cr, Recursive defiitio: Basis: s 0 = c, for some c R Recurrece: s +1 = s r, where r R is a fixed umber. Closed form: s = c r
Solvig recurreces Solvig a recurrece: fidig a closed form. Solvig the recurrece H()=2H(-1)+1 H() = 2 H 1 + 1 = 2 2H 2 + 1 + 1 = 2 2 H 2 + 2 + 1 = 2 3 H 3 + 2 2 + 2 + 1 = 2 4 H 4 + 2 3 + 2 2 + 2 + 1 Closed form: H = Σ 1 i=0 2 i = 2 1 Proof by iductio (comig i the ext lecture). Or by oticig that a biary umber 111...1 plus 1 gives a biary umber 10000 0 So addig oe more disk doubles the umber of steps. We say that the fuctio defied by H() grows expoetially Solvig recurreces i geeral might be tricky. Whe the recurrece is of the form T()=a T(/b)+f(), there is a geeral method to estimate the growth rate of a fuctio defied by the recurrece Called the Master Theorem for recurreces.
Fuctio growth. What does it mea to grow at a certai speed? How to compare growth rate of two fuctios? Is f()=100 larger tha g = 2? For small, yes. For > 100, ot so much As usually program take loger o larger iputs, performace o larger iputs matters more. Costat factors do t matter that much. So to compare two fuctios, check which becomes larger as icreases (to ifiity). Igorig costat factors, as they do t cotribute to the rate of growth.
Fuctio growth. How to estimate the rate of growth? Plottig a graph? Not quite coclusive: How do you kow what they will do past the graphed part?
O-otatio. We say that f() grows at least as fast as g() if There is a value 0 such that after 0, g is always at most as large as f More precisely, compare absolute values: g() vs. f() Moreover, igore costat factors: So if two fuctios oly differ by a costat factor, cosider them havig the same growth rate. Deote set of all fuctios growig at most as fast as g by O g Big-Oh of g(). g() is a asymptotic upper boud for f(). Whe both f O g ad g O f, write f Θ(g ) f() is i big-theta of g()). More geerally, for real-valued fuctios f(x) ad g(x), f x O g x iff x 0 R 0 c R >0 x x 0 f x c g x That is, from some poit x 0 o, each f x is less tha g(x) (up to a costat factor). Usually i time complexity have fuctios N R 0, so use for x ad igore.
O-otatio. f O g iff 0 N c R >0 0 f c g() f = 2, g = 2. Take c=1, 0 = 4. For every 0, f g Proof by iductio. So 2 O 2 f = 2, g = 10. Take arbitrary c ad look at 2 c 10. No matter what c is, whe > c 10, 2 c 10 So 2 O 10. You will see some O-otatio i COMP 1000 ad a lot i COMP 2002. f = 2 + 100, g = 10 2. Here, f O g ad also g O(f ) So f Θ(g ) f O g : c = 20 ad/or 0 = 100 work. g O f : Take c=10, 0 = 1. Ca igore ot oly costats, but also all except the leadig term i the expressio.