Dynamic Programming! CSE 417: Algorithms and Computational Complexity!

Transcription

1 Dynamc Programmng CSE 417: Algorthms and Computatonal Complexty Wnter 2009 W. L. Ruzzo Dynamc Programmng, I:" Fbonacc & Stamps Outlne: General Prncples Easy Examples Fbonacc, Lckng Stamps Meater examples RNA Structure predcton Weghted nterval schedulng Maybe others 1 2 Some Algorthm Desgn Technques, I General overall dea Reduce solvng a problem to a smaller problem or problems of the same type Greedy algorthms Used when one needs to buld somethng a pece at a tme Repeatedly make the greedy choce - the one that looks the best rght away e.g. closest par n TSP search Usually fast f they work (but often don't) Some Algorthm Desgn Technques, II Dvde & Conquer Reduce problem to one or more sub-problems of the same type Typcally, each sub-problem s at most a constant fracton of the sze of the orgnal problem e.g. Mergesort, Bnary Search, Strassen s Algorthm, Qucksort (knd of) 3 4

2 Some Algorthm Desgn Technques, III Dynamc Programmng Gve a soluton of a problem usng smaller subproblems, e.g. a recursve soluton Useful when the same sub-problems show up agan and agan n the soluton Dynamc Programmng Program A plan or procedure for dealng wth some matter " Webster s New World Dctonary 5 6 Dynamc Programmng Hstory Bellman. Poneered the systematc study of dynamc programmng n the 1950s. Etymology. Dynamc programmng = plannng over tme. Secretary of Defense was hostle to mathematcal research. Bellman sought an mpressve name to avod confrontaton. "t's mpossble to use dynamc n a pejoratve sense" "somethng not even a Congressman could object to" Reference: Bellman, R. E. Eye of the Hurrcane, An Autobography. A very smple case: Computng Fbonacc Numbers Recall F n = F n-1 + F n-2 and F 0 = 0, F 1 = 1 Recursve algorthm: Fbo(n)" f n=0 then return(0) else f n=1 then return(1) else return(fbo(n-1)+fbo(n-2)) 8 7

3 Call tree - start F (6)" Full call tree F (6)" F (5)" F (4)" F (5)" F (4)" F (4)" F (4)" F (2)" F (2)" F (2)" 1" 1" 1" 9 10 Memo-zaton (Cachng) Save all answers from earler recursve calls Before recursve call, test to see f value has already been computed Dynamc Programmng NOT memozed; nstead, convert memozed alg from a recursve one to an teratve one" (top-down bottom-up) Fbonacc - Memozed Verson ntalze: F[] " undefned for all F[0] " 0 F[1] " 1 FboMemo(n): f(f[n] undefned) { F[n] " FboMemo(n-2)+FboMemo(n-1) } return(f[n]) 11 12

4 Fbonacc - Dynamc Programmng Verson FboDP(n): F[0] " 0 F[1] " 1 for =2 to n do F[] " F[-1]+F[-2] For ths problem, keepng only last 2 entres nstead of full array suffces, but about the same speed end return(f[n]) Dynamc Programmng Useful when Same recursve sub-problems occur repeatedly Parameters of these recursve calls antcpated The soluton to whole problem can be solved wthout knowng the nternal detals of how the sub-problems are solved prncple of optmalty Makng change Gven: Large supply of 1, 5, 10, 25, 50 cons An amount N Problem: choose fewest cons totalng N Lckng Stamps Gven: Large supply of 5, 4, and 1 stamps An amount N Problem: choose fewest stamps totalng N Casher s (greedy) algorthm works: Gve as many as possble of the next bggest " denomnaton 15 16

5 How to Lck 27 A Smple Algorthm # of 5 stamps # of 4 stamps # of 1 stamps total number At most N stamps needed, etc. for a = 0,, N {" for b = 0,, N {" for c = 0,, N {" f (5a+4b+c == N && a+b+c s new mn)" {retan (a,b,c);}}}" output retaned trple;" Morals: Greed doesn t pay; success of casher s alg depends on con denomnatons 17 Tme: O(N 3 )" (Not too hard to see some optmzatons, but we re after bgger fsh ) 18 Better Idea Theorem: If last stamp n an opt sol has value v, then prevous stamps are opt sol for N-v. Proof: f not, we could mprove the soluton for N by usng opt for N-v Alg: for = 1 to n: M () = mn % & 0 M ("5) M ("4) M ("1) =0 #5 #4 #1 ' ( ) where M() = mn number of stamps totalng " 19 New Idea: Recurson M () = mn % & 0 M ("5) M ("4) M ("1) =0 #5 #4 #1 ' ( ) 27" "22 " "23 " "26" "22 25 Tme: > 3 N/5 20

6 Another New Idea:" Avod Recomputaton Tabulate values of solved subproblems Top-down: memozaton Bottom up: " & 0 = 0# for = 0,, N do M [ ] = mn M [ ( 5] ' 5 % M [ ( 4] ' 4" M [ ( 1] ' 1 Fndng How Many Stamps M() Mn(3,1,3) = 2" Tme: O(N) Fndng Whch Stamps:" Trace-Back M() " Mn(3,1,3) = 2 23 Trace-Back Way 1: tabulate all add data structure storng back-ponters ndcatng whch predecessor gave the mn. (more space, maybe less tme) Way 2: re-compute just what s needed TraceBack(): f == 0 then return; for d n {1, 4, 5} do f M[] == 1 + M[ - d] then break; prnt d; TraceBack( - d); M [ ] = mn & % [ ( 5] [ ( 4] M M M [ ( 1] ' = 0 ' ' # "

7 Complexty Note O(N) s better than O(N 3 ) or O(3 N/5 )" But stll exponental n nput sze " (log N bts)" (E.g., mserable f N s 64 bts c 2 64 steps & 2 64 memory.)" Note: can do n O(1) for 5, 4, and 1 but not n general. See NP-Completeness later. Elements of Dynamc Programmng What feature dd we use? What should we look for to use agan? Optmal Substructure " Optmal soluton contans optmal subproblems" A non-example: mn (number of stamps mod 2) Repeated Subproblems " The same subproblems arse n varous ways 25 26