Addtoal Decrease ad Coquer Algorthms For combatoral problems we mght eed to geerate all permutatos, combatos, or subsets of a set. Geeratg Permutatos If we have a set f elemets: { a 1, a 2, a 3, a } the how ca we geerate all! permutatos? The soluto s to geerate all (-1)! permutatos. Oce we solve ths, the we ca get a soluto to the larger oe by sertg to each of the postos amog every permutato of (-1) elemets. The recursve base case s that whe we have a sgle elemet, there s oly oe permutato. For example, to get the permutatos of {1,2,3}: Fd soluto to permutatos of {1, 2} Fd soluto to permutatos of {1} Ths s ust {1} Isert 2 to {1}, gvg: {2 1} ad {1 2} Isert 3 to {2 1} ad {1 2} gvg: {3 2 1} {2 3 1} {2 1 3} {3 1 2} {1 3 2} {1 2 3} We ed up creatg! permutatos, leadg to a rutme of Θ(!). Ths s horrbly slow for all but very small values of, but that s ot the fault of the algorthm but the problem s smply to geerate a huge umber of tems. Geeratg Subsets You mght recall the kapsack problem or subset sum problem whch may requre us to eumerate through all subsets utl we fd oe that solves the problem. We ca geerate subsets a maer smlar to geeratg all permutatos. To geerate all subsets requres geeratg the power set of the tems. The subsets of A = { a 1, a 2, a 3, a } ca be dvded to two groups: those that do ot cota a ad those that do. The frst group s smply all subsets of: S 1 = { a 1, a 2, a 3, a -1 }. The secod group s smply a added to each subset of S 1 uoed wth S 1.
For example, to get the power set of {1, 2, 3}: Fd all subsets of {1, 2} Fd all subsets of {1} Fd all subsets of {} Ths s ust Isert 1 to ad uo wth to get: {1} ad Isert 2 to: { {1}, } ad uo wth { {1}, } to get: { {1, 2}, {2}, {1}, } Isert 3 to: { {1, 2}, {2}, {1}, } ad uo wth { {1, 2}, {2}, {1}, } to get: { {1,2,3}, {2,3}, {1,3}, {3}, {1, 2}, {2}, {1}, } Josephus Problems The problem s amed after Flavus Josephus, a Jewsh hstora lvg the 1st cetury. As the leged goes, he ad hs 40 comrade solders were trapped a cave, surrouded by Romas. They chose sucde over capture ad decded that they wll form a crcle ad start kllg themselves usg a step of three. As Josephus dd ot wat to de, he was able to fd the safe place, ad stayed alve, later og the Romas who captured hm. The Josephus problem s, gve a scearo such as the above, where should you stad to be a survvor? Let s use a example where the step s 2. Ths would be the scearo where each ma dspatches hs eghbor the crcle. I the examples below, f we start wth perso 1: 6 1 2 6 7 1 2 5 3 5 3 4 4 The o the left, 5 s the last posto, ad o the rght, 7 s the last posto. To solve ths problem, frst cosder the case where, the umber of solders, s eve. After the frst pass we are left wth a problem of exactly half the sze of the orgal. The oly dfferece s posto umberg; 3 becomes 2, 5 becomes 3, etc. For a survvor posto, the prevous roud that perso was stadg posto 2 1. Ths holds for the fal survvor, so the case of a eve umber of solders here s the recurrece relato: J() = 2J(/2) 1 ; Fd soluto for problem of half sze, the shft postos
Next, cosder the case where s odd. The frst pass elmates people all eve postos. If we add to ths the elmato of perso posto 1 rght after the frst pass, we are left wth a problem that has a eve umber of people. Perso 3 becomes 1, 5 becomes 2, 7 becomes 3, etc. For a survvor posto, the prevous roud they were stadg posto 2 + 1. Ths hold for the fal survvor, so the case of a odd umber of solders here s the recurrece relato: J() = 2J(/2) + 1 ; Fd soluto for problem of half sze + 1, the shft The total recurrece relato s the: J() = 1 f = 1 = 2J(/2) 1 f s eve ad > 1 = 2J(/2) + 1 f s odd ad > 1 I more geeral terms: J() = 2J(/2) + (1) What s the rutme of ths algorthm?
The Selecto Problem - Varable Sze Decrease/Coquer Cosder the problem of fdg the th smallest elemet a set of usorted elemets. Ths s referred to as the selecto problem or the th order statstc. If =1 ths s fdg the mmum of a set = ths s fdg the maxmum of a set =/2 ths s fdg the meda or halfway pot of a set -- commo problem Selecto problem defed as: Iput: A set of umbers ad a umber, wth 1<=<= Output: The elemet x A that s larger tha exactly -1 other elemets A. Ca do Θ( lg ) tme easly by sortg wth Merge Sort, ad the pck A[]. But we ca do better! Cosder f the set of umbers s dvded as follows: S1: < p p S2: > p Note that the elemets S1 are ot sorted, but all of them are smaller tha elemet p (partto). We kow that p s the ( S1 +1)th smallest elemet of. Ths s the same dea used qucksort. Now cosder the followg algorthm to fd the th smallest elemet from Array A: Select a pvot pot, p, out of array A. Splt A to S1 ad S2, where all elemets S1 are <p ad all elemets S2 are >p If = S1 +1 the p s the th smallest elemet. Else f <= S1 the the th smallest elemet s somewhere S1. Repeat the process recursvely o S1 lookg for the th smallest elemet. Else s somewhere S2. Repeat the process recursvely lookg for the - S1-1 smallest elemet. Questo: How do we select p? Best f p s close to the meda. If p s the largest elemet or the smallest, the problem sze s oly reduced by 1. Always pck the same elemet, or 1 Pck a radom elemet Pck 3 radom elemets, ad pck the meda Other method we wll see later
Oce we have p t s farly easy to partto the elemets: If A cotas: [5 12 8 6 2 1 4 3] Ca create two subarrays, S1 ad S2. For each elemet x A, f x<p put t S1, f x>=p put t S2. p=5 S1: [2 1 4 3] S2: [5 12 8 6] Ths certaly works, but requres addtoal space to hold the subarrays. We ca also do the parttog -place, usg o addtoal space f we mata poters startg from the begg ad ed of the array as llustrated below: Partto(A,p,r) ; Parttos array A[p..r] x A[p] ; Choose frst elemet as partto elemet p-1 r+1 whle true do repeat -1 utl A[] x repeat +1 utl A[] x f < the exchage A[] A[] else retur ; dcates dex of parttos
Example: A[p..r] = [5 12 8 6 2 1 4 3] x=5 5 12 2 6 2 1 4 3 5 12 2 6 2 1 4 3 5 12 2 6 2 1 4 3 3 12 2 6 2 1 4 5 swap 3 12 2 6 2 1 4 5 3 12 2 6 2 1 4 5 3 4 2 6 2 1 12 5 swap 3 4 2 6 2 1 12 5 3 4 2 6 2 1 12 5 3 4 2 6 2 1 12 5 3 4 2 1 2 6 12 5 swap 3 4 2 1 2 6 12 5 3 4 2 1 2 6 12 5 3 4 2 1 2 6 12 5 crossover, >
Retur. All elemets A[p..] smaller or equal to x, all elemets A[+1..r] bgger or equal to x. (Note ths s a lttle dfferet tha the tal example, where we splt the sets up to < p, p, ad > p. I ths case the sets are <=p or >=p. (Cosder the case f all array elemets are detcal). If the pvot pot selected happes to be the largest or smallest value, t wll also be guarateed to splt off at least oe value). Ths route makes oly oe pass through the array A, so t takes tme Θ( ). No extra space requred except to hold dex varables. To use ths verso of Partto the Selecto algorthm, we eed to modfy the selecto algorthm a bt sce we are ot splttg to <p, p, ad >p. Here s the modfed algorthm: ; Select from array A, wth lower dex of p ad upper dex of r, the th largest umber Select(A,p,r,) If p = r retur A[p] Q Partto(A,p,r) // Q gets the dex of where we made the partto K Q p + 1 // Sze of left partto If <=K retur(select(a,p,q,) else retur(select(a,q+1,r,-k) Worst case rug tme of selecto: Pck m or max as partto elemet, producg rego of sze -1. T( ) = T( 1) + Θ ( ) subprob tme to splt Evaluate recurrece by teratve substtuto method: T( 1) = Θ( 1), T( 2) = Θ( 1) + Θ( 2), T( 3) = Θ( 1) + Θ( 2) + Θ ( 3),... Θ = 1 Θ( ) = 1 2 = Θ( ) Recurso tree for worst case: 1-1 1-2 1-1 -2 Total = Θ( 2 )
Best-case Parttog: I the best case, we pck the meda each tme. T( ) = T( ) + Θ ( ) 2 Usg the master method: a=1, b=2, f()= Θ( ) Case 3: 1 < 2 1 so the soluto s f()= Θ( ) Recurso Tree for Best Case: lg /2 /2 /4 /4 /2 /4 1... 1 Total = Θ() Average Case: Ca thk of the average case as alteratg betwee good splts where s splt half, ad bad splts, where a m or max s selected as the splt pot. Recurso tree for bad/good splt, good splt: ~ 2*lg /2 /2 1 (/2)-1 /2 /2 1... ((/2-1)/2 ((/2)-1)/2 / 4 1 Total = Θ() Both are Θ( ), wth ust a larger costat the evet of the bad/good splt. So average case stll rus tme Θ( ).
We ca solve ths problem worst-case lear tme, but t s trcker. I practce, the overhead of ths method makes t ot useful practce, compared to the prevous method. However, t has terestg theoretcal mplcatos. Basc dea: Fd a partto elemet guarateed to make a good splt. We must fd ths partto elemet quckly to esure Θ( ) tme. The dea s to fd the meda of a sample of medas, ad use that as the partto elemet. New partto selecto algorthm: Arrage the elemets to /5 groups of 5 elemets each, gorg the at most four extra elemets. (Costat tme to compute bucket, lear tme to put to bucket) Fd the meda of each group. Ths gves a lst M of /5 medas. (tme Θ( ) f we use the same meda selecto algorthm as ths oe or hard-code t) Fd the meda of M. Retur ths as the partto elemet. (Call partto selecto recursvely usg M as the put set) See pcture of meda of medas: x Guaratees that at least 30% of wll be larger tha pvot pot p, ad ca be elmated each tme! Rutme: T T 7 ( ) = ( ) + T( ) + O( ) 5 10 select recurse overhead of splt/select pvot subprob The O() tme wll domate the computato resultg O() ru tme.