Algorithms. Outline! Approximation Algorithms. The class APX. The intelligence behind the hardware. ! Based on

Transcription

1 6117CIT - Adv Topics in Computing Sci at Nathan 1 Algorithms The intelligence behind the hardware Outline! Approximation Algorithms The class APX! Some complexity classes, like PTAS and FPTAS! Illustration of some PTAS! Based on P. Schuurman and G. Woeginger (2001), Approximation Schemes - A Tutorial. M. Mastrolilli course notes 2 The class APX! (an abbreviation of "approximable").! The set of NP optimization problems that allow polynomial-time approximation algorithms with approximation ratio bounded by a constant (or constant-factor approximation algorithms for short).! Problems in this class have efficient algorithms that can find an answer within some fixed percentage of the optimal answer.! An approximation algorithm is called a!- approximation algorithm for some constant! if it can be proven that the solution that the algorithm finds is at most! times worse than the optimal solution. 3 1

2 Review from week 2! The vertex cover problem and traveling salesman problem with triangle inequality each have simple 2- approximation algorithms.! The traveling salesman problem with arbitrary edge-lengths can not be approximated with approximation ratio bounded by a constant as long as the Hamiltonianpath problem can not be solved in polynomial time. 4 Alternative view on PTAS! If there is a polynomial-time algorithm to solve a problem within every fixed percentage (one algorithm for each percentage), then the problem is said to have a polynomial-time approximation scheme (PTAS) Unless P=NP, it can be shown that there are problems that are in APX but not in PTAS; that is, problems that can be approximated within some constant factor, but not every constant factor. 5 More on APX! A problem is said to be APX-hard if there is a PTAS reduction from every problem in APX to that problem,! A problem is APX-complete if the problem is APX-hard and also in APX.! As a consequence of PTAS! APX, no APX-hard problem is in PTAS. 6 2

3 Focus on Optimization Problems! Notation We use I for an instance of an optimization problem We use I =n, for the length of the input instance We use Opt(I) for the value of the optimal solution! We focus on minimization problems in this lecture, but all concepts are symmetric for maximization problems. 7!- Approximation Algorithm We denote A(I) = solution value. An algorithm is an!-approximation Algorithm if A(I) "! Opt(I) for all instances and the running time is polynomial in I! = worst-case approximation ratio! >= 1 Good:! close to 1 8 PTAS: Polynomial Time Approximation Scheme This is a family {A#}#>0 of (1+#)-approximation algorithms with running time polynomial in I As observed in last lecture, the scheme fits the definition if the running time is exponential in 1/# : e.g. O( I 1/# ) NOTE: FPTAS: Fully PTAS running time also polynomial in 1/# : e.g. O( I /# 3 ) 9 3

4 Strongly and Weakly NP-hard! If a problem is NP-hard even if the input is encoded in unary, then it is called strongly NP-hard = NP-hard in the strong sense = unary NP-hard! If a problem is polynomially solvable under a unary encoding, then it is solvable in pseudo-polynomial time.! NP-Hard in the strong sense is contained within NP-Hard in the weak sense 10 Complexity Classes Relationships NP APX Pseudo-Poly PTAS FPTAS P 11 Some Known Approximation Algorithms Non-constant worst-case ratio Graph coloring O(n 1/2-e ) Total flow time O(n 1/2 ) Set covering O(log n) Vertex cover O(log n) Constant worst-case ratio TSP with triangle-inequalities 3/2 Max Sat PTAS Bin packing FPTAS Makespan on 2 machines 12 4

5 The first Approximation Algorithm (Graham 66)!""# $%& Makespan minimization on' $'identical machines (strongly'n!-hard) 1 - )* $'identical machines '''' +'jobs with lengths', -.', /.'0.', + Objective, smallest # $%& 1 / 1 $ # $%& 13 Algorithm: List-Scheduling (LS) LS: schedule jobs in any given order to the first available (i.e. idle) machine List:'2 -.'2 /'.'2 4'.'2 3'.'2 6'.' / / # $%& 14 Analysis of LS-Algorithm! Define the lower bound LB=max {max p j ;! p j /m}! Starting time of the FINAL job s f : starting time of the job that completes last! Observation (result by LS-Algorithm): 7 # $%& LS = s f +p f! Let E i be the completion (end) time of machine M i 15 5

6 LS Analysis (cont)! LS places the last job in the machine that is mostly available s f! E i (for all other machines i!f) s f = E f - p f! This implies (summing for each machine) that m s f! [" i=1 (E i )] - p f s f! (1/m)([" i=1 E i ]-p f ) =(1/m)([" p j ] -p f )! But C max LS =s f +p f!(1/m) " p j + p f (1-(1/m))! Thus C max LS! [2- (1/m)]Opt 16 LS: Analysis Theorem: LS is a (2-1/m)-approximation algorithm. The approximation ratio is tight. Example: p 1 = p 2 = 1 and p 3 = 2 M 1 J 1 M 2 J 2 J 3 C max =3 M 1 M 2 J 3 J 1 J 2 C max =2 17 Linear Programming based approximation algorithms IDEA ILP relax LP difficult poly time Opt Opt LP A(I) # $ Opt round to integral values 18 6

7 Example: R2 C max R2 C max : Makespan minimization on 2 unrelated machines (weakly NP-hard) I: 2 unrelated machines n jobs Job j has length p 1j on machine M 1 p 2j on machine M 2 M 1 M 2 C max 19 Integer Linear Program (ILP)! We will encode by x ij the fact that job j is placed in machine i! Then the ILP looks as follows Minimize C max Subject to x 1j + x 2j =1, for j=1, n (each jobs is assigned once)! j=1 n p 1j x 1j " C max! j=1 n p 1j x 1j " C max x 1j, x 2j! {0, 1} (it most be on one machine or the other) j=1,,n 20 Linear Program (LP) Relaxation! We will encode by x ij the fact that job j is placed in machine i! Then the ILP looks as follows Minimize C max Subject to x 1j + x 2j =1, for j=1, n (each jobs is assigned once)! j=1 n p 1j x 1j " C max! j=1 n p 1j x 1j " C max x 1j, x 2j " 0 (it most be on some machine or split) j=1,,n 21 7

8 Analysis of the number of fractional jobs! Known: a basic optimal LP solution has the property that the number of variables that get positive values is at most the number of rows in the constraint matrix Thus, there are at most n+2 variables with positive values! Since C max is always positive, at most n+1 of the x ij variables are positive We reduce the value C max of if we make any pair of variables ofr the same job to zero! Every job has at least one positive variable associated with it Because x 1j + x 2j =1! CONCLUSION: At most 1 (ONE) job has been split onto two machines 22 Rounding M 1 M 2 J 1 J 4 J 2 J 3 J 6 J 5 J 5 Opt LP!!Opt J 5 : fractional M 1 M 2 J 1 J 4 J 2 J 3 J 6 J 5 J 5!!Opt ROUNDING!!"Opt 23 How to get a PTAS Input I Algorithm A Output A(I): feasible sol. for I Add structure IDEA:! Add more structure (depending on ") as "##, additional structure # 0! Compare as "##$!%&'""()%*'"(!#&)* 24 8

9 Structuring the Input! I I # difficult poly time Opt Opt # A(I) " (1+!) Opt back in poly time 25 Example!"P2 C max P2 C max Makespan minimization on 2 identical machines (weakly NP-hard) I: 2 identical machines n jobs with lengths p 1, p 2,, p n M1 M 2 Lower bound is again LB=max {max p j ; # p j /2} Thus LB " Opt " 2 LB C max 26 How to round the input! I I # p j >! LB big p j "! LB small! p j# := p j! $S/ (! LB)% jobs of length! LB where S= # small p j 27 9

10 Analysis of Rounded Instance I # Recall that! " # $ " 2&' How many big jobs? a big job has " #$ ( # &' This implies $)big* " 2/# How many conglomerates jobs? ow man1 small #o4s 5 A conglomerate of small jobs has " #$ 6 # &' This implies $)conglomerates* " 2/# LEMMA: The rounded instance has a constant(#) number of jobs. COROLARY: We can find its optimal solution in constant time!! PROOF: Use exhaustive search 28 Back to a feasible solution #&' #&' #&' #&' #&' ;"t $ small #&' #&' #&' #&' #&' "#&' Sum of the small 29 Back to a feasible solution (ctd) #&' #&' #&' #&' #&' ;"t $ #&' #&' #&' #&' #&' C ma> " ;"t $? #&' " )1?#* ;"t $ ;"t $ 30 10

11 How much error is introduced? C max! Opt # + " LB Opt #! Opt + " LB C max! Opt + 2" LB! (1+2") Opt # Wait till next slide 31!pt vs.!pt % (Case 1:LUCKY)M 1 big big... big Opt #! Opt M 2 big... "LB "LB (Case 2:Optimal solution here has to be as good as LS) M 1 M 2 big big big big... "LB... "LB Opt #! C max LS =s f +p f!(1/m) # p j + p f (1-(1/m)) Since m=2, and (1/m) # p j! Opt and p f /2! "LB, we have Opt #! Opt + "LB 32 Structuring the execution of an algorithm IDEA: take an exact but slow algorithm A and interact with it while it is working. Clean-up part of its memory. As a result the algorithm becomes faster (less data to process) and generates incorrect output. IDEAL CASE: the time complexity becomes polynomial and the incorrect output is a good approximation Compare: Tabu search vs. branch & bound 33 11

12 Example: R2 C max Exact Algorithm (Dynamic Program) Encode a partial schedule for the first k jobs by a vector (state) [!, "]! = total processing time on M 1 " = total processing time on M 2 Assign job one by one and update the states appropriately 34 Dynamic Program 1. S 0 #{ [0,0] } 2. For k # 1 to n do 3. S k # {} 4. For all [!, "] in S k-1 do 5. add the two states 6. [! + p 1k, "] and 7. [!, " + p 2k ] to S k 8. End For 9. End For 10. Output min{ max{!, "} [!, "] in S n } 35 (Perturbed) Dynamic Program 1. S 0 #{ [0,0] } 2. For k # 1 to n do 3. S k # {} 4. For all [!, "] in S k-1 do 5. add the two states 6. [! + p 1k, "] and 7. [!, " + p 2k ] to S k 8. End For 9. CLEAN UP S k 10.End For 11.Output min{ max{!, "} [!, "] in S n } 36 12

13 P :=! j max{p 1j, p 2j } State space in PxP box " := (1+#) 1/n ", " 2, " 3,... State Space S k " i " i+1 " i+2 " i+3 " i+4 37 Clean up S k P :=! j max{p 1j, p 2j } State space in PxP box " := (1+#) 1/n ", " 2, " 3,... " i " i+1 " i+2 " i+3 " i+4 38 Running Time The running time is n times the number of boxes. The number of boxes is at most (b+1) 2 where " b = P Therefore b=log(p) / log(") Since " = (1+#) 1/n then b $ n log(p) /log(1+#) $ n log(p) (1+#)/# Polynomial on the size of the unary encoding 39 13

14 How much error is introduced by clean up? Clean up replaces state "!!"# by another state "!"!""# in the same box:!/# $!" $!# "/# $ "" $ "# Therefore, (derivations can show) error $ # There are n clean up phases, and each clean up phase has error # Overall error $ # n = 1(% In-Approximability Techniques $%& '% &e 'i*p,%-e.he e0i*.ence %3 a 5T789 MAIN IDEA: Prove that the existence of a polynomial time approximation algorithm for problem p with worst case guarantee better than! implies the existence of an exact polynomial time algorithm for an NP-hard problem q (which implies P=NP) 42 14

15 Other general techniques! For FPTAS: Strong NP-hardness implies no FPTAS! For PTAS: Gap-technique ==> implies no PTAS MAX-SNP-hardness Implies no PTAS 43 Example to disproof existence of an FPTAS (unless P=NP)!""# $%& Makespan minimization on $ identical machines () $ identical machines * jobs with lengths +, /- + * PROOF: Assume +, /- + * are integers encoded in unary (e.g., +, 01 write: +, 0,,,2 Known:!""# $%& is NP-hard even under unary encoding 3! )0! + 4 polynomial in size of input "! and 5+6 is an integer in 7 8-,-.- / -! 9 44 Continuation (no FPTAS)!"##$%& () *+,-!.$/ & 7$6#8&934:; " %(:; < $ = % ( " $ ##!& ' ( ( ( % B ) 6 + ' & 6(9 15$$%&! > = ()@ 7(88 *+,-!A?+ ( ) B + ( C"))3)D,36& E < ) ( 6 ( + #$8:)$63(8A! :;(2 " ;,<# ;, <,/.!2 5+6 " 5+6 <,/.! This gives :;(2 " 5+6 <, implies :;(205+6 implies!0>!?? 45 15

16 The gap technique IDEA: Let f be a minimization with integral objective function values (costs). Let g be a fixed integer. Assume that the problem of deciding whether an instance of p has a feasible solution with cost at most g is NP-hard. Then p does not have an approximation algorithm with ratio! < (g + 1)/g, unless P=NP. 46 The Gap Technique (ctd) hard to decide if the cost is " & & &'( REDUCTION: Assume for any instance " it exists an a-approximation algorithm # with! $ %& ' ()*&. If,-.%") " &, then #%") "! & $ & ' ( # /01 2 If,-.%") >= & ' (, then #%") >= & ' ( # 3, possible costs of p 47 Example Problem (bin packing or scheduling with deadline) Input: 4 jobs with lengths - (, - 2, 7, - 4 and a hard deadline 8 Goal: find the minimum number 9 of machines on which all jobs can be completed before their deadline

17 Example (ctd) THEOREM. Deciding if "! # is $%-hard& PROOF: (use partition). Gap Technique For the Bin packing problem no " -approximation algorithm is possible with " ' ()# (that is, no PTAS!), unless %*$%&