Outline. CS38 Introduction to Algorithms 5/23/2014. Linear programming. Lecture 16 May 22, coping with intractibility

Transcription

1 Outlie CS38 Itroductio to Algorithms Lecture 6 Ma, 04 Liear programmig L dualit ellipsoid algorithm * slides from Kevi Wae copig with itractibilit N-completeess Ma, 04 CS38 Lecture 6 Ma, 04 CS38 Lecture 6 L Dualit L Duals rimal problem. () ma 3A 3B s. t. 5A 5B 480 4A 4B 60 35A 0B 90 A, B 0 Caoical form. () ma c T mi T b s. t. A T c 0 Idea. Add oegative combiatio (C, H, M) of the costraits s.t. 3A 3B (5C 4H 35M ) A (5C 4H 0M ) B 480C 60H 90M Dual problem. Fid best such upper boud. mi 480C 60H 90M s. t. 5C 4H 35M 3 5C 4H 0M 3 C, H, M Double Dual Takig Duals Caoical form. L dual recipe. () ma c T mi T b s. t. A T c 0 rimal () maimie miimie Dual costraits a = b i a b a b i i urestricted i 0 i 0 variables ropert. The dual of the dual is the primal. f. Rewrite as a maimiatio problem i caoical form; take dual. variables j 0 j 0 urestricted a T c j a T c j a T = c j costraits (D' ) ma T b s. t. A T c 0 (DD) mi c T s. t. (A T ) T b 0 f. Rewrite L i stadard form ad take dual. 5 6

2 L Strog Dualit Theorem. [Gale-Kuh-Tucker 95, Datig-vo Neuma 947] For A R m, b R m, c R, if () ad are oempt, the ma = mi. Strog dualit () ma c T mi T b s. t. A T c 0 Geeralies: Dilworth's theorem. Köig-gervar theorem. Ma-flow mi-cut theorem. vo Neuma's miima theorem. Ma, 04 CS38 Lecture 6 7 f. [ahead] 8 L Weak Dualit rojectio Lemma Theorem. For A R m, b R m, c R, if () ad are oempt, the ma mi. Weierstrass' theorem. Let X be a compact set, ad let f() be a cotiuous fuctio o X. The mi { f() : X } eists. () ma c T mi T b s. t. A T c 0 rojectio lemma. Let X ½ R m be a oempt closed cove set, ad take ot i X. The there eists * X with miimum distace from. Moreover, for all X we have ( *) T ( *) 0. obtuse agle f. Suppose R m is feasible for () ad R is feasible for. 0, A b ) T A T b 0, A T c ) T A c T Combie: c T T A T b. * X 9 0 rojectio Lemma rojectio Lemma Weierstrass' theorem. Let X be a compact set, ad let f() be a cotiuous fuctio o X. The mi { f() : X } eists. Weierstrass' theorem. Let X be a compact set, ad let f() be a cotiuous fuctio o X. The mi { f() : X } eists. rojectio lemma. Let X ½ R m be a oempt closed cove set, ad take ot i X. The there eists * X with miimum distace from. Moreover, for all X we have ( *) T ( *) 0. rojectio lemma. Let X ½ R m be a oempt closed cove set, ad take ot i X. The there eists * X with miimum distace from. Moreover, for all X we have ( *) T ( *) 0. f. Defie f() =. f. * mi distace ) * for all X. Wat to appl Weierstrass, but X ot ecessaril bouded. X ot empt ) there eists X. Defie X' = { X : ' } so that X' is closed, bouded, ad mi { f() : X } = mi { f() : X' }. B Weierstrass, mi eists. * X' ' X B coveit: if X, the * + ² ( *) X for all 0 < ² <. * * ² ( *) = * + ² ( *) ² ( *) T ( - *) Thus, ( *) T ( - *) ½ ² ( *). Lettig ²! 0 +, we obtai the desired result.

3 Separatig Hperplae Theorem Farkas' Lemma Theorem. Let X ½ R m be a oempt closed cove set, ad take ot i X. The there eists a hperplae H = { R m : a T = } where a R m, R that separates from X. a T for all X a T < Theorem. For A R m, b R m eactl oe of the followig two sstems holds: (I) (II) m s. t. A T 0 T b 0 f. Let * be closest poit i X to. f. [ot both] Suppose satisfies (I) ad satisfies (II). B projectio lemma, ( *) T ( *) 0 for all X Choose a = * ot equal 0 ad = a T *. If X, the a T ( *) 0; * The 0 > T b = T A 0, a cotradictio. f. [at least oe] Suppose (I) ifeasible. We will show (II) feasible. Cosider S = { A : 0 } ad ote that b ot i S. thus ) a T a T * =. Let R m, R be a hperplae that separates b from S: Also, a T = a T (* a) = a < T b <, T s for all s S. 0 S ) 0 ) T b < 0 H = { R m : a T = } X T A for all 0 ) T A 0 sice ca be arbitraril large. 3 4 Aother Theorem of the Alterative L Strog Dualit Corollar. For A R m, b R m eactl oe of the followig two sstems holds: (I) (II) m s. t. A T 0 T b 0 0 Theorem. [strog dualit] For A R m, b R m, c R, if () ad are oempt the ma = mi. () ma c T mi T b s. t. A T c 0 f. [ma mi] Weak L dualit. f. [mi ma] Suppose ma <. We show mi <. f. Appl Farkas' lemma to: (I' ), s m s. t. A I s b, s 0 (II' ) m s. t. A T 0 I 0 T b 0 (I) c T (II) m, s. t. A T c 0 T b 0, 0 B defiitio of, (I) ifeasible ) (II) feasible b Farkas' Corollar. 5 6 L Strog Dualit (II) Let, be a solutio to (II). m, s. t. A T c 0 T b 0, 0 llipsoid algorithm Case. [ = 0] The, { R m : A T 0, T b < 0, 0 } is feasible. Farkas Corollar ) { R : A b, 0 } is ifeasible. Cotradictio sice b assumptio () is oempt. Case. [ > 0] Scale, so that satisfies (II) ad =. Resultig feasible to ad T b <. Ma, 04 CS38 Lecture

4 To fid a poit i : Geometric Divide-ad-Coquer To fid a poit i : Geometric Divide-ad-Coquer Maitai ellipsoid cotaiig. 9 0 To fid a poit i : Geometric Divide-ad-Coquer Maitai ellipsoid cotaiig. If ceter of ellipsoid is i stop; otherwise fid hperplae separatig from. ad cosider correspodig half-ellipsoid ½ = Å H To fid a poit i : Geometric Divide-ad-Coquer Maitai ellipsoid cotaiig. If ceter of ellipsoid is i stop; otherwise fid hperplae separatig from. Fid smallest ellipsoid ' cotaiig half-ellipsoid. separatig hperplae L-J ellipsoid separatig hperplae H H ' Geometric Divide-ad-Coquer Optimiatio to Feasibilit To fid a poit i : Maitai ellipsoid cotaiig. If ceter of ellipsoid is i stop; otherwise fid hperplae separatig from. Fid smallest ellipsoid ' cotaiig half-ellipsoid. Repeat. Stadard form. ma c T ' A b form., A b A T c c T b T 0 A b 0 dual feasible optimal 3 4 4

5 llipsoid Algorithm Shrikig Lemma Goal. Give A R m ad b R m, fid R such that A b. llipsoid. Give D R positive defiite ad R, the { : ( ) T D ( ) } llipsoid algorithm. Let 0 be a ellipsoid cotaiig. k = 0. eumerate costraits While ceter k of ellipsoid k is ot i : fid a costrait, sa a, that is violated b k let k+ be mi volume ellipsoid cotaiig k Å { : a a k } k = k + eas to compute half-ellipsoid ½ a a a k k+ is a ellipsoid cetered o with vol() = det vol(b(0, )) uit sphere Ke lemma. ver half-ellipsoid ½ is cotaied i a ellipsoid ' with vol( ) / vol() e /(+). H ' ½ k k 5 6 Shrikig Lemma: Uit Sphere Shrikig Lemma: Uit Sphere Special case. = uit sphere, H = { : 0 }. { : ( i ) i } { : ( ) ( i ) i } Special case. = uit sphere, H = { : 0 }. { : ( i ) i } { : ( ) ( i ) i } Claim. ' is a ellipsoid cotaiig ½ = Å H. f. If ½ : Claim. ' is a ellipsoid cotaiig ½ = Å H. f. Volume of ellipsoid is proportioal to side legths: i i i i i i ( ) i i 0 0 i 0 ½ ' vol( ) vol() e e e ( ) + e 0 ½ ' 7 8 Shrikig Lemma Shrikig Lemma Shrikig lemma. The mi volume ellipsoid cotaiig the half-ellipsoid ½ = Å { : a a } is defied b: { Da a T Da, D D : ( ) T (D ) ( ) } Daa T D a T Da Shrikig lemma. The mi volume ellipsoid cotaiig the half-ellipsoid ½ = Å { : a a } is defied b: { Da a T Da, D D : ( ) T (D ) ( ) } Daa T D a T Da Moreover, vol( ) / vol() < e /(+). ' H Moreover, vol( ) / vol() < e /(+). ½ f sketch. We proved = uit sphere, H = { : 0 } llipsoids are affie trasformatios of uit spheres. Volume ratios are preserved uder affie trasformatios. Corollar. llipsoid algorithm termiates after at most (+) l (vol( 0 ) / vol()) steps

6 llipsoid Algorithm Theorem. Liear rogrammig problems ca be solved i polomial time. f sketch. Shrikig lemma. Set iitial ellipsoid 0 so that vol( 0 ) cl. erturb A b to A b + ) either is empt or vol() -cl. Bit compleit (to deal with square roots). urif to verte solutio. Copig with itractabilit Caveat. This is a theoretical result. Do ot implemet. O(m 3 L) arithmetic ops o umbers of sie O(L), where L = umber of bits to ecode iput Ma, 04 CS38 Lecture Decisio problems + laguages A problem is a fuctio: f:σ * Σ * Simple. Ca we make it simpler? Yes. Decisio problems: f:σ * {accept, reject} Does this still capture our otio of problem, or is it too restrictive? Decisio problems + laguages ample: factorig: give a iteger m, fid its prime factors f factor : {0,} * {0,} * Decisio versio: give itegers m,k, accept iff m has a prime factor p < k Ca use oe to solve the other ad vice versa. True i geeral. Ma, 04 CS38 Lecture 6 33 Ma, 04 CS38 Lecture 6 34 Decisio problems + laguages For most compleit settigs a problem is a decisio problem: f:σ * {accept, reject} quivalet otio: laguage L Σ * the set of strigs that map to accept ample: L = set of pairs (m,k) for which m has a prime factor p < k Ma, 04 CS38 Lecture 6 35 Search vs. Decisio Defiitio: give a graph G = (V, ), a idepedet set i G is a subset V V such that for all u,w V (u,w) A problem: give G, fid the largest idepedet set This is called a search problem searchig for optimal object of some tpe comes up frequetl Ma, 04 CS38 Lecture

7 Search vs. Decisio We wat to talk about laguages (or decisio problems) Most search problems have a atural, related decisio problem b addig a boud k ; for eample: search problem: give G, fid the largest idepedet set decisio problem: give (G, k), is there a idepedet set of sie at least k Ma, 04 CS38 Lecture 6 37 The class N Defiitio: TIM(t()) = {L : there eists a TM M that decides L i time O(t())} = k TIM( k ) Defiitio: NTIM(t()) = {L : there eists a NTM M that decides L i time O(t())} N = k NTIM( k ) Ma, 04 CS38 Lecture 6 38 ol-time verifiers N = {L : L decided b pol-time NTM} Ver useful alterate defiitio of N: Theorem: laguage L is i N if ad ol if it is epressible as: L = { 9, k, (, ) R } where R is a laguage i. pol-time TM M R decidig R is a verifier Ma, 04 CS38 Lecture 6 39 ol-time verifiers N = {L : L decided witess b pol-time or NTM} certificate Ver useful alterate defiitio of efficietl N: Theorem: laguage L is i N if ad verifiable ol if it is epressible as: L = { 9, k, (, ) R } where R is a laguage i. pol-time TM M R decidig R is a verifier Ma, 04 CS38 Lecture 6 40 ol-time verifiers ample: 3SAT epressible as 3SAT = {φ : φ is a 3-CNF formula for which assigmet A for which (φ, A) R} R = {(φ, A) : A is a sat. assig. for φ} satisfig assigmet A is a witess of the satisfiabilit of φ (it certifies satisfiabilit of φ) R is decidable i pol-time ol-time reductios Tpe of reductio we will use: ma-oe pol-time reductio A es o f f es o B reductio from laguage A to laguage B Ma, 04 CS38 Lecture 6 4 Ma, 04 CS38 Lecture 6 4 7

8 ol-time reductios A es o es fuctio f should be pol-time computable Defiitio: f : Σ* Σ* is pol-time computable if for some g() = O() there eists a g()-time TM M f such that o ever w Σ*, M f halts with f(w) o its tape. Ma, 04 CS38 Lecture 6 43 f f o B ol-time reductios Defiitio: A B ( A reduces to B ) if there is a pol-time computable fuctio f such that for all w w A f(w) B coditio equivalet to: YS maps to YS ad NO maps to NO meaig is: B is at least as hard (or epressive) as A Ma, 04 CS38 Lecture 6 44 ol-time reductios Theorem: if A B ad B the A. roof: a pol-time algorithm for decidig A: o iput w, compute f(w) i pol-time. ru pol-time algorithm to decide if f(w) B if it sas es, output es if it sas o, output o Hardess ad completeess Reasoable that ca efficietl trasform oe problem ito aother. Surprisig: ca ofte fid a special laguage L so that ever laguage i a give compleit class reduces to L! powerful tool Ma, 04 CS38 Lecture 6 45 Ma, 04 CS38 Lecture 6 46 Hardess ad completeess Recall: a laguage L is a set of strigs a compleit class C is a set of laguages Defiitio: a laguage L is C-hard if for ever laguage A C, A pol-time reduces to L; i.e., A L. meaig: L is at least as hard as athig i C Hardess ad completeess Recall: a laguage L is a set of strigs a compleit class C is a set of laguages Defiitio: a laguage L is C-complete if L is C-hard ad L C meaig: L is a hardest problem i C Ma, 04 CS38 Lecture 6 47 Ma, 04 CS38 Lecture

9 Lots of N-complete problems logic problems 3-SAT = {φ : φ is a satisfiable 3-CNF formula} NA3SAT, (3,3)-SAT Ma--SAT fidig objects i graphs idepedet set verte cover clique sequecig Hamilto ath Hamilto Ccle ad TS problems o umbers subset sum kapsack partitio splittig thigs up ma cut mi/ma bisectio Ma, 04 CS38 Lecture