SELECTED SOLUTIONS, SECTION 4.3 1. Weak dualty Prove that the prmal and dual values p and d defned by equatons 4.3. and 4.3.3 satsfy p d. We consder an optmzaton problem of the form The Lagrangan for ths problem s fx : gx 0, x E }. Lx; λ = fx λ T gx, and the prmal and dual values d are defned, respectvely, as We have to show that p = sup Lx; λ, λ R m d = sup Lx; λ. λ R m p d. Wthout loss of generalty, we may assume that p < else ths nequalty s trval. Then the defnton of p mples that for every q > p there exsts some x E wth sup Lx; λ < q. λ R m In other words: For every q > p there exsts some x q E such that Lx q ; λ < q for all λ R m. d = sup Lx; λ sup Lx q ; λ < q. λ R m λ R m d < q for all q > p, whch mples that d p. 1 3. Slater and compactness Prove the Slater condton holds for problem 4.3.1 f and only f there s a pont ˆx E for whch the level sets λ R m : Lˆx; λ α } are compact. Note that the level sets of Lˆx; are always closed because L s contnuous n λ. the compactness of the level sets s equvalent to ther boundedness. Now recall that the Slater condton stated that there exsts ˆx dom f wth g ˆx < 0 for all. Assume that ths condton holds, and let α R. Let moreover λ R m satsfy Lˆx; λ α. Then, usng the fact that λ 0 for all and g ˆx < 0 for all, we obtan α fˆx λ g ˆx λ max g ˆx. Date: November 3, 015. 1
SELECTED SOLUTIONS, SECTION 4.3 As a consequence, λ α fˆx max g ˆx. the set gven n 1 s bounded. Conversely, assume that there exsts ˆx E such that the set gven n 1 s bounded for every α. Then, for each, the set of λ 0 such that fˆx λg ˆx α s bounded, whch mples that fˆx > ˆx dom f. Replacng now α by ˆα := α fˆx, we obtan the set of λ 0 such that λ g ˆx ˆα s bounded for all. Ths, however, s only possble f g ˆx < 0 for all, Slater s condton holds. 4. Examples of duals Lots and lots of them. The followng examples are orgnally stated wth condtons of the form a, x b for = 1,,..., m. I have rewrtten these nequaltes as Ax b, where A R n m denotes the matrx wth rows equal to the transposes of the vectors a. Also, the computatons soon become a bt repettve. a The lnear program We have b Another lnear program } c, x : Ax b. Lx; λ = c, x λ, Ax b Φλ = x Rnc, x λ, Ax b = λ, b c A λ, x f c A λ 0, = λ, b f c A λ = 0. Lx, λ = c, x δr n x : Ax b }. c, x λ, Ax b f x 0, f x 0. Φλ = c, x λ, Ax b x 0 = λ, b c x 0 A λ, x λ, b f c A λ 0, = f c A λ 0. c The quadratc program for C n } : Ax b. Lx; λ = xt Cx λ, Ax b.
SELECTED SOLUTIONS, SECTION 4.3 3 Φλ = λ, Ax b = λ, b A λ, x. Snce C S, n the mnmzaton problem to be solved here s convex and the mnmum s attaned at the pont where the gradent wth respect to x s zero. That s, at x satsfyng or and Cx A λ = 0, x = C 1 A λ. A λ, x = λt AC 1 A λ Φλ = λ, b λt AC 1 A λ. d The separable problem n } px j : Ax b. Lx; λ = Φλ = px j λ, Ax b = b, λ = b, λ = b, λ = b, λ = b, λ = b, λ px j λ, Ax b px j A λ, x pxj A λ j x j pt A λ j t t R sup t R A λ j t pt p A λ j p a j, λ, where a j denotes the j-th columns of the matrx A. e The penalzed lnear program } c, x ε lbx : Ax b.
4 SELECTED SOLUTIONS, SECTION 4.3 Recall that lbx = log x for x > 0. Φλ = c, x ε lbx λ, Ax b x>0 = λ, b c x>0 A λ, x ε lbx c A λ/ε, x lbx = λ, b ε sup x>0 = λ, b ε lb c A λ/ε Proposton 3.3.3 computaton of lb mples that λ, b ε lb c A λ/ε εn f c A λ > 0, Φλ = f c A λ 0. f The semdefnte program C, X δs n X : A, X b for = 1,,..., m }. Φλ = C, X = λ, b λ A, X b =1 C λ A, X. Now recall that S n s a self-dual cone, that s, N S n 0 = S n see Exercse.1.3., f C λ A S n, then C λ A, X 0 for all X S n C λ A, X = 0. On the other hand, f C λ A S n, then there exsts ˆX S n such that C λ A, ˆX > 0. C λ A, X C t>0 λ A, t ˆX =. Φλ = λ, b f C λ A S n, f C λ A S n.
SELECTED SOLUTIONS, SECTION 4.3 5 g The penalzed semdefnte program C, X ε ldx : A, X b for = 1,,..., m }. Φλ = C, X ε ldx = λ, b = λ, b ε sup = λ, b ε ld C C C λ A, X b λ A, X ε ldx λ A /ε, X ldx λ A /ε. Now Proposton 3.3.3 mples that Φλ = λ, b ε ld C λ A /ε εn. 9. Fenchel and Lagrangan dualty Let Y be a Eucldean space. By sutably rewrtng the prmal Fenchel problem } fx gax for gven functons f : E, ], g : Y, ], and lnear A: E Y, nterpret the dual Fenchel problem f A φ g φ } sup φ Y as a Lagrangan dual problem. We rewrte the problem as the constrant optmzaton problem fx gy} subject to ± Ax y 0. The Lagrangan of ths problem s Lx, y; λ = fx gy λ, Ax y µ, y Ax. Now the Lagrangan dual problem s sup fx gy λ µ, Ax y λ, µ Y = sup λ, µ Y fx µ λ, Ax gy λ µ, y = sup sup A µ λ, x fx sup λ µ, y gy λ, µ Y = sup f A µ λ g λ µ λ, µ Y = sup φ Y f A φ g φ.