Homework Set #3 - Solutions

Transcription

1 EE 15 - Applicatios of Covex Optimizatio i Sigal Processig ad Commuicatios Dr. Adre Tkaceko JPL Third Term 11-1 Homework Set #3 - Solutios 1. a) Note that x is closer to x tha to x l i the Euclidea orm sese if ad oly if we have the followig. x x x x l x x x x l x x ) T x x ) x x l ) T x x l ) x T x x T x + x T x x T x x T l x + xt l x l x l x ) T x x T l x l x T x. Note that the last coditio above defies a halfspace for every l. Thus we ca express V as V = {x : Ax b} with x 1 x x 1 x ) x T 1 A =. =. b = x 1 x T x.. x K x x K x ) x T K x K x T x b) Suppose we have P = {x : Ax b} with A R K ad b R K. The as P is assumed to have a oempty iterior we ca choose ay poit x {x : Ax b} ad the costruct K poits x l by takig the mirror image of x with respect to the hyperplaes { x : a T l x = b l}. I other words we choose xl of the form x l = x + λa l where λ is chose i such a way that the distace of x l to the hyperplae defied by a T l x = b l is equal to the distace of x to the hyperplae. This leads to the followig coditio: b l a T l x = a T l x l b l. Substitutig x l = x + λa l ito the above coditio ad solvig for λ yields λ = b l a T l x ) a l. Thus if we choose x l = x + b l a T l x ) a l a l l = 1... K the the polyhedro P is the Vorooi regio of x with respect to x 1... x K. c) A polyhedral decompositio of R ca ot always be described as Vorooi regios geerated by a set of poits {x 1... x m }. A couterexample i R is show below i Figure 1. I this figure R is decomposed ito 4 polyhedra P 1... P 4 by hyperplaes H 1 H. Suppose we arbitrarily pick x 1 P 1 ad x P. The x 3 P 3 must be the mirror image of x 1 ad x with respect to H ad H 1 respectively. However the mirror image of x 1 with respect to H lies i P 1 ad the mirror image of x with respect to H 1 lies i P so it is impossible to fid such a x 3.

2 P 1 P 1 P 4 P 3 P P H H 1 Figure 1: Polyhedral decompositio/vorooi regio partitio couterexample i R.. Recall that a set is covex if ad oly if its itersectio with a arbitrary lie of the form { x + tv : t R} is covex. We will use this property for both parts of this problem. a) Note that we have where we have x + tv) T A x + tv) + b T x + tv) + c = αt + βt + γ α v T Av β b T v + x T Av γ c + b T x + x T A x. The itersectio of C with the lie defied by x ad v is the set { x + tv : αt + βt + γ } which is covex if α. This is true for ay v if v T Av for all v i.e. A. To show that the coverse is false cosider the followig couterexample. Suppose A = 1 b = ad c = 1. The A but we have which is clearly covex. C = { x R : x 1 } = { x R : x + 1 } = R b) Suppose that we defie α β ad γ as i the previous part of the solutio. Note that we ow also have g T x + tv) + h = δt + ɛ where we have δ g T v ɛ g T x + h. Without loss of geerality we ca assume that x H i.e. ɛ =. The itersectio of C H with the lie defied by x ad v is { x + tv : αt + βt + γ δt = }. If δ = g T v the the itersectio is the sigleto { x} if γ or it is empty otherwise. I either case it is a covex set. If δ = g T v = the the set reduces to { x + tv : αt + βt + γ }

3 which is covex if α. Therefore C H is covex if g T v = = v T Av. 1) But this is true if there exists a λ such that A + λgg T ). I this case 1) holds because the we have v T Av = v T A + λgg T ) v for all v satisfyig g T v =. To show the coverse is false cosider the followig couterexample. Suppose we take [ ] [ ] [ ] 1 A = b = c = 1 g = h =. 1 The we clearly have [ λ A + λgg T = 1 for ay choice of λ. I this case though we have ] C = { x 1 x ) R : x 1 } = R ad so we have C H = H = { x 1 x ) R : x 1 = } which is clearly covex. 3. We first ote that the costraits p k k = 1... defie halfspaces ad p k = 1 defies a hyperplae so P is a polyhedro ad hece a covex set. a) The coditio here is α p k fa k ) β which is equivalet to two liear iequalities ad as such is covex i p. b) The coditio here is Pr {X > α} = k:a k >α p k β which is equivalet to a liear iequality ad as such is covex i p. c) The coditio here is equivalet to ) p k a k 3 α a k which is equivalet to a liear iequality ad as such is covex i p. d) The coditio here is p k a k α which is equivalet to a liear iequality ad as such is covex i p. e) The coditio here is p k a k α which is equivalet to a liear iequality ad as such is covex i p.

4 f) The coditio here is VarX) = E [ X ] E[X]) = p k a k p k a k ) α which is ot covex i geeral. As a couterexample take = a 1 = a = 1 ad α = 1 5. The p 1 p ) = 1 ) ad p 1 p ) = 1) are two poits which satisfy VarX) = 1 5 = α but the covex combiatio p 1 p ) = 1 1 ) does ot VarX) = 1 4 > 1 5 = α here). g) The coditio here is p k a k p k a k ) α By defiig the followig quatities: a a 1. a it follows that the coditio is equivalet to l=1 p k a k a l p l A aat b p T Ap b T p + α. a 1. a a k p k + α. However this defies a covex set i.e. it is covex i p sice A = aa T see the results of part a) of the previous problem). h) For emphasize the depedece of the first quartile o p let us deote Q 1 X) = fp). From Figure 1 of the homework set we have fp) = a. Usig this figure it ca be see that the coditio fp) α is equivalet to F X β) < 1 4 for all β < α. If α a 1 this is always true. Otherwise defie k max {l : a l < α}. This is a fixed iteger idepedet of p. The costrait fp) α holds if ad oly if F X a k ) = k p l < 1 4. This is a strict liear iequality i p which defies a ope halfspace. As such it is covex i p. i) Usig the otatio defied i the previous part of the solutio it follows that the coditio fp) α is equivalet to l=1 F X β) 1 4 for all β α. This ca be expressed as a liear iequality as follows: p l 1 4. l=k+1 Here if α a 1 we defie k =. As such this coditio is covex i p.

5 4. a) Followig the hit we have gt) = tr Z + tv) 1) = tr Z 1 ) ) = tr Z 1 I + tz 1 VZ 1 1 Z 1 I + tz 1 VZ 1 ) ) ) Z 1 1 ) ) = tr Z 1 I + tz 1 VZ 1 1. For sake of simplicity let us defie A Z 1 VZ 1 S ad let A = QΛQ T deote a eigevalue decompositio of A where Q is a uitary matrix of eigevectors of A ad Λ = diagλ 1... λ ) is a diagoal matrix of eigevalues of A. Substitutig this ito the expressio above yields gt) = tr Z 1 I + ta) 1) = tr Z 1 I + tqλq T ) ) 1 = tr Z 1 Q I + tλ) Q T ) ) ) 1 = tr Z 1 Q I + tλ) 1 Q T Q = tr T ZQ ) [ I + tλ) 1) Q T ZQ ] kk =. 1 + tλ k Now ote that from the last equality that gt) ca be expressed as a positive weighted 1 sum of covex fuctios 1+tλ k. Hece gt) is covex. b) I this part we will use the same eigevalue decompositio of A Z 1 VZ 1 = QΛQ T as was doe i the previous part. Here we have ) )) gt) = detz + tv)) 1 = det Z 1 I + tz 1 VZ 1 Z 1 1 ) = det Z 1 det I + tqλq T ) )) det Z 1 1 = detz)) 1 det I + tqλq T ) ) detz)) 1 1 = detz) det Q I + tλ) Q T )) 1 = detz)) 1 detq) deti + tλ) det Q T )) 1 = detz)) 1 deti + tλ)) 1 ) 1 = detz)) tλ k ). From the last equality we have show that gt) ca be expressed as a product of a positive costat ad the geometric mea of 1 + tλ k ) for k = 1... which is cocave i t. Hece the et result is that gt) is cocave. *5. First ote that we have ) P p C) = 1 P p C where C {1... } \ C is the complemet of C i the set {1... }. Hece we have ) ) ) )) P p C) P q C) = P p C + P q C = P p C P q C

6 ad so d mp p q) ca be expressed equivaletly as d mp p q) = max {P p C) P q C) : C {1... }}. As d mp p q) is the maximum of liear fuctios of p q) sice the umber of evets i the set S {1... } is the cardiality of the power set PS) which is ) it follows that d mp p q) is covex. To simplify the expressio for d mp p q) let us idetify a subset C that maximizes P p C) P q C) = k C We claim that such a optimal subset is give by p k q k ). ) C {k {1... } : p k > q k }. 3) To show this first ote that the idices for which p k = q k clearly do ot matter. Thus we will igore these idices ad assume without loss of geerality that for each idex either p k > q k or p k < q k. I other words we eed oly compute ) over relevat idices for which either p k > q k or p k < q k. Now cosider ay other subset C. If there is a elemet l i C but ot C the by addig l to C we icrease P p C) P q C)) by p l q l ) > ad so C could ot have bee optimal. Coversely suppose that l C \ C so p l q l ) <. If we remove l from C we would icrease P p C) P q C)) by q l p l ) > ad so C could ot have bee optimal. This proves that C as defied i 3) is optimal for maximizig the expressio i ) ad so we have d mp p q) = p k q k ). 4) k:p k >q k To express d mp p q) i terms of p q 1 ote that we have p k q k )+ p k q k ) = p k q k ) = p k q k = 1 T p 1 T q = 1 1 = k:p k >q k k:p k q k ad so k:p k >q k p k q k ) = k:p k q k p k q k ). Substitutig this ito 4) yields the followig: d mp p q) = 1 p k q k ) + 1 p k q k ) k:p k >q k k:p k >q k = 1 p k q k ) 1 p k q k ) k:p k >q k k:p k q k = 1 p k q k. Hece we coclude d mp p q) = 1 p q 1. From this it is very clear that d mp p q) is covex. The best way to iterpret this result is as a iterpretatio of the l 1 -orm for probability distributios. It states that the l 1 -distace betwee two probability distributios is twice the maximum differece i probability over all evets of the distributios.