Khintchine-Type Inequalities and Their Applications in Optimization

Transcription

1 Khntchne-Type Inequaltes and The Applcatons n Optmzaton Anthony Man-Cho So Depatment of Systems Engneeng & Engneeng Management The Chnese Unvesty of Hong Kong ISDS-Kolloquum Unvestaet Wen 29 June 2009

2 Backgound A cental queston n pobablty theoy s to undestand the behavo of a sum of ndependent andom vaables. Moment nequaltes Tal nequaltes Many applcatons Packet outng Appoxmate countng Randomzed oundng

3 Backgound Consde the followng vey smple settng. Let a be a vecto of eal numbes { ξ } be..d.. { ± } RVs What can you say about ξ a? In 923, Khntchne poved that: p E ξ a K p a 2 Consdeable effots have been focused on fndng the best constant K p. Haageup (982) showed that ( p / 2 K ) p = Θ p. p

4 Khntchne s Inequalty The above moment nequalty s known as Khntchne s nequalty. Note that an applcaton of Makov s s nequalty mmedately gves a tal bound: P ( ) ( ) p p p p ξ a > t = P ξ a > t t E ξ a The nequalty has snce been extended n many dectons, fo example: each a s a vecto each a s an element n some Banach space Ths gves se to many Khntchne-type type nequaltes.

5 Khntchne s Inequalty In ths talk we ae nteested n the followng settng: Q,,Q h ae m n eal matces { ξ } be..d.. { ± } RVs What can we say about ξ Q? Specfcally, can we bound the spectal nom (.e. the lagest sngula value) of ξ Q? Such a poblem ases n the analyses of many optmzaton poblems.

6 Khntchne s Inequalty The fom of Khntchne s nequalty suggests that we may want to look fo an nequalty of the fom: p 2 p E ξ Q K Q / S p ( ) 2 fo some appopate constant K p. Howeve, as we shall see, we may want some othe nomalzatons on the RHS as well. p S p

7 Applcaton : QP wth Nom Constant Consde the followng poblem: (P) whee: max s.t. A B C = 0 M m, n M m,n s the space of m-by-n matces equpped wth T T the Fobenus nne poduct Y = t Y = t Y A, B ae symmetc lnea mappngs, wth B psd C s a lnea mappng s the spectal nom (lagest sngula value) of fo =, K, L ( ) ( )

8 Motvaton Poblem (P) ases n many applcatons. The Pocustes Poblem: Gven: K collectons P,,P,P K of ponts n R n of the same cadnalty, say m. Goal: : Fnd otatons,,, K that make these collectons as close to each othe as possble. Mathematcally, we want to: m mn Al j A jl max < j K l = 2 2 T T t ( A j A j ) < j K T s.t.. = I fo =,,K,K.. Ths can be put nto the fom (P).

9 Pocustes: : A Geek Legend In Geek mythology, Pocustes was a bandt fom Attca who clamed that he had an on bed that fts eveyone. Howeve, f a guest was too shot, he would stetch hm by hammng o ackng the body to ft; f a guest was too tall, he would amputate the excess length. In ethe event, the guest ded. Eventually, Pocustes was made to taste hs own medcne by the Attc heo Theseus.

10 A Related Poblem A closely elated poblem, namely that wthout the nom constant, s petty well undestood. An O ( log L )-appoxmaton (whee L = no. of constants) s possble usng SDP elaxaton (cf. Nemovsk et al. 999). Many applcatons: clusteng, sgnal pocessng, etc. max s.t. A B C = 0 M m, n

11 A Related Poblem A closely elated poblem, namely that wthout the nom constant, s petty well undestood. An O ( log L )-appoxmaton (whee L = no. of constants) s possble usng SDP elaxaton (cf. Nemovsk et al. 999). Many applcatons: clusteng, sgnal pocessng, etc. max s.t. A B C = 0 Queston: Does a smla esult hold fo (P)? M m, n

12 An SDP Relaxaton (Nemovsk 07) The lnea mappngs A, B, C can be dentfed wth matces of the appopate dmenson. max s.t. A Y A B Y B C = 0 C Y = 0 Y a M m, n Gam matx Y S mn

13 An SDP Relaxaton (Nemovsk 07) The lnea mappngs A, B, C can be dentfed wth matces of the appopate dmenson. The nom constant s equvalent to: T I These can be expessed as LMIs usng appopate lnea mappngs. T I max s.t. A Y A B Y B C = 0 C Y = SY m, n M T Y Y a Gam matx Y S mn 0 I I

14 An SDP Relaxaton (Nemovsk 07) We now have the followng poblem: (P ) max s.t. B C Y = SY I T Y I Y A Y Y a Gam matx The standad move now s to elax the Gam matx constant to an psd constant. 0

15 An SDP Relaxaton (Nemovsk 07) We now have the followng poblem: (SDP) max s.t. B C Y = SY I T Y I Y A Y Y a Gam Y matx The standad move now s to elax the Gam matx constant to an psd constant. Note that whle S Y I and TY I ae edundant n (P ), they ae NOT edundant n (SDP). 0 0, Y S mn

16 Qualty of SDP Relaxaton So how well does (SDP) do? Nemovsk (2007) poved that ana ( {( ) }) 3 O max m + n /, log L appoxmaton s possble. He also conjectued that an O ( log max { m, n, L} ) He also conjectued that an appoxmaton should be achevable. Obsevaton (S. 2008): Nemovsk s conjectue s tue. The poof eles on cetan Khntchne-type type nequaltes.

17 Roundng the SDP Soluton A standad way of geneatng a soluton ˆ to (P) * fom a soluton Y to (SDP) s va andomzaton. Specfcally: * based on A extact fom Y a set of vectos { v, K, v mn } { ξ,, } geneate a Benoull andom vecto fom the (andom) vecto fom the (andom) vecto ζ = mn = K ξ mn and de-vectoze t to obtan a canddate soluton matx ˆ ξ v

18 Qualty of ˆ It s not had to show that: * ˆ A ˆ = and C ˆ = 0 v sdp follows fom constucton (P) max s.t. A B C = 0 M m, n

19 Qualty of ˆ It s not had to show that: * ˆ A ˆ = v sdp and C ˆ = 0 Thus, to analyze the qualty of ˆ, t emans to bound the followng quanttes: ( ) 2 P ˆ B ˆ and ( ) > t P ˆ > t If these ae small, then we can asset that ˆ / t s a feasble * 2 soluton to (P) of value v sdp t wth constant pobablty. (P) max s.t. * 2 / v / t A B C = 0 M m, n

20 Outlne of the Appoach Those two tal pobabltes can be estmated usng Khntchne-type type nequaltes. Fst, the poblem of boundng ( ) 2 P ˆ B ˆ > t can be shown to be equvalent to the followng: { ξ } { ± } { } Let be..d.. RVs. Let w be vectos 2 satsfyng w 2. Detemne an uppe bound on P ξ t. ( ) w 2 a nomalzaton condton

21 Outlne of the Appoach On the othe hand, the poblem of boundng ( ) P ˆ > t s equvalent to the followng: { } { ± } { } Let ξ be..d.. RVs. Let Q be m by n matces satsfyng Q Q T T I and Q Q I. Detemne an uppe bound on P t. ( ) ξ Q fom the constants SY I, T Y I

22 Tool: Khntchne-type type Inequaltes Fo the fst poblem { ξ } { ± } { } Let be..d.. RVs. Let w be abtay vectos. Theoem (Tomczak( Tomczak-Jaegemann 974): E ξ w p 2 p p / 2 ( ) 2 p w / 2 Note that the bound s ndependent of the numbe of vectos n the collecton! Coollay: : Let T=max {m,n,l}.. Then, = P P O ( ˆ B ˆ > Ω ( β log T )) ( ξ w > Ω ( β log T ) β ( T ) 2 2

23 Tool: Khntchne-type type Inequaltes On the othe hand { } { ± } { } Let ξ be..d.. RVs. Let Q be m by n matces satsfyng Q Q T T I and Q Q I. Theoem (Lust-Pquad 986, Pse 998, Buchholz 200): p p / 2 E Q p max m, n ξ { } Usng the fact that S S, we obtan: S p Coollay: : Let T=max {m,n,l}.. Then, = P P O S p ( ( ) ˆ > Ω β log T ( ξ Q > Ω ( β log T ) β ( T )

24 Puttng the Peces Togethe By pckng β appopately, the above esult shows that the oundng scheme of Nemovsk (2007) actually poduces a feasble soluton ˆ to (P) that s wthn a logathmc facto fom the optmum.

25 Applcaton 2: Chance-Constaned Constaned LMIs Consde the followng chance-constaned constaned optmzaton poblem: (P) mn s.t. c F x T P x ( x ) 0 A ( x ) ξ A ( x ) [ 0] 0 ε n R F s an effcently computable vecto-valued valued functon wth convex components; each A maps x nto a symmetc matx; we assume that A ( x ) 0 fo all x. 0 > Such a poblem ases, e.g., n contol theoy and s n geneal ntactable.

26 A Safe Tactable Appoxmaton One appoach fo pocessng (P) s the so-called safe tactable appoxmaton,,.e. a system H of constants such that: x s feasble fo (P) wheneve t s feasble fo H the constants n H ae effcently computable To develop H,, obseve that: [ A ( x) ξ A ( x) 0] P I A '( x) whee: P 0 ξ A ' / 2 / 2 ( x ) = A ( x ) A ( x ) A ( x ) 0 [ I ] 0

27 A Safe Tactable Appoxmaton Now, usng a matx veson of Khntchne s nequalty, one can show that fo nce ξ : P ( ξ '( x ) ) ε A wheneve: 2 (*) ( A '( x )) O I ln ( / ε ) The upshot of (*) s that t can be wtten as an LMI: γa0 ( x ) A ( x ) L Ah ( x ) ( ) ( ) ( ) A x γa0 x ( ) Z x 0, γ O M O ln / ε A ( x ) γa ( x ) h 0

28 A Safe Tactable Appoxmaton Thus, we obtan the followng safe tactable appoxmaton of (P): mn s.t. c F Z x T x ( x ) ( x ) R n 0 0

29 Concluson Moment nequaltes ae vey useful n analyzng andomzed algothms. SDP ank educton algothm bounds on stochastc optmzaton poblems analyss of SDP-based detecto fo MIMO channels Fnd moe applcatons!

30 Thank You!