Khntchne-Type Inequaltes and The Applcatons n Optmzaton Anthony Man-Cho So Depatment of Systems Engneeng & Engneeng Management The Chnese Unvesty of Hong Kong ISDS-Kolloquum Unvestaet Wen 29 June 2009
Backgound A cental queston n pobablty theoy s to undestand the behavo of a sum of ndependent andom vaables. Moment nequaltes Tal nequaltes Many applcatons Packet outng Appoxmate countng Randomzed oundng
Backgound Consde the followng vey smple settng. Let a be a vecto of eal numbes { ξ } be..d.. { ± } RVs What can you say about ξ a? In 923, Khntchne poved that: p E ξ a K p a 2 Consdeable effots have been focused on fndng the best constant K p. Haageup (982) showed that ( p / 2 K ) p = Θ p. p
Khntchne s Inequalty The above moment nequalty s known as Khntchne s nequalty. Note that an applcaton of Makov s s nequalty mmedately gves a tal bound: P ( ) ( ) p p p p ξ a > t = P ξ a > t t E ξ a The nequalty has snce been extended n many dectons, fo example: each a s a vecto each a s an element n some Banach space Ths gves se to many Khntchne-type type nequaltes.
Khntchne s Inequalty In ths talk we ae nteested n the followng settng: Q,,Q h ae m n eal matces { ξ } be..d.. { ± } RVs What can we say about ξ Q? Specfcally, can we bound the spectal nom (.e. the lagest sngula value) of ξ Q? Such a poblem ases n the analyses of many optmzaton poblems.
Khntchne s Inequalty The fom of Khntchne s nequalty suggests that we may want to look fo an nequalty of the fom: p 2 p E ξ Q K Q / S p ( ) 2 fo some appopate constant K p. Howeve, as we shall see, we may want some othe nomalzatons on the RHS as well. p S p
Applcaton : QP wth Nom Constant Consde the followng poblem: (P) whee: max s.t. A B C = 0 M m, n M m,n s the space of m-by-n matces equpped wth T T the Fobenus nne poduct Y = t Y = t Y A, B ae symmetc lnea mappngs, wth B psd C s a lnea mappng s the spectal nom (lagest sngula value) of fo =, K, L ( ) ( )
Motvaton Poblem (P) ases n many applcatons. The Pocustes Poblem: Gven: K collectons P,,P,P K of ponts n R n of the same cadnalty, say m. Goal: : Fnd otatons,,, K that make these collectons as close to each othe as possble. Mathematcally, we want to: m mn Al j A jl max < j K l = 2 2 T T t ( A j A j ) < j K T s.t.. = I fo =,,K,K.. Ths can be put nto the fom (P).
Pocustes: : A Geek Legend In Geek mythology, Pocustes was a bandt fom Attca who clamed that he had an on bed that fts eveyone. Howeve, f a guest was too shot, he would stetch hm by hammng o ackng the body to ft; f a guest was too tall, he would amputate the excess length. In ethe event, the guest ded. Eventually, Pocustes was made to taste hs own medcne by the Attc heo Theseus.
A Related Poblem A closely elated poblem, namely that wthout the nom constant, s petty well undestood. An O ( log L )-appoxmaton (whee L = no. of constants) s possble usng SDP elaxaton (cf. Nemovsk et al. 999). Many applcatons: clusteng, sgnal pocessng, etc. max s.t. A B C = 0 M m, n
A Related Poblem A closely elated poblem, namely that wthout the nom constant, s petty well undestood. An O ( log L )-appoxmaton (whee L = no. of constants) s possble usng SDP elaxaton (cf. Nemovsk et al. 999). Many applcatons: clusteng, sgnal pocessng, etc. max s.t. A B C = 0 Queston: Does a smla esult hold fo (P)? M m, n
An SDP Relaxaton (Nemovsk 07) The lnea mappngs A, B, C can be dentfed wth matces of the appopate dmenson. max s.t. A Y A B Y B C = 0 C Y = 0 Y a M m, n Gam matx Y S mn
An SDP Relaxaton (Nemovsk 07) The lnea mappngs A, B, C can be dentfed wth matces of the appopate dmenson. The nom constant s equvalent to: T I These can be expessed as LMIs usng appopate lnea mappngs. T I max s.t. A Y A B Y B C = 0 C Y = SY m, n M T Y Y a Gam matx Y S mn 0 I I
An SDP Relaxaton (Nemovsk 07) We now have the followng poblem: (P ) max s.t. B C Y = SY I T Y I Y A Y Y a Gam matx The standad move now s to elax the Gam matx constant to an psd constant. 0
An SDP Relaxaton (Nemovsk 07) We now have the followng poblem: (SDP) max s.t. B C Y = SY I T Y I Y A Y Y a Gam Y matx The standad move now s to elax the Gam matx constant to an psd constant. Note that whle S Y I and TY I ae edundant n (P ), they ae NOT edundant n (SDP). 0 0, Y S mn
Qualty of SDP Relaxaton So how well does (SDP) do? Nemovsk (2007) poved that ana ( {( ) }) 3 O max m + n /, log L appoxmaton s possble. He also conjectued that an O ( log max { m, n, L} ) He also conjectued that an appoxmaton should be achevable. Obsevaton (S. 2008): Nemovsk s conjectue s tue. The poof eles on cetan Khntchne-type type nequaltes.
Roundng the SDP Soluton A standad way of geneatng a soluton ˆ to (P) * fom a soluton Y to (SDP) s va andomzaton. Specfcally: * based on A extact fom Y a set of vectos { v, K, v mn } { ξ,, } geneate a Benoull andom vecto fom the (andom) vecto fom the (andom) vecto ζ = mn = K ξ mn and de-vectoze t to obtan a canddate soluton matx ˆ ξ v
Qualty of ˆ It s not had to show that: * ˆ A ˆ = and C ˆ = 0 v sdp follows fom constucton (P) max s.t. A B C = 0 M m, n
Qualty of ˆ It s not had to show that: * ˆ A ˆ = v sdp and C ˆ = 0 Thus, to analyze the qualty of ˆ, t emans to bound the followng quanttes: ( ) 2 P ˆ B ˆ and ( ) > t P ˆ > t If these ae small, then we can asset that ˆ / t s a feasble * 2 soluton to (P) of value v sdp t wth constant pobablty. (P) max s.t. * 2 / v / t A B C = 0 M m, n
Outlne of the Appoach Those two tal pobabltes can be estmated usng Khntchne-type type nequaltes. Fst, the poblem of boundng ( ) 2 P ˆ B ˆ > t can be shown to be equvalent to the followng: { ξ } { ± } { } Let be..d.. RVs. Let w be vectos 2 satsfyng w 2. Detemne an uppe bound on P ξ t. ( ) w 2 a nomalzaton condton
Outlne of the Appoach On the othe hand, the poblem of boundng ( ) P ˆ > t s equvalent to the followng: { } { ± } { } Let ξ be..d.. RVs. Let Q be m by n matces satsfyng Q Q T T I and Q Q I. Detemne an uppe bound on P t. ( ) ξ Q fom the constants SY I, T Y I
Tool: Khntchne-type type Inequaltes Fo the fst poblem { ξ } { ± } { } Let be..d.. RVs. Let w be abtay vectos. Theoem (Tomczak( Tomczak-Jaegemann 974): E ξ w p 2 p p / 2 ( ) 2 p w / 2 Note that the bound s ndependent of the numbe of vectos n the collecton! Coollay: : Let T=max {m,n,l}.. Then, = P P O ( ˆ B ˆ > Ω ( β log T )) ( ξ w > Ω ( β log T ) β ( T ) 2 2
Tool: Khntchne-type type Inequaltes On the othe hand { } { ± } { } Let ξ be..d.. RVs. Let Q be m by n matces satsfyng Q Q T T I and Q Q I. Theoem (Lust-Pquad 986, Pse 998, Buchholz 200): p p / 2 E Q p max m, n ξ { } Usng the fact that S S, we obtan: S p Coollay: : Let T=max {m,n,l}.. Then, = P P O S p ( ( ) ˆ > Ω β log T ( ξ Q > Ω ( β log T ) β ( T )
Puttng the Peces Togethe By pckng β appopately, the above esult shows that the oundng scheme of Nemovsk (2007) actually poduces a feasble soluton ˆ to (P) that s wthn a logathmc facto fom the optmum.
Applcaton 2: Chance-Constaned Constaned LMIs Consde the followng chance-constaned constaned optmzaton poblem: (P) mn s.t. c F x T P x ( x ) 0 A ( x ) ξ A ( x ) [ 0] 0 ε n R F s an effcently computable vecto-valued valued functon wth convex components; each A maps x nto a symmetc matx; we assume that A ( x ) 0 fo all x. 0 > Such a poblem ases, e.g., n contol theoy and s n geneal ntactable.
A Safe Tactable Appoxmaton One appoach fo pocessng (P) s the so-called safe tactable appoxmaton,,.e. a system H of constants such that: x s feasble fo (P) wheneve t s feasble fo H the constants n H ae effcently computable To develop H,, obseve that: [ A ( x) ξ A ( x) 0] P I A '( x) whee: P 0 ξ A ' / 2 / 2 ( x ) = A ( x ) A ( x ) A ( x ) 0 [ I ] 0
A Safe Tactable Appoxmaton Now, usng a matx veson of Khntchne s nequalty, one can show that fo nce ξ : P ( ξ '( x ) ) ε A wheneve: 2 (*) ( A '( x )) O I ln ( / ε ) The upshot of (*) s that t can be wtten as an LMI: γa0 ( x ) A ( x ) L Ah ( x ) ( ) ( ) ( ) A x γa0 x ( ) Z x 0, γ O M O ln / ε A ( x ) γa ( x ) h 0
A Safe Tactable Appoxmaton Thus, we obtan the followng safe tactable appoxmaton of (P): mn s.t. c F Z x T x ( x ) ( x ) R n 0 0
Concluson Moment nequaltes ae vey useful n analyzng andomzed algothms. SDP ank educton algothm bounds on stochastc optmzaton poblems analyss of SDP-based detecto fo MIMO channels Fnd moe applcatons!
Thank You!