On the Exponential Stability of Primal-Dual Gradient Dynamics*

Transcription

1 On the Exponential Stability of Primal-Dual Graient Dynamis* Guannan Qu 1 an Na Li 1 Abstrat Continuous time primal-ual graient ynamis that fin a sale point of a Lagrangian of an optimization problem have been wiely use in systems an ontrol. While the global asymptoti stability of suh ynamis has been wellstuie, it is less stuie whether they are globally exponentially stable. In this paper, we stuy the primal-ual graient ynamis for onvex optimization with strongly-onvex an smooth objetives an affine equality or inequality onstraints, an prove global exponential stability for suh ynamis. Bouns on eaying rates are provie. I. INRODUCION his paper onsiers the following onstraine optimization problem min f(x) (1) x Rn s.t. A 1 x = b 1 A 2 x b 2 where A 1 R m1 n, A 2 R m2 n an b 1 R m1, b 2 R m2 an f(x) is a strongly onvex an smooth funtion. Let L(x, λ) be the Lagrangian (or Augmente Lagrangian) assoiate with Problem (1). he fous of this paper is the following primal-ual graient ynamis, also known as sale-point ynamis, assoiate with the Lagrangian L(x, λ), ẋ = η 1 x L(x, λ) (2a) λ = η 2 λ L(x, λ) (2b) where η 1, η 2 > 0 are time onstants. Primal-Dual Graient Dynamis (PDGD), also known as sale-point ynamis, were first introue in 1, 2. hey have been wiely use in engineering an ontrol systems, for example in power gri 3, 4, wireless ommuniation 5, 6, network an istribute optimization 7, 8, game theory 9, et. Despite its wie appliations, general stuies on PDGD 1, 2, 8, have mostly fouse on its asymptoti stability (or onvergene), with few stuying its global exponential stability. It is known that the graient ynamis for the unonstraine version of (1) ahieves global exponential stability when f is strongly onvex an smooth. It is natural to raise the question whether in the onstraine ase, PDGD an also ahieve global exponential stability. Global exponential stability is a esirable property in pratie. Firstly, in ontrol systems espeially those in ritial infrastruture like the power gri, it is esirable to have strong stability guarantees. Seonly, when using PDGD as *his work is supporte by NSF , NSF CAREER , an ARPA-E NODES. 1 Guannan Qu an Na Li are with John A. Paulson Shool of Engineering an Applie Sienes, Harvar University, Cambrige MA 02138, USA. s: gqu@g.harvar.eu, nali@seas.harvar.eu omputational tools for onstraine optimization, isretization is essential for implementation. he global exponential stability ensures that the simple expliit Euler isretization has a geometri onvergene rate when the isretization step size is suffiiently small 23, 24. his is an appealing property for isrete-time optimization methos. Contribution of this paper. In this paper, we prove the global exponential stability of PDGD (2) uner some regularity onitions on problem (1) an we also give bouns on the eaying rates (heorem 1 an 2). Our proof relies on a quarati Lyapunov funtion that has non-zero offiagonal terms, whih is ifferent from the (blok-)iagonal quarati Lyapunov funtions that are ommonly use in the literature 6, 20 an are known being unable to ertify global exponential stability 6, Lemma 3. We also highlight that when hanling inequality onstraints, we use a variant of the PDGD base on Augmente Lagrangian 25 an is projetion free. his is ifferent from the projetion-base PDGD stuie in 18, 20, whih is isontinuous (see Footnote 2 for more isussions). Our variant of PDGD guarantees that the multipliers stay nonnegative without using projetion, an avois the isontinuity problem ause by projetion 18, 20 (see Remark 1 for more isussions). A. Relate Work here have been many efforts in stuying the stability of PDGD as well as its isrete time version. An inomplete list inlues 1, 2, 8, For instane, 17 stuies the subgraient sale point algorithm an proves its onvergene to an approximate sale point with rate O( 1 t ); 18 uses LaSalle invariane priniple to prove global asympoti stability of PDGD; 22 stuies the global asymptoti stability of the sale-point ynamis assoiate with general sale funtions; an 20 proves global asymptoti stability of PDGD with projetion, whih is extene in 21 by using a weaker assumption an proving input-to-state stability. Our work is losely relate to a reent paper 8 whih stuies sale-point-like ynamis an proves global exponential stability when applying suh ynamis to equality onstraine onvex optimization problems. he ifferene between 8 an our work is that for the equality onstraine ase, 8 onsiers a ifferent Lagrangian from ours. Further, the result in 8 annot be iretly generalize to inequality onstraine ase 8, Remark 3.9. Our work is also relate to the vast literature on spetral bouns on sale matries 26, 27. Suh bouns, when ombine with Ostrowski heorem 28, , an lea to loal exponential stability results of PDGD 25, Se , Prop as oppose to global exponential stability whih is the fous of this paper. It reently ame to our attention that 30 stuies a lass of ynamis, a speial ase of whih turns out to

2 be similar to our PDGD for affine inequality onstraints. 30 also proves global exponential stability. heir proof uses frequeny omain analysis, whih is ifferent from our time-omain analysis. It remains interesting to investigate the onnetion between the methos of 30 an our work. Notations. hroughout the paper, salars will be small letters, vetors will be bol small letters an matries will be apital letters. Notation represents Euliean norm for vetors, an spetrum norm for matries. For any symmetri matrix P 1, P 2 of the same imension, P 1 P 2 means P 1 P 2 is positive semi-efinite. II. ALGORIHMS AND MAIN RESULS In this setion we esribe our PDGD for solving Problem (1) an present stability results. hroughout this paper, we use the following assumption of f: Assumption 1. Funtion f is twie ontinuously ifferentiable, µ-strongly onvex an l-smooth, i.e. for all x, y R n, µ x y 2 f(x) f(y), x y l x y 2 (3) o streamline exposition, we will present the equality onstraine ase an the inequality onstraine ase separately. Integrating them will give PDGD with global exponential stability for Problem (1). Without ausing any onfusion, notations will be ouble-use in the two ases. A. Equality Constraine Case We first onsier the equality onstraine ase, min f(x) (4) x R n s.t. Ax = b Here we remove the subsript for A an b in Problem (1) for notational simpliity. Problem (4) has the Lagrangian, L(x, λ) = f(x) + λ (Ax b) (5) where λ R m is the Lagrangian multiplier. he PDGD is, ẋ = x L(x, λ) = f(x) A λ (6a) λ = η λ L(x, λ) = η(ax b) (6b) where without loss of generality, we have fixe the time onstant of the primal part to be 1. We make the following assumption on A, whih is the linear inepenene onstraint qualifiation for (4). Assumption 2. We assume that matrix A is full row rank an κ 1 I AA κ 2 I for some κ 1, κ 2 > 0. Let (x, λ ) be the equilibrium point of (6), whih in this ase is also the sale point of L. 1 he following theorem gives the global exponential stability of the PDGD (6). heorem 1. Uner Assumption 1 an 2, for η > 0, efine τ eq = min( ηκ1 4l, κ1µ 4κ 2 ). hen there exist onstants C 1, C 2 that epen on η, κ 1, κ 2, µ, l, x(0) x, λ(0) λ, s.t. x(t) x C 1 e 1 2 τeqt an λ(t) λ C 2 e 1 2 τeqt. 1 Assumption 1 an 2 guarantee that the sale point exists an is unique. B. Inequality Constraine Case Now we onsier the inequality onstraine ase, min f(x) (7) x R n s.t. Ax b where f an A satisfy Assumption 1 an 2. For the inequality onstraine ase, we use the Augmente Lagrangian 25, Se. 3.1, as oppose to the stanar Lagrangian in 18, In etails, let A = a 1, a 2,..., a m, with eah a j R n, an let b = b 1,..., b m. hen we efine the augmente Lagrangian, L(x, λ) = f(x) + H (a j x b j, λ j) (8) where > 0 is a free parameter, H (, ) : R 2 R is a penalty funtion on onstraint violation, efine as follows H (a j x b j, λ j) (a j x b j)λ j = + 2 (a j x b j) 2 if (a j x b j) + λ j 0 1 λ 2 j if (a 2 j x b j) + λ j < 0 We an then alulate the graient of H w.r.t. x an λ. xh (a j x b j, λ j) = max((a j x b j) + λ j, 0)a j λ H (a j x b j, λ j) = max((a j x b j) + λ j, 0) λ j e j where e j R m is a vetor with the j th entry being 1 an other entries being 0. he primal-ual graient ynamis for the augmente Lagrangian L is given in (9). We all it as Aug-PDGD (Augmente Primal-Dual Graient Dynamis). ẋ = xl(x, λ) = f(x) xh (a j x b j, λ j) = f(x) max((a j x b j) + λ j, 0)a j λ = η λ L(x, λ) = η = η λ H (a j x b j, λ j) max((a j x b j) + λ j, 0) λ j e j (9a) (9b) Remark 1. It is easy to hek that, if λ j (0) 0, then (9b) guarantees λ j (t) 0, t. his means that the ynamis (9) automatially guarantees λ j (t) will stay nonnegative as long as its initial value is nonnegative, without using projetion as is one in 18, 20, thus avoiing isontinuity issues ause by the projetion step. Sine the sale point of the Augmente Lagrangian (8) 2 We also onsiere using the stanar Lagrangian an the assoiate PDGD with projetion in 18, 20. However, the isontinuous projetion step therein reate iffiulties both in theoreti analysis an numerial simulations. heoretially, the projetion step is base on Euliean norm, an is onsistent with the blok iagonal Lyapunov funtion use in 18, 20, but it is not onsistent with the Lyapunov funtion with ross term use in this paper (f. (14)), whih is the key for proving global exponential stability. herefore we onjeture that the PDGD with projetion 18, 20 is not exponentially stable. Numerially, when we simulate the PDGD with the isontinuous projetion step using MALAB ODE solvers, we enounter many numerial issues. For these reasons, in this paper we stuy an alternative projetion-free PDGD base on the Augmente Lagrangian.

3 is the same as that of the stanar Lagrangian (see 25, Se. 3.1 for etails), we have the following proposition regaring the equilibrium point (x, λ ) of Aug-PDGD. For ompleteness we inlue a proof in our online report 31, Appenix-E. Proposition 1 ( 25). Uner Assumption 1 an 2, Aug- PDGD (9) has a unique equilibrium point (x, λ ) an it satisfies the KK onition of problem (7). Aug-PDGD (9) is globally exponentially stable, as state below. heorem 2. Uner Assumption 1 an 2, the Aug-PDGD (9) is globally exponentially stable in the sense that, for any η > 0, > 0, there exists onstant τ ineq = ηκ lκ 2max( κ 2 µ, l µ )2 max( η l, l µ )2 an onstants C 3, C 4 > 0 whih epen on η,, κ 1, κ 2, µ, l, x(0) x, λ(0) λ, s.t. x(t) x C 3 e 1 2 τineqt, λ(t) λ C 4 e 1 2 τineqt. Remark 2. We urrently only stuy the affine inequality onstraine ase an assume the matrix A satisfies Assumption 2. We onjeture that the results an be extene in two ways. Firstly, for affine inequality onstraints, Assumption 2 an be relaxe to the linear inepenene onstraint qualifiation, i.e. at the optimizer x, the submatrix of A assoiate with the ative onstraints has full row rank. Seonly, for nonlinear onvex onstraint J(x) 0 where J : R n R m, Assumption 2 an be replae by the onition κ 1 I J(x) x ( J(x) x ) κ 2 I where J(x) x R m n is the Jaobian of J w.r.t. x. We leave these extensions to the future work. III. SABILIY ANALYSIS In this setion, we prove global exponential stability. We also show global exponential stability ensures the geometri onvergene rate of the Euler isretization. A. he Equality Constraine Case, Proof of heorem 1 We stak x an λ into a larger vetor z = x, λ an similarly efine z = (x ), (λ ). We efine quarati Lyapunov funtion, V (z) = (z z ) P (z z ) with P 0 efine by ηi ηa P = R ηa I (m+n) (m+n) (10) where = 4 max(l, ηκ2 µ ).3 If we an show the following property of V (z) along the trajetory of the ynamis, V (z) τv (z) (11) t for τ = ηκ1 = min( ηκ1 4l, κ1µ 4κ 2 ), then we have prove heorem 1. he rest of the setion will be evote to proving (11). We start with the following auxiliary Lemma, whih an be prove by using mean value theorem. A similar lemma an be foun in 8, Lem. A.1, an for ompleteness we inlue a proof in our online report 31, Appenix-D. of. 3 We have P 0 as long as 2 > ηκ 2, whih is satisfie by our hoie Lemma 1. Uner Assumption 1, for any x R n, there exists a symmetri matrix B(x) that epens on x, satisfying µi B(x) li, s.t. f(x) f(x ) = B(x)(x x ). With Lemma 1, we an rewrite PDGD (6) as, ( xl(x, t z = λ) xl(x, λ )) η λ L(x, λ) η λ L(x, λ ) B(x)(x x = ) A (λ λ ) ηa(x x ) = B(x) A (z z ) := G(z)(z z ). (12) ηa 0 hen, tv (z) an be written as t V (z) = ż P (z z ) + (z z ) P ż = (z z ) (G(z) P + P G(z))(z z ) (13) herefore, to prove (11), it is suffiient to prove the following Lemma, whose proof is in Appenix-A. Lemma 2. For any z R n+m, we have G(z) P + P G(z) τp Lemma 2 an (13) lea to (11), onluing the proof. B. he Inequality Constraine Case, Proof of heorem 2 We start by emphasizing the notations in this setion is inepenent from the equality onstraine ase in Setion III-A. We stak x, λ into a larger vetor z = x, λ an similarly efine z = (x ), (λ ). Next, we efine the following quarati Lyapunov funtion V (z) = (z z ) P (z z ) with P 0 efine by, ηi ηa P = R ηa I (m+n) (m+n) (14) where = 20lmax( κ2 µ, l µ )2 max( η l, l κ2 µ )2 κ 1. 4 hen, the results of heorem 2 iretly follows from the following property of V (z), V (z) τv (z) (15) t where τ = ηκ1 2 = ηκ lκ 2max( κ 2. he rest of µ, l µ )2 max( η l, l µ )2 the setion will be evote to proving (15). o prove (15), we write the Aug-PDGD (9) in a linear form. In aition to Lemma 1 we nee the following Lemma. Lemma 3. For any j an z = x, λ R n+m, we have there exists γ j (z) 0, 1 that epens on z s.t. xh (a j x b j, λ j) xh (a j x b j, λ j ) = γ j(z)a j (x x )a j + γ j(z)(λ j λ j )a j λ H (a j x b j, λ j) λ H (a j x b j, λ j ) = γ j(z)a j (x x )e j + 1 (γj(z) 1)(λj λ j )e j Proof. he lemma iretly follows from that for any y, y R, there exists some γ 0, 1, epening on y, y s.t. max(y, 0) max(y, 0) = γ(y y ). o see this, when y y, set γ = max(y,0) max(y,0) y y ; otherwise, set γ = 0. of. 4 We have P 0 as long as 2 > ηκ 2, whih is satisfie by our hoie

4 For any z R n+m, we efine notation Γ(z) = iag(γ 1 (z),..., γ m (z)) R m m, where γ j (z) is from Lemma 3. With notation Γ(z), we an then rewrite the Aug- PDGD (9a) as ẋ = ( xl(x, λ) xl(x, λ )) = ( f(x) f(x )) xh (a j x b j, λ j) xh (a j x b j, λ j ) = B(x)(x x ) A Γ(z)A(x x ) A Γ(z)(λ λ ) where µi B(x) li (Lemma 1). We then rewrite (9b), λ = η λ L(x, λ) η λ L(x, λ ) = η λ H (a j x b j, λ j) λ H (a j x b j, λ j ) = ηγ(z)a(x x ) + η (Γ(z) I)(λ λ ) hen, the Aug-PDGD (9) an be written as, B(x) A Γ(z)A A Γ(z) ż = η ηγ(z)a (Γ(z) I) (z z ) := G(z)(z z ). (16) hen, tv (z) an be written as t V (z) = ż P (z z ) + (z z ) P ż = (z z ) (G(z) P + P G(z))(z z ) (17) herefore, to prove (15), it is suffiient to show the following Lemma, whose proof is in Appenix-B. Lemma 4. For any z R n+m, we have G(z) P + P G(z) τp Lemma 4 an (17) lea to (15), onluing the proof. C. Disrete ime Primal-Dual Graient Algorithm Lastly, we briefly isuss the stability of the isretization of (Aug-)PDGD. It is known that the Euler isretization of an exponentially stable ynamial system possesses geometri onvergene spee 23, 24, provie the isretization step size is small enough. For ompleteness, we provie the following Lemma 5, whose proof an be foun in our online report 31, Appenix-F. Lemma 5. Consier a ontinuous-time ynamial system ż = F (z) where F is ν-lipshitz ontinuous. Suppose z is an equilibrium point an there exists positive efinite matrix P, onstant τ > 0, an Lyapunov funtion V ( z) = z P z suh that t V ( z) τv ( z), where z := z z. hen its Euler isretization with step size δ > 0, y(k + 1) = y(k) + δf (y(k)) satisfies y(k) z C(e τδ 2 + κ P ν 2 δ 2 2 ) k, where κ P is the onition number of matrix P, an C > 0 is some onstant that epens on ν, τ, P an y(0) z. Further, e τδ 2 + κ P ν 2 δ 2 2 < 1 for small enough δ. Base on the proof in Setion III-A an III-B, both PDGD (6) an Aug-PDGD (9) satisfy the onitions in Lemma 5. Fig. 1: Simulation of PDGD for the equality onstraine ase. Upper: eaying rate as a funtion of η. Lower: istane to the equilibrium point for some values of η. Hene the isretize versions onverge geometrially fast for small enough isretization step size δ. IV. ILLUSRAIVE EXAMPLES A. Equality Constraine Case We numerially stuy PDGD with affine equality onstraints an quarati ost funtions. We let n = 5, m = 2, f(x) = 1 2 x W x, where W = 10I + W 0 W 0, an W 0 is a n-by-n Gaussian ranom matrix. A an b are also Gaussian ranom matries (or vetors). Sine the ost is quarati, the PDGD (6) beomes an Linear ime-invariant (LI) system an we an etermine the PDGD eaying rate by numerially alulating the eigenvalues of the resulting LI system. We plot the eaying rate as a funtion of η in the upper plot of Fig. 1. We also simulate the PDGD for a selete number of η s, an plot the istane to equilibrium point as a funtion of time in the lower plot of Fig. 1. In both plots, we observe that inreasing η beyon a ertain threshol oes not lea to faster eaying rate, an interesting phenomenon that may be worth further stuying. B. Inequality Constraine Case We numerially run the Aug-PDGD on a problem of size n = 50, m = 40. We use the loss for logisti regression 32 (with syntheti ata) as our ost funtion f. For the affine inequality onstraint Ax b, every entry of A, b is generate inepenently from stanar normal istribution. We fix = 1 but try ifferent η s, an show the results in Fig. 2. Similar to the equality onstraine ase, here we observe that when η is large, the eaying rate oesn t inrease with η. V. CONCLUSIONS In this paper, we stuy the primal-ual graient ynamis for optimization with strongly onvex an smooth objetive an affine equality or inequality onstraints. We prove the global exponential stability of PDGD. We also give expliit bouns on the eaying rates. Future work inlue 1) proviing tighter bouns on the eaying rates, espeially for

5 Fig. 2: Simulation of Aug-PDGD for the inequality onstraine ase uner ifferent values of η. the inequality onstraine ase; 2) relaxing Assumption 2 for the inequality onstraine ase. REFERENCES 1. Kose, Solutions of sale value problems by ifferential equations, Eonometria, Journal of the Eonometri Soiety, pp , K. J. Arrow, L. Hurwiz, H. Uzawa, an H. B. Chenery, Stuies in linear an non-linear programming. Stanfor University Press, C. Zhao, U. opu, N. Li, an S. Low, Design an stability of loasie primary frequeny ontrol in power systems, IEEE ransations on Automati Control, vol. 59, no. 5, pp , N. Li, C. Zhao, an L. Chen, Conneting automati generation ontrol an eonomi ispath from an optimization view, IEEE ransations on Control of Network Systems, vol. 3, no. 3, pp , M. Chiang, S. H. Low, A. R. Calerbank, an J. C. Doyle, Layering as optimization eomposition: A mathematial theory of network arhitetures, Proeeings of the IEEE, vol. 95, no. 1, pp , J. Chen an V. K. Lau, Convergene analysis of sale point problems in time varying wireless systemsontrol theoretial approah, IEEE ransations on Signal Proessing, vol. 60, no. 1, pp , J. Wang an N. Elia, A ontrol perspetive for entralize an istribute onvex optimization, in Deision an Control an European Control Conferene (CDC-ECC), th IEEE Conferene on. IEEE, 2011, pp S. K. Nieerläner an J. Cortés, Distribute oorination for nonsmooth onvex optimization via sale-point ynamis, arxiv preprint arxiv: , B. Gharesifar an J. Cortés, Distribute onvergene to nash equilibria in two-network zero-sum games, Automatia, vol. 49, no. 6, pp , H. Uzawa, Iterative methos in onave programming, Stuies in linear an non-linear programming, B. Polyak, Iterative methos using lagrange multipliers for solving extremal problems with onstraints of the equation type, USSR Computational Mathematis an Mathematial Physis, vol. 10, no. 5, pp , E. Golshtein, Generalize graient metho for fining salepoints, Matekon, vol. 10, no. 3, pp , I. Zabotin, A subgraient metho for fining a sale point of a onvex-onave funtion, Issle. Prikl. Mat, vol. 15, pp. 6 12, M. Kallio an C. H. Rosa, Large-sale onvex optimization via sale point omputation, Operations Researh, vol. 47, no. 1, pp , M. Kallio an A. Ruszzynski, Perturbation methos for sale point omputation, G. Korpelevih, he extragraient metho for fining sale points an other problems, Mateon, vol. 12, pp , A. Neić an A. Ozaglar, Subgraient methos for sale-point problems, Journal of optimization theory an appliations, vol. 142, no. 1, pp , D. Feijer an F. Paganini, Stability of primal ual graient ynamis an appliations to network optimization, Automatia, vol. 46, no. 12, pp , Holing an I. Lestas, On the onvergene to sale points of onave-onvex funtions, the graient metho an emergene of osillations, in Deision an Control (CDC), 2014 IEEE 53r Annual Conferene on. IEEE, 2014, pp A. Cherukuri, E. Mallaa, an J. Cortés, Asymptoti onvergene of onstraine primal ual ynamis, Systems & Control Letters, vol. 87, pp , A. Cherukuri, E. Mallaa, S. Low, an J. Cortés, he role of onvexity on sale-point ynamis: Lyapunov funtion an robustness, IEEE ransations on Automati Control, A. Cherukuri, B. Gharesifar, an J. Cortes, Sale-point ynamis: onitions for asymptoti stability of sale points, SIAM Journal on Control an Optimization, vol. 55, no. 1, pp , A. M. Stuart, Numerial analysis of ynamial systems, Ata numeria, vol. 3, pp , H. J. Stetter, Analysis of isretization methos for orinary ifferential equations. Springer, 1973, vol D. P. Bertsekas, Constraine optimization an Lagrange multiplier methos. Aaemi press, M. Benzi, G. H. Golub, an J. Liesen, Numerial solution of sale point problems, Ata numeria, vol. 14, pp , S.-Q. Shen,.-Z. Huang, an J. Yu, Eigenvalue estimates for preonitione nonsymmetri sale point matries, SIAM Journal on Matrix Analysis an Appliations, vol. 31, no. 5, pp , J. M. Ortega an W. C. Rheinbolt, Iterative solution of nonlinear equations in several variables. Siam, 1970, vol D. P. Bertsekas, Nonlinear programming. Athena sientifi Belmont, N. K. Dhingra, S. Z. Khong, an M. R. Jovanović, he proximal augmente lagrangian metho for nonsmooth omposite optimization, arxiv preprint arxiv: , G. Qu an N. Li, On the exponential stability of primal-ual graient ynamis, arxiv preprint arxiv: , (2012) Logisti regression. Online. Available: shalizi/uada/12/letures/h12.pf APPENDIX A. Proof of Lemma 2 Reall the efinition of G(z) in (12), G(z) = B(x) A ηa 0 It an be seen that G(z) epens on z through B(x), an B(x) satisfies µi B(x) li. In the remaining of this setion, we will rop the epenene of G(z) an B(x) on z an x, an prove G P + P G τp, for any symmetri B satisfying µi B li. Let Q = G P P G, then, Q τp = 2ηB 2η 2 A A η 2 κ 1I ηba η2 κ 1 A ηab η2 κ 1 A 2ηAA ηκ 1I 2ηB 2η 2 κ 2I η 2 κ 1I ηba η2 κ 1 A ηab η2 κ 1 A ηaa where we have use AA κ 1 I, A A κ 2 I. We will next use the Shur omplement argument. Consier (ηba η2 κ 1 A )(ηaa ) 1 (ηab η2 κ 1 A) = ηba (AA ) 1 AB η2 κ 1 (BA (AA ) 1 A + A (AA ) 1 AB) + η3 κ A (AA ) 1 A ηlb + η2 κ 1 2lI + η3 κ 2 1 I 2 where we have use A (AA ) 1 A I. Reall that = 4 max(l, ηκ2 µ ), κ 1 κ 2 an µ l. hen we have, i) 3 4 ηb 3 4 ηµi 3η2 κ 2 I 2η 2 κ 2 I + η 2 κ 1 I, ii) 1 4ηB ηlb, iii) 1 2 ηb 1 η2 2ηµI 2lκ 1I, an iv) 1 2 ηb 1 2 ηµi I. Summing them up, we have η 3 κ ηB 2η 2 κ 2I + η 2 κ 1I + ηlb + η2 2lκ1I + η3 κ 2 1 I 2

6 2η 2 κ 2I + η 2 κ 1I + (ηba η2 κ 1 A )(ηaa ) 1 (ηab η2 κ 1 A) As a result, we have Q τp 0. B. Proof of Lemma 4 Reall the efinition of G(z) in (16), B(x) A Γ(z)A A Γ(z) G(z) = η ηγ(z)a (Γ(z) I) It an be seen that G(z) epens on the state z through B(x), Γ(z). Note for any z R n+m, B(x) is a symmetri matrix satisfying µi B(x) li, an Γ(z) is a iagonal matrix with eah entry in 0, 1. In the following, to simplify notation, we will rop the epenene of G, B, Γ on z, an prove G P + P G τp for any symmetri B satisfing µi B li an for any iagonal Γ with eah entry boune in 0, 1. Let Q = G P P G, an Q1 Q 3 Q = Q τp = Q 3 Q 2 After straightforwar alulations, we have Q 1 = 2ηB + 2(η η 2 )A ΓA η2 κ 1 2 I Q 2 = η(γaa + AA Γ) + 2η ηκ1 (I Γ) 2 I. Q 3 = ηba + ηa ΓAA + η2 A (I Γ) η2 κ 1 2 A Q 5 {}}{ = η(b + A ΓA ηκ1 2 I )A + η η }{{} A (I Γ) }{{} Q 4 Q 6 Using the Shur omplement argument, to prove Q 0 it suffies to prove Q 2 0, Q 1 Q 3 Q 1 2 Q 3 0. o show this, we will first lower boun Q 2, then upper boun Q 3 Q 1 2 Q 3, next lower boun Q 1 an finally show Q 1 Q 3 Q 1 2 Q 3 0. Lower bouning Q 2. We will use the following lemma, whose proof is eferre to Appenix-C. Lemma 6. If κ 2, as long as Γ is a iagonal matrix with eah entry boune in 0, 1, we have η(γaa + AA Γ) + 2η (I Γ) 3 2 ηaa. Using Lemma 6, we have Q ηaa ηκ1 2 I ηaa. Upper bouning Q 3 Q 1 2 Q 3. Using the lower boun on Q 2, we alulate Q 3Q 1 2 Q 3 1 η Q3(AA ) 1 Q 3 = ηq 4A (AA ) 1 AQ 4 + ηq 4A (AA ) 1 Q 6 + ηq 6(AA ) 1 AQ 4 + ηq 6(AA ) 1 Q 6 ηq 4Q 4 + 2η Q 4A (AA ) 1 Q 6 I + η Q 6(AA ) 1 Q 6 I (18) where in the last inequality we have use A (AA ) 1 A I. o further boun Q 3 Q 1 2 Q 3, we use the following, 2η Q 4A (AA ) 1 Q 6 2η Q 4 A (AA ) 1 Q 6 2 η2 ηκ1 (l + κ2 + 2 ) κ2 := h 1() κ 1 (19) where we intentionally write the last quantity as a funtion h 1 () epening on for reasons to be lear later. We further boun η Q 6(AA ) 1 Q 6 η Q 6 2 (AA ) 1 η3 2 κ 2 κ 1 := h 2() hen, we boun ηq 4 Q 4 as (20) ηq 4 Q 4 = η(b + Q 5 ) 2 ηb 2 + 2η B Q 5 I + η Q 5 2 I ηlb + (2ηl(κ 2 + ηκ 1 2 ) + η(κ 2 + ηκ 1 2 )2 )I }{{} :=h 3() (21) Combining (18) (19) (20) (21), we have Q 3 Q 1 2 Q 3 ηlb+ (h 1 () + h 2 () + h 3 ())I. Lower bouning Q 1. It is easy to obtain, Q 1 2ηB (2η 2 κ 2 + η2 κ 1 )I (22) }{{ 2 } :=h 4() Proving Q 1 Q 3 Q 1 2 Q 3 0. Combining the above bouns on Q 1 an Q 3 Q 1 2 Q 3, we have, Q 1 Q 3 Q 1 2 Q 3 (2η ηl)b (h 1 () + h 2 () + h 3 () + h 4 ())I 2ηµ (ηlµ + h 1 () + h 2 () + h 3 () + h 4 ()) I Sine h 1 () + h 2 () + h 3 () + h 4 () is a stritly ereasing funtion in, we have Q 1 Q 3 Q 1 2 Q 3 0 as long as is suffiiently large. We an verify that our seletion of is large enough s.t. Q 1 Q 3 Q 1 2 Q 3 0. Due to spae limit, we omit the etails. herefore, Q τp. C. Proof of Lemma 6 Reall that Γ = iag(γ 1,..., γ m ) with eah γ i 0, 1. he matrix of interest is M(γ 1,..., γ m ) = η(γaa + AA Γ) + 2η (I Γ). It is easy to hek M(γ 1,..., γ m ) is a onvex ombination of 2 m matries {M(b 1,..., b m ) : b i = 0 or 1}. herefore, to prove the lower boun for M(γ 1,..., γ m ), without loss of generality, we only have to prove that for k = 0,..., m M k = M(1, 1,..., 1, 0,..., 0) 3 }{{} 2 ηaa (23) k entries Notie that M 0 = 2η I 3 2 ηaa, M m = 2ηAA, so (23) is true for k = 0 an m. Now assume 0 < k < m. We write AA in blok matrix form AA Λ1 Λ 3 = Λ (24) 3 Λ 2 with Λ 1 R k k, Λ 2 R (m k) (m k), Λ 3 R (k) (m k). hen, we an write M k as 2ηΛ1 ηλ 3 2ηΛ1 ηλ 3 M k = ηλ 3 2 η I ηλ 3 3 2ηΛ 2 2 ηaa where we have use the fat that Λ 2 Λ 2 I κ 2 I I (using κ 2 ).