PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.1/22

Size: px

Start display at page:

Download "PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.1/22"

Curtis Harper
5 years ago
Views:

1 PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University kocvara@utia.cas.cz PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.1/22

2 PBM Method for convex NLP R. Polyak 87 Ben-Tal, Zibulevsky 92, 97 Breitfeld, Shanno 94 Combination of: (exterior) Penalty meth., (interior) Barrier meth., Method of Multipliers (CP) min {f(x) : g i(x) 0, i = 1,..., m} x R n ϕ(t) b 1 ϕ (t) b PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.2/22

3 PBM in semidefinite programming Problem: where 1. f : R n R is C 2 min {f(x) : A(x) 0} x Rn 2. A : R n S d is generally nonconvex matrix opretaror PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.3/22

4 PBM in semidefinite programming Problem: where 1. f : R n R is C 2 min {f(x) : A(x) 0} x Rn 2. A : R n S d is generally nonconvex matrix opretaror Question: How can the matrix constraint A(x) 0 be treated by Penalty-Barrier approach? Idea: Find an augmented Lagrangian as follows: F(x, U, p) = f(x) + U, Φ p (A(x)) Sd PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.3/22

5 Construction of the penalty function Φ p Given: scalar valued penalty function ϕ matrix A = S ΛS, where Λ = diag (λ 1, λ 2,..., λ d ) Define A Φ p S T pϕ ( λ1 p ) 0 pϕ ( λ2 p ) pϕ ( λd p ) S any positive eigenvalue of A is penalized by ϕ PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.4/22

6 PBM algorithm for semidefinite problems We have A(x) 0 Φ p (A(x)) 0 and the corresponding augmented Lagrangian: PBM algorithm: F(x, U, p) := f(x) + U, Φ p (A(x)) Sd (i) Find x k+1 satisfying x F(x, U k, p k ) ε k (ii) U k+1 = D A Φ p (A(x); U k ) (iii) p k+1 < p k PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.5/22

7 PENNON for nonconvex problems Idea: replace Newton by Levenberg-Marquardt Given x, compute the gradient g and Hessian H at x. Compute the minimal eigenvalue λ min of H. If λ min < 10 3, set Compute the search direction Ĥ(α) = H + ( λ min + α)i d(α) = Ĥ(α) 1 g Line-search in direction d( α). Step-length s. Set x new = x + sd(α) PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.6/22

8 PBM for nonconvex problems Ĥ(α) = H + ( λ min + α)i d(α) = Ĥ(α) 1 g x new = x + sd(α) Simple version: α [ λ min, 2λ min ]; no convergence proof, works very well in praxis Sophisticated version: Full Trust-Region method; convergence proof, often slower in praxis PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.7/22

9 Computing β Choose initial β > 0. Perform Cholesky factorization of H + βi. If it fails, go to Step (i); otherwise go to Step (iii). (i) Set β 2 β. (ii) Perform Cholesky factorization of H + βi. If it fails, go to Step (i); otherwise stop and return β = β. (iii) Set β β/2. (iv) Perform Cholesky factorization of H + βi. If it fails stop and return β = 2 β; otherwise go to Step (iii). On output, β [ λ min, 2λ min ] PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.8/22

10 PENNON for NSDP: theory Based on Breitfeld-Shanno, 1993; generalized by M. Stingl, Assume: 1. f, A C 2 2. x Ω nonempty, bounded 3. Constraint Qualification Then an index set K so that: x k ˆx, k K U k Û, k K (ˆx, Û) satisfies first-order optimality conditions PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.9/22

11 The reciprocal barrier function in SDP Find a penalty function ϕ which allows direct computation of Φ, its gradient and Hessian: Φ(A) = (A I) 1 I (ϕ := 1 t 1 1) Then Φ(A(x)) = (A I) 1 A(A I) 1 x i x i PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.10/22

12 The reciprocal barrier function in SDP Find a penalty function ϕ which allows direct computation of Φ, its gradient and Hessian: Then Φ(A) = (A I) 1 I (ϕ := 1 t 1 1) 2 x i x j Φ(A(x)) = 1 A(x) 1 A(x) (A(x) I) (A(x) I) (A(x) I) 1 x i x j + (A(x) I) 1 2 A(x) x i x j (A(x) I) 1 1 A(x) 1 A(x) + (A(x) I) (A(x) I) (A(x) I) 1 x j x i PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.10/22

13 Construction of the penalty function Complexity of Hessian assembling - linear SDP: O(d 3 n + d 2 n 2 ) for dense matrices O(n 2 K 2 ) for sparse matrices (K... max. number of nonzeros in A i, i = 1,..., n) Compare to O(d 4 + d 3 n + d 2 n 2 ) in the general case { min b T x : A(x) 0 } x R n A : R n S d PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.11/22

14 BMI problems The same technique as for nonconvex NLP can be used for nonconvex SDP problems, in particular for optimization problems with bilinear matrix inequalities: s.t. min x R n c, x x i [a, b] i = 1,..., n n n n A 0 + x i A i + x i x j K ij 0 i=1 i=1 j=1 PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.12/22

15 BMI problems Feasibility BMI problem: min x R n,λ λ s.t. x i [a, b] i = 1,..., n n n n A 0 + x i A i + x i x j K ij λi 0 i=1 i=1 j=i PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.13/22

16 BMI problems Feasibility BMI problem: min x R n,λ λ + ρ x 2 s.t. x i [a, b] i = 1,..., n n n n A 0 + x i A i + x i x j K ij λi 0 i=1 i=1 j=i PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.13/22

17 GEVP optimization problems Special kind of BMI: generalized eigenvalue problem. Find, w.r.t. variable x, the maximal eigenvalue of GEVP optimization problem: A(x)ω = λb(x)ω. min x R n,λ λ s.t. A(x) λb(x) 0 This is a quasiconvex problem: there exists a unique global minimum PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.14/22

18 Quasiconvex functions Definition: A function f : R n R is called quasiconvex if its domain and all level sets {x domf f(x) α} are convex. PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.15/22

PENNON A Generalized Augmented Lagrangian Method for Convex NLP and SDP p.1/39

PENNON A Generalized Augmented Lagrangian Method for Convex NLP and SDP Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University