Introducing PENLAB a MATLAB code for NLP-SDP

Transcription

1 Introducing PENLAB a MATLAB code for NLP-SDP Michal Kočvara School of Mathematics, The University of Birmingham jointly with Jan Fiala Numerical Algorithms Group Michael Stingl University of Erlangen-Nürnberg Toulouse, July, 2013

2 PENNON collection PENNON (PENalty methods for NONlinear optimization) a collection of codes for NLP, SDP and BMI one algorithm to rule them all READY PENNLP PENSDP PENBMI AMPL, MATLAB, C/Fortran MATLAB/YALMIP, SDPA, C/Fortran MATLAB/YALMIP, C/Fortran (relatively) NEW PENNON (NLP + SDP) C/Fortran extended AMPL, MATLAB,

3 The problem Optimization problems with nonlinear objective subject to nonlinear inequality and equality constraints and semidefinite bound constraints: min f(x, Y) x R n,y 1 S p 1,...,Y k S p k subject to g i (x, Y) 0, i = 1,...,m g h i (x, Y) = 0, i = 1,...,m h λ i I Y i λ i I, i = 1,...,k. (NLP-SDP)

4 The algorithm Based on penalty/barrier functions ϕ g : R R and Φ P : S p S p : g i (x) 0 p i ϕ g (g i (x)/p i ) 0, Z 0 Φ P (Z) 0, Z S p. Augmented Lagrangian of (NLP-SDP): i = 1,...,m F(x,Y,u,U,U,p)=f(x,Y)+ m g i=1 u ip i ϕ g(g i (x,y)/p i ) + k i=1 U i,φ P(λ i I Y i ) + k i=1 U i,φ P (Y i λ i I) ; here u R mg and U i, U i are Lagrange multipliers.

5 The algorithm A generalized Augmented Lagrangian algorithm (based on R. Polyak 92, Ben-Tal Zibulevsky 94, Stingl 05): Given x 1, Y 1, u 1, U 1, U 1 ; p 1 i > 0, i = 1,...,m g and P > 0. For k = 1, 2,... repeat till a stopping criterium is reached: (i) Find x k+1 and Y k+1 s.t. x F(x k+1, Y k+1, u k, U k, U k, p k ) K (ii) u k+1 i = u k i ϕ g (g i(x k+1 )/p k i ), i = 1,..., m g (iii) p k+1 i U k+1 = D i A Φ P ((λ i I Y i ); U k i ), i = 1,..., k U k+1 i = D A Φ P ((Y i λ i I); U k i ), i = 1,..., k < p k i, i = 1,..., m g P k+1 < P k.

6 Interfaces How to enter the data the functions and their derivatives? Matlab interface AMPL interface c/fortran interface Key point: Matrix variables are treated as vectors

7 What s new PENNON being implemented in NAG (The Numerical Algorithms Group) library The first routines should appear in the NAG Fortran Library, Mark 24 (Autumn 2013) By-product: PENLAB free, open, fully functional version of PENNON coded in MATLAB

8 PENLAB PENLAB free, open, fully functional version of PENNON coded in Matlab Open source, all in MATLAB (one MEX function) The basic algorithm is identical Some data handling routines not (yet?) implemented PENLAB runs just as PENNON but is slower Pre-programmed procedures for standard NLP (with AMPL input!) linear SDP (reading SDPA input files) bilinear SDP (=BMI) SDP with polynomial MI (PMI) easy to add more (QP, robust QP, SOF, TTO... )

9 PENLAB The problem min f(x, Y) x R n,y 1 S p 1,...,Y k S p k subject to g i (x, Y) 0, i = 1,...,m g h i (x, Y) = 0, i = 1,...,m h A i (x, Y) 0, i = 1,...,m A λ i I Y i λ i I, i = 1,...,k (NLP-SDP) A i (x, Y)... nonlinear matrix operators

10 PENLAB Solving a problem: prepare a structure penm containing basic problem data >> prob = penlab(penm); MATLAB class containing all data >> solve(prob); results in class prob The user has to provide MATLAB functions for function values gradients Hessians (for nonlinear functions) of all f, g, A.

11 Structure penm and f/g/h functions Example: min x 1 + x 2 s.t. x x 2 2 1, x penm = []; penm.nx = 2; penm.lbx = [-0.5 ; -Inf]; penm.ngnln = 1; penm.ubg = [1]; penm.objfun deal(x(1) + x(2)); penm.objgrad deal([1 ; 1]); penm.confun deal([x(1)ˆ2 + x(2)ˆ2]); penm.congrad deal([2*x(1) ; 2*x(2)]); penm.conhess deal([2 0 ; 0 2]); % set starting point penm.xinit = [2,1];

12 Toy NLP-SDP example 1 1 min x R 2 2 (x x 2 2 ) 1 x subject to B + A 1 x 1 + A 2 x 2 := x x x 2 1 D. Noll, 2007

13 Structure penm and f/g/h functions B = [1-1 0; ; 0 0 1]; A{1} = [0 1 0; 1 0 0; 0 0 0]; A{2} = [0 0 0; 0 0 1; 0 1 0]; penm = []; penm.nx=2; penm.nalin=1; penm.lba=zeros(1,1); penm.objfun deal(-.5*(x(1)ˆ2+x(2)ˆ2)); penm.objgrad deal(-[x(1);x(2)]); penm.objhess deal(-eye(2,2)); penm.mconfun=@(x,y,k)deal(b+a{1}*x(1)+a{2}*x(2)); penm.mcongrad=@(x,y,k,i)deal(a{i});

14 Example: nearest correlation matrix Find a nearest correlation matrix: min X n (X ij H ij ) 2 (1) i,j=1 subject to X ii = 1, X 0 i = 1,...,n

15 Example: nearest correlation matrix The condition number of the nearest correlation matrix must be bounded by κ. Using the transformation of the variable X: The new problem: min z, X z X = X n (z X ij H ij ) 2 (2) i,j=1 subject to z X ii = 1, I X κi i = 1,...,n

16 Structure penm and f/g/h functions function [f,userdata] = objfun(x,y,userdata) YH = svec2(x(1).*y{1}-userdata.h); f = YH(:) *YH(:); function [df, userdata]=objgrad(x,y,userdata) YH=svec2(x(1).*Y{1}-userdata.H); df(1) = sum(2*svec2(y{1}).*yh); df(2:length(yh)+1) = 2*x(1).*YH; function [ddf, userdata] = objhess(x,y,userdata) YH=svec2(x(1).*Y{1}-userdata.H); yy = svec2(y{1}); n = length(yy); ddf = zeros(n+1,n+1); ddf(1,1) = 2*sum(yy.ˆ2); ddf(1,2:n+1) = 2.*(x(1).*yy+YH); ddf(2:n+1,1) = 2.*(x(1).*yy +YH ); for i=1:n, ddf(i+1,i+1) = 2*x(1)ˆ2; end

17 NLP with AMPL input Pre-programmed. All you need to do: >> penm=nlp_define( datafiles/chain100.nl ); >> prob=penlab(penm); >> prob.solve();

18 NLP with AMPL input problem vars constr. constr. PENNON PENLAB type sec iter. sec iter. chain = 1 14/ /56 pinene = 1 7/ /17 channel = 3 3/3 1 3/3 torsion / /26 lane emd / /49 dirichlet / /68 henon / /158 minsurf box 1 20/ /203 gasoil = & b 3 34/ /71 duct = & 6 19/ /11 marine & b 2 39/ /35 steering & b 1 9/9 7 19/40 methanol & b 2 24/ /67

19 Linear SDP with SDPA input Pre-programmed. All you need to do: >> sdpdata=readsdpa( datafiles/arch0.dat-s ); >> penm=sdp_define(sdpdata); >> prob=penlab(penm); >> prob.solve();

20 Bilinear matrix inequalities (BMI) Pre-programmed. All you need to do: >> bmidata=define_my_problem; %matrices A, K,... >> penm=bmi_define(bmidata); >> prob=penlab(penm); >> prob.solve(); min x R n ct x s.t. n n A i 0 + x k A i k + k=1 k=1 l=1 n x k x l Kkl i 0, i = 1,...,m

21 Polynomial matrix inequalities (PMI) Pre-programmed. All you need to do: >> load datafiles/example_pmidata; >> penm = pmi_define(pmidata); >> problem = penlab(penm); >> problem.solve(); with 1 min x R n 2 x T Hx + c T x subject to b low Bx b up A i (x) 0, i = 1,..., m A(x) = i x κ(i) Q i where κ(i) is a multi-index with possibly repeated entries and x κ(i) is a product of elements with indices in κ(i).

22 Other pre-programmed modules Nearest correlation matrix Truss topology optimization (stability constraints) Static output feedback (input from COMPlib, formulated as PMI) Robust QP

23 Availability PENNON: Free time-limited academic version of the code available PENLAB: Free open MATLAB version available from NAG

24 What s missing? SOCP (Second-Order Conic Programming) - nonlinear, integrated in PENLAB (and PENNON) Postdoctoral research position in Birmingham (sponsored by NAG) development of NL-SOCP algorithm (compatible with PENNON algorithm) implementation in PENLAB and PENNON Alain Zemkoho, started April 2013

25 Sensor network localization (Euclidean distance matrix completion, Graph realization) We have (in R 2 (or R d )) n points a i, anchors with known location m points x i, sensors with unknown location d ij known Euclidean distance between close points d ij = x i x j, (i, j) I x d kj = a k x j, (k, j) I a Goal: Find the positions of the sensors! Find x R 2 m such that x i x j 2 = d 2 ij, a k x j 2 = d 2 kj, (i, j) I x (k, j) I a

26 Sensor network localization Example, 4 anchors, 36 sensors

27 Sensor network localization Applications Wireless sensor network localization habitat monitoring system in the Great Duck Island detecting volcano eruptions industrial control in semiconductor manufacturing plants structural health monitoring military and civilian surveillance moving object tracking asset location Molecule conformation...

28 Sensor network localization Solve the least-square problem min x 1,...,x m (i,j) I x ( x i x j 2 d 2 ij ) 2 + (i,j) I a to global minimum. This is an NP-hard problem. ( ) a k x j 2 d 2 2 kj

29 (P. Biswas and Y. Ye, 04) SDP relaxation Let X = [x 1 x 2... x n ] be a d n unknown matrix. Then x i x j 2 = (e i e j ) T X T X(e i e j ) [ ] a k x j 2 = (a k ; e j ) T Id X T [I d X](a k ; e j ) and the problem becomes (e i e j ) T X T X(e i e j ) = d 2 ij (a k ; e j ) T Z(a k ; e j ) = d 2 kj ( ) Id X Z = X T X T X Z 1:d,1:d = I d, Z 0, Z has rank d

30 SDP relaxation Now relax Z 1:d,1:d = I d, Z 0, Z has rank d to Z 1:d,1:d = I d, Z 0 Relaxed problem: min 0 subject to (0; e i e j ) T Z(0; e i e j ) = d 2 ij (i, j) I x (a k ; e j ) T Z(a k ; e j ) = d 2 kj (k, j) I a Z 1:d,1:d = I d Z 0 Full SDP relaxation, FSDP (linear SDP)

31 SDP relaxation Equivalent formulation: min (i,j) I x + subject to (k,j) I a Z 1:d,1:d = I d Z 0 ( (0; e i e j ) T Z(0; e i e j ) dij 2 ) 2 ( ) (a k ; e j ) T Z(a k ; e j ) d 2 2 kj Full SDP relaxation, FSDP (nonlinear SDP)

32 SDP relaxation For larger problems, FSDP is not solvable numerically: many variables (number of sensors) large and full matrix constraint (although low-rank) Can we exploit sparsity of I x and I a at the relaxation modelling level? Recently several approaches: Wolkowicz Toh Kojima Su Simple, yet powerful way: Edge-based relaxation (ESDP, Wang-Zheng-Ye-Boyd, 08).

33 Example, 16 anchors, 455 sensors

34 Example, 16 anchors, 455 sensors

35 Example, 16 anchors, 455 sensors

36 Example, 16 anchors, 455 sensors problem rmsd out-3 out-2 E-linear E-quadratic SDP: 6714 variables, 5349 (4 4) LMIs

37 Solution refinement Take the SDP solution as initial approximation for the original unconstrained nonconvex problem. Solve both by PENNON.

38 Example, 16 anchors, 455 sensors problem rmsd out-3 out-2 E-linear orig from lin

39 Example, 16 anchors, 455 sensors problem rmsd out-3 out-2 E-linear E-quadratic orig from lin orig from qua 0 0 0

40 Happy Birthday,