Exploitig Structure i SDPs with Chordal Sparsity Atois Papachristodoulou Departmet of Egieerig Sciece, Uiversity of Oxford Joit work with Yag Zheg, Giovai Fatuzzi, Paul Goulart ad Adrew Wy CDC 06 Pre-coferece Workshop
OULINE Chordal Graphs ad Positive Semidefiite Matrices ADMM for Primal ad Dual Sparse SDPs 3 CDCS: Coe Decompositio Coic Solver 4 Coclusio
Chordal Graphs A graph G = ( V, E) is chordal if every cycle of legth 4 has a chord. 3 6 5 9 0 4 4 3 7 8 Ca recogise chordal graphs i O ( V + E ) time
Chordal Graphs A graph G = ( V, E) is chordal if every cycle of legth 4 has a chord. 3 6 5 9 0 4 4 3 7 8 Ca recogise chordal graphs i O ( V + E ) time
Maximal Cliques A maximal clique is a clique that is ot a subset of aother clique. e.g. C = {,,6}. 3 C Ca fid the maximal C C 3 C C 3 4 cliques of a chordal graph i O ( V + E ) time. 6 C 44 5
Examples of Chordal Graphs a) c) b) d)
Sparse Matrices Z z z 0 0 0 z 6 z z z 3 z 4 0 z 6 0 z 3 z 33 z 34 0 0 0 z 4 z 34 z 44 z 45 z 46 0 0 0 z 45 z 55 z 56 z 6 z 6 0 z 46 z 56 z 66 { Z S Z } ij i j Z S ( E,0) = = 0, (, ) E S ( E,0) = { Z S ( E,0) Z 0} + G( Z ) = ( V, E) 3 6 5 4 4
Agler s heorem heorem: Let G = ( V, E) be a chordal graph with set of maximal cliques that Z S C = { C, C,, C }. Suppose ( E,0). he Z S ( E,0) if ad oly if there exists a set of matrices p Z Z p {,,, } such k k that Z = k Z, Z 0, Zij = 0 if ( i, j) / Ek Ek. k = + p Z
Example: Applyig Agler s heorem 3 C C C 3 4 6 C 4 5
Example: Applyig Agler s heorem 3 C C C 3 4 4 4 6 6 6 C 4 5
. SDPs with Chordal Sparsity SDPs with Chordal Sparsity mi CX, max by, X yz, Dual subject to A( X ) = b subject to A ( y ) + Z = C X S+ Z S+ Applicatios: cotrol theory, fluid mechaics, machie learig, polyomial optimizatio, combiatorics, operatios research, fiace, etc. Secod-order solvers : SeDuMi, SDPA, SDP3 Large-scale cases: exploit the iheret structure of the istaces (De Klerk, 00): Low Rak ad Algebraic Symmetry. Chordal Sparsity: Secod-order methods: Fukuda et al., 00; Nakata et al., 003; Aderse et al., 00; First-order methods: Madai et al., 05; Su et al., 04.
Decomposig Sparse SDPs Primal SDP Duality Dual SDP Groe s heorem Chordal Graphs Agler s heorem Decomposed Primal SDP Duality Decomposed Dual SDP
. SDPs with Chordal SDPs Sparsity with Chordal Sparsity S { X S X } ij i j E ( E,0) = = 0, (, ) S ( E,0) = { X S ( E,0) X 0} + G = ( V, E) S ( E,?) = partial symmetric matrices with etries defied o E S+ ( E,?) = { X S ( E,?) M 0, Mij = Xij, ( i, j) E} S ( E,?) ad S ( E,0) are dual coes of each other + +
Groe s theorem. SDPs with Chordal Sparsity Cosider a choral graph G = ( V, E) with a set of maximal cliques C,, C. p Groe s heorem: X S ( E,?) if ad oly if X( C ) 0, k =,, p. + k X( C ) 0 X( C ) 0 X + S ( E,?) i.e., matrix X is PSD completable X( C ) 0 3
Coe Decompositio of Primal ad Dual SDPs. ADMM for Primal ad Dual Sparse SDPs Primal Dual mi CX, X subject to A( X ) = X S+ b max by, yz, subject to A ( y ) + Z = Z S + C S ( E,?) S ( E,0) X + Z + mi CX, X s.t. A( X) = b k E XE S C, k =,, p Ck Ck + max by, yz, s.t A ( y ) + E Z E = C p k = Ck Ck Ck Z S, k =,, p + k
OULINE Chordal Graphs ad Positive Semidefiite Matrices ADMM for Primal ad Dual Sparse SDPs 3 CDCS: Coe Decompositio Coic Solver 4 Coclusio
. ADMM Alteratig for Primal Directio ad Dual Method Sparse of SDPs Multipliers A operator splittig method for a problem of the form f, g may be osmooth. mi f( x) + g( z) s.t. x = z Lagragia: ρ L = f( x) + g( z) + x z+ λ ρ ADMM: + ρ x = argmi f( x) + x z + λ x ρ + ρ + z = argmi g( z) + x z+ λ z ρ + + + λ = λ + ρ( x z )
. ADMM Reformulatio Primal ad ad decompositio Dual Sparse of the SDPs PSD costrait mi CX, X s.t. A( X) = b X = E XE, k =,, p k Ck Ck Ck X S, k =,, p k + Usig idicator fuctios xx,, xp Regroup the variables 0 p δsk k = mi xx,,, xp s.t. mi cx+ δ ( Ax b) + ( x) s.t. x = H x, k =,, p k k k cx Ax = b x = H x, k =,, p k k x S, k =,, p Augmeted Lagragia p ρ L = cx+ δ ( Ax b) + ( x) x Hx 0 δ + + λ Sk k k k k k = ρ { x} ; { x, x }; { λ, λ } p p X Y Z k k
. ADMM for Primal ad Dual Sparse SDPs ) Miimizatio over mi x s.t. ρ cx x Hx Ax = b ) Miimizatio over mi xk s.t. x S k k 3) Update multipliers λ = λ + ρ x H x ADMM for Primal SDPs p ρ L = cx+ δ ( Ax b) + ( x) x Hx 0 δ + + λ Sk k k k k k = ρ ρ p ( ) ( ) + + λ k k k k = x { x} ; { x, x }; { λ, λ } p p X Y Z X Y H x + λ ρ ( + ) ( ) k k k ( ) ( + ) ( ) ( + ) ( + ) k k k k QP with liear costrait the KK system matrix oly depeds o the problem data. Projectios i parallel
ADMM for Primal ad Dual SDPs. ADMM for Primal ad Dual Sparse SDPs he duality betwee the primal ad dual SDP is iherited by the decomposed problems by virtue of the duality betwee Groe s ad Agler s theorems.
3. ADMM for the Homogeous Self-dual Embeddig mi xx, k s.t. cx Ax = b x = H x k k x S, k =,, p k Notatioal simplicity ADMM for the Homogeeous self-dual embeddig KK coditios Primal feasible k Dual feasible Zero-duality gap mi y,zk s.t. by x z v H s, z, v, H x z v H p p p p Ax r = b, r = 0, s + w = Hx, w = 0, s S A y + H v + h = c, h = 0, z v = 0, z S cx by = 0 p Ay+ Hv = c k k k = z v = 0, k =,, p k k z S, k =,, p k k
ADMM for the Homogeeous self-dual embeddig 3. ADMM for the Homogeous Self-dual Embeddig h 0 0 A H c x z 0 0 0 I 0 s r = A 0 0 0 by w H I 0 0 0 v κ c 0 b 0 0 τ Notatioal simplicity Feasibility problem τ, κ are two o-egative ad complemetary variables h x 0 0 A H c z s 0 0 0 I 0 m d v r, u y, Q A 0 0 0 b, K = S w v H I 0 0 0 κ τ c 0 b 0 0 fid ( uv, ) s.t. v = Qu ( uv, ) K K +
ADMM for the Homogeeous self-dual embeddig fid ( uv, ) s.t. v = Qu, ( u, v) K K ADMM steps (similar to solver SCS []) k+ k k uˆ = ( I + Q) ( u + v ) k+ k+ k u = P( uˆ v ) K Projectio to a subspace Projectio to coes k+ v k k+ = v uˆ k+ + u Q is highly structured ad sparse 0 0 A H c 0 0 0 I 0 Q A 0 0 0 b H I 0 0 0 c 0 b 0 0 Block elimiatio ca be applied here to speed up the projectio; he, the per-iteratio cost is the same as applyig a splittig method to the primal or dual aloe. [] O Dooghue, B., Chu, E., Parikh, N. ad Boyd, S. (06). Coic optimizatio via operator splittig ad homogeeous self-dual embeddig. Joural of Optimizatio heory ad Applicatios, 69(3), 04 068
OULINE Chordal Graphs ad Positive Semidefiite Matrices ADMM for Primal ad Dual Sparse SDPs 3 CDCS: Coe Decompositio Coic Solver 4 Coclusio
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver A ope source MALAB solver for partially decomposable coic programs; CDCS supports costraits o the followig coes: Free variables o-egative orthat secod-order coe the positive semidefiite coe. Iput-output format is aliged with SeDuMi; Works with latest YALMIP release. Sytax: [x,y,z,ifo] = cdcs(at,b,c,k,opts); Dowload from https://github.com/oxfordcotrol/cdcs
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Radom SDPs with block-arrow patter Block size: d Number of Blocks: l Arrow head: h Number of costraits: m Numerical Result Numerical Compariso SeDuMi sparsecolo+sedumi SCS sparsecolo+scs CDCS ad SCS =0
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Bechmark problems i SDPLIB hree sets of bechmark problems i SDPLIB (Borchers, 999): ) Four small ad medium-sized SDPs ( theta, theta, qap5 ad qap9); ) Four large-scale sparse SDPs (maxg, maxg3, qpg ad qpg5); 3) wo ifeasible SDPs (ifp ad ifd). 3
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Result: small ad medium-sized istaces
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Result: large-sparse istaces
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Result: Ifeasible istaces
CDCS: Coe Decompositio Coic Solver 4. CDCS: Coe Decompositio Coic Solver Result: CPU time per iteratio Our codes are curretly writte i MALAB SCS is implemeted i C.
5. Coclusio Coclusio Itroduced a coversio framework for sparse SDPs Developed efficiet ADMM algorithms Primal ad dual stadard form; he homogeeous self-dual embeddig; Ogoig work suitable for firstorder methods Develop ADMM algorithms for sparse SDPs arisig i SOS. Applicatios i etworked systems ad power systems
hak you for your attetio! CDCS: Dowload from https://github.com/oxfordcotrol/cdcs. Zheg, Y., Fatuzzi G., Papachristodoulou A., Goulart, P., ad Wy, A. (06) Fast ADMM for Semidefiite Programs with Chordal Sparsity. arxiv preprit arxiv:609.06068. Zheg, Y., Fatuzzi, G., Papachristodoulou, A., Goulart, P., & Wy, A. (06) Fast ADMM for homogeeous self-dual embeddigs of sparse SDPs. arxiv preprit arxiv:6.088