Semidefinite approximations of the matrix logarithm

Transcription

1 1/26 Semidefinite approximations of the matrix logarithm Hamza Fawzi DAMTP, University of Cambridge Joint work with James Saunderson (Monash University) and Pablo Parrilo (MIT) April 5, 2018

2 2/26 Logarithm Concave function Information theory: Entropy H(p) = n i=1 p i log p i (Concave). Kullback-Leibler divergence (or relative entropy) D(p q) = n p i log(p i /q i ) i=1 Convex jointly in (p, q).

3 3/26 Matrix logarithm function X symmetric matrix with positive eigenvalues (positive definite) ( λ1 X = U... U log(x ) = U λ n ) ( log(λ1)... log(λn) ) U where U orthogonal.

4 3/26 Matrix logarithm function X symmetric matrix with positive eigenvalues (positive definite) ( λ1 X = U... U log(x ) = U λ n ) ( log(λ1)... log(λn) ) U where U orthogonal. von Neumann Entropy of X : H(X ) = Tr[X log X ]. Concave in X. Quantum relative entropy: D(X Y ) = Tr[X (log X log Y )] Convex in (X, Y ) [Lieb-Ruskai, 1973].

5 4/26 Concavity of matrix logarithm Matrix logarithm is operator concave: log(λa + (1 λ)b) λ log(a) + (1 λ) log(b) where A, B 0 and λ [0, 1] X Y means X Y positive semidefinite (Löwner order)

6 5/26 Convex optimisation How can we solve convex optimisation problems involving matrix logarithm? CVX modeling tool developed by M. Grant and S. Boyd at Stanford % Maximum entropy problem cvx_begin variable p(n) maximize sum(entr(p)) subject to p >= 0; sum(p) == 1; A*p == b; cvx_end For scalar logarithm, CVX uses a successive approximation heuristic. Works good in practice but: sometimes fails (no guarantees) slow for large problems does not work for matrix logarithm.

7 6/26 Semidefinite programming This talk: New method to treat matrix logarithm and derived functions using symmetric cone solvers (semidefinite programming) Based on accurate rational approximations of logarithm Much faster than successive approximation heuristic for scalars

8 7/26 Outline Semidefinite representations Approximating matrix logarithm Numerical examples, comparison with successive approximation (for scalars) and other matrix examples

9 8/26 Semidefinite programming Problem data: C, A, b minimize C, X s.t. A(X ) = b, X 0 X S n Available solvers: SeDuMi, SDPT3, Mosek, SDPA, etc. (e.g., sedumi(a,b,c)) Generalization of linear programming where S n + x R n X S n x 0 X 0 A(X) = b

10 9/26 Semidefinite formulation Not all optimisation problems are given in semidefinite form... Example: maximise x,y R 2x + y s.t. x 2 + y 2 1

11 9/26 Semidefinite formulation Not all optimisation problems are given in semidefinite form... Example: maximise x,y R 2x + y s.t. x 2 + y 2 1 Formulate as semidefinite optimisation using the fact that: [ ] x 2 + y 2 1 x y 1 0 y 1 + x

12 10/26 Examples of semidefinite formulation [ ] x t x t 0 t 1 t x

13 10/26 Examples of semidefinite formulation [ ] x t x t 0 t 1 t x t 1 x t [ ] x t x

14 11/26 Semidefinite representations Concave function f has a semidefinite representation if: f (x) t S(x, t) 0 for some affine function S : R n+1 S d Key fact: if f has a semidefinite representation then can solve optimisation problems involving f using semidefinite solvers.

15 11/26 Semidefinite representations Concave function f has a semidefinite representation if: f (x) t u R m : S(x, t, u) 0 for some affine function S : R n+1+m S d Key fact: if f has a semidefinite representation then can solve optimisation problems involving f using semidefinite solvers.

16 11/26 Semidefinite representations Concave function f has a semidefinite representation if: f (x) t u R m : S(x, t, u) 0 for some affine function S : R n+1+m S d Key fact: if f has a semidefinite representation then can solve optimisation problems involving f using semidefinite solvers. Book by Ben-Tal and Nemirovski gives semidefinite representations of many convex/concave functions.

17 12/26 Back to logarithm function Goal: find a semidefinite representation of logarithm. t log(x) t x Logarithm is not semialgebraic! We have to resort to approximations.

18 13/26 Integral representation of log Starting point of approximation is: log(x) = 1 0 x s(x 1) ds

19 13/26 Integral representation of log Starting point of approximation is: log(x) = 1 0 x s(x 1) ds Key fact: integrand is concave and semidefinite rep. for any fixed s! [ ] x s(x 1) s(x 1) t st

20 13/26 Integral representation of log Starting point of approximation is: log(x) = 1 0 x s(x 1) ds Key fact: integrand is concave and semidefinite rep. for any fixed s! [ ] x s(x 1) s(x 1) t st Get semidefinite approximation of log using quadrature: log(x) m j=1 Right-hand side is semidefinite representable w j x s j (x 1)

21 14/26 Rational approximation log(x) m x 1 w j 1 + s j (x 1) j=1 }{{} r m(x) r m = m th diagonal Padé approximant of log at x = 1 (matches the first 2m + 1 Taylor coefficients) Log m=3

22 14/26 Rational approximation log(x) m x 1 w j 1 + s j (x 1) j=1 }{{} r m(x) r m = m th diagonal Padé approximant of log at x = 1 (matches the first 2m + 1 Taylor coefficients) Log m=3 Improve approximation by bringing x closer to 1 and using log(x) = 1 h log(x h ) (0 < h < 1): r m,h (x) := 1 h r m(x h )

23 14/26 Rational approximation log(x) m x 1 w j 1 + s j (x 1) j=1 }{{} r m(x) r m = m th diagonal Padé approximant of log at x = 1 (matches the first 2m + 1 Taylor coefficients) Log m=3 Improve approximation by bringing x closer to 1 and using log(x) = 1 h log(x h ) (0 < h < 1): r m,h (x) := 1 h r m(x h ) Remarkable fact: r m,h is still concave and semidefinite representable!

24 15/26 Quadrature + exponentiation r m,h (x) := 1 h r m(x h ) Semidefinite representation of r m,h (say h = 1/2 for concreteness): { x 1/2 y r m,1/2 (x) t y 0 s.t. r m (y) t/2 Uses fact that r m is monotone and x 1/2 is concave and semidefinite rep. Can do the case h = 1/2 k with iterative square-rooting.

25 16/26 Approximation error Approximation error r m,h log on [1/a, a] (h = 1/2 k ): 10 2 Approximation error m=2, k=2 m=3, k=3 m=4, k= a

26 16/26 Approximation error Approximation error r m,h log on [1/a, a] (h = 1/2 k ): 10 2 Approximation error m=2, k=2 m=3, k=3 m=4, k= a Recap: Two ingredients Rational approximation via quadrature Use log(x) = 1 h log(x h ) with small h to bring x closer to 1. Key fact: resulting approximation is concave and semidefinite representable.

27 17/26 Matrix logarithm What about matrix logarithm? Integral representation is valid for matrix log as well: log(x ) = 1 0 (X I )(I + s(x I )) 1 ds

28 17/26 Matrix logarithm What about matrix logarithm? Integral representation is valid for matrix log as well: log(x ) = 1 0 (X I )(I + s(x I )) 1 ds Key fact: integrand is operator concave and semidefinite rep. for any fixed s (use Schur complements) [ ] (X I )(I + s(x I )) 1 I + s(x I ) I T 0 I I st

29 17/26 Matrix logarithm What about matrix logarithm? Integral representation is valid for matrix log as well: log(x ) = 1 0 (X I )(I + s(x I )) 1 ds Key fact: integrand is operator concave and semidefinite rep. for any fixed s (use Schur complements) [ ] (X I )(I + s(x I )) 1 I + s(x I ) I T 0 I I st Get semidefinite approximation of matrix log using quadrature: log(x ) m j=1 Right-hand side is semidefinite representable w j X s j (X 1)

30 18/26 Exponentiation Exponentiation idea also works for matrices: r m,h (X ) := 1 h r m(x h ) (0 < h < 1) r m is not only monotone concave but operator monotone and operator concave. Also X X h is operator concave and semidefinite rep. [ ] X 1/2 X T T 0 T I

31 18/26 Exponentiation Exponentiation idea also works for matrices: r m,h (X ) := 1 h r m(x h ) (0 < h < 1) r m is not only monotone concave but operator monotone and operator concave. Also X X h is operator concave and semidefinite rep. [ ] X 1/2 X T T 0 T I Approximation log(x ) r m,h (X ) called inverse scaling and squaring method by Kenney-Laub, widely used in numerical computations. Remarkable that it preserves concavity and can be implemented in semidefinite programming.

32 19/26 From (matrix) logarithm to (matrix) relative entropy log(x) r m,h (x) Perspective transform (homogenization): f : R R concave g(x, y) := yf (x/y) also concave on R R ++

33 19/26 From (matrix) logarithm to (matrix) relative entropy log(x) r m,h (x) Perspective transform (homogenization): f : R R concave g(x, y) := yf (x/y) also concave on R R ++ Perspective of log is (x, y) y log(x/y) related to relative entropy. Can simply approximate with the perspective of r m,h : y log(x/y) yr m,h (x/y) Semidefinite representation is obtained by homogenization (replace 1 by y).

34 19/26 From (matrix) logarithm to (matrix) relative entropy log(x) r m,h (x) Perspective transform (homogenization): f : R R concave g(x, y) := yf (x/y) also concave on R R ++ Perspective of log is (x, y) y log(x/y) related to relative entropy. Can simply approximate with the perspective of r m,h : y log(x/y) yr m,h (x/y) Semidefinite representation is obtained by homogenization (replace 1 by y). What about for matrices? What is the perspective transform?

35 20/26 Matrix perspective Matrix perspective of f : g(x, Y ) = Y 1/2 f (Y 1/2 XY 1/2 )Y 1/2 Theorem [Effros, Ebadian et al.]: If f operator concave then matrix perspective of f is jointly operator concave in (X, Y ).

36 20/26 Matrix perspective Matrix perspective of f : g(x, Y ) = Y 1/2 f (Y 1/2 XY 1/2 )Y 1/2 Theorem [Effros, Ebadian et al.]: If f operator concave then matrix perspective of f is jointly operator concave in (X, Y ). For f = log matrix perspective is related to operator relative entropy D op (X Y ) = X 1/2 log(x 1/2 YX 1/2 )X 1/2 Approximate with the matrix perspective of r m,h : D op (X Y ) X 1/2 r m,h (X 1/2 YX 1/2 )X 1/2 Semidefinite representation obtained by homogenization

37 21/26 Quantum relative entropy (Umegaki) quantum relative entropy: D(X Y ) = Tr[X (log X log Y )] Operator relative entropy: D op (X Y ) = X 1/2 log(x 1/2 Y 1 X 1/2 )X 1/2 Get SDP approximation of Umegaki rel. entr. via D op : D(X Y ) = φ(d op (X I I Y )) where φ : R n2 n 2 R is the linear map that satisfies φ(a B) = Tr[A T B]. Note: SDP approximation of Umegaki rel. entr. has size n 2!

38 22/26 CVXQUAD New CVX functions: quantum entr ρ Tr[ρ log ρ] Concave trace logm ρ Tr[σ log ρ] Concave (σ 0 fixed) quantum rel entr (ρ, σ) Tr[ρ(log ρ log σ)] Convex lieb ando (ρ, σ) Tr[K ρ 1 t Kσ t ] Concave (t [0, 1]) op rel entr epi cone matrix geo mean hypo cone D op(ρ σ) T A# tb T

39 23/26 Numerical experiments: maximum entropy problem maximize subject to n i=1 x i log(x i ) Ax = b x 0 (A R l n, b R l )

40 23/26 Numerical experiments: maximum entropy problem maximize subject to n i=1 x i log(x i ) Ax = b x 0 (A R l n, b R l ) CVX s successive approx. Our approach m = 3, h = 1/8 n l time (s) accuracy time (s) accuracy s 6.635e s 2.767e s 2.662e s 1.164e s 2.927e s 2.743e s 1.067e s 1.469e-04 accuracy measured wrt specialized MOSEK routine CVX s successive approx.: Uses Taylor expansion of log instead of Padé approx + successively refine linearization point

41 24/26 Relative entropy of entanglement Quantify entanglement of a bipartite state ρ min D(ρ τ) s.t. τ Sep ρ n Cutting-plane [Zinchenko et al.] s 0.55 s s 0.51 s s 0.69 s s 0.82 s s 1.74 s s 5.55 s Our approach m = 3, h = 1/8 Sep = separable states cvx_begin sdp variable tau(na*nb,na*nb) hermitian; minimize (quantum_rel_entr(rho,tau)); subject to tau >= 0; trace(tau) == 1; Tx(tau,2,[na nb]) >= 0; % Positive partial transpose constraint cvx_end

42 25/26 Relative entropy of recovery (with Omar Fawzi) Question: Is it true that for any tripartite quantum state ρ ABC : I (A : C B)? 1.6 min Λ:B BC D(ρ ABC (id A Λ)(ρ AB )). Relative entropy of recovery vs. conditional mutual information D(ρABC (ida Λ opt )(ρab)) I(A : C B)

43 26/26 Conclusion Approximation theory with convexity Approach extends to other operator concave functions via their integral representation (Löwner theorem) Open questions: Dependence on n: Our SDP approximation for Umegaki relative entropy has size n 2. Is there a representation of size O(n)? Dependence on ɛ: Our approximation for scalar log has size (second-order cone rep.) log(1/ɛ) where ɛ error on [e 1, e]. Is this best possible? Paper: arxiv: Code:

44 26/26 Conclusion Approximation theory with convexity Approach extends to other operator concave functions via their integral representation (Löwner theorem) Open questions: Dependence on n: Our SDP approximation for Umegaki relative entropy has size n 2. Is there a representation of size O(n)? Dependence on ɛ: Our approximation for scalar log has size (second-order cone rep.) log(1/ɛ) where ɛ error on [e 1, e]. Is this best possible? Paper: arxiv: Code: Thank you!