Entropic and Displacement Interpolation: Tryphon Georgiou

Transcription

1 Entropic and Displacement Interpolation: Schrödinger bridges, Monge-Kantorovich optimal transport, and a Computational Approach Based on the Hilbert Metric Tryphon Georgiou University of Minnesota IMA workshop Optimization and Parsimonious Modeling January 29, 2016 Supported by NSF-ECCS , AFOSR FA and FA , the Vincentine Hermes-Luh Chair and University of Padova CPDA

2 Joint work with Yongxin Chen, Michele Pavon University of Minnesota & University of Padova A Venetian Schrödinger bridge

3 Interpolation of distributions

4 Optimal Mass Transport (OMT) Gaspard Monge 1781 Monge s memoir Leonid Kantorovich 1976 Work in early 1940 s, Nobel 1975 CIA file on Kantorovich (wikipedia)

5 Monge-Kantorovich OMT Le mémoire sur les déblais et les remblais Gaspard Monge 1781 Monge s drawing

6 Monge-Kantorovich OMT (cont d) inf c(x, y)dπ(x, y) π Π(µ,ν) R n Rn where Π(µ, ν) are couplings of µ and ν. Here, c(x, y) = 1 2 x y Brenier s theorem: If µ does not give mass to sets of dimension n 1,! optimal plan π (Kantorovich) induced by a map T (Monge) T = ϕ, ϕ convex, π = (I ϕ)#µ, ϕ#µ = ν. From now on µ(dx) = ρ 0 (x)dx, ν(dy) = ρ 1 (y)dy.

7 OMT as a control problem Observe 1 2 x y 2 = inf ẋ(t) 2 dt, x(0) = x, x(1) = y, and C 1 Inf attained at constant speed geodesic x (t) = (1 t)x + ty Also the inf of any probabilistic average 1 2 x y 2 = inf P xy D 1 (δ x,δ y ) E Pxy { 1 0 } 1 2 ẋ(t) 2 dt, D 1 (δ x, δ y ) are probability measures on C 1 ([0, 1]; R n ) with delta marginals

8 OMT as a control problem (cont d) 1 inf π Π(µ,ν) R n R n 2 x y 2 dπ(x, y) = inf E P P D(µ,ν) { 1 0 } 1 2 ẋ(t) 2 dt. OMT stochastic control problem inf v E with atypical boundary constraints { v(xv (t), t) 2 dt } ẋ v (t) = v(x v (t), t), a.s., x(0) ρ 0 dx, x(1) ρ 1 dy.

9 Dynamic version of OMT Fluid-dynamic version of OMT (Benamou and Brenier (2000)): 1 1 inf (ρ,v) R n 0 2 v(x, t) 2 ρ(x, t)dtdx ρ t + (vρ) = 0 ρ(x, 0) = ρ 0 (x), ρ(y, 1) = ρ 1 (y)

10 Schrödinger s Bridges Erwin Schrödinger Work in 1926, Nobel 1935 Bridges 1931/32

11 Schrödinger s Bridge Problem (SBP) Cloud of N independent Brownian particles (N large) empirical distr. ρ 0 (x)dx and ρ 1 (y)dy at t = 0 and t = 1, resp. ρ 0 and ρ 1 not compatible with transition mechanism 1 ρ 1 (y) p(t 0, x, t 1, y)ρ 0 (x)dx, 0 where [ ] p(s, y, t, x) = [2π(t s)] n x y 2 2 exp, s < t 2(t s) Particles have been transported in an unlikely way Schrödinger (1931): Of the many unlikely ways in which this could have happened, which one is the most likely?

12 Minimum relative entropy formulation of SBP [ Minimize H(Q, W ) = E Q log dq ] dw over Q D(ρ 0, ρ 1 ) the family of distributions on path-space with given marginals Föllmer 1988: This is a problem of large deviations of the empirical distribution on path space connected through Sanov s theorem to a maximum entropy problem.

13 SBP (cont d)

14 SBP (cont d)

15 SBP (cont d)

16 SBP (cont d)

17 SBP (cont d) Schrödinger: solution (bridge from ρ 0 to ρ 1 over Brownian motion), has at each time a density ρ that factors as ρ(x, t) = ϕ(x, t) ˆϕ(x, t), where ϕ and ˆϕ solve Schrödinger s system ϕ(x, t) = p(t, x, 1, y)ϕ(y, 1)dy, ϕ(x, 0) ˆϕ(x, 0) = ρ 0 (x) ˆϕ(x, t) = p(0, y, t, x) ˆϕ(y, 0)dy, ϕ(x, 1) ˆϕ(x, 1) = ρ 1 (x). Existence and uniqueness for Schrödinger s system: Fortet 1940, Beurling 1960, Jamison 1974/75, Föllmer 1988.

18 Solution of the Schrödinger system

19 SBP as a control problem The minimum relative entropy formulation of SBP can be turned using Girsanov s theorem, into a stochastic control problem! SBP: 1 1 inf (ρ,v) R n 0 2ɛ v(x, t) 2 ρ(x, t)dtdx, ρ t + (vρ) ɛ ρ = 0, 2 ρ(x, 0) = ρ 0 (x), ρ(y, 1) = ρ 1 (y). Blaquière, Dai Pra, Pavon-Wakolbinger, Filliger-Hongler-Streit,... cf. with OMT: 1 1 inf (ρ,v) R n 0 2 v(x, t) 2 ρ(x, t)dtdx ρ t + (vρ) = 0 ρ(x, 0) = ρ 0 (x), ρ(y, 1) = ρ 1 (y)

20 Time-symmetric fluid-dynamic formulation connection with OMT SBP: 1 [ 1 inf (ρ,v) R n 0 2ɛ v(x, t) 2 + ɛ ] log ρ(x, t) 2 ρ(x, t)dtdx, 8 ρ + (vρ) = 0, t ρ(0, x) = ρ 0 (x), ρ(1, y) = ρ 1 (y). Chen-Georgiou-Pavon, On the relation between optimal transport and Schrödinger bridges: A stochastic control viewpoint, J. Opt. Theory Appl., 2015 Cf. Benamou and Brenier: extra term Fisher information integrated over time.

21 Schrödinger s Bridges and OMT Can we use SBP as an approximation of OMT? Yes Mikami 2004, Mikami-Thieullen 2006,2008, Léonard 2012, Chen-Georgiou-Pavon 2015 Is this useful to compute solution to OMT? Other directions: 1. Earlier proofs of SBP solution not in implementable form 2. Earlier formulation of SBP only for non degenerate diffusions (which excluded most engineering applications) 3. Steady-state theory 4. OMT with nontrivial prior, etc.

22 Schrödinger bridges LQG version! stochastic control problem bridge Problem 1. Find a control u minimizing { } T J(u) := E u(t) 2 dt, among those which achieve the transfer dx t = A(t)X t dt + B(t)u(t)dt + B 1 (t)dw t, X 0 N (0, Σ 0 ), X T N (0, Σ 1 ). Engineering applications thermodynamic systems, collective response magnetization distribution in NMR spectroscopy,.. 0

23 Schrödinger bridges LQG version (cont d) a system of two matrix Riccati equations Schrödinger system (Lyapunov equations if B = B 1 ) in the finite horizon case Π = A Π ΠA + ΠBB Π Ḣ = A H HA HBB H Σ 1 0 = Π(0) + H(0) Σ 1 T = Π(T ) + H(T ). + (Π + H) ( BB B 1 B 1) (Π + H). log(ρ) = log(φ) + log( ˆφ) Σ 1 = Π + H

24 Motivating example: Cooling Efficient steering from initial condition ρ 0 to ρ at finite time Efficient asymptotic steering of a system of stochastic oscillators to desired steady state ρ - Y. Chen, T.T. Georgiou and M. Pavon, Fast cooling for a system of stochastic oscillators, J. Math. Phys. Nov

25 Cooling (cont d) Nyquist-Johnson noise driven oscillator Ldi L (t) = v C (t)dt RCdv C (t) = v C (t)dt Ri L (t)dt + u(t)dt + dw(t)

26 Cooling (cont d) Inertial particles with stochastic excitation dx(t) = v(t)dt dv(t) = u(t)dt + dw(t)

27 Cooling (cont d) Inertial particles: dx(t) = v(t)dt dv(t) = v(t)dt x(t)dt + u(t)dt + dw(t) control effort u(t) trajectories in phase space transparent tube: 3σ region

28 OMT and SBP: example on convergence Smoluchowski model for highly overdamped planar Brownian motion in a force field is, in a strong sense, high-friction limit of full Ornstein-Uhlenbeck model in phase space dx t = V (X t )dt + ɛdw t, V (x) = Ax, A = m 0 = m 1 = [ 5, and Σ 5] 0 = [ ] 5, and Σ 5 1 = [ ] [ ] [ ] 3 0, 0 3 Transparent tube represent the 3σ region (x m t )Σ 1 t (x m t ) 9.

29 OMT and SBP: Example (cont d) Schrödinger bridge with ɛ = 9 Schrödinger bridge with ɛ = 0.01 Schrödinger bridge with ɛ = 4 Optimal transport with prior

30 OMT and SBP computations How to effectively compute in the general non Gaussian case: iterative techniques to solve the Schrödinger system based on the Garrett Birkhoff (1957)-Bushell(1973) theorem. M. Cuturi, Sinkhorn distances: Lightspeed computation of optimal transport, NIPS 2013 Chen-Georgiou-Pavon, Entropic and displacement interpolation: a computational approach using the Hilbert metric, arxiv: v1 Georgiou-Pavon, Positive contraction mappings for classical and quantum Schrödinger systems J. Math. Phys., March Benamou-Carlier-Cuturi-Nenna-Peyre, Iterative Bregman projections...,siam J. Sci.Comp

31 Numerical solution of the Schrödinger system p 0 (x 0 ) = ˆφ(x 0 ) p T (x T ) = φ(x T ) p(x 0, x T )φ(x T )dx T = ˆφ(x 0 ) p(x 0, x T ) ˆφ(x 0 )dx 0 = φ(x T ) φ 0 (x 0 ) {}}{ Π 0,T φ(x T ) ˆφ T (x T ) {}}{ Π 0,T ˆφ(x 0 ) and p t (x t ) = ˆφ t (x t )φ(x t ) where φ t (x t ) := Π t,t φ(x T ) ˆφ t (x t ) := Π 0,t ˆφ(x0 )

32 Hilbert metric Positive cones / examples: i) positive cone in R n ii) positive definite Hermitian matrices K closed solid cone x y y x K, Hilbert metric: M(x, y) := inf {λ x λy} m(x, y) := sup{λ λy x}. d H (x, y) := log ( ) M(x, y). m(x, y) Contraction ratio (gain/h-norm) Π H := inf{λ d H (Π(x), Π(y)) λd H (x, y), for all x, y K\{0}}.

33 Birkhoff-Bushell theorem Let Π positive, linear, monotone i.e., x y Π(x) Π(y), then Π H 1 and the (possibly stronger) bound Π H = tanh( 1 4 (Π)) also holds ( (Π) := sup{d H (Π(x), Π(y)) x, y K\{0}}).

34 Numerical solution of the Schrödinger system (cont.) ˆϕ 0 Π ˆϕ T ˆϕ 0 (x 0 ) = p 0(x 0 ) ϕ 0 (x 0 ) ϕ T (x T ) = p T (x T ) ˆϕ T (x T ) where ϕ 0 Π ϕt D T : ϕ 0 ˆϕ 0 (x 0 ) = p 0(x 0 ) ϕ 0 (x 0 ) D T : ˆϕ T ϕ T (x T ) = p T (x N ) ˆϕ T (x T ) are componentwise division of vectors d H -isometries! The composition is strictly contractive: ˆϕ 0 Π ˆϕ T D T ϕ T Π ϕ0 D 0 ( ˆϕ 0 ) next

35 Numerical approximation of OMT via SBP Marginal distributions Dispacement interpolation Entropic interpolation

36 OMT and SBP: Image interpolation Interpolation of 2D images to a 3D model: MRI slices at two different points, designated t = 0 and t = 1 Interpolation with = 0.01, for t {0.2, 0.4, 0.6, 0.8}

37 References Y. Chen, T. Georgiou and M. Pavon On the relation between optimal transport and Schrödinger bridges: A stochastic control viewpoint, JOTA, 2015 Fast cooling for a system of stochastic oscillators, J. Math. Phys., 2015 Optimal steering of a linear stochastic system to a final probability distribution, Part I, IEEE TAC, May 2016 Optimal steering of a linear stochastic system to a final probability distribution, Part II, IEEE TAC, May 2016 Optimal steering of inertial particles diffusing anisotropically with losses, ACC2015 Optimal mass transport over bridges, GSI 15 Optimal transport over a linear dynamical system, in review Entropic and displacement interpolation: a computational approach using the Hilbert metric, in review Georgiou & Pavon, Positive contraction mappings for classical and quantum Schrödinger systems, J. Math. Phys., 2015 Y. Chen & Georgiou, Stochastic bridges of linear systems, IEEE TAC, 2016

38 Thank you for your attention Thanks to the organizers, Maryam, Jiawang, and Pablo and thanks to the

39 I hope to see you all at the MTNS 2016 in July