Schrodinger Bridges classical and quantum evolution

Transcription

1 .. Schrodinger Bridges classical and quantum evolution Tryphon Georgiou University of Minnesota! Joint work with Michele Pavon and Yongxin Chen! LCCC Workshop on Dynamic and Control in Networks Lund, October 2014

2 History of Schrodinger bridges Bridges for Markov chains The Hilbert metric Bridges for quantum (TPTP) evolutions Bridges for Gauss-Markov process

3 Schrodinger 1931/32: The time reversal of the laws of nature! Kolmogoroff: The reversibility of the statistical laws of nature! Bernstein 1932 Fortet 1940 Beurling 1960 Jamison 1974/75 Follmer 1988! connections to Nelson s stochastic mechanics Zambrini, Wakolbinger, Dai Pra, Pavon,Ticozzi and others

4 Hilbert metric:! Hilbert 1895 Birkhoff 1957 Bushell 1973! Sepulchre, Sarlette, Rouchon 2010 Reeb, Kastoryano, Wolf 2011!

5 Schrodinger 1931/1932: suppose a large number of Brownian particles observed at two different times to evolve between two empirical distributions. What is the most likely intermediate distribution at any point in time?

6

7

8

9

10 Given initial and final distribution p 0 (x), p T (x) and transition p(x, y) Schrödinger hypothesised that p T ( ) 6= Z p(x, )p 0 (x)dx =: 0T (p 0 (x)) 0t : q 0 (x)! q = 1 2 2, q(0,x)=q 0 (x). Large deviations Sample paths Relative entropy Stochastic Control

11 Schrödinger system discretized time, space, N-particles Stirling s approximation optimized, lagrange multipliers the most likely joint density and transition probability P? (x 0,x T )= ˆ(x 0 )p(x 0,x T ) (x T ) and p? (x 0,x T )=p(x 0,x T ) (x T ) (x 0 ) Z p 0 (x 0 ) = ˆ(x 0 ) Z p T (x T ) = (x T ) p(x 0,x T ) (x T )dx T = ˆ(x 0 ) p(x 0,x T ) ˆ(x 0 )dx 0 = (x T ) 0(x 0 ) z } { 0,T (x T ) {z } T (x T ) ˆT (x T ) z } { 0,T ˆ(x0 ) {z } ˆ0(x 0 ) and p t (x t )= ˆt(x t ) (x t ) where t(x t ):= t,t (x T) ˆt(x t ):= 0,t ˆ(x0 )

12 Schrödinger system Schrödinger: there exists a solution except possibly for very nasty p 0, p T because the question leading to the pair of equations is so reasonable. Existence/uniqueness Fortet 1940 Résolution d un system d equations de M. Schr dinger Beurling 1960 An automorphism of product measures

13 Markov chains {1,...,N} states, x =(x 0,x 1,...,x T ) sample path t stochastic matrices, t 2 {1,...,T} P 2 probability induced by s on {1,...,N} T +1 P (x 0,...,x T )=P (x 0,x T )P (x 1,...,x T 1 x 0,x T ) Schrödinger question given p 0, p T p T 6= T 1 p 0 find Q(x 0,...,x T )=Q(x 0,x T )Q(x 1,...,x T 1 x 0,x T ) such that P P x T Q(x 0,x T )=p 0 (x 0 ) x 0 Q(x 0,x T )=p T (x T ) and minimizes the relative entropy X Q log Q P = X Q(x 0,x T ) log Q(x 0,x T ) P (x all x 0,x 0,x T ) +X T all Q( x 0,x T ) log Q( x 0,x T ) P ( x 0,x T ) Q(x 0,x T )

14 Lagrangian L(Q) = X Q(x 0,x T )log Q(x 0,x T ) P (x x 0,x 0,x T ) T X (x 0 X x 0 xt Q(x 0,x T ) p T (x T ) A + X x T µ(x T ) X x 0 Q(x 0,x T ) p 0 (x 0 )! (x 0 ) ˆ0 µ(x T ) T such that with = T 1

15 Schrödinger system if there is a solution ˆT = ˆ0 0 = T p 0 = 0 ˆ0 p T = T ˆT? = D T D 1 0 [Q(x 0,x T )] x0,x T = D T D ˆ0

16 Hilbert metric S real Banach space K closed solid cone in S x y, y x 2 K, M(x, y) := inf{ x y} define the Hilbert metric: m(x, y) := sup{ y x}. d H (x, y) :=log M(x, y) m(x, y) Examples: i) positive cone in R n ii) positive definite Hermitian matrices.

17 d H -gain bound of positive maps is a positive map: : K\{0}! K\{0}. Projective diameter ( ) :=sup{d H ( (x), (y)) x, y 2 K\{0}} Contraction ratio, orgain/h-norm k k H := inf{ d H ( (x), (y)) apple d H (x, y), for all x, y 2 K\{0}}.

18 Birkho -Bushell theorem Let positive, monotone, homogeneous of degree m, i.e., x y ) (x) (y), and then ( x) = m (x), k k H apple m. For the special case where is also linear, the (possibly stronger) bound k k H =tanh( 1 4 ( )) also holds.

19 Solution of the Schrödinger system Lemma Let > e 0(element-wisepositive)stochasticmatrix p 0, p T probability vectors then k k H < 1. proof i) ( ) =sup{d H ( (x), (y)) x, y 2 K\{0}} remains the same if we restrict x, y to be probability vectors ii) d H ( (x), (y)) < 18x, y. iii) the probability simplex is compact.

20 Solution of the Schrödinger system Consider ˆ' 0! ˆ'T ˆ' 0 (x 0 )= p 0(x 0 ) ' 0 (x 0 ) " # ' T (x T )= p T (x T ) ˆ' T (x T ) where ' 0 ' T D T : ' 0 7! ˆ' 0 (x 0 )= p 0(x 0 ) ' 0 (x 0 ) D T : ˆ' T 7! ' T (x T )= p T(x N ) ˆ' T (x T ) are componentwise division of vectors ) d H -isometries! The composition ˆ' 0! ˆ'T D T! ' T! ' 0 D 0! ( ˆ' 0 ) next is strictly contractive in the Hilbert metric.

21 Sinkhorn s theorem If > e 0, then 9 a i, b j such that [ ij a i b j ] i,j doubly stochastic. Cf. p 0 = 1, p T = 1? = D T D 1 doubly stochastic 0 i.e., (? ) 1 = 1 but also (? )1 = 1

22 Quantum analogues Density matrices: D = { 0 trace( ) =1} TPTP: E : D! D :! = P n E i=1 E i E i with i.e., E (I) =I n E X i=1 E i E i = I E is positivity improving: if 0 ) E( ) > 0

23 Reference quantum evolution TPCP maps {E t ;0apple t apple T 1} with Kraus representation E t : t 7! t+1 = X i E t,i t E t,i, t =0, 1,...,T 1. Consider the composition E 0:T := E T 1 E 0. initial and a final 0 and T Problem Find F 0:T = F T 1 F 0 such that and F close to E F 0:T ( 0 )= T.

24 rank-1 corrections F t ( ) = t+1 E t ( 1 t ( ) t ) t+1 1 i.e., F t = t+1 E t t where are rank-1 Kraus maps, n =1 Corresponds to the commutative case via: =

25 Quantum version of Sinkhorn s thm Suppose E 0:T is positivity improving Then, 9 observables 0, T such that, for any factorization the map 0 = 0 0, and T = T T, F( ) := T E 0:T ( 1 0 ( ) 0 ) T is a doubly stochastic Kraus map, in that F(I) =I as well as F (I) =I.

26 Proof ˆ0 E 0,T! ˆT ˆ0 = 1 0 " # T = ˆ 1 T The composition map C : ˆ0 starting 0 E 0,T! ˆT E 0,T T ( ) 1 E 0,T ( ) 1! T! 0! ˆ0 next is strictly contractive the steps are identical

27 General case Given E 0:T and 0 and T if 9 0, T, ˆ0, ˆT solving E 0:T ( T) = 0, E 0:T ( ˆ0) = ˆT, Then, for any factorization the map 0 = 0 ˆ0 0, T = T ˆT T. 0 = 0 0, and T = T T, F( ) := T E 0:T ( 1 0 ( ) 0 ) T is a quantum bridge for (E 0:T, 0, T ), namely F(I) =I and F ( 0 )= T.

28 Conjecture The quantum Schrödinger system has a solution for arbitrary 0, T Snag in the proof:! ˆ and ˆ! are not isometries, e.g., 1/2 2 D T : ˆT 7! T = 1/2 T 1/2 ˆ 1 T 1/2 1/2 T T ˆD 0 : 0 7! ˆ0 =( 0 ) 1/2 ( 0 ) 1/2 Extensive numerical evidence that the composition has a fixed point Software for numerical experimentation bridge/

29 Pinned bridge E 0:T positivity improving and two pure states 0 = v 0 v 0 and T = v T v T (i.e., v 0,v T are unit norm vectors), define and F ( ) := 1/2 T E ( 0 := E(v T v T ) T := v T v T, 1/2 0 ( ) 1/2 0 ) 1/2 T 1/2 (where, clearly, T = T = v T v T ). Then, F is TPTP and satisfies the marginal conditions T = F ( 0 ).

30 Example E 1 = E( ) =E 1 ( )E 1 + E 2( )E 2 + E 3( )E 3 # " # 2 q ,E 2 = q,e 1 3 = 4q " # " # 1/4 0 2/3 0 0 = and 1 = 0 3/4 0 1/3 "q = 1 = ˆ0 = ˆ1 = " # 1/2 0 " 0 1/2 # 2/3 0 " 0 1/3 # 1/2 0 " 0 3/2 # F 1 = " p 2/ #, F 2 = " # " p # /3 p, F 3 p 0 1/3 1/3 0

31 Recap Hilbert metric ) constructive existence proofs for i) classical Schrödinger systems ii) quantum Sinkhorn version (uniform marginals) iii) general case open Final topic: Schrödinger bridges for degenerate classical linear stochastic systems anewtypeofoptimalcontrolproblem

32 Optimal steering of state-densities min relative entropy $ Girsanov $ minimum energy stochastic control dx = bdt + dw di usion dx =(b + u)dt + dw controlled di usion min{e{kuk 2 } p 0,p T } relative entropy from prior (dai Pra) our interest: inertial particles, cooling of oscillators dx = vdt dv =(b + u)dt + dw controlled degenerate di usion

33 Optimal steering of state-densities dx(t) = A(t)x(t)dt + B(t)u(t)dt + B(t)dw(t) Given initial and terminal (target) Gaussian densities with covariances 0, T. Find u(t) witht 2 [0,T]thatsteersthesystem from the initial to the target state density and minimizes Z T E{ 0 u(t) 0 u(t)dt}

34 Optimal steering of state-densities Theorem (Gauss-Markov Schrödinger bridge): There exists a unique solution to the following (analogue of the Schrödinger system) Q(T ), P (0) values for matrices satisfying 1 0 = Q(0) 1 + P (0) 1 1 T = Q(T ) 1 + P (T ) 1 and Q(0), P (T )obtainedvia Q(t) =A(t)Q(t)+Q(t)A(t) 0 + B(t)B(t) 0 P (t) =A(t)P (t)+p (t)a(t) 0 B(t)B(t) 0 with Q(t) invertibleover[0,t].

35 The optimal control is u(t) = B(t) 0 Q(t) 1 x(t) The controlled degenerate di usion is the closest to the uncontrolled di usion in the relative entropy sense. Q(0) = N(T,0) 1/2 S 1/2 0 S I S 1/2 S 0 T S 1/ I 1/2! 1 S 1/2 0 N(T,0) 1/2 N(T,0) is the controllability Grammian.

36 Gauss Markov model for inertial particles dx(t) = v(t)dt dv(t) = u(t)dt + dw(t) ntrol force at our disposal, re

37 Gauss Markov model for inertial particles

38 Gauss Markov model for Nyquist-Johnson noise driven oscillator Ldi L (t) = v C (t)dt RCdv C (t) = v C (t)dt Ri L (t)dt + u(t)dt + dw(t) th all parameters in compatible units. Withou

39 X Gauss Markov model for inertial particles: X X state-cost ~ particles with losses dx(t) =f(x(t),t)dt + (X(t),t)dw(t) inf (,ũ) Z R N Z T 0 Z Z apple apple 1 2 kuk2 + V (x, t) (x, + r ((f + u) ) =1 2 (0,x)= 0 (x), i,j=1 (T,y)= T 2 (a ij j, a ij (x, t) = X k X ik(x, t) kj (x, + r (f(x, t) )+V (x, t) = 1 2 X NX 2 (a ij (x, t) j

40 Z Z apple Schrödinger + f(x, t) r' + 1 NX X 2 + r (f(x, t) ˆ') 1 NX X 2 i,j=1 a 2 j = V 2 (a ij j = V ˆ', '(x, 0) ˆ'(x, 0) = 0 (x), '(x, T )ˆ'(x, T )= T (x). ( u (x, + r ((f + ar log ') ) = r log '(x, t), NX 2 (a ij j,

41 Controllability of Fokker-Planck - Linear-Gaussian dx(t) =Ax(t)dt + Bu(t)dt + B 1 dw(t) with x(0) = x 0 a.s. Thm: (A,B) controllable is sufficient to steer the system from any initial Gaussian distribution to a final one at t=t. Thm: A Gaussian state-pdf can be sustained with constant state-feedback iff the state covariance satisfies (A BK) + (A 0 K 0 B 0 )+B 1 B 0 1 =0. equivalently, " # A + A 0 + B 1 B1 0 B rank B 0 " # 0 B =rank B 0

42 Compare with conditions for: i) steering the system to a given state - controllability ii) steering within the positive cone? iii) maintaining the state at a given value ẋ(t) =Ax(t)+Bu(t), 0=(A BK) + Bu

43 Schrödinger system = A 0 A + BB 0 Ḣ = A 0 H HA HBB 0 H +( +H)(BB = (0) + H(0) T 1 = (T )+H(T). B 1 B 0 1)( +H)

44 Fast cooling + stationary control & for dw anywhere

45 Open problem Density matrices: e.g. D = { 0 symmetric 2 R n n with trace( ) =1} E i with i =1,...,n E and P n E i=1 E i E i = I (typically n E n 2 for positivity-improving : 0 ) E( ) > 0) TPTP: E : D! D :! = P n E i=1 E i E i Data: 0, T, E. Problem: Prove that the iteration: E : ˆ0 7! ˆT = E( ˆ0) D T : ˆT 7! T = 1/2 T 1/2 T E : T 7! 0 = E ( T ) ˆD 0 : 0 7! ˆ0 =( 0 ) 1/2 ( 0 ) 1/2 ˆ 1 1/2 T 1/2 2 1/2 T has an attractive fixed point. Software for numerical experimentation

46 Thank you for your attention

47 Positive contraction mappings for classical and quantum Schrodinger systems Stochastic bridges of linear systems Optimal steering of inertial particles diffusing anisotropically with losses arxiv.org/abs/ Optimal steering of a linear stochastic system to a final probability distribution! arxiv.org/abs/ Optimal steering of a linear stochastic system to a final probability distribution, Part II