Stochastic Thermodynamics of Langevin systems under time-delayed feedback control

Transcription

1 New Frontiers in Non-equilibrium Physics 215 Stochastic Thermodynamics of Langevin systems under time-delayed feedback control M.L. Rosinberg in coll. with T. Munakata (Kyoto), and G. Tarjus (Paris) LPTMC, CNRS and Université. P. et M. Curie, Paris

2 Both practical and theoretical interest: Time-delayed feedback processes are ubiquitous in biological regulatory networks and engineering. These systems are typically «autonomous» machines that operate in a nonequilibrium steady state (NESS) where work is permanently extracted from the environment. The non-markovian character of the dynamics raises issues that go beyond the current framework of stochastic thermodynamics and that do not exist when dealing with a discrete (non-autonomous) feedback control. Main theme of the talk: Because of the delay, the timereversal operation becomes highly non-trivial. However, one cannot understand the behavior of the system (in particular the fluctuations) without referring to the unusual properties of the reverse process.

3 TALK ROADMAP A. SECOND LAW-LIKE INEQUALITIES: (bounds for the average extracted work) For more details, see PRL 112, 1861 (214) and Phys. Rev. E 91, (215). B. FLUCTUATIONS (work, heat, entropy production): large-deviation functions and fluctuation relations For more details, see cond-mat. arxiv soon...

4 A. SECOND-LAW-LIKE INEQUALITIES Langevin equation: m v t = v t + F (x t )+F fb (t)+ p 2 T (t) with F fb (t) =F fb (x t + t ) Inertial effects play an important role in human motor control and in experimental setups involving mechanical or electromechanical systems. Deterministic feedback control: no measurement errors Stochastic Delay Differential Equations (SDDEs) have a rich dynamical behavior (multistability, bifurcations, stochastic resonance, etc.). However, we will only focus on the steadystate regime.

5 Consequences of non-markovianity 1) The full description of the time-evolving state of the system in terms of pdf s requires the knowledge of the whole Kolmogorov hierarchy p(x, v, t),p(x 1,v 1,t 1 ; x 2,v 2,t 2 ), etc. There is an infinite hierarchy of Fokker-Planck (FP) equations that has no close solution in general. The definition of the Shannon entropy depends on the level of description. There is no unique entropy-balance equation from the FP formalism (nor unique second-law-like inequality), but a set of equations and inequalities. 2) The time-reversal operation is non-trivial and leads to another second-law-like inequality (in this sense, one looses the nice consistency of stochastic thermodynamics).

6 3) Preparation effects are crucial due to the memory of the dynamics. We will only focus on the steady-state regime and on the asymptotic behavior in the long-time limit (we will not consider transients).

7 Second-law-like inequalities obtained from the FP description FP equation for the one-time pdf: t p(x, v, t) x J x (x, v v (x, v) J x (x, v, t) =vp(x, v, t) J v (x, v, t) = 1 m [ v + F (x)+ F fb (x, v, t)]p(x, v, t) and F fb (x, v, t)] := 1 p(x, v, t) Z 1 1 T m vp(x, v, t) dyf fb (y)p(x, v, t; y, t ) is an effective time-dependent force obtained by formally integrating out the dependence on the variable y t := x t

8 Corresponding Shannon entropy Z S xv (t) = dx dv p(x, v, t)lnp(x, v, t) d/dt +FP equation => Entropy balance equation: d dt Sxv (t) =Ṡxv i (t) Q(t) T S xv pump(t) where and Q(t) = m (mhv 2 t i T ) Ṡi xv (t) = m2 T Ṡ xv pump(t) = Z dxdv [J irr v (x, v)]2 p(x, v, t) 1 m h@ v F fb (x, v, t)i heat exchanged with the bath non-negative «EP» rate «Entropy pumping» rate that describes the influence of the continuous feedback. The effective force contributes to the balance equation because it is velocity - dependent (i.e., it contains a piece which is antisymmetric under time-reversal).

9 In the steady state regime, one then obtains a secondlaw-like inequality Ẇ ext T apple S xv pump (Ẇext = one can extract work from the heat bath if Q) S xv pump > (this depends on the delay, among other things) Similarly, by working in momentum space only, and defining the Shannon entropy as Z S v (t) = dx dv p(v, t)lnp(v, t) one obtains another inequality Ẇ ext T apple S pump v apple Spump xv

10 The entropy pumping rates have no direct interpretation in terms of information-theoretic measures, but one can also consider information flows that reveal how the exchange of information between the system and the controller is affected by the time delay, e.g. Z I xv;y flow,v (t) := dx dv v J v (x, v, t; y, t )ln p(x, v, t; y, t ) p(x, v, t)p(y, t ) For more details, see Phys. Rev. E 91, (215)

11 Second-law-like inequality obtained from time reversal In the case of non-autonomous feedback control with measurements and actions performed step by step at regular time intervals (e.g. Szilard engines), one can record the measurement outcomes and define a reverse process that does not involve any measurement nor feedback (see recent review in Nature Phys. 11, 131, 215). This is not possible when the feedback is implemented continuously. One must also reverse the feedback The feedback force then depends on the future! The «conjugate» dynamics is acausal. m v t = v t + F (x t )+F fb (x t+ )+ p 2 T (t)

12 Generalized local detailed balance equation: P[X Y] probability to observe X = {x s } t given the previous path Y = {x s } P[X Y] / J e S[X,Y] S[X, Y] = Onsager-Machlup action functional S[X, Y] = 1 4 Z t ds mẍ s + ẋ s F (x s ) F fb (x s ) J path-independent Jacobian (contains the factor e 2m t ) Fluctuating heat: q[x, Y] = = Z t Z t ds [ v s p 2 T s ] v s ds [m v s F (x s ) F fb (x s )] v s The heat is odd under time reversal if is changed into

13 Local detailed balance with continuous time-delayed feedback control: P[X Y] P[X x i, Y ] = J J [X] e Q[X,Y] P[X x i, Y ] / J [X]e S[X,Y ] with S[X, Y] = 1 4 Z t ds mẍ s + ẋ s F (x s ) F fb (x s+ ) J [X] = non-trivial Jacobian due to the violation of causality in general path dependent

14 hr cg [X]i = Z Ẇ ext T S 1 J := lim t!1 t hln DX P[X]ln P[X] P[X ] which satisfies an integral fluctuation theorem apple S J J J [X] i st he R cg[x] i =1 In the steady state, this leads to another second-law-like inequality: where One can then define a generalized «entropy production» (Kullback-Leibler divergence): (this quantity can be computed exactly in a linear system but this requires a careful analysis of the «response function» associated to the acausal conjugate Langevin equation in Laplace space.) R + i!?? c???????

15 Example for a linear system: (a) Rates in the NESS Solid black line: extracted work red and blue lines: various bounds τ

16 B. FLUCTUATIONS To be concrete, we will consider a linear Langevin equation, i.e. a stochastic harmonic oscillator submitted to a linear feedback m v t = v t kx t + k x t + p 2 T (t) Γ = ( + ) + describes accurately the dynamics of nanomechanical resonators (e.g. the cantilever of an AFM) used in feedback cooling setups. In reduced units: 3 parameters v t = x t 1 Q v t + g Q x t + t Quality factor: Q =! (! = p k/m, = m/ ) Gain: g =(k /k)q

17 We study the fluctuations of 3 observables: Work: W[X, Y] = 2g Q 2 Z t ds x s v s Heat: Q[X, Y] = W[X, Y] U(x i, x f ) 1 = W[X, Y] (x 2 f x 2 i + vf 2 vi 2 ) Q Pseudo EP [X, Y] = Q[X, Y]+ln p st(x i ) p st (x f ) Quantities of interest: probability distribution functions P A (A, t) =h (A A[X, Y])i st Z Z Z xf = dx f DY P st [Y] DX (A A[X, Y])P[X Y] x i and the corresponding moment generating functions Z A (,t)=he A[X,Y] i st = Z +1 1 da e A P A (A, t)

18 Expected long-time behavior of the pdfs: P A (A = at) e I A(a)t where denotes logarithmic equivalence and I(a) is the LDF Similarly: Z A (,t) g A ( )e µ A( )t 1 where µ A ( )= lim t!1 t lnhe A[X,Y] i st is the SCGF Scaled Cumulant Generating Function) and the pre-exponential factor g A ( ) typically arises from the average over the initial and final states. Here the initial state is Y The 3 observables only differ by «boundary» terms that are not extensive in time. However, since the potential V(x) is unbounded, these terms may fluctuate to order t! Pole singularities in the prefactors and exponential tails in the pdf s (e.g. for the heat)

19 Numerical study: Q = 34.2, g/q =.25 The quality factor corresponds to the cantilever of the AFM used in recent experiments by Ciliberto et al (Eur. Phys. Lett. 89, 63 (21)) g/q The feedbackcontroled oscillator h a s a c o m p l e x dynamical behavior as a function of the delay or the gain g: multistability regime τ As an exemple, we will study the fluctuations in the second lobe where the system can reach a stationary state.

20 Fluctuations of the 3 observables for different noise realizations: The length of the trajectory is t=1 2 τ=7.6 4 τ=8.4 W, Q, Σ Noise realizations Noise realizations WFIG. 3 P W (W Qg/Q = t = Sing th noise: The so from E large-d on the obtain Boundary terms are still non negligible. Fluctuations are correlated but the qualitative behavior depends on the delay! The

21 obability distributions are shown in Figs. 3 and 4. Probability distribution functions: 1 1 τ=8.4 Probability distributions Probability distributions τ= ,1 -.5 w, q, or σ w, q, or σ.1.2 FIG. 4: (Color on line) Same as Fig. 3 for I(w)t = 8.4. G. 3: (Color on line) Probability distribution functions The solid black curve (W for = qwt) eis the theoret blue line onpthe /.42 (W = wt), PQ (Q = qt), line and P the ( =theoretical t) for Qdashed = 34.2, W l.h.s. IQ (q)t curve e is obtained from Eq. (72). Q =.25 and = 7.6. The duration of the trajectory the dashed solid line takes by into = and 1. Points represent numerical data obtained solv-account finite-time corrections. the Langevin equation (45) for 2.16 realizations of the se: W (black circles), Q (blue stars), and (red squares). Main Puzzle: How can we explain the precisely change(1/t) oflnbehavior (more ZA (, t)) areofshown in e solid black line is the theoretical curve e IW (w)t tions obtained 5. Again we observe a striking di erence in the beha m Eq. (66), and the dashed black line is the semi-empirical for = 7.6 and = 8.4. It is also ge-deviation given by Eq. (69). The = dashed lines functions P ( t)ofredthese PQ (Qform = qt) with? and )t the pre-exponential factors play an essen the l.h.s. for /.48 is the theoretical curvevious e I ( that role. ained from Eq. (72).

22 Two origins: 1) Existence of integral fluctuation theorems (IFT):.2.15 (1/t) ln Z A (1,t).1.5 1/Q τ Z Q (1,t) he Q[X,Y] i st = e t/m (exact result for an underdamped Langevin dynamics) Z (1,t) he [X,Y] i st e S J t ( S 1 J := lim t!1 t,t!1 ln ) J J (not yet fully proved; some similarity with Sagawa-Ueda relation involving the so-called «efficacy parameter»)

23 Such IFT s imply a peculiar behavior of the generating functions in the long-time limit: Pole in the prefactor at =1 inthelimit t!1 Boundary layer in the vicinity of but finite =1 for t large µ Q ( = 1) 6= lim!1 µ Q ( ) But this also depends on the value of the delay!

24 2) The behavior of the pdf s also depends on whether the conjugate, acausal dynamics reaches or does not reach a stationary state. Inserting the local detailed balance equation into the definition of the generating function, one finds (for instance for the heat) Z Q (,t) e S J t Z dx i Z DY e (1 ) U(x i,x f ) P st [Y ] Z xf x i DX e es [X,Y] where es [X, Y] is the OM action associated with the conjugate Langevin equation The boundary terms become irrelevant in the longtime limit when the conjugate acausal dynamics reaches a stationary state: no dependence on the state of the system in the far past or the far future.

25 Acausal respeonse function,8 As expected, the fluctuations of the three observables are strongly correlated. However, despite the long du,6 ration of the observed trajectory, the boundary terms Acausal response function in the (which,4 are non-extensive in time) are still not negligible. The most striking feature is that they contribute di ercase where the conjugate,2 ently to the observables depending on the value of : for dynamics reaches a stationary = 7.6, the quantity that exhibits the largest fluctua tions is, whereas it is Q for = 8.4. Note that the sysstate -,2 tem operates in the feedback cooling regime in both cases (Tx /T-,4.42, Tv /T.36, W ext W.19 for = 7.6, and T x /T.72, Tv /T.84,2 W ext.5-2 t for = 8.4). To get a also more quantitative picture, thethe corresponding This depends on delay and can be related to the probability distributions are shown in Figs. 3 and 4. position the acausal response function in the ponse function of e(t) the for Qpoles = 2, g/qof =.55 and = 2.5. (In the final version, we will.) complex-frequency plane. 1 1 τ=8.4 Probability distributions Probability distributions τ=7.6 1 t) are1over the two poles in the l.h.s. of the complex plane and all the poles in the ven by an infinite but converging sum of exponentials, s. -1 Z -2 1 A(s) -3 1 t s(t t ) dt e 1 (t ) + X s2r.h.s. B(s) Z 1 t dt e s(t t) (t ), (81) -3 1 ed for a given noise history (in practice of course, one can only include a finite number the specific -.1values of the parameters)..1.2 he quality of the-,1 approximation depends on w, q, or σ w, q, or σ presentative ensemble of trajectories and obtain the statistics of the modified work -4-4

26 Z Modified f Crooks FT efor the e work: When the acausal dynamics reaches a stationary state, one can show that P W (W = wt) ep ( W f = wt) e(w+ S J )t,t!1 Probability distributions 1, W P W (W = wt) P W (W = wt)e wt P W ( W S = wt)e J t In the long-time limit, the atypical trajectories that dominate he W i st are the conjugate twins (Jarzynski 26) of typical realisations of the reverse (acausal) process

27 Alternatively, one can determine the properties of the atypical noise that generates the rare events. Since the conjugate dynamics converges, the solution of the acausal Langevin equation is ex(t) = Z 1 1 Z t 1 dt e(t t ) (t ) dt e + (t t) (t )+ or in the frequency domain: Z 1 Inserting into the original Langevin equation yields: atyp (!) = e(!) (!) (!). The atypical noise is Hence h atyp (t) atyp (t )i = (t t ) colored! with (t) =2 T apple (t)+ t Z +1 1 dt e (t t ) (t e ) ex(!) e(!) (!) d! 2 Acausal respeonse function,8,6,4,2 -,2 -,4-2 2 t [ e(!) (!) 2 1]e i!t

28 Variance of the atypical noise.4 ν(t)-2 γ T δ(t) t

29 CONCLUSION One can extend the framework of stochastic thermodynamics to treat non-markovian effects induced by a time-delayed feedback. This introduces a new and interesting phenomenology. Experimental tests? Thank you for your attention!