Stochastic and deterministic models of biochemical reaction networks

Stochastic and deterministic models of biochemical reaction networks David F. Anderson Department of Mathematics University of Wisconsin - Madison Case Western Reserve November 9th, 215

Goals of the talk Introduce mathematical models from molecular biology reaction networks: Stochastic models continuous time Markov chains with stochastic mass action kinetics. Deterministic models ODEs with deterministic mass action kinetics. Introduce the field of chemical reaction network theory Focus on two results how do models behave in large time-frames? Collaborators: Gheorghe Craciun, Tom Kurtz, German Enciso, Matthew Johnston, Daniele Cappelletti Funding: Army Research Office, grant W911NF-14-1-41

Reaction networks Models usually begin with something called a reaction network. A + B 2B B A or E + S ES E + P E The network is a static object. We are not yet choosing a dynamics.

Reaction networks Biochemical/population networks can range from simple to very complex. Example 1: A. Example 2: A + B 2B B A Example 3: Gene transcription & translation: G G + M transcription M M + P translation M degradation P degradation G + P B Binding/unbinding Cartoon representation: 1 1 J. Paulsson, Physics of Life Reviews, 2, 25 157 175.

Reaction networks Example 4: EnvZ/OmpR signaling system 2 2 Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical Reaction Networks, Science, 21

Big picture Hanahan and Weinberg, The Hallmarks of Cancer, Cell, 2.!

Big picture Metabolic Pathways Roche Applied Science!

Big picture Goal of chemical reaction network theory: discover how the dynamics of the mathematical model depend upon properties of the reaction network. Key point of research: want results that are applicable to whole classes of reaction networks, not studying a single model.

Short (incomplete) history of chemical reaction network theory Dates back to at least work of Horn, Jackson, Feinberg in 1972: Detailed balancing is not necessary for many nice results. Developed idea of complex balancing. 197s: development of deficiency theory to predict dynamics of deterministic models based solely on easily checked network conditions (deficiency = and weak reversibility). 2s: many more people started to join this research area 1. Large focus on global attractor conjecture. 2. Focus on possibility of multiple equilibria or oscillations. 3. Algebraists (Sturmfels, Shiu, Miller, Dickenstein) got involved: Toric varieties, Birch s theorem,... 4. Beginning to be a focus on stochastic models (green fluorescent proteins, etc.)

Reaction Networks: {S, C, R} Example: A S = {A}. C = {, A}. R = { A, A }.

Reaction Networks: {S, C, R} Example: A + B 2B B A S = {A, B}. C = {A + B, 2B, B, A}. R = {A + B 2B, B A}.

Reaction Networks: {S, C, R} Example: Species: S = {A, B, C, D, E}. Complexes: C = {A, 2B, A + C, D, B + E}. Reactions: R = {A 2B, 2B A, A+C D, D A+C, D B+E, B+E A+C}.

Reaction Networks: {S, C, R} Definition A chemical reaction network, {S, C, R}, consists of: 1. Species, S := {S 1,..., S d }: constituent molecules undergoing a series of chemical reactions. 2. Complexes, C: linear combinations of the species representing those used, and produced, in each reaction. 3. A set of reactions, R := {y k y k}. Denote reaction vectors ζ k = y k y k.

Dynamics deterministic with mass-action kinetics Example: A + B κ 1 2B B κ 2 A (R1) (R2) Let x(t) R 2 give concentrations of molecules of A and B: [ 1 ẋ = r 1 (x) 1 ] [ 1 + r 2 (x) 1 ]. Deterministic mass-action kinetics says: so r 1 (x) = κ 1 x A x B, and r 2 (x) = κ 2 x B. ẋ A = κ 1 x A x B + κ 2 x B ẋ B = κ 1 x A x B κ 2 x B.

Dynamics deterministic Consider a general system with S = {S 1,..., S d }, and kth reaction y k y k The rate of kth reaction is r k : R d R. As before: or ẋ(t) = k x(t) = x() + k r k (x(t))(y k y k ), ( t ) r k (x(s))ds (y k y k ).

Deterministic mass-action kinetics Consider reaction Then rate is y k y k r k (x) = κ k x y k = κ k d i=1 x y ki i. Example: If S 1 anything, then r k (x) = κ k x 1. Example: If S 1 + S 2 anything, then r k (x) = κ k x 1 x 2. Example: If 2S 2 anything, then r k (x) = κ k x2 2. Example: If 3S 1 + 2S 2 + S 3 anything, then r k (x) = κ k x1 3 x2 2 x 3. Yields: ẋ = k κ k x y k (y k y k )

Dynamics deterministic Example:

Dynamics: discrete stochastic Example: α β A (R1/R2) For Markov models can take X(t) = X() + R 1 (t) R 2 (t). R 1 (t) = Y 1 (αt) R 2 (t) = Y 2 (β t ) X A (s)ds where Y 1, Y 2 are independent unit-rate Poisson processes. t ) X(t) = X() + Y 1 (αt) Y 2 (β X A (s)ds.

Dynamics: discrete stochastic Example: A + B κ 1 2B B κ 2 A (R1) (R2) For Markov models can take [ 1 X(t) = X() + R 1 (t) 1 R 1 (t) = Y 1 ( κ 1 t R 2 (t) = Y 2 ( κ 2 t ] [ 1 + R 2 (t) 1 ) X A (s)x B (s)ds ) X B (s)ds where Y 1, Y 2 are independent unit-rate Poisson processes. ].

Model of viral infection: Srivistava, You, Summers, and Yin, J. Theor. Biol., 218 (22) R 1 ) T 1 T + G, R 2 ) G.25 T, R 3 ) T 1 T + S, R 4 ) T.25, R 5 ) S 2, R 6 ) G + S 7.5 1 6 V, X G (t) = X G () + Y 1 ( t t X S (t) = X S () + Y 3 (1 t X T (t) = X T () + Y 2 (.25 ( t X V (t) = X V () + Y 6 7.5 1 6 ) ( X T (s)ds Y 2.25 ) ( X T (s)ds Y 5 2 t t ) ( X G (s)ds Y 4.25 ) X G (s)x S (s)ds ) t ) X G (s)ds Y 6 (7.5 1 6 X G (s)x S (s)ds ) ) t X S (s)ds Y 6 (7.5 1 6 X G (s)x S (s)ds t. 1 5 1 4 ) X T (s)ds structure Counts 1 3 1 2 virus genome 1 1 template 1 2 4 6 8 1 12 14 16 18 2 Time (Days)

Dynamics: discrete stochastic For general system, we have S = {S 1,..., S d }, with kth reaction y k y k The rate (or intensity or propensity) of kth reaction is λ k : Z d R. As before: X(t) = X() + k R k (t)(y k y k ), with X(t) = X() + k Y k ( t Y k are independent, unit-rate Poisson processes. ) λ k (X(s))ds (y k y k ),

Mass-action kinetics The standard intensity function chosen is stochastic mass-action kinetics: λ k (x) = κ k i x i! (x i y ik )!. Example: If S 1 anything, then λ k (x) = κ k x 1. Example: If S 1 + S 2 anything, then λ k (x) = κ k x 1 x 2. Example: If 2S 2 anything, then λ k (x) = κ k x 2 (x 2 1) κ 2 x 2 2 if x 2 1.

Dynamics. Example A + B α 2B B β A (R1) (R2) Stochastic equations X(t) = X() + Y 1 (α t ) [ 1 X A (s)x B (s)ds 1 ] ( t ) [ 1 + Y 2 β X B (s)ds 1 ]. Deterministic equations x(t) = x() + α t [ 1 x A (s)x B (s)ds 1 ] t [ 1 + β x B (s)ds 1 ],

Other ways to understand model Could just say that for n Z d, n + y 1 y 1, n + y 2 y 2, n. n + y r y r, with rate λ 1 (n) with rate λ 2 (n) with rate λ r (n) where y k y k Z d. I.e. a continuous time Markov chain with infinitesimal generator Af (x) = k λ k (x)(f (x + y k y k ) f (x)). and forward (chemical master) equation p t (x) = k p t(x y k + y k )λ k (x y k + y k ) k p t(x)λ k (x)

First connection (Tom Kurtz 197s) The ODE model is a limit of the stochastic model. Assume V is a scaling parameter: volume times Avogadro s number, or 1. X i = O(V ), and X V (t) = def V 1 X(t), λ k (X(t)) V ( κ k X V (t) y k ),

Connection: Law of large numbers (Tom Kurtz 197s) Then, X V (t) 1 V X + k ( 1 t ) V Y k V κ k X V (s) y k ds (y k y k ) LLN for Y k says 1 V Y k(vu) u ( lim sup V 1 Y k (Vu) u =, V u U a.s. ) so a good approximation is solution to x(t) = x() + k t κ k x(s) y k ds (y k y k ), where u v = u v 1 1 uv d d, is standard mass-action kinetics. See Tom Kurtz s works...

LLN: Example Stochastic models: A + B 2/V 2B B 1 A (R1) (R2) with X() = [3V, V ] so that [A V, B V ] = X/V satisfies A V () = 3, B V () = 1. ODE model of A + B 2 2B B 1 A, with x() = [3, 1].

LLN: Example, A + B 2/V 2B B 1 A 4.5 4 3.5 3 2.5 2 1.5 1.5 V=1 1 2 3 4 5 A B 4.5 4 3.5 3 2.5 2 1.5 1.5 V=1 B A 1 2 3 4 5

LLN: Example, A + B 2/V 2B 4.5 4 3.5 3 2.5 2 1.5 1.5 B 1 A V=1 1 2 3 4 5 A B 4.5 4 3.5 3 2.5 2 1.5 1.5 V=5 1 2 3 4 5 A B

LLN: Example, A + B 2/V 2B 4.5 4 3.5 3 2.5 2 1.5 1.5 B 1 A V=1 A B 1 2 3 4 5 4.5 4 3.5 3 2.5 2 1.5 1.5 V=1 1 2 3 4 5 A B

Next questions. The limit is: over [, T ] with T finite. X V x So what about infinite time-frames?

Story # 1: Complex balanced networks Theorem (Horn, Jackson, Feinberg - 197s) Let {S, C, R} be a chemical reaction network with rate constants κ k. Suppose: 1. the network is weakly reversible, and 2. has a deficiency of zero. Then, for the ODE system (i) within each stoichiometric compatibility class (invariant manifold), there is a unique positive equilibria. (ii) each equilibria is complex-balanced. (iii) each equilibria is globally asymptotically stable relative to the interior of its compatibility class.

Connectivity Definition The connected components of the reaction network are called the linkage classes. C 2 C 3 C 5 C 6 C 7 C 1 C 4 C 8 Example Has two linkage classes. A + B α 2B (Linkage Class 1) B β A (Linkage Class 2)

Connectivity Definition A chemical reaction network, {S, C, R}, is called weakly reversible if each linkage class is strongly connected. A network is called reversible if y k y k R whenever y k y k R. Weakly Reversible C 1 Reversible C 1 C 3 C 2 C 3 C 2

Connectivity The following is not weakly reversible: C 2 C 3 C 5 C 6 C 7 C 1 C 4 C 8 Neither is the following: A + B α 2B (Linkage Class 1) B β A (Linkage Class 2)

Network properties Definition S = span {yk y k R}{y k y k } is the stoichiometric subspace of the network. Denote dim(s) = s. Implication: Solutions bound to translations of S. Example: Reaction network 1 9 8 7 A + B 2B B A # B Molecules 6 5 4 3 2 1 2 4 6 8 1 # A Molecuels

Deficiency. deficiency of {S, C, R} = δ = n l s, where 1. n = # of complexes. 2. l = # of linkage classes. 3. s = dimension of span of reaction vectors. So it is easy to check. Example A + B 2B B A (R1) (R2) n = 4, l = 2, s = 1 = δ = 1.

Deficiency Example n = 5 l = 2 s = 3 = δ = 5 2 3 =.

Deficiency Of course, you are thinking I have no idea what deficiency is that was pointless. In reality, ẋ = f (x) = YA κψ(x), where 1. Y and A κ are matrices, 2. and deficiency = δ = dim(ker(y ) imagea κ).

Story # 1: Complex balanced networks Theorem (Horn, Jackson, Feinberg - 197s) Let {S, C, R} be a chemical reaction network with rate constants κ k. Suppose: 1. the network is weakly reversible, and 2. has a deficiency of zero. Then, the ODE system is complex-balanced and, (i) within each stoichiometric compatibility class (invariant manifold), there is a unique positive equilibria. (ii) each equilibria is complex-balanced. (iii) each equilibria is globally asymptotically stable relative to the interior of its compatibility class. c is complex-balanced if for each η C, κ k c y k = k:y k =η k:y k =η κ k c yk. (1)

Complex-balanced Example Balance for complex A + C means For complex A means κ A+C D c A c C = κ D A+C c D + κ B+E A+C c B c E κ A 2B c A = κ 2B A c 2 B Together gives [ ] f (c) A = κ A 2B c A + κ 2B A cb 2 +[κ D A+C c D + κ B+E A+C c B c E κ A+C D c A c C ] =

Story # 1: Deficiency Zero Theorem - stochastic Theorem (A., Craciun, Kurtz, 21) Let {S, C, R} be a chemical reaction network with rate constants κ k. Suppose: 1. the network is weakly reversible, and 2. has a deficiency of zero. Then, for any irreducible communicating equivalence class, Γ, the stochastic system has a product form stationary distribution π(x) = 1 Z V d i=1 e c c x i i i, x Γ, (2) x i! where Z V is a normalizing constant and c is a complexed-balanced equilibrium of the corresponding ODE. David F. Anderson, Gheorghe Craciun, and Thomas G. Kurtz, Product-form stationary distributions for deficiency zero chemical reaction networks, Bulletin of Mathematical Biology, Vol. 72, No. 8, 1947-197, 21.

Example: Enzyme kinetics Consider the possible model of enzyme kinetics given by E + S ES E + P, E S Easy to see that deficiency is and that Γ = Z 4. Thus, in distributional equilibrium, the species numbers are independent and have Poisson distributions.

Enzyme kinetics Consider the slightly different enzyme kinetics given by E + S ES E + P, E We see S + ES + P = N. In distributional equilibrium: E has Poisson distribution, S, ES, P have a multinomial distribution, and E is independent from S, ES, and P.

Differences Story #2: are there times the models behave very differently?

Story 2: Absolute Concentration Robustness Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical Reaction Networks, Science, 21. A + B α 2B B β A (R1) (R2) ẋ A (t) = αx A (t)x B (t) + βx B (t) ẋ B (t) = αx A (t)x B (t) βx B (t) M = def x A () + x B (), Solving for equilibria: x A = β α, x B = M β α, Network has absolute concentration robustness in species A.

Theorems: deterministic and stochastic Theorem (M. Feinberg and G. Shinar, Science, 21 deterministic) Consider a deterministic mass-action system that has a deficiency of one. admits a positive steady state and has two non-terminal complexes that differ only in species S, then the system has absolute concentration robustness in S.

Differing in one species Examples: 1. differ in species B. A, A + B 2. differ in species Y p. XT, XT + Y p 3. differ in species G. T, T + G

Terminal and non-terminal complexes k 1 XD X XT X p k 2 [ D] k 6 k 3 k4 X p +Y X p Y X+Y p k 7 k 8 k 9 [ T] XD+Y p XDY p XD+Y k 1 k 11 k 5 The orange complexes are called terminal. The blue complexes are called non-terminal.

Theorems: deterministic and stochastic Theorem (M. Feinberg and G. Shinar, Science, 21 deterministic) Consider a deterministic mass-action system that has a deficiency of one. admits a positive steady state and has two non-terminal complexes that differ only in species S, then the system has absolute concentration robustness in S. What about stochastically modeled systems? Theorem (A., Enciso, Johnston stochastic) Consider a reaction network satisfying the following: has a deficiency of one, the deterministic model admits a positive steady state, has two non-terminal complexes that differ only in species S, (new) is conservative, then with probability one there there is a last time a nonterminal reaction fires. 2 David F. Anderson, Germán Enciso, and Matthew Johnston, Stochastic analysis of biochemical reaction networks with absolute concentration robustness, J. Royal Society Interface, Vol. 11, 213943, February 12, 214.

Example Reaction network A + B 2B B A has state space 1 9 8 7 # B Molecules 6 5 4 3 2 1 2 4 6 8 1 # A Molecuels

Example X tot := X + XD + XT + X p + X py + XDY p Y tot := Y + X py + XDY p + Y p.

Idea of proof 1. Suppose not true: then there is a positive recurrent state for which a nonterminal complex has positive intensity (rate). 2. This provides a series of stopping times, τ N. 3. Use stochastic equation: = X (τ N ) X () = k = [ ( 1 τn κ k τn κ k k (X (s)) y k ds Y k κ k ( τn ) Y k κ i (X (s)) y k ds (y k y k ) ) (X (s)) y k ds 1 t N ] τn (X (s)) y k ds (y k y k ) 4. In limit N, find something in kernel of YA κ that can not be there.

Extinction can be rare event. What s real behavior? 1 9 8 A + B α 2B B β A X A () + X B () = M, # B Molecules 7 6 5 4 3 2 1 2 4 6 8 1 # A Molecuels Can show that quasi-stationary distribution for A converges to Poisson πm(x) Q e (β/α) (β/α) x, as M. x!

Quasi-stationary distribution: EnvZ-OmpR signaling system 3 3 Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical Reaction Networks, Science, 21

Quasi-stationary distribution Quasi stationary probabilities.8.6.4.2 X tot = 1 Y tot = 35 X tot = 1 Y tot = 35 X tot = 1 Y tot = 35 Poisson 1 2 3 4 5 Molecules of Y p

Behavior on compact time intervals Let N be a scaling parameter (total conserved value), and consider N 1. Let q be the value of the ACR equilibrium value and let J Poisson(q). (3) Theorem (A., Cappelletti, Kurtz, 215 being written now) Suppose T > and that some technical assumptions hold on the reaction network and let J be as in (3). Then, for any continuous function ĝ : R χ ACR R with at most polynomial growth rate we have or equivalently, P sup T t 1,t 2 [,T ] t 1 <t 2 t2 t 1 (ĝ(x N ACR (s)) E[ĝ(J)]) ds = N. N (ĝ(xacr (s)) E[ĝ(J)]) ds > ε N

That is the story. Thanks! Main References: 1. David F. Anderson, Gheorghe Craciun, Thomas G. Kurtz, Product-form stationary distributions for deficiency zero chemical reaction networks, Bulletin of Mathematical Biology, Vol. 72, No. 8, 1947-197, 21. 2. Guy Shinar and Martin Feinberg, Structural Sources of Robustness in Biochemical Reaction Networks, Science, 21. 3. David F. Anderson, Germán Enciso, and Matthew Johnston, Stochastic analysis of biochemical reaction networks with absolute concentration robustness, J. Royal Society Interface, Vol. 11, 213943, February 12, 214. 4. David F. Anderson, Gheorghe Craciun, Manoj Gopalkrishnan, and Carsten Wiuf, Lyapunov functions, stationary distributions, and non-equilibrium potential for reaction networks, Bulletin of Math. Biol., 215. 5. David F. Anderson and Thomas G. Kurtz, Stochastic analysis of biochemical systems, Springer, 215. 6. Jeremy Gunawardena, Chemical reaction network theory for in-silico biologists, 23, http://vcp.med.harvard.edu/papers/crnt.pdf