Munsky; q-bio Stochastic Gene Expression: Modeling, Analysis, and Identification Mustafa Khammash University of California, Santa Barbara Center for Control, Dynamical Systems and Computation CC DC
Outline Randomness at the molecular level Exploiting the randomness Stochastic modeling framework Simulation and Analysis Methods Identification from Noise Conclusions
Randomness at the Molecular Level
Stochastic Influences on Phenotype Fingerprints of identical twins Cc, the first cloned cat and her genetic mother J. Raser and E. O Shea, Science, 1995. J. Raser and E. O Shea, Science, 1995. gen gen gen.. gen variability in gene expression Elowitz et al, Science 2002 Piliated Unpiliated
Capturing Randomness in Gene Expression Models Deterministic model protein γ p φ k p mrna γ r φ k r DNA
Capturing Randomness in Gene Expression Models Stochastic model protein γ p φ Probability a single mrna is transcribed in time dt is k r dt. Probability a single mrna is degraded in time dt is (#mrna) γ r dt k p mrna γ r φ k r DNA...
Fluctuations at Small Copy Numbers protein k p γ p φ Protein Molecules 800 600 400 200 0 0 1000 2000 3000 4000 5000 Time (s) mrna DNA k r γ r φ mrna Molecules 25 20 15 10 5 0 0 1000 2000 3000 4000 5000 Time (s)
Fluctuations at Small Copy Numbers γ p φ (protein) protein k p mrna γ r φ (mrna) DNA k r C v = coefficient of variation = standard deviation mean
Exploiting the Randomness
Noise Induced Oscillations Circadian rhythm Vilar, Kueh, Barkai, Leibler, PNAS 2002 Oscillations disappear from deterministic model after a small reduction in deg. of repressor (Coherence resonance) Regularity of noise induced oscillations can be manipulated by tuning the level of noise [El-Samad, Khammash]
Stochastic Focusing: Fluctuation Enhanced Sensitivity Signaling Circuit 400 350 stochastic Number of Molecules of P 300 250 200 150 deterministic 100 P 50 0 10 20 30 40 50 60 70 80 90 100 Time mean k s reduced by 50% Stochastic mean value different from deterministic steady state Noise enhances signal! after shift S before shift Johan Paulsson, Otto G. Berg, and Måns Ehrenberg, PNAS 2000
Bacterial Competence ¼ Competence is a process by which bacteria takes up foreign DNA Only a fraction of cells become competent 350 7 SSA runs for 40 hours 300 250 competence ComS 200 150 100 50 0 20 40 60 80 100 120 140 ComK Suel et al, 2006
Stochastic Modeling Framework
A Simple Example mrna k γ φ mrna copy number N(t) is a random variable Transcription: Probability a single mrna is transcribed in time dt is k r dt DNA Degradation: Probability a single mrna is degraded in time dt is nγdt k k k k k k N 0 1 2... n 1 n n + 1... γ 2γ 3γ (n 1)γ nγ (n + 1)γ
Key Question: k k k k k k 0 1 2. n 1 n n + 1 γ 2γ 3γ (n 1)γ nγ (n + 1)γ. Find p(n, t), the probability that N(t) = n. P (n, t + dt) = P (n 1, t) kdt + P (n + 1, t) (n + 1)γdt Prob.{N(t) = n 1 and mrna created in [t,t+dt)} Prob.{N(t) = n + 1 and mrna degraded in [t,t+dt)} + P (n, t) (1 kdt)(1 nγdt) Prob.{N(t) = n and mrna not created nor degraded in [t,t+dt)} P (n, t + dt) P (n, t) = P (n 1, t)kdt + P (n + 1, t)(n + 1)γdt P (n, t)(k + nγ)dt +O(dt 2 ) Dividing by dt and taking the limit as dt 0 The Chemical Master Equation d P (n, t) = kp (n 1, t) + (n + 1)γP (n + 1, t) (k + nγ)p (n, t) dt
mrna Stationary Distribution We look for the stationary distribution P (n, t) = p(n) t The stationary solution satisfies: d dtp (n, t) = 0 From the Master Equation... (k + nγ)p(n) = kp(n 1) + (n + 1)γp(n + 1) n = 0 n = 1 n = 2. kp(0) = γp(1) kp(1) = 2γp(2) kp(2) = 3γp(3) kp(n 1) = nγ p(n)
kp(n 1) = nγ p(n) p(n) = k γ =. = 1 n 2 k 1 γ k γ We can solve for p(0) using the fact n k 1 1 = γ n! p(0) n=0 We can express p(n) as a function of p(0): p(n 1) n 1 n 1 n 1 n! p(0) p(n 2) n=0 p(n) = 1 = e k/γ p(0) p(0) = e k/γ p(n) = e aan n! a = k γ Poisson Distribution
Poisson, a =3!#&$"!#&" probability!#%$"!#%"!#!$"!" 0 1 2 3 4 5 6 7 8 9 mrna count (n) Stationary distribution: P (n) =e aan n! a = k γ Poisson Distribution
Formulation of Stochastic Chemical Kinetics Reaction volume=ω Key Assumptions (Well-Mixed) The probability of finding any molecule in a region dω is given by dω Ω. (Thermal Equilibrium) The molecules move due to the thermal energy. The reaction volume is at a constant temperature T. The velocity of a molecule is determined according to a Boltzman distribution: f vx (v) = f vy (v) = f vz (v) = m 2πk B T e m 2k B T v2
Stochastic Chemical Kinetics population of S2... 0 1 2 3 4 5 6 7 1 2 3 4 6 7 8 5... (N-species) S 1,...,S N. Population of each is an integer r.v.: X(t) =[X 1 (t),...,x N (t)] T (M-reactions) The system s state can change through any one of M reaction: R k : k {1, 2,...,M}. (State transition) Firing of reaction R k causes a state transition from X(t) =x to X(t + )=x + s k. 0 1 2 3 4 5 6 7 population of S 1 Stoich. matrix: S = s 1 s M (Transition Probability) The probability that reaction R k fires in the next dt time units is: w k (x)dt. Example: w 1 (x) =c 1 ; w 2 (x) =c 2 x 1 x 2 ; w 3 (x) =c 3 x 1 ;
The Chemical Master Equation X(t) is Continuous-time discrete-state Markov Chain p(x, t) := prob(x(t) = x) The Chemical Master Equation (Forward Kolmogorov Equation) dp(x, t) dt = p(x, t) k w k (x)+ k p(x s k,t)w k (x s k )
From Stochastic to Deterministic Define X Ω (t) = X(t) Ω. Question: How does X Ω (t) relate to Φ(t)? Fact: Let Φ(t) be the deterministic solution to the reaction rate equations dφ dt = Sf(Φ), Φ(0) = Φ 0. Let X Ω (t) be the stochastic representation of the same chemical systems with X Ω (0) = Φ 0. Then for every t 0: lim sup Ω s t X Ω (s) Φ(s) = 0 a.s.
Simulation and Analysis Tools
1. Sample Paths Computation Gillespie s Stochastic Simulation Algorithm: To each of the reactions {R 1,...,R M } we associate a RV τ i : τ i is the time to the next firing of reaction R i Fact 0: τ i is exponentially distributed with parameter w i We define two new RVs: τ =min{τ i } i (Time to the next reaction) µ =argmin{τ i } i (Index of the next reaction) Fact 1: τ is exponentially distributed with parameter i w i Fact 2: P (µ = k) = w k i w i
Stochastic Simulation Algorithm Step 0 Initialize time t and state population x Step 1 Draw a sample τ from the distribution of τ r 1 U([0, 1]) 1 Cumulative distribution of τ : F (t) =1 exp( k w kt) 0 τ = 1 k w k log 1 1 r 1 time (s) Step 2 Draw a sample µ from the distribution of µ r 2 U([0, 1]) 1 0 Cumulative distribution of µ 1 2 3 4 5 reaction index µ (w 1 + w 2 + w 3 + w 4 )/ k w k (w 1 + w 2 + w 3 )/ k w k (w 1 + w 2 )/ k w k w 1 / k w k Step 3 Update time: t t + τ. Update state: x x + s µ.
2. Moment Computations Let w(x) =[w 1 (x),...,w M (x)] T be the vector of propensity functions Moment Dynamics de[x] dt de[xx T ] dt = S E[w(X)] = SE[w(X)X T ] + E[Xw T (X)]S T + S diag(e[w(x)]) S T Affine propensity. Closed moment equations. Quadratic propensity. Not generally closed. Mass Fluctuation Kinetics (Gomez-Uribe, Verghese) Derivative Matching (Singh, Hespanha)
3. SDE Approximation Let X Ω (t) := X(t) Ω Write X Ω = Φ 0 (t) + 1 Ω V Ω where Φ 0 (t) solves the deterministic RRE Linear Noise Approximation dφ dt = Sf(Φ) V Ω (t) V (t) as Ω, where dv (t) = A(t)V (t)dt + B(t)dW t A(t) = d[sf(φ)] dφ (Φ 0(t)), B(t) := S diag[f(φ 0 (t))] Linear Noise Approximation: X Ω (t) Φ(t) + 1 Ω V (t)
ω (white gaussian noise) Ẏ = AY + ΩB ω Y (t) = ΩV (t) Ω φ (mean) + X(t)
Density Computation Goal: Compute p(x, t), the probability that X(t) = x. Enumerate the state space: X = {x 1,x 2,x 3,...} Form the probability density state vector P (X, ) :R + 1 P (X,t):=[p(x 1,t) p(x 2,t) p(x 3,t)... ] T The Chemical Master Equation (CME): dp(x, t) dt = p(x, t) k w k (x)+ k p(x s k,t)w k (x s k ) can now be written in matrix form: Ṗ (X,t)=A P (X,t)
The Finite State Projection Approach
The Finite State Projection Approach A finite subset is appropriately chosen
The Finite State Projection Approach A finite subset is appropriately chosen The remaining (infinite) states are projected onto a single state (red)
The Finite State Projection Approach A finite subset is appropriately chosen The remaining (infinite) states are projected onto a single state (red) Only transitions into removed states are retained The projected system can be solved exactly!
Finite Projection Bounds Notation: For a matrix A, leta J to be the principle submatrix of A indexed by J, where J =[m 1...m n ]. Projection Error Bounds Consider any Markov process described by the Forward Kolmogorov Equation: Ṗ (X ; t) =A P (X ; t). If for an indexing vector J: 1 T exp(a J T )P (X J ; 0) 1, then P (XJ ; t) exp(aj t)p (X J ; 0) < t [0,T] P (X J ; t) 0 1 Munsky and Khammash, Journal of Chemical Physics, 2006
Example: Analysis of A Synthetic Stochastic Switch Two repressors, u and v. s 2 v s 2 s 2 Gene Promoter s 1 Promoter s 1 Gene Gardner, et al., Nature 403, 339-342 (2000) s 1 u v inhibits the production of u: a 1 (u, v) = α 1 1+v β ν 1 = [ 1 0 ] u inhibits the production of v: a 3 (u, v) = α 2 1+u γ ν 3 = [ 0 1 ] u and v degrade exponentially: a 2 (u, v) =u a 4 (u, v) =v [ 1 ν 2 = 0 [ 0 ν 4 = 1 ] ] α 1 =50 α 2 =16 β =2.5 γ =1 u(0) = v(0) = 0
Using Noise to Identify Model Parameters
Why use noise? A?? B 0 hr 3 hr 4 hr 5 hr Noise provides an excitation source for the network dynamics Resulting distributions of proteins can be measured Probability Density q =0 LacI laci Promoter laci laci lac GFP Such distributions q =0 provide a lot of information about the dynamics Can they be used to identify model parameters? q =0 q =1 q =2 q =5 q =2 q =2 q =4 q =2 q =2 q =3 Noise has been used to discriminate among competing models Dunlop et. al (2008). Nature Genetics. Regulatory activity revealed by dynamic correlations in gene expression noise. Total Florescence: GFP + Background (Arbitrary Units)???
Identification from Moment Information y y protein γ 2 φ x mrna k 2 γ 1 φ k 1 k 12 y DNA Identifiability Can one identify the parameters λ = {k 1, γ 1,k 2, γ 2,k 21 } from measurements of the moments v(t)?
Identifying Using Steady-State Moments y protein γ 2 φ Can the stationary distribution be used to identify all the parameters? x mrna k 2 γ 1 φ k 1 k 12 y DNA Full Identifiability with Stationary Moments Full identifiability is impossible using only v. Munsky et. al, MSB, 2009 Identifiability is possible if lim t E[x(t)x(t + s)] is available. Cinquemani et al, lect. notes comp. sci, 2009
Identifiability from Transient Time-Measurements y protein γ 2 φ x mrna k 2 γ 1 φ Multiple Measurements Suppose v j := v(t j ) has been measured at equally separated points in time {t 0,t 1,...,t m } k 1 k 12 y DNA Identifiability with Multiple Moment Measurements For m = 6 the model parameters are identifiable. I 0 A = 1 τ log(g)
Identification with Two Measurements x mrna γ φ Identifiability of Transcription Parameters Suppose the mean and variance are known at two times t 0 <t 1 <, and define (µ 0, σ 0 ) := (µ(t 0 ), σ(t 0 )) and (µ 1, σ 1 ) := (µ(t 1 ), σ(t 1 )). DNA k Then the transcription parameters are identifiable, and γ = 1 σ 2 2τ log 1 µ 1 σ0 2 µ k = γ µ 1 exp( γτ)µ 0. 0 1 exp( γτ) (τ := t 1 t 0 ) protein y γ 2 φ Identifiability of Transcription & Translation Parameters x mrna k 2 γ 1 φ Given v(t 0 ) and v(t 1 ), identifiability of all parameters k 1,k 2, γ 1, γ 2 is generically possible. k 1 An expression exists for finding the parameters. DNA
Using Densities to Identify Network Parameters Moment equations can be written only in special cases. Densities (distributions) contain much more information than first two moments. Using the Chemical Master Equation, we propose to use density measurements for model identification. Using Density: Suppose we measure P at different times: P(t 0 ), P(t 1 ),...,P(t N 1 ) We can use these to identify unknown network parameters λ: Find λ subject to Ṗ FSP = A(λ)P FSP P FSP (t 0 ) = P(t 0 ) P FSP (t 1 ) = P(t 1 ). P FSP (t N 1 ) = P(t N 1 )
Tsien Lab, UCSD Agar Plate of Fluorescent Bacteria Colonies Flow cytometry protein A histogram joint pdf protein B histogram
Identification of lac Induction A Model φ k L B LacI LacI δ L φ Probability Density φ w G GFP δ G φ Experiment laci LacI laci Promoter laci lac 0 hr 3 hr 4 hr 5 hr q =0 GFP w G = q =0 q =1 q =2 q =5 δ L = δ (0) L + δ(1) L [IPTG] IN 1+α[LacI] η, q =2 q =2 q =4 IPTG IN = IPTG OUT (1 e rt ) q =0 q =2 q =2 q =3 k G GFP 9 unknown parameters! E. coli strain Total DL5905 Florescence: GFP + Background (Arbitrary Units) 0 hr 3 hr 4 hr 5 hr Induced Cwith different IPTG concentrations: 5,10, 20, 40, 100 um Induction times: 0, 1, 2, 3, 4, 5 hours before flow cytometry
Identified Parameters Identified Model vs. Experiment Model Predictions A B Probability Density C laci can specify the rate δ G =3.8 10 N s, corresponding to a half life of thirty minutes. The remaining seven parameters can then be identified as: k L =1.7 10 3 s 1 k G =1.0 LacI 10 1 s 1 η =2.1 δ (0) L =3.1 10 4 N 1 s 1 δ (1) L =5.0 laci Promoter 10 2 (µm N) 1 s 1 α =1.3 10 4 N η r =2.8 10 5 s 1 µ GFP = 220 AU σ GFP = 390 AU laci lac GFP where N refers to molecule number. Since the 0 hr assumed hr model 3 hr represents hr a simplified 4 hr hr description 5 hr hr of multiple events (folding dynamics, 10elongation, 2 4 spatial 10 2 motion, 10 4 etc...), 10 2 these 10 4 parameters 10 2 10 are 4 best viewed as empirical measurements in q the =0context of q the =1assumed model. q =2Still, it is possible q =5 to make some comparisons 0.1 between the identified parameters and previous analyses. First, the production and degradation rates of 0 LacI yield a mean number of k L /γ (0) L 5 molecules per cell at steady state in the absence of IPTG, on the same magnitude of wild-type levels of about ten per cell. Second, the level 0.1 of LacI q required =0 for half q =2 occupancy of q the =2lac operon q is =4[LacI] 1/2 = (1/α) 1/η =0.012 which compares well to values 0.006-0.6 molecules (10 11 10 9 M, Oehler et al., 1990). Third, 0a Hill coefficient of 2.1 is reasonable considering that LacI binds to the operon as a tetramer. Finally, the degradation rate LacI, δ (0) L 0.1 q =0 q =2 q =2 is close to q the =3 dilution rate of 3.8 10 4, reflecting the high stability of that protein. In addition to comparing the parameters to values in the literature, we have used the parameter set identified from 4, 10, and 20 µm IPTG induction to predict 0 the fluorescence under 40µM IPTG. Fig. 3C shows that these predictions match the 10 2 10 4 10 2 10 4 10 2 10 4 10 2 10 4 subsequent experimental measurements very well despite the vastly different shapes observed at the high Total induction Florescence: levels. GFP + Background (Arbitrary Units) 0 hr hr 3 hr hr 4 hr hr 5 hr hr With single cell experimental techniques such as flow cytometry, it has become possible to efficiently 0.1 measure the fluctuations in cellular species. When properly extracted and processed 0.05with rapidly improving computational tools, these measurements contain sufficiently rich information 0 as to enable the unique identification of parameters. In principle, this can be accomplished 10 10 22 10 10 44 when 10 2 accurate 2 10 44 distributions 10 22 10 4 are 4 measured 10 22 10 4 at 4 only two distinct time points. More time points are needed if the distributions are poorly measured, but the idea remains the 0.1 same. In this study we have used experimental measurements of cell variability to identify 0.05 the parameters of candidate models, and we have shown that noise and transient dynamics are 0 important 10 10 22 to 10 this 10 44 effort. 10 10 22 It is10 10 easy 44 to10 10 envision 22 10 4 the 4 next 10 22 iterative 10 44 step in the scientific process, where one will use these identified models and design the next set of experiments to improve Total Florescence: GFP + Background (Arbitrary Units) the identification. Hence, the proposed exploitation of single cell measurements and stochastic B. Munsky, B. Trinh, M. Khammash, Nature Molecular Systems Biology, in press., Model Experiment
Conclusions Randomness noise leads to cell-cell variability Stochastic models are necessary Some stochastic analysis tools available (more needed) Kinetic Monte Carlo Moment approximation Linear noise approximation (van Kampen) Density computation (FSP) Noise reveals network parameters Enabling technologies: flow cytometry and FISH/microscopy A small number of transient measurements suffices FSP exploits full pdf measurements Cellular noise (process noise) vs. measurement noise (output noise)
Acknowledgement Brian Munsky, UCSB, LANL Brooke Trinh UCSB (lac induction)