Inference of genomic network dynamics with non-linear ODEs

Transcription

1 Inference of genomic network dynamics with non-linear ODEs University of Groningen ernst 11 June 2015 Joint work with Mahdi Mahmoudi, Itai Dattner and Ivan Vujačić

2 Motivation: dynamic genomic processes The amount of RNA of a particular gene depends on the rate of DNA decay (depends on temperature,...) the amount of transcription factor the rate of transcription (depends on other factors,...)

3 Law of Mass Action Given a basic reaction A + B k 1,k 1 C the rate of forward and backward and backward reactions is linearly proportional with concentration A, B and C respectively: d[a] dt = k 1 [C] k 1 [A][B].

4 Enzymes (typically) large proteins catalysts that change the speed of reaction, but not its extent. highly specific: speed up (typically) a single biochemical reaction. are highly regulated themselves by: phosphorylation calcium ATP their own products, etc.

5 Basic problem of enzyme kinetics Let S be a substrate which together with enzyme E results in a product P: S + E P + E. Problem: according to Law of Mass Action rate increases linearly with [S], but it actually saturates. Michaelis and Menten (1913): an intermediate complex C S + E k 1,k 1 C k 2 P + E.

6 Pseudo steady-state approximation Mass action equilibrium: (k 1 + k 2 )[C] = k 1 [S][E] combined with the finite amount of enzyme m: m = [C] + [E], We get so K[C] = [S](m [C]), with K = (k 1 + k 2 )/k 1 [C](K + [S]) = [S]m. The rate of production of product P v P = k 2 [C] = k 2[S] K + [S]

7 E-Coli SOS system 30 genes controlled by protein called LexA. 1 ODE per gene, particular kinetics parameters.

8 Modelling transcription kinetics in a SIM Mean transcription dynamics follow Michaelis-Menten kinetics: µ i (t) = p i (t, η) δ i µ i (t), where p i (t, η) = RNA production term of gene i by protein η, δ i µ i (t) = natural RNA decay of gene i. If gene i only depends on activity η of one protein: η(t) p i (t, η) = α i + β i γ i + η(t) }{{} Activator or p i (t, η) = α i + β i 1 γ i + η(t) }{{} Repressor where α i = basal transcription rate of gene i, β i = maximal reaction-rate for gene i, γ i = half-saturation constant.

9 Cooperativity Enzyme can bind two substrate molecules at different binding sites: which results in the following rate of production of P v P = (k 2K 2 + k 4 [S])m[S] K 1 K 2 + K 2 [S] + [S] 2.

10 Independent binding sites If there are 2 independent binding sites to which enzyme can bind: which results in doubling the rate of production of P v P = k 2 [C] = 2k 2[S] K + [S]

11 ... or more complex systems: fatty acid β-oxidation Differential equations play crucial role in science

12 Statistical model Consider x (t) = f (x(t), t; θ), t [0, T ], Noisy observations of solution x(t; θ, ξ) are available: y(t i ) = x(t i ; θ, ξ) + ε(t i ), i = 1,..., n, where 0 t 1 t n T and with initial values x j (0) = ξ j. Goal: inverse ODE problem 1 Estimate the vector of parameters θ. 2 When initial values ξ are unknown, they are also estimated.

13 Possible approaches 1 Maximum likelihood (Khanin et al. 2007, Xue et al. 2010): PRO: has guaranteed properties of estimator. CON: requires solution of ODE. 2 Reproducing Kernel Hilbert Space (Gonzalez et al. 2014): CON: has no guaranteed properties of estimator. PRO: does not require solution of ODE. 3 Collocation Method (Ramsay et al. 2007): PRO: consistency. PRO: does not require solution of ODE. CON: Iterative procedure with tuning parameters

14 AIM: INFER DYNAMIC PARAMETERS OF SYSTEM Let θ = (θ, ξ), including initial state. Our proposed framework: iteration of 1. x(θ) = argmin x Xm T α,γ (x θ), 2. θ n = argmin θ Θ M n (θ x(θ), Y). Aim Define T α,γ and M n such that: It yields asymptotically efficient estimator. It can handle partially observed systems.

15 Well-posedness in sense of Hadamard Let F : X Y where X, Y are linear normed spaces and solve: F (x) = y, (1) x X, y Y. The problem (1) is well-posed in the sense of Hadamard on (X, Y) if: 1 The solution of (1) exists. 2 It is unique. 3 It is continuous with respect to y. The problem (1) is ill-posed on (X, Y) if it is not well-posed.

16 Solving ODEs Consider for fixed θ. x (t) = f (x(t), t; θ), t [0, T ], Suppressing dependence on θ, define F (x) = x f (x,, θ), then ODE is equivalent to F (x) = 0 d. In principle, this is an ill-posed problem: lack of uniqueness

17 Quasi-solution for ill-posed problems Equation F (x) = y, can be solved on a set S X by minimizing objective functional J (x) = F (x) y 2. Quasi-solution Minimizer of J on S is called quasi-solution, pseudo solution or least squares solution of F (x) = y. Remark: Idea dates back to beginning of 19th century: Gauss, Legendre.

18 J : ODE objective functional Given F (x) = x f (x,, θ), the corresponding ODE objective functional is J (x) = x f (x,, θ) 2 2,w. where di=1 L 2 norm, x 2,w = w T i 0 x i 2(t)dt. with possible weights w = (w 1,..., w d ). Minimizing J = solving ODE

19 What subset S? Finite-dimensional approximation Consider finite-dimensional subspaces X m X (m = 1, 2,...), such that 1 X 1 X m=1 X m is dense in X. X m C 1 [0, T ] linear subspace with basis {h 1,..., h m }, i.e. x X m m x i (t) = β ik h k (t) = βi h(t), k=1 such as cubic splines with increasing knots. Approximate ODE solution The solution x = argmin x Xm x f (x,, θ) 2,w, is quasi-solution of ODE. Remarks: in statistics literature X m s are called sieves.

20 What subset S? Finite-dimensional approximation Approximate ODE solution The solution x = argmin x Xm x f (x,, θ) 2,w, is quasi-solution of ODE. Remarks: in statistics literature X m s are called sieves.

21 Tikhonov regularization Ω is stabilizing functional, which incorporates a priori information on smoothness of solution x. is usually given by norm or semi-norm on X. Tikhonov regularization involves minimization of Tikhonov functional T α (x) = J (x) + αω(x), where α 0 is regularization parameter For ODE we consider norm in Sobolev space {H 2 [0, T ]} d d T Ω(x) = v i {x i (t)} 2 dt. i=1 0

22 Generalized Tikhonov regularization S is similarity functional, which incorporates a priori information on values of x. Generalized Tikhonov regularization involves minimization of generalized Tikhonov functional T α,γ (x) = J (x) + αω(x) + γs(x), where γ 0 is penalty parameter. We consider (for example) minus log-likelihood: d 1 n S(x) = log p(y i (t j ) x i (t j )). i=1 j=1

23 General outline of our method Consider x β = β t h, such that m x i (t) = β ik h k (t) = βi h(t). k=1 1 For fixed θ, GTR solution x = x β is given by 2 For fixed x, obtain solution β = argmin β R dmt α,γ (x β ). θ n = argmin θ Θ M n (θ x(θ), Y). Then iterate steps 1 and 2.

24 Consistency and asymptotic efficiency Theorem (Consistency) Let Assumptions 1-5 from Qi and Zhao (2010) hold. If as n 1 α n 0 2 γ n 0 then θ n θ 0 = o P (1). Theorem (Asymptotic normality & efficiency) Let Assumptions 1-6 from Qi and Zhao (2010) hold. If α n = o(n 2 ) and γ n = o(n 2 ), then θ n is asymptotically normal and efficient.

25 ODE Systems linear in parameters Nonlinear ODE systems that are linear in parameters θ x (t) = g(x(t))θ, where g : R d R d p. Example 1. Lotka-Volterra system { x 1 (t) = θ 1 x 1 (t) θ 2 x 1 (t)x 2 (t), x 2 (t) = θ 3x 2 (t) + θ 4 x 1 (t)x 2 (t), is linear in parameter θ = (θ 1, θ 2, θ 3, θ 4 ) where ( ) x1 (t) x g(x(t)) = 1 x 2 (t) 0 0, 0 0 x 2 (t) x 1 x 2 (t) i.e. (x 1 (t), x 2 (t)) = g(x(t))θ.

26 Example 2. FitzHugh-Nagumo FitzHugh-Nagumo system { x 1 (t) = c{x 1 (t) x 1 (t) 3 /3 + x 2 (t)}, x 2 (t) = 1 c {x 1(t) a + bx 2 (t)}, is linear in parameter θ = (c, 1/c, a/c, b/c) where g(x(t)) = i.e. (x 1 (t), x 2 (t)) = g(x(t))θ. ( x1 (t) x1 3(t)/3 + x ) 2(t) 0 0 0, 0 x 1 (t) 1 x 2 (t)

27 GTR: Window-smoothing Approach We consider X m = piecewise constant functions on m levels. and consider Tikhonov functional (non-dependent on θ!) T 0, (x) = S(x). Obtain x = argmin x X d m T,0 (x) and estimate θ with objective function M n (θ) = T 0 x (t) f ( x(t), θ) q w(t)dt, 1-step GTR estimator!

28 STEP 1. Window smoother for x The window smoother of x is defined as x n (t) = 1 y(t j ), S(t) t j S(t) t S(t). x time x time

29 STEP 2. Objective function for θ Estimate parameters θ and ξ by minimizing M n (ξ, θ) = T 0 x n(t) ξ t 0 g( x n (s)) ds θ 2 dt

30 STEP 2. Objective function for θ Estimate parameters θ and ξ by minimizing M n (ξ, θ) = where T 0 ( T Id Ân = (ξ, θ) x n(t) ξ Â n Ĝ n (t) = Â n = B n = B n t t 0 ) ( ξ θ 0 T 0 T 0 2 g( x n (s)) ds θ dt ) ( T x n(t)dt Ĝn(t) x n (t)dt g( x n (s))ds, Ĝ n (t)dt, T 0 Ĝ n (t) Ĝn(t)dt. ) ( ) ξ. θ

31 STEP 2. Standard least squares form of estimators Minimizing previous quadratic form yields ( ) 1 ( ( ξ, θ) T T Id = Ân  0 x ) n(t)dt n B T n 0 Ĝn(t) x. n (t)dt The estimators of ξ and θ have the form: ξ n = ( TI d θ n = B 1 n T 0 ) Ân B 1 1 T { n  n 0 Id Ĝn(t) { x n (t) ξ n }dt, Ân B 1 n Ĝ n (t) } x n (t)dt,

32 STEP2.... even better! Explicit forms i Ĝ n (t i ) = g( x n (S m )), m=1 Â n (i) = tĝn(t) t i t i (2i 1) 2 g( x n (S i )), Â n = T Ĝn(T ) 2 I (2i 1)g( x n (S i )). 2 i=1 B n = Ĝn(T ) Ân 1 I g( x n (S i )) t 2Ĝn(t) t i 2 t i=1 i I (3i 2 1)g( x n (S i )) g( x n (S i ))) + 3 I 1 (2i 1)Ri g( x n (S i )), 6 2 i=1 i=1 where R i = I m=i+1 g( x n (S m )).

33 Postscript Computational complexity Computing x n : O(nd). Computing ξ n and θ n : O(p 3 + d 3 + ndp 2 ). Consistency: Window-based estimators of θ n and ξ n are n consistent. A reviewer s comment (Statistics and Computing): It s an unusual idea and at first glance it is surprising that it works.

34 A simulation example FitzHugh-Nagumo system, given by the equations, is linear in the parameter FhNdata consist of x 1 (t) = c{x 1(t) x 1 (t) 3 /3 + x 2 (t)}, x 2 (t) = 1 c {x 1(t) a + bx 2 (t)}, θ = (θ 1, θ 2, θ 3, θ 4 ) = (c, 1/c, a/c, b/c). 41 equally spaced observations on the interval [0,20] obtained from the above model with noise levels σ 1 = σ 2 = 0.5 and parameters a = 0.2, b = 0.2 and c = 3.

35 FhNdata from R package CollocInfer x time x time

36 Window estimator provides a sensible initial guess Window, Window + Generalized Profiling, Generalized Profiling u = (1, 1, 1). Estimation of parameters in Fitz-Hugh Nagumo system Method Ini guess a = 0.2 b = 0.2 c = 3 Rel. error Time(sec) Window NO Ramsay Window u u u u u u 6e+99 6e+99 1e+56 6e

37 Application 1. E-Coli SOS system 30 genes controlled by protein called LexA. 1 ODE per gene, particular kinetics parameters.

38 Application 1. SOS system in Ecoli: Model m genes x 1 (t),..., x m (t) and one transcription factor η(t). Model: ẋ k (t) = β k 1 γ k + η(t) + ϕ k δ k x k (t), for k = 1,..., m, t [0, T ]. Gene-dependent kinetics parameters: β k, γ k, δ k and ϕ k.

39 Application 1. SOS system in Ecoli: Data Data set of 14 expression genes. The genes are targets of the master repressor LexA and their expression is studied under UV exposure (40J/m2 ). Abundance of the mrna molecules measured at 0, 5, 10, 20, 40 and 60 minutes. The LexA abundance is unobserved.

40 Application 1. SOS system in Ecoli: Results Reconstructed the activity of the repressor η(t) gene dini. Reconstructed TF LexA Reconstructed gene dini TFA Gene Expression Time Time Confidence intervals for the parameters CI 95% (β) = (0.56, 1.39) CI 95% (δ) = (0.15, 0.27) CI 95% (γ) = (1.71, 2.83), σ = 0.04.

41 Application 1. But not just that... The aim of statistics is not just to fit data, but also compare scientific hypotheses draw scientific conclusions from the models For example, the following scientific conclusions can be drawn: reca has the fastest effective production rate, r = β/(γ + η), uvrb the slowest. reca and umuc are heavily affected by RNA decay. recn, uvrb and umuc has a non-negligible basal production rate α, so they are possibly also transcribed by another TF.

42 Application 2. Auto-regulated system Streptomyces Streptomyces Coelicolor produces antibiotics. CdaR is a crucial regulator. TF CdaR also activates its own transcription.

43 x Application 2. Generalized MM formulation ẋ = β zm γ + z m + ϕ δx ż = ρx τz A B Stability F 1 High abundant Hysteresis loop mrna F 2 γ Bistability γ+ Low abundant mrna Stability z x β = 6 ρ = 0.5 δ = 0.5 τ = 1 ϕ = γ γ C D Value Stability γ = 2.5 x z Value Bistability γ = 7.5 x 1 z 1 x 2 z Time Time

44 Application 2. CdaR data and inference log(aic) A Abundance B mrna, SCO3217 TF, cdar ^ > Hill coefficient Time order A AIC selects Hill coefficient 2. B Data for autoregulated gene SCO3217 (cdar) and obtained smooth functions after estimating parameters. Parameter estimates lie in bistable region, which may provoke bimodal effects at populational level.

45 Discussion Strange that Statistics has not dealt with ODEs until now! Ramsay et al. (2007) has been an important first step. In this talk, We extended it via Generalized Tikhonov Regularization We simplified it via Window Smoothing approach. Genetics may be quite static, genomics is about dynamics. More good work is needed!