CS-E5880 Modeling biological networks Parameter estimation for biological networks

Transcription

1 CS-E5880 Modeling biological networks Parameter estimation for biological networks Jukka Intosalmi Department of Computer Science Aalto University January 16, 2018

2 Outline ODE model calibration as an optimization problem Local gradient based optimization Identifiability and reparameterization ODE model calibration as a statistical estimation problem Practical considerations

3 Parameter estimation Parameter estimation is an important part of ODE modeling Estimation of the reaction rates and initial values Testing if a hypothetical model can produce observed dynamics Model construction in a data-driven manner Experimental data is typically noisy

4 Parameter estimation as an optimization problem (1) Let us consider a one dimensional initial value problem dx dt = f (x, θ), x(0) = x 0, where θ = (θ 1,..., θ d ) is a parameter vector. The system has the solution x(t, θ). The solution can be obtained numerically (recall Euler method) The observed values are denoted by y(t i ), i = 1,..., n where t i are the measurement times.

5 Parameter estimation as an optimization problem (2) Our goal is to minimize the distance between the solution of the ODE model and data In other words, we wish to minimize the objective function L(θ) = n (y(t i ) x(t i, θ)) 2. i=1 For ODE models, closed-form solutions are only rarely available and numerical optimization techniques need to be used. Typically a non-convex optimization problem

6 Optimization techniques Global and local optimization techniques Figure: Illustration of optimization problem (from Fröchlich et al., 2017) In this lecture, we will concentrate on deterministic local optimization Extentions to global optimization through multistart approach

7 Gradient based optimization Search for the nearest minimum of the objective function following the direction of the gradient

8 Gradient descent method 1. Set k = 0 and define initial values for the model parameters θ k 2. Evaluate the objective function L(θ k ) 3. Compute the gradient of the objective function at θ k, L(θ k ) 4. Line search: solve for the r = ˆr minimizing L(θ k r L(θ k )) 5. Set θ k+1 = θ k ˆr L(θ k ) 6. Compare the current objective function value with the previous value; if L(θ k+1 ) > L(θ k ) or if θk+1 θ k < ɛ then the solution is θ k 7. Otherwise, set k := k + 1 and go to step 2

9 Differentiation of the objective function To obtain the gradient, we need to differentiate the objective function L(θ) θ j = n i=1 2(y(t i ) x(t i, θ)) x(t i, θ) θ j. The gradient depends on x(t i,θ) θ j, i = 1,..., n; j = 1,..., d

10 Finite differences approximation for x(t i,θ) θ j The approximation is computed by perturbing the parameters x(t i, θ) x(t i, θ) x(t i, θ + he j ) θ j h where e j is the jth unit vector and h is sufficiently small constant. Requires several numerical solutions for the ODE system No generic way of choosing the constant h In general finite differences approach may result in a poor approximation Better results can be obtained using the sensitivity equations

11 Sensitivity equations We can take a total derivative of the ODE system w.r.t. θ j d dθ j dx dt = d dθ j f (x, θ). By evaluating the total derivative and rearranging, we obtain d dt dx dθ j = f (x, θ) x dx + dθ j f (x, θ) θ j. The sensitivity equation for the parameter θ j can be written in the form ds j f (x, θ) f (x, θ) = s j +, dt x θ j where s j (t) = x(t,θ) θ j are called sensitivities.

12 Solving the sensitivities Analytical formulation for s j (t) = x(t,θ) θ j The sensitivities and the original ODE model form a coupled system and they are solved simultaneously using numerical solvers. The accuracy of the computed sensitivities depends only on the numerical error of the ODE solver!

13 Comparing the finite differences and sensitivity equation approaches Figure: From Raue et al., 2013

14 Example of sensitivity equation construction (1) Let us consider the initial value problem dx dt = α + θ x 1 x + K θ 2x, x(0) = x 0 }{{} f (x,θ) where θ 1 and θ 2 are unknown rate parameters. We have f (x, θ) K = θ 1 x (x + K) 2 θ 2 f (x, θ) = x θ 1 x + K f (x, θ) = x. θ 2

15 Example of sensitivity equation construction (2) The sensitivity equations together with the actual ODE model form the following ODE system dx dt = α + θ x 1 x + K θ 2x ( ) ds 1 dt = K θ 1 (x + K) 2 θ 2 s 1 + x x + K ( ) ds 2 dt = K θ 1 (x + K) 2 θ 2 s 2 x with x(0) = x 0 and s 1 (0) = s 2 (0) = 0.

16 Example of sensitivity equation construction (3) And the system can be solved x s s Time (t) Time (t) Time (t)

17 Sensitivity equation in the general form Let us consider an m-dimensional (x R m ) initial value problem dx dt = f(x, θ), x(0) = x 0, where θ = (θ 1,..., θ d ) is a parameter vector. The sensitivity equations can be expressed in the general form d dx dt dθ = f(x, θ) x dx dθ f(x, θ) +. θ

18 Practical considerations In practice, it is beneficial to carry out the parameter estimation using log-transformed parameters Automated construction of the sensitivity equation Selection of the numerical solver (e.g. Sundials CVODES) Least squares optimization has been used in many applications

19 Global optimization through multistart approach A non-convex optimization problem Multiple local minima Parameter space explored using multiple optimization runs Parallelizable for high-dimensional problems Figure: Illustration of multistart approach (from Raue et al, 2013)

20 Choosing the initial points for multistart (1) Latin hypercube sampling (LHS) Figure: From Raue et al, 2013

21 Choosing the initial points for multistart (2) Figure: From Raue et al, 2013

22 Statistical formulation of the parameter estimation problem Formulating the existing information about the system in statistical terms If we assume normally distributed measurement errors, we can write y(t i ) = x(t i, θ) + ɛ(t i ), where ɛ(t i ) N(0, σ 2 i ). This is equivalent to y(t i ) N(x(t i, θ), σ 2 i ). By assuming independent measurements, we can express the likelihood of the data given the parameters in the form l(θ) = n 1 exp ( (y(t i) x(t i, θ)) 2 ) 2πσi 2 2σi 2. i=1

23 Maximum likelihood estimation The maximum likelihood principle is to choose the value θ = ˆθ which maximizes l(θ). In other words, the maximum likelihood estimate is It is equivalent to solve ˆθ = arg max l(θ) θ ˆθ = arg min L(θ), θ where L(θ) = 2 log(l(θ)) is the negative log-likelihood.

24 Negative log-likelihood (1) n L(θ) = 2 log i=1 1 2πσi 2 exp ( (y(t i) x(t i, θ)) 2 ) 2σ 2 i ( n = log(2πσ2 i ) 1 ( (y(ti ) x(t i, θ)) 2 ) ) 2 σ 2 i=1 i n = (log(2π) + 2 log(σ i ) + (y(t i) x(t i, θ)) 2 ) = i=1 n (2 log(σ i ) + (y(t i) x(t i, θ)) 2 ) + C 1 i=1 σ 2 i σ 2 i

25 Negative log-likelihood (2) If σ i = σ, i = 1,..., n, we have L(θ) = n (2 log(σ) + (y(t i) x(t i, θ)) 2 ) + C 1 i=1 σ 2 = 1 n σ 2 (y(t i ) x(t i, θ)) 2 + C 2 i=1

26 Parameter identifiability In many cases, the parameters are not well determined Infinitely many parameter settings may have equal likelihood The effect of changing one parameter can be compensated by tuning other parameters Figure: From Raue et al, 2010

27 Profile likelihood (1) Parameter identifiability can be studied e.g. using the profile likelihood method. The profile likelihood is defined by PL j (p) = where LL is the log-likelihood. max LL(θ), θ {θ θ j =p} Structural and practical non-identifiability

28 Profile likelihood (2) Figure: From Raue et al, 2010

29 Reparameterization example (1) Modeling metabolic switching in yeast cells. GLUCOSE STATE ethanol production ETHANOL STATE QUIESCENT STATE low glucose level induced transition m g m e m q Figure: Schematic illustration of the model. (Intosalmi et al., unpublished)

30 Reparameterization example (2) dm g dt dm e dt dm q = β 2 m g + β 3 m e dt dg dt = µ 1 m g g γ 1 de dt = µ 1 m g g µ 2 m e e γ 2 γ 3 = µ 1 m g g β 1 1 g + K mg β 2 m g = µ 2 m e e + β 1 1 g + K mg β 3 m e The model output to be linked with data is m g + m e + m q = m.

31 Reparameterization example (3) The initial level of glucose is non-negative Glucose level is a latent variable Normalization w.r.t. the initial glucose level g(0) Formally, Further, we can reparameterize where x 1 = m g, x 4 = x 4 (0) = 1. 1 dg g(0) dt = µ 1 m g g. g(0)γ 1 dx 4 dt = θ 7x 1 x 4, g g(0), θ 7 = µ 1 γ 1, and the initial condition is

32 Reparameterization example (4) In the case of ethanol, we set γ 3 = 1. This results in the expression which has the dimensionless form de dt = µ 1 γ 2 m g g µ 2 m e e dx 5 dt = θ 8x 1 x 4 θ 5 x 2 x 5, where x 5 = e, θ 5 = µ 2, and θ 8 = µ 1 γ 2.

33 Reparameterization example (5) The full system can then be written in the form dx 1 dt = θ 1 1x 1 x 4 θ 2 x 1 θ 4 x 1 x 4 + θ 3 dx 2 dt = θ 1 5x 2 x 5 + θ 2 x 1 θ 6 x 2 x 4 + θ 3 dx 3 dt = θ 4x 1 + θ 6 x 2 dx 4 dt = θ 7x 1 x 4 dx 5 dt = θ 8x 1 x 4 θ 5 x 2 x 5, where (x 1, x 2, x 3, x 4, x 5 ) = (m g, m e, m q, g, e), θ 2 = β 1, θ 3 = K, θ 4 = β 2, and θ 6 = β 3.

34 Reparameterization example (6) Figure: Fitted model (Intosalmi et al., unpublished)

35 T helper cell differentiation example Naive T cells sense the environment and react to different cytokine milieu and differentiate to functionally different T helper subsets Proper regulation and balance between various lymphocyte subsets is central the human immune system For example, so-called T helper 1 cells are involved in defense against intracellular bacteria Molecular mechanisms which control T helper cell differentiation are not at all fully understood A more thorough understanding would allow e.g. better drug design to modulate immune response and help in autoimmune diseases

36 T helper cell differentiation example (2) Different T cell subsets Figure: From (Brusselle et al, 2011)

37 T helper cell differentiation example (3) Immunity Key factors involved Sequential in and Th1 Cell driving Induction T helper by IFN-g 1 differentiation and IL-12 include IFN-γ, T-bet, IL-12 and IL-12Rβ2 The standard/assumed molecular model, until recently, has been the following A B CFigure: From (Schulz et al, 2009) D

38 mrna and protein concentration, respectively. Since naïve cells express T helper cell differentiation example (4) neither IL-12R 2 nor IFN-, only for T-bet we assumed a basal transcript considered had the following form, only differing in the functions f gene de The actualtranscription molecularrate. mechanisms are assumed to follow the following ODE system dtbet dt mrna f 1 Tbet IFN Pr ot,re c Pr ot, Ag Tbet Tbet mrna dtbet dt Pr ot Tbet mrna Tbet Tbet Pr ot dre c dt mrna f Re c Tbet Pr ot, Ag Re c Re c mrna dre c dt Pr ot Re c mrna Re c Re c Pr ot difn dt mrna f IFN Tbet Pr ot,re c Pr ot, Ag IFN IFN m RNA difn dt Pr ot IFN m RNA IFN IFN Pr ot The Figure: rates of From transcription (Schulz are ettaken al, 2009) as saturating, hyperbolic functions Independent activation by multiple signals is described by additive te

39 Sequential Th1 Cell Induction by IFN-g and IL-12 T helper cell differentiation example (5) Figure 3. The Two-Loop Mode (A) Network structure of the literature (B) Schematic representation of t described in Figures 1 and 2: IL-1 T-bet expression and antigen-depen (C and D) Circles represent the e Th cells, isolated from C57BL/6 mice tative example of five independent e shown. Lines represent the best fit of A B One-Loop Model, Can Explain Kinetics of T-Bet, IL-12Rb2, during Th1 Cell Priming loop model. model, integrating the regulatory sion of IL-12Rb2. C D T-bet, IL-12Rb2, and IFN-g mrna over 5 days of Th1 cell differentiatio (C) or the two-loop (D) model. To test the model, mrna time-course measurements have been collected after inducing the Th1 differentiation ODE model is fit to the experimental data using annealing Figure: From (Schulz et al, 2009) r r r flow cytometry in the early phase and in the late phase in WT and Ifngr / cells in the presence and absence of IL-12 and tion of IL-12Rb2 and advance the second wave of T-bet r (Figures 4D and 4E, left). Usi block activation of the calcine pathway, we found that ex inhibition of antigen-depende after 24 hr of TCR stimulation the predicted effect (Figures 4 right). It has been shown previ the induction of IFN-g and IL T-bet could be indirect and m T-bet-dependent repression o r which otherwise would repr (Usui et al., 2006). In the pr T-bet, elevated Stat4 would enh signaling, induce IFN-g, and IL-12Rb2 chain expression. We Stat4 and GATA-3 in the T cell activation and found, in g correlation with T-bet con (Figure S3). In particular, we fin ence in Stat4 and GATA-3 mrn sion when comparing WT and r cient cells in the first 48 hr of s a phase when T-bet and IFN-g are vastly different (Figures 4A Similarly, at hr after th stimulation, comparison of ce in the presence and absence of IL-12 showed that, al expression of T-bet and IL-12Rb2 differed greatly,

40 In Model C, induction of T-bet by IL-12 was added to Mod Sequential Th1 Th1 Cell Cell Induction Induction by IFN-g by IFN-g and and IL-12 IL-12 T helper cell differentiation example (6) Model C Ag (t ) IFN Re c A A B K1B Ag (t ) K2 IFN Pr ot K3 Re c Pr Figure ot Figure 3. The 3. One-Loop One-Loop Mod Tbet Pr ot f Kinetics Kinetics of T- o Re c 4 Immunity, Volume 30 K8 Tbet Pr ot during during Th1 Cell Th1 (A) Network (A) Network struc loop model. loop model. In Model A (one-loop model), T-bet (B) Schematic (B) r In Model is induced D (two-loop by synergistic model), action antigen-dependent of IFN- and repr ant model, model, integratin signals and IL-12R 2 is induced by dependent T-bet: induction of T-bet was added to model described described A: in Figi T-bet T-bet expression sion of sion IL-12Rb2. of Model A (One loop model) Model D (Two - loop model) (C and (C D) andcircle D) C C T-bet, D D T-bet, IL-12Rb2, Ag (t ) IFN Pr ot Ag (t ) IFN Pr ot Re c over 5over days 5 days of T ftbet 2 Pr ot ftbet 2 3 K Ag (t ) K IFN Th cells, Th cells, isolated 1 2 Pr ot K1 Ag (t ) K2 IFN Pr ot K3 Re c Pr ot tative tative example examof Tbet Pr ot Tbet K shown. shown. LinesLine repr fre c 4 Pr ot 4 f Re c 4 K Tbet (C) or(c) theor two-loo the 8 Pr ot K8 Tbet Pr ot K4 Ag (t ) Pr ot ot f Something wrong with thetbet model? 2 another network 3 model Pr Figure: From (Schulz et al, 2009) In Model B, an additional Term was added, describing repression of IL-12R 2 by ant signals (Ag): tion tion of IL-12Rb of the secon w the second r r r r (Figures (Figures 4D an 4 Model B block block activatio pathway, pathway, we Ag (t ) IFN 9 Pr ot ftbet 2 inhibition inhibition of a K1 Ag (t ) K2 IFN Pr ot after after 24 hr24ofht

41 Sequential Th1 Cell Induction by IFN-g and IL-12 T helper cell differentiation example (7) A B C D Figure 3. The Two-Loop Model, Not the One-Loop Model, Can Explain Expression Kinetics of T-Bet, IL-12Rb2, and IFN-g during Th1 Cell Priming (A) Network structure of the literature-based oneloop model. (B) Schematic representation of the two-loop model, integrating the regulatory interactions described in Figures 1 and 2: IL-12-dependent T-bet expression and antigen-dependent repression of IL-12Rb2. (C and D) Circles represent the expression of T-bet, IL-12Rb2, and IFN-g mrna, measured over 5 days of Th1 cell differentiation with naive Th cells, isolated from C57BL/6 mice. A representative example of five independent experiments is shown. Lines represent the best fit of the one-loop (C) or the two-loop (D) model. tion of IL-12Rb2 and advanced onset of the second wave of T-bet expression r r (Figures 4D and 4E, left). Using CsA to block activation of the calcineurin-nfat pathway, we found that experimental inhibition of antigen-dependent signals after 24 hr of TCR stimulation resulted in the predicted effect (Figures 4D and 4E, right). It has been shown previously that the induction of IFN-g and IL-12Rb2 by T-bet could be indirect and mediated by T-bet-dependent repression of GATA-3, r r which otherwise would repress Stat4 (Usui et al., 2006). In the presence of T-bet, elevated Stat4 would enhance IL-12 signaling, induce IFN-g, and increase IL-12Rb2 chain expression. We measured Stat4 and GATA-3 in the course of T cell activation and found, in general, no correlation with T-bet concentrations (Figure S3). In particular, we find no difference in Stat4 and GATA-3 mrna expression when comparing WT and Ifngr-deficient cells in the first 48 hr of stimulation, r r a phase when T-bet and IFN-g expression are vastly different (Figures 4A and 4C). Similarly, at hr after the onset of Figure: From (Schulz et al, 2009) stimulation, comparison of cells cultured flow cytometry in the early phase and in the late phase in WT in the presence and absence of IL-12 showed that, although the /

42 T helper cell differentiation example (8) Immunity The ODE model is predictive Sequential Th1 Cell Induction by IFN-g and IL-12 A B r r Figure 4. The Two-Loop Model Can Simulate and Predict Expression Kinetics under Various Experimental Conditions (A C) The left column shows the expression kinetics of T-bet (A), IL-12Rb2 (B), and IFN-g mrna (C) simulated on the basis of the two-loop model (Figure 3B) under Th1 cell-inducing conditions (black) and in the absence of IFN-g (blue) or IL-12 signaling (red). All parameters were simultaneously fitted to experimental data. As shown in the right column, naive Th cells, isolated from C57BL/6 (black and red) or Ifngr / mice (blue), were stimulated for 5 days with IL-12 (black and blue) or IFN-g, and blocking antibodies to IL-12 (red). T-bet (A), IL-12Rb2 (B), and IFN-g mrna kinetics (C) were quantified. The experiment was performed three times; one representative example is shown. (D and E) As shown in the left column, expression kinetics of T-bet (D) and IL-12Rb2 mrna (E) are compared in silico in cells stimulated with antigen for 24 hr (red) or 48 hr (black) under Th1 cellinducing conditions. As shown in the right column, naive Th cells, isolated from C57BL/6 mice, were cultured under Th1 cellinducing conditions, and cyclosporine A (CsA) was added after 24 hr of stimulation as indicated (red). T-bet (D) and IL-12Rb2 mrna (E) were quantified over 5 days of culture. One representative example out of three independent experiments is shown. r r C expression. Through a mathematical model we could show that these regulatory interactions between T-bet, IFN-g, IL-12Rb2, and the antigen stimulus can explain the major features of the expression kinetics during primary activation, such as two-peaked T-bet expression with a single IFN-g peak and delayed induction of IL-12Rb2. Therefore, we propose that the model described here constitutes the core of the gene network r r controlling Th1 cell differentiation. T-bet imprints the Th1 cell phenotype in that histones D are modified and chromatin is remodeled in regulatory regions on the Ifng gene (Chang and Aune, 2005; Hatton et al., 2006; Shnyreva et al., 2004). Here, we show a quantitative dose dependence of the frequency of IFN-g-producing cells in a recall stimulation on the expression of T-bet in the late phase of primary activation (>48 hr). Importantly, this T-bet dose dependence of Th1 cell priming held true for different stimulation r protocols (e.g., with or without IFN-g signaling), r strongly suggesting that T-bet expression in a specific E time window, rather than other factors, is the decisive event. The underlying mechanism might involve epigenetic regulation rather than direct transcriptional activation, given that the observed dose dependence is maximal between 72 and 120 hr and decreased after 144 hr when the recall response is induced. A temporal constraint on Th1 cell priming has been shown previously in that memory expression of the Ifng gene requires entry into the S phase of the first r r cell cycle (Bird et al., 1998; Richter et al., 1999). However, because this occurs at 20 hr of stimulation, expression through a positive feedback loop of IL-12R and T-bet. whereas the T-bet effect is strongest at 96 hr, S phase entry The expression of T-bet in the late phase, and not in the early seems to be only one of a number of necessary events that allow phase, correlates strongly with the frequency of IFN-g-producing or mediate T-bet action. In agreement with previous reports, we cells in a later recall response, and thus this late IL-12-dependent show that Runx3 and Hlx, which are induced by and cooperate T-bet induction governs the imprinting of the Th1 cell for IFN-g re- with T-bet, were specifically expressed in the late phase of Figure: From (Schulz et al, 2009)

43 Figure S6. Best fit by Model A-D T helper cell differentiation example (9) (A) Network structure of Model A, B, C and D. (B) circles: Expression level of T-bet (top), IL-12R 2 (middle) and IFN- (bottom) mrna was measured over 5 days of Th1 differentiation More using networks naive Th cells, models isolated from C57BL/6 amice. fundamental lines: The best fit of Models question A-D. concerns choosing the correct biological network model Figure: From (Schulz et al, 2009)

44 References Fröchlich et al. (2017) Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes, arxiv preprint arxiv: Raue et al. (2013) Lessons learned from quantitative dynamical modeling in systems biology, PLoS One, 8(9), e Raue et al. (2010) Identifiability and observability analysis for experimental design in nonlinear dynamical models, Chaos, 20, Brusselle GG et al, (2011) New insights into the immunology of chronic obstructive pulmonary disease, Lancet, 378(9795): Schulz et al, (2009) Sequential Polarization and Imprinting of Type 1 T Helper Lymphocytes by Interferon-g and Interleukin-12, Immunity, 30,