Sensitivity analysis of complex stochastic processes

Transcription

1 Sensitivity analysis of complex stochastic processes Yannis Pantazis join work with Markos Katsoulakis University of Massachusetts, Amherst, USA Partially supported by DOE contract: DE-SC February, 2013

2 Outline Introduction Discrete-time MC Continuous-time MC p53 reaction network Langevin process ZGB lattice model

3 Sensitivity analysis - Definition Definition: Sensitivity analysis is the study of the impact in the output of a mathematical model or system caused by perturbations in the input. Q: What could be an input? A: Initial data (deterministic or random), intrinsic model parameters (such as temperature, pressure, masses, reaction rates, etc.) Q: What could be an output? A: Output data, observables of the process (such as mean values, variances, higher moments, correlations, etc.), histograms. An input parameter is sensitive when even small changes to this parameter may significantly alter the system output.

4 Sensitivity analysis - Applications Perspectives on the design and control of multiscale systems, Braatz et al., 2006

5 Sensitivity analysis - Applications Optimal experimental design: Construct experiments in such a way that the parameters can be estimated from the resulting experimental data with the highest statistical quality. Various optimality criteria (A-optimality, D-optimality, E-optimality, etc.) are based on the Fisher information matrix. Robustness: The persistence of a system to a desirable state under the (controllable and/or uncontrollable) perturbations of external conditions. Identifiability: The ability of the experimental data and/or the model to estimate the model parameters with high fidelity. Reliability: To ensure that the performance of a system meets some pre-specified target with a given probability. Uncertainty quantification: The science of quantitative characterization and reduction of uncertainties in applications. For instance, not only the values of the model parameters might contain errors but also the model itself could be an approximation.

6 Sensitivity analysis - Global vs Local Global sensitivity analysis: Study the effect of an input parameter to the output under a wide range of values or under a specified distribution. Insensitive input parameters could be treated without big effort (assign a nominal value) while sensitive input parameters should be treated carefully. Variance-based methods for global SA are: Sobol indices (decompose variance in a ANOVA-like manner), Fourier amplitude sensitivity test (FAST), Morris method. Information-based methods for global SA are: relative entropy (or Kullback-Leibler divergence), mutual information (quantifies the input-output mutual dependence). Local sensitivity analysis: Fix input parameters to a value and study the effect of small perturbations around the fixed parameter to the output. Local SA is typically a prerequisite for global SA.

7 Outline Introduction Discrete-time MC Continuous-time MC p53 reaction network Langevin process ZGB lattice model

8 Deterministic SA System of ODEs: ẏ = f (t, y; θ), y(0) = y 0 R N Goal: Perform SA on the model parameters θ R K. Define sensitivity indices: s k = y θ k A new system of ODEs is derived and augmented to the previous: ṡ k = f y s k + f θ k, k = 1,..., K

9 Stochastic SA - Observable-based By stochastic process we merely mean either discrete-time Markov chains (DTMC) with possibly separable state space or continuous-time jump Markov chains (CTMC) with countable state space. Well-mixed reaction networks, spatially-extended on lattices systems, discretizations of stochastic differential equations are models which belong to the above classes of stochastic processes. A typical example for stochastic SA: {xt} is a MC, θ R is a model parameter and f is a test function (or observable). Compute S(θ, t) = θ E Pt θ [f (x)]

10 Stochastic SA - Observable-based Finite difference approach. Approximate parameter sensitivity with finite difference, S(θ, t) = E P [f (x)] E t θ+ɛ P θ t [f (x)] + O(ɛ) ɛ Applying typical sampling approaches, Ŝ(θ, t) = 1 n ( f (x θ+ɛ,i t ) f (xt θ,i ) ) nɛ with variance, i=1 var (f (xt θ+ɛ ) f (xt θ )) = var (f (xt θ+ɛ )) + var (f (xt θ )) 2cov (f (xt θ+ɛ ), f (xt θ )) Importance quantities: Bias due to finite difference. Variance due to sampling of two different stochastic processes.

11 Stochastic SA - Observable-based Likelihood ratio method, P. Glynn ( ). Parameter sensitivity is rewritten as, S(θ, t) = θ E P [f (x)] = f (x) t θ θ Pt θ (x)dx = E P θ t [f (x) θ log P θ t (x)] = E Q θ 0,t [f (x) θ log Q θ 0,t(x)] where Q0,t is the path distribution of the process in the interval [0, t]. Derivative-free approach. Variance increases with time.

12 Stochastic SA - Density-based A more holistic approach based on the probability densities of the process. Assuming that xt θ Pt θ (x)dx and xt θ+ɛ Pt θ+ɛ (x)dx, why not comparing directly the distributions? Majda and Gershgorin (2010) compared the distributions between a coarse-grained climate model and a statistical climate model based on relative entropy. Komorowski et al. (2011) computed the Fisher information matrix of the path distribution of a linearized CTMC process.

13 Outline Introduction Discrete-time MC Continuous-time MC Introduction Discrete-time MC Continuous-time MC p53 reaction network Langevin process ZGB lattice model

14 Preliminaries Introduction Discrete-time MC Continuous-time MC Restrict, for the moment, to DTMC processes and to the steady state regime. Steady state means that the probability density of the process an any time instant is independent of the particular time instant. For the parameter vector θ R K, a DTMC, {σ m } m Z+, is defined with path distribution in time instants 0, 1,..., M given by Q θ 0,M( σ0,, σ M ) = µ θ (σ 0 )p θ (σ 0, σ 1 ) p θ (σ M 1, σ M ) where µ θ (σ) is the stationary (or invariant) probability density function. p θ (σ, σ ) is the transition probability function. Similarly for the DTMC process, { σ m } m Z+, induced from the parameter vector θ + ɛ.

15 Relative entropy in path-space Discrete-time MC Continuous-time MC Suggest performing parameter sensitivity analysis by comparing the path distributions utilizing relative entropy. Definition: The path-wise relative entropy of Q0,M θ w.r.t. Qθ+ɛ 0,M is ( ) ( ) dq θ R Q0,M θ Q θ+ɛ 0,M 0,M := log dq θ+ɛ dq0,m θ 0,M ( ) Properties: (i) R Q0,M θ Qθ+ɛ 0,M 0 and ( ) (ii) R Q0,M θ Qθ+ɛ 0,M = 0 iff Q0,M θ = Qθ+ɛ 0,M ( ) ( a.e. ) (iii) R Q0,M θ Qθ+ɛ 0,M R Q0,M+1 θ Qθ+ɛ 0,M+1

16 Relative entropy decomposition Discrete-time MC Continuous-time MC Its not difficult to see that path-wise relative entropy at stationary regime is decomposed into two parts; one independent of time and one that grows linearly with time: ( ) R Q0,M θ Q θ+ɛ 0,M = E M 1 µ θ (σ 0 ) p θ (σ i, σ i+1 ) E i=0 µ θ (σ 0 ) M 1 i=0 log pθ (σ i, σ i+1 ) µ θ+ɛ (σ 0 ) M 1 i=0 pθ+ɛ (σ i, σ i+1 ) dσ 0 dσ M ( ) = MH Q0,M θ Q θ+ɛ 0,M + R ( µ θ µ θ+ɛ) where ( ) H Q θ 0,M Q θ+ɛ 0,M is the relative entropy rate (RER): ( ) [ ] H Q0,M θ Q θ+ɛ 0,M = E µ θ p θ (σ, σ ) log pθ (σ, σ ) p θ+ɛ (σ, σ ) d σ

17 SA - Relative entropy rate Discrete-time MC Continuous-time MC For long times, RER is a sensible measure of parameter sensitivity. Properties: Infer information about the path distribution. Consequently, it contains information not only for the invariant measure but also for the stationary dynamics such as metastable dynamics, exit times, time correlations, etc.. No need for explicit knowledge of invariant measure. Thus, it is suitable for reaction networks and non-equilibrium steady state systems. RER is an observable of known test function tractable and statistical estimators can provide easily and efficiently its value. Different ɛ s SA at different directions. However, the perturbed process is not needed to be simulated.

18 SA - Fisher information matrix Discrete-time MC Continuous-time MC RER reminds the discrete derivative SA method. Q: Can we avoid the (discrete) perturbation of θ by ɛ? A: Yes, by constructing the corresponding Fisher Information Matrix (FIM). It holds, under smoothness assumption on θ, that ( ) H Q0,M θ Q θ+ɛ 0,M = 1 2 ɛt F H ( Q θ 0,M ) ɛ + O( ɛ 3 ) where Fisher information matrix is defined as [ ] ( F H Q θ ) 0,M = Eµ θ p θ (σ, σ ) θ log p θ (σ, σ ) θ log p θ (σ, σ ) T d σ E

19 SA - Fisher information matrix Discrete-time MC Continuous-time MC FIM constitutes a derivative-free sensitivity analysis method. In optimal experimental design, the maximization of the determinant of the FIM constitutes the D-optimality test while the minimization of the trace of the inverse of the FIM constitutes the A-optimality test. Robustness on parameter perturbations as well as parameter identifiability can be inferred from the FIM. Wider FIM implies a more robust model. Steeper FIM implies more identifiable parameters. Eigenvalue analysis of FIM gives the most/least sensitive directions.

20 Discrete-time MC Continuous-time MC Preliminaries A continuous-time jump Markov process is fully determined by the transition rates, c(σ, σ ). The total rate is defined by λ(σ) = σ c(σ, σ ). For the parameter vector θ R K, a CTMC, {σ t } t R+, is defined. Its path distribution in the interval [0, T ] is given by Q θ 0,T ( {σt } t [0,T ] ). Similarly for the CTMC, { σ m } m Z+, induced from the parameter vector θ + ɛ.

21 Relative entropy decomposition Discrete-time MC Continuous-time MC Q: How to compute the Radon-Nikodym derivative, 0 dq θ 0,T dq θ+ɛ 0,T A: Apply Girsanov theorem which asserts that dq0,t θ { ({σ t}) = µθ (σ T 0) dq θ+ɛ µ θ+ɛ (σ exp log cθ (σ s, σ s) 0,T 0) 0 c θ+ɛ (σ s, σ s) dns T } [λ θ (σ s) λ θ+ɛ (σ s)] ds Substituting the above Girsanov formula to the path-wise relative entropy and exploiting the fact that we are at the steady state regime, we obtain that ( R Q θ 0,T Q θ+ɛ 0,T ) ( = T H Q θ 0,T Q θ+ɛ 0,T? ) + R ( µ θ µ θ+ɛ) Relative Entropy Rate for CTMC is given by ( ) [ H Q0,T θ Q θ+ɛ 0,T = E µ θ c θ (σ, σ ) log cθ (σ, σ ) ] c θ+ɛ (σ, σ ) (λθ (σ) λ θ+ɛ (σ)) σ E

22 Discrete-time MC Continuous-time MC Fisher information matrix Generalize the notion of Fisher information theory to the case of CTMC processes. FIM is defined as [ ] F H (Q0,T θ ) := E µ θ c θ (σ, σ ) θ log c θ (σ, σ ) θ log c θ (σ, σ ) T σ E

23 Discrete-time MC Continuous-time MC Fisher information matrix Generalize the notion of Fisher information theory to the case of CTMC processes. FIM is defined as [ ] F H (Q0,T θ ) := E µ θ c θ (σ, σ ) θ log c θ (σ, σ ) θ log c θ (σ, σ ) T σ E Similar derivations can be obtained for time-inhomogeneous Markov chains, semi-markov processes and probably for processes with memory. Sensitivity analysis on the logarithmic scale is also possible.

24 Connection with previous methods Discrete-time MC Continuous-time MC Pinsker (or Csiszar-Kullback-Pinsker) inequality: ( ) Q0,T θ Q θ+ɛ 0,T TV 2R Q0,T θ Qθ+ɛ 0,T If Q0,T θ = qθ 0,T dx and respectively Qθ+ɛ 0,T = qθ+ɛ 0,T dx, E Q θ 0,T [f ] E Q θ+ɛ[f ] f q0,t θ q θ+ɛ 0,T 0,T 1 ( f 2R q θ 0,T qθ+ɛ 0,T Thus, if RER is insensitive to a parameter then any observable is insensitive to this particular parameter. Overall, in terms of specific observables, RER and FIM are conservative estimates of their parameter sensitivity. )

25 Outline Introduction p53 reaction network Langevin process ZGB lattice model Introduction Discrete-time MC Continuous-time MC p53 reaction network Langevin process ZGB lattice model

26 Well-mixed reaction systems - p53 p53 reaction network Langevin process ZGB lattice model The p53 gene model is an extensively studied system which plays a crucial role for effective tumor suppression in human beings as its universal inactivation in cancer cells suggests. The p53 gene is activated in response to DNA damage and it constitutes a negative feedback loop with the oncogene protein Mdm2. Event Reaction Rate 1 x c 1 (σ) = b x 2 x c 2 (σ) = a x x + a k y x+k x 3 x x + y 0 c 3 (σ) = b y x 4 y 0 y c 4 (σ) = a 0 y 0 5 y c 5 (σ) = a y y x corresponding to p53, y 0 to Mdm2-precursor while y corresponds to Mdm2. Parameter vector: θ = [b x, a x, a k, k, b y, a 0, a y ] T.

27 p53 - Concentrations Introduction p53 reaction network Langevin process ZGB lattice model p53 mdm2 precursor mdm2 120 Number of Molecules Time Figure: Molecule concentration of p53, Mdm2-precursor and Mdm2. Concentration oscillations as well as time delays (phase shifts) between the species are present due to the negative feedback loop. Furthermore, the concentration of p53 periodically hits the zero and since negative concentrations are not allowed, the stochastic characteristics of p53 are far from Gaussian.

28 p53 - RER Introduction p53 reaction network Langevin process ZGB lattice model Relative Entropy Rate Time ε e 0 1 ε 0 e 4 ε e 0 7 Relative Entropy Rate ε 0 FIM based ε 0 0 e1 e2 e3 e4 e5 e6 e7 Figure: Upper panel: The Relative Entropy Rate in time for the parameter perturbation of b x (blue), k (green) and a y (red) by +10%. The relaxation time of the RER as an observable is ultra fast. Lower panel: RER for various perturbation directions computed either directly (blue and red bars) or based on FIM (green bars).

29 p53 - FIM Introduction p53 reaction network Langevin process ZGB lattice model Path wise FIM LNA based FIM b_x a_x a_k k b_y a_0 a_y b_x a_x a_k k b_y a_0 a_y b_x a_x a_k k b_y a_0 a_y b_x a_x a_k k b_y a_0 a_y Figure: The proposed path-wise Fisher Information Matrix (left) based on RER as well as the FIM based on LNA (Komorowski et al., Sensitivity, robustness, and identifiability in stochastic chemical kinetics models, PNAS, 2011).

30 p53 - Comparison Introduction p53 reaction network Langevin process ZGB lattice model unperturbed b perturb x b y perturb p53 50 mdm2 precursor Time unperturbed b x perturb b y perturb Time unperturbed b x perturb b y perturb mdm Time Figure: Time-series of the species for the unperturbed parameter regime (blue), when b x is perturbed by +10% (red) as well as when b y is perturbed by +10%. The same sequence of random numbers is used in all cases.

31 p53 reaction network Langevin process ZGB lattice model Langevin out-of-equilibrium process - Definition SDE system with N particles: dq t = 1 m p tdt dp t = F(q t )dt γ m p tdt + σdb t Force: F(q) = q V (q) + αg(q) Interaction potential: V (q) = i,j<i V M( q i q j ) Morse potential: VM (r) = D e(1 e a(r re ) ) 2 The divergence-free component ( q G = 0) is taken to be a simple antisymmetric force: G i (q) = q i+1 q i 1, i = 1,..., N If α = 0 then Langevin process is reversible. If α 0 then Langevin process is out of equilibrium.

32 p53 reaction network Langevin process ZGB lattice model Langevin out-of-equilibrium process - Scheme Sensitivity analysis on Morse parameters: θ = [D e, a, r e ]. Langevin process is time-discretized utilizing BBK integrator: p i+ 1 = p i F(q 2 i ) t 2 γ m p t i 2 + σ W i q i+1 = q i + 1 m p i+ 1 2 t p i+1 = p i+ 1 F(q i+1) t 2 2 γ m p t i σ W i+ 1 2 Transition probability of the discretized process: where P(q q, p) = 1 Z 0 e m2 P(q, p, q, p ) = P(q q, p)p(p q, q, p) σ 2 t 3 q q+(p F(q) t 2 +p tγ 2m ) t m 2 P(p q, q, p) = 1 Z 1 e 1 γ t σ 2 (1+ t 2m )p ( t m (q q) F(q ) t 2 ) 2

33 a a Introduction p53 reaction network Langevin process ZGB lattice model Langevin out-of-equilibrium process - FIM r e 0 r e D e D e a r e 0 r e D e D e a Figure: Upper plots: Level sets (or neutral spaces) for the reversible case (α = 0). Lower plots: Level sets for the irreversible case (α = 0.1).

34 ZGB - Definition Introduction p53 reaction network Langevin process ZGB lattice model ZGB (Ziff-Gulari-Barshad) is a simplified spatio-temporal CO oxidization model without diffusion. Despite being an idealized model, the ZGB model incorporates the basic mechanisms for the dynamics of adsorbate species during CO oxidation on catalytic surfaces. Event Reaction Rate 1 CO (1 σ(j) 2 )k 1 2 O 2 (1 σ(j) 2 )(1 k 1 ) 3 CO + O CO 2 + des. 1 2 σ(j)(1 + σ(j))k 2 4 O + CO CO 2 + des. 1 2 σ(j)(σ(j) 1)k 2 #vacant n.n. total n.n. #O n.n. total n.n. #CO n.n. total n.n. Table: The rate of the kth event of the jth site given that the current configuration is σ is denoted by c k (j; σ) where n.n. stands for nearest neighbors.

35 ZGB - RER Introduction p53 reaction network Langevin process ZGB lattice model 0.2 Relative Entropy Rate Time e 1 e Relative Entropy Rate ε 0 FIM based, ε 0 0 e1 Various directions e2 Figure: Upper plot: Relative entropy rate as a function of time for perturbations of both k 1 (solid line) and of k 2 (dashed line). An equilibration time until the process reach its metastable regime is evident. Lower plot: RER for various directions. The most sensitive parameter is k 1.

36 ZGB - Configurations p53 reaction network Langevin process ZGB lattice model Original Most sens. direction Least sens. direction Figure: Typical configurations obtained by ɛ 0-perturbations of the most and least sensitive parameters. The comparison with the reference configuration reveals the differences between the most and least sensitive perturbation parameters.

37 Conclusions Introduction p53 reaction network Langevin process ZGB lattice model Sensitivity analysis of steady state stochastic processes utilizing RER and FIM. Even though path-wise distributions was considered, the resulted observables where easy to compute. Long-time stochastic dynamics are also taken into consideration. No need to know the stationary distribution. A Relative Entropy Rate Method for Path Space Sensitivity Analysis of Stationary Complex Stochastic Dynamics, M.K. - Y.P., J. of Chemical Physics (2013).