Controlling Systemic Inflammation Using NMPC. Using Nonlinear Model Predictive Control with State Estimation

Transcription

1 Controlling Systemic Inflammation Using Nonlinear Model Predictive Control with State Estimation Gregory Zitelli, Judy Day July 2013 With generous support from the NSF, Award

2 Motivation We re going to be interested in the acute inflammatory response to biological stress in the form of a bacterial pathogen. The inflammatory response aims to remove the presence of the pathogen. However, an excessive response may lead to collateral tissue damage, organ failure, or worse.

3 Motivation To recover from a high-inflammation state, it is necessary for the inflammatory mechanisms to downregulate itself through the use of some anti-inflammatory mediator. We will consider a mathematical formulation of these ideas in terms of a highly coupled, nonlinear ODE model, as well as control applied to both the pro and anti-inflammatory influences within the system.

4 Interaction Diagram

5 Nonlinear Model P represents the bacterial pathogen population. N represents the concentration of pro-inflammatory mediators, such as activated phagocytes and their produced cytokines. D acts as a marker of tissue damage. C A represents the concentration of anti-inflammatory mediators.

6 Nonlinear Model ( dp dt = k pgp 1 P ) k pms m P P µ m + k mp P k pnf(n )P dn dt = s nrr(p, N, D) µ nr + R(P, N, D) µ nn dd dt = k f(n ) 6 dn x 6 dn + f(n ) 6 µ dd dc A dt = s c + k cn f(n + k cnd D) 1 + f(n + k cnd D) µ cc A

7 Nonlinear Model The system (P, N, D, C A ) has three fixed points corresponding to the three biologically relevant scenarios: P = N = D = 0 and C A = sc µ c, where the patient is healthy. All states elevated, where the patient is septic. P = 0, and N, D, C A > 0, where the patient is aseptic. For values of k pg (the pathogen growth rate) in the interval (0.5137, 1.755), all three states are stable. (Reynolds et al 2006)

8 ??? So what does this model actually look like?

9 Septic Simulation P (state ( )) 6 4 N* (state ( )) Time Time D (state ( )) Time C a (state ( )) Time

10 Aseptic Simulation P (state ( ),est( )) Time N* (state ( ),est( )) Time D (state ( ),est( )) C a (state ( ),est( )) Time Time

11 Healthy Simulation P (state ( )) N* (state ( )) Time Time D (state ( )) C a (state ( )) Time Time

12 Goals We would like to apply some kind of control to this model, to direct septic and aseptic patients towards the healthy fixed state. This is accomplished using Nonlinear Model Predictive Control, or NMPC. The NMPC is applied to the pro and anti-inflammatory mediators, N and C A. We only apply positive control, and there are restrictions on how much can be introduced within certain time frames. We assume that the level of pro and anti-inflammatory mediators, N and C A, are measurable with Gaussian noise.

13 Overview of NMPC

14 Overview of NMPC Our reference trajectory is zero levels of P and D, and minimal amounts of control. J = min AI(t),P I(t) Γ DD Γ P P Γ AI AI Γ P I P I 2 2

15 Note on Observations We take noisy measurements of N and C A at each time interval which update our predictive model. The levels of P and D are not measured directly, and are instead estimated from the predictive model. Being unable to measure P often leads the NMPC to be too aggressive. Many virtual patients were unnecessarily harmed under this scheme. Since there are biologically relevant scenarios for loose measurements of P (indicators like body temperature or blood pressure give an idea whether or not the infection persists), a pathogen update is done every four timesteps.

16 Pathogen Update If the predictive model reads low (< 0.05) but the pathogen levels are high (P > 0.5) then the level in the predictive model is reset to 0.5. If the predictive model reads high (> 0.5) but the pathogen levels are low (P < 0.05) then the level in the predictive model is reset to 0.

17 Virtual Patient Pool 1,000 patients are randomly generated with different parameters, including unique values of the pathogen growth rate k pg. 620 acquire elevated P values to suggest treatment. Of the 620, 251 (40%) will return to a healthy state on their own.

18 Initial Results (Patient Snapshot) P (state ( ),est( )) N* (state ( ),est( )) Time Time D (state ( ),est( )) C a (state ( ),est( )) Time Time

19 Initial Results (Day et al 2010) Therapy Type: Placebo Mismatch k pg = 0.52 Mismatch k pg = 0.6 Mismatch k pg = 0.8 Percentage Healthy: 40% (251) 60% (369) 82% (510) 83% (513) Percentage Aseptic: 37% (228) 19% (120) 8% (49) 17% (107) Percentage Septic: 23% (141) 21% (131) 10% (61) 0% (0) Percentage Harmed: na 0% (0/251) 1% (2/251) 6% (16/251) Percentage Rescued: na 32% (118/369) 71% (261/369) 75% (278/369)

20 Particle Filter One way to go about improving these results is to impliment robust state estimation for the unobserved variables P and D. This is accomplished using a particle filter, which tracks its accuracy by comparing its predictions of N and C A to the observed values.

21 Particle Filter The particle filter is initialized with 1,000 particles randomized near our measurements for N and C A and initial predictions for P and D. Each particle p i = (P i, N i, C A,i, D i ) is simulated for one time step. At the next time step, the particles p i are assigned weights q i depending on how close N i, C A,i are to the new measurements for N and C A.

22 Particle Filter At the end of the time step, each of the 1,000 slots holding a particle is randomly assigned a new particle p i according to their weights q i. This causes bad particles with low weights q i to die off, while good particle with high weights replicate.

23 Particle Filter Results Therapy Type: Original k pg = 0.52 Original k pg = 0.6 Original k pg = 0.8 Percentage Healthy: 60% (369) 80% (497) 84% (519) Percentage Aseptic: 19% (121) 10% (60) 16% (101) Percentage Septic: 21% (130) 10% (63) 0% (0) Therapy Type: Particles k pg = 0.52 Particles k pg = 0.6 Particles k pg = 0.8 Percentage Healthy: 60% (369) 80% (493) 82% (511) Percentage Aseptic: 19% (120) 10% (66) 18% (109) Percentage Septic: 21% (131) 10% (61) 0% (0)

24 Particle Filter Results The particle filter does slightly worse than our original mismatched predictive model. On the other hand, the particle filter is self correcting, so it needs no pathogen update to correct a misled pathogen prediction.

25 Controllability Consider the general nonlinear system with affine control ẏ = F (y) + w = h(y) m g i (x)u i i=1 where f : R n R n and h : R n R k. We typically take u : [0, ) R m locally integrable.

26 Controllability Rank Condition ẏ = F (y) + w = h(y) m g i (x)u i i=1 Definition (Controllability Rank Condition) Given the same nonlinear system with affine control, we say that the system satisfies the controllability rank condition if a finite sub-matrix of the following matrix has rank n [ g1... g m [F, g 1 ]... [F, [F, g 1 ]]... ]

27 Controllability If the system is linear, Controllability Rank Condition = Locally Controllable If the system is nonlinear, Controllability Rank Condition = Locally Accessible If the system is nonlinear and has symmetric control (for each control u there is a u such that F (y, u) = F (y, u ) for every y R n ), then Controllability Rank Condition = Locally Controllable

28 Controllability It can be shown that our system does satisfy the controllability rank condition in the first octant. This means that the system is locally accessible there. Unfortunately, our control does not satisfy the symmetry property, so this does not imply that the system is locally controllable.

29 What s Next? Djouadi and Bara are working on the use of adaptive control for this model, to help overcome the mismatch between the predictive model and patient model parameters like k pg. Although the pathogen update was necessary for the variable P, it appears that the damage variable D is well estimated by our predictive model under many different initial conditions and model parameters. It would be nice to make this notion precise. Are there elements of this model that can be exploited to increase the effectiveness of our filter?

30 The End? Thank you very much!

31 Model ( dp dt = k pgp 1 P ) k pms m P P µ m + k mp P k pnf(n )P dn dt = s nrr(p, N, D) µ nr + R(P, N, D) µ nn dd dt = k f(n ) 6 dn x 6 dn + f(n ) 6 µ dd dc A dt = s c + k cn f(n + k cnd D) 1 + f(n + k cnd D) µ cc A

32 Model Where R(P, N, D) = f(k np P + k nn N + k nd D) f(x) = x 1 + ( CA c ) 2 c is chosen so that when C A is at its highest, f(x) 1 4 x.

33 Pathogen The bacterial pathogen follows logistic growth. Healthy individuals have a baseline, non-specific, local immune response. We assume through the initial analysis of this interaction that the local response reaches a quasi-steady state value of k pm µ m+k mpp. The pathogen are directly attacked by the phagocytic immune cells N, which may be inhibited by the presence of the anti-inflammatory C A. ( dp dt = k pgp 1 P ) k pms m P P µ m + k mp P k pnf(n )P

34 Pro-Inflammatory Mediator The first term comes from the assumed quasi-steady state of a subsystem involving resting and activated phagocytes. The second term represents the gradual decay in the concentration of pro-inflammatory mediators. dn dt = s nrr(p, N, D) µ nr + R(P, N, D) µ nn

35 Damage Collateral tissue damage from activated phagocytes motivated the first term. The damage saturates, with the large Hill coefficient necessary to produce a more realistic basin of attraction for the healthy fixed state. The second term represents tissue regeneration. dd dt = k f(n ) 6 dn x 6 dn + f(n ) 6 µ dd

36 Anti-Inflammatory Mediators The initial biological source term s c is augmented by the Michaelis-Menten term. The second term represents the gradual decay in the concentration of anti-inflammatory mediators. dc A dt = s c + k cn f(n + k cnd D) 1 + f(n + k cnd D) µ cc A

37 Essential Components of NMPC (I) Reference Trajectory Desired trajectory we want the system to approach. Our reference trajectory will be P = D = 0. (II) Prediction of Process Output A predictive model is used to provide an estimation of the states which are unobserved. (III) Objective Function An objective function will describe how are current predictions deviate from the reference trajectory. Our objective function is a weighted sum of squares for the undesireable variables P and D, as well as for the control levels. J = min AI(t),P I(t) Γ DD Γ P P Γ AI AI Γ P I P I 2 2

38 Essential Components of NMPC (IV) Sequence of Control Moves at Each Time Step A discrete control is designed for k future steps to minimize the objective function over each time interval. (V) Error Prediction Update The measured values of N and C A from the patient model are compared to the predictive model after the control has been applied to both. It is at this step that implementations of the particle filter can be refined through this comparison.

39 Controllability Consider the general nonlinear system with affine control ẏ = F (y) + w = h(y) m g i (x)u i i=1 where f : R n R n and h : R n R k. We typically take u : [0, ) R m locally integrable. For us, k pg y 1 (1 y 1 F (y) = P ) kpmsmy 1 µ m+k mpy 1 k pn f(y 2 )y 1 s nrr(y 1,y 2,y 3 ) µ nr+r(y 1,y 2,y 3 ) µ ny 2 k dn f(y 2 ) 6 x 6 dn +f(y 2) 6 µ d y 3 s c + k cn f(y 2 +k cnd y 3 ) 1+f(y 2 +k cnd y 3 ) µ cy 4

40 Controllability Definition We say that a point a R n is attainable from ŷ if there exists an appropriate control u y (locally integrable) such that the trajectory given by m ẏ = F (y) + g i (x)u y i i=1 y(0) = ŷ is such that y(t ) = a for some finite trajectory [0, T ].

41 Controllability Definition If ŷ R n and O is an open set containing ŷ, we let A O (ŷ, T ) denote the set of all points in O which are attainable from ŷ in time T, such that y(t) O for all t [0, T ]. Definition We say that the nonlinear system is locally accessible at a point ŷ R n if the sets A O (ŷ, T ) has nonempty interior for every open set O and every T > 0.

42 Controllability Definition Given the same nonlinear system with affine control, we say that the system satisfies the controllability rank condition if a finite sub-matrix of the following matrix has rank n [ g1... g m [F, g 1 ]... [F, [F, g 1 ]]... ]

43 Controllability If our system was linear, then it turns out that the controllability rank condition implies that the system is controllable. In the nonlinear case, we have the following results. Theorem Given the same nonlinear system with affine control, if the system satisfies the controllability rank condition at a point ŷ R n then it is locally accessible there. Theorem If for a nonlinear system ẏ = F (y, u) we have that for any locally integrable control u there exists another locally integrable control u such that F (y, u) = F (y, u ) for all y R n and the system satisfies the controllability rank condition at a point ŷ R n, then the system is locally controllable at ŷ.

44 Controllability Recall our system is in the following form k pg y 1 (1 y 1 kpmsmy 1 µ m+k mpy 1 k pn f(y 2 )y 1 s nrr(y 1,y 2,y 3 ) µ ẏ = nr+r(y 1,y 2,y 3 ) µ ny 2 + u 1 f(y k 2 ) 6 dn x 6 dn +f(y µ 2) 6 d y 3 f(y s c + k 2 +k cnd y 3 ) cn 1+f(y 2 +k cnd y 3 ) µ cy 4 + u u u 2 P )