ProbReach: Probabilistic Bounded Reachability for Uncertain Hybrid Systems

Transcription

1 ProbReach: Probabilistic Bounded Reachability for Uncertain Hybrid Systems Fedor Shmarov, Paolo Zuliani School of Computing Science, Newcastle University, UK 1 / 41

2 Introduction ProbReach tool for probabilistic reachability analysis in uncertain hybrid systems. Uncertain hybrid systems: random and nondeterministic parameters. 2 / 41

3 Introduction ProbReach tool for probabilistic reachability analysis in uncertain hybrid systems. Uncertain hybrid systems: random and nondeterministic parameters. Reachability = deciding whether a goal state is reachable. Systems with random parameters: computing probability of reachability. Nondeterministic parameters introduce a probability reachability function. 2 / 41

4 Introduction ProbReach tool for probabilistic reachability analysis in uncertain hybrid systems. Uncertain hybrid systems: random and nondeterministic parameters. Reachability = deciding whether a goal state is reachable. Systems with random parameters: computing probability of reachability. Nondeterministic parameters introduce a probability reachability function. ProbReach implements two approaches: formal and statistical. Formal approach: stronger guarantees (absolute vs. statistical). Statistical approach: lower complexity with respect to the number of parameters (constant vs. exponential). 2 / 41

5 Introduction ProbReach tool for probabilistic reachability analysis in uncertain hybrid systems. Uncertain hybrid systems: random and nondeterministic parameters. Reachability = deciding whether a goal state is reachable. Systems with random parameters: computing probability of reachability. Nondeterministic parameters introduce a probability reachability function. ProbReach implements two approaches: formal and statistical. Formal approach: stronger guarantees (absolute vs. statistical). Statistical approach: lower complexity with respect to the number of parameters (constant vs. exponential). ProbReach can be applied to realistic models. Artificial pancreas model. 2 / 41

6 Hybrid Systems reset (q1,q 2 ) jump (q1,q 2 ) init q 1 flow q1 invt q1 q 2 flow q2 invt q2 reset (q2,q 1 ) jump (q2,q 1 ) 3 / 41

7 Hybrid Systems reset (q1,q 2 ) jump (q1,q 2 ) init q 1 flow q1 invt q1 q 2 flow q2 invt q2 reset (q2,q 1 ) jump (q2,q 1 ) init and reset computable functions, flow Lipschitz-continuous ODEs, invt and jump Boolean logic formula m {>, }, f i,j computable function. i=1 ( ki j=1 ( fi,j (x) 0 )), 3 / 41

8 Bounded Reachability Reachability is undecidable even for linear hybrid systems (Alur, Courcoubetis, Henzinger, Ho. 1993). 4 / 41

9 Bounded Reachability Reachability is undecidable even for linear hybrid systems (Alur, Courcoubetis, Henzinger, Ho. 1993). Bounded reachability: computable goal predicate, and finite reachability depth, and bounded time domain in each mode. Does the hybrid system reach a goal state within a finite number of (discrete) steps? 4 / 41

10 Bounded Reachability Reachability is undecidable even for linear hybrid systems (Alur, Courcoubetis, Henzinger, Ho. 1993). Bounded reachability: computable goal predicate, and finite reachability depth, and bounded time domain in each mode. Does the hybrid system reach a goal state within a finite number of (discrete) steps? Nonlinear arithmetics (with trigonometric functions) over the reals is undecidable (Tarski, 1951). 4 / 41

11 Bounded Reachability Reachability is undecidable even for linear hybrid systems (Alur, Courcoubetis, Henzinger, Ho. 1993). Bounded reachability: computable goal predicate, and finite reachability depth, and bounded time domain in each mode. Does the hybrid system reach a goal state within a finite number of (discrete) steps? Nonlinear arithmetics (with trigonometric functions) over the reals is undecidable (Tarski, 1951). Bounded reachability is δ-decidable. δ-complete decision procedure (Gao, Avigad, Clarke. LICS 2012). 4 / 41

12 Uncertain Hybrid Systems reset (q1,q 2 )(p) jump (q1,q 2 ) (p) init(p) q 1 flow q1 (p) invt q1 (p) q 2 flow q2 (p) invt q2 (p) reset (q2,q 1 )(p) jump (q2,q 1 ) (p) 5 / 41

13 Uncertain Hybrid Systems reset (q1,q 2 )(p) jump (q1,q 2 ) (p) init(p) q 1 flow q1 (p) invt q1 (p) q 2 flow q2 (p) invt q2 (p) Parametric Hybrid System (PHS): p P parameter, P parameter space, dp dt = 0. reset (q2,q 1 )(p) jump (q2,q 1 ) (p) 5 / 41

14 Uncertain Hybrid Systems reset (q1,q 2 )(p) jump (q1,q 2 ) (p) init(p) q 1 flow q1 (p) invt q1 (p) q 2 flow q2 (p) invt q2 (p) Parametric Hybrid System (PHS): p P parameter, P parameter space, dp dt = 0. reset (q2,q 1 )(p) jump (q2,q 1 ) (p) Stochastic PHS (SPHS): PHS with random parameters. 5 / 41

15 Uncertain Hybrid Systems reset (q1,q 2 )(p) jump (q1,q 2 ) (p) init(p) q 1 flow q1 (p) invt q1 (p) q 2 flow q2 (p) invt q2 (p) Parametric Hybrid System (PHS): p P parameter, P parameter space, dp dt = 0. reset (q2,q 1 )(p) jump (q2,q 1 ) (p) Stochastic PHS (SPHS): PHS with random parameters. init and reset computable functions, flow Lipschitz-continuous ODEs, invt and jump Boolean logic formula m {>, }, f i,j computable function. i=1 ( ki j=1 ( fi,j (x, p) 0 )), 5 / 41

16 SPHS: Running Example 10 S y 5 υ 0 α S x 6 / 41

17 SPHS: Running Example 10 S y 5 υ 0 α S x Parameters: Random: υ 0 N (25, 3) initial speed, α = { : 0.9, : 0.09, : 0.01} angle to horizon, Nondeterministic: K [0.5, 0.9] speed loss coefficient. 6 / 41

18 Bounded Reachability Probability What is the probability that the system reaches a goal state in a finite number of steps? 7 / 41

19 Bounded Reachability Probability What is the probability that the system reaches a goal state in a finite number of steps? Let P = P N P R P R is the domain of random parameters, P N is the domain of nondeterministic parameters 7 / 41

20 Bounded Reachability Probability What is the probability that the system reaches a goal state in a finite number of steps? Let P = P N P R P R is the domain of random parameters, P N is the domain of nondeterministic parameters The bounded reachability probability function is: Pr : P N [0, 1]. If P N = then Pr is constant. 7 / 41

21 Computing Bounded Reachability Probability Approach Formal Statistical Principle Formal Reasoning Monte Carlo Sampling Probability dp E[X ] 1 N N i=1 X i G G = {p P : goal(p)}, X i = 1 if goal(p), G C = P \ G. X i = 0 otherwise. Guarantees Absolute Statistical Complexity Exponential Constant (number of parameters) 8 / 41

22 Computing Bounded Reachability Probability Approach Formal Statistical Principle Formal Reasoning Monte Carlo Sampling Probability dp E[X ] 1 N N i=1 X i G G = {p P : goal(p)}, X i = 1 if goal(p), G C = P \ G. X i = 0 otherwise. Guarantees Absolute Statistical Complexity Exponential Constant (number of parameters) Both approaches need a procedure which given a non-empty B P identifies whether B G or B G C. 8 / 41

23 Evaluation Procedure (I) Given a non-empty B P we define two formulae Reach(H, l, B), and Reach (H, l, B). 9 / 41

24 Evaluation Procedure (I) Given a non-empty B P we define two formulae Reach(H, l, B), and Reach (H, l, B). Reach(H, l, B) true if H satisfies goal in l steps for some p B, Reach (H, l, B) := B p : Reach(H, l, {p}). Reach (H, l, B) Reach(H, l, {p}). 9 / 41

25 Evaluation Procedure (I) Given a non-empty B P we define two formulae Reach(H, l, B), and Reach (H, l, B). Reach(H, l, B) true if H satisfies goal in l steps for some p B, Reach (H, l, B) := B p : Reach(H, l, {p}). Reach (H, l, B) Reach(H, l, {p}). Reach and Reach are bounded L R -sentences Can be verified by the δ-complete decision procedure 9 / 41

26 Evaluation Procedure (I) Given a non-empty B P we define two formulae Reach(H, l, B), and Reach (H, l, B). Reach(H, l, B) true if H satisfies goal in l steps for some p B, Reach (H, l, B) := B p : Reach(H, l, {p}). Reach (H, l, B) Reach(H, l, {p}). Reach and Reach are bounded L R -sentences Can be verified by the δ-complete decision procedure Remember!!! Only unsat answer can be trusted and δ-sat is subject to over-approximation δ. 9 / 41

27 Evaluation Procedure (II) Based on the unsat (trusted) answer of δ-decision procedures Algorithm 1: evaluate(h, l, B, δ) ( ) 1 if δ-decision Reach(H, l, B) ( == δ-sat then ) 2 if δ-decision Reach (H, l, B) == δ-sat then 3 return undet; 4 return sat; 5 return unsat; sat goal is reached for all parameter values in B, unsat goal is reached for no parameter values in B, undet goal is reached for some parameter values in B, 10 / 41

28 Evaluation Procedure (II) Based on the unsat (trusted) answer of δ-decision procedures Algorithm 2: evaluate(h, l, B, δ) ( ) 1 if δ-decision Reach(H, l, B) ( == δ-sat then ) 2 if δ-decision Reach (H, l, B) == δ-sat then 3 return undet; 4 return sat; 5 return unsat; sat goal is reached for all parameter values in B, unsat goal is reached for no parameter values in B, undet goal is reached for some parameter values in B, OR undet one of the formulae is not robust for the given δ. 10 / 41

29 Computing Bounded Reachability Probability Approach Formal Statistical Principle Formal Reasoning Monte Carlo Sampling Probability dp E[X ] = 1 N N i=1 X i G G = {p P : goal(p)}, X i = 1 if goal(p), G C = P \ G. X i = 0 otherwise. Guarantees Absolute Statistical Complexity Exponential Constant (number of parameters) 11 / 41

30 Formal Approach: In a nutshell The bounded reachability probability is a function of nondeterministic parameters obtained as: Pr(p N ) = dp G(p N ) P - probability measure of random parameters, G(p N ) - system s goal set for the given p N. 12 / 41

31 Formal Approach: In a nutshell The bounded reachability probability is a function of nondeterministic parameters obtained as: Pr(p N ) = dp G(p N ) P - probability measure of random parameters, G(p N ) - system s goal set for the given p N. How to compute Pr(p N )??? 12 / 41

32 Formal Approach: In a nutshell The bounded reachability probability is a function of nondeterministic parameters obtained as: Pr(p N ) = dp G(p N ) P - probability measure of random parameters, G(p N ) - system s goal set for the given p N. How to compute Pr(p N )??? Identify G(p N ) already solved! Partition P R with boxes B, Evaluate each {p N } B using procedure evaluate. 12 / 41

33 Formal Approach: In a nutshell The bounded reachability probability is a function of nondeterministic parameters obtained as: Pr(p N ) = dp G(p N ) P - probability measure of random parameters, G(p N ) - system s goal set for the given p N. How to compute Pr(p N )??? Identify G(p N ) already solved! Partition P R with boxes B, Evaluate each {p N } B using procedure evaluate. Compute B dp for each box with desired precision ˆɛ > 0. Find an estimate which is at most ˆɛ far from B dp. 12 / 41

34 Formal Approach: Algorithm We reason about parameter boxes B N P N for which we compute enclosures [ P over [B N ], P under [B N ] ] such that: p N B N : Pr(p N ) [ P over [B N ], P under [B N ] ]. 13 / 41

35 Formal Approach: Algorithm We reason about parameter boxes B N P N for which we compute enclosures [ P over [B N ], P under [B N ] ] such that: p N B N : Pr(p N ) [ P over [B N ], P under [B N ] ]. 1 B N = P N ; B R = P R ; 2 P over [B N ] = 1; P under [B N ] = 0; 3 while for each B N : (P over [B N ] P under [B N ] > ɛ) or (B N > ρ) do 4 switch evaluate(h, l, B R B N, B R ) do 5 case unsat do P over [B N ] = P over [B N ] B R dp ; 6 case sat do P under [B N ] = P under [B N ] + B R dp ; 7 case undet do bisect B R, B N ; Fedor Shmarov and Paolo Zuliani, PlanHS / 41

36 Formal Approach: Running Example 10 S y 5 υ 0 α S x Parameters: Random: υ 0 N (25, 3) initial speed, α = { : 0.9, : 0.09, : 0.01} angle to horizon, Nondeterministic: K [0.5, 0.9] speed loss coefficient. 14 / 41

37 Formal Approach: Running Example 10 S y 5 υ 0 α S x Parameters: Random: υ 0 N (25, 3) initial speed, α = { : 0.9, : 0.09, : 0.01} angle to horizon, Nondeterministic: K [0.5, 0.9] speed loss coefficient. Compute the probability (Pr : [0.5, 0.9] [0, 1]) of landing further than 100 metres (S x 100) after bouncing once (l = 1). 14 / 41

38 Formal Approach: Running Example (II) The probability reachability function Pr(K) can be obtained as: Pr(K) = [ ] 3 i=1 f α (α i ) f υ0 (x)dx sin(2α i )(K 2 +1) Pr(K) K 15 / 41

39 Formal Approach: Running Example (III) Pr(K) Probability enclosure precision ɛ = Red boxes computed for ρ = Blue boxes computed for ρ = K 16 / 41

40 Formal Approach: ɛ-guarantee Size of probability enclosures depends on nondeterministic parameter precision ρ, solver precision δ. 17 / 41

41 Formal Approach: ɛ-guarantee Size of probability enclosures depends on nondeterministic parameter precision ρ, solver precision δ. Probability enclosures can be arbitrarily tight (up to the required ɛ > 0) if formulae Reach and Reach are robust for all p P, reachability probability function is continuous, at least one continuous random parameter. 17 / 41

42 Formal Approach: ɛ-guarantee Size of probability enclosures depends on nondeterministic parameter precision ρ, solver precision δ. Probability enclosures can be arbitrarily tight (up to the required ɛ > 0) if formulae Reach and Reach are robust for all p P, reachability probability function is continuous, at least one continuous random parameter. 1 B N = P N ; B R = P R ; 2 P over [B N ] = 1; P under [B N ] = 0; 3 while for each B N : (P over [B N ] P under [B N ] > ɛ) or (B N > ρ) do 4 switch evaluate(h, l, B R B N, B R ) do 5 case unsat do P over [B N ] = P over [B N ] B R dp ; 6 case sat do P under [B N ] = P under [B N ] + B R dp ; 7 case undet do bisect B R, B N ; 17 / 41

43 Formal Approach: Running Example (IV) Pr(K) Probability enclosure precision ɛ = Nondeterministic parameter precision ρ is ignored. K 18 / 41

44 Computing Probabilistic Bounded Reachability Approach Formal Statistical Principle Formal Reasoning Monte Carlo Sampling Probability dp E[X ] = 1 N N i=1 X i G G = {p P : goal(p)}, X i = 1 if goal(p), G C = P \ G. X i = 0 otherwise. Guarantees Absolute Statistical Complexity Exponential Constant (number of parameters) 19 / 41

45 Statistical Approach: In a nutshell For each p N P N and p R P R let: { 1 if goal is reached for (p N, p R ), X (p N, p R ) = 0 otherwise. Then Pr(p N ) = E[X (p N )] = G(p N ) dp. 20 / 41

46 Statistical Approach: In a nutshell For each p N P N and p R P R let: { 1 if goal is reached for (p N, p R ), X (p N, p R ) = 0 otherwise. Then Pr(p N ) = E[X (p N )] = G(p N ) dp. How to compute Pr(p N )??? 20 / 41

47 Statistical Approach: In a nutshell For each p N P N and p R P R let: { 1 if goal is reached for (p N, p R ), X (p N, p R ) = 0 otherwise. Then Pr(p N ) = E[X (p N )] = G(p N ) dp. How to compute Pr(p N )??? Sample p R using the parameters distribution. Evaluate X (p N, p R ). 20 / 41

48 Statistical Approach: In a nutshell For each p N P N and p R P R let: { 1 if goal is reached for (p N, p R ), X (p N, p R ) = 0 otherwise. Then Pr(p N ) = E[X (p N )] = G(p N ) dp. How to compute Pr(p N )??? Sample p R using the parameters distribution. Evaluate X (p N, p R ). We CANNOT evaluate X (p N, p R ) (undecidability!!!) 20 / 41

49 Statistical Approach: Confidence Intervals We define two random variables: { 1 if evaluate(h, l, {p N, p R }, δ) = sat, X sat (p N, p R ) = 0 otherwise. { 0 if evaluate(h, l, {p N, p R }, δ) = unsat, X usat (p N, p R ) = 1 otherwise. 21 / 41

50 Statistical Approach: Confidence Intervals We define two random variables: { 1 if evaluate(h, l, {p N, p R }, δ) = sat, X sat (p N, p R ) = 0 otherwise. { 0 if evaluate(h, l, {p N, p R }, δ) = unsat, X usat (p N, p R ) = 1 otherwise. X sat (p N, p R ) and X usat (p N, p R ) can be sampled, X sat (p N, p R ) X (p N, p R ) X usat (p N, p R ). E[X sat (p N )] E[X (p N )] = Pr(p N ) E[X usat (p N )] 21 / 41

51 Statistical Approach: Confidence Intervals (II) Given accuracy ξ > 0 and confidence c (0, 1) compute intervals [p sat ξ, p sat + ξ] and [p usat ξ, p usat + ξ]. ( ) Probability E[X sat (p N, p R )] [p sat ξ, p sat + ξ] c, ( ) Probability E[X usat (p N, p R )] [p usat ξ, p usat + ξ] c. 22 / 41

52 Statistical Approach: Confidence Intervals (II) Given accuracy ξ > 0 and confidence c (0, 1) compute intervals [p sat ξ, p sat + ξ] and [p usat ξ, p usat + ξ]. ( ) Probability E[X sat (p N, p R )] [p sat ξ, p sat + ξ] c, ( ) Probability E[X usat (p N, p R )] [p usat ξ, p usat + ξ] c. ( ) Probability Pr(p N ) [p sat ξ, p usat + ξ] c. The size of [p sat ξ, p usat + ξ] can be greater than 2ξ non-robustness for the given δ, or undecidability in general. Shmarov and Zuliani. HVC / 41

53 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. 23 / 41

54 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. 23 / 41

55 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). 23 / 41

56 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). Step 1 generate samples p N using f ( ; v). 23 / 41

57 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). Step 1 generate samples p N using f ( ; v). Step 2 update v using the fittest (probability-wise) samples. 23 / 41

58 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). Step 1 generate samples p N using f ( ; v). Step 2 update v using the fittest (probability-wise) samples. Step 3 go to Step 1 if f ( ; v) variance is greater than the desired ˆσ / 41

59 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). Step 1 generate samples p N using f ( ; v). Step 2 update v using the fittest (probability-wise) samples. Step 3 go to Step 1 if f ( ; v) variance is greater than the desired ˆσ 2. There must be v such that f ( ; v) approximates single-point distributions arbitrarily well. 23 / 41

60 Statistical Approach: Cross-Entropy Algorithm We compute maximum/minimum reachability probability. approximate value p N where the minimum/maximum probability is achieved, Probability ( Pr(p N ) [p sat ξ, p usat + ξ] ) c. CE aims at obtaining the optimal parameter distribution [Rubinstein and Kroese, 2008]. Step 0 start with nondeterministic parameters distributed with PDF f ( ; v). Step 1 generate samples p N using f ( ; v). Step 2 update v using the fittest (probability-wise) samples. Step 3 go to Step 1 if f ( ; v) variance is greater than the desired ˆσ 2. There must be v such that f ( ; v) approximates single-point distributions arbitrarily well. Cross-Entropy can fall into a local extremum. 23 / 41

61 Statistical Approach: Cross-Entropy Algorithm (II) CE terminates when the user-defined variance ˆσ 2 is reached. 24 / 41

62 Statistical Approach: Cross-Entropy Algorithm (II) CE terminates when the user-defined variance ˆσ 2 is reached. 24 / 41

63 Statistical Approach: Cross-Entropy Algorithm (II) CE terminates when the user-defined variance ˆσ 2 is reached. 24 / 41

64 Statistical Approach: Running Example Pr(K) can be obtained analytically Pr(K) K K K min max K Confidence Interval Pr(K) min [ , ] max [ , ] / 41

65 Statistical Approach: CE Result Quality Number of samples per iteration of CE algorithm, the more the better. 26 / 41

66 Statistical Approach: CE Result Quality Number of samples per iteration of CE algorithm, the more the better. Terminal variance value, the smaller the better. 26 / 41

67 Statistical Approach: CE Result Quality Number of samples per iteration of CE algorithm, the more the better. Terminal variance value, the smaller the better. Initial distribution parameters, need to provide sufficient initial coverage to avoid local extrema. 26 / 41

68 Statistical Approach: CE Result Quality Number of samples per iteration of CE algorithm, the more the better. Terminal variance value, the smaller the better. Initial distribution parameters, need to provide sufficient initial coverage to avoid local extrema. Accuracy for estimating the confidence intervals, the higher the better. 26 / 41

69 ProbReach Implemented in C++. Uses OpenMP for parallelisation. Uses several libraries CAPD, IBEX, GSL / 41

70 ProbReach Implemented in C++. Uses OpenMP for parallelisation. Uses several libraries CAPD, IBEX, GSL. Any SAT ODE solver supporting δ-decisions can be used. dreal 1 [Sicun Gao, Soonho Kong] isat3 2 [Martin Fränzle et al.] Available at Shmarov and Zuliani. HSCC / 41

71 Demonstration 28 / 41

72 Discussion We presented ProbReach tool for probabilistic bounded reachability in uncertain hybrid system. It features formal and statistical approaches. Formal approach: computes probability enclosures containing the range of the probability reachability function. Complexity grows exponentially with the number of system parameters. Statistical approach: computes confidence intervals containing the approximate maximum/minimum probability value. Complexity remains constant with respect to the number of system parameters. ProbReach is publicly available at 29 / 41

73 Automated Synthesis of Safe and Robust PID Controllers for Stochastic Hybrid Systems Fedor Shmarov 1, Nicola Paoletti 2, Ezio Bartocci 3, Shan Lin 2, Scott A. Smolka 2, Paolo Zuliani 1 1 Newcastle University, UK, 2 Stony Brook University, NY, USA, 3 TU Wien, Austria 30 / 41

74 Artificial Pancreas Closed-loop (with feedback) control of insulin treatment for Type 1 diabetes. Continuous glucose monitor Control algorithm Insulin pump basal constant dose (automatic) bolus single high dose (manual) MINIMED 670G by Medtronic / 41

75 Artificial Pancreas Closed-loop (with feedback) control of insulin treatment for Type 1 diabetes. Continuous glucose monitor Control algorithm Insulin pump basal constant dose (automatic) bolus single high dose (manual) Objective MINIMED 670G by Medtronic 3 Design automatic closed-loop control of bolus insulin for keeping blood glucose level between 4-12 mmol/l. Temporary hyperglycemia is allowed while hypoglycemia should be avoided / 41

76 Automatic Control Control Objective Given an external disturbance reduce the difference between the measured system output and the desired value by adjusting the control variable. System output Time disturbance: amount of carbohydrates (D G ) system output: blood glucose level (G(t)) desired level (set-point): G sp = 6.11 [mmol/l] control variable: insulin admission (u(t) + u b ) difference (error): e(t) = G sp G(t) 32 / 41

77 PID Controller P I PLANT PHS D P roportional - present value of the error, I ntegral - past errors, D erivative - predicted future errors. 33 / 41

78 PID Controller P I PLANT PHS D P roportional - present value of the error, I ntegral - past errors, D erivative - predicted future errors. Synthesis Objective Find values of K p, K i and K d (gains) minimising e(t). 33 / 41

79 Stochastic Parametric Hybrid Systems Meal ( Flow G(t), u(t) + u }{{ b, G } sp insulin }{{} glucose }{{} desired value, D G }{{} meal size PID(K p, K i, K d, G(t), u(t) + u b, G sp ) ) a. a Hovorka, R.: Closed-loop insulin delivery: from bench to clinical practice. Nature Reviews Endocrinology 7(7), (2011) 34 / 41

80 Stochastic Parametric Hybrid Systems (II) D G := D G1 (D G := D G2 ) (t := 0) (D G := D G3 ) (t := 0) Meal 1 Meal 2 Meal 3 t = T1 t = T 2 Size of each meal: D G1 N (40, 10), D G2 N (90, 10), D G3 N (60, 10). Parameters: Time between the meals: T 1 N (300, 10), T 2 N (300, 10). 35 / 41

81 Safety and Robustness Safety An unsafe state should be reached with very small probability. Unsafe: G(t) [4, 16]. 36 / 41

82 Safety and Robustness Safety An unsafe state should be reached with very small probability. Unsafe: G(t) [4, 16]. Robustness (not in the sense of δ-robustness) Difference between the system output and the desired value should be small. Fundamental Index: FI (t) = t 0 (e(τ))2 dτ System output should converge to the steady-state. Weighted Fundamental Index: FI w (t) = t 0 τ 2 (e(τ)) 2 dτ Non-robust: (FI (t) > ) (FI w (t) > ). 36 / 41

83 Automated Synthesis Safety and robustness analysis is performed through bounded reachability. Bounded Reachability Can the unsafe state be reached within: finite number of discrete steps, and bounded time interval. Bounds: 3 meals, 24 hours. 37 / 41

84 Automated Synthesis Safety and robustness analysis is performed through bounded reachability. Bounded Reachability Can the unsafe state be reached within: finite number of discrete steps, and bounded time interval. Bounds: 3 meals, 24 hours. Automated Synthesis Objective Synthesise a PID controller minimizing the probability of reaching an unsafe state or violating the robustness constraint during 3 meals within 24 hour period. 37 / 41

85 Results Insulin Administration u(t) }{{} + u }{{} b PID(K p, K i, K d, e(t)) basal rate Both safety and robustness were taken into account 38 / 41

86 Results Insulin Administration u(t) }{{} + u }{{} b PID(K p, K i, K d, e(t)) basal rate Basal rate synthesis (formal): with G(0) = G sp and no external disturbances G(t) reaches [G sp 0.05, G sp ] in 2000 minutes and remains there for another 1000 minutes. Domain Result Chosen Value u b [0,1] [ , ] Both safety and robustness were taken into account 38 / 41

87 Results Insulin Administration u(t) }{{} + u }{{} b PID(K p, K i, K d, e(t)) basal rate Basal rate synthesis (formal): with G(0) = G sp and no external disturbances G(t) reaches [G sp 0.05, G sp ] in 2000 minutes and remains there for another 1000 minutes. Domain Result Chosen Value u b [0,1] [ , ] PID controller synthesis (statistical): # K d K i K p CI C [ , ] C [ , 1] C [ , ] C [ , ] C [0.3324, ] Both safety and robustness were taken into account 38 / 41

88 One-day Scenario 50 grams, 100 grams, 70 grams in 5 hour intervals. 25 C 0 Glucose Level (mmol/l) C 1 C 2 C 3 sp Time (min) 0.3 C 0 Insulin Administration (U) C 1 C 2 C 3 # Safety FI 10 6 FI w 10 9 C0 1, C 0 2 Unsafe C 1 Safe C 2 Unsafe C 3 Safe Time (min) 39 / 41

89 Discussion Conclusions: We presented a technique for the automated synthesis of safe and robust PID controllers using ProbReach. The presented approach was applied to an artificial pancreas model. Future work: PID controllers with nonlinear gains. Discrete-time PID controllers. 40 / 41

90 Thank you Questions? 41 / 41