A stochastic model-based approach to online event prediction and response scheduling M. Biagi, L. Carnevali, M. Paolieri, F. Patara, E. Vicario Department of Information Engineering, University of Florence, Italy marco.biagi@unifi.it Chios, EPEW October 5, 26 This is about: prediction and scheduling in partially observable systems with a stochastic model-based approach / 23
Aim Partially observable system: any system where part of the state remains hidden to observations Diagnosis: estimate current state based on observations Prediction: evaluate future state probabilities given a distribution of initial current state Scheduling: select an action type and a time point in the future 2 / 23
An application context Ambient Assisted Living (AAL): aims at providing assistance to people living within smart environments through the integration and exploitation of new sensing technologies, data processing techniques, and services 3 / 23
An application context T.vanKasteren, A.Noulas, G.Englebienne, B.Krse, Accurate Activity Recognition in a Home Setting, UbiComp 8 4 / 23
Diagnosis given a stream of Observations (EventType and Timestamp), evaluate the current ongoing Activity of the subject and a measure of likelihood of such Activity diagnosis 2 provides a list of triplets π, x, R(t) where π [, ] is a likelihood measure x S is a state of the system R(t) is the PDF of the remaining sojourn time in x (estimated numerically on a grid of time points). 2 L.Carnevali, C.Nugent, F.Patara, E.Vicario, A Continuous-Time Model-Based Approach to Activity Recognition for Ambient Assisted Living QEST 5 5 / 23
Prediction an activity type is chosen as target the aim is to predict when the next instance of the target activity type will begin at each new diagnosis result, estimate the first-passage transient probabilities to the target activity 6 / 23
Prediction technique: a model for prediction model-based approach a predefined parametric structure: stochastic parameters derived automatically from the statistics in process mining terminology: enhancement without elicitation p xy idle xy act x idle yx p yx act y p xz idle zx idle p zy p yz zx p zy idle xz act z idle yz 7 / 23
Prediction technique: a model for prediction model structure is chosen so as to fit prediction needs take into account GEN state sojourn time distribution idle states are specialized, this allows us to: preserve memory of the foregoing activity (trade-off memory/complexity) duration depends on previous and following activities. Allows to have idles durations with a lower CV that is a good feature for prediction never observed states (e.g from dinner to dinner) are not included allowing to reduce the space of possible behaviors p xy idle xy act x idle yx p yx act y p xz idle zx idle p zy p yz zx p zy idle xz act z idle yz 8 / 23
Prediction technique: derivation of stochastic parameters sojourn time distributions are evaluated via a basic approximants fitting of expected value and variation coefficient 3 for CV >, a mixture of 2 EXP for > CV > 2 a sequence of 2 EXP for 2 CV, a shifted EXP transition probabilities p xy that activity act y will follow act x is estimated as p xy = { i < n α i = act x and α i+ = act y } { i < n α i = act x } 3 W. Whitt, Approximating a Point Process by a Renewal Process, I: Two Basic Methods, Operations Research, 982 9 / 23
Prediction technique: distribution to critical event π A =.7 A D E π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) for each possible path to the critical state, compose the remaining time in the current activity with the Semi Markov sojourn in subsequent activities / 23
Prediction technique: distribution to critical event π A =.7 A D E π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) 2 sum up first passage time distributions weighted by the diagnosed state likelihood PDF (t) A to F π A PDF (t) B to F π B PDF (t) to F..8.6.4.4.3 + =.2.8.6.4.2. t 2 3 4 5 2 3 4 5.2 t 2 3 4 5 t / 23
Scheduling Select and schedule a response action to be taken before or at the beginning of some target activity (e.g.: remind pills intake 5 minutes before a meal) 2 / 23
Scheduling: problem formulation 4 Response actuation time: response action must be executed t w time units before the beginning of the target activity. Response duration: response action is active for a fixed duration : target activities are successfully handled by the system if they start t w time units after the response action was active. dinner t ev t t w time E.g. pills intake reminder: t w = 3s = 2s Remind at time t 4 F. Salfner et al., A Survey of Online Failure Prediction Methods, ACM Comput. Surv., 2 3 / 23
Scheduling: reward and decision policy performance evaluation: based on the number of true positives TP, false positives FP and false negatives FN maximum probability interval policy: activation time is evaluated searching for the maximum probability interval of width if maximum probability is higher than a fixed threshold, schedule at t = x t w, where x is the start of the interval otherwise, no response action is scheduled dinner t ev t PDF (t) x t w t time 4 / 23
Scheduling: an example a = α, τ, δ e τ t τ + δ t () PDF (t) t w t a 2 = α 2, τ 2, δ 2 τ 2 τ 2 + δ 2 time 5 / 23
Scheduling: an example e e 2 τ t t2 τ + δ PDF (t) a = α, τ, δ PDF 2 (t) t w e 3 t3 t (2) t a 2 = α 2, τ 2, δ 2 e 4 e 5 τ 2 t4 t5 τ 2 + δ 2 time t () Schedule t () is updated at e 2 t w PDF 3 (t) I t (3) t w I 2 t Schedule t (2) results in a FP Schedule t (3) results in a TP t 6 / 23
Experiments: methodology two types of datasets considered: synthetic datasets real AAL dataset 5 compare also different fitting strategies for sojourn distributions: the mean value with an EXP distribution the mean value with a two phases Erlang distribution the mean value and CV by the Whitt approach 6 exact state diagnosis assumed as input, allowing to focus on predictor and scheduler performance results as precision = TP/(TP + FP) and recall = TP/(TP + FN) 5 T.vanKasteren, A.Noulas, G.Englebienne, B.Krse, Accurate Activity Recognition in a Home Setting, UbiComp 8 6 W. Whitt, Approximating a Point Process by a Renewal Process, I: Two Basic Methods, Operations Research, 982 7 / 23
Experiments: generating a synthetic datasets generated by simulation in order to analyze how real phenomena sojourn distributions affects system predictability predefined model structure x, y p x,y =.5 events with Exp() inter-times p, p,2 p 2,3 act idle, act idle,2 act 2 idle 2,3 act 3 p, p, p 2, p 3, idle, idle, idle 2, idle 3, 8-phase Erlang: Erlang(k = 8, λ = 4) µ = 2, CV = 8 2-phase Erlang: Erlang(k = 2, λ = ) µ = 2, CV = 2 2-phase Hyper-exp: hyper-exp µ = 2, CV.92 increase CV 8 / 23
Experiments: synthetic datasets results.9.8.7.6.5.4.3.2. Precision 5 5 2 t w = t w = 6 t w = 2 t w = 3.9.8.7.6.5.4.3.2. Recall 5 5 2 t w = t w = 6 t w = 2 t w = 3 precision and recall with Whitt fitting on 8-phase Erlang 9 / 23
Experiments: synthetic datasets results.9.8.7.6.5.4.3.2. Whitt Erlang Exp t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2 comparing precisions of fitting strategy on 8-phase Erlang synthetic dataset 2 / 23
Experiments: synthetic datasets results.9.8.7.6.5.4.3.2. 8-ph Erlang CV = 8 2-ph Erlang CV = 2 Hyper-exp CV.92 t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2 comparing precisions with Whitt fitting on different synthetic sojourn distributions 2 / 23
Experiments: real AAL dataset results increase F F = 2 (precision recall) (precision+recall) Preparing dinner Preparing breakfast Preparing breakfast (filtered) (s) 5 3 6 2 8 /.63/.533.265/.684.52/.684 /.86 /.86 6 /.5/.25.2/.385.2/.385.68/.78.68/.78 t w (s) 2 /.2/.385.2/.385.2/.385.68/.78.68/.78 3 / / / /.48/.6.48/.6 t w (s) t w (s) /.93/.385.74/.385.74/.385.929/.448.929/.448 6 /.24/.36.24/.36.24/.36.462/.25.462/.25 2 / / / /.24/.36.24/.36 3 / / / /.2/.36.2/.36 /.465/.74.489/.759.557/.88.66/.87.672/.93 6 /.222/.556.39/.667.33/.735.48/.828.476/.896 2 /.63/.26.28/.467.225/.69.288/.778.338/.839 3 /.53/.28.88/.429.78/.643.277/.8.329/.836 22 / 23
Conclusion and future development we presented a model-based approach to online event prediction and response scheduling fitting both sojourn times mean value and CV improves precision of the prediction performance of the technique depends on the predictability of the system this technique can be applied to AAL for small values of response actuation time future evolution often depends on the history of previous activities, breaking the hypothesis of semi-markov models performance achieved by different model structures will be investigated in the future applicability beyond the limits of AAL perhaps better suited to more structured and predictable systems e.g. monitoring of industrial system e.g. gas/water distribution network 23 / 23
24 / 23
An application context Observed: Open fridge, open bathroom door Hidden states: Dinner, toileting,... Floorplan of the house, red rectangle boxes indicate sensor nodes 7 7 T.vanKasteren, A.Noulas, G.Englebienne, B.Krse, Accurate Activity Recognition in a Home Setting, UbiComp 8 25 / 23
A reference dataset for AAL a sequence of Observations with manually annotated Activities 8 7+ Activity Types from the index of Activities of Daily Living 28 Event Types (On Off of 4 state-change Sensors) 28 days, 245 annotated Activities and 29 unclassified intervals (Idle), 2638 Observations,... a Idling k- a Idling k a k+ δ δ k-,k k e n-u e n-u+ e n- e n e n+ e n+h............... end t k- t k start t k end start t k+ single subject, no concurrent Activities, reliable sensors, some inaccuracies start and completion of Activities do not coincide with Observations (and they are thus not observable) 8 T.vanKasteren, A.Noulas, G.Englebienne, B.Krse, Accurate Activity Recognition in a Home Setting, UbiComp 8 26 / 23
Prediction technique: a model for prediction Activity µ CV bed 26832.3.43 toilet 88.82.87 breakfast 5.5.72 shower 485.7.3 leave 39764.94.6 drink 35.25.35 dinner 862.3.63 idle 88.77 2. Activity µ CV idle bed 56.2.52 idle toilet 377.69.47 idle breakfast 244.5.58 idle shower 22.87 2.8 idle leave 687.33.54 idle drink 363.75.86 idle dinner 735.3.76 Activity µ CV idle bed toilet 434.75.9 idle bed breakfast 863.5.87 idle toilet bed 58.32 2.4 idle toilet toilet 3568.56.8 idle toilet breakfast 8.93.5 idle toilet shower 463.4.9 idle toilet leave 595.45.53 idle toilerink 364.5.97 idle toileinner 439.33.97 idle breakfast toilet 47.75.66 idle breakfast shower 53.73.38 idle breakfasrink 425.. idle shower toilet 2.. idle shower leave 722.38.73 idle shower dinner 2525.. idle leave bed 22.. idle leave toilet 82.96.5 idle leave shower 75..92 idle leave leave 359.. idle leave drink 28.67.2 idle leave dinner 76.. idle drink toilet 37.69.86 idle drink breakfast 8.. idle drink drink 4282..68 idle dinner toilet 6256..76 idle dinner leave 3474.. idle dinner drink 95.29.68 27 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 28 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 29 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 2 evaluate next states reached after current state (C and D from A) 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 3 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 2 evaluate next states reached after current state (C and D from A) 3 P DF (t) and P CF (t) are the distributions from next states to the critical state and was evaluated offline with transient analysis 9 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 3 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 2 evaluate next states reached after current state (C and D from A) 3 P DF (t) and P CF (t) are the distributions from next states to the critical state and was evaluated offline with transient analysis 9 4 evaluate convolution between R A (t) and P CF (t) weighted by p AC 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 32 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 2 evaluate next states reached after current state (C and D from A) 3 P DF (t) and P CF (t) are the distributions from next states to the critical state and was evaluated offline with transient analysis 9 4 evaluate convolution between R A (t) and P CF (t) weighted by p AC 5 evaluate convolution between R A (t) and P DF (t) weighted by p AD 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 33 / 23
Prediction technique: distribution to critical event π A =.7 A p AD D E p AC π B =.3 B C F for each current plausible state (A and B) evaluate the first-passage distribution to the critical state (gray state) mark critical state as absorbing 2 evaluate next states reached after current state (C and D from A) 3 P DF (t) and P CF (t) are the distributions from next states to the critical state and was evaluated offline with transient analysis 9 4 evaluate convolution between R A (t) and P CF (t) weighted by p AC 5 evaluate convolution between R A (t) and P DF (t) weighted by p AD 6 their sum is the first passage time distribution from A to F 9 A. Horváth, M. Paolieri, L. Ridi, E.Vicario, Transient analysis of non-markovian models using stochastic state classes Perform. Evaluation, 22. 34 / 23
Experiments: synthetic datasets results.9.8.7.6.5.4.3.2. Precision t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. Recall t w = t w = 6 t w = 2 t w = 3 5 5 2 Whitt.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2 EXP comparing fitting strategy on 8-phase Erlang synthetic dataset.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2 Erlang 35 / 23
Experiments: synthetic datasets results.9.8.7.6.5.4.3.2. Precision t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. Recall t w = t w = 6 t w = 2 t w = 3 5 5 2 8-ph Erlang.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2.9.8.7.6.5.4.3.2. t w = t w = 6 t w = 2 t w = 3 5 5 2 2-ph Erlang comparing results with Whitt fitting on different underlying distributions.9.8.7.6.5.4.3.2. 5 5 2 t t w = t w = 6 t w = 2 t w = 3.9.8.7.6.5.4.3.2. 5 5 2 t t w = t w = 6 t w = 2 t w = 3 Hyper-exp 36 / 23