Context-based Reasoning in Ambient Intelligence - CoReAmI -

Transcription

1 Context-based in Ambient Intelligence - CoReAmI - Hristijan Gjoreski Department of Intelligent Systems, Jožef Stefan Institute Supervisor: Prof. Dr. Matjaž Gams Co-supervisor: Dr. Mitja Luštrek

2 Background What is Ambient Intelligence (AmI)? Refers environments with embedded technologies that are sensitive and responsive to the presence of people What is context? [A. Dey, 00] Context is any information that characterizes the circumstances in which an event occurs J. C. Augusto et al. (007) Context-based reasoning To reason about something using contextual information

3 Motivation () Context Numerous systems that provide services and reason about a user: health-care systems, elderly support systems, fitness applications, etc. The various sensors used in these systems provide rich contextual information: location, activity, etc. Illustrative example, which shows the importance of the context User sits and has high heart rate an alarming situation? Not always depends on the context, e.g., running as a previous activity Context-based reasoning approach CoReAmI

4 Motivation () Multi-view from multiple points of view (Multi-view reasoning) An intuitive example: sensing/reasoning about food Sight Smell Taste Complete picture of the food CoReAmI includes the idea of using multiple points of view Each view is created by using each source of information as a context individually In accordance with the: Principle of multiple knowledge Gams Ensemble learning (combining multiple ML models) Dietterich Gams M. Weak intelligence: through the principle and paradox of multiple knowledge. New York: Nova Science Publishers, 00. Dietterich TG. Ensemble methods in Machine Learning. In Proc. of the first International Workshop on Multiple Classifier Systems

5 Outline Background and Motivation Hypothesis CoReAmI Approach CoReAmI Case Studies Scientific Contributions Conclusions and Future Work 5

6 Hypothesis Extracting and combining multiple sources of information by using a context-based approach (that is, using each source of information as a context) can lead to better reasoning performance compared to conventional approaches in an AmI domain. To validate the hypothesis we developed the CoReAmI approach and tested it on three AmI problem domains: o Activity recognition o Energy expenditure estimation o Fall detection 6

7 Outline Introduction and Motivation Hypothesis CoReAmI Approach CoReAmI Case Studies Scientific Contributions Conclusions and Future Work 7

8 Outline Introduction and Motivation Hypothesis CoReAmI Approach Context Extraction Context Modeling Context Aggregation CoReAmI Case Studies Scientific Contributions Conclusions and Future Work 8

9 Outline Introduction and Motivation Hypothesis CoReAmI Approach Context Extraction Context Modeling Context Aggregation CoReAmI Case Studies Scientific Contributions Conclusions and Future Work 9

10 CoReAmI Approach s s s s m Sensor Data c c n A: Context Extraction v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation 0

11 CoReAmI Approach s s s s m Sensor Data c c n A: Context Extraction v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation

12 CoReAmI Approach s s s s m Sensor Data Accelerometer Heart rate sensor Location sensor c c n A: Context Extraction v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation

13 CoReAmI Approach A: Context Extraction s s s m Each context is a nominal variable (feature) For example: Activity (standing, sitting) Context Extraction segmentation, filtering, synchronization, classification, Heart rate (low, medium, ) Breath rate (low, medium, ) c c c c n

14 CoReAmI Approach s s s s m Sensor Data Accelerometer Heart rate sensor Location sensor c c n A: Context Extraction Location Activity Breath rate v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation 4

15 CoReAmI Approach s s s s m Breath rate Low c c n Sensor Data Accelerometer Heart rate sensor Location sensor A: Context Extraction Location Activity Breath rate v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c ML Expert rules v c Aggregation C: Context Aggregation Final decision 5

16 CoReAmI Approach B: Context Modeling Activity Heart rate Breath rate Decision Standing Low: 50 Low: 0 Yes Sitting Medium: 85 Low: No Sitting Low: 60 High: 9 No Standing Medium: 9 High: No Sitting Medium: 9 Low: No Running High: 5 Medium: No Sitting Medium: 0 Low: Yes Running High: 8 Medium: Yes Sitting Low: 59 Low: 6 Yes Standing High: 45 Medium: 9 Yes Running High: 44 High: 6 Yes The fundamental phase of CoReAmI Provides the multiple views Performs context-based partitioning Constructs an ensemble of models Breath rate Low Medium High Activity Heart rate Breath rate Decision Standing 50 0 Yes Sitting 85 No Sitting 9 No Sitting 99 Yes Sitting 59 6 Yes for the low breath rate Activity Heart rate Breath rate Decision Running 5 No Running 8 Yes Standing 45 9 Yes for the medium breath rate Activity Heart rate (min - ) Breath rate (min - ) Decision Sitting 60 9 No Standing 9 No Running 44 6 Yes for the high breath rate The same for: heart rate and activity 6 m breath rate = low m breath rate = medium m breath rate = high

17 CoReAmI Approach s s s s m Sensor Data Accelerometer Heart rate sensor Location sensor c c n A: Context Extraction Activity Heart rate Location v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation 7

18 CoReAmI Approach s s s s m Sensor Data Accelerometer Heart rate sensor Location sensor c c n A: Context Extraction Activity Heart rate Location v c v c v c Rv c Rv c Rv c R R R B: Context Modeling m c m c m c v c Aggregation Final decision C: Context Aggregation 8

19 CoReAmI Approach C: Context Aggregation Executed while evaluating a new instance Models that correspond to the instance s context values are invoked and their decisions are aggregated. m A=Standing m A=Sitting m A=Running m HR=Low m HR=Medium m HR=High Instance Activity (A) Standing Sitting Running Running Aggregation Heart rate (HR) Low Medium High Medium Decision Aggregation techniques: average, weighted average, median, voting, stacking Assumption: it is better to find a good aggregation function instead of choosing the best single model (Gams, Dietterich) Gams M. Weak intelligence: through the principle and paradox of multiple knowledge. New York: Nova Science Publishers, 00. Dietterich TG. Ensemble methods in Machine Learning. In Proc. of the first International Workshop on Multiple Classifier Systems

20 CoReAmI Summary CoReAmI General approach for context-based reasoning in Ambient Intelligence Contexts Multiple-views s s s s 4 s m c c c n c c c n Three phases: Context extraction Context modeling Context aggregation c c c c c cn v c v v x v v v y v v Rv c Rv c Rv c Rv c Rv c Rv c Rv c Rv c Rv c x y z m c m c m c m c m c m c v c v c cn v z cn Aggregation Final decision 0

21 Outline Introduction and Motivation Hypothesis CoReAmI Approach CoReAmI Case Studies Scientific Contributions Conclusions and Future Work

22 Outline Introduction and Motivation Hypothesis CoReAmI Approach CoReAmI Case Studies Energy Expenditure Estimation Activity Recognition Fall Detection Scientific Contributions Conclusions and Future Work

23 Case Study: EE Estimation Problem Description Human Energy Expenditure (EE) estimation Directly reflects the level of physical activity For sports training, weight control, management of metabolic disorders (e.g., diabetes), etc. True EE is difficult to measure by traditional means Solution: Automatic, unobtrusive and accurate EE estimation using various sensors (accelerometers, heart rate, temperature)

24 Case Study: EE Estimation Experimental Setup Sensor equipment Zephyr chest strap Shimmer accelerometers BodyMedia arm band Raw sensor used as input for CoReAmI EE estimation output used for comparison Cosmed ground truth (MET) Experimental scenario 90-minute Supervised by an expert Including 5 activities: lying, walking, running, cycling, shoveling snow, washing dishes, etc. Recorded by 0 volunteers 4

25 Case Study: EE Estimation CoReAmI for EE Estimation Context Extraction Activity Acceleration peaks count Heart rate Breath rate Chest skin temperature Galvanic skin response Arm skin temperature Near-body temperature Context modeling Discretization Decision Tree splitting criteria adapted for regression by Yong et al. 46 regression models 5 regression learning methods Context Aggregation Average Median 5

26 Case Study: EE Estimation Results RMSE Root Mean Square Error (RMSE) for CoReAmI's MET estimation compared to : single regression, Random Subspace and Bagging using 5 base learners: o Artificial Neural Network (ANN) o Support Vector machine for Regression (SVR) o Multiple Linear Regression (MLR) o Gaussian Processes for Regression (GPR) o Model tree (M5P) Base learner Single regression Random Subspace Bagging CoReAmI (average) CoReAmI (median) Improvement Improvement ANN % % SVR % % MLR % % GPR % % M5P % % CoReAmI achieves the lowest RMSE regardless of the baseline learner Statistically significant, confirmed by T-Test p = (0.05) 6

27 RMSE Case Study: EE Estimation Results CoReAmI's MET estimation compared to each of the contexts used individually (only the context models learned for the particular context). Individual contexts CoReAmI Activity Peak count Breath rate Heart rate Chest temp GSR Ambient temp Arm temp CoreAmI 7

28 Estimeted MET Case Study: EE Estimation Results Measured and estimated METs for different types of activities using: CoReAmI BodyMedia (SOTA commercial EE estimation device) ANN trained on chest-accelerometer only (ANN-Acc) CoReAmI BodyMedia ANN-Acc Vigorous stationary cycling Light stationary cycling Running Method RMSE overall CoReAmI Moderate to vigorous household activities BodyMedia.6 Walking Light household activities & exercise Sedentary ANN-Acc True MET 8

29 Case Study: EE Estimation Summary EE Estimation A novel approach to EE estimation using CoReAmI Outperformed the competing approaches The main advantages of CoReAmI: Exploits the complementarity of multiple context-based regression models. Semantically split the set (by using contexts) and not by sampling it with replacement (Bagging) or randomly choosing features (Random Subspace). The results confirmed the hypothesis. 9

30 Case Study Activity Recognition The RaReFall system (Real-time Activity Recognition and Fall detection) Evaluated as the best performing at the EvAAL-AR competition* Uses two accelerometers Problem: distinguishing standing and sitting with single torso accelerometer CoReAmI significantly improved the recognition of the two activities, by using their context The results confirm the hypothesis 80% 70% F-measure 65% 68% 78% 60% 55% 50% Baseline J48 with context features RF with context features CoReAmI *Gjoreski H, Kozina S, Gams M, Luštrek M, Álvarez JA, Hong JH, Ramos J, Dey AK, Bocca M, Patwari N. Competitive Live Evaluation of Activity-recognition Systems. IEEE Pervasive Computing, accepted, (04). JCR:.0 0

31 Case Study Fall Detection Goal: Accurate fall detection with as few sensors as possible Combining contexts extracted from inertial and location sensors by using expert rules CoReAmI significantly improved the detection performance achieving 96.6% F-measure with minimal sensor configuration inertial and location sensor The results confirm the hypothesis F-measure in % AFP MLA CoReAmI Falls Non-fall (fal-like) Non-fall Overall

32 Outline Introduction and Motivation Hypothesis CoReAmI Approach CoReAmI Case Studies Energy Expenditure Estimation Activity Recognition Fall Detection Scientific Contributions Conclusions and Future Work

33 Scientific Contributions. Development of a novel, general, context-based reasoning approach in AmI, called CoReAmI. The approach extracts multiple contexts from sensor and performs reasoning about the user using multiple models constructed for each of the contexts individually.. Application/Adaptation of CoReAmI to three AmI problem domains, which resulted in novel approach in each domain (outperforming the conventional approaches):. CoReAmI for activity recognition. CoReAmI for energy expenditure estimation. CoReAmI for fall detection. Development of a novel method for context-based partitioning of a set into multiple subsets and this way creating multiple views on the by using each feature as a context. Also applicable outside of CoReAmI.

34 Conclusions The thesis addressed the problem of combining multiple sources of information by using contextual information. The hypothesis was confirmed on all three AmI problem domains. The key idea is to partition the set (problem) using each source of information as a context individually. The CoReAmI approach can be adapted and applied to different problem domains, for which the available information can be presented by multiple contexts. 4

35 Future Work Optimizing the hyper-parameters. (Auto Weka) Context-based ensembles. Ensemble-learning algorithm for general purpose ML, (similar to Random forest). Part of WEKA. Directions of how to use CoReAmI on a new problem domain. Release the code, its documentation and appropriate sample applications. Dealing with missing context values. Dealing with redundant or similar context information. (Feature selection algorithms) Deep learning and CoReAmI 5