Human Activity Recognition and Estimation of Calorie Expenditure - A Data Analytic Approach

Human Activity Recognition and Estimation of Calorie Expenditure - A Data Analytic Approach Mushtaque Ahamed A Dr. Snehanshu Saha PESIT-Bangalore South Campus December 4, 2015 Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 1 / Appr 24

1 Introduction 2 Experimental Setup 3 Data Collection 4 Feature Extraction Feature Selection and Normalization Dimensionality Reduction 5 Multiclass Support Vector Machines Formulation of Binary Support Vector Machine Training Cross Validation 6 Estimation of Calorie Expenditure 7 Results and Dicussion 8 Refereneces Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 2 / Appr 24

Introduction Recognition of human activity is useful in monitoring the health status of a person. Information about the type of activities that a person has done in a day is vital for doctors who remotely monitor the health their clients. The model helps determine the type of activity of a person, the amount of time and calories spent during that activity. The information thus gathered can profile the activities of a person towards maintaining their health. The challenge here is to accurately detect the type of activity done by a person. The following set of measures was executed to solve this problem which includes data Collection, feature extraction, dimensionality reduction, support vector machine training and cross validation and testing in real time. The amount of time were determined, we estimated the amount of calories burnt using a standard table of Metabolic Equivalents Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 3 / Appr 24

Experimental Setup Figure : Arduino - An Open Source Electronics Hardware Platform Figure : MPU6050 - A Tri-axes Accelerometer by Invensense Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 4 / Appr 24

. Two test subjects were considered for the experiment. A Tri-axis Accelerometer called MPU6050[6] and Arduino Uno, a micro controller were used to collect the data. Five activities were defined for classification. The activities are sleeping, sitting, standing, walking and running, necessarily in that order. The sensor was worn by the subjects as lockets and different activities were performed, each for about a minute. The activities were labeled 1 to 5 in increasing levels of intensity from sleeping to running Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 5 / Appr 24

Data collection A 4 dimensional dataset was recorded, which includes acceleration along the three co-ordinate axes and the time when it was sampled, accurate to milliseconds. The sensor is sensitive to accelerations from -2g to +2g, with a resolution of 16 bits. The raw data from the sensor consisted of values from -32768 to 32767. It was normalized in range from -2 to +2. The sensor was set to sample at a rate of 20Hz. The data was recorded in CSV file format. 10 samples were taken for each of the five activities. Each file contains 1000 records. Subsequently 500 samples were created from 50 files by splitting them evenly with a window size 100 without any overlap. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 6 / Appr 24

Feature Extraction A MATLAB script was written to extract these feature from the samples and a dataset of features, consisting of 500 rows and 23 columns was created. Each of the 23 features listed above were placed in a row. The data set was labeled from 1 to 5 as shown in table 2. Metabolic Equivalent [4, 5] (MET) values were extracted from the National Cancer Institute. The MET values in the following table is the amount of calories spent by a human body in one hour, per kg of the body weight, while engaged in the corresponding activity. Activity Label Metabolic Equivalent Sleeping 1 0.92 Sitting 2 1.12 Standing 3 1.30 Walking 4 3.80 Running 5 7.50 Table : List of Activities and their MET values. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 7 / Appr 24

Feature Selection and Normalization The following 23 features[2, 2] were extracted from the 500 samples in the data set, where each sample consisted of 100 values of x, y and z axes data, as shown in table below. Feature Feature Feature Feature Number Description Number Description 1 mean of x axis 13 kurtosis of x axis 2 mean of y axis 14 kurtosis of y axis 3 mean of z axis 15 kurtosis of z axis 4 avg. mean of 3 axes 16 avg. kurtosis of 3 axes 5 std. dev of x axis 17 energy of x axis 6 std. dev of y axis 18 energy of y axis 7 std. dev of z axis 19 energy of z axis 8 avg. std. dev of 3 axes 20 avg. eneryg of 3 axes 9 skewness of x axis 21 corr. between x and y axis 10 skewness of y axis 22 corr. between x and z axis 11 skewness of z axis 23 corr. between y and z axis Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 8 / Appr 24

Dimensionality Reduction The feature vector was of 23 dimensions and the data set was of 500 samples in size, with 5 classes. Principal Component Analysis [1] was performed on the data set and it was found that the principal 8 dimensions captured 85% of the variance of the data. Dimensionality Reduction[3] was performed on the data set in which the dimensionality of the data set was reduced from 23 to 8. This new data set with 500 samples and 8 dimensions was considered for Principal component were calculated as follows. Letting X be matrix of 500 rows and 23 columns, representing the data set, with each sample in a row. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December - A4, Data 2015 Analytic 9 / Appr 24

The covariance matrix Σ was calculated as Σ = X T X m where m was the number of samples. The covariance matrix Σ was decomposed into the matrices U, S and V using the singular value decomposition such that, USV = Σ The first eight diagonal elements constituted the top eight eigen values of Σ such that they added up to 85% of the total of all eigen values. the corresponding 8 eigen vectors formed the matrix U and were exploited to reduce the dimension. The dataset X was projected onto the reduced dimension space( subspace) as X reduced = X U column(1:8) Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 10 / Appr 24

Figure : Original 23 Dimensional data set Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 11 / Appr 24

Multiclass Support Vector Machine Five Binary Support Vector Machine models were trained on the labeled, dimensionally reduced data set of 350 samples Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 12 / Appr 24

Formulation of Binary Support Vector Machine Consider the reduced dataset X reduced which has 500 samples arranged in rows, and each row has 8 dimensions( columns). Let X i represent the ith sample of the datasetx reduced and let y i represent the label of the data set. If the model is to classify the activity a then y i is 1 if the the sample i is a positive sample and y i is -1 if the sample i is a negative sample. The Support vector machine model creates a hyperplane of the feature vector space which is identified by its normal vector w and the bias b which represents the offset hyperplane incurs from the origin. Once the normal vector w and the bias b are known, given a data sample x to classify, the following classification function may be deployed. Classify(x) = 1 if w T x b 1 +1 if w T x b +1 0 if 1 w T x b +1 Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 13 / Appr 24

To obtain the model parameters w and b, we formulate an optimization problem as follows. Combining the positive and the negative test cases, we can write the following equation. y i (w T x i b) 1, 1 i m where m is the number of the samples in the training data set. The hyperplane is such that it has maximum perpendicular distance from the data points in feature vector space. This perpendicular distance is inversely proportional to the magnitude of w, that is w. Thus the following optimization problem can be posed to solve for w and b. arg min (w,b) 1 2 w by introducing lagrangian multiplier α, the above constrained problem can be posed as follows arg min (w,b) max α 0 { 1 2 w m i=1 ] α i [y } i (w T x i b) 1 Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 14 / Appr 24

This can be solved by using quadratic programming technique.the stationary Karush Kuhn Tucker [4] condition implies that the solution can be expressed as a linear combination of the training vectors. and w = b = 1 m SV m α i y i x i i=1 m SV (w T x i y i ) where m SV is the number of support vectors selected by the model. i=1 Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 15 / Appr 24

Training Each of the five models were trained to test for positivity of a sample to a class. 70% of the data set was used to train the model where each class had about 70 positive samples and about 280 negative samples. The model was used to test accuracy of the training by making it predict the class of a sample. Five Binary classification models were used to build the multi-class SVM. To predict the class to which a sample belonged, the sample was tested on all five models. The affiliation of a sample to a particular class was determined by checking which of the models tested the sample as positive. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 16 / Appr 24

Cross Validation Cross Validation was performed to check the accuracy of the training model. 150 samples were used to test. Confusion matrix [1] was computed based the training data and the test data. Sensitivity, specificity and accuracy for each of the class was computed from confusion matrix. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 17 / Appr 24

Activity Confusion Matrix Sensitivity Specificity Accuracy Sleeping Sitting Standing Walking Running 71 0 0 279 67 0 0 283 74 0 0 276 69 0 0 281 69 0 0 281 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 Table : Summary of test results for training data set Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 18 / Appr 24

Activity Confusion Matrix Sensitivity Specificity Accuracy Sleeping Sitting Standing Walking Running 26 3 0 121 31 2 4 113 26 0 6 118 30 1 2 117 31 0 2 117 1.0 0.9758 0.98 0.8857 0.9826 0.96 0.8125 1.0 0.96 0.9375 0.9915 0.98 0.9393 1.0 0.9866 Table : Summary of test results for testing data set Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 19 / Appr 24

Estimation of Calories The following is the proposed method to estimate the amount of calories spent of various activities. The sensor will be worn by the user as a locket. The sensor will send the stream of data via bluetooth or some other wireless interface to the smart phone. The model will run on the sample as input and predict the type of activity the person is undergoing. The time-stamp of the sample is recorded. Using the standard MET table, the weight of the user and the time and activity data, the following formula is employed to calculate amount of calories burnt. Energy calorie = MET activity Weight user Time hours Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 20 / Appr 24

Results and Dicussion In this paper, the authors have discussed a model to classify human activity by using multi-class support vector machine. We have collected the data, extracted the features, normalized it and dimensionally reduced it from 23 dimensions to 8 dimensions. The data set was used to train five SVM classifiers, one for each class of activity. We tested the model with the cross validation data set and found it to be 97.332% accurate in classifying the accelerometer sample into one of the five activities. The proposed model elucidates vital information about the health of a person. It estimates the calorie expenditure for each of the activities. Such a model is very useful for the doctors who are monitoring their clients remotely. With our model, the doctors can remotely diagnose the user and predict the user s health. All this is achieved with a wearable device and a simple but trustworthy computational model. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 21 / Appr 24

References Abdi, Hervé, and Lynne J. Williams. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics 2.4 (2010): 433-459. Cleland I, Kikhia B, Nugent C, Boytsov A, Hallberg J, Synnes K, McClean S, Finlay D. Optimal Placement of Accelerometers for the Detection of Everyday Activities. Sensors. 2013; 13(7):9183-9200. Keogh, Eamonn, et al. Dimensionality reduction for fast similarity search in large time series databases. Knowledge and information Systems 3.3 (2001): 263-286. Kuhn, H. W.; Tucker, A. W. (1951). Nonlinear programming. Proceedings of 2nd Berkeley Symposium. Berkeley: University of California Press. pp. 481 492. MR 47303 Lavie, Carl J., and Richard V. Milani. Metabolic equivalent (MET) inflation-not the MET we used to know. Journal of cardiopulmonary rehabilitation and prevention 27.3 (2007): 149-150. Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 22 / Appr 24

References Contd... Schuldt, Christian, Ivan Laptev, and Barbara Caputo. Recognizing human actions: a local SVM approach. Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on. Vol. 3. IEEE, 2004. Soliman, Samir S., and S-Z. Hsue. Signal classification using statistical moments. Communications, IEEE Transactions on 40.5 (1992): 908-916. Suykens, Johan AK, and Joos Vandewalle. Least squares support vector machine classifiers. Neural processing letters 9.3 (1999): 293-300. http : //appliedresearch.cancer.gov/atus met/met.php http : //playground.arduino.cc/main/mpu 6050 http : //www.invensense.com/mems/gyro/documents/ps MPU 6000A 00v 3.4.pdf Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 23 / Appr 24

Acknowledgement Thanks to ICCC VVIT, Bangalore And Thank You Everyone Mushtaque Ahamed A Dr. Snehanshu Saha Human (PESIT-Bangalore Activity Recognition South Campus) and Estimation of Calorie Expenditure December -4, A2015 Data Analytic 24 / Appr 24