Meta-Activity Recognition: A Wearable Approach for Logic Cognition-based Activity Sensing

Transcription

1 Met-Ativity Reognition: A Werle Approh for Logi Cognition-se Ativity Sensing Lei Xie, Xu Dong, Wei Wng, n Dwei Hung Stte Key Lortory for Novel Softwre Tehnology, Nnjing University, Chin Emil: lxie@nju.eu.n, ongxu@isl.nju.eu.n, ww@nju.eu.n, hungw@isl.nju.eu.n Astrt Ativity sensing hs eome key tehnology for mny uiquitous pplitions, suh s exerise monitoring n eler re. Most tritionl pprohes trk the humn motions n perform tivity reognition se on the wveform mthing shemes in the rw t representtion level. In regr to the omplex tivities with reltively lrge moving rnge, they usully fil to urtely reognize these tivities, ue to the inherent vritions in humn tivities. In this pper, we propose werle pproh for logi ognition-se tivity sensing sheme in the logil representtion level, y leverging the mettivity reognition. Our solution extrts the ngle profiles from the rw inertil mesurements, to epit the ngle vrition of lim movement in regr to the onsistent oy oorinte system. It further extrts the met-tivity profiles to epit the sequene of smll-rnge tivity units in the omplex tivity. By leverging the lest eit istne-se mthing sheme, our solution is le to urtely perform the tivity sensing. Bse on the logi ognition-se tivity sensing, our solution hieves lightweight-trining reognition, whih requires smll quntity of trining smples to uil the templtes, n user-inepenent reognition, whih requires no trining from the speifi user. The experiment results in rel settings shows tht our mettivity reognition hieves n verge ury of 92% for user-inepenent tivity sensing. I. INTRODUCTION Nowys tivity sensing hs eome key tehnology for mny uiquitous pplitions suh s exerise monitoring n eler re. For exmple, in the ily exerise monitoring, it is essentil to figure out wht kins of exerises the humn sujets i everyy. The rising of the werle evies hs provie new opportunities for tivity sensing uring humn motion. The werle evies suh s the smrt wthes re usully emee with inertil sensors like the elerometers, gyrosopes n mgnetometers. They re le to ontinuously trk the humn sujet s movements n lssify them into the orresponing tivities y mthing the wveforms of inertil mesurements ginst the templtes. However, numer of ommon tivities, e.g., umell url n rope skipping, elong to the omplex tivities. The omplex tivity refers to n tivity whih hs lrge rnge of movement n inurs rottions on multiple joints of the lims, e.g., the movement hs ngle hnge of more thn 45 n involves more thn 2 joints of the lims. Moreover, it usully hs two omplex spets: the wiespre vritions in tivity etils n the lrge movement rnge. Due to the user-speifi hrters like the heights, lim lengths n moving ehviors, there exist ovious evitions in the rw inertil mesurements from ifferent humn sujets uring the proess of the omplex tivity. Therefore, tritionl tivity sensing shemes [] [3] re either se on the user-epenent reognition, whih requires to reor the trining t from the urrent user to improve the reognition ury, or relying on hevytrining, whih requires to ollet lrge quntity of trining smples to uil the templtes. It is essentil to propose rn-new tivity sensing sheme, suh tht the erive reognition moels n e slle to ny ritrry humn sujets in user-inepenent n light-trining pproh. In this pper, we propose werle pproh for logi ognition-se tivity sensing, y leverging the met-tivity reognition in the logil representtion level. We minly fous on the omplex tivities from humn sujets, s shown in Fig.. Our pproh is se on the oservtion tht when the humn sujet is performing n ritrry tivity, he/she is experiening very similr sequene of smll-rnge-tivity units in the logil spet, espite of the etile ifferenes in the wveforms of the rw inertil mesurements. We leverge the notion met-tivity to enote the smll-rnge-tivity units whih ompose ommon tivity of humn sujet. Given n ritrry tivity, our pproh first extrts the ngle profiles from the rw mesurements to epit the ngle vrition of lim movement in the onsistent oy oorinte system. Then, it further extrts the met-tivity profiles to epit the sequene of smll-rnge-tivity units in the speifi tivity. By leverging the lest eit istne-se mthing sheme, our solution is le to urtely perform the tivity sensing. Sine slle reognition moel is erive from the met-tivity-se templtes in the logil representtion level, our solution hieves lightweight-trining reognition, whih requires smll quntity of trining smples to uil the templtes, n user-inepenent reognition, whih requires no trining from the speifi user. :Upright Brell Row 2:Dumell Curl 6: Rope Skipping 7: Butterfly 3:Dumell Flies 8: Cle Crossover 4:Dumell Lterl Rise 9: Ping-Pong Swing Fig.. Exmple Complex Ativities 5:Dumell Trieps Extension : Bminton Swing

2 There re two key tehnil hllenges in relizing the tivity sensing sheme. The first hllenge is to relize the tivity sensing in user-inepenent pproh, suh tht the erive reognition moel n e extene to reognize the tivities of ny ritrry humn sujets, regrless of the etile ifferenes n inherent evitions in the tivities from ifferent humn sujets. To ress this hllenge, we propose to leverge the ngle profiles, i.e., the ngles etween the speifie lim n the oorinte xes, to epit the lim movements. The ngle profiles re le to pture the ngle vrition of the lim movements reltive to the humn oy, whih tkle the evition etils use y the user-speifi hrters like the height. Moreover, we propose the metho of met-tivity reognition to perform tivity sensing in the logil representtion level, se on the sequene of mettivity profiles, so s to tkle the vritions in the long sequene of smll-rnge tivities. Speifilly, oring to the inertil mesurements ollete from humn motion, inste of performing the wveform-se mthing like ynmi time wrping, we eompose the omplex tivity into sequene of met-tivities, n use this sequene to reognize the omplex tivity vi the lest eit istne-se mthing. The seon hllenge is to uil onsistent sheme to epit the humn motion oring to the inertil mesurements from the werle evies. Sine the humn sujets my perform the tivities towrs ny ritrry iretion uring the humn motion, this uses the templtes for tivity reognition to epen hevily on the tul iretion the humn oy is fing, n further enhnes the omplexities in performing tivity sensing ue to the inonsisteny. To ress this hllenge, we epit ll the inertil mesurements of humn motion in terms of oy oorinte system in onsistent pproh. Speifilly, oring to the grvity iretion n the mgneti iretion extrte from the inertil mesurements, we trnsform the mesurements from the wth oorinte system (WCS) to the glol oorinte system (GCS). Then, y speifying two signl gestures, i.e., extening the rm to the front n ropping the rm ownwr, we n figure out the orienttion of the humn oy in the glol oorinte system oring to the mesurements in the signl gestures, thus we further trnsform the mesurements to the oy oorinte system (BCS). To the est of our knowlege, this pper presents the first stuy of using the metho met-tivity reognition for logil ognition-se tivity sensing. Speifilly, we mke three key ontriutions in this pper. ) Inste of performing wveform mthing on the inertil mesurements in the rw t level, we extrt the ngle profiles to epit the ngle vrition of lim movements, n leverge the met-tivity profiles to epit the omplex tivities in the logil representtion level, suh tht the erive reognition moel is slle enough for the tivity reognition on ny ritrry humn sujets. 2) We uil oorinte system trnsformtion sheme to trnsform the inertil mesurement from the wth oorinte system to the oy oorinte system, suh tht the lim movement n e epite in onsistent pproh, regrless of the ext orienttion of the humn oies. 3) We hve implemente prototype system to evlute the rel performne, the experiment results in rel settings shows tht our met-tivity reognition hieves n verge ury of 92% for user-inepenent tivity sensing. II. PROBLEM FORMULATION In this pper, we investigte the werle pproh for tivity sensing, i.e., werle evie is worn y the humn sujet to ontinuously ollet the inertil mesurements of humn motion, then n tivity sensing sheme is require to urtely reognize the omplex tivities of lim movements from humn sujets. The omplex tivity refers to the kin of tivity with lrge rnge of movement, suh s sit-ups n umell lterl rise. Without loss of generlity, we leverge the smrt wth to sense the humn motions, whih is emee with inertil sensors inluing the elerometer, gyrosope n mgnetometer. In this pper, we im to esign n tivity sensing sheme, y onsiering the following metris in system performne: ) Aury: The expete ury for the tivity sensing sheme to suessfully mth speifi tivity to orret tivity shoul e greter thn speifie threshol, e.g., 85%. 2) Time-effiieny: The time ely of the tivity reognition proess shoul e less thn speifie threshol, e.g., 5ms. 3) User-inepenene: When performing tivity sensing, no trining t shoul e require from the speifie user. 4) Lightweight-trining: The essentil quntity of the trining smples to uil the templtes shoul e smll enough.. III. MODELING THE HUMAN MOTION A. Coorinte System Trnsformtion In regr to tivity sensing, s the rw inertil mesurements re ollete from the emee inertil sensors in the smrt wth, they re mesure y referene to the oy frme of the smrt wth. However, the wth oorinte system is ontinuously hnging with the rm/wrist movement uring the proess of humn motion, thus the mesurements from the wth oorinte system nnot e use s stle referene for the speifie tivities. In ft, sine the humn sujet my e performing the tivity towrs ny ritrry iretion, the movements shoul e epite s the movement of rms or legs reltive to the humn oy, regrless of the solute moving iretion of the lims. Therefore, in orer to perform tivity sensing in slle pproh, it is essentil to trnsform the mesurement of lim movements from the wth oorinte system to the oy oorinte system. ) From Wth Coorinte System to Glol Coorinte System: Fig. 2() shows the three xes of the wth oorinte system. The X w -xis refers to the iretion of the lower rm when the wth is worn on the wrist, the Y w -xis refers to the iretion of the strp of the wth, n the Z w -xis refers to the iretion whih is perpeniulr to the wth surfe. Aoring to the elertion mesurements from the elerometer, we n extrt onstnt grvittionl elertion s vetor g from the low pss filter (suh s

3 Zw Zg Xw Xg Yw Zg(Z) Yg () The reltionships etween WCS () The reltionships etween GCS n GCS n BCS Fig. 2. The reltionship etween ifferent oorinte systems X Xg! G! Y Wth the Butterworth filter [4]) in the wth oorinte system. Moreover, oring to the mgneti mesurements from the mgnetometer, we n extrt the mgneti fore s vetor m in the wth oorinte system. Then, we n uil glol oorinte system (GCS) se on the grvity iretion n mgneti iretion in the wth oorinte system. The proeure is s follows: After we otin the grvity vetor g, we erive its opposite vlue n normlize this vetor s z g = g g, we then set this vetor z g to represent the glol Z g -xis s it is in the opposite iretion of the grvittionl elertion n it is perpeniulr to the horizontl plne. After omputing the ross prout y = g m, we otin vetor y tht is perpeniulr to the plne etermine y the two istint ut interseting lines orresponing to g n m. We normlize this vetor s y g = y y. Sine the vetor y g is on the horizontl plne, we set this vetor y g to represent the glol Y g -xis. After tht, y omputing the ross prout x = g y, we otin vetor x tht is orthogonl to the plne etermine y the two istint ut interseting lines orresponing to g n y. We normlize this vetor s x x x g = to represent the glol X g-xis.. Fig. 2() further shows the reltionship etween the three xes (x w, y w n z w ) of WCS n the three xes (x g, y g n z g ) of GCS. To quntify the orienttion ifferene etween the wth oorintes n glol oorintes, we use the iretion osine representtion [5]. In the iretion osine representtion, the orienttion of the glol oorinte reltive to the wth oorinte system is speifie y 3 3 rottion mtrix C, in whih eh olumn is unit vetor long one of the wth oorinte xes speifie in terms of the glol oorinte xes. A vetor quntity v w efine in the wth oorinte system is equivlent to the vetor v g = C v w efine in the glol oorinte system. In this wy, we re le to trnsform ny inertil mesurement v w from WCS to the orresponing inertil mesurement v g in GCS. During the humn motion, the iretions of g n m re ontinuously upte in WCS to trk the three xes of GCS, so s to further upte the rottion mtrix C in rel-time pproh. 2) From Glol Coorinte System to Boy Coorinte System: During the humn motion, the humn sujet my e fing ny ritrry iretion in regr to the glol oorinte system. Hene, lthough we n erive the inertil mesurement of lim movements in GCS, these mesurements my not e onsistent with eh other even if they elong to the sme tivity, ue to the ifferenes in the fing Yg iretions. Therefore, it is essentil to uil oy oorinte system (BCS) to epit the lim movements in onsistent pproh y referene to the humn oy. In regr to the oy oorinte system, we set the vetor orresponing to the heing iretion of the humn sujet to represent the Z xis. For the horizontl plne whih is orthogonl to the Z xis, we set the vetor whih is prllel to the physil plne of the oy to represent the X xis, n set the vetor whih is perpeniulr to the physil plne of the oy to represent the Y xis. Fig. 2() shows the three xes (X, Y, Z ) of BCS n the three xes (X g, Y g, Z g ) of GCS in regr to the physil plne of the humn oy, respetively. Consiering tht the humn sujet n perform the tivity with ifferent orienttions of the physil plne of the oy, e.g., stning on the floor or lying on the floor, in ll situtions, we n trnsform ny inertil mesurement from the GCS to BCS y lso using the iretion osine representtion. The orienttion of the oy oorinte system reltive to the glol oorinte system is speifie y 3 3 rottion mtrix C, in whih eh olumn is unit vetor long one of the glol oorinte xes speifie in terms of the oy oorinte xes. A vetor quntity v g efine in GCS is equivlent to the vetor v = C v g efine in BCS. In this wy, we n trnsform ny inertil mesurement from the GCS to the BCS. In Setion IV, we will introue the pproh to ompute the rottion mtrix C, y leverging two signl gestures. In regr to the tivities where the physil plne of the humn oy is ontinuously hnging, e.g., sit-ups, we n set the initil physil plne of the humn oy s the referene oy oorinte system. In this wy, eh of the following inertil mesurements re mesure in terms of the referene oy oorinte system. B. Moeling the Humn Motion with Met-Ativity Eh omplex tivity, e.g., umell sie rise n entover umell lterls, is performe with lrge rnge of movement. So it n e eompose into series of smllrnge movements whih re sequentilly performe over time. Therefore, we leverge the term met-tivities to enote these smll-rnge movements. Eh met-tivity is efine s unit movement with logilly the miniml grnulrity in regr to the moving rnge. We n efine the whole set of omplex tivities s set C, n the whole set of met-tivities s set M. Then, oring to the ove efinition, eh omplex tivity i C n e epite s series of met-tivities, i.e., i = m j,, m jk, where m j M. ) Angle Profiles: In regr to the tivity sensing, ue to the ifferenes in humn-speifi hrters suh s the height, rm length, n moving ehvior, ifferent humn sujets my perform the sme tivity with ifferent spees n mplitues. This uses nonnegligile evitions mong the inertil mesurements of the sme tivities in oth time omin n spe omin. Therefore, the met-tivity shoul e epite in slle pproh, suh tht the tivity sensing sheme n e tolernt to the vrines in the lim movements. However, tritionl inertil mesurements suh

4 s the liner elertions re very sensitive to the spees n mplitues of the lim movements, whih fil to epit the met-tivity in slle pproh. Fortuntely, it is foun tht, uring the proess of lim movements, the ngle vritions etween the lim n the oy re muh more stle thn the tritionl inertil mesurements, whih re regrless of the humn-speifi hrters suh s the height n rm length. Therefore, in this pper, we propose to leverge the ngle profiles, i.e., the ngles etween the lower rm n the three xes in the oy oorinte system, to epit the met-tivities of the lim movements. Speifilly, sine the iretion of X w xis in the wth oorinte system is onsistent with the lower rm iretion, we n use the vetor x w to epit the lower rm iretion in the oy oorinte system. Fig. 3() shows the vetor x w to epit the lower rm iretion in the BCS. As shown in Fig. 3(), we respetively enote the ngle profiles, i.e., the ngles etween the lower rm n the X, Y n Z xes in the BCS, s α, β n γ. In orer to ompute the ngle profiles, we tke the ngle α s n exmple, suppose the lower rm vetor n the vetor of the X-xis re v (v = x w ) n u, respetively, in the BCS. Then α n e ompute oring to the osine vlue s follows: os α = v u v u = v x u x + v y u y + v z u z v 2 x + v 2 y + v 2 z u 2 x + u 2 y + u 2 z. () For ny speifie vlue of os α, there exist two solutions of α in the rnge etween n 36. Hene, we first ompute the orresponing solution α within the rnge [, 8 ], we then further etermine the vlue of α s follows: { α if vy α = 36 (2) α if v y <. Similrly, we n ompute os β n os γ oringly, then the vlues of β n γ n e etermine s follows: { β if vz β = 36 β (3) if v z <. { γ if vx γ = 36 (4) γ if v x <. In this wy, the ngle profiles α, β, γ in the BCS n e etermine within the rnge [, 36 ]. We further onute empiril stuies to vlite the ove jugement. We invite four humn sujets (,, n ) with ifferent heights n geners to perform the speifie omplex tivities, n respetively reor the orresponing elertion mesurements n the ngle profiles in regr to eh of the xes in the oy oorinte system. We normlize ll the mesurements to the rnge [, ] for fir omprison. Fig.4() n Fig.4() respetively shows the elertion mesurements n ngle profiles of the tivity Dumell Curl. It is foun tht, mong ifferent humn sujets, there exist As mentione in Setion III, the vetor x w in BCS n e ompute oring to the iretion osine representtion, it n e ontinuously upte in rel time pproh. ovious vrines in the elertion mesurements, wheres the vrines in the ngle profiles re reltively smll. We further ompute the DTW istnes etween eh pir of mesurements from ifferent humn sujets, n otin the verge istne s the metri to quntify the orresponing vrines. Fig.4() shows the DTW istnes, respetively, for the tivity Dumell Curl n Sit-Up. It is foun tht for oth ses the ngle profiles hieve muh smller istnes thn the elertion mesurement, whih implies tht the ngle profile is more stle metri to epit the humn motion. X Boy Z Coorinte System Y xw Lower Arm Diretion () The lower rm iretion X xis α Z xis γ β () The ngle profiles xw Y xis Fig. 3. Derive the ngle profiles in the oy oorinte system 2) Met-Ativity Profiles: Ielly, in orer to epit the lim movements of humn sujet, the ngle profiles of ll skeletons in the oy oorinte system re require to e pture. Nevertheless, sine the lower rm usully experienes movement with firly lrge rnge uring the proess of humn motion, it is lrey representtive to perform the tivity sensing se on the ngle profiles of the lower rm. For the ngle profiles α, β n γ, oring to the efinition, they hve the following reltionship: (os α) 2 + (os β) 2 + (os γ) 2 =. (5) Given ny two vlues of the α, β n γ, the other one n e ompute oring to Eq.(5). However, it still hs two nite solutions oring to the orresponing osine vlue. Therefore, the rm-iretion in the BCS n e uniquely etermine vi the three prmeters α, β, γ. For ny speifie met-tivity, while it is eing performe, the ngle profiles α, β, γ re ontinuously hnging. The met-tivity shoul hve the following properties for ny of the prmeters α, β, γ : ) The vrition rnge of ny ngle profile shoul e less thn threshol δ, e.g., 3. 2) The vrition tren of ny ngle profile shoul e monotoni, e.g., monotonilly inresing or eresing. 3) The time urtion of the met-tivity shoul e less thn threshol t, e.g., 5ms. The first property implies tht the moving rnge of the met-tivity shoul e smll enough, the seon property implies tht the moving iretion of the met-tivity shoul e monotoni, the thir property implies tht the time urtion of the met-tivity shoul e limite, even if the moving rnge is still smll enough. Therefore, for eh imension of the ngle profiles α, β, γ, we n uniformly ivie the rottion rnge [, 36 ] into multiple setors, while the ngle of eh setor is no greter thn the threshol δ. In this wy, we n use the speifie

5 () Alertion mesurements of Dumell Curl () Angle profiles of Dumell Curl setor to epit the orresponing met-tivity in the speifie imension. Moreover, onsiering the rm rottion n e nti-lok-wise or lok-wise, it is essentil to further lel eh setor oring to the rottion tren. Therefore, suppose the numer of setors is m, we n lel eh setor with ifferent ID from to m in n nti-lok-wise pproh: for the ith setor, if the rottion iretion is nti-lok-wise, then we lel it with s j, otherwise, we lel it with S j. Fig. 5 shows n exmple of these met-tivity setors, where eh setor hs n ngle of 3. These setors re lele from s to s if the rottion iretion is nti-lok-wise, n they re lele from S to S otherwise. In this wy, we n use these isrete sttes rther thn the ontinuous wveforms to represent the met-tivities. In omprison to the ontinuous wveform-se representtion in the rw t level, this isrete stte-se representtion is se on the logi ognition of the humn motion, whih is more slle to the inherent vrines use y user speifi hrters. S2 S3 S4 S s s s s s2 s3 s4 S S5 s5 s6 S s9 s8 s7 S Aelertion Angle Profiles Dumell Curl Fig. 4. The elertion mesurements n ngle profiles in X, Y n Z-xes S7 S s: º-3º s: 3º-6º S9 s: 33º-36º S8 S: 3º-º S: 6º-3º S: 36º-33º Fig. 5. The setors to epit met-tivity in eh imension of ngle profiles IV. SYSTEM DESIGN The overll system is ompose of three mjor moules, s shown in Fig.6: Dt Aquisition n Preproessing tkes the rw inertil mesurements s input. It first performs the oorinte trnsformtion to trnsform the mesurement from WCS to BCS. Then, it extrts the ngle profiles n further split the series into seprte omplex tivities. Met-Ativity Segmenttion n Clssifition segments single omplex tivity into series of met-tivities, n lssifies the segmente met-tivities into orresponing tegories. Complex Ativity Reognition performs tivity reognition se on the sequenes of met-tivities from the test omplex tivity, y leverging the lest eit istne-se mthing sheme. A. Dt Aquisition n Preproessing ) Coorinte Trnsformtion: As mentione in Setion III, we n trnsform the mesurement from the WCS to Rw Sensor Dt Sit-Up () DTW istne in ifferent omplex tivities Dt Aquisition n Preproessing Coorinte Trnsformtion Angle Profile Extrtion Segmenttion Met-Ativity Segmenttion n Clssifition Met-Ativity Segmenttion Met-Ativity Clssifition Complex Ativity Reognition Lest Eit Distne-se Mthing Reognition Result Fig. 6. The system frmework BCS, y using the Diretion Cosine metho. To figure out the orienttion ifferene, i.e., the rottion mtrix C etween BCS n GCS, efore the humn sujet performs the omplex tivities, he/she is require to perform the following two signl gestures in vne: ) Exten the rm to the front: let the humn sujet exten his/her rm to the front of the oy, the rm iretion is onsistent with the Y xis in the BCS; 2) Drop the rm ownwr: let the humn sujet rop the rm ownwr long his/her legs, the rm iretion is opposite to the Z g xis in the BCS. Fig.7() n Fig.7() shows n exmple of the two signl gestures, respetively. In this wy, y omputing the orresponing vetor of the rm iretion in the GCS, we re le to figure out the rottion mtrix C, whtever the humn sujet is stning or lying on the floor.! Arm Diretion Glol Coorinte System Glol Coorinte System Arm Diretion () Signl gesture : exten the rm () Signl gesture 2: rop the rm to the front ownwr Fig. 7. The signl gestures 2) Angle Profiles Extrtion: As forementione in Setion III, tke the smrt wth s n exmple, we use the vetor x w, i.e., the iretion of X w xis in the WCS, to epit the rm iretion in the BCS. Then, oring to the rm vetor x w (t) t time t, we n extrt the ngle profiles α(t), β(t), γ(t) over time in the BCS oring to Eq. (4)-(7). 3) Segmenttion: In prtie, the humn sujet my ontinuously perform series of omplex tivities. Therefore, the reognition system shoul first split these series of omplex tivities into seprte tivities, then we n further ientify whih tivity pttern the urrent movement elongs to. As the humn sujet usully tkes short puse etween two

6 jent omplex tivities, we use ngle hnges to etet the strt n en of omplex tivity. Speifilly, we leverge sliing winow (whih is set to 3s in our implementtion) to ontinuously store the reent ngle profiles α(t), β(t), γ(t). For eh ngle profile, the system omputes erivtive of these vlues, i.e., the ifferene etween the urrent n the previous smple. When the ifferene from one or more ngle profiles exees ertin threshol (whih is set to in our implementtion), the system etets the eginning of omplex tivity; similrly, when the ifferene from lmost ll ngle profiles flls ellow the sme threshol, the system etets the en of omplex tivity. B. Met-Ativity Segmenttion n Clssifition ) Met-Ativity Segmenttion: After preproessing, we re le to otin the series of ngle profiles α(t), β(t), γ(t) (t [t s, t e ]) for single omplex tivity, where t s n t e re respetively the strt time n the en time of the omplex tivity. Then, it is essentil to further segment the omplex tivity into series of met-tivities, oring to the series of ngle profiles. Bse on the properties of the met-tivity mentione in Setion III.B, we n erive the segmenttion onitions for ny of the ngle profiles (i.e., α, β or γ). A stright-forwr solution for met-tivity segmenttion is to segment the series of ngle profiles y heking ll the three imensions of the ngle profiles simultneously. If the segmenttion onition is stisfie in ny of the imensions, the entire series of ngle profiles re segmente s single met-tivity. However, this solution uses the series of ngle profiles for single met-tivity to e too frgmente fter segmenttion, sine the series of ngle profiles in the other imensions re not yet stisfie for the segmenttion onition to e omplete met-tivity, whih is not suitle for the following met-tivity lssifition. Therefore, we propose to perform the met-tivity segmenttion over eh imension of the ngle profiles in seprte pproh. For eh imension, we leverge sliing winow (whih is set to 5ms in our implementtion) to ontinuously store the reent ngle profiles. Our solution sns the series of ngle profiles to verify if ny of the segmenttion onition is stisfie, we then segment the ngle profile series s single met-tivity in the orresponing imension. After the mettivity segmenttion, we n otin seprte segmenttion for eh imension of the ngle profiles. Fig. 8 shows n exmple of the met-tivity segmenttion for the tivity Dumell Curl in the three imensions of the ngle profiles. s s2 s3 s4 s5 S6 S4s8 S2 S S S s s s2 S8 S2 s2 s2 s9 s ss s Fig. 8. An exmple of the met-tivity segmenttion 2) Met-Ativity Clssifition: After the met-tivity segmenttion, oring to eh imension of the ngle profiles, the met-tivity shoul e further lssifie into one speifi setor se on its moving rnge n rottion iretion. As the met-tivity epits proess of movement rnging from strt ngle θ s to n en ngle θ e, it usully hs some flututions in the wveforms of the ngle profiles. Hene, when performing met-tivity lssifition, it is essentil to onsier the vrition tren of ngle profiles, inste of only the strt n en ngles. For exmple, test met-tivity my inrese the ngle profile slowly in prt of the setor s 5, n then inrese the ngle profile rpily in prt of the setor s 6, so it shoul e lssifie to the setor s 5 sine its mjor ngle profile vries in this re. Therefore, we leverge the metho of Dynmi Time Wrping (DTW) to mth test met-tivity to orresponing setor y referring to the vrition tren of ngle profiles. The proeure is s follows: In regr to ny speifi setor, onsiering the rottion rnge n iretion, its strt ngle profile is η s n its en ngle profile is η e. Then, suppose the orresponing met-tivity is performe with uniform spee in the time intervl of t ( t=5ms in our implementtion), we n use liner funtion f(t) to epit the templte of this met-tivity, i.e., f(t) = η s + ηe ηs t t. Then, given test met-tivity with the ngle profile η(t), n the templtes of met-tivities orresponing to the setors, we n use vetor P l with length l n vetor R l with length l to enote the test met-tivity s ngle profile n the speifie templte met-tivity s ngle profile, respetively. For eh pir of test met-tivity P l n the templte mettivity R l, we onstrut istne mtrix D l l s n input to the DTW lgorithm, where eh element D i,j is efine s the Eulien istne etween eh pir of ngle profiles V P,i n V R,j. The output of DTW is wrping pth π = {π,..., π k } suh tht the istne etween the sequenes is minimize: rgmin π π = k i= D x(π),y(π). Hene, given the test mettivity, we n enumerte ll templte met-tivities n leverge DTW to ompute the orresponing istne. We then selet the templte met-tivity with the smllest istne s the lssifition result. C. Complex Ativity Reognition After the met-tivity lssifition, eh omplex tivity n e eompose into sequene of met-tivities, respetively, in regr to the three imensions of ngle profiles. We ll them met-tivity profiles of the omplex tivity. In Tle I, we show four exmple met-tivity profiles for the speifie omplex tivities. In this pper, we leverge the Lest Eit Distne-se mthing sheme (LED) to perform the omplex tivity reognition. LED ompres the test omplex tivity ginst the templte omplex tivities in regr to the orresponing met-tivity profiles. It omputes the lest eit istne etween eh pir of test omplex tivity n templte omplex tivity, n selet the templte omplex tivity with the smllest istne s the mthing result. By referring to the eit istne [7] for mesuring the ifferene etween two

7 Complex Ativity Met-Ativity Profiles Dumell α : S 9, S 8, S 7, S 5, s 7, s 8 Trieps β : s 8, S 8 Extension γ : s 5, s 6, S 7 Upright α : s, s 2, s 3, S 3, S 2, S Brell β : s, S, S, S Row γ : s 3, s 8, S 8 Dumell α : s, s 2, s 3, s 9, s Lterl β : s 2 Rise γ : S 2, S, S, s, s, s Butter α : s 2 Fly β : S 2, S, S, S, s, s 2 γ : s, s 2, S 2, S, S TABLE I EXAMPLE META-ACTIVITY PROFILES FOR THE COMPLEX ACTIVITIES sequenes of strings, we leverge this term to enote the ifferene etween two sequenes of met-tivity profiles. In orer to ompute the eit istne etween two pirs of omplex tivities, it is essentil to first onsier the istne etween two met-tivities. As forementione in Setion III, for eh imension of the ngle profiles, the met-tivity is proess whih is performe in speifie setor with speifie rottion iretion. When onsiering the istne etween two met-tivities, we shoul tke these two issues into onsiertion, i.e., the istne etween setors n the istne etween rottion iretions. Consiering the istne etween setors, ssume the numer of setors is m, for ny two met-tivities m i n m j, suppose their orresponing setor numers re respetively s i n s j ( s i s j < m), the istne etween them is efine s follows: s (m i, m j ) = min{(s j s i )mom, (s i s j + m)mom}. The istne is the minimum istne etween the two setors s i n s j either lokwise or nti-lokwise. Consiering the istne etween rottion iretions (lok-wise or nti-lokwise), for ny two met-tivities m i n m j, if they hve the sme rottion iretion, then we set the istne r (m i, m j ) to. Otherwise, we set the istne r (m i, m j ) to Ω (Ω = m/4 in our implementtion). Hene, onsiering the ove two issues, the istne etween two met-tivities m i n m j is s follows: (m i, m j ) = s (m i, m j ) + r (m i, m j ). (6) Therefore, for eh imension of the ngles profiles, omplex tivity i C n e epite s sequene of the met-tivities, i.e., i = m j,, m jk, where m j M. Then, for speifie imension, onsiering ny two omplex tivities, e.g., n, we n ompute their istne L, (, ) y referring to the Levenshtein istne [7]: mx(i, j) µ if min(i, j) =, L L, (i, j) =, (i, j) + µ (7) min L, (i, j ) + µ otherwise. L, (i, j ) + ( i, j ) where L, (i, j) is the istne etween the first i mettivities of n the first j met-tivities of, µ is the verge istne etween ny two met-tivities (µ =.75 m in our implementtion), n ( i, j ) is the istne etween the ith met-tivity of n the jth met-tivity of. After tht, for ny two omplex tivities i n j, we the istnes from ll three imensions together, n otin the overll istne etween the two omplex tivities s follows: L i, j = L 2 + i(α), j(α) L2 + i(β), j(β) L2. (8) i(γ), j(γ) Therefore, given test omplex tivity i, we enumerte ll templte met-tivity profiles of ll omplex tivities j C n ompute their istne L i, j, then we selet the tegory of the templte omplex tivity with the lest istne s the reognition result. V. PERFORMANCE EVALUATION A. Experimentl Setup We hve implemente prototype system using the nroi phone (SAMSUNG Glxy S5) 2, whih is tthe to the wrist of the humn sujet, s shown in Fig.9. The nroi phone is emee with inertil sensors inluing elerometers n mgnetometers. The lower-rm iretion is onsistent with the Y -xis of the smrt phone s lol oorinte system. In the experiment, we let volunteers perform tegories of omplex tivities, they hve ifferent heights, geners, n ges. For eh tegory of omplex tivity, 2 smples of inertil mesurements re ollete for eh sujet. In orer to evlute the performne for user-inepenent tivity sensing, we leverge the n-fol ross-vlition s follows: for eh roun of evlution, we selet one humn sujet s the test se, n otin the templte profiles from n of the remining humn sujets. We then evlute the reognition ury n time ely for the three solutions: ) Aelertion-se Mthing (AM): It uses the DTW to perform wveform-se mthing in terms of the elertion mesurements. 2) Angle Profiles-se Mthing (APM): It uses the DTW to perform wveform-se mthing in terms of the ngle profiles. 3) Met-Ativity Reognition (MAR): It uses the lest eit istne-se mthing in terms of the met-tivity profiles. B. Prmeter Seletion Z-xis X-xis Y-xis Fig. 9. Experimentl Setup For the met-tivity reognition, the ngle of mettivity setor, i.e., δ, is very ruil to the performne in terms of reognition ury n time effiieny. It iretly etermines the numer of met-tivities within speifie omplex tivity. Therefore, we onut experiments to evlute the performne with ifferent vlues of δ. We set the numer of humn sujets in templte onstrution to 5. 2 As COTS smrt wthes re still not emee with mgnetometers to help uil the oy oorinte system, so in this pper we hoose to use the nroi phone s the testing werle evies.

8 Reognition ury The ngle of the met-tivity setor () Aury for ifferent setor ngles A A2 A3 A4 A5 A6 A7 A8 A9 A A A2 A3 A4 A5 A6 A7 A8 A9 A () Confusion mtrix for AM A A2 A3 A4 A5 A6 A7 A8 A9 A Time ely(ms) The ngle of the met-tivity setor () Time ely for ifferent setor ngles A A2 A3 A4 A5 A6 A7 A8 A9 A (e) Confusion mtrix for APM (f) Confusion mtrix for MAR Fig.. The experiment results We first evlute the performne in terms of reognition ury when the ngle δ is vrie from 5 to 45, s shown in Fig. (). It is foun tht ll the reognition uries re greter thn 83% when the ngle is vrie from 5 to 45. The highest ury is hieve when the ngle is set to, wheres the lowest ury is hieve when the ngle is set to 45. The reson is tht, when the ngle is firly lrge, e.g., 45, the grnulrity of the met-tivity is too orse to epit the movement of humn sujets, thus it les to mny mismthes in tivity reognition. However, when the ngle is firly smll, e.g., 5, the grnulrity of the met-tivity is too fine to tolerte the etile evitions ue to humnspeifi hrters, thus the performne is lso egre in omprison to the optimum se. Therefore, the prmeter of the setor ngle shoul e refully selete for improving the performne in reognition ury. We then evlute the performne in terms of time effiieny when the ngle δ is vrie from 5 to 45, s shown in Fig. (). It is foun tht, s the ngle δ inreses from 5 to 3, the verge time ely rpily ereses from 45ms to 2ms. The reue time ely is use y the inresing grnulrity of the met-tivity, whih reues the proessing time ost. However, when the ngle δ further inreses to 45, the verge time ely slightly inreses to 49ms, s the time ely for the met-tivity lssifition inreses ue to the inrese size of input to the DTW lgorithm. Therefore, to hieve n pproprite tre off etween the ury n time effiieny, in the following experiment, we set the ngle δ to in MAR to hieve the optimize performne. C. Evlute the Reognition Aury ) Sensitivity to the numer of trining smples: Sine we im to hieve the lightweight-trining reognition, we require the numer of trining smples to e s smll s possile. Therefore, s we ollet 2 smples from eh humn sujet to uil the templtes for eh omplex tivity, we vry the numer of humn sujets involve in the templte onstrution from to 8, n evlute the verge reognition A A2 A3 A4 A5 A6 A7 A8 A9 A Reognition Aury A A2 A3 A4 A5 A6 A7 A8 A9 A AM APM MAR The numer of humn sujets in templte onstrution () Aury for ifferent numer of trining smples Time ely(ms) 3 2 AM APM MAR The numer of humn sujets in templtes (g) Time ely for ifferent solutions ury, s shown in Fig. (). It is foun tht, in lmost ll situtions, MAR hieves the est performne wheres AM hieves the worst performne. Speifilly, when the numer of humn sujets in templte onstrution is, the numer of trining smples is smll. AM hieves poor ury of 6% s it lks enough templtes for urte mthing. APM leverges the ngle profiles to mitigte the impt of vrines in the rw inertil mesurement, thus it enhnes the ury from 6% to 8%. Due to the hrter of logil ognition, MAR further improves the ury to 87% even if the trining smples re so limite. This implies tht our met-tivity reognition hieve rther goo performne for user-inepenent reognition while requiring lightweighttrining. As the numer of templtes inreses, the uries of the three solutions re ll inresing to lose vlue of 92%. Nevertheless, MAR lwys hieves the lest vrines in reognition ury sine it les to very stle performne. 2) The mthing rtios mong multiple tivities: We further investigte the onfusion mtries for the three solutions, s shown from Fig.() to Fig.(f). We set the numer of humn sujets in templte onstrution to 2, n the tivities re liste from A to A oring to the orer in Fig.. Aoring to the mthing results in the three onfusion mtries, it is foun tht APM is le to reue most of the mismthes use y AM, so APM hieves the reognition rtio of % for most tivities. However, APM still fils to urtely reognize some tivities suh s A 6, A 9 n A, sine these tivities usully hve lrger movement rnges n more movement vritions in etils. Fortuntely, MAR is le to further reue these mismthes n improve the reognition ury to firly high level. Moreover, MAR hieves the lest vrines in reognizing multiple tivities in omprison to the other two solutions. D. Evlute the Time Effiieny We further evlute the time ely of proessing the tivity sensing, respetively, for AM, APM n MAR. We vry the numer of humn sujets in templte onstrution from to

9 8 n evlute the orresponing time ely. It is foun tht, in ll situtions, MAR hieves muh smller time ely (ll less thn 45ms) thn AM n APM. Moreover, s the numer of humn sujets in templte onstrution inreses from to 8, the time ely of AM n APM inreses rpily, wheres the time ely of MAR keeps firly stle. The reson is s follows: As oth AM n APM uses the DTW for mthing, it requires lrge mount of time to proess the mesurements with smll grnulrity in the rw t level, however, MAR proesses the mesurements with muh lrger grnulrity in the met-tivity level. It tkes most of the proessing time on the met-tivity segmenttion rther thn mthing. Thus, MAR hieves the est time effiieny in tens of milliseons. VI. RELATED WORK Werle Devie. Reent reserhes onsier leverging the inertil sensors emee in werle evies to etet n monitor the user s tivities [], [8] [3]. Wrist mounte inertil sensors re wiely use for rm-se tivity sensing [], [2]. RistQ [] leverges the elertions from wrist strp to etet n reognize smoking gestures. Krts et l. [2] uses wrist mounte inertil sensors to trk steering wheel usge n ngle. Foot-mounte inertil sensors re leverge for inoor loliztion y sensing the ptterns of footsteps [2], [3]. LookUp [3] uses shoe-mounte inertil sensors for lotion lssifition se on surfe grient profile n step ptterns. Roertson et l. [2] proposes n pproh for simultneous mpping n loliztion for peestrins se on oometry with foot mounte inertil sensors. Wireless Signls. Another rnh of tivity reognition solutions exploit the hnge of wireless signls (inluing WiFi signls, RF-signls, et.) inurre y the humn tivities [4] [9]. FEMO [4] provies free-weight exerise monitoring sheme y tthing RFID tgs on the umells n leverging the Doppler shift profile of the reflete kstter signls for tivity reognition. Wng et l. [5] propose CSI se humn tivity reognition n monitoring system, y quntittively uiling the orreltion etween CSI vlue ynmis n speifi humn tivity. E-eyes [6] presents evie-free lotion-oriente tivity ientifition t home through the use of fine-grine WiFi signtures. RF-IDrw [7] n infer humns writing y trking pssive RFID tg tthe to his/her fingers. However, most of the ove tivity reognition shemes leverge the tritionl wveform-se mthing to proess the inertil mesurement/wireless signls in the rw t level. In this pper, we propose the met-tivity reognition, whih elongs to logi ognition-se tivity sensing. Our pproh hieves lightweight-trining reognition, whih requires smll quntity of trining smples to uil the templtes, n user-inepenent reognition, whih requires no trining from the speifi user. VII. CONCLUSION In this pper, we propose werle pproh for logi ognition-se tivity sensing sheme in the logil representtion level, y leverging the met-tivity reognition. Our solution extrts the ngle profiles to epit the ngle vrition of lim movement in the onsistent oy oorinte system. It further extrts the met-tivity profiles to epit the sequene of smll rnge tivities in the omplex tivity. By leverging the lest eit istne-se mthing sheme, the experiment results shows tht our solution hieves n verge ury of 92% for user-inepenent tivity sensing. ACKNOWLEDGMENTS This work is supporte in prt y Ntionl Nturl Siene Fountion of Chin uner Grnt Nos , , 63249, ; JingSu Nturl Siene Fountion, No. BK2539. This work is prtilly supporte y Collortive Innovtion Center of Novel Softwre Tehnology n Inustriliztion. REFERENCES [] A. Prte, M. C. Chiu, C. Chowitz, D. Gnesn, n E. Klogerkis. Risq: Reognizing smoking gestures with inertil sensors on wristn. In Proeeings of ACM MoiSys, 24. [2] C. Krts, L. Liu, H. Li, J. Liu, Y. Wng, S. Tn, J. Yng, Y. Chen, M. Gruteser, n R. Mrtin. Leverging werles for steering n river trking. In Proeeings of IEEE INFOCOM, 26. [3] S. Jin, C. Borgittino, Y. Ren, M. Gruteser, Y. Chen, n C. Fin Chisserini. Lookup: Enling peestrin sfety servies vi shoe sensing. In Proeeings of ACM MoiSys, 25. [4] S. Butterworth. On the theory of filter mplifiers. Wireless Engineer, 7(6):536 54, 93. [5] M. R. Spiegel, S. Lipshutz, n D. Spellmn. Vetor nlysis. Shums Outlines (2n e.), pges 5 25, 29. [6] K. Shoemke. Animting rottion with quternion urves. Computer Grphis, 9(3): , 985. [7] G. Nvrro. A guie tour to pproximte string mthing. ACM Computing Surveys, 33():3 88, 2. [8] S. Consolvo, D. W. MDonl, T. Tosos, M. Y. Chen, J. Froehlih, B. Hrrison, P. Klsnj, A. LMr, L. LeGrn, n R. Liy. Ativity sensing in the wil: A fiel tril of uifit gren. In Proeeings of ACM CHI, 28. [9] K.-H. Chng, M. Y. Chen, n J. Cnny. Trking free-weight exerises. In Proeeings of ACM UiComp, 27. [] A. Khny, S. Mellory, E. BerlinN, R. Thompsony, R. MNneyy, P. Oliviery, n T. Plotzy. Beyon tivity reognition: Skill ssessment from elerometer t. In Proeeings of ACM UiComp, 25. [] N. Roy, H. Wng, n R. R. Chouhury. I m smrtphone n I n tell my user s wlking iretion. In Proeeings of MoiSys, 24. [2] P. Roertson, M. Angermnn, n B. Krh. Simultneous loliztion n mpping for peestrins using only foot-mounte inertil sensors. In Proeeings of ACM UiComp, 29. [3] O. Woomn n R. Hrle. Peestrin loliztion for inoor environments. In Proeeings of ACM UiComp, 28. [4] H. Ding, L. Shnggun, Z. Yng, J. Hn, Z. Zhou, P. Yng, W. Xi, n J. Zho. Femo: A pltform for free-weight exerise monitoring with rfis. In Proeeings of ACM SenSys, 25. [5] W. Wng, A. X. Liu, M. Shhz, K. Ling, n S. Lu. Unerstning n moeling of wifi signl se humn tivity reognition. In Proeeings of ACM MoiCom, 25. [6] Y. Wng, J. Liu, Y. Chen, M. Gruteser, J. Yng, n H. Liu. E-eyes: evie-free lotion-oriente tivity ientifition using fine-grine wifi signtures. In Proeeings of ACM MOBICOM, 24. [7] J. Wng, D. Vsisht, n D. Kti. RF-IDrw: Virtul touh sreen in the ir using RF signls. In Pro. of ACM SIGCOMM, 24. [8] X. Zheng, J. Wng, L. Shnggun, Z. Zhou, n Y. Liu. Smokey: Uiquitous smoking etetion with ommeril wifi infrstrutures. In Proeeings of IEEE INFOCOM, 26. [9] Q. Pu, S. Gupt, S. Gollkot, n S. Ptel. Whole-home gesture reognition using wireless signls. In Proeeings of ACM MOBICOM, 23.