A MACHINE LEARNING APPROACH FOR HUMAN POSTURE DETECTION IN DOMOTICS APPLICATIONS

Transcription

1 A MACHINE LEARNING APPROACH FOR HUMAN POSTURE DETECTION IN DOMOTICS APPLICATIONS L.Pann*, R.Cucchara*, *D.S.I. Unversy of Modena, va Vgnolese Modena, Ialy emal: ; ABSTRACT Ths paper descrbes an approach for human posure classfcaon ha has been devsed for ndoor survellance n domoc applcaons. The approach was nally nspred o a prevous works of Haraoglou e al. [2] ha uses hsogram projecons o classfy people s posure. We modfy and mprove he generaly of he approach by addng a machne learnng phase n order o generae probably maps. A sasc classfer has hen defned ha compares he probably maps and he hsogram profles exraced from each movng people. The approach resuls o be very robus f he nal consrans are sasfed and exhbs a very low compuaonal me so ha can be used o process lve vdeos wh sandard plaforms. 1. INTRODUCTION Human posure deecon s now wdely explored n many applcaon conexs, rangng from conen-based rereval, survellance, ndoor and oudoor monorng, vrual realy unl anmaon and eneranmen. In parcular, we are neresed n human posure deecon n he framework of home-human nerface. The scope s wofold. The former s o mprove he conrol of elecronc pars n he house, n order o ad he nhabans (especally f dsabled) n daly lfe. The laer s o fnd a way o monor people healh whou some nvasve approaches ofen proposed n ele-medcne. In hs case an nellgen vdeo-based ele-asssence servce, for remoe conrol of dsabled or elderly people, needs advanced compuer vson echnques able o deec movng objecs, classfy people, localze hem n he envronmen and deec her posure and behavor. A ypcal applcaon s deecng f he person s walkng, sng or fallng down and lyng on he floor and evenually sendng an alarm o a remoe human operaor. To hs am hs paper proposes an approach for people posure classfcaon. Ths opc s very popular n hs momen and many papers and research works address hs problem. Neverheless mos of he proposal sll lack of generaly and are oo alored o manually se models. In hs paper we explore an approach ha negraes and mproves some assessed mehods of human posure analyss [1,2,3] wh machne learnng phase o model people posure feaures. These feaures are hen used n sascal classfer n order o predc he human posure. Fnally a fne sae char s adoped o supervse rackng and classfcaon process and generae alarms f requred. In he nex secon some relaed works are descrbed. Secon 3 saes some Inal assumpons and workng hypohess. The defned approach s proposed n Secon 4. Then he learnng envronmen s descrbed and evenually Secon 6 shows resuls on dfferen ess. Conclusons end he paper. 2. RELATED WORKS In recen years an ncreasng number of compuer vson projecs dealng wh deecon and rackng of human posure have been proposed and developed. Analyzng leraure's works, wo basc approach o he problem sand ou. From one sde, sysems (lke Pfnder[5] or W 4 [3]) use a drec approaches, based he analyss on a dealed human body model: an effecve example s he Cardboard Model[4]. In many of hese cases, an ncremenal predc-updae mehod s used, rerevng nformaon from every body par. Neverheless hese sysems are generally oo sensve, when loosng nformaon abou feaures, needng ofen a reboo phase. For hs reason, he segmenaon process has o be as precse as possble usng specfc human cues (eg. skn deecon). Ofen hs can conrbues o sysem nsably because some of hese feaures could no be found n every frame (caused e.g. by overlappng). In order o bypass hese drawbacks, where no body pars conrol needed, many researchers deal wh he problem n a ndrec way usng less, bu more robus, nformaon abou he body. Many of hese approaches are based on human body slhouee analyss. The work of Fujyosh e. Al. [1] uses a synhec represenaon (Sar Skeleon) composed by exremal boundary pons. In [2] Haraoglu e al. add o W 4 framework [3] some echnques for

2 human body analyss usng only nformaon abou slhouee and s boundary. They frs use herarchcal classfcaon n man and secondary posures, processng vercal and horzonal hsogram profles from he body slhouee. Then hey locae body pars on he slhouee boundary s corner. Our approach s smlar o [2], as well as concernng hsogram-based feaures, bu dfferenly from, s no based on a pror defned model. In fac he man srengh of our approach s a machne learnng phase, exploed o creae feaure models, furher used n he classfer. I S A K B O T Back upd MVO deecon Blob Analyss People Tracks People model-free classfer P HPMS 3. ASSUMPTIONS AND CONSTRAINTS The approach ams o defne a framework for posure classfcaon over prevously deeced people. Thus he par of movng objec deecon and people classfcaon s no he srengh of he paper. In order o be as flexble as possble, we consran he problem dmensonaly o he 2D case, reducng he analyss o monocular mages. Ths allows our applcaon o be appled also n hose suaons where a 3D reconsrucon can be ssued, eher for envronmenal or compuaonal lms. Therefore we suppose o have a pon of vew where he human posure can be easly perceved whou ambgues also n he mage plane. We consder mages from ndoor envronmen wh an unknown bu fxed camera poson. The mehod can be exploed n oudoor envronmen oo bu we exclude hgh lumnance varaon and oher arfacs ha could add dffcules n movng objec deecon. Moreover, one people a me s consdered: n hs paper we don address problems of people overlappng, neher problems of human pars hdng, problems ha we could suppose solved by a very precse movng objec deecon and rackng sysems or by a mul-camera sysem. 4. THE PROPOSED APPROACH T Trackng Non-people Tracks PREclassfer Posure Monor Alarm Alarm Conroller Head Fnder H Fgure 1 Layou of he sysem VdeoServer Sms Conroller Phone Messenger A hgh level sysem layou s represened n Fgure 1, where gray block ndcaes pars referred o hs paper. As prevously saed, we assume o have he capably o exrac and rack human bodes from a move. In our ess we explo a sysem called Sakbo (Sascal & Knowledge-Based Objec Deecon) [6] able o deec and rack movng objecs (called MVOs, Movng Vsual Objecs). The basc process s based on a adapve background suppresson, where background s modeled usng sascal and knowledge-based daa. Sakbo can exrac foreground objecs, dsngush real MVOs from shadows and oher arfacs such as ghoss (.e. errors n background modelng), and rack MVOs durng he me. Every racked MVO s hen processed by a pre-classfyng phase, n order o dsngush beween people and nonpeople racks. For hs work, we dd no address hs problem, usng a very smple classfer, based on geomercal feaures. As n Fgure 1, MVOs classfed as People racks are hen passed o HPMS (Human Posure Monorng Sysem) wch deecs and analyzes he posure of he person n order o recognze f dangerous suaons occurred (e.g. fallng or long me nacvy). Ths par s bascally composed by hree block. The frs, PMFC (Posure Model-Free Classfer) s a sascal examplebased classfer able o dsngush beween four man body posures. I uses a human-free model creaed by a specfc ranng phase. When human body posure s recognzed, he sysem pass o he nex block (Head Fnder) wch nvesgae he slhouee boundary, lookng for he op-of-he-head pon. Ths s done usng a Jarvs- Convex Hull scan and lgh rackng phase wh opologcal rules. The Head Fnder has he man goal o make he posure deecon more effecve and robus, dscrmnang beween ambguous cases. Fnally he conrol swches o he hrd block (Posure Monor) whch deec dangerous suaon judgng he hsory of body posure and head poson. Ths s done usng a fne sae char whch models person s behavour. The posure monor generaes, f requred, an alarm as ndcaed n Fgure 1.

3 4-1. Posure model free classfer In order o recognze human body posure, we used a smlar knowledge classfcaon, proposed by [2]. In he maer of fac, we dscrmnae four posures whch represen our classfer s saes: sandng, crawlng, layng and sng. An algorhm layou s repored n Fgure 2. Every MVO, classfed as a person, s processed exracng nformaon from s slhouee as n [6]. Le B a cloud of 2D pons, we compue he cardnaly ( ndcaed wh #) of horzonal and vercal projecons respecvely π and θ as follow: { p p p } θ ( x) = # ( x, y ) B x = x (1) { p p p } π ( y) = # ( x, y ) B y = y (2) These wo neger funcons represen he basc classfer feaures. Alhough oher feaures have been explored and could be added o he classfers, n our ess, hey have been proved o be relable enough. The classfer works n wo dfferen modales: ranng and run-me. In he frs phase we consruc wo probably maps for every posure (sae), usng he respecve slhouee projecons. θ θ people racks Vercal&Horzonal Projecons π Algnmen Bes Score MUX π ranng hgh-level Knowledge S τ =ΘΠ (, ) esmaed posure runung posures PMFC Knowledge Creaon Le B ( xy, ), =1.. T, a ranng se of T 2Dmages referred o a he -h sae, and le ( θ, π ) s projecon slhouee couple seres. (a) (b) (c) (d) Fgure 3 (a),(b) Sandng pose examples Horzonal(c) and Vercal(d) probably map for sandng sae We consruc he couple of 2D probably maps of he sae ( Θ, Π ) as follow, 1 Θ (, ) ( xy = gθ ) (3) T T where g( f) s 1 f y = f ( x) g( f)( x, y) = (4) 0 elsewhere The consrucon for Π s analogous, usng π nsead of θ. In run-me modaly, he PMFC module compares he curren projecon couple wh he corresponden probably maps for every sae. Frs, he Algnmen sep shfs he π and θ hsograms n he cenral par o he probably map and he Bes score module (see Fgure 2) compues he S score quany. To defne he bes classfyng rule we have esed fve dfferen funcons over dfferen es-se models. The problem s how o compare an 1D hsogram wh a 2D probably map consruced by hsograms overlap. The fve mehods belong o hree dfferen approaches: Fgure 2 - Block Dagram for PMFC

4 mehod 1 and 2 provde a 2D comparson by machng he feaure hsogram drecly over he map and combnng he wh a produc and sum operaor, respecvely mean of he mached probables. mehod 3 synheszes he map n wo 1D hsograms, consderng for every hsogram s enry he mean and varance values. Then a Mahalanobs dsance s compued beween hese hsograms and he π and θ hsograms. Fnally, he 4 and 5 approaches consder a comparson beween he feaure and he mean hsogram, usng respecvely nersecon and dfference beween hsograms. measure funcon effcacy rae me (msec) 1 96,63% 0, ,34% 0, ,57% 0, ,34% 0, ,30% 0,0009 Table 1: Resul on dfferen measure funcons The comparson resuls beween classfyng rules are repored n Table 1. Ths shows ha he frs devsed classfer ouperforms oher mehods. We beleve ha n hs framework, as frequenly happens, a mul-classfer could mprove performance consderably. Wh hs a comparable execuon me has been acheved wh. Below a bref explanaon of he mehod s exposed. Le τ = ( Θ, Π) a probably map couple for he sae. We consder he quany obaned as: S θ x= wdh( θ ) θ 1 S = Θ( x, θ ( x) ) (1.5) wdh( θ ) x= 0 In he same way we defne S π usng Π. The fnal score S s compued as he correlaon beween he wo scores as : θ π S = S S (1.6) 5. A FRAMEWORK FOR MACHINE LEARNING We se up a dedcaed vsual framework n order o have he complee conrol n he ranng phase. In parcular he probably maps of Fgure 3 have been compued whou any a pror defned model bu only wh a machne learnng phase. The GUI (graphc user nerface) depced n Fgure 4(a) allows he user o load a ranng vdeo. The ools ndcaes he deeced and racked MVOs, frame by frame, askng he user s supposed posure. A very frendly envronmen prevens he necessy o ask a frame by frame user neracon: n fac he has o ndcae when he people change s saus from a posure o anoher. Also he unknown saus s acceped. In hs manner n few mnues very relable probably maps can be se ( No Classfed n Fgure 4(a) ). Ths ranng phase can be mproved, eraed or changed whenever requred. 6. EXPERIMENTAL RESULTS The sysem has been desgned o mee real-me consrans and o process a suffcen number of frame per second o be reacve and adapve enough for possble alarm. Here we repor some resuls on vdeos acqured n dfferen conexs. In parcular we descrbe hree ess: 1. he effcacy n posure deecon over he same vdeos where he ranng phase has been provded: hs resuls could be neresng for a domoc survellance applcaon, supposng ha an nal ranng s done n he specfc conex on he specfc people, as a sor of nal calbraon of he sysem; 2. he effcacy n posure deecon over oher vdeos, bu aken from he same camera sysem; 3. he effcacy and he generaly of he model n posure deecon on dfferen vdeos (dfferen camera, dfferen scene, dfferen acors) w.r.. he ranng se; 4. he effcency n erms of frame rae. In parcular, here we presen resuls agans ground-ruh on sx vdeos wh hree dfferen acors. Examples of frames are repored n Fgure 4(b) (f). Vdeos have been aken wh wo dfferen cameras, n 320x240 frame sze, compressed n MPEG-2 forma before her processng. For processng evaluaon we used a es machne wh hs characerscs: 1,4 GHz P4, 256 MB RAM. Three vdeos are aken from he same oudoor envronmen (Fgure 4(b) (d) ), whle he las hree ones, n Fgure 4(d) and 4(e), from a dfferen ndoor scene. In Table 2 he effcacy rae s shown o descrbe he sysem behavour over he hree es menoned (pon 1,2,3): he columns of he able repor n order: name of he vdeo (ook from s subjec), he name of he vdeo used for model ranng, he pose number,.e. he number of MVOs classfed as person whch posure has o be classfed, he effcacy rae, ha s he number of correcly deeced poses over he oal poses number, and he acheved frame per second.

5 Vdeo Model Pose Effcacy Example Rae fps Luca1 Luca ,10% 13,95 Luca2 Luca ,83% 13,04 Alberna1 Luca ,93% 10,42 Robero2 Robero ,79% 12,46 Robero2 Luca ,04% 11,33 Robero3 Luca ,31% 11,18 Table 2 - Tes resul on dfferen vdeos The resuls of he second rows esfy he robusness of he mehod (as for he pon 1) when he same envronmen and he same acor s used boh for ranng and for esng. The frs row repors resuls on anoher vdeo wh he same acor and he same camera (bu a dfferen vdeo nsance), whle n he hrd row anoher acor s used and classfed wh he models raned wh he oher person. The sysem exbs a que robusness (abou 90%) n every es where he model comes from he same envronmen. Ths wll be a very common case n our applcaon, where we can quely suppose o have correspondence beween ranng and run-me envronmen (and subjec). Good maches came even where subjec was dfferen from ranng: see he las row of Table 2. We repor also he errors ha can occur when he camera s bad algned. In fac he fourh and he ffh row repor resuls on a vdeo (Robero 2, see for nsance Fgure 4(d) ) where a person falls down n he same drecon of he feld of vew. In hese cases he shape n he 2D mage plane s more smlar o a crawlng model han o a lyng model. Thus he very low effcacy rae (54% and 67% only) s due o hs hgh number of poses ha canno be classfed from ha pon of vew. Therefore hs mehod wll be enhanced n a mul-camera envronmen. In Table 3 we repor he confuson marx ha s he number of wrong classfcaon beween he four saes. be relable and robus f he workng consrans are sasfed. Thus can be consdered a frs sep of a complee sysems ha wll explo a mul-camera sysem n order o provde he bes pon of vew n a complex and cluered envronmen as he domesc one. We would lke o hanks he Domoca per dsabl projec ha suppored hs research and he people of he Imagelab Saff for her valuable help. REFERENCES [1] H.Fujyosh, A.J.Lpon. Real-Tme Human Moon Analyss by Image Skeleonzaon n Fourh IEEE Workshop on Applcaons of Compuer Vson, WACV '98. [2] I.Haraoglu, D.Harwood, L.S.Davs. Ghos: A Human Body Par Labelng Sysem Usng Slhouees n 14h In. Conf. on Paern Recognon, Brsbane, 1998 [3] I.Haraoglu, D.Harwood, L.S.Davs W 4 : Real-Tme Survellance of People and Ther Acves IEEE Trans. On Paern Analyss and Machne Inellgence,22(8),pp , Aug 2000 [4] S.X.Ju, M.J.Black, Y.Yacob Cardoboard People: A Parameerzed Model of Arculaed Image Moon n 2º Inernaonal Conf. on Auomac Face & Gesure Recognon, 1996 [5] C.R.Wren, A.Azarbayejan, T.Darrel, A.P.Penland Pfnder: Real-Tme Trackng of he Human Body IEEE Trans. On Paern Analyss and Machne Inellgence,19(7),pp , Jul 1997 [6] R. Cucchara, C. Grana, M. Pccard, A. Pra, Deecng Movng Objecs, Ghoss and Shadows n Vdeo Sreams n press on IEEE Transacons on Paern Analyss and Machne Inellgence, 2003 g\class No Classfed Sandng Crawlng Sng Layng Table 3: Confuson marx from analyzed vdeos. As a fnal remark we would lke o underlne he speed for he process, snce we are able o process always more han en frames per second wh a sandard PC. 7. CONCLUSION AND ACKNOWLEDGES The paper dscusses nal resuls of deecng human posure for survellance and behavor monorng n domoc applcaons. The dscussed approach proved o

6 (b) (a) (c) (d) (e) (f) Fgure 4: Some snapsho from he Framework: (a) Machne Learnng envronmen (b,c,d,e,f) Frames from vdeo es se