Proceedngs of he IEEE ICRA 9 Workshop on People Deecon and Trackng Kobe, Japan, May 9 Esmaon of Pedesran Dsrbuon n Indoor Envronmens usng Mulple Pedesran Trackng Muhammad Emaduddn Robocs & AI Deparmen Naonal Unversy of Scences and Technology H-1, Islamabad, Paksan emadudd@usc.edu Absrac - We propose a wo-er daa analyss approach for esmang dsrbuon of pedesran locaons n an ndoor space usng mulple pedesran deecon and rackng. Mulple pedesran deecon uses laser measuremen for sensng pedesrans n a heavly occluded envronmen whch s usually he case wh mos ndoor envronmens.. We adap a parcle fler based mulple pedesran racker o address he consrans of a lmed number of sensors, heavy occluson and real-me execuon. Under hese condons any deecon and rackng echnque s lkely o encouner a degree of error n cardnaly and poson of pedesrans. A compleely new approach s employed whch measures he error n racker oupu due o occluson and uses o esmae a probably densy funcon whch represens he probable number of pedesrans locaed a a parcular exhb a a parcular me. The end resul of he sysem s a varable represenng cardnaly of pedesrans a a parcular exhb. Ths varable follows a dsrbuon whch s approxmaely normal where he varance of he probably dsrbuon funcon s drecly proporonal o he error encounered by he racker because of occluson. The accuracy of our deecon and rackng algorhm was esed boh separaely and n conuncon wh he second-er pedesran dsrbuon analyss and found marked mprovemen makng our average pedesran counng accuracy o a leas 9% for all he pedesran poson daa ha we gahered wh average pedesran densy a.34 pedesrans per sq. meer. Snce he envronmen consrans for our sysem are unprecedened, we were unable o compare our resul o any prevous expermens. We recorded he number of people a each exhb manually o esablsh he ground ruh and compare our resuls. I. INTRODUCTION Indoor deecon and rackng of pedesrans has a wde specrum of applcaons rangng from archecural desgn of walkways o conrollng pedesran flow a publc places lke heares, museums, arpors, spors arenas, convenons ceners and parks. Our effor n hs paper s o devse a sysem capable of rackng and counng pedesrans n real-me usng mnmal resources. The word mnmal here refers o he fewes possble laser measuremen sensors wh consrans on her orenaon and placemen. In real lfe applcaons, (e.g. narrow walkways, mounng on vehcles ec he se of feasble locaons for deployng sensors can be severely consraned. In our experence he requremens for non-nrusveness of sensors.e. relable elecrcal power and maxmum sensor coverage, lm he number and placemen of sensors. Among Ths work was parally suppored by.us Naonal Scence Foundaon under her Crosscung Human and Socal Dynamcs (HSD program. Dylan A. Shell Compuer Scence Deparmen Unversy of Souhern Calforna Los Angeles, CA 989, USA dshell@robocs.usc.edu he se of sensors ha are avalable for rackng pedesrans, laser-range fnders (LADAR are presenly among he mos relable and accurae; hey relably provde sub-cenmeer accuracy a mllsecond frequences n range of envronmens. Bu even wh he hgh fdely ha laser sensors provde, crcumsances exs n whch laser-based echnques fal o produce dependable pedesran rackng resuls. Whle he echnques nroduced n [1], [3], [4], [5], [6] and [7] are among he mos successful n erms of rackng accuracy, hey are sgnfcanly lmed when dealng wh occlusons [] and many have a compuaonal complexy ha means hey reman unsuable for real-me applcaons. Whle our developed sysem s no as accurae as he onlne-learnng racker descrbed n [4], produces dependable resuls n heavly occluded envronmens whle no compromsng s real-me applcaons. II. EXPERIMENT SETUP Our es-bed for he deecon and rackng algorhm consss of a unnel lke pahway whch has fve exhbs along s pah and wo access doorways o an unobserved heare exhb close o he cenre of he pahway as shown n Fgure. 1. Pedesrans can ener and ex he secon of museum under dscusson usng any of he wo accesses o he pahway. Pedesrans can also ener n and ou from any of he doorway accesses o he unobserved exhb. Ths pahway was chosen o be our es case as allows varous suaons ha can nroduce complcaons n ndoor pedesran deecon and rackng o be esed. These suaons nclude: ( Pedesrans move n a narrow unnel lke space hus here exss a hgh probably of occluson due o close proxmy of people: ( The pahway conans secons ha can help us observe compleely dsnc behavour of pedesrans e.g. a he exhbs where we expec pedesrans o sop and gaher, away from exhbs where we expec pedesrans o walk wh a relavely longer srde and a enrances where pedesrans are usually n an exploraory mode and end o change walkng drecon very quckly: ( The wo only access doorways o he crcular heare are observed by our laser scanners hus we were able o keep rack of people presen whn he heare whou even drecly observng hem by smple coun-keepng of people leavng and enerng he heare, (v Pedesrans vsng he exhbs were boh aduls and chldren whch requred us o une deecon o accep a relavely wde range of values for srde of a pedesran, (v Pedesran groups,
Enrances/exs for he arena Exhb 1 Exhb Exhb 4 Exhb 5 Fg.1. Tes arena whch were usually a group of sudens lead by a eacher were a frequen occurrence a our es bed. In order o mee our obecve of rackng a farly large number of people ulzng mnmum possble resources, we decded o place wo SICK Laser Measuremen Sensors (LMS a a dsance of approxmaely 8 meers from each oher o cover an area of roughly 7 sq. meers. Ranges of our laser sensors overlapped for almos 16 sq. meers of area ou of he oal hus gvng us a relavely accurae coun n he overlapped area. The oal area was dvded no 5 cells each represenng an exhb (as shown wh red lnes n Fgure 1. These cells wll be laer used o gaher coun of pedesrans vsng each exhb a any gven me. The off-he-ground hegh of roang mrror whn laser sensor was se a 9.9cm for all observaons durng he proec. Ths hegh plays a crucal role n deecon and assocaon of clusers o he pedesrans snce lowerng he sensor hegh gves us dscree clusers represenng fee bu a he same me decreases our chances of deecon of fee snce we rase our fee whle walkng. On he oher hand ncrease n hegh ends o gnore dscree clusers from fee of chldren or people wh shor heghs. The effecve scannng frequency of laser sensors s abou 39Hz. The foreground pons from he laser sensors were exraced easly by background learnng and subracng from laser sensor readngs. III. THE SYSTEM Laser sensor Exhb 3 Enrances/exs o unobserved exhb Laser sensor We presen a sysem ha s capable of deecng, rackng and he gvng us he probably of pedesran coun a requred locaons. I comprses of wo ers explaned n deal below Ter 1: Deecng and Trackng Pedesrans As wll be shown, hs nvolves a non-rval adapaon and exenson of he echnques developed n [1]. We descrbe he hree pars below. A. Cluserng: Our algorhm sars by cluserng ncomng pons from laser sensors usng mean shf cluserng algorhm. The sysem needs he sze of cluser parameer a hs pon whch s equvalen o he average area A of fooprn of an adul foo.e..4 sq. meers []. B. Temporal Correlaon Analyss: Afer classfcaon of pons no clusers we erae hrough clusers and esablsh whch clusers belong o whch pedesran based on he noon ha each pedesran can be assocaed wh a maxmum of wo clusers n nh frame whch le closes o he pedesran n (n- 1s frame, we call hs sep as emporal correlaon sep. We dvde hs sep no wo phases ( Phase one sars wh denfcaon of poenal fee of pedesrans by calculang closes clusers and separang hese as pars. Only hose clusers qualfy as fee par whch le whn a parameer know as ner-fee dsance I and have szes n he vcny of A sq. meers. es _ par( C, C { Par ( C, C dsa nce( C, C mn( dsa nce( C, C } max( sze( C, sze( C.4 Par 1 ( C, C I The remanng unpared clusers are hough o be clusers whch are formed due o he fac ha we cross our fee whle walkng hus renderng a sngle cluser n he laser sensor readngs. The area of such clusers can be a mos wce he fooprn area of an average human foo ( Second phase consss of deermnng wheher each cluser par belongs o a newly deeced pedesran or should be consdered an updae for an already racked pedesran P on he scene. Ths s done usng assocaon dsance D ha s he maxmum dsance ha a pedesran can ravel beween readngs colleced by laser sensors. Therefore he value of D s dependen upon he maxmum walkng speed of pedesrans n he arena. assocae ( Par { updae ( Par, P, P 1 1 1 dsa nce ( Par, P D mn( dsa nce ( Par, P 1 } Inroducng above condon lms he dsance ravelled by pedesrans whle beng occluded and sll beng effecvely racked as a unque pedesran. We observed ha he perodc moon of pedesran fee descrbed n [1] remans undeecable mos of he me n envronmens cluered wh occlusons. Algorhm n [1] defnes merge as a sage durng walk when clusers of boh fee of a pedesran come close ogeher and her clusers merge whle spl s descrbed as a case when he pedesran connues o walk afer a merge and clusers of boh fee spl par. Whle merge and spl cases were occasonally encounered durng (1 (
our expermen, we found ou ha deecon of pedesrans n hs manner s boh naccurae and compuaonally burdensome. The reason of naccuracy les n followng noons (a Mos of he me we observe pedesrans walkng n close proxmy o oher pedesrans or n he shape of groups, hs ends o produce merges and spls ha nvolve fee of wo dfferen pedesrans (b Due o frequen occluson (see Fgure We are lkely o mss spls and merges belongng o a pedesran hus renderng our spl/merge deecon mechansm useless under hs suaon (c Pedesrans may no always walk, hey mgh us sand for a whle. Our soluon o hese problems as evden by (1 and ( s o gnore he merge and spl cases compleely hus reducng he me complexy of emporal correlaon sep o (n logn + (nm.log(nm/3 where n s he number of clusers and m s he number of pedesrans on he scene. Afer hs sep, deeced pedesrans along wh her assocaed clusers are provded o he racker.e. our nex sep n sequence. C. Trackng: The racker s he componen of our sysem ha s responsble for esmang he parameers of moon and locaon aached wh our pedesran based on gven updaes from emporal correlaon sep. I uses a parcle fler o esmae he poson p, srde s, drecon d and phase ph of a pedesran as already employed n [1]. In bref he racker keeps rack of he pedesrans n hree sub-seps ( Updae Sep: Tracker weghs each pedesran's parcles proporonal o her dsance o he pons belongng o s assocaed clusers: ( Samplng Sep: Afer updae sep, he racker randomly samples he weghed parcles where he lkelhood of any parcle o be chosen s proporonal o s wegh. Thus a ceran predefned number of parcles M are chosen: ( Propagaon Sep: In he las sep of rackng he sampled M parcles are propagaed hrough a muldmensonal space represenng he moon of he racked pedesran accordng o he walk model descrbed n deal n [1]. Ths sep modfes he poson, srde, drecon and he walkng phase of a pedesran and s performed whou akng no accoun wheher a pedesran has receved updaes or no. The propagaon of pedesrans ha do no receve updaes helps our racker o rack occluded pedesrans up ll a ceran amoun of dsance D. Durng rackng each foreground pon belongng o he pedesran s used for calculang s dsance wh each of M parcles belongng o he same pedesran n racker. For a maxmum densy of 1.8 pedesrans per sq. meer under whch our racker can perform opmally, performs on he average nearly 54, calculaons o updae, sample and propagae 16 pedesrans hrough a sngle eraon. Gven such hgh a penaly n erms of execuon me, we deemed exremely mporan for our algorhm o produce resuls wh nearly same accuracy usng fewer less compuaonal resources n order o reman useful n real-me applcaons. Consderng hs requremen, we were able o successfully rack pedesrans wh very lle degradaon of accuracy by skppng unnecessary observaons from laser sensors (See Occluson caused by anoher pedesran Fg. An S-T represenaon of observable fee daa Table 1. The laser sensors provde our sysem wh observaons effecvely afer every. seconds. We forced our sysem o consder observaons afer every.5 seconds.e. n effec droppng every second observaon. Ths reduced he oupu accuracy by a very neglgble value bu he performance gan was more han mes. Snce our sysem s specfcally desgned o handle occluson, skppng an observaon makes our sysem behave as f he skpped observaon s due o an occluson, hus by ncreasng he D parameer n emporal correlaon module compensaes for mos of he loss n accuracy. The resulan sysem descrbed up ll now s relavely robus and accurae means of deecng and rackng pedesrans gven he fac ha we are performng hese seps n real-me. Ter : Pedesran Dsrbuon Analyss Self occluson caused by one foo n fron of he oher Laser sensor Alhough relably n he resuls could have been acheved by negrang echnques lke onlne-supervsed learnng [4], Mulple Hypohess Trackng [3] or Auxlary Parcle Fler swchng [] n he frs er, bu dong so wll exclude our racker compleely from he realm of real-me sysems. Thus, he second er of our sysem s desgned o furher enhance he relably of he pedesran coun oupu for each exhb whle keepng he compuaonal complexy growh nearly consan. We erm hs er as he pedesran dsrbuon analyss er as s concerned wh keepng rack of pedesrans crossng n and ou of each cell cells whn he envronmen. A cell comprses of area n fron of an exhb defned usng cell boundares (as marked n Fgure 1. By mananng nformaon abou he dsrbuon of people over cells, alhough he sysem canno answer quesons abou where parcular pedesrans are, one may sll nvesgae quesons abou he flow of people and how her (average roue selecon depends on he (average presence or absence of people.
18 18 14 5 1 1 5 1 3 4 5 6 7 55 3 35 15 Fg.3. Pedesran Dsrbuon Analyss er Oupu Deecng number of people crossng no and ou of each cell we were able o deduce he number of people N n each cell a each me-sep. Ths number conans a ceran error drecly proporonal o he percenage of he cell boundary hdden from laser sensors due o he pedesrans sandng/walkng very close o he laser sensors. In order o facor-n he error presen n hs number, we choose o represen he oupu of he sysem for each cell as a dsrbuon over he number of people. A dsrbuon varable X for each cell a any gven me s a sae of our belef ha represens all pas observaons ncludng he curren one. Ths s acheved va updang he dsrbuon varable X for each exhb a each me-sep. Varable X s defned as X 1 1 1, u N r, N ( r 1,..., N r (3 u Here u s an ndex ha runs hrough he range of weghs whch represen our probably densy funcon (pdf. Mos generally he range adusmen value r s subec o he requremen of he analys whch dffers wh he applcaon of our sysem. (We used he physcal capacy of he exhbs o place lms on hs range of values. Changng he value of r ncreases or decreases he doman of our dsrbuon funcon. 1 N s a number ha has he maxmum wegh 1 u 1 assocaed o n he dsrbuon X from prevous mesep. Followng seps updae he varable va a Gaussan updae U X a each me-sep whose varance s deermned by he percenage of cell boundary occluded a any gven momen. The updae sep s gven below. X 1 where and X U 1 1 ( U 1, u N u X 1 1 r,..., N 1 r (4 1 1 Here f U has hgh varance relave o U hen s 1 small hus has lle mpac on value of X. Ths ensures ha updaes whch have more chance of error are facored-n 1 less no our curren belef X. n updae dsrbuon crera : s he adused-varance U and s deermned usng hs nuve Here g s he Gaussan varance of updae U. The crera descrbed n (5, ses he varance o be drecly proporonal o he rao of lengh of occluded boundary of cell o (calculaed a every me-sep o he oal vsble lengh l of he cell boundary. ( g Pedesran Dsrbuon analyss er hus represens snapshos of pdfs for each cell a each me-sep whch gves us a measured dea abou he confdence ha we can place on he pedesran coun n each cell (see Fgure 3. IV. DISCUSSION o ( l Trackng pedesrans a exs and enrances proved o be one of he rckes pars durng he sysem desgn. We know ha he racker oupu grows accurae wh ncrease n he me for whch a arge s observed snce racker ges more chances o updae and propagae s parcles so ha hese can mach arge dynamcs. Thus he places he racker ends o be mos naccurae are he enrances o he observed area where he observed me for enerng arges s lmed. In order o esmae by wha margn our racker fals o rack enerng pedesrans, we performed an expermen by frs measurng he number of pedesrans crossng eas o wes across a lne dvdng he observed area no wo halves. We dd hs because our racker s relavely accurae abou pedesrans n he mddle of observed area snce he racker had enough me o rack hese pedesrans. Then we consdered he same lne as an enrance and ran he racker for he second me on he same (5
8 8 runnng average of error percenage 6 4 runnng average of error percenage 6 4 se of observaons for people enerng n eas o wes drecon consderng updaes only from one half of he observed area and gnorng he res. The dfference beween he numbers of people crossng eas o wes n boh cases provded us wh he bas he racker had n rackng pedesrans near he enrances. We used hs bas b n followng manner o adus he number of people n cells ha are suaed a he enrances: N N b Usng updaed cardnaly as an npu o he second er of our sysem proved o be benefcal n erms of accuracy bu we resraned o declare a formal par of our sysem snce would make edous expermenaon o learn bas, a prerequse for deployng our sysem hus lmng s applcaons. V. RESULTS We esed our sysem n erms of accuracy and compuaonal effcency. In daa collecon phase we manually recorded he pedesran crossngs over ceran epsodes of me observed va laser daa sream for each of he cells. These me-samped recordngs were accurae up o 1 second resoluon and served as our ground ruh. For accuracy measuremen we compued followng wo errors. ( ( N ground _ ruh for exhbs =1 o 7 (Fgure 4a shows a sngle epsode depcng he error for each of he cells. Here error s calculaed usng pre er- measuremen.e. N from er-1. Here he cumulave average counng error for all our observaons for all he exhbs oalled o be 13.8%. ( ( ground _ ruh for exhbs =1 o 7 where s he value wh hghes probably n he pdf represenng X (Fgure 4b shows he same epsode as shown n fg. 4a 1 99 197 95 393 491 589 687 785 883 981 79 1177 175 1373 1471 1569 - me n secs (a Error before Ter- applcaon depcng he error for each of he cells. Ths error s compued usng oupu from er- of our sysem. The average counng error for all our observaons for all he exhbs n hs case sood a 9.83% whch shows marked mprovemen as a resul of applyng er-. 1 99 197 95 393 491 589 687 785 883 981 79 1177 175 1373 1471 1569 - Fg. 4 Sysem counng error comparson me n secs (b Error afer Ter- applcaon By applyng our er approach o laser daa colleced by recordng over 5 hours of museum vsors, we are able o plo locaons of hgh-raffc. Ths s shown n Fgure 5 usng a colour coded scheme n whch red hghlghs reflec he posons ha people spend mos of her me n. In a sense, hs represens he me-averaged dsrbuon from er-. VI. CONCLUSION Technques descrbed n [1], [3], [4] and [6] sress he rackng accuracy. Our effor s focused on rerevng analysable resuls usng fas rackng echnques n order o ge relable pedesran coun n heavly occluded envronmens. Our pedesran deecon and rackng algorhm s exremely compuaonally nensve as s he case wh all oher mulple arge rackng algorhms [7] and hs happens n our case due o compuaons lke ner-cluser, cluser o pedesran dsance calculaon and propagaon of a hgh number of parcles n parcle fler a each me-sep. Durng our expermen phase we were able o produce suffcenly accurae resuls n a more relable forma for scenfc analyss of pedesran dsrbuon n ndoor envronmens. ACKNOWLEDGEMENTS Suppor from Ineracon lab, Unversy of Souhern Calforna (USC s graefully acknowledged. Also suppor from all undergraduae sudens who worked under NSF s Research Experence for Undergraduaes (REU program s apprecaed. Scholarshp gran from Fulbrgh Commsson s acknowledged and apprecaed as funded he research asssanshp for one he auhors. We hank Professor Krsna Lerman from USC Informaon Scences Insue for her consan advce and menorng durng all phases of our research. Lasly hs work was made possble by he movaon gven o us by our ever helpful Professor Maa Maarc.
Fgure 5: Locaons of hgh raffc whn he museum exhb TABLE I COMPUTATIONAL EFFICIENCY FOR VARYING PEDESTRIAN DENSITY (Sysem: Ubunu 8.4, kernel.6.4-18, Inel Penum Moble 17 MHz Processor REFERENCES Frame skp rae Every ou of 3 Every ou of 3 Every ou of 3 Every oher Every oher Every oher None skpped None skpped None skpped Average execuon me for 1 sec of frames Peak densy encounered (people per sq. m Average densy (people per sq. m Average counng error % (error/ruh*.58 sec.33. 11.8.8 sec 1.94.35 11.9.9 sec.7.54 13.6.71 sec.33. 9.7.94 sec 1.94.35 9.4 1.7 sec.7.54. 1.94 sec.33. 8.5.6 sec 1.94.35 8.4 3.1 sec.7.54 9.3 [1] Shao.X, Zhao.H, Nakamura.K, Kaabra.K, Shbasak.R, Deecon and Trackng of Mulple Pedesrans by Usng Laser Range Scanners n 7 IEEE/RSJ Inernaonal Conference on Inellgen Robos and Sysems, Aprl 7. [] Bando.T, Shbaa. T, Doya. K, Ish. S, Swchng Parcle Flers for Effcen Real-me Vsual Trackng n Proceedngs of he 17h Inernaonal Conference on Paern Recognon 4, vol., pp. 7-73, Aug 4. [3] Arras.K, Grzonka.S, Luber.M, Burgard.W, Effcen People Trackng n Laser Range Daa usng a Mul-Hypohess Leg-Tracker wh Adapve Occluson Probables n 8 IEEE Inernaonal Conference on Robocs and Auomaon, pp. 17-1715, May 8. [4] Song.X, Cu.J, Wang.X, Zhao.H, Zha.H, Trackng Ineracng Targes wh Laser Scanner va On-lne Supervsed Learnng n 8 IEEE Inernaonal Conference on Robocs and Auomaon, pp. 71-76, May 8. [5] D. Red, An algorhm for rackng mulple arges, IEEE Transacons on Auomac Conrol, vol. 4, pp. 843 854, Dec 1979. [6] Wang.J, Makhara.Y, Yag.Y, Human Trackng and Segmenaon Suppored by Slhouee-based Ga Recognon n 8 IEEE Inernaonal Conference on Robocs and Auomaon, pp. 1698-173, May 8. [7] Khan.Z, Balch.T, Dellaer.F, MCMC-Based Parcle Flerng for Trackng a Varable Number of Ineracng Targes n IEEE Transacons on Paern Analyss and Machne Inellgence, Vol. 7, Issue. 11, pp. 185-1819, Nov. 5. [8] Thrun, S., Parcle flers n robocs, Proceedngs of he 17h Annual Conference on Uncerany n AI (UAI,. [9] Hollnger.G, Dugash.J, Sngh.S, Trackng a Movng Targe n Cluered Envronmens wh Rangng Rados, n 8 IEEE Inernaonal Conference on Robocs and Auomaon, pp. 143-1435, May 8. [] Hawes, Mchael R., Quanave morphology of he human foo n a Norh Amercan populaon n Ergonomcs, Vol. 37, Issue. 7, pp 113, 1994.