Deecon of Wavng Hands from Images Usng Tme eres of Inensy Values Koa IRIE, Kazunor UMEDA Chuo Unversy, Tokyo, Japan re@sensor.mech.chuo-u.ac.jp, umeda@mech.chuo-u.ac.jp Absrac Ths paper proposes a mehod of deecng wavng hands from mages as a mehod for man-machne nerface. FFT s appled o me seres of nensy mages. The mages are convered o low-resoluon ones, and FFT s appled o each pxel of he low-resoluon mages. The proposed mehod s robus o lghng condon and ndvdual dfference of skn color, because doesn use color nformaon a all. Expermens show he sably and robusness of he proposed mehod. Key Words: Image Processng, Gesure Recognon, FFT, Low-Resoluon Image, Man-Machne Inerface 2. Cyclc change of nensy values of mages caused by hand wavng Hand wavng s descrbed as he horzonal cyclc moon of a hand, wh he frequency of usually 3 or 4Hz. When a hand s waved, he nensy value of a pxel correspondng o he hand regon vbraes beween hand regon and background. As a pre-processng, we make he resoluon of he mage lower. By hs process, he paern of he vbraon s smoohed as shown n Fg., and addonally, he robusness for noses s acqured and calculaon cos s reduced.. Inroducon o as o realze man-machne nerface ha s naural for an operaor, s mporan o recognze he exsence of he operaor and hs/her nenon of operaon. Hand wavng s ofen used for man-man nerface o communcae one s nenon o oher person, and hus s hough o be effcen for he purpose. There are several sudes ha deal wh deecng wavng objecs[][2]. For deecon of hand regons, color nformaon s usually ulzed[3][4][5]. However, hs mehodology s sensve o lghng condon and ndvdual dfference of skn color, because s ndspensable o exrac he skn color from mages. In hs paper, we propose a mehod o recognze wavng hands from mages ha doesn ulze color nformaon. FFT(Fas Fourer Transform)[6] s appled o me seres of nensy mages. The mages are convered o low-resoluon ones, and FFT s appled o each pxel of he low-resoluon mages. The proposed mehod s robus o lghng condon and ndvdual dfference of skn color, because doesn use color nformaon a all. Addonally, he mehod s very smple, because doesn requre mage processng o recognze hand regons. Fg. Cyclc change of nensy values by hand wavng The exen of converng an mage o low-resoluon s evaluaed as follows. uppose he dsance o he hand s L[m], he wdh of hand wavng s L H [m], he horzonal angle of he camera s θ[rad], and number of horzonal pxels of he obaned mage s a. Then he wdh of hand wavng n he mage H[pxel] (see Fg.2) s obaned by alh H =. () θ L 2 an 2 I s necessary ha wdh of hand wavng n he low-resoluon mage s roughly larger han one pxel. Therefore, H>P lm should be sasfed, where P lm s he
number of horzonal pxels assgned o he pxel of he low-resoluon mage. Fg.2 Wdh of hand wavng 3. Applcaon of FFT o me seres of nensy values (a)inpu mage As menoned n Chaper 2, each mage s convered o low-resoluon, and me seres of low-resoluon mages are obaned. uppose he number of pxels of he mages s m n, and I(,j,) s he nensy value of (,j) pxel (=,2,,m, j=,2,,n) of -h frame, as shown n Fg.3. (b)low-resoluon mage Fg.3 Tme seres of low-resoluon mages Fg.4 llusraes he applcaon of FFT o low-resoluon mages. Orgnal mage Fg.4(a) s convered o he low-resoluon mage Fg.4(b). The pxels n he recangle of Fg.4(b) correspond o he regon of he wavng hand. The nensy value I(,j,) of hese pxels changes as llusraed n Fg.4(c), snce he rae of he hand and he background changes perodcally, accordng o he wavng of he hand. As hs change of nensy value s perodc wh a consan cycle, we can ulze FFT for quanfyng. We apply FFT o nensy values of every pxels I(,j,), and deec wavng hands from he specrum, llusraed as Fg.4(d). To remove he effec of noses lke flcker of fluorescen lgh and reduce he calculaon cos, FFT s appled o he pxels ha sasfe I(, j, + ) I(, j, ). (2) I df (c)tme seres of nensy value (d) Power specrum Fg.4 Applcaon of FFT o me seres of nensy values
4. The mehod of recognzng wavng hands 4.. Feaures Feaures are exraced from he power specrum obaned from he me seres of nensy values. We ulze he wo feaures: he maxmum value G max and he mean value of he power of he specrum. G max and are gven by eq.(3) and eq.(4) respecvely, where s he samplng number and W s he wddle facor of DFT(Dscree Fourer Transform). max(f) represens he maxmum value of f. = G max max I k W (3) nk k= 2 2 = G (4) = 4.2. Recognon by dscrmnan analyss The lnear dscrmnan mehod[7] s appled o he feaure space of G max and for recognon of wavng hands. uppose he feaure vecor s x=[g max,], he class of wavng hand s ω, he class of oher moons s ω 2, and average vecor of each class s m,m 2. Then he scaer marces of each class and 2 are defned by eq.(5). ( x m )( x m ) (5) x χ Usng all feaure vecors of wo classes, he whn-class scaer marx and he beween-class scaer marx are defned by eq.(6) and eq.(7) respecvely, where n s he number of samples of ω and m s he average vecor of every sample. + = ( x m )( x m (6) W 2 ) =,2 x χ ( m m)( m ) =,2 B n m (7) The lnear dscrmnan funcon g(x) s gven by g( x ) = A x + a0, A = W ( m m 2 ). (8) To defne he hreshold a 0 n eq.(8), we selec he mehod o dvde nernally wh he sandard devaon of each class[8]. Therefore, m σ 2 + m 2σ a0 = m, (9) + m2 where m and σ are he average and he sandard devaon of pons n G max and space projeced on normal vecor of decson boundary lne. When g( x) < 0 ( x ω) for eq.(8), he pxel s regarded as correspondng o he wavng hand. o as o make he recognon more robus, he deecon of hand wavng s done when g(x)<0 connues for several frames. I s formulaed as eq.(0) and (). f g(x) <0 hen D = else D =0 (0) ar Conver an mage o low-resoluon Oban nensy value of each pxel ecessary samplng number? Change of nensy value I df FFT o sequence of nensy values Exracon of Gmax and Dscrmnan analyss J= Wavng hand s deeced End he calculaon for all pxels? Fg.5 Flow char of deecng wavng hands c = + J D k, f J= hen hand wavng () k = Fg.5 shows he flow of he recognon of wavng hands descrbed n Chaper 2 o 4. e ha FFT s appled o every pxel (excep for he pxel wh consan value) of low-resoluon mage, ndvdually. 5. Expermens of deecng wavng hands In hs Chaper, we show some expermenal resuls o show he effecveness of he proposed mehod o deec wavng hands. Every calculaon, ncludng FFT for every pxel and recognon, s performed by a PC (Penum IV.4GHz). For npung mages and converng he mages o low-resoluon, we used an mage board PcPor Color(Leuron Vson) and an mage processng sofware HALCO(MVTec). As a CCD camera, we used EVI-G20 (OY) ha has he funcon of pan-l rackng. Therefore, we can pan and l he camera so as o move he deeced wavng hand a he cener of he mage.
The number of samplng was se o 6. I n eq.(2) was se o 5[pxel]. The samplng perod was abou 80[ms]. 5.. Decson of mage resoluon The resoluon of he mages was decded by eq.(). The parameers are, a=640[pxel], θ π/4[rad], and H l =0.3[m]. We se he maxmum measuremen dsance o 8[m]. Then H becomes 29[pxel]. Therefore, we se P lm o 25[pxel] ha s less han H, and 25 25 pxels of he orgnal mage was assgned o one pxel of he low-resoluon mage. As a resul, he number of he pxels of he low-resoluon mage was se o 25 9[pxel]. 5.2. Decson of he lnear dscrmnan funcon The cluser of hand wavng ω and he cluser of oher moons ω 2 were formed by expermens. The oher moons were varous random moons, e.g., walkng randomly n he room. Fg.6 and Fg.7 show he dsrbuon of G max - for ω and ω 2 respecvely. The dsance was se from 3 o 8[m]. The number of daa for ω and ω 2 was abou 2500. Whn-class scaer marx W and average of paern m of eq.(6) were obaned as follows. W G = G max, G max max, G max,, 296.3 = 466.3 466.3 230.2 G 24.6 max G 6.5 max m = =, m 2 = = 75.6 48.6 Then he marx A n eq.(8) became 0.02657 A = 0.044 and a 0 n eq.(9) became 0.43. Consequenly, he lnear dscrmnan funcon was gven as 0.02657 Gmax g( x ) = 0.43 0.044. (2) Here we evaluae he obaned g(x). We defne he rae of dscrmnan error p as nw p =. (3) n where n s he number of samples n cluser and nw s he number of dscrmnan error. The raes for he daa n Fg.6 and Fg.7 are, p =4.20% (ype error: recognzng he pxel for hand wavng as for oher moon), and p 2 =2.28% (ype 2 error: recognzng he pxel for oher moon as for hand wavng). The error raes are prey small, and addonally, he rae can be mproved by consderng he seres of g(x) as descrbed n econ 4.2. 400 350 300 250 200 50 00 50 0 400 350 300 250 200 50 00 50 0 0 00 200 300 400 500 600 Gmax Fg.6 Dsrbuon of wavng hand: ω 0 00 200 300 400 500 600 Gmax Fg.7 Dsrbuon of oher moons: ω 2 5.3 Deecon of wavng hands 5.3.. Recognon rae of hand wavng Expermens were performed for 5 subjecs, by changng dsance and lghng condon. Fluorescen lghs were used for lghng. The llumnaon around he hand was 60-90[lux] (condon : dark), and 250-35[lux] (condon 2: brgh). The execued moon was as follows. () Wavng a hand a arbrary posons n he camera angle for abou wo seconds, (2) uspendng he hand wavng for abou wo seconds, and hen nex wavng. The moon was repeaed for 20 mes. When he hand wavng was deeced n wo seconds, we regarded he recognon of hand wagng was successful. Table shows he expermenal resuls. I s shown ha prey hgh recognon rae s realzed for he range of 4-8[m]. When he dsance becomes larger (7,8[m]), he wdh of hand wavng becomes small and recognon rae becomes lower. Addonally, s shown ha he resuls are beer for condon, darker one. Ths raher srange phenomenon s caused by he dfference of llumnaon beween hand regon and background. A he darker condon, he background wall was much darker, 30-50[lux], and he dfference of nensy beween hand regon and background was larger. On he conrary, he background wall a he brgher condon was
Fg.8 Dsance: 4m, Lghng: Brgh Fg.9 Dsance: 4m, Lghng: Dark Fg.0 Dsance: 8m, Lghng: Brgh 230-300[lux], and he dfference of nensy was smaller. Fg.8- shows he example of deecon of a hand wavng. Table Recognon rae of hand wavng Dsance 60-90 lux 250-35 lux 4m 99% 96% 5m 00% 96% 6m 00% 97% 7m 96% 92% 8m 9% 83% 5.3.2. Expermens for recognon error o as o evaluae he robusness of he proposed mehod, we made expermens for recognon error. One person made varous moons excep hand wavng, e.g., walkng randomly n he room. The dsance vared from o 8[m]. The moons were performed connuously for 600 seconds. In hs expermens, he recognon error dd no occur a all. The flcker of he fluorescen lgh dd no affec he resuls. Ths resul ndcaes he robusness of he proposed mehod. 6. Conclusons We proposed a mehod o deec wavng hands, whch doesn requre color nformaon and s robus for llumnaon. FFT s appled o me seres of nensy values for low-resoluon mages. The calculaon cos s no hgh, and he mehod s applcable for praccal use. Fg. Dsance: 8m, Lghng: Dark Because FFT can be consruced by hardware, s possble o make he calculaon cos much lower. Expermens show he sably and robusness of he proposed mehod. Fuure works nclude more expermenal evaluaon, and he mprovemen as he problem of paern recognon. References [] R. Culer, L.Davs, Vew-based Deecon and Analyss of Perodc Moon, Proc. of he Inernaonal Conference on Paern Recognon, 998. [2] R. Culer, L.. Davs, Robus Real-Tme Perodc Moon Deecon, Analyss, and Applcaons, IEEE Transacons on Paern Analyss and Machne Inellgence, Vol. 22,. 8, pp. 78-796, 2000. [3] J. herrah and. Gong, VIGOUR: A ysem for Trackng and Recognon of Mulple People and her Acves, Proc. of he Inernaonal Conference on Paern Recognon, 2000. [4] P. Hong, M.Turk, T.. Huang, Gesure Modelng and Recognon Usng Fne ae Machnes, IEEE In. Conf. on Auomac Face and Gesure Recognon, 2000. [5] H. Wu, T. hoyama, and H. Kobayash, pong Recognon of Head Gesures from Color Image eres, Proc. of he Inernaonal Conference on Paern Recognon, pp.83-85, 998. [6] W. Press,. Teukolsky, W. Veerlng, and B. Flannery, umercal Recpes n C, Cambrdge Unversy Press, 988. [7] Duda, R.O. and Har, P.E. Paern Classfcaon and cene Analyss, John Wley & ons, 973. [8] Fukunaga K., Inroducon o ascal Paern Recognon (2nd ed.), Academc Press, 990.