An Auomaic Doo Senso Using Image Pocessing Depamen o Elecical and Eleconic Engineeing Faculy o Engineeing Tooi Univesiy MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC
1. Inoducion Auomaic doo is equipped a he enance o many buildings and inside he buildings. - I opens when pesons appoach o he doo. - vey useul o pesons wih lage luggage & disabled pesons. Vey useul because he doo is auomaically opened. On he ohe hand hee is anohe aim Regulaed Sociey o he building Keep Secuiy o he building and Save he ai-condiioning cos Auomaic doo should be closed as long as possible. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 2
Many kinds o sensos inaed ay senso & supesonic senso applied o he auomaic doo someime ail o deec appoaching obec. Pesons wihou will o ene he doo Snow ain o alling leaves NOREN Japanese usual clohes hang in on o enance o shops To solve hese diiculies Auomaic doo senso sysem using image pocessing Coninual images In on o he doo is aken by he CCD camea. Image pocessing Open and close conol As a is sep we develop a echnique o classiy he cuen image. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 3
2. Open and Close Conol Sysem o Auomaic Doo Using Image Pocessing CCD Camea Daa CCD camea 2-3[m] Deph Requiemens o he CCD camea Deph :10 [m] Nea side widh:20 [m] In ode o ge his shooing aea necessay angle anges o he CCD camea insalled a a posiion 3.0 [m] high is: Shooing Aea Hoizonal angle ange: 147.2[deg] Veical angle ange : 73.3 [deg] Fig.2.1 Shooing aea and is measue. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 4
2. Open and Close Conol Sysem o Auomaic Doo Using Image Pocessing 2-3[m] CCD camea Unounaely he camea we could acquied has poo spec as ollows. Then an expeimen mus be done by using his camea. Deph Angle anges o ou CCD camea Hoizonal angle ange : 42.4 [deg] Veical angle ange : 32.0 [deg] Shooing Aea Then we insalled his camea a a posiion o 3.2 [m] high o expeimens o his eseach. Fig.2.1 Shooing aea and is measue. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 5
Fig.2.2 An example image aken by he CCD camea insalled a a posiion 3.2[m] high. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 6
The classiicaion based on he nomalized coelaion maching The opical low esimaion equies long compuaion ime. The low only in paial egions including moving obec should be compued. The sysem classiies he cuen image a each paial egion ino conaining moving obecs. conaining a saic obec. conaining he backgound image only. Classiicaion Based on he nomalized coelaion maching value beween he backgound image & he cuen ame image and ohe 3 paamees. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 7
The pocessing low START iniial pocess obain he cuen image speciy paial egion R In he iniial pocess paial egions ae deined. The iniial image is acquied i is used o he iniial backgound image classiicaion i is applied wih he gay-scaling pocess and i is divided ino he paial egions. compue C D ΔC and ΔD ΔC H 1 D ΔD H 2 C<H 3 moving obec opical low esimaion saic obec deec size o block backgound image modiy backgound image all paial egion? open & close conol inishing condiion END Fig.2.3 A pocessing low o he classiicaion. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 8
The pocessing low START iniial pocess obain he cuen image speciy paial egion R The cuen image aken by he CCD camea is also divided ino he classiicaion paial egions ae he gay-scaling pocess. compue C D ΔC and ΔD ΔC H 1 D ΔD H 2 C<H 3 moving obec opical low esimaion saic obec deec size o block backgound image modiy backgound image all paial egion? open & close conol inishing condiion END Fig.2.3 A pocessing low o he classiicaion. This loop is execued wih obaining he cuen image. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 9
The pocessing low START iniial pocess obain he cuen image A one o paial egion he nomalized Fis one o paial egion is coelaion maching C he epesenaive speciied. gay scale value D and hei ime vaiaions ΔC and ΔD ae compued. classiicaion speciy paial egion R compue C D ΔC and ΔD ΔC H 1 D ΔD H 2 C<H 3 moving obec opical low esimaion saic obec deec size o block backgound image modiy backgound image all paial egion? In his classiicaion pocess he cuen image a each paial egion By means o hese 4 open paamees & close conol he cuen is image classiied ino hee ypes. o he paial egion is classiied as inishing condiion conaining moving obec conaining saic obec o END Fig.2.3 A pocessing low o he classiicaion. conaining backgound image only. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 10
The pocessing low START iniial pocess obain he cuen image classiicaion speciy paial egion R compue C D ΔC and ΔD ΔC H 1 D ΔD H 2 C<H 3 moving obec opical low esimaion saic obec deec size o block backgound image modiy backgound image all paial egion? open & close conol inishing condiion Ae classiicaion appopiae pocess o each ype is execued especively. These pocesses ae shown lae in deail. END Fig.2.3 A pocessing low o he classiicaion. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 11
The pocessing low START iniial pocess obain he cuen image classiicaion speciy paial egion R compue C D ΔC and ΔD ΔC H 1 D ΔD H 2 C<H 3 moving obec opical low esimaion saic obec deec size o block backgound image modiy backgound image all paial egion? open & close conol inishing condiion END Fig.2.3 A pocessing low o he classiicaion. Ae he cuen image is pocessed o all paial egions opical low esimaion is compued only a paial egions conaining moving obec. And hen open and close conol is execued. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 12
Deiniion o paial egions Ignoed egion Each paial egion is omed as a seco deined by he pola coodinae sysem. The ange on he agumen is equally divided ino ive pas and he deph on he adius is equally divided ino ou pas. Toally 18 paial egions ae deined. Auomaic Doo Fig.2.4 Deiniion o he paial egions. Images o he auomaic doo neighbohood MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 13
Gay-scaling pocess Fo he image pocessing he oiginal colo image is conveed o he gay-scale image. P i 0.299 P i 0.587 P i 0.114 P i Q R G R i 0.299 Q i 0.587 Q i 0.114 Q i R G R 1 2 i :posiion o image pixel P :gayscale value o he backgound image PR PG PB :colo elemens o he backgound image Q :gayscale value o he cuen image QR QG QB:colo elemens o he cuen image Fig.2.5 Gay-scale pocess. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 14
MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 15 R l L K k Fou paamees 4 : Time vaiaion o he value o D D 1 : The nomalized coelaion maching C 3 : Time vaiaion o he value o C C 2 : The epesenaive gay scale value D Fig.2.6 Deiniion o he paial egions. R R i R i R i i Q i P i Q i P C 2 2 3 R i i Q D 2 4 1 C C C 5 1 D D D 6 :an index o he cuen ame.
Classiicaion 1 When one o hose wo condiions is saisied he paial egion is decided as conaining moving obecs. Condiion 1 C H D 1 H 1 D: A heshold uncion o D deined by H1 D D :consan values. 7. 8 Condiion 2 H 2 D H : A consan heshold value. 2 9 In his egion he diecion o he moving obec is deemined by he opical low esimaion. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 16
Classiicaion 2 When he pevious condiions abou he moving obec ae no saisied and when he condiion 3 is saisied he paial egion is decided as conaining a saic obec. C H Condiion 3 3 H 3 : A consan heshold value. 10 This paial egion is oined o ohe paial egions similaly conaining he saic obec aound his aending egion. The size o a block o hese paial egions denoes he size o he saic obec. Especially i he saic obec is deeced a paial egions igh in on o he auomaic doo The saic obec is consideed as a peson sanding sill a he posiion. The doo mus be always open while his siuaion. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 17
Classiicaion 3 When he condiions abou moving obec and abou saic obec ae boh no saisied he paial egion is decided as conaining he backgound image only. I his siuaion coninues o a consan numbe o ames an image giving he maximum C among he ames is subsiued o he backgound image a he paial egion. By means o his echnique he sysem can always soe he exac backgound image adaping o gay scale change accoding o weahe change and ime pogess. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 18
MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 19 good accuacy sho compuaion ime. 3. Deecion o Moving Obec Diecion by Opical Flow Esimaion We have adoped he Lucas-Kanade mehod. Alhough vaious mehods o opical low esimaion has been poposed R y x R y x y y x Q y x Q y y x Q v 2 R y x R y x x y x Q y x Q x y x Q u 2 11 12 : x elemen o he opical low esimaion a he paial egion R. : y elemen o he opical low esimaion a he paial egion R. u v
MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 20 R y x R y x Q i Q i Q i Q i Q i Q i u 2 1 1 1 R y x R y x Q i Q i Q i Q i Q i Q i v 2 1 1 1 13 14 In he compue pogam he opical low esimaion is compued in he discee coodinae sysem. Equaion 11 Equaion 12
4. Classiicaion expeimen Expeimen Condiions CCD camea CCD-CAMI-O DATA Hoizonal ange: 42.4[deg] Veical ange : 32.0[deg] Compue DynabookTOSHIBA OS :Micoso Windows Me CPU :celeon 640[MHz] Fig.4.1 Sample images o his expeimen. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 21
Case1 The paial egion R 00 is an example which iniially conains he backgound image only and ae ha conains moving obec. R 00 Fig.4.2 Posiion o paial egion R 00. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 22
he nomalized he ime vaiaion o C ΔC he ime vaiaion o D ΔD he epesenaive gay scale value D coelaion macing C 0.05 0.04 0.03 0.02 0.01 0-0.01-0.02-0.03-0.04-0.05 1 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 ame numbe A ames 6 and A ohe 7 his ames egion his can egion be classiied can be classiied as conaining as moving conaining obec. backgound image only. 2 1.6 1.2 0.8 0.4 0 ame numbe a he nomalized coelaion maching C. b he epesenaive gay scale value D ame numbe 0.3 0.2 0.1 0-0.1-0.2-0.3 ame numbe c he ime vaiaion o C Δ C. d he ime vaiaion o D Δ D. Fig.4. 3 Gaphs o C D Δ C and Δ D o each ame a he paial egion R 00. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 23
Case2 The paial egion R 01 is also same example. Bu he moving obec aken in he egion is oo small. R 01 Fig.4.4 Posiion o paial egion R 01. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 24
he nomalized coelaion macing C 1 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 ame numbe he epesenaive gay scale value D 2 1.6 1.2 0.8 0.4 0 ame numbe a he nomalized coelaion maching C. b he epesenaive gay scale value D he ime vaiaion o C ΔC 0.05 0.04 0.03 0.02 0.01 0-0.01-0.02-0.03-0.04-0.05 ame numbe he ime vaiaion o D ΔD 0.3 0.2 0.1 0-0.1-0.2-0.3 ame numbe c he ime vaiaion o C ΔC. d he ime vaiaion o D ΔD. Fig.4.5 Gaphs o C D ΔC andδd o each ame a he paial egion R 01. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 25
Case3 The paial egion R 02 is an example which conains he backgound image only. R 02 Fig.4.6 Posiion o paial egion R 02. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 26
he nomalized coelaion macing C 1 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 ame numbe he epesenaive gay scale value D 2 1.6 1.2 0.8 0.4 0 ame numbe a he nomalized coelaion maching C. b he epesenaive gay scale value D he ime vaiaion o C ΔC 0.05 0.04 0.03 0.02 0.01 0-0.01-0.02-0.03-0.04-0.05 ame numbe he ime vaiaion o D ΔD 0.3 0.2 0.1 0-0.1-0.2-0.3 ame numbe c he ime vaiaion o C ΔC. d he ime vaiaion o D ΔD. Fig.4.7 Gaphs o C D ΔC andδd o each ame a he paial egion R 02. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 27
Case4 The paial egion R 22 is an example which iniially conains he backgound image only and hen conains moving obec and ae ha conains he backgound image only. Paial egions such as his case R 22 Fig.4.8 Posiion o paial egion R 22. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 28
he nomalized coelaion macing C 1 0.99 0.98 0.97 0.96 0.95 0.94 0.93 0.92 0.91 ame numbe he epesenaive gay scal value D 2 1.6 1.2 0.8 0.4 0 ame numbe a he nomalized coelaion maching C. b he epesenaive gay scale value D he ime vaiaion o C ΔC 0.05 0.04 0.03 0.02 0.01 0-0.01-0.02-0.03-0.04-0.05 ame numbe he ime vaiaion o D ΔD 0.3 0.2 0.1 0-0.1-0.2-0.3 d ame numbe c he ime vaiaion o C ΔC. d he ime vaiaion o D ΔD. Fig.4.9 Gaphs o C D ΔC andδd o each ame a he paial egion R 22. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 29
Case 5 The paial egion R 32 is an example which iniially conains he moving obec and hen conains he backgound image only. R 32 Fig.4.10 Posiion o paial egion R 32. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 30
he nomalized coelaion macing C 1 0.98 0.96 0.94 0.92 0.9 ame numbe he epesenaive gay scale value D 2 1.6 1.2 0.8 0.4 0 ame numbe a he nomalized coelaion maching C. b he epesenaive gay scale value D he ime vaiaion o C ΔC 0.05 0.04 0.03 0.02 0.01 0-0.01-0.02-0.03-0.04-0.05 ame numbe he ime vaiaion o D ΔD 0.3 0.2 0.1 0-0.1-0.2-0.3 ame numbe c he ime vaiaion o C ΔC. d he ime vaiaion o D ΔD. Fig.4.11 Gaphs o C D ΔC andδd o each ame a he paial egion R 32. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 31
Consideaion om he expeimen Fom hese expeimens he heshold value H 3 should be se as ollows. 0.978 H 3 0.980 15 When small pa o he moving obec is aken in he paial egion C does no disincly change especially o lage sized paial egions. Theeoe dieen heshold is equied accoding o he size o egion. The ime vaiaion C o he moving obec. is eecive o ecogniion o exisence We could no ined clea eeciveness o paamees D and D. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 32
5. Conclusion We ied a echnique o classiy he paial egions o he open-close conol sysem by using image pocessing. The CCD camea akes coninual images. The coninual images ae divided ino he paial egions. The paial egion is classiied ino 3 ypes by means o 4 paamees. Expeimen o he classiicaion is illusaed. I is claiied ha he nomalized coelaion maching and is ime vaiaion ae eecive o he classiicaion. Eeciveness o he epesenaive gay scale value and is ime vaiaion could no ound. In he uue 1. We will ind eecive paamees o he classiicaion and develop he es o he sysem. 2. We will esablish a new echnique o compue he opical low esimaion by using GA which we ae doing eseach on. 3. A lens wih wide ange is also consideed o be insalled on he CCD camea o he sysems. MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 33
Tha s all! Thank you! MENDEL 2004 -Insiue o Auomaion and Compue Science- in BRNO CZECH REPUBLIC 34