Principal Component Analysis)

Transcription

1 jp Principal Componen Analysis) n p i i,...,, =1,2 x φ x 3 i T i x z φ = φ 2 φ 3 φ 1 x 2 n s n i i T / = = x φ x 1 φ φ φ φ φ ) / ( ) / ( / ) ( n n n s n i T i i T n i T i i T n i T i = = = = = = x x x x x

2 2

3 For each sub image a each scale Objec Exracion from Image Daabase car person person PASCALVO C 2010 person car bike bike Semanic no easy o ge Which objecs ha an image conain? Objecives: Combine muliple feaures o deec as many objecs as possible 5 Objec deecion approach Inpu image Preprocessing sandard image Example of raining Edge/Corner feaure (*) Corner map Saisfy Corner map Se of raining images Saisfy Orienaion SURF, (*) Line, Circle, Color, Edge map Saisfy Edge map (1) (2) Objec (*) Oher feaures are also rained 6 3

4 Edge map and corner map (raining) Corner map Edge map Fig. 2. Making Edge map & Corner map from raining se 7 SURF feaure (aero plane, rain) Corner map Edge map SURF feaure SURF poins Naïve Bayes Calculae SURF descripors k Quanize o k bins (,I) P(Cj I) = P(C j)p(i C j) = P(C j) N P(v C j) = 1 I: inpu image; Cj: class (plane or non plane) v : vocabulary (bin[]); N (,I) :hisogramofv in I Plane Non plane Choose he class : argmax{p(c j I)} 8 4

5 Color feaure (horse, sheep) Inpu image Corner map Edge map None-Horse-color Horse-color Region Region Hue Image in HSV Sauraion HShisogram Training image daabase Color map Color Maching Calculae: M color x, y CM: Color map; H: HS Hisogram of image Check: M color > hreshold = CM( x, y)* H( x, y) 9 Bicycle deecion False posiive Fig. 8. Example of bicycle deecion resul True negaive 10 5

6 Auomaically choose feaures None-Horse-color Region Hue Color map of horse Horse-color Region Sauraion Two sronges lines of rain Line and Circle of bicycle Which feaures are good for a specific objec? Can sysem auomaically choose good feaures or no? chosen feaure range Horse objec peak of saisfied score Saisfied score Posiive score Negaive score no chosen feaure range H1 E9 E5 E3 CR2 S1 E7 C13 C3 E13 E15 E18 E20 C1 C5 C7 C9 C chosen feaure range Side view car objec peak of saisfied score Saisfied score Posiive score Negaive score no chosen feaure range CL1 C6 C1 L8 C12 C14 E4 C11 E3 E5 E7 E9 E11 E13 E15 C3 C5 C7 C9 C13 C15 E1 S2 1 peak of saisfied score Tower objec Saisfied Score Posiive Score Negaive Score 1 peak of saisfied score C6 HoG29 HoG26 CL2 HoG32 E3 E15 HoG13 C5 C11 C17 HoG28 HoG12 HoG31 HoG24 HoG11 HoG27 C2 HoG17 HoG1 HoG23 C18 HoG7 E6 Flower objec 12 6

7 Objec wih auomaically chosen feaures Objec Edge Corner Line Circle HoG SURF Color F/r car Side car Bike Train Aero plane Moorbike Horse Sheep Tower Flower Table 2. Feaures are good for recognizing objec 13 Objec TRAINING STAGE Average Precision Average Recall TESTING STAGE Average Precision Average Recall F/r car 96.48% 90.21% 95.12% 90.14% Side car 97.20% 94.92% 94.21% 91.73% Bike 85.80% 81.32% 84.02% 79.14% Train 87.24% 77.16% 83.65% 75.31% Aero plane 86.65% 84.55% 85.75% 84.51% Moorbike 90.23% 87.32% 89.38% 85.47% Horse 88.93% 82.09% 87.81% 81% 75.40% Sheep 87.32% 75.91% 86.25% 73.24% Tower 89.07% 90.39% 84.33% 82.64% Flower 82.71% 75.15% 81.57% 74.42% Table 3. AP & AR a raining/esing sage of auomaically choosing feaures 7

8 Fig. 9. Example of side view car (op) and plane (boom) deecion resul Objec Evaluaion Our sysem PASCAL VOC 2010 (1) Average Precision Average Precision Auhors Fron/rear car 95.12% Side car 94.21% 49.10% UOCTTI_LSVM_MDPM Bicycle 84.02% 55.30% NLPR_HOGLBP_MC_LCEGCHLC Train 83.65% 50.30% MITUCLA_HIERARCHY Aero plane 85.75% 58.40% UVA_GROUPLOC Moorbike 89.38% 56.30% Horse 87.81% 51.90% NUS_HOGLBP_CTX_CLS_RESCO RE_V2 NUS_HOGLBP_CTX_CLS_RESCO RE_V2 Sheep 86.25% 37.80% UVA_DETMONKEY Tower 84.33% 84.00% HENA_LU_DU (2) Flower 81.57% 80.00% JZU_HONG_CHEN_LI (3) Table 5. Comparison beween our sysem wih PASCAL 10 1 PASCAL 10 hp://pascallin.ecs.soon.ac.uk/challenges/voc/voc2010/resuls/ (abou 20 caegories) 2 Lu Yang, Du Xiao wei, 2 nd Inl Asia Conference on Informaics in Conrol, Auomaion and Roboics 2010, p.p Hong e al. / J Zhejiang Univ SCI (7): hp:// 16 8

9 Conclusion Combine various feaures for objec deecion. Auomaically uo ychoose suiable feaures. es. Fuure works f i f i+1 f i+2 f i (f i+1 f i+2 ) very small objec Selec 17 suiable scale Dynamic Moion Planning for Efficien Visual Navigaion under Uncerainy 9

10 Objecive Safely and quickly reaching a goal posiion Basic assumpion for he firs case Indoor environmen is known Landmarks are given Use sereo vision for localizaion 10

11 Problem Moion and vision include uncerainy. Vision requires high compuaional cos. Uncerainy Moion uncerainy conrol error rolling, slippage and ec. 3σ area of covariance marix Vision uncerainy quanizaion error calibraion error 11

12 Non sop and speed conrolled navigaion sraegy Esimaed uncerainy a -1 Esimaed uncerainy a Prediced uncerainy rajecory X 1 X X +1 Uncerainy afer Uncerainy afer observaion a X observaion a 1 X Speed conrolled moion beween planned viewpoins Non sop navigaion sraegy Esimaed uncerainy a -1 Esimaed uncerainy a Prediced uncerainy rajecory X 1 X X +1 Uncerainy afer Uncerainy afer observaion a X observaion a 1 X Speed conrolled moion beween planned viewpoins 12

13 Case 1: fas bu dangerous Case 2: safe bu slow 13

14 Case 3: adapive observaion Adapive viewpoin planning mehod considering uncerainy Adapive viewpoin 1. Observaion posiion o guaranee he safey 2. Observaion posiion o reach he goal posiion quickly 14

15 15 Idea Safey considering uncerainy Quickly navigaion wih minimum observaion non sop and speed conrolled navigaion Adapi e ie poin planning nder ncerain Adapive viewpoin planning under uncerainy Observaion model 0 O X G = = x L L cx x ) i( ) ( ) cos( ) sin( ), ( φ θ φ θ φ θ φ θ = y x X = cx O + + y L cy y ) sin( ) cos( φ θ φ θ = y θ X = cy O

16 16 Sereo observaion model + r l u u u B(u ) = cy cx O = = r l r l u u u u 2BF ) Z(I O = r l u u I y r Trajecory planning

17 Offline planned viewpoins Dynamic planning mehod Move o he farhes safe posiion on he arge rajecory prediced wors posiion arge rajecory planned rajecory obsacle region esimaed uncerainy planned nex viewpoin prediced uncerainy prediced rajecory consrained by safey 17

18 Dynamic viewpoin planning Move o a posiion o guaranee safey prediced wors posiion arge rajecory esimaed uncerainy planned rajecory planned nex viewpoin prediced uncerainy obsacle region prediced rajecory consrained by safey Moion resul using online planning 18

19 Landmark observaion Verical segmen as landmark Selecion candidaes by posiion and similariy consrains Posiion consrain Lef image Righ image 19

20 Similariy consrain Lef image Righ image Similariy of sereo segmen pair 1. Orienaion similariy 2. Overlap lengh 3. Lengh similariy 1 ( if dir( li ) = dir( rj )) OS( li, rj ) = 0 ( else ) 1 δy δy OL( li, rj ) = [ + ] 2 len( li ) len( rj ) min[ len( li ), len( rj )] LS ( li, rj ) = max[ len( l ), len( r )] i j 20

21 Example of maching Resul of landmark deecion Prediced landmark posiion on image Deeced landmark posiion on image 21

22 Mobile robo sereo CCD camera hos compuer and monior moor for PAN quad-swicher for sereo inpu UPS PAN and robo driving module AC/DC converer seering driving module 22

23 Range sensors Omnidirecional Sereo Laser Range FinderLRF) Generaing probabilisic occupancy maps Represening probabiliies of obsacle exisence in each grid. Omnidirecional sereo LRF 23

24 E Inerpreaion of curren sensor daa : obsacle exiss. O: obsacle is observed. : free space is observed. O ( ) P E O P = P P( O E) P( E) ( O E)() P E + P( O E) P( E ) P ( ) ( O E ) P( E ) E O = P( O E ) P( E ) + P( O E ) P( E ) P( E) : prior probabiliy iniialized o 0.5 P ( O E) ( O E) : observaion models P Map generaion inegraion of wo sensors Inegraion by a logical rule. L R F Omnidirecional sereo obsacle undecidedwo undecidedw free obsacle obsacle obsacle obsacle obsacle undecidedw o obsacle obsacle obsacle obsacle undecidedw obsacle obsacle obsacle free free obsacle undecidedw obsacle : undeeced by a free sensor free undecidedwo : no observed ye 24

25 Formulaion The robo moves a a speed o sufficienly observe an undecided region. N : he number of observaions recognizing a free space d : Disance o undecided region T : Observaion cycle (consan) v : Robo speed d d vt N Maximum speed v = max d NT Experimen (real ime movie) 25

26 Free space maps and planned pahs Moving disance: 30[m] in 45[sec] (maximum speed: 1[m/s]) (wihou speed conrol, 150[sec]) mapping robo working robo 26

27 Scale Invarian Feaure Transform example : n=2 sable keypoin T L R 27

28 r( ) = [ X ( ) Z( ) ( )] T Λ i z c x O c S i w z c w x c O r ( 1) ) 28

29 High Low 29

30 keypoin mapping 0.75 [sec/frame] + 30

31 mapping robo x[mm] z[mm] θ[deg] mm x +100 mm -100 mm z +100 mm - 10 degθ+ 10 deg (x:10mmz:10mmθ:1deg) 0.86 [sec/frame] + Working robo 31

32 x[mm] z[mm] φ[deg] [sec/frame] x[mm] z[mm] φ[deg] [sec/frame] x[mm] z[mm] θ()[deg] 32

33 [sec] ~~ [sec] ~~

34 Move area Z frame X θ powerbo frame X Z θ /10/24 68 Z X powerbo frame X Z θ

35 X X Map P P Ps 2013/10/24 69 re() (x,z,θ) r() (x,z,θ) ( ) r r = r 1) r e ( r r + r ( ) ( 1) = e( ) Map Z YZ X Map r () powerbo powerbo powerbo r e() (x,z,θ)=(0,0,0) Z r () r r e() 2013/10/24 Y X 70 35

36 re() (x,z,θ) r() (x,z,θ) ( ) Y r Map Z X Map powerbo r () powerbo powerbo r e() (x,z,θ)=(0,0,0) 2013/10/24 Y Z X powerbo r () 71 36

37 X P n m 2013/10/24 73 n m P = (1 P ) P Pn Pm 2013/10/

38 2013/10/24 75 Real-ime 3-D hand posure esimaion from 2-D appearance Gesure recogniion for human inerface Inpu image Camera Esimaion resul 38

39 Conour Feaure Exracion r i Sar poin candidae r i i Scale normalized feaure Muliple sar poins are ried i Previous Approaches of Gesure Esimaion Single Image Sequenial Image 3 D Model dlfiing Arbirary posures High compuaion cos Moion consrains can be derived from he model Direc Image Maching Limied posures Low compuaion cos Moion consrain needs o be learned 39

40 Real ime processing experimenal environmen inpu image resul Previous Approaches of Gesure Esimaion Single Image Sequenial Image 3 D Model dlfiing Arbirary posures High compuaion cos Moion consrains can be derived from he model Direc Image Maching Limied posures Low compuaion cos Moion consrain needs o be learned 40

41 Learning of Shape Transiion Efficien Maching Using Transiion Nework Try only possible posures which are reached from he curren posure 41

42 Shape Esimaion Resul Inpu image Mached model Hand Shape Esimaion under Complex Backgrounds for Sign Language Recogniion 42

43 HMM 43

44 44

45 45

46 1 46

47 HMM HMMLef o Righ

48 a a b = a a b = a =0.7 2 a23 = a 0.5 a 0.0 b 0.5 b 1.0 HMM a 11 =0.3 a 22 a b a b = a =0.7 2 a 23 = a 0.5 a 0.0 b 0.5 b

49 RGBDEPTHVierbi a =0.3 a b a a b = a =0.7 2 a 23 = a 0.5 a 0.0 b 0.5 b 2010 Medal of Honor Winner: Andrew J. Vierbi 1.0 RGB

50

51 51