Rejection Based Face Detection

Transcription

1 Rejection Based Face Detection Michael Elad* Scientific Computing and Computational Mathematics Stanfod Univesit The Compute Vision Wokshop Mach 18 th, 00 * Collaboation with Y. Hel-O and R. Keshet 1

2 Pat 1 Intoduction of the Poblem and Basic Consideations

3 Pat 1 1. The Task Input image Face (taget) Detecto Comments: 1. Etension to colo. Faces vs. geneal tagets 3

4 Pat 1. Requiements Detect fontal & vetical faces: All spatial position, all scales An peson, an epession Robust to illumination conditions Old/oung, man/woman, hai, glasses. Design Goals: Fast algoithm Accuate (False Positive / False Negative) 4

5 Pat 1 3. Fontal-Vetical Faces Taken fom the ORL Database 5

6 Pat 1 4. Scale-Position Input Image Suspected Face Positions Compose a pamid with 1:f esolution atio (f=1.) Daw L*L blocks fom each location and in each esolution lae Classifie Face Finde 6

7 Pat 1 5. Classifie Design A classifie is a paametic (J paametes) function C(Z,θ) of the fom C L J + { Z, θ} : R R { 1, 1} Need to answe two questions: Q1: What paametic fom to use? Linea o nonlinea? What kind of non-linea? Etc. Q: Having chosen the paametic fom, how do we find appopiate set of paametes θ? 7

8 Pat 1 6. Algoithm Compleit Seaching faces in a given scale, fo a 1000 b 000 piels image, the classifie is applied e6 times (Q1) Choosing the paametic fom: keep in mind that the algoithm s compleit is govened b the classifie compleit 8

9 Pat 1 7. Taining b Eamples N N >> N k = 1 { X k } k { } = 1 Y k (Q) Finding Suitable Paametes: 1 k N, C = k N X Y, C { X, θ} k { Y, θ} = 1 k 9

10 Pat 1 8. Geometic Intepetation C(Z,θ) is to dawing a sepaating manifold between the two classes N X k k= 1 { } X +1-1 L R N Y k k= 1 { } Y 10

11 Pat SOME Pevious Wok 11

12 Pat 1. Neual Netwoks Choose C(Z,θ) to be a Neual Netwok (NN). Add pio knowledge in ode to: Contol the stuctue of the net, Choose the pope kind (RBF?), Pe-condition the data (clusteing) Repesentative Pevious Wok: Juel & Mach (1996), and Rowle & Kanade (1998), and Sung & Poggio (1998). NN leads to a Comple Classifie 1

13 Pat. Suppot Vecto Machine Choose C(Z,θ) to be a based on SVM. Add pio knowledge in ode to: Pune the suppot vectos, Choose the pope kind (RBF, Polnomial?), Pe-condition the data (clusteing) Repesentative Pevious Wok: Osuna, Feund, & Giosi (1997), Bassiou et.al.(1998), Teillon et. al. (000). SVM leads to a Comple Classifie 13

14 Pat 3. Rejection Based Build C(Z,θ) as a combination of weak (simple to design and activate) classifies. Appl the weak classifies sequentiall while ejecting non-faces. Repesentative Pevious Wok: Rowle & Kanade (1998) Elad, Hel-O, & Keshet (1998), Amit, Geman & Jedank (1998), Osdachi, Gotsman & Keen (001), and Fast (and Viola & Jones (001). accuate) classifie 14

15 Pat 4. The Rejection Idea Input Blocks Rejected Rejected Detected Rejected Weak Classifie # 1 Weak Classifie # n Weak Classifie # Classifie Rejected Weak Classifie # 3 Weak Classifie # 4 Rejected 15

16 Pat 5. Suppoting Theo (Ada) Boosting Feund & Schapie ( ) Using a goup of weak classifies in ode to design a successful comple classifie. Decision-Tee Tee stuctued classification (the ejection appoach hee is a simple dadic tee). Rejection Naa & Bake (1995) - Application of ejection while appling the sequence of weak classifies. Maimal Rejection Elad, Hel-O & Keshet (1998) Geed appoach towads ejection. 16

17 Pat 3 Maimal Rejection Classification 17

18 Pat 3 1. Linea Classification (LC) We popose LC as ou weak classifie: { } { } T C Z, θ = sign Z θ θ N X k k= 1 { } X L R Hpeplane N Y k k= 1 { } Y 18

19 Pat 3. Maimal Rejection Find θ 1 and two decision levels d,d 1 1 such that the numbe of ejected non-faces is maimized while finding all faces Non-Faces N { } Y Y k k= 1 Pojected onto θ 1 Faces N { } X X k k= 1 d d 1 Rejected non-faces 19

20 Pat 3 3. Iteations Rejected points Taking ONLY the emaining non-faces: Find θ and two decision levels d,d 1 such that the numbe of ejected non-faces is maimized while finding all faces d 1 Pojected onto θ 1 d Pojected onto θ 0

21 Pat 3 4. Maimizing Rejection N X k k= 1 { } X N Y k k= 1 { } Y Maimal Rejection Maimal distance between these two PDF s We need a measue fo this distance which will be appopiate and eas to use 1

22 Pat 3 5. One Sided Distance Define a distance between a point and a PDF b ( α α) 0 1{ 0 ( )} α ( α0 m) + = ( ) D α,p α = P α dα α 0 P ( α) α { ( ) ( )} { } D P α,p α = D α,p( α) P ( α)dα = 1 α + + (m m ) This distance is asmmetic!! It descibes the aveage distance between points of Y to the X-PDF, P X (α).

23 Pat 3 6. Final Measue (m { ( ) ( )} m ) + + (m m ) α α = + D P,P P(Y) P(X) In the case of face detection in images we have P(X)<<P(Y) f {} θ = T We Should Maimize (GEP) { T } MX MY MX MY RX RY θ + + θ θ T R X θ 3

24 7. Diffeent Method 1 f Maimize the following function: {} θ = NY NX T T Xk Y j j1k1 = = θ θ NX NX T T θ Xk θ X j j1k1 = = Maimize the distance between all the pais of [face, non-face] T C θ R = θ = T θ Qθ The same Epession Minimize the distance between all the pais of [face, face] 4

25 8. Diffeent Method N X k k= 1 { } X N Y k k= 1 { } Y If the two PDF s ae assumed Gaussians, thei KL distance is given b (m { } m ) + + KL D P,P = + ln + 1 And we get a simila epession 5

26 Pat 3 9. Limitations The disciminated zone is a paallelogam. Thus, if the faces set is non-conve *, zeo false alam discimination is impossible Solution: Second lae. Even if the faces-set is conve, convegence to zeo falsealams is not guaanteed. Solution: Clusteing the nonfaces. * Moe accuatel, if in the conve hull of the face set thee ae non-faces 6

27 Pat 3 8. Conveit? Can we assume that the Faces set is conve? - We ae dealing with fontal and vetical faces onl - We ae dealing with a low-esolution epesentation of the faces - Ae the an non-faces that ae conve combination of faces? 7

28 Chapte 4 Results & Conclusions 8

29 Pat 4 1. Details Kenels fo finding faces (15 15) and ees (7 15). Seaching fo ees and faces sequentiall - ve efficient! Face DB: 04 images of 40 people (ORL-DB afte some sceening). Each image is also otated ±5 and veticall flipped - to poduce 14 Face images. Non-Face DB: 54 images - All the possible positions in all esolution laes and veticall flipped - about non-face images. Coe MRC applied (no second lae, no clusteing). 9

30 Pat 4. Results - 1 Out of 44 faces, 10 faces ae undetected, and 1 false alam (the undetected faces ae cicled - the ae eithe otated o stongl shadowed) 30

31 Pat 4 3. Results - All faces detected with no false alams 31

32 Pat 4 4. Results - 3 All faces detected with 1 false alam (looking close, this false alam can be consideed as face) 3

33 Pat 4 5. Moe Details A set of 15 kenels - the fist tpicall emoves about 90% of the piels fom futhe consideation. Othe kenels give a ejection of 50%. The algoithm equies slightl moe that one convolution of the image (pe each esolution lae). Compaed to state-of-the-at esults: Accuac Simila to (slightl infeio in FA) to Rowle and Viola. Speed Simila to Viola much faste (facto of ~10) compaed to Rowle. 33

34 Pat 4 6.Conclusions Rejection-based classification - effective and accuate. Basic idea goup of weak classifies applied sequentiall followed each b ejection decision. Theo Boosting, Decision tee, Rejection based classification, and MRC. The Maimal-Rejection Classification (MRC): Fast in close to one convolution we get face detection, Simple eas to tain, appl, debug, maintain, and etend. Modula to match hadwae/time constaints. Limitations Can be ovecome. Moe details 34

35 35

36 36

37 7. Moe Topics 1. Wh scale-invaiant measue?. How we got the final distance epession? 3. Relation of the MRC to Fishe Linea Disciminant 4. Stuctue of the algoithm 5. Numbe of convolutions pe piel 6. Using colo 7. Etending to D otated faces 8. Etension to 3D otated faces 9. Relevanc to taget detection 10. Additional ingedients fo bette pefomance 37

38 1. Scale-Invaiant ( α α) 0 1{ 0 ( )} α ( α0 m) + = ( ) D α,p α = P α dα α 0 P ( α) α Same distance fo P ( α) P ( α) α 0 α α 0 α 38

39 f {} θ = T { T } MX MY MX MY RX RY θ + + θ θ T R X θ In this epession: 1. The two classes means ae encouaged to get fa fom each othe. The Y-class is encouaged to spead as much as possible, and 3. The X-class is encouaged to condense to a neaconstant value Thus, getting good ejection pefomance. back 39

40 . The Distance Epession { Z } N k k= 1 1 N M= N R k Z k k= 1 N T Zk M Zk M k= 1 1 = N T T T k z = θ Z m=θ M, =θ Rθ 40

41 41 ( )( ) ( )( ) [ ] θ θ θ + θ = = + T T T T M M M M m m m m R R back

42 3. Relation to FLD* Minimize vaiances Assume that N X k k= 1 { } X and { } N Y Y k k= 1 Gaussians Maimize mean diffeence *FLD - Fishe Linea Disciminant 4

43 { Z } N k k= 1 1 N M= N R k Z k k= 1 N T Zk M Zk M k= 1 1 = N T T T k z = θ Z m=θ M, =θ Rθ 43

44 f {} θ = θ T T M X X θ T T Maimize M Y θ R θ+θ R θ Y = Minimize T θ M M M M θ X Y X Y T θ RX + RY θ T θ θ T T R M X X θ θ θ T T R M Y Y θ 44

45 45 ( ) ( ) m m P(X) m m P(Y) In the MRC we got the epession fo the distance The distance of the Y points to the X-distibution The distance of the X points to the Y-distibution If P(X)=P(Y)=0.5 we maimize ( ) ( ) m m m m

46 46 Instead of maimizing the sum ( ) ( ) m m m m Minimize the invese of the two epessions (the invese epesent the poimit) ( ) ( ) ( ) m m Min m m m m Min + = back

47 4. Algoithm Stuctue N X k k 1 { } X { } N Y (0) 0 Y k k 1 = Compute R,M X X = Compute R,M Y Y Minimize f(θ) & find thesholds { Sub-set Y j+ 1 k { θ,d 1,d} j Remove T θ Yk < d k o T θ Yk > d N (j+ 1) } Y k= 1 1 N Y (j) < Theshold? END 47

48 No moe Kenels Face { θ,d,d } J 1 j = 1 Poject onto the net Kenel Is value in d,d 1 j Yes No Non Face j = j+ 1 back 48

49 5. Counting Convolutions Assume that the fist kenel ejection is 0<α<1 (I.e. α of the incoming blocks ae ejected). Assume also that the othe stages ejection ate is 0.5. Then, the numbe of oveall convolutions pe piel is given b ( 1 α) α 1+ k= k 0.5 k 1 ~1 = 3 α= α= 0.99 α= 0.9 α= 0.6 back 49

50 6. Using Colo Seveal options: Tivial appoach use the same algoithm with blocks of L-b-L b 3. Eploit colo edundanc wok in HSV space with decimated vesions of the Hue and the Satuation laes. Rejection appoach Design a (possibl non-spatial) colo-based simple classifie and use it as the fist stage ejection. back 50

51 7. D-Rotated Faces Input block Pose Estimation and Alignment Fontal & Vetical Face Detecto Face/ Non- Face Remaks: 1. A set of otated kenels can be used instead of actuall otating the input block. Estimating the pose can be done with a elativel simple sstem (few convolutions). back 51

52 8. 3D-Rotated Faces A possible solution: 1. Cluste the face-set to same-view angle faces and design a Final classifie fo each goup using the ejection appoach. Appl a pe-classifie fo fast ejection at the beginning of the pocess. 3. Appl a mid-classifie to map to the appopiate cluste with the suitable angle Input block Cude Rejection Mid-clas. Fo Angle Final Stage Face/ Non- Face back 5

53 9. Faces vs. Tagets Teating othe tagets can be done using the same concepts of Teatment of scale and location Building and taining sets Designing a ejection based appoach (e.g. MRC) Boosting the esulting classifie The specific chaacteistics of the taget in mind could be eploited to fine-tune and impove the above geneal tools. back 53

54 10. Futhe Impovements Pe-pocessing linea kind does not cost Regulaization compensating fo shotage in eamples Boosted taining enich the non-face goup b finding false-alams and tain on those again Boosted classifie Use the sequence of weak-classifie outputs and appl et anothe classifie on them use ada-boosting o simple additional linea classifie Constained linea classifies fo simple classifie Can appl kenel methods to etend to non-linea vesion back 54