Two-Layered Face Detection System using Evolutionary Algorithm

Transcription

1 Two-Layered Face Detection System using Evolutionary Algorithm Jun-Su Jang Jong-Hwan Kim Dept. of Electrical Engineering and Computer Science, Korea Advanced Institute of Science and Technology (KAIST), Guseong-dong, Yuseong-gu, Daejeon, , Republic of Korea jsjang, Abstract- This paper proposes a novel face detection method based on Principal Component Analysis (PCA) and Evolutionary Algorithm (EA). In a view-based approach to face detection, the face is treated as an input vector of high dimension; and a multivariate Gaussian model is often employed in representing these faces. A near-face is a vector which, according to certain specified distance measure, is close to being a face. EA is employed to estimate the covariance matrix of this model, which discriminates between face class and nearface class. The proposed face detection system is characterized by EA-based two-layered classifier which are designed with a cascade structure for efficient performance and computation. The performance of the proposed method is experimentally verified on BioID face database. 1 Introduction Face detection is one of the most important part in object detection problems. Face is the most spotlighted in computer vision field among the many kinds of target objects, such as car, door, stair, and so on. It also has many application areas such as face recognition, human tracking, crowd surveillance, intelligent human-computer interaction. It is the most fundamental and the initial step of these applications. It is a difficult problem since the face is a highly nonrigid object. Many research demonstrations and commercial applications have been developed and recent results have demonstrated excellent outputs, but there exist a lot of difference in performance between human vision system and state-of-the-art face detectors [1, 2, 3]. There are many good literatures of face detection techniques [4, 5] and a lot of approaches are going on. Early research in face detection started at the beginning of the 1970s, where simple heuristic technique was used [5]. Most approaches to face detection fall into one of two categories. They are either feature-based approach or viewbased approach. In the former category, human knowledge and facial features such as color, edge, shape, eyes, mouth and some other constraints are used to verify face patterns [6, 7]. In the latter category, 2-D images are directly classified into face groups using pattern recognition algorithms. This approach uses face knowledge implicitly in the system through learning or training face (or non-face) images, while feature-based approach uses the human knowledge explicitly. Many feature-based approaches have been studied intensively, however most of them suffer from the unpredictability of face appearance and environmental conditions. And rapid development of computer technology helped the new approach in which face detection is treated as a statistical pattern classification problem. In view-based approach, there are many pattern classification methods such as neural network [1], Support Vector Machine (SVM) [8] and Adaboost with rectangle features [3], Principal Component Analysis (PCA) [9], etc. One of the most famous methods among view-based approaches is PCA, which is well known as eigenfaces [10]. Given an ensemble of different face images, the technique first finds the principal components of the face training data set, expressed in terms of eigenvectors of the covariance matrix of the face vector distribution. Since the face reconstruction by its principal components is an approximation, a residual reconstruction error is defined in the algorithm as a measure of faceness. Moghaddam further developed this technique within a probabilistic framework [9]. PCA is an appropriate way of constructing a subspace for representing an object class in many cases, but it is not necessarily optimal for distinguishing between the face class and the non-face class. The face class might be better represented. Sung and Poggio proposed the mixture of multidimensional Gaussian model [11]. They used an adaptively changing normalized Mahalanobis distance metric. Afterward, many face space analysis algorithms have been investigated and some of them have outstanding performance. But the problem of the PCA-related approaches lies in making decision rules or distance measures. Many research used maximum likelihood estimation using Gaussian model with PCA [12]. However, they may not be the optimal ones for distinguishing between face images and non-face images. To solve such a complex problem, evolutionary algorithm (EA) is an efficient tool. EA has powerful search and optimization performance in a complex problem. Prakash and Murty studied optimal subset of principal component selection and applied to vowel recognition [13]. They used genetic algorithm to select the principal components which maximize the recognition rate. Liu and Wechsler proposed a different approach to maximize the recognition rate using evolutionary algorithm [14]. They attempted to find optimal basis for the dual purpose of data compression and pattern classification. Jang et al. [15, 16] applied face verification and detection task to estimate the likelihood function using quantum-inspired evolutionary algorithm [17, 18]. In this paper, we propose a novel face de-

2 tection system as a further study of our previous result [16]. This paper is organized as follows. Section 2 describes PCA and discriminant functions briefly. Section 3 presents the proposed classifier including two-layered architecture and training method. Section 4 presents the experimental results, and conclusions follow in Section 5. 2 PCA and Discriminant Functions In this section, we present PCA concept and discriminant functions using Gaussian densities as a basic technique of this study. Many view-based face detection methods can be understood in a probabilistic framework. An image or feature vector extracted from input image is treated as a random variable x, and it is characterized for class-conditional density functions p(x ω f ), where ω f is a face class. Bayesian classification or maximum likelihood can be used to classify a candidate window region as face or non-face [12]. However, it is difficult to find the class-conditional density function, p(x ω f ) due to the high dimensionality of x. Unfortunately, it is not yet understood if there exist natural parameterized forms for p(x ω f ). Many works in view-based approach concern empirically validated parametric or nonparametric approximation for p(x ω f ) [4, 9]. 2.1 PCA Concept Given a set of m n pixels images I 1,I 2,...,I K }, we can form a set of N-dimensional vectors X =x 1,x 2,...,x K }, where x i R N=mn, i = 1, 2,..., K. The data set might be a set of face images. The mean and the covariance matrix of the set are given by Σ = 1 K µ = 1 K K x i (1) i=1 K (x i µ)(x i µ) t (2) i=1 where µ is the mean, and Σ is the covariance matrix of X. The basis functions for the Karhunen-Loève Transform are obtained by solving the following eigenvalue problem: Λ = Φ t ΣΦ (3) where Φ is the eigenvector matrix of Σ, and Λ is the corresponding diagonal matrix of eigenvalues. We can obtain M largest eigenvalues of the covariance matrix and their corresponding eigenvectors. Then feature vector is given as follows: y = Φ t M x (4) where x = x µ is the difference between the image vector and the mean image vector, and Φ M is a submatrix of Φ containing the M largest eigenvectors. These principal components preserve the major linear correlations in the given set of image vectors. By projecting to Φ t M, original image vector x is transformed to feature vector y. It is a linear transformation which reduces N dimensions to M dimensions as follows: y = T (x) : R N R M. (5) After applying the linear transformation Φ t, the set of transformed vectors, feature vectors, has scatter Φ t ΣΦ. PCA chooses Φ so as to maximize the determinant of the scatter matrix of the transformed vectors. By selecting M largest eigenvectors, two subspaces can be obtained. One is the principal subspace (or feature space) F containing the principal components represented by distance-in-featurespace (DIFS). The other is the orthogonal space F represented by distance-from-feature-space (DFFS) [9]. An original input vector x can be approximately reconstructed from its feature vector y as follows: x = Φ M y + µ. (6) Note that the first M largest eigenvectors are the optimal basis vectors for minimizing the reconstruction error in case of the reconstructing an input image with any combination of M basis vectors. In a partial Karhunen-Loeve expansion, the residual reconstruction error is defined as ɛ 2 (x) = N i=m+1 y 2 i = x 2 M y 2 i (7) i=1 and this is the DFFS as stated above, which is basically the Euclidean distance. The component of x which lies in the feature space F is referred to as the DIFS. In practice, the covariance matrix, Σ is often singular. However, the eigenvectors can still be estimated using singular value decomposition. 2.2 Discriminant Functions Many view-based methods can be understood in a probabilistic framework. Usually, the class-conditional density function is considered as multivariate Gaussian model as follows: p(x ω f ) = exp 1 2 (x µ)t Σ 1 (x µ)} (2π) N/2 Σ 1/2 (8) where p(x ω f ) is the class-conditional density function, and ω f is a class of the face vector x. Since we assume the parametric form of the class-conditional density function as multivariate Gaussian, the parameters to be estimated are mean vector ˆµ and covariance matrix ˆΣ. By maximum likelihood estimation theory, estimations are given by (1) and (2). It says that the maximum likelihood estimation for the unknown population mean is just the arithmetic average of the training samples, and also estimation for the unknown covariance matrix is the arithmetic average of the matrices (x i ˆµ)(x i ˆµ) t. There are many different methods to represent pattern classifiers. One of the most useful is in terms of discriminant functions g i (x). The classifier is to assign an input vector x to class ω i if

3 g i (x) > g j (x) for all j i. (9) In face detection task, it can be regarded as two-category classification problem between face class and non-face class. Non-face class may form much greater number of images than face class since it covers rest of the possible images excluding face images. Therefore non-face class is represented by complementary set ωf c whose discriminant function is regarded as proper threshold rather than probability term. A posteriori probability for face class, P (ω f x), and threshold can be used discriminant function. Using log-likelihood function for simple calculation, discriminant function for face detection is described as g f (x) P (ω f x) ln p(x ω f ) + ln P (ω f ) and decision rule is face if gf (x) > δ x non-face otherwise (10) (11) where δ is threshold which represents complementary set ω c f. Using maximum likelihood estimation for parameter µ f and Σ f, the estimated parameter can be found using face training data set. The classifier based on the maximum likelihood is called ML classifier. As shown in subsection 2.1, input vector is transformed to feature vector using PCA. Then the discriminant function is represented by using feature vector y f, g f (x) = 1 } y t f 2 Λ 1 f y f + N ln 2π + ln Λ f +ln P (ω f ) (12) where Λ f is a diagonal matrix of eigenvalues. This discriminant function will be used for collecting the near-face training data in next section. 3 Classifier Design Overall structure of the proposed face detection system is shown in Figure 1. The detection system consists of variance filters, preprocessing, feature extraction, two-layered classifier and post-processing. By window scanning technique pixels subwindows are extracted from an input image, and then each window is classified by following stages. Variance filters are employed by heuristic, but it is efficient to discard the non-face images with simple calculations. In pre-processing stage, lighting compensation is applied by subtracting the best-fit linear plane from original window, and then histogram equalization is employed to obtain a uniform histogram. PCA is used for feature extraction and two-layered classifier is applied as a main classifier. In post-processing stage, overlapping detections are averaged to make final face locations. Face Test input Extract subwindow Variance filters pass Pre-processing Feature extraction 2 layer classifier pass Post-processing Non-face discard discard Figure 1: Overall structure of the proposed face detection system 3.1 Variance Filters To improve the detection rate and computation time, simple heuristic filters are employed. These filters exclude subwindows which could not be a face at all, such as homogeneous background. When main classification tasks are applied to current subwindows, a lot of computations are required including lighting compensation and histogram equalization. Therefore, early exclusion of those non-face subwindows can save computation time. Some subwindows which are likely classified to face class by main classifier can be easily discarded through simple filters, especially homogeneous background-like images. This process is easily performed by calculating the variance of the pixel values in some rectangles. Figure 2 shows two rectangles in which variance of the pixel values are calculated. The first region is the area around two eyes and the second region is the area around the mouth. (0,0) (x,y) w R1 R2 h (x+w,y+h) (W,H) Figure 2: Variance filters Given a normalized window of W H pixels, a rectangle

4 R is specified as follows: R = x, y, w, h} (13) with 0 x, x + w W, 0 y, y + h H, x, y 0, w, h > 0, and W = H = 19. In Figure 2, two rectangles, R1 = 2, 3, 16, 4} and R2 = 3, 12, 15, 16} are determined. Similar rectangle feature was proposed by Viola and Jones [3]. They searched efficient feature set and structure of cascade classifier using AdaBoost learning algorithm. Note that only two rectangular features are selected by heuristic in this paper because the variance filters are not main classifier. The variance of the pixel values in a rectangle R is denoted by V ar(r). Then filtering is performed by comparing V ar(r) to proper threshold. The threshold can be determined to pass all face images while maximizing the discard of non-face images. For all input windows, calculate V ar(r1) and compare the threshold. If the variance is larger than a threshold, then current input window passes to the second variance filter, otherwise current input window is regarded as non-face. Cascade structure provides computational efficiency because early discarding cases do not require further classification processes which are expected to need more computations. To improve the computation time, variance calculation is performed by the integral image method [3]. 3.2 Training Classifiers using EA Generally, view-based approaches are using a large number of non-face images to train the face detector. It is very difficult to cover nearly infinite non-face images, since there exist much greater number of non-face images than face images. Many researcher used non-face training data collection technique such as bootstrapping [11, 1]. Similarly we collect false positive cases as near-face. It is gathered by applying variance filters and ML classifier with 7 eigenvectors. The ML classifier is performed by evaluating the discriminant function, equation (12), with 7 eigenvectors, and comparing the proper threshold. All images in the face training data have the larger discriminant function value than threshold. After gathering the near-face data, we focus on near-face data rather than non-face. Near-face is particular subset of non-face, which is close to face set. By limiting the field of the detection problem as two specific classes, face and near-face, it can be solved more efficiently. The ML estimation guarantees quite good performance, but it is not optimal for discriminating face images from the near-face images. In the discriminant function, parameters to be estimated are mean and covariance. We use estimated mean from maximum likelihood estimation. It is reasonable because the mean is given by the arithmetic average of the face training data. Nevertheless, covariance term needs further consideration. After applying PCA to input vector, covariance matrix has a diagonal form. It means that each diagonal element represents the weight of corresponding basis. It can be natural that the eigenvalue is the diagonal element and the value effects the DIFS component 30 genes for Λ f 30 genes for a Λ n a1 a2 L 30 b1 b2 L 30 Figure 3: Structure of the chromosome corresponding eigenvector. To minimize the reconstruction error when dimension reduction is employed, the eigenvector with the largest eigenvalue is the first candidate basis. But reconstruction error which is equivalent to the sum of pixel intensity differences is not a measure deciding whether the input is a face or not. For this reason, Prakash and Murty attempted to select the particular eigenvector subset for discrimination point of view [13]. Our approach is estimating the covariance matrix to acquire better discrimination than ML estimation. The estimation can be regarded as numerical optimization of each component of covariance matrix. It is characterized by M eigenvalues, which are the diagonal elements of the covariance matrix after dimension reduction. To estimate the diagonal elements, EA is employed. EA-based estimation of a discriminant function is performed by using two training data set, face training data and near-face training data. The discriminant function for face training data is given by equation (12), and that for near-face training data, g n (x) is similarly given by g n (x) = 1 y t 2 n Λ 1 n y n + N ln 2π + ln Λ n } + ln P (ω n ) (14) where y n is the feature vector of near-face training data. Then classification can be performed simply comparing the two discriminant functions with a proper threshold. The classifier is to assign an input vector x to face if b g f (x) > g n (x) + τ (15) where τ is the control parameter adjusting detection rate and false positive rate. Since the near-face data are collected using 7 eigenvectors, more eigenvectors need to discriminate between face and near-face. We set 30 eigenvectors for each set of training data by experiment. Then the total number of parameters to be estimated are 60. These parameters are the diagonal components of Λ f and Λ n in equations (12) and (14), respectively. They are represented by following equations: Λ f = diag(a 1, a 2,..., a 30 ), (16) Λ n = diag(b 1, b 2,..., b 30 ). (17) To find the estimated parameters using EA, genes are encoded as real-values. Each gene represents a diagonal component of the two covariance matrices. Therefore a chromosome represents the two covariance matrices for face and non-face. Figure 3 shows the structure of the chromosome. The boundary of the diagonal components are given by using maximum value and minimum value of the eigenvalues.

5 Fitness EA Input images from training database x ML Feature extraction using PCA y Calculate the discriminant function Λˆ Generation gˆ f (x) > gˆ (x) +τ n EA (maximize the fitness) Figure 4: Training curve Calculate the fitness 0.1λ min < a i, b i < 10λ max, (1 i 30) (18) where λ min and λ max are the minimum and maximum values of the eigenvalues of the covariance matrices for face and non-face training data, respectively. To define appropriate fitness value, we evaluate two kinds of score, S f for face training data, S n for near-face training data. Basically, it is added by +1 for every correct classification. Exceptionally, penalty function is applied if the score is lower than some minimum value. It means that minimum bound of detection rate and false positive rate are considered. Finally, the fitness value is the sum of the two scores: F itness = S f + S n. (19) Using this fitness function, evolution process performed to find the solution which has maximum fitness value. Population size is set to 40, and 20 parents are selected for reproduction. 20 parents and 20 new offsprings make up the next generation. Total 7634 training data which consist of 3350 face images and 4284 near-face images were used. Figure 4 shows a training curve which indicates the fitness value of the best solution. Dash line indicates the fitness value of the ML estimation, which uses the eigenvalues of covariance matrix of the training data. Before 400 generations, EA found the better solution than ML estimation. After evolution process, covariance matrices can be estimated by the best solution. Let estimated covariance matrices, ˆΛ f and ˆΛ n. Then EA-based discriminant functions are presented by rewriting equations (12) and (14) as follows: ĝ f (x) = 1 2 ĝ n (x) = 1 2 } y t 1 f ˆΛ f y f + N ln 2π + ln ˆΛ f } +ln P (ω f ) (20) y t 1 n ˆΛ n y n + N ln 2π + ln ˆΛ n +ln P (ω n ) (21) and decision rule is described as follows: x face if ĝf (x) > ĝ n (x) + τ near-face otherwise. (22) Figure 5: Flowchart of the estimation process using EA By adjusting the control parameter τ, several pairs of detection rate and false positive rate can be acquired. Some result pairs are given in Table 1. Proposed methodology for estimating the covariance matrices is shown in Figure 5. It shows each generation of evolution which maximizes the fitness value. After reaching a termination condition, the final values of ˆΛ f and ˆΛ n are obtained, which maximize the fitness value on the training data. By using the obtained covariance matrices, equation (22) is calculated for classification. 3.3 Two-Layered Classifier In this subsection, two-layered classifier structure is described. As shown in previous subsection, there are two kinds of classifiers. One is the ML classifier with 7 eigenvectors and the other is the EA-based classifier with 60 eigenvectors (30 for faces and 30 for near-faces). Two-layered classifier is a cascade structure of these two classifiers. The first layer is the ML classifier with 7 eigenvectors. It discards a lot of images while passing the face and near-face images. The second layer is the EA-based classifier which focuses on discriminating between face and near-face images. Figure 6 shows the two-layered classifier. Figure 6: Two-layered classifier In case of discarding at the first layer, further computations are not required. The advantage of the two-layered classifier can be described in two aspects. One is that each layer is designed for different purpose. The first layer focuses on reducing the face candidate images from the huge

6 amount of possible inputs. Then the second layer focuses on discriminating task within the relatively small set of face and near-face classes. It provides better detection rate than a single classifier. The other is computation efficiency. Early exclusion in the first layer cause further computations to be unnecessary. The computation in the second layer is much more than the first layer. Because the second layer uses two eigen models and each model has 30 dimensions while the first layer has a single 7-dimensional model. (a) (b) 4 Experimental Results Experimental results are provided to evaluate the performance of the proposed face detection system. It is trained by 7634 training data as described in subsection 3.2. It is tested to detect frontal faces from BioID database [19]. It consists of 1521 gray-scale images with pixels. Each one shows the frontal view of a face of one out of 23 different test persons. Figure 8: Successful face detection examples Detector Performance Two layer (EA) Two layer (ML) Single layer Detection rate (%) Case1 95.1% / % / % / 3 Detection rate / False positives Case2 95.4% / % / % / 16 Case3 96.4% / % / % / 34 Table 1: Detection rate and false positives Two layer (EA) Two layer (ML) Single layer (ML) Case4 96.6% / % / % / False positives Figure 7: ROC curve To detect faces by window scanning, we set the minimum size of extracted window at pixels. It means that detectable face size is larger than pixels. It can be implemented by setting a starting scale as a factor of Starting from the scale factor 1.875, input image was scaled down by a factor of 1.25, while base window with pixels was applied to face detection task. The performance of the classifier usually evaluated by detection rate with false positives. A good classifier should have high detection rate at small number of false positives. Three classifiers, EA-based two-layered classifier, ML classifier with (a) (b) Figure 9: Missed and false positive examples two layer and ML classifier with a single layer were tested. A single layer classifier is ML classifier with 7 eigenvectors used in collecting the near-face training data. Table 1 show the performance of the proposed classifier on BioID database including the detection rate and corresponding false positives, respectively. By changing the threshold values several cases of the detection rate and the false positives are given. The results show that EA-based two-layered classifier has better performance than others. Through the pairs of detection rate and false positives, Receiver Operating Characteristic (ROC) curve is given in Figure 7. Instead of false positive rate, the number of false positives was used for the x-axis of the ROC curve. The Y-axis indicates the detection rate. The detection rate of the single layer classifier was increasing together as false positives were increasing. Figure 8 shows some successful detection results. And examples of missed face and false positive are shown in Figure 9. The proposed detection system misses the face with large variation of pose as in Figure 9(a) or makes the false positive as in Figure 9(b). At this false positive case, the region which consists of beard and mustaches looks like face. 5 Conclusions In this paper, EA-based frontal face detection system was proposed. The performance of the proposed detection system was tested on BioID database. The results showed EAbased two-layered classifier had better performance in detection rate and false positives than classifier with ML estimation. It was caused by the two-layered architecture: the former layer focused on reducing the face candidate images of the huge amount of possible inputs, and then the latter

7 layer focused on discriminating with the relatively small set of face and near-face classes. Acknowledgments This work was supported by the ITRC-IRRC (Intelligent Robot Research Center) of the Korea Ministry of Information and Communication in Bibliography [1] H.A. Rowley, S. Baluja and T. Kanade, Neural Network-Based Face Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp , [2] H. Schneiderman and T. Kanade, A Statistical Method for 3D Object Detection Applied to Faces and Cars, in Proc. IEEE Int l Conf. Computer Vision and Pattern Recognition, vol. 1, pp , [3] P. Viola, M. Jones, Rapid Object Detection using a Boosted Cascade of Simple Features, in Proc. IEEE Int l Conf. Computer Vision and Pattern Recognition, [4] M.-H. Yang, D. Kriegman, and N. Ahuja, Detecting Faces in Images: A Survey, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp , [5] E. Hjelmås and B. K. Low, Face detection: A survey, Computer Vision and Image Understanding, vol. 83, pp , [6] M.-H. Yang and N. Ahuja, Detecting Human Faces in Color Images, in Proc. IEEE Int l Conf. Image Processing, vol. 1, pp , [7] G. Holst, Face detection by facets : Combined bottom-up and top-down search using compound templates, in Proc. Int l Conf. Image Processing, TA07-08, [12] C. Liu, A Bayesian Discriminating Features Method for Face Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp , [13] M. Prakash and N. Murty, A genetic approach for selection of (near-) optimal subsets of principal components for discrimination, Pattern Recognition Letters, vol. 16, pp , [14] C. Liu and H. Wechsler, Evolutionary Pursuit and Its Application to Face Recognition, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp , [15] J.-S. Jang, K.-H. Han and J.-H. Kim, Evolutionary algorithm-based face verification, Pattern Recognition letters, vol. 25, pp , [16] J.-S. Jang, K.-H. Han and J.-H. Kim, Face Detection using Quantum-inspired Evolutionary Algorithm, in Proc. IEEE Congress on Evolutionary Computation, pp , [17] K.-H. Han and J.-H. Kim, Quantum-inspired Evolutionary Algorithm for a class of combinatorial optimization, IEEE Trans. Evolutionary Computation, vol. 6, no. 6, pp , [18] K.-H. Han and J.-H. Kim, Quantum-inspired Evolutionary Algorithms with a New Termination Criterion, Hε Gate, and Two Phase Scheme, IEEE Trans. Evolutionary Computation, vol. 8, no. 2, pp , [19] O. Jesorsky, K.J. Kirchberg, and R.W. Frischholz, Robust face detection using the hausdorff distance, in Proc. Int l Conf. Audio- and Video-based Biometric Person Authentication, pp , [8] E. Osuna, R. Freund and F. Girosi, Training Support Vector Machines: An Application to Face Detection, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp , [9] B. Moghaddam and A. Pentland, Probabilistic Visual Learning for Object Representation, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp , [10] M. Turk and A. Pentland, Eigenfaces for Recognition, J. Cognitive Neuroscience, vol. 3, no. 1, pp , [11] K.-K. Sung and T. Poggio, Example-Based Learning for View-Based Human Face Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp , 1998.