PARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH. Luoluo Liu, Trac D. Tran, and Sang Peter Chin

Transcription

1 PARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH Luoluo Liu, Trac D. Tran, and Sang Peter Chin Det. of Electrical and Comuter Engineering, Johns Hokins Univ., Baltimore, MD 21218, USA {lliu69,trac,schin11}@jhu.edu ABSTRACT Partial face recognition is a roblem that often arises in ractical settings and alications. We roose a sarse reresentation-based algorithm for this roblem. Our method firstly trains a dictionary and the classifier arameters in a suervised dictionary learning framework and then aligns the artially observed test image and seeks for the sarse reresentation with resect to the training data alternatively to obtain its label. We also analyze the erformance limit of sarse reresentation-based classification algorithms on artial observations. Finally, face recognition exeriments on the oular AR data-set are conducted to validate the effectiveness of the roosed method. Index Terms Face Recognition, Image Alignment, S- arse Reresentation, Dictionary Learning 1. INTRODUCTION The face recognition task attracts much attention because of its ractical use for safety and surveillance uroses as well as the otential to understand how human-beings identify d- ifferent eole. Various aroaches have been roosed. A- mong them, sarse reresentation-based classification (SRC) ioneered by J. Wright et al. is one of the simlest methods but still rovides state-of-the-art erformance [1]. The authors showed, by using an over-comlete dictionary, sarse reresentation can be treated as a high-dimensional reresentation and classification is reformed on to of that. However, as with almost all of other current face recognition systems, sarse reresentation-based method relies highly on the success of solving sarse coding, which requires accurate alignment between training data and testing data. In ractice, much of test data are collected in uncontrolled conditions with the ossibility of severe occlusion or missing information. This leads to the artial face recognition roblem. In these settings, one of the most common issues is that the artial test data obviously do not align well with the training data. One may re-rocess the data to achieve the same We gratefully acknowledge that this work was in art suorted by NSF (NSF-DMS , NSF-CCF , NSF-CCF , NSF-EECS ) and AFOSR (FA ). alignment. A oular aroach would be detecting local features and align data by normalizing the detected features geometrically. However, this aroach will fail whenever feature detection fails, which is highly likely since given artial data usually lacks these features. Another solution is accounting for every ossible translations via a suer-redundant dictionary, which certainly would result in exorbitant comutational/memory cost. Some of artial face recognition algorithms are based on finding transform-invariant features such as Scale-invariant feature transform (SIFT) in MKD-SRC [2], Gabor Ternary Pattern (GTP) [3], ooled sarse codes from SIFT [4] or local atches [5], etc. Others recover the transformation and solve the sarse code simultaneously as in [6], [7], [8]. Reorted success of local-feature-based holistic face recognition algorithms imlies that the ossibility of recognition based on a few discriminative local features. Moreover, Insired by human-beings ability to identify face relying on artial observations, another motivation for us to study the artial face recognition roblem is the intriguing question: given limited observations, how much information is enough to reliably identify a erson? In this aer, we roose an aroach to solve the aforementioned roblem by alternatively finding the alignment and sarse coding. Main contributions are: (i) Our roosed framework make recognition task with artial observations efficiently solvable with different aligning conditions in suervised dictionary learning (SDL) framework [9]; (ii) Our method can be easily extend to artial observations almost in the form of any shae although for simlicity we demonstrate the one atch case; (iii) Sufficient conditions that lead to correctly classification on artial observations are exlored. 2. METHOD The roblem is formulated as follows: given well-aligned holistic faces with labels as training data, artial face recognition task aims at classifying artially observed faces without any alignment information. Under suervised dictionary learning framework, we roose an aroach that alternatively finds the sarse code as well as the best ossible alignment /16/$ IEEE 2389 ICASSP 2016

2 Notations: Let T 1, T 2 be the sets of training samles, and the testing samles resectively, where y i T 1, y i R m and y t T 2 are not necessarily of the same size. Also, k N +, [k] := {1, 2,.., k}. N denotes the number of different classes in the training data set. Let l i be the label associated with y i T 1, then and l i L := [N]. K is the label matrix of T 1. D R m n denotes the dictionary and x R n reresents sarse code and W rovides the arameter of the classifier. Let f(x, W) reresents the model of the linear classifier: f(x, W) = Wx + b [10]. We define a artial observation oerator ( ) Λ, Λ [m] as follows. Let I Λ denote sub rows of identity matrix with row indices in the set Λ. Then for y R m, (y) Λ := I Λ y. In short, Λ contains the suort of image ixels that we are able to observe whereas its comlement Λ c indicates the suort of missing information. For D R m n, D Λ := I Λ D simly selects the corresonding rows in the observation set Suervised Dictionary Learning and Classification Suervised Dictionary Learning A classical aroach to seek the dictionary that yields sarse reresentation is given as [11] [12]: min (1/2) y i Dx i ρ sarsity(x i ), ρ > 0, (1) D,x i T 1 where sarsity( ) denotes a regularizer that encourages sarsity where common choices are l 0, l 1 norms. Let R s (l i, f(x, W)) indicates the classification loss. By minimizing classification loss and data fidelity y i Dx i 2 2, the suervised dictionary learning framework can be described as follows: (D, x, W ) = arg min R s (l i, f(x i (y i, D), W)) D,x,W i T 1 +β y i Dx i ρ sarsity(x i ), β > 0. (2) For solving sarse coding, l 0 minimization can be solved by Iterative Hard Thresholding (IHT) [13] whereas Orthogonal Mathching Pursuit (OMP) [14] for corresonding constraint minimization and the l 1 minimization can be solved via Least Absolute Shrinkage and Selection Oerator [15], or Gradient Projection for Sarse Reconstruction [16] Classification If test data y t T 2 has the same alignment condition as the training data, then x can be solved by traditional sarse coding. The technique of solving x for artial data will be - resented in Section 2.2. After obtaining x, the label assignment bases on the class that yields the minimum classification loss among all classes: ĉ(y t ) = arg min R s (l i, f(x (y t, D ), W )). (3) 2.2. Alternating Alignment and Sarse Coding Given artially observed test data y t, let y h reresent the corresonding holistic face, which is unknown (and in this scenario, the recovery is not necessary). We would like to solve for sarse code of the test data with artial constraint on the unknown observed set Λ. We know that (D x) Λ = I Λ (D x) = (I Λ D )x = D Λ x and y t = y h Λ. Considering minimizing the data fidelity term on Λ: (y h D x) Λ 2 2 = y h Λ D Λ x 2 2, then the roblem can be recast as: (x, Λ ) = arg min (1/2) y t D Λ x ρ sarsity(x), α R n,λ i S (4) where S is the subset of all candidates that Λ could take from. (4) is solved by Alternating Minimization. Details are deicted in Algorithm 1. However, for a learnt dictionary D, each column no longer has the hysical meaning as a vector in the training data-set, so as the alignment. Udating alignment Λ seeks a otimal artial set resulting smaller reresentation error and sarsity level with resect to the otimal dictionary. Algorithm 1 Partial Face Recognition based on Alternating Alignment and Sarse Coding(AASP) 1: Require: Training set, label set T 1, K; Testing set T 2 2: Suervised Dictionary Learning: Inut: T 1, K, Outut: (D, W ). 3: for i = 1, 2,.. S do 4: x i = arg min(1/2) y t D Λi x ρ sarsity(x), x R n 5: end for 6: Λ = arg min(1/2) y t D Λi x i ρ sarsity(x i ) Λ i S 7: x = arg min(1/2) y t D Λ x ρ sarsity(x) x R n 8: Return: c(y t ): classification by (3) Imlementation Details In our imlementation, we use linear rediction with zero offset (b = 0) as classifier and the squared loss as the loss function R s. The label is reresented by their binary exression q. For l i = j, j [N], then q i = e T j = [0, 0,..1, 0,.., 0] T. Therefore, R s (l i, f(x, W)) = q i Wx i 2 2. In testing, we demonstrate a simle aroach to find the Λ in ste 6 of Algorithm 1. A more comlicated solution which extensively searching alignment uses otical flow is given in [8]. Here, Λ is initialized via mean face matching. The mean face y m of the training set is calculated by taking the average of training samles. In the case when artially observed data is connected and rectangular, the artial observation oerator Λ can be uniquely determined by coordinates of the to left ixel ( i, j ) and its width b and height h. Let B and H be the width and height of the holistic image resectively; 2390

3 S Ω contains all oerators in the form of Λ( 1, 2, b, h) that selects a atch same size as the test, then mean face matching is simly: Λ mean( a, b, b, h) = arg min Λ S y t (y m ) Λ 2. (5) To reduce the comutational comlexity, we restrict the ossible region from S Ω to S, which includes be all oerators gives artial observations that are within a local neighbourhood of c n ixels of (y) Λ mean. 3. ANALYSIS In general, artially observed data cannot contain more information than entirely observed data. Unlike the scenario in matrix comletion with randomly missing observations, the observed set is highly satially coherent. It is not ossible to recover the entire data with high accuracy and then reform classification. However, in ractice, successful classification based on local discriminative features shows the ossibility of accurate classification based on artial data Data Model In the sarse reresentation based classification framework, we assume that the data model is y = D x + z, where y, z R m, x R n and x is s-sarse, s < n and the sarse reresentation x for data of different classes are quite distinctive to each other. Hence we might exect to find good reresentation D along with a reasonable classifier arametrized by W from the learning rocess. We also assume that there exists a recovery algorithm that can erfectly recover the true suort set Λ of the artial data. To simlify the notations, we dro the subscrit from D, W, Λ in this section. Let us assume that we get the otimal dictionary D, and test data is correctly classified in solving y = Dx. Now the question becomes: does it exist a artial region Λ [m] such that solving x via y Λ = D Λ x still yields the same correct classification result? 3.2. The Noiseless Case Remark 1 z = 0, for Λ [m], Λ = rank(d), and rows of D Λ are linearly indeendent, then solving y Λ = D Λ x is equivalent to solving y = Dx. In solving y = Dx, only rank(d) number of equations is needed. For m > n, rank(d) n, at most n features are needed for solving y = Dx. Λ can be comuted by finding linearly indeendent rows in D. When x is fairly sarse, comressive sensing theory [17] indicates that the number of measurements needed is O(s log(n)), which is ossibly smaller than n The Noisy Case Now, consider the case when z = 0, we analyze the sarse enalty being the l 1 norm for its sub-differentiability. Estimation of labels can be formulated as solving the system: Q(H): arg min R s (l i, f(x h (y, D), W)) subject to x h = arg min(1/2) y Dx ρ x 1. (6) On the other hand, similar rocedure based on artial observation Λ is equivalent to solving: Q(P): arg min R s (l i, f(x (y Λ, D Λ ), W)) subject to x = arg min(1/2) y Λ D Λ x ρ x 1. (7) As long as the solutions for Q(H) and Q(P) match, the classification based on artial observation is as accurate as the case with comlete observation. Ideally, sarse codes are the same, which yields to the following sufficient condition. Lemma 2 (Sufficient Condition): For Λ such that x = arg min(1/2) y Λ D Λ x ρ x 1, if x also solves min(1/2) y Λ c D Λ cx 2 2, then classification from artially observed data is just as accurate as with comlete data. Proof: Let x = arg min(1/2) y Λ D Λ x ρ x 1. Let g 1,Λ := (1/2) y Λ D Λ x ρ x 1, then 0 ( g 1,Λ / x) x=x. Also, let g Λ c := (1/2) y Λ c D Λ cx 2 2. Since y Dx 2 2 = y Λ D Λ x y Λ c D Λ cx 2 2, x h = arg min ρ x 1 + (1/2) y Λ D Λ x (1/2) y Λ c D Λ cx 2 2 = arg min g 1,Λ + g Λ c. If x solves g 1,Λ and g Λ c, then 0 ( g 1,Λ / x) x=x + ( g Λ c/ x) x=x, x h = x. {x : 0 ( g Λ c/ x)} has close form solution {D Λ cy + v : v Null(D Λ c)}, where denotes the seudo-inverse. If this solution set intersects with {x : x = arg min g 1,Λ }, then such a x = x h exists. If l y is the true label, then arg min R s (l i, f(x(y, D), W)) = arg min R s (l i, f(x(y Λ, D Λ ), W)) = l y. Given y, D and Λ, we could tell whether the set of features is good enough to return the same label as solving Q(H). 4. EXPERIMENTAL RESULTS We erform our exeriments on the croed AR data-set [18] containing 100 subjects with half males and half females. For faster comutation, we down-samle the data-set by a ratio of 0.5 and the resulting size is Under the assumtion that training data are well-aligned holistic faces, we exclude ictures taken with scarfs or sunglasses and only use tyes 1 7 and tyes for each erson in the data-set, which contains 4 different emotions and 4 lighting conditions. In the exeriment, we randomly artition the data-set: half for training and remaining for testing. To test our algorithm, we generate 3 secific atterns (Tye 1-3) with fixed 2391

4 coordinates and 3 random atterns with controlled sizes (Tye 4-6) in which the width b and height h are randomly generated from a redefined range: b [r 1 B, r 2 B], h [r 1 H, r 2 H] and (r 1, r 2 ) are (0.5, 0.9), (0.3, 0.8) and (0.2, 0.5) resectively. Examles are shown in Fig. 1. Different dictionary learning algorithms are adoted in training. We aly online dictionary learning (ODL) [19], and label consistent K-SVD dictionary learning (LC-KSVD) [20] for constraint l 1, l 0 minimization resectively. We comare our algorithm with suervised dictionary learning methods using the same mean-face matching initialization (denoted by MF-Match) and one of the state-of-the-art feature-based artial face recognition algorithms: MKD-SRC [2]. In our exeriments, all dictionary learning algorithms share the same arameter setting: ρ = 0.1, β = 1, dictionary size n = 700, and number of iterations is 10. The sarsity level of OMP is set to be 30 and the neighborhood size c n is 5. Table 1 lists the averaged accuracy among all classes. Cumulative Matching Curves for two hard cases (Tye 3, 6 with small observation sets) are deicted in Fig. 2 and Fig. 3. Our algorithm erforms better than others in both relatively easy cases (Tye 1, 4) and hard cases (Tye 3, 6). In SDL with MF Match and AASP, l 1 and l 0 minimization algorithms erform similarly. For holistic face recognition (Tye 0), two methods coincidence since S Ω contains only one element: the whole observation set. All algorithms erform reasonably well when size of the observation set is large and when decreasing, they all suffer. By Section 3, larger size of Λ leads to tighter relaxation of the original constraint in (6), and therefore one would exect better result. Ours is relatively robust against the decreasing size of Λ and esecially will not be effected that much when observation contains only art of major local areas such as eyes used for feature extraction as feature based methods. Fig. 2: Tye 3: Cumulative Match Score Curves Fig. 3: Tye 6: Cumulative Match Score Curves MKD MF-Match AASP Tyes -SRC [2] -ODL -LC-KSVD -ODL -LC-KSVD Table 1: Rank-1 accuracy with various artial atterns 5. CONCLUSIONS AND FUTURE WORK Fig. 1: Examles: Col. 1: holistic faces; Col. 2 (to to bottom): fixed atterns 1 (a large artial block), 2 (left face), 3 (mouth and chin); Col. 3, 4, 5 : random atterns 4, 5, 6. Size of artial faces among tyes: 0 > 1 > 2 > 3, 0 > 4 > 5 > 6 We develo a sarse reresentation-based classification algorithm called AASP to solve the artial face recognition roblem. Exeriments shows that our roosed method erforms better than other comarison methods esecially in the case of severe information missing. In the future, we lan to extend our method to handle various other tyes of corrutions and exlore the lower bound of size of artial data that still rovides correct classification in the noisy case. 2392

5 6. REFERENCES [1] J. Wright, A. Y Yang, A. Ganesh, S. S. Sastry, and Y. Ma, Robust face recognition via sarse reresentation., IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, , [2] S. Liao and A. K. Jain, Partial face recognition: An alignment free aroach, in Biometrics (IJCB), 2011 International Joint Conf. on. IEEE, 2011, [3] S. Liao, A. K Jain, and S. Z Li, Partial face recognition: Alignment-free aroach, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, no. 5, , [4] J. Yang, K. Yu, and T. Huang, Suervised translationinvariant sarse coding, in Comuter Vision and Pattern Recognition (CVPR), 2010 IEEE Conf. on. IEEE, 2010, [5] Y. Chen, T. T. Do, and T. D. Tran, Robust face recognition using locally adative sarse reresentation, in Image Processing (ICIP), th IEEE Conf. on. IEEE, 2010, [6] A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, and Y. Ma, Toward a ractical face recognition system: Robust alignment and illumination by sarse reresentation, Pattern Analysis and Machine Intelligence, IEEE Trans. on, vol. 34, no. 2, , [7] J. Huang, X. Huang, and D. Metaxas, Simultaneous image transformation and sarse reresentation recovery, in Comuter Vision and Pattern Recognition, CVPR IEEE Conf. on. IEEE, 2008, [8] X. Sun, N. M. Nasrabadi, and T. D. Tran, Sarse coding with fast image alignment via large dislacement otical flow, arxiv; International Conference on A- coustics, Seech, and Signal Processing(ICASSP), 2016 IEEE Conf. on (to be aear). [9] J. Mairal, J. Ponce, G. Sairo, A. Zisserman, and F. R. Bach, Suervised dictionary learning, , [12] M. Elad, M. Aharon, and A. M. Bruckstein, The k- SVD: An algorithm for designing of overcomlete dictionaries for sarse reresentations, IEEE Trans. Image Process, vol. 15, no. 12, , [13] T. Blumensath and M. E. Davies, Iterative hard thresholding for comressed sensing, Alied and Comutational Harmonic Analysis, vol. 27, no. 3, , [14] J. A. Tro and A. C. Gilbert, Signal recovery from random measurements via orthogonal matching ursuit, Information Theory, IEEE Trans. on, vol. 53, no. 12, , [15] R. Tibshirani, Regression shrinkage and selection via the Lasso, Journal of the Royal Statistical Society. Series B (Methodological), , [16] M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, Gradient rojection for sarse reconstruction: Alication to comressed sensing and other inverse roblems, Selected Toics in Signal Processing, IEEE Journal of, vol. 1, no. 4, , [17] E.J. Candes, J. Romberg, and T. Tao, Robust uncertainty rinciles: Exact signal reconstruction from highly incomlete frequency information, Information Theory, IEEE Trans. on, vol. 52, no. 2, , [18] A. M. Martínez and A. C. Kak, PCA versus LDA, Pattern Analysis and Machine Intelligence, IEEE Trans. on, vol. 23, no. 2, , [19] J. Mairal, F. Bach, J. Ponce, and G. Sairo, Online learning for matrix factorization and sarse coding, The Journal of Machine Learning Research, vol. 11, , [20] Z. Jiang, Z. Lin, and L. S. Davis, Learning a discriminative dictionary for sarse coding via label consistent K-SVD, in Comuter Vision and Pattern Recognition (CVPR), 2011 IEEE Conf. on. IEEE, 2011, [10] J. Mairal, J. Ponce, G. Sairo, A. Zisserman, and F. R. Bach, Suervised dictionary learning, in Advances in neural information rocessing systems, 2009, [11] K. Engan, S. O. Aase, and J. H. Husøy, Frame based signal comression using method of otimal directions (mod), in Circuits and Systems, ISCAS 99. Proceedings of the 1999 IEEE International Symosium on. IEEE, 1999, vol. 4,