A Double-objective Rank Level Classifier Fusion Method

Size: px

Start display at page:

Download "A Double-objective Rank Level Classifier Fusion Method"

Amice Ramsey
6 years ago
Views:

1 ½ 33 ½ 2 Þ Vol. 33, No ACTA AUTOMATICA SINICA December, 2007 è Î Á Ë Ãàß Ñ ý Melnik Â Åè Ë Ã, ÃÄ É Æ, Õ Ë Â è²². ð Melnik Â Ãàß, á Ç ÇĐ, Ç î µá»â Ã. Melnik Ù, Â Åè Ãàß, Ìàß Ë ÆÆ, ÍÅ» Ý Melnik Â Á., Åè ø Ä Ä, ÇÅ Ù Æ àß. ÇÅÃ ä ä Ç, Å Ã Â. ÂÄ àß ÔÙ àß Melnik àß. ³ Ã, µ Æ Ç, Ç, ä Ç TP39. 4 A Double-objective Rank Level Classifier Fusion Method LIU Ming YUAN Bao-Zong MIAO Zhen-Jiang Abstract Recently, Melnik proposed a new rank level classifier fusion idea, which managed to keep a balance between the preference for the specific classifier and the confidence it had in any specific rank. However, Melnik s classifier fusion method suffers from the curse of dimensionality. The number of parameters increases exponentially with the increase of the number of classifiers. Inspired by Melnik s idea, we propose a new fusion method, which achieves Melnik s objectives through combination of the rank transforming and the weighted classifier integration. Furthermore, a continuously differentiable classification error expression is given. Based on that, a gradient descendent parameter tuning algorithm is designed. We develop a multi-modal identity recognition system by fusion of palmprint and finger image data. Many experiments have been conducted to test the performance of our method under the condition of different classifier numbers. The experimental results show that the performance of our method is better than those of traditional methods and Melnik s method. Key words Classifier fusion, biometrics, palmprint recognition, finger image recognition Ç, Ã Ǒè Ǒ, Â Å Ã àß, Åàß 3 : æ Ã, Ë Ã Ã []. ËÆ Ù Ë Ã Ë ÃàßÕ : ßË ß [2]. ß Í Ù è, Ì, è Í Ç Ë ß þú þ ÍÚ Received October 25, 2006; in revised form April 24, 2007 Á Ù Ç Ý (973 Ç Ý) (2006CB30305, 2004CB380), Á ( ), á (2003SZ002) Supported by National Basic Research Program of China (973 Program) (2006CB30305, 2004CB380), National Natural Science Foundation of China ( ), and University Key Research Project (2003SZ002). ² Institute of Information Science, Beijing Jiaotong University, Beijing DOI: 0.360/aas ½ Ë, íõ Í àß, Ë ß ³ Ë ß BC (Baoda count) àß HR (Highest rank) àß à ß. BC àß HR àß³ ß, Á ; àß ËÍ, ÍÅ Ä Ë, Æ Ã Ã. Saranli [3] ÙÇÒ Ë Ã ½ÅÒ, ûâ Åè Ãñ. ðìàßá ÇË Þè Í Ý Î, Ç ³, ³ ìõ Ùµ á»â Ã. Ǒ, Saranli Â Å ÄÝ ÆèÑ Ç µ à., è Ë Ã, è Ý àßðè éµ Â Ã., Melnik [4] Ë Ã Ã Åè Ò. Ì Å Ë Ã ³ Ã: èð Æ ; ð Ë íõ è²². Æ, ù Ò à, Ã Â ÃǑ.

2 2 ñè Î Á Ë Ãàß 277 Ë íõ è²², ð ÇÖ Ë Ã Â ÃǑ. Ë ãä³ Ô è², Ë ãä³ è². Ë ³ î, Ë Þ è² ÞÃǑ³ ; Ë ³ î, Ë Þ è² Þ ÃǑ³. ǑÅÄ à, ù Ò à û Ë Â è²² Û ð Ë Ã Á, ÍÅ Ý Á Ë Ãàß ǑÎ Á Ë Ã àß. Þ [4] Åè Æ Ë Ãà ß MGR (Mixed group ranks) àß. MGR àß ðè Î Á Ë Ãàß, ð àó Ù Ã ê ìõ, ÃÁ ù ÇĐ. Melnik Â Ãñ Â Åè Î Á Ë Ãàß. Ìàß Ë è² ³ Ë ½, ÇÖ Ë ÃǑ, ¾ ½Ã, û î Ã ½ Þ. Ãàß, èå Þ Å è²àßâôã Í [5 8], ð Þ [5] Â Å Ã î Þ àß, àß³ Õ. ð Ã, Þ ð è² ½, ðǒåµ è² ê Ã. ð Ë ½, ðǒåçö Ë Ã Â ÃǑ. MGR àß ³, àßíõ, Åè àß ù, Ô. àß Ç Ã ½Åý, ÇÅÃ ä ä Ç, û àß àß MGR àß ½Å³, ÂÄ àß Ç ÔÙàß. 2 Î Á Ë Ã 2. ³ Đ è Ç Ã, L : {ω,ω 2,,ω L }, M ( Ã ): {C,C 2,,C M }. Ù ûè x, ¹ r m l (x) Ä C m ω l Ë. r m l (x) è Å R(x), R(x) Þè Ùè, Þè½ Ùè. Ã Ã ð Å R(x) Ç x ÙÞè ω l Óè² S l (x), ùè è²ø S l (x) = f l (R(x)), l =,2,,L () Ë Ãàß Á ð», ÂÔ, ãä àßð àß [2]. àß Ã Ãð, Ä» Ǒ S l (x) = w m r m l (x) (2) m= w m ( m =,2,,M) ðá ù àßíå»µ Ã, Ã Ã Ù Ë Ã Đ è ³, Ë, Ë Þ Ã Â ÃǑð. Ë, Ë ³ î, Ë Þ è² ÞÃǑ³ ; Ë ³ î, Ë Þ è² Þ ÃǑ ³. èýæ, Melnik Â Å Ë Ã àß è Á, ÇÖ Ë Ã Â ÃǑ. ǑÅ ÃÄ ÍÅ Á, Melnik Åè Ë Ãñ, û Ì ñ Â Å MGR àß. Melnik Ãñ ð Äùþè è²ø ï, Å ïð: ) Ù è è²ø Ù Ì Ë ³, Ù û ω l S l (x) = f l (r l (x), r2 l (x),, rm l (x)) (3) 2) Æ. è²ø S l (x) (l =, 2,, L) Ìá ûè Ë Ç, Ù û ω l Ë u = (u, u 2,,u M ), v = (v, v 2,, v M ) u m v m, m =, 2,, M f l (u) f l (v) (4) 3) Ù û 0 λ, è²ø S l (x) (l =, 2,, L) Ì Ú f l (u) f l (v) f l (u) f l (λu + ( λ)v) (5) ï )» Åè²ø á ; ï 2) Ý Ù è, ³ Ë Â è²óð Ù³ Ë Â è²; ï 3) ÍÅÇÖ Ë Ã Â ÃǑ. Melnik Ãñ Â Å Ë Ã è Á, ÇÖ Ë ÃǑ, Ú èãñ àß ǑÎ Á Ë Ãàß. ÙÌÃñ, Melnik Å MGR àß. à ß Ã Ã Ä Ǒ f l (r l (x),, rm l (x)) = w A min{r m l (x) : m A} (6) A {, 2,, M}

3 278 Þ 33 Ô A Ǒ {, 2,, M} û, Ǒ 2 M ; w A 0 Ǒ, àß. Ã ÃǑ Ǒð, ð Ã Ã «Ç Þèá Ù è, Ì íõ î, Ð ü è Ã, ðü ³, Å èà, Þ ð è, è, Ð Õ Å Þ [4] Ý, MGR àß Ú Melnik 3 ï, Ð ÍÅÇÖ ² ÃǑ, ÍÅ» Ë Â è²² ûþ, MGR àßǒ Ë ÃÒ á Å ð ð Ç î, ÇĐ, µ á»â Ã 2.2 àß Î Á Ë Ãàß Ë Ãà ß ï Çð Ë ÃÄ ÇÖ Ë ÃǑ Ù, Â Ë íõè²² Þ Þ Ë ½, ÇÖ Ë ÃǑ, ¾ ½Ã Ã Ã Ä Ǒ f l (r l (x),r 2 l (x),,r M l (x)) = w m g m (r m l (x)) m= (7) w m 0, g m ( ) Ǒ Æ ø, Ë ½. ÆðÝ Ã Ã Ú Melnik 3 ï., Ù è è²ø ÙÌ Ë ³, Ú ï );, (7) Þèá ð Æ ø, Ó è²á ûë Ç, Ú ï 2);, ¹ λ = 0, Ã f l (λu + ( λ)v) = f l (v), Ð f l (u) f l (λu + ( λ)v) f l (u) f l (v) Ý (5) ÁÝ Í Ñ. Ǒ è²ø f l (u) Þ ùþè Γ u = {u : f l (u) s}, û Þ Ë ù þ ¾Ǒ ( ). ÙÞ g m ( ) ð ø, ø ðø, (7) ù þ è²ø ðø. Õ Ù ùþ ø ð, Þ Γ u ð Ù û Ë u = (u, u 2,, u M ), v = (v, v 2,, v M ), f l (u) f l (v), Ã f l (u) λf l (u) + ( λ)f l (v) f l (λu + ( λ)v) É Ë Ǒ î, (5), Ó Ë Ǒ î, (5), (7) ùþ Ã ÃǑ Ú ï 3)., Â Ãàß Melnik Â Ãñ Þ [4], Ë íõè²² Þ Ǒ: Ë ³ î, Ë Þ è² ÞÃǑ³ ; Ë ³ î, Ë Þ è² Þ ÃǑ³ (7) ø g m ( ) (m =, 2,, M) Æ, á Ç, ø, ǑÃ èø º Ù ø ³ è, Ä ÙÝèø Ù ø, Đ ø g m ( ) ù Ǒ ǑÃ èø,» Ǒ g m (t) = t bm (8) b m 0 ðæ ù. (8) ã (7), ÆÃ ÃǑ f l (r l (x),r 2 l (x),,r M l (x)) = w m (r m l (x) bm ) m= (9) Â Ã Ã íõ : è w m (m =, 2,, M), Æ ÃǑ; è b m (m =, 2,, M), ÇÖ Ë ÃǑ MGR àß ³, àß 3 : ) ( Ã Ù 2 î); 2) Ëûþ; 3) ø Ä µ Å Ùó ùó,» Ý Melnik Â Ã,»µ Ã Ã 3 µ ǑÅ ùã Ã, Åè Ëàß ß. Ǒ Á ùþè Áø. Ã, Áø à «(Minimum squared error, MSE) Áø «(Minimum classification error, MCE) Áø. MSE Áø» ³ ß, ð Í «Þ. Þ [9] MCE Áø

4 2 ñè Î Á Ë Ãàß 279 Þ ½Å Â, ùþ MCE Áø Á 3 ù:, è, î ; Ì ø ÕÄ ; è ø ½. Ä 3 ù Áø» ð³. Paredes [0] îâ Å è ³ ß ùþ MCE Áø àß. Ì àß, Þè, üâè Ð, üâ ùþè» ø Ù üâ ÃǑ, ø ð. Ǒ, Paredes ù Ë Þî, üâú ÃǑ, Ð ÕǑ üâ ùþ ø ð. Á Þ [9] ù MCE Áø ½ 2 ù, Áø ³ ß.» Å Paredes, ù MCE Áø àß Â. ù x Ù ω q(x), (9) Ç x Ù ω q(x) Óè² S q(x) (x) = w m (rq(x)(x) m bm ) (0) m= Ç x Ù Óè² S l( x), l Λ, (Λ = {, 2,, M} {q(x)}). x «Ä Ǒ err(x) = S q(x) (x) min{s l (x) : l Λ} () Ù Ë å, err(x) = 0 ìõ Ý, err(x) ìõ: ) err(x) > 0, î x ð ; 2) err(x) < 0, î x ð. ùþè ø step(t) = + e Qt (2) Q ðè ³, Q = 00. Ìø «½, Ä, àß err (x) = + e Q(S q(x)(x) min{s l (x):l Λ}) (3) Ǒ D, Ã àß Ä Ǒ E D = x D + e Q(S q(x)(x) min{s l (x): l Λ}) (4) ǑÅ ßÇ è²ø, Á Ä ð, (4), Đ è Ä min{s l (x) : l Λ}. Ǒ, ù min{s l (x) : l Λ} = S p(x), x Ç ω q(x), Đ è è² Ô ω p(x). ½ î, ùã Ã è Ë Þ, Ð ÕǑ Þ ÃǑ è² Ë, Þ p(x) ³. (4) ßÞǑ E D = x D x D + e Q(S q(x)(x) S p(x) (x)) = + e Q M m= wm(rm q(x) (x) bm r m p(x) (x) bm ) (5) (5) ùþ Õ Ǒ Áø, Þ, ß Ã. Ǒ Á Í (6) (7) Ç E D Ù Ã ÅÓÖ ä []. Ìä íõ ä ä, óù ½ Ã. Ç Ǒµ Æ Ç è, à [2 5]. ä Çð Ý è E D = Qe Q(S q(x)(x) S p(x) (x)) w m ( + e Q(S q(x)(x) S p(x) (x)) ) 2(rm q(x) r m (x) bm p(x) ) (6) (x) bm x D E D = Qw m e Q(S q(x)(x) S p(x) (x)) b m ( + e Q(S q(x)(x) S p(x) (x)) ) q(x)(x) bm ln(rq(x)(x) m bm ) rp(x)(x) m bm ln(rp(x)(x) m bm )) 2(rm (7) x D

5 280 Þ 33 àå ÓÖ [] Â Å Ù ä Ýàß, û Åè Ù Æ² Æ Â àß; Ribaric [6] Â ÅÃ ä ä Ý ; Đ [7] Â Å Ù ³ Ýàß. ÓÖ ä, Þ äíµ ä ä ä ò ä ¹ ä è ü Å ä ä ä. Çð Ǒ ò ¹ Æð, Ùò ¹ Ç ß ù ³ «4.2 Ç ä Ǒ 3 :, ý. íõþ 3, 98 3 ä, ; íõþ 3, 98 3 ä, Ã ß; ä ý. ä Ǒ 32 32, ä Ǒ Þ ä Ç Ä àß (Principal component analysis, PCA) Ç (Linear discrimination analysis, LDA) ½ á [8], ÞǑ 32 á Æ, Ä ½ Þ Ç ( ), ø 6. Ù û, Þ Ç Ì Þ Ð, û Ð Ë, è Ë, ÂÒ è Ë Å., àß àß MGR ½Å³ Ä Ò Ë Å Ã ß, ý Ä Ò Ë Å Ã ß ý. àß Ä, ß Ëã Ǒ 00, w m Ǒ /M, b m Ǒ. à ß MGR àß, Þ [2], Ë Ǒ 0. ǑÅ³ Ãàß Í, ½Å Ǒ 2 3 ìõ Ð 6 ü 2 ½Ã, 5 ìõ, ½ 5 ; Ã 3, ½ 20. Çàß ³ û Û Ã ß Í 4.3 Â ½Å Ç, ä, Ù îñ,, ³ Ô, Ä Table Ä Classification error rates of the base classifiers Ù PCA (%) Ù LDA (%) ½è ðã, àß àß MGR ½³. Ð 6 üâ, ½ 5, Å, Ä 2. Û MGR àß Ù à ß, ðí ³. àß ³ MGR àß Å Þ. àß 4 Û Ù àß MGR àß, è, ÔÙ MGR àß. Table 2 Ã Fig. Fusion experiments of two classifiers Ä 2 Ã î Average classification error rates of fusion of two classifiers (%) MGR(%) àß (%) ½ ðã 3, àß àß MGR àß ½³., àß î Í Ä àß. ǑÅ Ù, ó Ý Ǒ ³. ó ½ ³, Ð 6 üâ 3, ü Â 3 2 è, ½ 2, Â 2. ½, Ð 6 üâ 3

6 2 ñè Î Á Ë Ãàß 28, ½ 8, Â 3. Ã Ò Ä 3. Ð Û Ç Åè, Ã ß ÂÔ. ÑĐ ³³ î, àß Ùàß, ðí. Ñð ³ î, àß Þè Ù MGR àß àß. Fig. 2 2 ÃÒ ³ Fusion experiments of three correlated classifiers 3 ÃÒ Fig. 3 Fusion experiments of three independent classifiers Table 3 Ä 3 ÃÒ î Average classification error rates of fusion of three classifiers (%) MGR(%) àß (%) ³ ½Ò ðã 6. Ù î Â MGR àß, Ã Ç 2 6 = 63, Ã ÃÄÙ, ó àß àß ½Å³. Ä 4 Å àß. Û, Ã ß Õ èù, àß Ù àß. Table 4 Ä 4 Ã 6 Classification error rates of fusion of six classifiers (%) àß (%) ½Å, Â Û àß ÔÙ Ë Ã àß Melnik Â Ãàß. ð Ù : ) àß è Ò, ÍÅ» Ý Ë Ã Á; 2) àßá è Þ ß, Í ÅÐ Ë ó Ã, Ñ ³ ê ß, àß à Û ÄÅ àß MGR àß Ã 3, Ñð ³ î, àß Û Ù àß MGR àß, ð ³ î, àß àß «Û ç èý æ ð Ù 3 è è î, Ñ«Ç ÃǑÇ Å, Ë «ÃǑ Å ì õ, Î Á Ë Ãàß Å Ã 6, ÙÞ µ, î Ñ «Ç ÃǑ Å, Ë «ÃǑ, àß ÕÇ Å 5 Melnik Ã Â ÅÎ Á Ã Î, û Åè Ë Ãàß. Ìàß Ë ê ÆÆ, û è Þàß., ÇÅÃ ä ä ä Ç, ½Å ì õ. ÂÄ, àß Ë Ãàß Melnik Ãàß ³ ËÍ Ô. References Xu L, Krzyzak A, Suen C Y. Methods for combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics, 992, 22(3):

282 Þ 33 2 Ho T K, Hull J J, Srihari S N. Decision combination in multiple classifier systems.

Pattern Recognition, 200, 34(4): 865 884 4 Melnik O, Vardi Y, Zhang C H. Mixed group ranks: preference and confidence in classifier combination.

Pattern Recognition, 2005, 38(): 28 6 Kittler J. A framework for classifier fusion: is it still needed? Lecture Notes in Computer Science, 2000, 876: 45 7 Lin X F, Ding X Q, Chen M, Zhang R, Wu Y S.

7 282 Þ 33 2 Ho T K, Hull J J, Srihari S N. Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 994, 6(): Saranli A, Demirekler M. A statistical unified framework for rank-based multiple classifier decision combination. Pattern Recognition, 200, 34(4): Melnik O, Vardi Y, Zhang C H. Mixed group ranks: preference and confidence in classifier combination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(8): Liu C L. Classifier combination based on confidence transformation. Pattern Recognition, 2005, 38(): 28 6 Kittler J. A framework for classifier fusion: is it still needed? Lecture Notes in Computer Science, 2000, 876: 45 7 Lin X F, Ding X Q, Chen M, Zhang R, Wu Y S. Adaptive confidence transform based classifier combination for Chinese character recognition. Pattern Recognition Letters, 998, 9(0): Artukorale A S, Suganthan P N. Combining classifiers based on confidence values. In: Proceedings of the 5th International Conference on Document Analysis and Recognition. Bangalore, India: IEEE, Juang B H, Katagiri S. Discriminative learning for minimum error classification. IEEE Transactions on Signal Processing, 992, 40(2): Paredes R, Vidal E. Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(7): 00 0 Li Q, Qiu Z D, Sun D M. Personal identification using knuckleprint. Lecture Notes in Computer Science, 2004, 3338: Han C C, Cheng H L, Lin C L, Fan K C. Personal authentication using palm-print features. Pattern Recognition, 2003, 36(2): Li Qiang, Qiu Zheng-Ding, Sun Dong-Mei, Liu Lu-Lu. Online palmprint identification based on improved 2D PCA. Acta Electronica Sinica, 2005, 33(0): (ÓÖ, ù, êýö, öö. ÙÍ á Ç., 2005, 33(0): ) 4 Yuan Guo-Wu, Wei Xiao-Yong, Xu Dan. Palmprint-based personal verification. Journal of Computer-aided Design and Computer Graphics, 2005, 7(2): ( Á, ïì, Å. Ù Ç. Ç Ç», 2005, 7(2): ) 5 Wu Xiang-Qian. Palmprint Feature Analysis for Personal Authentification [Ph. D. dissertation], Harbin Institute of Technology, 2004 ( å. å Ç Æ [ ], Å Ì â, 2004) 6 Ribaric S, Fratric I. A biometric identifition system based on eigenpalm and eigenfinger features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(): Wang Chang-Yu, Song Shang-Ling, Sun Feng-Rong, Mei Liang-Mo. A novel biometrics technology-finger-back articular skin texture recognition. Acta Automatica Sinica, 2006, 32(3): ( Đ, Ý, êðâ, Ö. è µ Æ Ç - ³ é Ç. Þ, 2006, 32(3): ) 8 Liu Qing-Shan, Lu Han-Qing, Ma Song-De. A survey: subspace analysis for face recognition. Acta Automatica Sinica, 2003, 29(6): ( ã, è, Û. Ò Ç èñàß. Þ, 2003, 29(6): 900 9) µ. àåǒ µ Æ Ç, ä Ò. ². liuming@hbu.edu.cn (LIU Ming Ph. D. candidate at Beijing Jiaotong University. His research interest covers multi-modal biometrics and image processing. Corresponding author of this paper.) Ñ. àåǒç ü Â Ý ä Ò Ç» ² Ò, ÜÆ² Ò ². bzyuan@center.njtu.edu.cn (YUAN Bao-Zong Professor. His research interest covers digital signal processing, speech signal processing, image processing, computer vision, computer graphics, multimedia information processing, and data communication.) Ë ÙßÁ Á Ò ², àåǒ Ô, óç, Â Ý, ä Ò. zjmiao@bjtu.edu.cn (MIAO Zhen-Jiang Professor at Beijing Jiaotong University. From 994 to 2004, he worked at French Institut National Polytechnique de Toulouse and National Research Council of Canada. His research interest covers human-computer interaction, pervasive computing, virtual reality, and image and speech processing.)

â, Đ (Very Long Baseline Interferometry, VLBI)

â, Đ (Very Long Baseline Interferometry, VLBI) ½ 55 ½ 5 Í Vol.55 No.5 2014 9 ACTA ASTRONOMICA SINICA Sep., 2014» Á Çý è 1,2 1,2 å 1,2 Ü ô 1,2 ï 1,2 ï 1,2 à 1,3 Æ Ö 3 ý (1 Á Í 200030) (2 Á Í û À 210008) (3 541004) ÇÅè 1.5 GHz Á è, î Í, û ÓÆ Å ò ½Ò ¼ï.