Random Sampling Based SVM for Relevance Feedback Image Retrieval

Transcription

1 Random Samplng Based or Relevance Feedback Image Retreval Dacheng Tao and Xaoou Tang Department o Inormaton Engneerng The Chnese Unversty o Hong Kong {dctao, xtang}@e.cuhk.edu.hk Abstract Relevance eedback (RF) schemes based on support vector machne () have been wdely used n content-based mage retreval. However, the perormance o based RF s oten poor when the number o labeled postve eedback samples s small. Ths s manly due to three reasons:. classer s unstable on small sze tranng set;. s optmal hyper-plane may be based when the postve eedback samples are much less than the negatve eedback samples; 3. overttng due to that the eature dmenson s much hgher than the sze o the tranng set. In ths paper, we try to use random samplng technques to overcome these problems. To address the rst two problems, we propose an asymmetrc baggng based. For the thrd problem, we combne the random subspace method (RSM) and or RF. Fnally, by ntegratng baggng and RSM, we solve all the three problems and urther mprove the RF perormance.. Introducton Relevance eedback (RF) [] s an mportant tool to mprove the perormance o content-based mage retreval (CBIR) []. In a RF process, the user rst labels a number o relevant retreval results as postve eedbacks and some rrelevant retreval results as negatve eedbacks. Then the system renes all retreval results based on these eedbacks. The two steps are carred out teratvely to mprove the perormance o mage retreval system by gradually learnng the user s percepton. Many RF methods have been developed n recent years. One approach [] adjusts the weghts o varous eatures to adapt to the user s percepton. Another approach [3] estmates the densty o the postve eedback examples. Dscrmnant learnng has also been used as a eature selecton method or RF [4]. These methods all have certan lmtatons. The method n [] s only heurstc based. The densty estmaton method n [3] loses normaton contaned n negatve samples. The dscrmnant learnng n [4] oten suers rom the matrx sngular problem. Recently, classcaton-based RF [5-7] becomes a popular technque n CBIR and the Support Vector Machne () based RF (RF) has shown promsng results owng to ts good generalzaton ablty. However, when the number o postve eedbacks s small, the perormance o RF becomes poor. Ths s manly due to the ollowng reasons. Frst, classer s unstable or small sze tranng set,.e. the optmal hyper-plane o s senstve to the tranng samples when the sze o the tranng set s small. In RF, the optmal hyperplane s determned by the eedbacks. However, more oten than not the users would only label a ew mages and cannot label each eedback accurately all the tme. Hence the perormance o the system may be poor wth the nexactly labeled samples. Second, n the RF process there are usually much more negatve eedback samples than postve ones. Because o the mbalance o the tranng samples or the two classes, s optmal hyper-plane wll be based toward the negatve eedback samples. Consequently, RF may mstake many query rrelevant mages as relevant. Fnally, n RF, the sze o the tranng set s much smaller than the dmenson o the eature vector, thus may cause the over ttng problem. Because o the exstence o nose, some eatures can only dscrmnant the postve and negatve eedbacks but cannot dscrmnant the relevant or rrelevant mages n the database. So the learned classer cannot work well or the remanng mages n the database. In order to overcome these problems, we desgn several new algorthms to mprove the based RF or CBIR. The key dea comes rom the Classer Commttee Learnng (CCL) [8-0]. Snce each classer has ts own unque ablty to classy relevant and rrelevant samples, the CCL can pool a number o weak classers to mprove the recognton perormance. We use baggng and random subspace method to mprove the snce they are especally eectve when the orgnal classer s not very stable /04 $ IEEE

2 . n CBIR RF [,] s a very eectve bnary classcaton algorthm. Consder a lnearly separable bnary classcaton problem: x y and = { +, } = {(, )} N y, () where x s an n-dmenson vector and y s the label o the class that the vector belongs to. separates the two classes o ponts by a hyper-plane, w T x + b = 0, () where x s an nput vector, w s an adaptve weght vector, and b s a bas. nds the parameters w and b or the optmal hyper-plane to maxmze the geometrc margn w, subject to y T ( + b) + wx. (3) The soluton can be ound through a Wole dual problem wth Lagrangan multple α : m m jyyj j =, j= m αy = 0. = Q( α) = α αα ( x x ), (4) subject to α 0 and In the dual ormat, the data ponts only appear n the nner product. To get a potentally better representaton o the data, the data ponts are mapped nto the Hlbert Inner Product space through a replacement: x x j φ( x) φ( x j) = K( x, x j), (5) where K (). s a kernel uncton. We then get the kernel verson o the Wole dual problem: m m Q( α) = α αα dd K( x x ). (6) j j j =, j= Thus or a gven kernel uncton, the classer s gven by F( x) = sgn( ( x )), (7) where l ( x) = αyk ( x, x ) + b s the output hyperplane = decson uncton o the. In general, when ( x ) or a gven pattern s hgh, the correspondng predcton condence wll be hgh. On the contrary, a low ( x ) o a gven pattern means the pattern s close to the decson boundary and ts correspondng predcton condence wll be low. Consequently, the output o, ( x ) has been used to measure the dssmlarty [5,6] between a gven pattern and the query mage, n tradtonal based CBIR RF. 3. CCL or s To address the three problems o RF descrbed n the ntroducton, we propose three algorthms n ths secton. 3.. Asymmetrc Baggng Baggng [8] strategy ncorporates the benets o bootstrappng and aggregaton. Multple classers can be generated by tranng on multple sets o samples that are produced by bootstrappng,.e. random samplng wth replacement on the tranng samples. Aggregaton o the generated classers can then be mplemented by majorty votng rule (MVR) [0]. Expermental and theoretcal results have shown that baggng can mprove a good but unstable classer sgncantly [8]. Ths s exactly the case o the rst problem o based RF. However, drectly usng Baggng n RF s not approprate snce we have only a very small number o postve eedback samples. To overcome ths problem we develop a novel asymmetrc Baggng strategy. The bootstrappng s executed only on the negatve eedbacks, snce there are ar more negatve eedbacks than the postve eedbacks. Ths way each generated classer wll be traned on a balanced number o postve and negatve samples, thus solvng the second problem as well. The Asymmetrc Baggng (AB) algorthm s descrbed n Table. Table : Algorthm o Asymmetrc Baggng. + Input: postve tranng set S, negatve tranng set S, weak classer I (), nteger T (number o generated classers), x s the test sample.. For = to T {. S = bootstrap sample rom S, wth S = S. 3. C = I( S, S ) 4. } 5. C ( x) = aggregaton{ C( x, S, S ), T}. Output: classer C. In AB, the aggregaton s mplemented by Majorty Votng Rule (MVR). The asymmetrc Baggng strategy solves the classer unstable problem and the tranng set unbalance problem. However, t cannot solve the small sample sze problem. We wll solve t by the Random Subspace Method (RSM) n the next secton /04 $ IEEE

3 3.. Random Subspace Method Smlar to Baggng, RSM [9] also benets rom the bootstrappng and aggregaton. However, unlke Baggng that bootstrappng tranng samples, RSM perorms the bootstrappng n the eature space. For based RF, over ttng happens when the tranng set s relatvely small compared to the hgh dmensonalty o the eature vector. In order to avod over ttng, we sample a small subset o eatures to reduce the dscrepancy between the tranng data sze and the eature vector length. Usng such a random samplng method, we construct a multple number o s ree o over ttng problem. We then combne these s to construct a more powerul classer. Thus the over ttng problem s solved. The RSM based (R) algorthm s descrbed n Table. Table : Algorthm o RSM. Input: eature set F, weak classer I (), nteger T (number o generated classers), x s the test sample.. For = to T {. F = bootstrap eature rom F. 3. C = I( F ) 4. } 5. C ( x) = aggregaton{ C( x, F ), T}. Output: classer C Asymmetrc Baggng RSM Snce the asymmetrc Baggng method can overcome the rst two problems o RF and the RSM can overcome the thrd problem o the RF, we should be able to ntegrate the two methods to solve all the three problems together. So we propose an Asymmetrc Baggng RSM (ABR) to combne the two. The algorthm s descrbed n Table 3. In order to explan why Baggng RSM strategy works, we derve the proo ollowng a smlar dscusson on Baggng n [8]. y, x be a data sample n the tranng set L Let ( ) wth eature vector F, where y s the class label o the sample x. L s drawn rom the probablty dstrbuton P. Suppose ϕ ( x, L, F ) s the smple predctor (classer) constructed by the Baggng RSM strategy, and the aggregated predctor s ϕ x, P = E E ϕ x, L, F. ( ) ( ) Let random varables ( Y, X ) be drawn rom the A F L dstrbuton P ndependent o the tranng set L. The ϕ x, L, F, s average predctor error, estmated by ( ) ( ( )) ea = EFELEY, X Y ϕ X, L, F. The correspondng error estmated by the aggregated predctor s e ( ( )) A = EY, X Y ϕ A X, P. (8) M N M N Usng the nequalty ( zj ) zj M j N = = M j= N, = we have: E (,, ) ( (,, )) FELϕ X L F EFELϕ X L F (9) E ( ) Y, XEE F Lϕ X, LF, EY, XϕA( X, P) (0) Thus, ea = EY, XY EY, XYϕA + EY, XEFELϕ ( X, L, F). () EY, X( Y ϕ A) = ea Thereore, the predcted error o the aggregated method s reduced. From the nequalty, we can see ϕ x, L, F, the more that the more dverse s the ( ) accurate s the aggregated predctor. In CBIR RF, the classer s unstable both or the tranng eatures and the tranng samples. Consequently, the Baggng RSM strategy can mprove the perormance. Here we made an assumpton that the average perormance o all the ndvdual classer ϕ ( x, L, F ), traned on a subset o eature and tranng set replca s smlar to a classer, whch use the ull eature set and the whole subset tranng set. Ths can be true when the sze o eature and tranng data subset s adequate to approxmate the ull set dstrbuton. Even when ths s not true, the drop o accuracy or each smple classer may be well compensated n the aggregaton process. Table 3: Algorthm o Asymmetrc Baggng RSM. Input: postve tranng set S +, negatve tranng set S, eature set F, weak classer I (), nteger Ts (number o Baggng classers), nteger T (number o RSM classers), x s the test sample.. For j = to T s {. S j = bootstrap sample rom S. 3. or = to T { 4. F = bootstrap sample rom F. 5. C, j= I( F, S j, S ). 6. } 7. } (,,, ) Cj xf S j S 8. C ( x) = aggregaton T, j Ts Output: classer C /04 $ IEEE

4 Snce Baggng RSM strategy can generate more dversed classers than usng Baggng or RSM alone, t should outperorm the two. In order to acheve maxmum dversty, we choose to combne all generated classers n parallel as shown n Fgure. Ths s better than combnng Baggng or RSM rst then combng RSM or Baggng. are quantzed nto 8, 8, and 4 bns respectvely. Texture s extracted rom Y component n the YCrCb space by pyramd wavelet transorm (PWT) wth Haar wavelet. The mean value and standard devaton are calculated or each sub-band at each decomposton level. The eature length s 4 3. For shape eature, edge hstogram [4] s calculated on Y component n the YCrCb color space. Edges are grouped nto our categores, horzontal, 45 dagonal, vertcal, and 35 dagonal. We combne the color, texture, and shape eatures nto a eature vector, and then we normalze each eature to a normal dstrbuton. Fgure. Aggregaton structure o the ABR. For a gven test sample, we rst recognze t by all T T weak classers: s { C C( F, S ) T, j T} j j s =. () Then, an aggregaton rule s used to ntegrate all the results rom the weak classers or nal classcaton o the sample as relevant or rrelevant. 3.4 Dssmlarty Measure For a gven sample, we rst use the MVR to recognze t as query relevant or rrelevant. Then we measure the dssmlarty between the sample and the query as the output o the ndvdual classer, whch gves the same label as the MVR and produces the hghest condence value (the absolute value o the decson uncton o the classer). 4. Image Retreval System To evaluate the perormance o the proposed algorthms, we develop the ollowng general CBIR system wth RF. In the system, we can use any RF algorthm or the block Relevance Feedback Model. In Fgure, when a query mage s nput, the lowlevel eatures are extracted. Then, all mages n the database are sorted based on a smlarty metrc (here we use Eucldean dstance). The user labels some top mages as postve and negatve eedbacks. Usng these eedbacks, a RF model s traned based on a. Then the smlarty metrc s updated based on the RF model. All mages are resorted by the updated smlarty metrc. The RF procedure wll be teratvely executed untl the user s satsed wth the outcome. In our retreval system, three man eatures, color, texture, and shape are extracted to represent the mage. For color eature, we use the color hstogram [3] n HSV color space. Here, the color hstogram s quantzed nto 56 levels. Hue, Saturaton and Value Fgure. Flowchart o the mage retreval system. 5. Expermental Results In ths secton, we compare the new algorthms wth exstng algorthms through experments on 7, 800 mages o 90 concepts rom the Corel Photo Gallery. The experments are smulated by a computer automatcally. Frst, 300 queres are randomly selected rom the data, and then RF s automatcally done by the computer: all query relevant mages (.e. mages o the same concept as the query) are marked as postve eedbacks n the top 40 mages and all the other mages are marked as negatve eedbacks. In general, we have about 5 mages as postve eedbacks. The procedure s close to the real crcumstances, because the user typcally would not lke to clck on the negatve eedbacks. Thus requrng the user to mark only the postve eedbacks n top 40 mages s reasonable. In ths paper, precson and standard devaton (SD) are used to evaluate the perormance o a RF algorthm. Precson s the percentage o relevant mages n the top N retreved mages. The precson curve s the averaged precson values o the 300 queres, and SD curve s the SD values o the 300 queres precson values. The precson curve evaluates the eectveness o a gven algorthm and SD curve evaluates the robustness o the algorthm. In the precson and SD /04 $ IEEE

5 curves 0 eedback reers to the retreval based on Eucldean dstance measure wthout RF. We compare all the proposed algorthms wth the orgnal based RF [5] and the constraned smlarty measure (C) based RF [7]. We K, e ρ x = y xy wth ρ = chose the Gaussan kernel ( ) (the deault value n the OSU- [5] MatLab TM toolbox) or all the algorthms. The perormances o all the algorthms are stable over a range o ρ. Precson Precson Vs. Number o s Top Number o s 5.. Perormance o Asymmetrc Baggng 5 Standard Devaton Vs. Number o s Fgure 3 shows the precson and SD values when usng derent number o s n AB. The results show that the number o s wll not aect the perormance o the asymmetrc Baggng method. Standard Devaton Top 0 Precson Standard Devaton Precson Vs. Number o s Top Number o s Standard Devaton Vs. Number o s Top Number o s Fgure 3. Asymmetrc Baggng based RF. The second experment compares the perormances o the AB usng 5 weak classers and the standard and C based RF. The expermental results are shown n Fgure 5. From these experments, we can see that 5 weak s are enough or AB, and AB clearly outperorms the and C. The precson curve o AB s hgher than that o and C, and SD curve s lower than that o and C. 5.. Perormance o RSM Fgure 4 shows the precson and SD values when usng derent number o s o R. The results show that the number o s does not aect the perormance o R Number o s Fgure 4. RSM based RF. The second experment compares the R usng 5 weak classers wth the standard and C based RF. The expermental results are also shown n Fgure 5. The results demonstrate that 5 weak s are enough or the R, and R outperorms the and C Perormance o Asymmetrc Baggng RSM Ths experment evaluates the perormance o the proposed ABR, AB, and R based RF. In ths experment, we chose T s = 5 or AB, T = 5 or R, and Ts = T = 5 or ABR. The results n Fgure 5 show that the ABR gves the best perormance ollowed by R then AB. They all outperorm and C Computatonal Complexty To very the ecency o the proposed algorthms, we record the computatonal tme when conductng the experments. The rato or the tme used by derent methods are : C: AB: R: ABR = 5: 5: : 3: 5. Ths s show that the new based algorthms are much more ecent than exstng based algorthms /04 $ IEEE

6 6. Concluson In ths paper, we desgn a new Asymmetrc Baggng Random Subspace Method or based RF. The proposed algorthm can address the classer unstable problem, the unbalanced tranng set problem, and the small sample sze problem eectvely. Extensve experments on a Corel Photo database wth 7, 800 mages show that the new algorthm can mprove the perormance o relevance eedback sgncantly. 7. Acknowledgement The work descrbed n ths paper was ully supported by a grant rom the Research Grants Councl o the Hong Kong SAR. (Project no. AoE/E-0/99). 8. Reerences [] Y. Ru, T. S. Huang, and S. Mehrotra. Content-based mage retreval wth relevance eedback n MARS, In Proc. IEEE ICIP, 997. [] A.W.M. Smeulders, M. Worrng, S. Santn, A. Gupta, and R. Jan, Content-based mage retreval at the end o the early years, IEEE Trans. on PAMI, vol., no., pp , Dec [3] Y. Chen, X. Zhou, and T. S. Huang, One-class or learnng n mage retreval, In Proc. IEEE ICIP, 00. Precson 0.9 Retreval precson n top0 results 0.5 ABR RSM AB C Number o teratons Retreval precson n top60 results Standard devaton [4] X. Zhou and T. S. Huang, Small sample learnng durng multmeda retreval usng basmap, In Proc. IEEE CVPR, 00. [5] L. Zhang, F. Ln, and B. Zhang, Support vector machne learnng or mage retreval, In Proc. IEEE ICIP, 00. [6] P. Hong, Q. Tan, and T. S. Huang, Incorporate Support Vector Machnes to Content-based Image Retreval wth Relevant Feedback, In Proc. IEEE ICIP, 000. [7] G. Guo, A. K. Jan, W. Ma, and H. Zhang, Learnng smlarty measure or natural mage retreval wth relevance eedback, IEEE Trans. on NN, vol., no. 4, pp.8-80, July 00. [8] L. Breman, Baggng Predctors, Int. J. on Machne Learnng, no. 4, pp 3-40, 996. [9] T. K. Ho, The Random Subspace Method or Constructng Decson Forests, IEEE Trans. On PAMI. vol. 0, no. 8, pp , Aug [0] J. Kttler, M. Hate, P.W. Dun, and J. Matas, On Combnng Classers, IEEE Trans. On PAMI. Vol. 0, no. 3, pp. 6-39, Mar [] Vapnk, V.: The Nature o Statstcal Learnng Theory. Sprnger-Verlag, New York (995). [] J. C. Burges, A Tutoral on Support Vector Machnes or Pattern Recognton, Int. J. on. DMKD, pp.-67, 998. [3] M.J. Swan and D.H. Ballard, Color ndexng, IJCV, 7():--3, 99. [4] B.S Manjunath, J. Ohm, V. Vasudevan, and A. Yamada, Color and texture descrptors, IEEE Trans. on CSVT, Vol., pp , Jun 00. [5] Retreval standard devaton n top0 results 0.5 ABR RSM 0. AB C Number o teratons Retreval standard devaton n top60 results 5 Precson 0.5 ABR RSM AB C Number o teratons Standard devaton 5 5 ABR RSM AB 0.5 C Number o teratons Fgure 5. Perormance o all proposed algorthms compared to exstng algorthms. The algorthms are evaluated over 9 teratons /04 $ IEEE