Coectve Networ of Evoutonry Bnry Cssfers for Content-Bsed Imge Retrev Sern Krnyz, Stefn Uhmnn, Jenn Punen nd Moncef Gbbouj Dept. of Sgn Processng Tmpere Unversty of Technoogy Tmpere, Fnnd {frstnme.stnme}@tut.f Abstrct The content-bsed mge retrev (CBIR) hs been n ctve reserch fed for whch sever feture extrcton, cssfcton nd retrev technques hve been proposed up to dte. However, when the dtbse sze grows rger, t s common fct tht the over retrev performnce sgnfcnty deterortes. In ths pper, we propose coectve networ of (evoutonry) bnry cssfers (CNBC) frmewor to cheve hgh retrev performnce even though the trnng (ground truth) dt my not be entrey present from the begnnng nd thus the system cn ony be trned ncrementy. The CNBC frmewor bscy dopts Dvde nd Conquer type pproch by octng sever networs of bnry cssfers (NBCs) to dscrmnte ech css nd performs evoutonry serch to fnd the optm bnry cssfer (BC) n ech NBC. In such n evouton sesson, the CNBC body cn further dynmcy dpt tsef wth ech new ncomng css/feture set wthout fu-sce re-trnng or re-confgurton. Both vsu nd numerc performnce evutons of the proposed frmewor over benchmr mge dtbses demonstrte ts scbty; nd sgnfcnt performnce mprovement s cheved over trdton retrev technques. Keywords- evoutonry cssfers, content-bsed mge retrev, mut-dmenson prtce swrm optmzton I. INTRODUCTION For content-bsed mge cssfcton nd retrev, the ey questons, e.g. ) how to seect certn fetures so s to cheve hghest dscrmnton over certn csses, 2) how to combne them n the most effectve wy, 3) whch dstnce metrc to ppy, 4) how to fnd the optm cssfer confgurton for the cssfcton probem n hnd, 5) how to sce/dpt the cssfer f rge number of csses/fetures re ncrementy ntroduced nd fny, 6) how to trn the cssfer effcenty to mze the cssfcton ccurcy, st remn unnswered. The current stte-of-the-rt cssfers such s SVMs, Byesn, Artfc Neur Networs (ANNs), etc. cnnot cope wth such requrements snce snge cssfer, no mtter how powerfu nd we-trned t my be, cnnot dscrmnte effcenty vst mount of csses, over n ndefntey rge set of fetures. Furthermore, snce both csses nd fetures re not sttc, rther dynmcy vryng, s ntur consequence of mge repostores, sttc nd fxed-structured snge cssfers cnnot sce such chnges wthout proper confgurton updtes nd fu-sce re-trnng. In order to ddress these probems nd hence to mze the cssfcton ccurcy whch w n turn boost the retrev performnce, n ths pper, we sh focus on gob frmewor desgn tht embodes coectve networs of evoutonry cssfers. Specfcy n ths pproch, the foowng objectves w be trgeted: Turer Ince Fcuty of Computer Scence Izmr Unversty of Economcs Izmr, Turey Em: turer.nce@eu.edu.tr I. Evoutonry Serch: Seeng for the optmum networ rchtecture mong coecton of confgurtons (the so-ced Archtecture Spce, AS). II. Evoutonry Updte n the AS: Keepng ony the best ndvdu confgurton n the AS mong ndefnte number of evouton runs. III. Feture Scbty: Support for vryng number of fetures. Any feture cn be dynmcy ntegrted nto the frmewor wthout requrng fu-sce ntzton nd re-evouton. IV. Css Scbty: Support for vryng number of csses. Any css cn dynmcy be nserted nto the frmewor wthout requrng re-evouton. V. Hgh effcency for the evouton (or trnng) process: Usng s compct nd smpe cssfers s possbe n the AS. VI. Onne (ncrement) Evouton: Contnuous onne/ncrement trnng (or evouton) sessons cn be performed to mprove the cssfcton ccurcy. VII. Pre processng: Cssfers cn be evoved usng sever processors worng n pre. In ths wy, we sh cheve s compct cssfers s possbe, whch cn be evoved nd trned n much more effcent wy thn snge but compex cssfer, nd the optmum cssfer for the cssfcton probem n hnd cn be serched wth n underyng evoutonry technque, e.g. s n []. At gven tme, ths ows creton nd desgnton of dedcted cssfer for dscrmntng certn css type from the others bsed on snge feture. Ech ncrement evouton sesson w ern from the current best cssfer confgurtons nd cn mprove them further, possby s resut of n (ncrement) optmzton, whch my fnd nother confgurton n the rchtecture spce (AS) s the optm. Moreover, wth ech ncrement evouton, new csses/fetures cn so be ntroduced whch sgns the coectve cssfer networ to crete new correspondng networs nd cssfers wthn to dpt dynmcy to the chnge. In ths wy the coectve cssfer networ w be be to dynmcy sce tsef to the ndexng requrements of the mge dtbse whst strvng for mzng the cssfcton nd retrev ccurces thns to the dedcted cssfers wthn. In order to cheve these objectves, we dopt Dvde nd Conquer type of pproch, whch s bsed on nove frmewor encpsutng networ of (evoutonry) bnry cssfers (NBCs). Ech NBC s devoted to unque css nd further encpsutes set of evoutonry Bnry Cssfers (BCs), ech of whch s optmy chosen wthn the AS, dscrmntng the css of the NBC wth unque feture set (or sub-feture). The optmty theren cn be set wth user-defned crteron. The proposed Coectve NBC (CNBC) frmewor currenty supports two common ANN types, the Mut-Lyer Perceptrons (MLPs) nd the Rd Bss Functon (RBF) networs. Besdes the exhustve serch wth the numerous runs of the Bc-Propgton method, the recenty proposed mut-dmenson Prtce Swrm 978--4244-9977-9//$26.00 20 IEEE 47
Optmzton (MD-PSO) [2], [] s used s the prmry evouton technque. In the current wor, due to spce mttons we sh restrct ony on the CNBC desgn wth evoutonry MLPs. The rest of the pper s orgnzed s foows. Secton II presents evoutonry rtfc neur networs. The proposed CNBC frmewor ong wth the evoutonry updte mechnsm s expned n det n Secton III. Secton IV provdes n extensve set of cssfcton nd retrev resuts over two benchmr mge repostores ong wth evutons of the proposed ncrement CNBC evouton. Fny, Secton V concudes the pper nd dscusses topcs for future wor. II. EVOLUTIONARY NEURAL NETWORKS In ths secton we sh dscuss the methodoogy for chevng the frst objectve tht s the evoutonry serch for the optm cssfer confgurton. Frst the evoutonry technque, MD-PSO, w be brefy expned nd then ts ppcton over feed-forwrd ANNs (the MLPs) sh be ntroduced. A. Mut-Dmenson Prtce Swrm Optmzton As the evoutonry method, we sh use the mut-dmenson (MD) extenson of the bsc PSO (bpso) method, the MD-PSO, recenty proposed n [2]. Insted of opertng t fxed dmenson N, the MD- PSO gorthm s desgned to see both poston nd dmenson optm wthn dmenson rnge, { D mn, D}. In order to ccompsh ths, ech prtce hs two sets of components, ech of whch hs been subjected to one of the two ndependent nd consecutve processes. The frst one s regur poston PSO,.e. the trdton veocty updtes nd due poston shfts n N dmenson serch (souton) spce. The second one s dmenson PSO, whch ows the prtce to nvgte through dmensons. Accordngy, ech prtce eeps trc of ts st poston, veocty nd person best poston (pbest) n prtcur dmenson so tht when t re-vsts the sme dmenson t ter tme, t cn perform ts regur poston updte usng ths nformton. The dmenson PSO process of ech prtce my then move the prtce to nother dmenson where t w remember ts poston sttus nd w be updted wthn the poston PSO process t ths dmenson, nd so on. The swrm, on the other hnd, eeps trc of the gbest prtce n ech dmenson, ndctng the best (gob) poston so fr cheved. Smry, the dmenson PSO process of ech prtce uses ts person best dmenson n whch the person best ftness score hs so fr been cheved. Fny, the swrm eeps trc of the gob best dmenson, dbest, mong the person best dmensons. The gbest prtce n the dbest dmenson represents the optmum souton nd dmenson, respectvey. In MD-PSO process t tme (terton) t, ech prtce n the swrm wth S prtces, ξ = { x,.., x,.., x S }, s represented by the foowng chrcterstcs: xd ( t ) xx, j : j th component (dmenson) of the poston of prtce, n dmenson xd xd vx, j : j th component (dmenson) of the veocty of prtce, n dmenson xd xd xy, j : j th component (dmenson) of the person best poston of prtce, n dmenson xd gbest(d) : Gob best prtce ndex n dmenson d xyˆ d j : j th component (dmenson) of the gob best poston of swrm, n dmenson d xd : Dmenson of prtce vd : Dmenson veocty of prtce ~ xd ( t ): Person best dmenson component of prtce Let f denote the ftness functon tht s to be optmzed wthn certn dmenson rnge, { D, D mn }. Wthout oss of generty ssume tht the objectve s to fnd the mnmum of f t the optmum dmenson wthn mut-dmenson serch spce. Assume tht the prtce vsts (bc) the sme dmenson fter T tertons (.e. xd = xd( t + T) ), then the person best poston cn be updted n terton t+t s foows, xd xd ( t+ T) xd xd ( t T) xy, j f f ( xx ( t T)) f ( xy ) + + > xy, j ( t + T) = xd ( t+ T) xx, j ( t + T) ese () j =,2,..., xd( t + T) Furthermore, the person best dmenson of prtce cn be updted n terton t+ s foows, ~ ~ xd ( t+ ) xd ~ xd t f f xx t + > f xy t ( ) ( ( )) ( ( )) xd ( t + ) = (2) xd ( t + ) ese {MD PSO Prtce ( Dmenson ) ( Poston ) xx 2 : 2 vx 2 : 2 xy 2 : 2 xd xx 3 : 2 3 vx 3 : 2 3 vd xy 3 : 2 3 ~ xd ( t ) xx 9 : 2 3 4 5 7 vx 9 : 2 3 4 5 6 7 8 9 xy 9 : 2 3 4 5 6 7 8 9 } 6 8 9 Fgure : Smpe MD-PSO (eft) vs. bpso (rght) prtce structures. For MD-PSO { D mn = 2, D= 9} nd t tme t, ~ xd = 2 nd xd = 3. Fgure shows smpe MD-PSO nd bpso prtces denoted s. Prtce n bpso prtce s t (fxed) dmenson, N=5, nd contns ony poston components; wheres n MD-PSO prtce contns both poston nd dmenson components, respectvey. In the fgure the dmenson rnge for MD-PSO s gven by { D mn, D} = {2, 9}, therefore the prtce contns 9 sets of poston components. In ths exmpe the prtce currenty resdes t dmenson 2 ( xd = 2 ); wheres ts person best dmenson s 3 ~ ( xd = 3 ). Therefore, t tme t poston PSO updte s frst performed over the poston eements, xx 2 ( t ) nd then the prtce my move to nother dmenson wth respect to the dmenson PSO. Rec tht ech poston eement, xx 2, represents potent souton n the serch spce of the probem. B. MD-PSO for Evovng MLPs As stochstc serch process n mut-dmenson serch spce, MD-PSO sees (ner-) optm networs n n rchtecture spce (AS), whch cn be defned over ny type of ANNs wth ny propertes. A networ confgurtons n the AS re enumerted nto hsh tbe wth proper hsh functon, whch rns the networs wth respect to ther compexty,.e. ssoctes hgher hsh ndces to networs wth hgher compexty. MD-PSO cn then use ech ndex s unque dmenson of the serch spce where prtces cn me nter-dmenson nvgtons to see n optmum dmenson (dbest) dbest nd the optmum souton on tht dmenson,. The former Ths wor ws supported by the Acdemy of Fnnd, project No. 23462 (Fnnsh Centre of Exceence Progrm (2006-20) xˆ y bpso Prtce ( Poston ) x : 2 3 4 5 v : 2 3 4 5 y : 2 3 4 5 { } 48
corresponds to the optm rchtecture nd the tter encpsutes the optmum networ prmeters (connectons, weghts nd bses). Suppose for the se of smpcty, rnge s defned for the mnmum nd mum number of yers, { L, L mn } nd number of neurons for the hdden yer, { N mn, N }. The szes of both nput nd output yers re determned by the probem nd hence fxed. The AS cn then be defned ony by two rnge rrys, L L Rmn = { N, Nmn,..., Nmn, No } nd R = { N, N,..., N, No}, one for mnmum nd the other for the mum number of neurons owed for ech yer of MLP. The sze of both rrys s ntury L where the correspondng entres defne the rnge of the th + hdden yer for those MLPs, whch cn hve n th hdden yer. The sze of the nput nd output yers, { N, N o }, s fxed nd s the sme for confgurtons n the AS. L mn nd L cn be set to ny vue menngfu for the probem encountered. The hsh functon then enumertes potent MLP confgurtons nto hsh ndces, strtng from the smpest MLP wth L mn hdden yers, ech of whch hs mnmum number of neurons gven byr mn, to the most compex networ wth L hdden yers, ech of whch hs mum number of neurons gven by R. Let N h be the number of hdden neurons n yer of MLP wth nput nd output yer szes N nd N o, respectvey. The nput neurons re merey fn-out unts snce no processng tes pce. Let F be the ctvton functon pped over the weghted nputs pus bs, s foows: p, p, p, p, y = F ( s ) where s = w j y j + θ (3) p y, j where s the output of the th neuron of the th hdden/output yer when the pttern p s fed, w j s the weght from the j th neuron n yer - to the th neuron n yer, nd θ s the bs vue of the th neuron of the th hdden/output yer. The trnng men squre error,, s formuted s foows: N o p = ( ) p, o 2 t y (4) 2PN o p T = p po, where t s the trget (desred) output nd y s the ctu output from the th neuron n the output yer, =o, for pttern p n the trnng dtset T wth sze P, respectvey. At tme t, suppose tht prtce, hs the poston component formed s, xd() t xd() t 0 2 2 o o o xx =Ψ {{ wj},{ wj},{ θ},{ wj},{ θ},...,{ wj },{ θ },{ θ} where { w j} nd { θ } represent the sets of weghts nd bses of xd the yer of the MLP confgurton, Ψ. Note tht the nput yer (=0) contns ony weghts wheres the output yer (=o) hs ony bses. By mens of such drect encodng scheme, prtce represents potent networ prmeters of the MLP rchtecture t the dmenson (hsh ndex) xd. As mentoned erer, the dmenson rnge, { D mn, D}, where MD-PSO prtces cn me nter-dmenson jumps, s determned by the AS defned. Aprt from the regur mts such s (poston) veocty rnge, V, V }, dmenson veocty rnge, VD, VD }, the dt { mn { mn spce cn so be mted wth some prctc rnge,.e. xd Xmn < xx < X. Settng n Eq. (4) s the ftness functon enbes MD-PSO to perform evoutons of both networ prmeters nd rchtectures wthn ts ntve process. Further dets nd n extensve set of experments demonstrtng the optmty of the networs evoved wth respect to sever benchmr probems cn be found n []. III. THE CNBC FRAMEWORK Ths secton descrbes n det the proposed frmewor: Coectve Networ of (Evoutonry) Bnry Cssfers, the CNBC, whch uses the trnng dtset to confgure ts ntern structure nd to evove ts bnry cssfers (BCs) ndvduy. Before gong nto dets of CNBC, the evoutonry updte mechnsm, whch eeps ony the best networs wthn the AS of ech BC w be deted next. A. Evoutonry Updte n the Archtecture Spce Snce the evoutonry technque, MD PSO, s stochstc optmzton method, n order to mprove the probbty of convergence to the gob optmum, sever evoutonry runs cn be performed. Let N be the number of runs nd R N be the number of C confgurtons n the AS. For ech run the objectve s to fnd the optm (the best) cssfer wthn the AS wth respect to pre-defned crteron. Note tht ong wth the best cssfer, other confgurtons n the AS re so subject to evouton nd therefore, they re contnuousy (re-) trned wth ech run. So durng ths ongong process, between ny two consecutve runs, ny networ confgurton cn repce the current best one n the AS f t surpsses t. Besdes the MD PSO evoutonry serch, ths s so true for the exhustve serch,.e. ech networ confgurton n the AS s trned by N R BP runs nd the sme evoutonry updte rue ppes. Fgure 2 demonstrtes n evoutonry updte operton over smpe AS contnng 5 MLP confgurtons. The tbe shows the trnng whch s the crteron used to seect the optm confgurton t ech run. The best runs for ech confgurtons re hghghted nd the best confgurton n ech run s tgged wth *. Note tht t the end of the three runs, the over best networ wth = 0.0 hs the confgurton: 5x2x2 nd thus used s the cssfer for ny cssfcton ts unt ny other confgurton surpsses t n future run. In ths wy, ech BC confgurton n the AS cn ony evove to better stte, whch s the mn purpose of the proposed evoutonry updte mechnsm. Fgure 2: Evoutonry updte n smpe AS for MLP confgurton rrys R mn = {5,,2 } nd R = {5,4,2} where N R = 3 nd N C = 5. The best run for ech confgurtons s hghghted nd the best confgurton n ech run s tgged wth *. FV CV 49
B. Coectve Networ of Bnry Cssfers ) The Topoogy To cheve the thrd nd fourth objectves mentoned erer,.e. the scbty wth respect to vryng number of csses nd fetures, nove frmewor encpsutng networ of bnry cssfers (NBCs) s deveoped, where NBCs cn evove contnuousy wth the ongong evouton sessons.e. usng the trnng dtset whch s n prctce obtned by cumutng the ground truth dt (GTD) from sever reevnce feedbc sessons. Ech NBC corresponds to unque mge css nd sh contn vryng number of evoutonry bnry cssfers (BCs) n the nput yer where ech BC performs bnry cssfcton usng snge (sub-) feture. Therefore, whenever new feture s extrcted, ts correspondng BC w be creted, evoved (usng the vbe GTD so fr), nd nserted nto ech NBC, yet eepng ech of the other BCs s s. On the other hnd, whenever n exstng feture s removed, the correspondng BC s smpy removed from ech NBC n the system. In ths wy scbty wth respect to ny number of fetures s cheved nd the over system cn vod re-evoutons from scrtch. {, FV,..., FV N } FV FV N CV 0 BC N NBC 0 FV FV N CV { c * } FV FV N Fgure 3: Topoogy of the proposed CNBC frmewor wth C csses nd N FVs (sub-fetures). Ech NBC hs fuser BC n the output yer, whch coects nd fuses the bnry outputs of BCs n the nput yer nd genertes snge bnry output, ndctng the reevncy of ech FV to the NBC s correspondng css. Furthermore, CNBC s so scbe to ny number of csses snce whenever new css s defned by the user, new NBC cn smpy be creted (nd evoved) ony for ths css wthout requrng ny need for chnge or updte the other NBCs uness ther performnce sgnfcnty deterortes wth the ntroducton of the new css Ths wy the over system dynmcy dpts to vryng number of mge csses. As shown n Fgure 3, the mn de n ths pproch s to use s rge number of cssfers s necessry, so s to dvde mssve ernng probem nto mny NBC unts ong wth the BCs wthn, nd thus prevent the need of usng compex cssfers s the performnce of both trnng nd evouton processes degrdes sgnfcnty s the compexty rses due to the curse of dmensonty. A mjor beneft of our pproch wth respect to effcent trnng nd evouton process s tht the confgurtons n the AS cn be ept s compct s possbe vodng unfesby rge storge nd trnng tme requrements. Ths s sgnfcnt dvntge especy for the trnng methods performng oc serch, such s BP snce the mount of decevng oc mnm s sgnfcnty ow n the error spce for such smpe nd compct ANNs. Furthermore, BC N NBC CV C BC N NBC C when BP s pped exhustvey, the probbty of fndng the optmum souton s sgnfcnty ncresed. In order to mze the cssfcton ccurcy, we pped dedcted css seecton technque for CNBC. We used -of-n encodng scheme n BCs, nd the output yer sze of BCs s wys two (.e. n = 2). Let CV nd CV c, c, 2 be the frst nd second output of the c th BC s css vector (CV). The css seecton n -of-n encodng scheme cn smpy be performed by comprng the ndvdu outputs, e.g. sy postve output f CV c, 2 > CVc,, nd vce vers for negtve. Ths s so true for the fuser BC, the output of whch mes the output of ts NBC. FVs of ech dtset tem re fed to ech NBC n the CNBC. Ech FV s propgted through ts correspondng BC n the nput yer of the NBC. The outputs of these BCs re then fed to the fuser BC of ech NBC to produce CVs. Fny, the css seecton boc shown n Fgure 3 coects them nd seects the postve css(es) of the CNBC s the fn outcome. Ths seecton scheme, frst of, dffers wth respect to the dtset css type,.e. the dtset cn be ced s un-css, f n tem n the dtset cn beong to ony one css, otherwse ced s mutcss. Therefore, n un-css dtset there must be ony one css, * the c, seected s the postve outcome wheres n mut-css dtset, there cn be one or more NBCs, { c * }, wth postve outcome. In the css seecton scheme the wnner-tes- strtegy s utzed. Therefore, for un-css dtsets, the postve css ndex, * c, ( the wnner ) s determned to be the css where the dfference CV CV s m,.e., c, 2 c, c * = rg ( CV c,2 CV c, ) c [0, C ] (5) In ths wy the erroneous cses (fse negtve nd fse postves) where none or more thn one NBC exsts wth postve outcome cn be propery hnded. However, for mut-css dtsets the wnner tes strtegy cn ony be pped when no NBC yeds postve outcome,.e. CVc, 2 CVc, c [ 0, C ], otherwse mutpe NBCs wth postve outcome my ndcte the mutpe true-postves nd hence cnnot be further pruned. As resut, for mut-css dtset the (set of) postve css ndces, { c * }, s seected s foows: rg ( CVc,2 CVc,) f CVc,2 CVc, c [ 0, C ] * c [0, C ] { c } = (6) { rg ( CVc,2 > CVc,)} ese c [0, C ] 2) Evouton of the CNBC The evouton of subset of the NBCs or the entre CNBC s performed for ech NBC ndvduy wth two-phse operton, s ustrted n Fgure 4. As expned erer, usng the feture vectors (FVs) nd the trget css vectors (CVs) of the trnng dtset, the evouton process of ech BC n NBC s performed wthn the current rchtecture spce (AS) n order to fnd the best (optm) BC confgurton wth respect to gven crteron (e.g. trnng/vdton men-squre-error () or cssfcton error (CE)). Durng the evouton, ony NBCs ssocted wth those csses represented n the trnng dtset re evoved. If the trnng dtset contns new csses, whch do not hve correspondng NBC yet, new NBC s creted for ech, nd evoved usng the trnng dtset. In Phse, see top of Fgure 4, the BCs of ech NBC re evoved gven n nput set of FVs nd trget CV. Rec tht ech CV s ssocted wth unque NBC. The fuser BCs re not used n 50
ths phse. Once n evouton sesson s over, the AS of ech BC s then recorded so s to be used for potent (ncrement) evouton sessons n the future. Rec tht ech evouton process my contn sever runs nd ccordng to the forementoned evoutonry updte rue, the best confgurton cheved w be used s the cssfer. Hence once the evouton process s competed for BCs n the nput yer (phse ), the best BC confgurtons re used to forwrd propgte FVs of the tems n the trnng dtset to compose the FV for the fuser BC from ther output CVs, so s to evove the fuser BC n the second phse. Aprt from the dfference n the generton of the FVs, the evoutonry method (nd updte) of the fuser BC s sme s ny other BC hs n the nput yer. In ths phse, the fuser BC erns the sgnfcnce of ech ndvdu BC (nd the correspondng sub-feture) for the dscrmnton of tht prtcur css. Ths cn be vewed s the dptton of the entre feture spce to dscrmnte specfc css n rge dtset, or n other words, s wy of ppyng n effcent feture seecton scheme s some FVs my be qute dscrmntve for some csses wheres others my not nd the fuser, f propery evoved nd trned, cn weght ech BC (wth ts FV), ccordngy. In ths wy the usge of ech feture (nd ts BC) sh optmy be fused ccordng to ther dscrmnton power of ech css. Smry, ech BC n the frst yer sh n tme ern the sgnfcnce of ndvdu feture components of the correspondng FV for the dscrmnton of ts css. In short the CNBC, f propery evoved, sh ern the sgnfcnce (or the dscrmnton power) of ech FV nd ts ndvdu components. FV FV 0 FV N BC N NBC 0 CV 0 = 0 0 = 0 FV FV 0 FV N Fgure 4: Iustrton of the two-phse evouton sesson over BCs rchtecture spces n ech NBC. 3) Increment Evouton of the CNBC To ccompsh yet nother mjor objectve, the proposed CNBC frmewor s desgned for contnuous ncrement evouton sessons where ech sesson my further mprove the cssfcton performnce of ech BC usng the dvntge of the evoutonry updtes. The mn dfference between the nt nd the subsequent ncrement evouton sessons s the ntzton of the evouton process: the former uses rndom ntzton for ech confgurton n the AS wheres the tter strts from the best prmeters found for ech cssfer confgurton n the st AS for ech BC. Note tht the trnng dtset used for the ncrement evouton sessons my be dfferent from the prevous ones, nd ech sesson my contn sever runs. Thus the evoutonry updte rue compres the performnce of the st recorded nd the current (fter the run) networ over the current trnng dtset. Durng ech ncrement evouton phse, exstng NBCs re (ncrementy) evoved ony f BC N NBC FV FV 0 FV N CV CV CV CV C CV = 0 C FV FV N BC N NBC 0 FV FV N BC N NBC BC N NBC C FV FV N CV0 = 0 CV = 0 CV C = 0 BC N NBC C they cnnot ccurtey cssfy the trnng dtset of the new (emergng) csses. In tht, n emprc threshod eve (e.g. 95%) s used to determne the eve of cssfcton ccurcy requred. The NBCs for the new csses re obvousy due for evouton wthout ny such verfcton. Consequenty, the proposed MD PSO evoutonry technque used for evovng MLP confgurtons s ntzed wth the current AS prmeters of the networ. Tht s the swrm prtces re rndomy ntzed (s n the nt evoutonry step) except tht one of the prtces (wthout oss of generty we ssume the frst prtce wth =0) hs ts person best set to the optm souton found n the prevous evoutonry sesson. For MD PSO evouton over MLPs, ths cn be expressed s, d d 0 2 2 O O O xy0 (0) Ψ { wj},{ wj},{ θ },{ wj},{ θ },...,{ wj },{ θ },{ θ } (7) d [, N ] where { j } C { w nd } θ represent the sets of weghts nd bses of d the yer of the MLP networ, Ψ, whch s the d th (MLP) confgurton retreved from the st AS record. It s expected tht especy t the ery stges of the MD PSO run, the frst prtce s ey to be the gbest prtce n every dmenson, gudng the swrm towrds the st souton otherwse eepng the process ndependent nd unconstrned. Prtcury f the trnng dtset s consderby dfferent n the ncrement evouton sessons, t s qute probbe tht MD PSO cn converge to new souton whst tng the pst souton (experence) nto ccount. In the terntve evoutonry technque, the exhustve serch v repettve BP trnng of ech networ n the AS, the frst step of n ncrement trnng w smpy be the ntzton of the weghts w j nd bses θ wth the prmeters retreved from the st AS of tht BC. Strtng from ths s the nt pont, nd usng the current trnng dtset wth the trget CVs, the BP gorthm cn then perform ts grdent descent n the error spce to converge to new souton. IV. EXPERIMENTAL RESULTS In ths secton, we frst det the benchmr dtsets used nd the feture extrcton technques performed for the extensve set of cssfcton nd content-bsed mge retrev experments. We then nvestgte the cssfcton performnce of the proposed CNBC frmewor usng dfferent feture combntons nd we so compre the resuts obtned wth btch trnng nd ncrement evoutons. Fny we sh demonstrte the performnce gn n terms of mproved retrev ccurcy tht cn be cheved usng the proposed CNBC frmewor s compred to the trdton (ds-) smrty bsed retrevs. A. Dtbse Creton nd Feture Extrcton We used MUVIS frmewor [4], to crete nd to ndex the foowng two mge dtbses by extrctng 4 (sub-) fetures for ech. ) Core_0 Imge Dtbse: There re 000 medum resouton (384x256 pxes) mges obtned from Core repostory [5] coverng 0 dverse csses: - Ntves, 2 - Bech, 3 - Archtecture, 4 - Bus, 5 - Dno Art, 6 - Eephnt, 7 - Fower, 8 - Horse, 9 - Mountn, nd 0 - Food. 2) Core_Ctech_30 Imge Dtbse: There re 4245 mges from 30 dverse csses tht re obtned from both Core nd Ctech [6] mge repostores. 5
As deted n Tbe, some of the bsc coor (e.g. MPEG-7 Domnnt Coor Descrptor, HSV coor hstogrm nd Coor Structure Descrptor [7]), texture (e.g. Gbor [8], Loc Bnry Pttern [9], nd Ordn Co-occurrence Mtrx [0]) nd edge (e.g. Edge Hstogrm Drecton [7]) fetures, re extrcted. Some of them re creted wth dfferent prmeters to extrct sever sub-fetures nd the tot feture dmenson s obtned s 2335. Such hgh feture spce dmenson cn thus gve us opportunty to test the performnce of the proposed CNBC frmewor gnst the curse of dmensonty nd scbty wth the vryng number of fetures. Tbe : 4 Fetures extrcted per MUVIS dtbse. H=6, S=2, V=2 24 HSV Coor Hstogrm 2 H=8, S=4, V=4 28 3 N 27 DC = 6, TA = 2%, TS = 5 Domnnt Coor 4 N DC = 8, TA = 2%, TS = 5 35 5 32 bns 32 6 64 bns 64 7 28 bns 28 Coor Structure 8 256 bns 256 9 52 bns 52 0 024 bns 024 Loc Bnry Pttern 6 2 Gbor sce=4, orent.=6 48 3 Ordn Co-occurrence d=3, o=4 36 4 Edge Hstogrm Dr. 5 B. Cssfcton Resuts Both dtbses re prttoned n such wy tht the mjorty (55%) of the tems s spred for testng nd the rest ws used for evovng the CNBC. To demonstrte the feture scbty property of the CNBC, we evoved two CNBCs ndvduy usng 7 (FVs, 3, 5,, 2, 3 nd 4 n Tbe wth tot dmenson of 88) nd 4 ( FVs wth tot dmenson of 2335) fetures. Therefore, the frst CNBC hs 7+=8 BCs nd the second hs 4+=5 BCs n ech NBC. The evouton (nd trnng) prmeters re s foows: For MD-PSO, we use the termnton crter s the combnton of the mum number of tertons owed (terno = 00) nd the cut-off error ( ε =0 4 ). Other prmeters were emprcy set s: the swrm sze, C S=50, V = x/5= 0. 2 nd VD = 5, respectvey. For exhustve BP, the ernng prmeter s set s λ = 0. 0 nd terton number s 20. We use the typc ctvton functon: hyperboc tngent ( x x e e tnh( x) = ). For the AS, we used smpe confgurtons wth x x e + e the foowng rnge rrys: R mn = { N,8,2} nd R = { N,6, 2}, whch ndcte tht besdes the snge yer perceptron (SLP), MLPs hve ony snge hdden yer,.e. L = 2, wth no more thn 6 hdden neurons. Besdes the SLP, the hsh functon enumertes MLP confgurtons n the AS, s shown n Tbe 2. Fny, for both evouton methods, the exhustve BP nd MD PSO, N R = 0 ndependent runs re performed. Note tht for exhustve BP, ths corresponds to 0 runs for ech confgurton n the AS. Tbe 3 presents the cssfcton performnces cheved for Core_0 dtbse by both evoutonry technques over the smpe AS gven n Tbe 2. The resuts ndcte tht MD PSO cheves the owest nd CE eves (nd hence the best resuts) wthn the trnng set wheres vce vers s true for the exhustve BP wthn the test set. CNBC n gener demonstrtes sod robustness gnst the mjor feture dmenson ncrese (.e. from 7 (88-D) to 4 sub-fetures (2335-D)) snce the cssfcton performnce does not show ny deterorton, on contrry, wth both technques better performnce s cheved wth enhnced generzton bty. Ths s n expected outcome snce CNBC cn beneft from the ddton dscrmnton cpbty of ech ncomng (sub-) feture thns to ts Dvde nd Conquer type desgn where n effcent feture seecton scheme s embedded. Tbe 2: The rchtecture spce used for MLPs. Dm. Conf. Dm. Conf. Dm. Conf. 0 N x 2 6 N x6x2 2 N x2x2 N xx2 7 N x7x2 3 N x3x2 2 N x2x2 8 N x8x2 4 N x4x2 3 N x3x2 9 N x9x2 5 N x5x2 4 N x4x2 0 N x0x2 6 N x6x2 5 N x5x2 N xx2 Tbe 3: Cssfcton performnce of ech evouton method per feture set for Core_0 dtbse. Feture Set Evo. Method Trn Trn CE % Test Test CE % 7 subfetures MDPSO 0.32 3.55.6 20.8 BP 0.45 4.88.27 8.36 4 subfetures MDPSO 0.28 3.8.46 8.63 BP 0.34 3.33.25 4.54 Tbe 4 presents the confuson mtrx of the best cssfcton resut over the test set,.e. cheved by the exhustve BP method usng 4 sub-fetures. It s worth notng tht the mjor source of error resuts from the confuson between the 2 nd (Bech) nd 9 th (Mountn) csses where ow-eve fetures cnnot rey dscrmnte due to excessve coor nd texture smrtes mong those csses. Ths s so true for the 6 th css (Eephnt) from whch the bcground of some mges shre hgh smrty wth both csses. Tbe 4: Confuson mtrx of the evouton method, whch gve the best (owest) test CE n Tbe 3. 42 2 0 5 0 0 3 2 37 4 0 0 0 9 2 3 46 0 0 0 2 0 0 53 0 0 0 0 0 0 0 0 0 0 55 0 0 0 0 0 2 4 2 0 0 37 0 8 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 55 0 0 0 8 0 0 0 0 0 46 0 0 2 0 2 46 The CNBC evoutons so fr performed re much e to the (btch) trnng of trdton cssfers (such s ANNs, -NN, Byesn) where the trnng dt (the fetures) nd (number of) csses re fxed nd the entre GTD s used durng the trnng (evouton). As deted erer, the CNBC cn so be evoved ncrementy,.e. ncrement evoutons cn be performed whenever 52
new fetures/csses cn be ntroduced nd the CNBC cn dynmcy crete new BCs nd/or NBCs s the need rses. In order to evute the ncrement evouton performnce, the trnng dtset s dvded nto three dstnct prttons, ech of whch contns 5 (csses -5), 3 (csses 6-8) nd 2 (csses 9 nd 0) csses, respectvey. Therefore, three stges of ncrement evoutons hve been performed where t ech stge the CNBC s further evoved ony wth the prtcur dtset prtton, whch beongs to the new csses n tht prtton. After the frst phse, ony three out of fve exstng NBCs were ncrementy evoved over the trnng dtset of the three new csses (csses 6-8). Smry, t the thrd phse, three out of 8 NBCs dd not undergo for ncrement evouton snce ther cssfcton ccurcy over the trnng dtset of those new csses (9 nd 0) re redy bove the mnmum cssfcton ccurcy threshod (95%) requred. Due to spce mttons, we hd to sp dets on the cssfcton performnces cheved t the ntermedte stges. Tbe 5 presents the fn cssfcton performnce of ech evouton method per feture set. The resuts ndcte few percent osses on both trnng nd test cssfcton ccurces, whch cn be expected snce the ncrement evouton ws purposefuy spped for some NBCs whenever they surpss 95% cssfcton ccurcy over the trnng dtset of the new (emergng) csses. Ths mens, for nstnce, some NBCs (e.g. the one corresponds to css 4, the Bus) evoved wth ony over subset of the entre trnng dtset. Tbe 5: Fn cssfcton performnce of 3-stge ncrement evouton per evouton method nd feture set for Core_0 dtbse. Feture Set Evo. Method Trn Trn CE % Test Test CE % 7 subfetures MDPSO.36 6.89 2.6 28.63 BP 0.82 4.22.85 2.63 4 subfetures MDPSO.23 6.66 2.39 26.36 BP 0.9 7.55.83 23.8 Fny, the CNBC evouton for Core_Ctech_30 dtbse ows testng nd evuton of ts cssfcton performnce when the dtbse sze nd number of csses re sgnfcnty ncresed. For both evouton technques, we used the sme prmeters s presented erer except tht the number of epochs (tertons) for BP nd MD PSO were ncresed to 200 nd 500 n order to compenste the ncrese n the dtbse sze. Tbe 6 presents the cssfcton performnces of ech evouton method per feture set. Due to spce mttons we hd to sp the resuts from ncrement evoutons n ths dtset. As compred wth the resuts from Core_0 dtbse n Tbe 3, t s evdent tht both evouton methods cheved smr cssfcton performnce n the trnng set (.e. smr trn CEs) whst certn degrdton occurs n the cssfcton ccurcy n test set (.e. 0-5% ncrese n the test CEs). Ths s n expected outcome snce the c of dscrmnton wthn those ow-eve fetures cn eventuy yed poorer generzton especy when the number of csses s trped. Tbe 6: Cssfcton performnce of ech evouton method per feture set for Core_Ctech_30 dtbse. Feture Set Evo. Trn Trn Test Test Method CE CE 7 subfetures BP 0.24 2.95 2.6 MD PSO 0.54 8. 2.3 33.40 34.67 4 subfetures MD PSO 0.33 5.47 2.52 36.33 BP 0.074.3 2.69 33.86 Fgure 5: 4x2 smpe queres n Core_0 (qa nd qb), nd Core_Ctech_30 (qc nd qd) dtbses Top-eft s the query mge. 53
C. Retrev Resuts The trdton retrev process n MUVIS s bsed on the query by exmpe (QBE) operton. The (sub-) fetures of the query tem re used for (ds-) smrty mesurement mong the fetures of the vsu tems n the dtbse. Rnng the dtbse tems ccordng to ther smrty dstnces yeds the retrev resut. The trdton (ds- ) smrty mesurement n MUVIS s ccompshed by ppyng dstnce metrc such s L2 (Eucden) between the feture vectors of the query nd ech dtbse tem. When CNBC s used for the purpose of retrev, the sme (L2) dstnce metrc s now pped to the css vectors t the output yer of the CNBC (0x2=20-D for Core_0 nd 30x2=60-D for Core_Ctech_30 dtbses). In order to evute the retrev performnces wth nd wthout CNBC, we use verge precson (AP) nd verge normzed modfed retrev rn (ANMRR) mesures, both of whch re computed queryng mges n the dtbse (.e. btch query) nd wthn retrev wndow equ to the number of ground truth mges, N(q) for ech query q. Ths henceforth mes the AP dentc to verge rec nd verge F mesures, too. Over ech dtbse, four btch queres re performed to compute the verge retrev performnces, two wth nd two wthout usng the CNBC. Whenever used, the CNBC s evoved wth the MD PSO nd the exhustve BP, the former wth 7 nd the tter wth 4 sub-fetures, respectvey. As sted n Tbe 7, t s evdent tht the CNBC cn sgnfcnty enhnce the retrev performnce regrdess of the evouton method, the feture set nd the dtbse sze. The resuts (wthout CNBC) n the tbe so confrm the enhnced dscrmnton obtned from the rger feture set, whch ed to better cssfcton performnce nd n turn, eds to better retrev performnce. Tbe 7: Retrev performnces (%) of the four btch queres n ech MUVIS dtbses. Core_0 Core_Ctech_30 Evo. Method ANMRR AP ANMRR AP For vsu evuton, Fgure 5 presents four typc retrev resuts wth nd wthout usng the proposed CNBC frmewor. A query mges re seected mong the test set nd the query s processed wthn the entre dtbse. V. CONCLUSIONS In ths pper, nove CNBC frmewor s ntroduced to ddress the probem of effcent nd ccurte content-bsed cssfcton nd retrev wthn rge mge dtbses. CNBC s Dvde nd Conquer type of pproch, whch reduces both feture nd css vector dmensons for ndvdu cssfers sgnfcnty to enbe the use of s compct cssfers s possbe. Such compct cssfers cn be evoved nd trned better thn snge yet more compex cssfer. The optmum cssfer for ech cssfcton probem t hnd cn be serched seprtey nd t gven tme, ths ows to crete new dedcted cssfers (BCs) for dscrmntng certn css type from the others wth the use of snge (sub-) feture to ccommodte new fetures or to crete new NBC to ow ntroducton of new csses. Ech (ncrement) evouton sesson erns from the current best cssfer nd cn mprove t further, possby usng nother confgurton n the AS. Moreover, when trned propery, the fuser BC cn correct the erroneous cssfcton of ny BC n the nput yer, whch further ncreses the cssfcton ccurcy. Thus cssfcton performnce, the mn dvntges of the proposed frmewor re the mproved cssfcton performnce nd the effcent souton provded to the probems of scbty nd dynmc dptbty by owng both feture spce dmensons nd the number of csses n dtbse to be unmted nd dynmc (ncrement). Furthermore, the CNBC frmewor s desgned for both onne (ncrement) nd offne (btch) evoutons, whch cn be performed n mutpe runs. Durng ech run, ny new confgurton cn repce the current one n the AS f t outperforms t. Such n evoutonry updte mechnsm ensures tht the AS contnng the best confgurtons, s wys ept ntct nd tht ony the best confgurton t ny gven tme s used for cssfcton nd retrev. Athough the resuts ndcte tht the forementoned objectves hve been successfuy fufed, even hgher ccurcy eves cn st be expected from the CNBC frmewor wth the ddton of new powerfu fetures wth superor dscrmnton nd content descrpton cpbtes. REFERENCES [] S. Krnyz, T. Ince, A. Ydrm nd M. Gbbouj, Evoutonry Artfc Neur Networs by Mut-Dmenson Prtce Swrm Optmzton, Neur Networs, vo. 22, pp. 448 462, do:0.06/j.neunet.2009.05.03, Dec. 2009. [2] S. Krnyz, T. Ince, A. Ydrm nd M. Gbbouj, Frcton Prtce Swrm Optmzton n Mut-Dmenson Serch Spce, IEEE Trns. on Systems, Mn, nd Cybernetcs Prt B, pp. 298 39, vo. 40, No. 2, 200. [3] Y. Chuvn nd D. E. Rumehrt, Bc Propgton: Theory, Archtectures, nd Appctons, Lwrence Erbum Assoctes Pubshers, UK, 995. [4] MUVIS. http://muvs.cs.tut.f/ [5] Core Coecton / Photo CD Coecton (www.core.com ) [6] L. Fe-Fe, R. Fergus nd P. Peron, Lernng genertve vsu modes from few trnng exmpes: n ncrement Byesn pproch tested on 0 object ctegores, IEEE CVPR Worshop on Genertve-Mode Bsed Vson, vo. 2, pp.78, 2004. [7] B. S. Mnjunth, J.-R. Ohm, V. V. Vsudevn, nd A. Ymd, Coor nd Texture Descrptors, IEEE Trns. On Crcuts nd Systems for Vdeo Technoogy, vo., pp. 703-75, Jun. 200. [8] B. Mnjunth, P. Wu, S. Newsm, H. Shn, A texture descrptor for browsng nd smrty retrev, Journ of Sgn Processng: Imge Communcton, vo. 6, pp. 33-43, Sep. 2000. [9] T. Oj, M. Petnen, D. Hrwood, A comprtve study of texture mesures wth cssfcton bsed on feture dstrbutons, Pttern Recognton, vo. 29, pp. 5-59, 996. [0] M. Prto, B. Crmruc, M. Gbbouj, "An Ordn Co-occurrence Mtrx Frmewor for Texture Retrev", EURASIP Journ on Imge nd Vdeo Processng, vo. 2007, Artce ID 7358, 5 pges, 2007. do:0.55/2007/7358. 54