Automatic Multi-Label Image Annotation System Guided by Firefly Algorithm and Bayesian Classifier

Transcription

1 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. Auomac Mul-abel Image Annoaon Sysem Guded by Frefly Algorhm and Bayesan Classfer Saad M. Darwsh, Mohamed A. El-Isandaran, and Guar M. Shawa Absrac owadays, he amoun of avalable mulmeda daa s connuously on he rse. The need o fnd a requred mage for an ordnary user s a challengng as. Tradonal mehods as conen-based mage rereval CBIR compue relevance based on he vsual smlary of low-level mage feaures such as color, exures, ec. However, here s a gap beween low-level vsual feaures and semanc meanngs requred by applcaons. The ypcal mehod of brdgng he semanc gap s hrough he auomac mage annoaon AIA ha exracs semanc feaures usng machne learnng echnques. In hs paper, we propose a novel aemp for mul-nsance mul-label mage annoaon MIM. Frsly, mages are segmened by Osu mehod whch selecs an opmum hreshold by maxmzng he varance nra clusers n he mage. Osu s mehod s modfed usng frefly algorhm o opmze runme and segmenaon accuracy. Feaure exracon echnques based on colour feaures and regon properes are appled o oban he represenave feaures. In he annoaon sage, we employ a Gaussan model based on Bayesan mehods o compue poseror probably of conceps gven he regon clusers. Ths model s effcen for mul-label learnng wh hgh precson and less complexy. Expermens are performed usng Corel Daabase. The resuls show ha he proposed sysem s beer han radonal ones for auomac mage annoaon and rereval. Index Terms feaure exracon, feaure selecon, mage annoaon, classfcaon W I. ITRODUCTIO h he developmen of new capurng echnology and growh of he World Wde Web, he amoun of mage daa s connuously growng. Effcen mage searchng, browsng and rereval ools are requred by users from varous domans, ncludng remoe sensng, medcal magng, crmnal nvesgaon, archecure, communcaons, and ohers. For hs purpose auomac mage annoaon AIA echnques have become ncreasngly mporan and large number of machne learnng echnques has been appled along wh a grea deal of research effors [-4]. There are generally wo ypes of AIA approaches [5]. The frs approach s based on radonal classfcaon mehods. I reas each semanc eyword or concep as an Manuscrp receved February 5, 06; revsed Aprl 06, 06. Saad M. Darwsh s a Professor n he Insue of Graduae Sudes and Research correspondng auhor; 6 Horreya Avenue, El-shaby, Alexandra, Egyp; e-mal: saad.darwsh@alex-gsr.edu.eg. Mohamed A. El-Isandaran s a Professor n he Insue of Graduae Sudes and Research e-mal: Isandaran@alex-gsr.edu.eg. Guar M. Shawa s wh he Insue of Graduae Sudes and Research e-mal: garshawa@ yahoo.com. ndependen class and rans a correspondng classfer based on he ranng se o denfy mages belongng o hs class. The common machne learnng ncludes suppor vecor machnes SVM, arfcal neural newor A and decson ree DT [5]. Ths approach s consdered supervsed as a se of ranng mages wh and whou he concep of neres was colleced and a bnary classfer raned o deec he concep of neres [6]. The problem of hs approach s ha doesn consder he fac ha many mages belong o mulple caegores. The second approach s based on generave model. I reas mage and ex as equvalen daa and focuses on learnng he vsual feaures and semanc conceps by esmang he on dsrbuon of feaures and words. The nfluenal wor s Cross Meda Relevance Model CMRM [], whch was subsequenly mproved hrough models as connuous relevance model CRM, mulple Bernoull relevance model MBRM [7] and dual CMRM [8]. Ths approach s consdered unsupervsed, he relaonshp beween eywords and mage feaures s denfed by dfferen hdden saes [6]. The generave based mehod s a more reasonable approach, because assgns an mage o several caegores and assgns an mage o a caegory wh a confdence value whch assss mage ranng [5]. Images may be assocaed wh number of nsances and number of labels smulaneously, hus provdes effcen envronmen for mul-nsance mul-label learnng MIM, however s challengng n hree aspecs. abel ocaly: mos labels are only relaed o her correspondng semanc regons. For example, some sngle semanc labels, such as ree and mounan, are relaed o sngle semanc regons. Whle some oher labels, such as beach, are relaed o a mulple semanc regons sand and beach. Thus s necessary o propose a mehod o decompose he feaure represenaon wh effcen segmenaon mehods. Iner-abel Smlary: here are several relaonshps beween dfferen labels, such as herarchcal relaonshp, correlave relaonshp and so on. These relaonshps, are que necessary o be consdered o mprove he accuracy n label propagaon [9]. Thus s very aracve o develop new algorhms for characerzng he ner-concep smlary conexs more precsely and deermnng he ner-relaed learnng ass auomacally [0]. And Iner-abel Dversy: for each label, s correspondng regons a dfferen mages can be dfferen. For example, he label sy could nfer varous expressons, such as cloudy, dar, clear sy and so on. We need o eep n mnd on nra- label ISB: ISS: Prn; ISS: Onlne WCE 06

2 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. dversy so ha o elmnae he gap among dfferen expressons n label propagaon. To encouner he menoned challenges, we consder a modfed generave model ha apples MIM annoaon and pays more aenon o segmenaon n am o provde beer performance. Frsly, he mage s segmened usng Osu mehod whch selecs an opmum hreshold by maxmzng he varance nra clusers n he mage. Osu s mehod s modfed usng frefly algorhm, so ha he nerlabel localy can be releved. Also hs mehod provdes he opmal mulple hresholds, hgher convergng speed, and less compuaon rae []. Secondly, each segmen was represened by feaures o mprove he ner-label smlary. asly, model based on Bayesan mehods was used for annoaon n am o mprove he ner-label dversy by provdng beer correspondence beween he words and he segmens. The model used s consdered unsupervsed, n he sense ha, he words are avalable only for he mages, no for he ndvdual regons. The paper s organzed as follows. Secon II brefly menons relaed wor n mage annoaon. Secon III formulaes he problem of learnng, Secon IV descrbes our annoaon mehodology. Secon V presens he resuls of he expermens. Fnally, secon VI concludes he paper and presens drecons for fuure research. II. REATED WOR Image annoaon researches have been nroduced by many researchers o assocae vsual feaures wh semanc conceps. Valaya e al [] aemped o capure hgh-level conceps from low-level mage feaures by usng bnary Bayesan classfers. Ther wor focused on herarchcal classfcaon of vacaon mages. A vecor quanzer was used and class-condonal denses of he feaures were esmaed. Duygulu e al [] guaraneed he mage vsual words blobs vocabulary by cluserng and dscrezng he regon feaures. He proposed a machne ranslaon mehod o descrbe mages usng a vocabulary of blobs. In addon o hs, Ble and Jordan [4] employed correspondence laen Drchle allocaon DA model o buld a language-based correspondence beween words and mages. The model s a generave process ha frs generaes he regon descrpons and subsequenly generaes he capon words. Jeon e al [] proposed CMRM based on he machne ranslaon model. The prmary dfference s he underlyng one-o-one correspondence beween blobs and words and assumng a se of blobs s relaed o a se of words. Monay and Gaca-Perez [5] nroduced laen varables o ln mage feaures wh words as a way o capure co-occurrence nformaon. Ths s based on laen semanc analyss SA. The addon of a probablsc model o SA resuled n he developmen of PSA. avreno e al. [7] proposed smlar CRM, n whch he word probables are esmaed usng mulnomal dsrbuon and he blob feaure probables usng a nonparamerc ernel densy esmae. Whle Feng e al [5] modfed he above model [7] usng a mulple-bernoull dsrbuon. In addon, hey smply dvded mages no recangular les nsead of applyng auomac segmenaon algorhms. Ther MBRM acheved furher mprovemen on performance. Yavlnsy e al [6] ISB: ISS: Prn; ISS: Onlne descrbed a smple framewor for AIA usng non-paramerc models of dsrbuons of mage feaures. They showed ha under hs framewor que smple mage properes provde a srong bass for relably annoang mages. Ru e al [7] proposed an approach for AIA, hey frs performed cluserng of regons by ncorporang par-wse consrans whch were derved by consderng he language model underlyng he annoaons assgned o ranng mages. Second, hey employed a sem-naïve Bayes model o compue he poseror probably of conceps gven he regon clusers. Zhou e al [] formalzed MIM learnng where an example s assocaed wh mulple nsances and mulple labels smulaneously. They proposed algorhms, MIMBOOST and MIMSVM, whch acheved good performance n he applcaon o scene classfcaon. M. Wang e al. [8] explored he use of hgher level semanc space wh lower dmenson by cluserng correlaed eywords no opcs n he local neghbourhood, hey also reduced he bas beween vsual and semanc spaces by fndng opmal margn n boh spaces. Bao e al [9] proposed he hdden concep drven mage annoaon and label ranng algorhm whch conduced label propagaon based on he smlary over a vsually semancally conssen hdden concep spaces. Xue e al [0] developed a srucured max-margn learnng algorhm by ncorporang he vsual concep newor, max-margn Marov newor and mul as learnng o address he ssue of huge nerconcep vsual smlary more effecvely. Johnson e al [9] presened an obec recognon sysem whch learned from mul-label daa hrough boosng and mproved on sae-ofhe-ar mul-label annoaon and labelng sysems. Vanarasmhan e al [0] presened an acve learnng framewor ha predced he radeoff beween he effor and nformaon gan assocaed wh a canddae mage annoaon. They developed MIM approach ha accommodaes mul-obec mage and a mxure of srong and wea labels. Mos of he prevous wor ddn consder he segmenaon sep more horoughly. Our proposed mehod nroduces a sysem ha apples mul-label annoaon and pays more aenon o segmenaon n am o provde beer precson. III. PROBEM FORMUATIO e J denoes he esng se of un-annoaed mages, and le T denoes he ranng collecon of annoaed mages. Each esng mage I J s represened by s regonal vsual feaures f { f,... f }, and each ranng mage I T s represened by boh a se of regonal vsual feaures f { f,... f } and a eyword ls W V, where f,... s he vsual feaures for regon, {... M V } he vocabulary and,... M he h eyword n V. The goal of mage annoaon s o selec a se of eywords W ha bes descrbes a gven mage I from he vocabularyv. The ranng se,t, consss of mage-eyword pars T { I, W,..., I, W } [6]. The ey dea of learnng s o run a cluserng algorhm on WCE 06

3 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. he low-level feaure space, and hen esmae he on densy of eywords and low-level feaures P,. In he unsupervsed learnng formulaon, he relaonshp beween eywords and mage feaures s denfed by dfferen hdden saes. A laen varable z z z,... z } encodes hdden saes of he word. {.e. " sy " sae," e " sae. A sae defnes a on dsrbuon of mage feaures and eywords..e. P f blue, whe, fuzzy, " sy"," cloud "," blue" " sy" Sae, wll have hgh probably. We can sum over he saes varable o fnd he on dsrbuon. Where z s varable of he hdden sae, s he number of he possble saes of Z []. P f, P f, z P z The model s oo large o be represened as a unque on probably dsrbuon, herefore s requred o nroduce some sparse and srucural a pror nowledge. The probablsc graphcal models, especally Bayesan newors are a good way o solve hs nd of problem. In fac whn Bayesan newors he on probably dsrbuon s replaced by a sparse represenaon only among he varables drecly nfluencng one anoher. Ineracons among ndrecly-relaed varables are hen compued by propagang nfluence hough a graph of hese drecs connecons. Consequenly he Bayesan mehods are a smple way o represen a on probably dsrbuon over a se of random varables, o vsualze he condonal properes and o compue complex operaons le probably learnng, and nference wh graphcal manpulaons []. The smples model adoped maes each mage n he ranng daabase a sae of he laen varable, and assumes condonal ndependence beween mage feaures and eywords,.e. P f, z P f z p z P f, P z P f z P z Where s he ranng se sze []. The mxure of Gaussan s assumed for he condonal probably P f z [4, 4]. p f z Where, e T f f and are he dmenson, covarance marx and mean of vsual feaures belongng o respecvely. Followng he maxmum lelhood prncple, P f z, P z and P z can be deermned by maxmzaon of he loglelhood funcon [4]. u log P f Z u log P z p f z 5 u he number of annoaon words for mage M n f f. z 4 log P f, 6 n denoes he wegh of annoaon word occurrence frequency, for mage,.e., f.the sandard procedure ISB: ISS: Prn; ISS: Onlne for maxmum lelhood esmaon n laen varable models s he EM algorhm. In E-sep, applyng Bayes heorem o, one can oban P z P f z P z P z f, 7 P z P f z P z In M-sep, one has o maxmze he expecaon of he complee-daa log-lelhood M n log[ P z p f z P z ] P Z F, 8, z,,,, V M s P z f s, s, P Z F, V. In 8 he noaon s he concep varable ha assocaes wh he feaure- word par f,, where,. Maxmzng 8 wh agrange mulplers o P f z and P z respecvely under he followng consrans P z, P z f, 9 For any f, z and, he parameers are deermned as s u f p z f 0 u p z f Z P z s s u p z f f f s s u p z f M n M s P z n T f, A annoaon me, he feaure vecors F exraced from he query are used o oban a funcon of W ha provdes a naural orderng of he relevance of all possble capons for he query. Ths funcon can be he on densy or he poseror densy accordng o Bayesan mehods. pror * lelhood poseror evdence P f P f, P P W F F, W F f Annoaon nvolves he words ha maxmze he probably dsrbuon model, * arg max W, P F, f P W, F f IV. THE PROPOSED AOTATIO SYSTEM 4 As menoned he ey for mage annoaon s o learn a sascal model whch correlaes he mage feaures wh he annoaon words. We sar wh a se of ranng mages, each of whch has a se of accompanyng annoaon words. Typcally, mages are frs segmened no mulple homogenous regons usng Maxmum Varance Inra cluserng Osu modfed by he Frefly algorhm. Image feaures are exraced o represen each mage regon, hen a model based on Bayesan mehods are appled o learn he correspondence beween regons and words. Fnally, gven a new es mage, he same se of mage feaures are exraced, and words are predced accordng o he relaonshp beween mage feaures and annoaon words esablshed by he model. The proposed model s shown n Fg.. WCE 06

4 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. A. Image Segmenaon The frs sep n semanc undersandng s o exrac effcen vsual feaures from he mage. These feaures can be exraced eher locally or globally. Global mehods compue a sngle se of feaures from he enre mage. As naural mages are no homogenous, hs sngle se of feaures may no be meanngful unless hey are appled n doman specfc applcaons. ocal mehods dvde mages no regons or blocs, a se of feaures s compued for each of he regons. As a resul, feaures can represen mages a obec level and provdes spaal nformaon. However, regon feaures may no be accurae due o he usually unsupervsed segmenaon. Segmenaon performance usually depends on applcaons. Some common mage segmenaon algorhms are grd based, cluserng based, conour based, sascal model based and regon growng based mehods [6]. Fg.. The Proposed Mullabel Image Annoaon Sysem One of he effcen mehods for mage segmenaon s maxmzng he varance Inra-cluser, whch selec a global hreshold value by maxmzng he separably of he classes n grey level mages [6]. Osu s based on maxmum varance nra cluser. The basc dea of Osu s mehod s o dvde he pxels no wo groups a a hreshold and calculae he varance beween hem. The bgger he varance shows he more dfference beween wo pars. The mage sze s M and he mage grey level s. The grey range s 0~-. The pxels number of grayscale level I s n. Thus, he number of he mage pxels s n n M, he probably dsrbuon s: 0 Inpu mage Image Segmenaon usng Maxmum varance nra cluserng and Fre Fly algorhm Feaure Exracon Color Feaure, Regon Properes Annoaon usng Gaussan model based on Bayesan mehod Mullabel annoaed n P, p 0 5 n The mage s dvded no wo classes wh he sandard hreshold. The class cncludes he pxel and he class c ncludes he pxel. Cumulave probably of c and c s: w p 0, w p w 6 ISB: ISS: Prn; ISS: Onlne Calculaed mean levels: p w 0, p w 7 Varance of class c and class c s: p w 0, p w 8 Varance ner cluser s: w w w 9 Varance nra cluser s: B w T w T w w 0 The bes hreshold value T B should sasfy he condon afer he mage s dvded no wo caegores c and c : max[ B ] T B When he mage s segmened o more han wo caegores, he Osu mehod should be exended o more hresholds, he equaons wll be as follows: w w w w B w T w T w T max[ ] Th 4, Th B However, wh ncreasng he number of caegores, compung rae and oal runmes also ncrease. Here we use modfed Osu whch s a combnaon of he Frefly algorhm wh Osu s mehod o fnd opmal hreshold of mages and ncreasng segmenaon resul accuracy [6]. B. Frefly Algorhm The Frefly algorhm FA s a novel meaheursc, whch s nspred by he behavour of frefles [7, 8, 9]. In he Frefly algorhm, here are hree dealzed rules. Frs, all frefles are unsex, and hey wll move owards more aracve and brgher ones regardless of her sex. ex, he aracveness r of a frefly s proporonal o s brghness whch decreases as he dsance from he oher frefly ncreases. If here s no a more aracve frefly han a parcular one, wll move randomly. For maxmzaon problems, he brghness s proporonal o he value of he obecve funcon [6]. e r o r 5 Where denoes he maxmum aracveness a r 0 o and s he lgh absorpon coeffcen, whch conrols he decrease of he lgh nensy. The dsance nra any wo frefles and a x and x, respecvely, s he Caresan dsance [8]. r x x Where x of d x, s he x, x, 6 h componen of he spaal coordnae h frefly and d denoes he number of dmensons. The movemen of a frefly s deermned by he followng form [8]. x x e r o x x rand The frs erm s he curren poson of a frefly, he second erm denoes a frefly s aracveness and he las 7 WCE 06

5 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. erm s used for he random movemen f here are no any brgher frefly rand s a random number generaor unformly dsrbued n he range 0,. For mos cases 0 0,. In pracce he lgh absorpon coeffcen γ vares from 0. o 0. Ths parameer descrbes he varaon of he aracveness and s value s responsble for he speed of FA convergence [8]. The flowchar of he algorhm s shown n Fg.. and sar Inalze frefles and parameers Calculae he varance for each segmen Beer han he correspondng frefles Move frefles o new poson bgges eraon Fg.. The Flowchar of he Maxmum Varance Inracluser usng Frefly algorhm Maxmum Varance Inra Cluser Osu usng Frefly algorhm: The pseudo code of he algorhm s:. Inalze algorhm s parameers: - umber of frefles n acc.o hresholds - α, ß and γ accordng o [8] - Maxmum number of generaons Max-Gen. -Defne obecve funcon f x varance of segmen -Generae nal populaon of frefles -gh nensy of frefly I a x s obecve fn. f x. Whle < MaxGen // = : MaxGen For = : n //all n frefles For = : n If I > I move frefly o n acc. o Eq. 7; End f Oban aracveness acc. o Eq. 5 and Eq. 6 Fnd new soluons and updae lgh nensy End for End for Ran he frefles and fnd he curren bes End whle. Fnd he frefles wh he hghes lgh nensy. Image s segmened by he opmal resul obaned. Y e be he opmal hreshold. Segmen mages wh hs opmal hreshold end C. Feaure Exracon The erm feaure exracon comes from paern classfcaon heory [9]. Is goal s wofold. Frsly, o ransform he orgnal mage daa no feaures ha are more useful for classfcaon han he orgnal daa. Secondly, reduce he compuaonal complexy [9, 0, ]. Some colour descrpors le correlogram, colour srucure descrpor CSV and colour coherence vecor CVV capure boh colour and exure feaures. However, hey are appled o enre mage. Texure descrpors le edge hsogram, co-occurrence marx, dreconaly and specral mehods capure boh exure and shape feaures [5]. Global feaures are calculaed on he whole mage usng he momen hsogram. These are mean and sandard devaon. Ths gves feaures. Regonal Color Feaures: are wdely used for mage represenaon because of her smplcy and effecveness. Colors can be represened by varable combnaons of he hree RGB prmary color space [4,, 7]. However some oher color spaces can be used for represenng he color feaure, such as HSV and ab. A ab color space s a color-opponen space wh dmenson for lghness whle a and b for he color-opponen dmensons. The ab color space ncludes all percevable colors. One of he mos mporan arbues of he ab model s devce ndependence. In our wor we calculae he mean, sandard devaon, and sewness for each channel n he ab color space. G 0 p 5 G 0 p 6 G p 0 7 Where s a random varable ndcang nensy, p s he hsogram of he nensy levels n a regon and G s he number of possble nensy levels. Ths yelds a 9-dmensonal color vecor. Regon properes descrbe he ouer shape va some sascal expresson. Several calculaons are made usng Malab [], whch are: Area, Boundary/area and Convexy. Regon properes yeld feaures. Thus he oal feaures used are 4 feaures. Fg.. abelng of segmened mages ISB: ISS: Prn; ISS: Onlne WCE 06

6 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. D. abelng Segmened Images The segmened mages now conss of blobs, and each blob s labeled wh word Fg.. Thus we oban a daase of labeled mages. A. Auo annoaon sraegy Annoaon can be done by predcng words wh hgh poseror probably gven he mage. In order o oban he word poseror probables for he whole mage, he word poseror probables of he regons n he mage, provded by he probably able, are summed ogeher. For he mage I, we can wre, p I p 8 Fg. 4. Auo-annoaon sraegy. Word poseror probables for he regons of he mage.then he bes n words wh he hghes probably are chosen o annoae he mage b' s are he blobs n he mage and s number of words n he mage. Then, he sum of hese word poseror probables s normalzed o one. Fg. 4 shows an example for obanng he word poseror probables for he mage. In order o auo-annoae he mages we predc n words wh he hghes probably, where n s a predefned number. :hp:// b V. EXPERIMETS A. Daabase To valdae our mehod we use he publcly avalable Corel daase, snce conans mul-label mages and a varable number of obecs per mage. I also allows comparsons wh anoher MIM approach and oher saeof he-ar mehods. In he followng expermens we use he Corel daabase.the daabase conans ses, Corela and Corelb. Corela, has 00 mages wh 8 words n he ran se. Corelb daa se conans a oal of 97 mages wh 7 words n he ran se. Boh ses are dvded no ran and es ses n he rao :. Sample of he mages s shown n Fg. 5. In our expermens, we used nds of Segmenaon. In he frs, we used a unform grd of paches over he mage, n he second we used he Osu mehod based on he Maxmum Varance Inra-Cluser as menoned before, hen fnally we used he proposed modfed Osu. The feaures were exraced as descrbed before, hus obanng 4 feaures. The framewor provded by magerans was used exensvely for buldng a Gaussan model []. For evaluaon of he resuls several mercs are used o measure accuracy and rereval effecveness, he followng sascs were colleced: Precson p: The rao of correcly classfed nsances um correc o he oal number of nsances classfed as he class under consderaon um rereved. abel % label word frequency: represens he probably of fndng a parcular word n an mage regon; and Annoaon% annoaon word freq: represens he probably of fndng a parcular word n he manually-annoaed mage. Resuls for evaluaon of he mage annoaon are shown n Tables I o IV The precson of he used model averaged over he 6 rals. Precson s defned as he probably he model s predcon s correc for a parcular word and blob. oe ha some words do no appear n boh he ranng and es ses, hence he n/a. For Corela and Corelb ses of mages, he labelword frequency and annoaon word frequency are shown for he used words Fg. 6 and Fg. 7. They are roughly he same for all caegores n he daa se, whch s a good hng and proves ha he evaluaon sysem s worng rgh and was able o predc he label% usng he annoaon %. TABE I RESUTS O COREA SET USIG: OTSU Usng Osu Mehod Fg. 5. Orgnal mages and annoaed mages usng Grd and Modfed Osu ISB: ISS: Prn; ISS: Onlne WCE 06

7 freq freq Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. TABE II RESUTS O COREA SET USIG: MODIFIED OTSU Usng Modfed Osu Mehod TABE IV RESUTS O COREB SET USIG: MODIFIED OTSU Usng Modfed Osu Mehod The precson for each ndvdual word was also calculaed for he used mehods and shown n Table VII. The ndvdual precson s defned as he probably he predcon s correc for a parcular word and blob. The decrease n performance of hese classes may be arbued o he small ranng and es mages of hese classes. Also he oal precson was calculaed. I s defned as he probably he predcon s correc for all words and all blobs. The oal precson showed mprovemen from o 0.09 o 0. for corela se and 0.0 o 0.85 o 0.5 for corelb se. The resuls are shown n Table V. Some obecs decreased a lle, bu oher obecs mproved sgnfcanly le grass, rees and waer whose predcon were correc mos of he me. Obvously, s dffcul problem, so wll be hard o acheve 00% accuracy. Alhough he ndvdual precson may no be hgher n some obecs, ye he mos mporan s he overall precson as he mage s annoaed wh more han words no only one word. TABE III RESUTS O COREB SET USIG: OTSU Usng Osu Mehod The oal precson for he modfed Osu mproved sgnfcanly compared o he ordnary Osu mehod, hs s arbued o he segmenaon echnque ha mproved he ner-label localy. Furhermore, he feaures used mproved he ner-label smlary. The Bayesan model appled provded beer correspondence beween words and segmens. Ths llusraes he advanage of he mullabel mul-nsance learner proposed abelwordfreq & Annowordfreq for corela se arplane boa cow grass Words house mounan roc sy rees abelwordfreq Annowordfreq Fg. 6. abelword freq and Annowordfreq for Corela se abelwordfreq & Annowordfreq for corelb se abelwordfreq Annowordfreq arplane brd coral fox polarbear sand Words snow racs waer Fg. 7. abelword freq and Annowordfreq for Corelb se TABE V PRECISIO USIG OTSU AD MODIFIED OTSU O CORE A AD CORE B DATASETS Grd Precson Osu Precson Mod Precson Corela Corelb Osu ISB: ISS: Prn; ISS: Onlne WCE 06

8 Proceedngs of he World Congress on Engneerng 06 Vol I WCE 06, June 9 - July, 06, ondon, U.. VI. DISCUSSIO AD COCUSIOS Ths sudy has evaluaed he effecveness of feaure exracon and selecon echnques appled o daa usng Bayesan mehod. The mos noceable effec s he mprovemen n he correcly classfed nsances whch was affeced grealy by he rgh choce of he segmenaon echnque based on he proposed Frefly algorhm and usng Translaon model. The performance can be mproved whou much effor snce mos of hese echnques are no me/compung nensve. Ths s rue for any learnng algorhm, snce he complexy of he daa used drecly affecs he learnng algorhm's performance. Feaure selecon, when used along wh any learnng sysem, can help mprove performance of hese sysems even furher wh mnmal addonal effor. By selecng useful feaures from he daa se, we are essenally reducng he number of feaures needed for hese cred-rs evaluaon decsons. Ths n urn ranslaes o reducon n daa gaherng coss as well as sorage and manenance coss assocaed wh feaures ha are no necessarly useful for he decson problem of neres REFERECES [] Z. Zhou, M. Zhang, Mul-nsance mul-label learnng wh applcaon o scene classfcaon, Proceedngs of he Inernaonal conference on Advances n eural Informaon Processng Sysems, Canada, 006, vol. 9, pp [] T. Sumah, C.. Devasena, An overvew of auomaed mage annoaon approaches, Inernaonal Journal of Research and Revews n Informaon Scences, vol., pp. 6, 0. [] J. Jeon, V. avreno, R. Manmaha, Auomac mage annoaon and rereval usng cross-meda relevance models, Proceedngs of he 6 h Inernaonal ACM Conference on Research and Developmen n Informaon Rereval, Canada, pp. 9-6, 004 [4] C. F. Tsa, C. Hung, Auomacally annoang mages wh eywords: a revew of mage annoaon sysems, Recen Paens on Compuer Scence, vol., no., pp , 008 [5] D. Zhang, M. Islam, G. u, 'A revew on auomac mage annoaon echnques, Paern Recognon, 0, vol. 45, no., pp [6] G. Carnero,. Vasconelos, Formulang semanc mage annoaon as a supervsed learnng problem, Proceedngs of he IEEE Conference on Compuer Vson and Paern Recognon, San Dego, 005, pp [7] V. avreno, R. Manmaha, J. Jeon, A model for learnng he semancs of pcures, Proceedngs of he Inernaonal Conference on eural Informaon Processng Sysems, Canada, 00, pp [8] J. u, B. Wang, M., e al. Dual Cross-Meda Relevance Model for Image Annoaon, Proceedngs of he 5 h Inernaonal Conference on Mulmeda, Germany, 007, pp [9] B.. Bao, T., S. Yan, Hdden-concep drven mul-label mage annoaon and label ranng, IEEE Transacons on Mulmeda,, vol. 4, no., pp. 99-0, 0. [0] X. Xue, H. uo, J. Fan, Srucured max-margn learnng for mullabel mage annoaon, Proceedngs of he 9 h ACM Inernaonal Conference on Image and Vdeo Rereval, 00, pp [] X. S. Yang, X.S. He, Frefly Algorhm: recen advances and applcaons, Inernaonal ournal of swam nellgence, vol., pp. 6-50, 0. [] A. Valaya, M. A. T. Fgueredo, A.. Jan, e al. Image classfcaon for conen-based ndexng, IEEE Transacons on Image Processng, vol. 0, no., pp. 7-0, 00. [] P. Duygulu,. Barnard, J. D. Freas, e al. Obec recognon as machne ranslaon: learnng a lexcon for a fxed mage vocabulary, Proceedngs of he 7 h European Conference on Compuer Vson, Denmar, May 00, pp. 97- [4] D. M. Ble, M. I. Jordan, Modelng annoaed daa, Proceedngs of he 6 h Inernaonal ACM Conference on Research and Developmen n Informaon Rereval, Canada, 00, pp. 7-4 [5]. Feng, R. Manmaha, V. avreno, Mulple bernoull relevance models for mage and vdeo annoaon, Proceedngs of he IEEE Inernaonal Conference on Compuer Vson and Paern Recognon, USA, 004, pp [6] A. Yavlnsy, E. Schofeld, S. Ruger, Auomaed mage annoaon usng global feaures and robus nonparamerc densy esmaon, Proceedngs of he Inernaonal Conference on Image and Vdeo Rereval, Sngapore, 005, pp [7] S. Ru, W. Jn, T. S. Chua, A novel approach o auo mage annoaon based on parwse consraned cluserng and sem-naı ve Bayesan model, Proceedngs of he h Inernaonal Conference on Mulmeda Modelng, Ausrala, 005, pp. -7. [8] M. Wang, X. Zhou, T. S. Chua, Auomac mage annoaon va mul-label classfcaon, Proceedngs of he Inernaonal Conference on Conen-Based Image and Vdeo Rereval, Canada, 008, pp [9] M. Johnson, R. Cpolla, Improved mage annoaon and labelng hrough mul-label boosng, Proceedngs of he 6 h Brsh Machne Vson conference, Brsh Vson Machne assocaon, U, 005, pp. -6. [0] S. Vaayanarasmhan,. Grauman, Wha s gong o cos you?: predcng effor vs nformaveness for mul-label mage annoaons, Proceedngs of he IEEE Conference on Compuer Vson and Paern Recognon, USA, 009, pp [] T. sumah, C.. Devasena, R. Revah, e al. Auomac mage annoaon and rereval usng mul-insance mul-label learnng, Bonfrg Inernaonal Journal of Advances n Image Processng, vol., 0. []. Sea, H. Burhard, Feaure selecon for auomac mage annoaon, Proceedngs of he 8 h Symposum of he German Assocaon for Paern Recognon, Germany, 006, pp [] S. Barra, S. Tabbone, Modelng, classfyng and annoang wealy annoaed mages usng bayesan newor, Journal of Vsual Communcaon Image Rereval, vol.,, pp. 55-6, 00. [4] R. Zhang, M. Zhang, W. Y. Ma, A probablsc semanc model for mage annoaon and mul-modal mage rereval, Proceedngs of he 0 h IEEE Inernaonal Conference on Compuer Vson, Chna, 005, pp [5] T. Hassanzadeh, H. Vood, M. E. Mghadam, An mage segmenaon approach based on maxmum varance nra-cluser mehod and frefly algorhm, Proceedngs of he 7 h Inernaonal conference on aural compuaon, Chna, July 0, pp. 87-8, [6] X. S. Yang, Frefly algorhm, sochasc es funcons and desgn opmsaon, Inernaonal Journal of Bo-Inspred Compuaon, vol., no., pp , 00. [7] J. wecen, J. B. Flpowcz, Frefly algorhm n opmzaon of queueng sysems, Bullen of he Polsh Academy of Scences Techncal Scences, vol. 60, 0. [8] X. S. Yang, Frefly algorhms for mulmodal opmzaon, sochasc algorhms: foundaons and applcaons, ecure oes n Compuer Scences, vol. 5, pp , 009. [9] S. uhan, D. Brown, Exracon of arbues, naure and conex of mages, Inernaonal Conference of Paern Recognon and Image Processng, Ausra, 005. [0] W. uo, Comparson for edge deecon of Colony Images, Inernaonal Journal of Compuer Scence and ewor Secury, vol. 6, no. 9, pp. -5, 006. [] R. C. Gonzalez, R. E. Woods, S.. Eddns, Dgal Image Processng and Image Processng usng MATAB ' Prence Hall, 004 [] P. Carboneo,. D. Freas,. Barnard, A sascal model for general conexual obec recognon, Proceedngs of he 8h European Conference on Compuer Vson, Czech Republc, 004, pp [] F. ang, R. Jn, J. Y. Cha, Regularzng ranslaon models for beer auomac mage annoaon, Proceedngs of he h ACM Conference on Informaon and nowledge Managemen, USA, 004 [4] Z., Z. Tang, Z., e al. Combnng generave /dscrmnave learnng for auomac mage annoaon and rereval, Inernaonal Journal of Inellgence Scence, vol., pp. 55-6, 0. [5] F. Monay, D. G. Perez, PSA-based mage auo-annoaon: consranng he laen space, Proceedngs of he h ACM Conference on Mulmeda, Sngapore, 004, pp ISB: ISS: Prn; ISS: Onlne WCE 06