Fusion of Retrieval Models at CLEF 2008 Ad-Hoc Persian Track

Transcription

1 Fusio of Rtrival Modls at CLEF 008 Ad-Hoc Prsia rack Zahra Aghazad*, Nazai Dhghai* Lili Farzivash* Razih Rahimi* Abolfazl AlAhmad* Hadi Amiri Farhad Oroumchia** * Dpartmt of ECE, Uivrsity of hra {z.aghazadh,.dhghay, l.farzivash, r.rahimi}@c.ut.ac.ir ** Uivrsity of Wollogog i Dubai FarhadO@uow.du.au Abstract Mtasarch gis submit th usr qury to svral udrlyig sarch gis ad th mrg thir rtrivd rsults to grat a sigl list that is mor ffctiv to th usrs iformatio ds. Accordig to th ida bhid mtasarch gis, it sms that mrgig th rsults rtrivd from diffrt rtrival modls will improv th sarch covrag ad prcisio. I this study, w hav ivstigatd th ffct of fusio of diffrt rtrival tchiqus o th prformac of Prsia rtrival. W us a xtsio of Ordrd Wightd Avrag (OWA) oprator calld IOWA ad a wightig schma, NOWA for mrgig th rsults. Our xprimtal rsults show that mrgig by OWA oprators producs bttr prcisio. Catgoris ad Subjct Dscriptors Iformatio Sarch ad Rtrival, Rtrival Modls. Kywords Iformatio Rtrival, Iformatio Fusio, Prsia xt Rtrival.. Itroductio With th rapid growth of th volum of th data, improvig th ffctivss of iformatio rtrival systms is sstial. May approachs ad mthods hav dvlopd to xhibit bttr rtrival gis []. I this study, w try to us th ida bhid mtasarch gis i ordr to improv th rsults of Prsia iformatio rtrival. W cosidr ach rtrival modl as a dcisio makr ad th fus thir dcisios with a OWA oprator i ordr to icras th ffctivss. his work has b do as our first participatio i th CLEF valuatio campaig. For th Ad-Hoc Prsia track w submittd lv xprimts (rus): UNLPDB3BB, UNLPDB3BM5, UNLPDB3DFR, UNLPDB3IFB, UNLPDB3INEXPB, UNLPDB3INEXPC, UNLPDB3INL, UNLPDB3PL, UNLPDB3FIDF, UNLPDB3NOWA ad UNLPDB3OWA. Our mai goal was to study th ffct of fusio oprators ad whthr fusig rtrival modls ca brig additioal prformac improvmts. h collctio that is usd i this study is a stadard tst collctio of Prsia txt which is calld Hamshahri ad was mad availabl to CLEF by Uivrsity of hra [], [3]. I sctio two, w prst a brif dscriptio of th rtrival mthods that hav b usd i our xprimts. Prvious xprimts hav dmostratd that ths mthods hav good prformac o Prsia rtrival. I sctio thr, OWA oprator ad its xtsios that ar usd for mrgig th rsults ar dscribd. O ky poit i th OWA oprator is to dtrmi its associatd wights. I this study, w us a wightig modl which is basd o Normal distributio ad a IOWA xtsio. hr ar two approachs to fus th rtrivd lists: Combi th rsults of distict rtrival mthods. Combi th rsults of th sam mthod but with diffrt typs of toks Rus that submittd to CLEF 008 us th first approach ad rsults show that usig this approach dos ot ld itslf to a sigificat improvmt. It sms although th rtrival mthods ar diffrt but thir prformac ad rsult st is similar. I aothr word, thos rtrival mthods provid th sam visio of th data. Aftr

2 CLEF rsults wr publishd, w trid th scod approach ad w wr abl to improv th ffctivss up to 5.67% ad rachd th 45.% avrag prcisio o tst st. Sctio four dscribs th xprimts ad thir rsults.. Rtrival Mthods I this work, for th purpos of fusio, w dd diffrt rtrival mthods. Aftr studyig diffrt rtrival toolkits, fially w choos rrir [4]. Diffrt mthods hav b implmtd i rrir toolkit. Amog ths mthods, w slctd i of thm. h wightig modls ad a brif dscriptio of thm (from [5]) ar illustratd i tabl. Wightig Modl BB abl A dscriptio of rtrival mthods Dscriptio Bos-Eisti modl for radomss, th ratio of two Broulli's procsss for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio BM5 h BM5 probabilistic modl DFR_BM5 IFB I_xpB I_xpC IL PL F_IDF h DFR vrsio of BM5 Ivrs rm Frqucy modl for radomss, th ratio of two Broulli's procsss for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio Ivrs xpctd documt frqucy modl for radomss, th ratio of two Broulli's procsss for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio Ivrs xpctd documt frqucy modl for radomss, th ratio of two Broulli's procsss for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio with atural logarithm Ivrs documt frqucy modl for radomss, Laplac succssio for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio Poisso stimatio for radomss, Laplac succssio for first ormalizatio, ad Normalizatio for trm frqucy ormalizatio h tf*idf wightig fuctio, whr tf is giv by Robrtso's tf ad idf is giv by th stadard Sparck Jos' idf abl dpicts th rsult obtaid from ruig th abov i mthods dscribd i abl o th traiig st of quris.

3 abl Compariso btw diffrt wightig modls Wightig Modl Avrag Prcisio R-Prcisio BB BM DFR_BM IFB I_xpB I_xpC IL PL F_IDF OWA Fuzzy Oprator his sctio dscribs th Ordr Wightd Avrag (OWA) oprator, ormal distributio-basd wightig ad IOWA xtsio. 3.. OWA Dfiitio A OWA oprator of dimsio is a mappig, OWA: w = ( w w,..., w, ) such that w j [0,] ad Whr w j =. Furthrmor, OWA ( a, a,..., a ) = R R, that has a associatd vctor b j w j b j is th j th largst lmt of th collctio of th aggrgatd objcts 3.. IOWA A IOWA oprator is dfid as follows: IOWA( u =, a, u, a,..., u, a ) w j = j b j, that w j =, 0 w j ad b j is th a i valu of th OWA pair a, a,..., a [6]. w = ( w, w,..., w ) whr is a wightig vctor, such u i, ai havig th j th largst u i, ad u i i u i, a i is rfrrd to as th ordr iducig variabl ad a i as th argumt variabl. It is assumd that ai is a xact umrical valu whil ui ca b draw from ay ordial st Ω [7] NOWA Suppos that w wat to fus prfrc valus providd by diffrt idividuals. Som idividuals may assig uduly high or uduly low prfrc valus to thir prfrrd or rpugat objcts. I such a cas, w shall assig vry low wights to ths fals or biasd opiios, that is to say, th closr a prfrc valu (argumt) is to th mid o(s), th mor th wight it will rciv; covrsly, th furthr a prfrc valu is from th mid o(s), th lss th wight it will hav [8]. Lt w = ( w, w,..., w ) b th wight vctor of th OWA oprator; th w dfi th followig:. [( i µ ) ] σ w i =, i =,,..., πσ Whr µ is th ma of th collctio of,,...,, ad σ ( σ > 0) is th stadard dviatio of th collctio of,,...,. µ adσ ar obtaid by th followig formulas, rspctivly:

4 Cosidr that w j w µ σ ( + ) + = = = i= = ad 0 w j th w hav: i = πσ πσ i µ ) σ ] j µ ) σ ] = ( i µ ) i µ ) σ ] j µ ) σ ], i =,,..., 4. Exprimt For th xprimts, CLEF has obtaid th stadard Prsia tst collctio which is calld Hamshahri. Hamshahri collctio is th largst tst collctio of Prsia txt. his collctio is prpard ad distributd by Uivrsity of hra. h third vrsio of Hamshahri collctio is 600MB i siz ad cotais mor tha 60,000 distict txtual ws articls i Prsia [9]. hr wr 50 traiig quris with thir rlvac judgmts ad 50 tst quris prpard for th Prsia ad-hoc track. For th CLEF, w choos i mthods of documt rtrival dscribd abov ad fus th top hudrd rtrivd rsults from ach of thm. W us OWA oprator basd o ormal distributio wightig for mrgig th lists. I this problm, w hav i dcisio makrs, so th wightig vctor is as th followig: = 9, µ 0 9 = 5, σ 9 =, ors( w) = 0.5, disp( w) =.95, 3 w = (0.0506,0.0855,0.43,0.557,0.678,0.557,0.43,0.0855,0.0506) h prcisio-rcall diagram obtaid aftr submittig th OWA ru to CLEF is illustratd i figur. Figur - h rsult of ruig NOWA publishd by CLEF 008 IOWA xtsio was also tstd. W usd 50 traiig quris i ordr to calculat th wightig vctor for this mthod. W ra th i slctd rtrival mthods o th collctio. h followig wightig vctor is obtaid by usig th avrag prcisio of ach mthod as its wight: {0.467/3.8409, /3.8409, /3.8409, 0.438/3.8409, 0.439/3.8409, 0.446/3.8409, 0.4/3.8409, /3.8409, 0.40/3.8409} ( is th sum of th obtaid avrag prcisios) Figur illustrats th prcisio-rcall diagram of IOWA ru with th abov wightig vctor.

5 Figur - h rsult of ruig IOWA publishd by CLEF Aalyzig th Rsults ad Mor Exprimts W submittd top hudrd rtrivd documts for our rus to CLEF, whil CLEF valuats th rsults by top thousad documts which dcrasd avrag prcisio about 0% i avrag. hrfor, i futur w itd to calculat our Prcisio-Rcall charts ad othr masurmts basd th top thousad rtrivd documts. h rsults publishd by CLEF for our fusio rus show that usig fusio tchiqus o ths mthods dos ot yild to improvd rsults ovr th idividual mthods. By aalyzig th lists obtaid from th rtrival mthods, w obsrvd that ths rsult lists for ths i diffrt mthods hav high ovrlap amog thm. O th othr had, fusio mthods work wll wh thr ar sigificat diffrcs btw dcisio makrs. hrfor, w hav cocludd that although th mthods ar diffrt thy ar ot sigificatly diffrt from ach othr ad basically thy provid th sam viw of th collctio. Aftr th CLEF rsults wr publishd, w dcidd to ivstigat th scod approach for fusio ad look ito th ffct of diffrt toks i rtrival. For this purpos w chos a vctor spac modl ad ra it o th traiig st thr tims with thr diffrt typs of toks amly 4-grams, stmmd sigl trms ad ustmmd sigl trms. o obtaiig bst rsults, w ra PL mthod of trrir toolkit o 4gram trms, idri of lmur toolkit [0] o stmmd trms ad F_IDF of trrir toolkit o ustmmd trms. h w applid th abov OWA mthods ad as show i tabl 3, w obtaid 9.97% improvmts ovr idividual rus. abl 3 Rsults of applyig fusio mthods o traiig st Rtrival Mthod Avrag Prcisio R-Prcisio Dif F_IDF with ustmmd sigl trms PL with 4gram trms Idri with stmmd trms IOWA NOWA Aftr that, w cotiud this approach ad did mor xprimts with th CLEF tst st. O th tst st, this approach lad oly to 5.67% improvmts o th avrag prcisio ovr idividual rus usig NOWA mthod ad 5.6% usig IOWA mthod. abl 4, figur 3 ad figur 4 dmostrat th obtaid rsults. abl 4 Rsults of applyig fusio mthods o tst st Rtrival Mthod Avrag Prcisio R-Prcisio Dif F_IDF with ustmmd sigl trms PL with 4gram trms Idri with stmmd trms IOWA NOWA

6 Figur 3 Compariso of prcisio btw NOWA ad idividual mthods. Figur 4 - Compariso of prcisio at top rtrivd documts btw NOWA ad idividual mthods. 6. Coclusio Our motivatio for participatio i th Ad-Hoc Prsia track of CLEF was ivstigatig th ifluc of fusio tchiqus o th ffctivss of Prsia rtrival mthods. First w us i rtrival mthods ad th fus th rsults by NOWA ad IOWA. h obtaid rsults showd that fuctioality of ths mthods hav high ovrlap ad thr wr o cosidrabl improvmt by applyig fusio tchiqus. I th scod stag, w chagd our approach to us diffrt vrsios of a sam mthod. o rach this goal, w focusd o workig with diffrt trms istad of diffrt mthods. Rsults idicats fusio tchiqus works wll o th circumstacs which th dcisio makrs hav diffrt viws.

7 I futur, w will ivstigat th ffcts of diffrt tok typs ad rtrival gis o Prsia rtrival ad will try to fi tu a gi basd o fusio. Rfrcs [] Ia H. Witt, Alistair Moffat ad imothy C. Bll. Maagig Gigabyts: Comprssig ad Idxig Documts ad Imags. Morga Kaufma Publishrs, Los Altos, USA, 999 [] Oroumchia F, Darrudi E. Exprimts with Prsia xt Comprssio for Wb. WWW 004. [3] [4] [5] [6] Yagr RR. O ordrd wightd avragig aggrgatio oprators i multicritria dcisio makig. IEEE ras Syst Ma Cybr 988; 8: [7] Yagr RR, Filv D.P. Iducd ordrd wightd avragig oprators. IEEE rasactios o Systms, Ma, ad Cybrtics Part B 9 (999) [8] Xu Z. Iducd ucrtai liguistic OWA oprators applid to group dcisio makig. Itratioal Joural of Itlligt Systms, Vol. 0, (005). [9] Amiri H, Alahmad A, Oroumchia F, Lucas C, Rahgozar M. Usig OWA Fuzzy Oprator to Mrg Rtrival Systm Rsults, 007 [0]