A Tri-Valued Belief Network Model for Information Retrieval

Transcription

1 December 200 A Tr-Vlued Belef Networ Model for Informton Retrevl Fernndo Ds-Neves Computer Scence Dept. Vrgn Polytechnc Insttute nd Stte Unversty Blcsburg, VA IR models t Combnng Evdence Grphcl Models [Perl88] [Buntne94] [Jensen200] llow for greter flexblty t modelng reltons mong dfferent sources, when compred wth other methods le LSI, or stndrd vector models, becuse ech source cn be modeled ndependently. Grphcl models rely on subectve nsted of frequentst understndng of probbltes. Such probbltes do not necessrly represent reltve frequences (lthough they re sometmes clculted le they were, but nsted they represent the degree of certnty, belef or support bout certn feture ( term, ln, etc., nd bout one feture gven nother feture (condtonl probbltes. The most common type of grphcl model ppled to IR s Byesn Networs. The frst successful pplcton of Byesn networs to IR s the Inference Networ Model n [Fung95], [Turtle90]. Belef Networs [Rbero-Neto96] [Slv2000] re n lterntve pproch to Inference Networs when explctly modelng n nformton retrevl system. Whle smlr n expressve power to nference networs, belef networs cn express ny nference networ used to retreve documents by content smlrty, whle the opposte s not necessrly true. The ey dfference s n the modelng of p(d t (probblty of document gven set of terms or concepts n belef networs, s opposed to p(t d used n Byesn networs. Snce n Byesn networ (both of the nference nd belef type, nstnttng d mes p(t d nd p(t 2 d defned nd mutully ndependent, then p(t d cn be clculted from the ndependent probbltes p(d t n belef networ, nd so belef networs cn reproduce the rnng generted by n nference networ. Genertng the rnng produced by belef networ from n nference networ s not lwys possble, s belef networ cn reproduce the cosne smlrty rnng, whle ths s not possble for n nference networ (see [Rbero96] for detls. Fgure 4 llustrtes the dfferences n the probblty structure between the two networs (note the drecton of the rrows between the terms nd the documents: Inference Networ Belef Networ d d2 dm d d2 dm t t 2 t n t t 2 t n q q Fgure 4. Exmples of Inference nd Belef Networs to clculte relevnce of document gven query.

2 In these grphs, n rrow between two nodes represent the condtonl probblty of the vrble ponted by the rrowhed, gven the vlue of vrble t the tl of the rrow. Vrbles not connected by rrows re ssumed to be ndependent. Whle these networs loo lmost dentcl, the condtonl probblty structure s dfferent, nd t hs two mportnt consequences: The hmmoc" structure between document nd query cn be mde to represent vector dotproduct, wth p(d t nd p(q t the two vectors of dmenson n to multply, wth p(d t nd p(q t beng the ndvdul components of the vectors. From ths pont of vew, belef networs consttute tool to combne the best of vector nd probblstc opertons. 2 Belef networs cn clculte ny orderng clculted by n nference networ, ncludng the trdtonl cosne smlrty used n SMART, whle the opposte s not true. An nference networ, on the other hnd, rns the documents by clcultng p(q d usng the chn rule: for the set U of vrbles n the model P ( U = p(vr prents(vr. Inference networs cn model structure tht clcultes rnng smlr to the cosne smlrty, but the clculton hs n extr term tht s document-dependent, nd so for Byesn networs cnnot replcte the rnng gven by cosne smlrty. It s worth notng tht we re showng here only the bsc networs. It s possble to nclude relevnce feedbc s prt of the networ s new nodes, or reltons between terms le synonyms s new nodes connectng relted terms (s long s they do not ntroduce cycles. New nodes lso rse from other sources of nformton, s clusters, or hubs nd uthortes le n CLEVER [Klenberg98]. [Turtle90], [Hnes93] nd [Slv2000] provde more nformton on these extensons of the bsc networ model. Our retrevl scheme, explned n detl below, llows not only for the specfcton of document fetures tht re mportnt or rrelevnt to the user need, but lso llows for neutrl nterpretton, lettng the user sy I don t now. Document fetures tht do not explctly pper n the query formulton re ten s undetermned by the system. Document content s treted s usul. 2. How we perform Retrevl Our model for retrevl s belef networ combnng multple sources of evdence. A belef networ let us combne probbltes nd vector opertons, llowng us fst clculton of smlrty vlues even for bg collectons. Our belef networ s formlly descrbed s follows: 2. Notton An uppercse letter le A letter represents set. One or more lowercse letters wthout subndex, (le d, for exmple, re subsets of other sets (context gven where s used. A lowercse letter wth subndex, le, represents the th element of the set nmed wth the sme letter but n uppercse. A s the number of elements of set A. A s the vector representton of the set A, gven some orderng of the set elements, nd s the th component of A. Note tht vectors nd sets re equvlent.

3 n For set of vrbles x x n, x = = x n 2.2 Vrbles nd ther Reltonshps: 2.2. Structure. The defnton of the belef networ s bsed on the bsc belef networ n [Rbero96]. In our cse, there s one belef networ per document d wth the followng structure: Q Q Q r 2... u... 2 n r r 2... r m K G A G R G d C C... 2 C L Fgure 2. Belef Networ to clculte nd combne evdence from terms, uthors nd references. where d s document, D s the set of ll documents n the collecton, Q ; Q ; Q r represent the query nformton bout document content, uthors nd references, respectvely, KG; AG;RG represent the set of Keywords (Keyword Group, Authors nd References, respectvely, q, q, q r n re the vector representtons of q Q, q Q, q r Q n r, for KG, AG, n RG,, ; r represent prtculr uthor, eyword or reference, C l represents document cluster, for l D. By the structure of the belef networ, we tret content-bsed nd structurl nformton the sme wy. Vrbles, nd r represent ln (lthough not explct between ll documents tht contn the

4 nformton represented by the vrble, whch must pper n t lest one document for the vrble to belong to the byesn networ Vrbles. The followng tble descrbes the domn (possble set of vlues of the vrbles descrbed bove: Vrble Domn Q ; Q ; Q r {, 0} q, q, q {yes, no, unnown} r n,, r {yes, no} KG, AG, RG {relevnt, rrelevnt} d {relevnt, rrelevnt} C l {, 0} Tble 2. Vrbles of the Belef Networ n Fgure 2, nd ther possble vlues. 2.3 Interpretton of vrble vlues. For certn document d; = yes ff the eyword or concept represented by s relevnt for d, otherwse s consdered non-relevnt for d nd = no. In the sme wy nd for the sme document d, = yes or r = yes f the correspondng uthor or reference ppers n d. We cll the sets KG; AG nd RG sources of evdence of d. A query Q from the user s represented s three ndependent nd dsont query subsets Q ; Q ; Q r, so Q = Q Q Qr Q, nd q Q, q Q, qr Qr.If the query formulton ncluded the eyword then q = yes, otherwse q = unnown unless the user hs explctly sd tht q s not relevnt for the query subset Q, cse n whch then. q = no. For exmple, f Q = { ; 4 ; not( 0 }, then q = q = yes, q no 4 = q 0 = no, nd ll other q unnown 0 =. Under ths nterpretton, bsence of evdence bout relevnce of eyword, uthor or reference n the query formulton does not consttute proof of ts rrelevnce. The sme defntons lso pply for ll the q nd r q. Any of query subsets Q ; Q nd Q r s relevnt f the probblty of t lest one of the elements of the query subset s non-ero. Tht s, f we cll G one of the query subsets of Q nd there s some v G such tht p(g v 0, then G s relevnt. A document d s relevnt gven query Q f t lest one of Q ; Q ; Q r, s relevnt. A cluster C l = f t lest one document belongs to C l, nd p(c l = d s the probblty of the nformton n cluster C l beng relted to the content of document d.

5 2.4 Defnton of Probbltes: The dvntge of ths type of belef networ s tht every hmmoc structure cn be nterpreted s vector dot-product, nd so t let us combne vector opertons wth probblstc opertons. When combnng evdence for ll the sources, we wnt tht Absence of evdence from one source does not ffect other sources Evdence from more thn one source s more effectve tht evdence from only one source If source of evdence unquely dentfes document, ll other sources re rrelevnt. A nosy-or [Good6] of the sources stsfes the bove condtons, under the dded ssumpton tht ll sources re ndependent: p(d = relevnt KG = relevnt; AG = relevnt; RG = relevnt = [( w(p(d KG ( w(p(d AG ( wr(p(d RG] [] Here w, w, nd wr weght-dustng functons, tht trnsform ech of the probbltes ccordng to some mesure of the mportnce of the sources. The functons w, w, nd wr must be monotonclly ncresng nd between 0 nd. For the se of dervng some of the probbltes, nd to smplfy the explnton, we re gong to use AG s the source of evdence, n nd s the prtculr nstntton of tht source n prtculr document. Becuse of the smlr structure, the sme rtonle cn be ppled lso to KG n RG, unless otherwse notced. Let us defne the sets A = { set of ll uthors n ll documents }, nd the set to be subset of uthors such tht A, nd vectors d nd d n such tht d 0 f s n uthor of d, nd 0 otherwse, nd d 0 f s not n uthor of d, nd 0 otherwse. n By the fundmentl rule of probbltes we hve p(ag Q p( AG Q =, nd p( Q p( AG Q = p( AG p( Q p( A [2] (see the orgnl Belef Networ pper [Rbero96] for the rtonle of ths equton. If we defne p(d AG = p(ag Q, then gven p(ag ; p(q, nd p( we cn clculte equton [].

6 Let x = p(ag = yes, y = p(ag = no = p(q A - A x s the probblty tht the uthor s one of the uthors of document d. y s the probblty tht uthor s not one of the uthors of document d. s the probblty tht q =. Beng 0 otherwse. We gnore ny uthor tht s not n uthor of ny document n the collecton. For the cse were for ll q Q ech q ether yes or no (the set of uthors of document s completely specfed n the query, both the ones tht re uthors nd the ones tht re not, we desre complete exct mtch between Q nd A to be the df(, the nverse of the number of documents tht hve the set nd only s uthors. The reson for ths s tht for n exct specfcton we wnt the probblty to be proportonl to the number of those cses n the document collecton. To sy ths formlly, for the cse = d dn (tht s, Q + Q s ll s, we wnt, gven tht p( s constnt for ll documents nd ll uthors ( resonble nd common ssumpton for -pror uthor probbltes, Ths we cn pproxmte by x + y p( = = = df ( x + y = = = df ( [3] Snce p( s constnt tht ffects ll document n the sme wy. Plese note tht for the sme ndex., x nd y re never smultneously. Note lso tht = x = d nd = y = A dn 2.5 Boolen Approch. One wy to gve vlues to x, y nd s

7 x = / A ff =yes, y = / A ff =no, nd = ff Q =. Tht s, the probblty of n uthor beng relevnt or non-relevnt s constnt. Then, ( x + = = y = A = For the cse ( ( p( q = (tht s, ll g, where g = / A. q mtched the correspondng, we hve tht [3] then becomes g g A df ( df ( = = = s we wnted. Also, = = A for the cse f only postve nformton (tht s, only uthors who my be uthors of d re specfed n Q, d Q ( equton [3] becomes sm Q, d =, whch s the ntersecton of the boolen sets Q nd d.(q s Boolen n ths cse becuse for postve nformton only, ech component hs ether the vlue yes or unnown. 2.6 Another Approch. However, ths s not the only possble defnton of probbltes. If we defne x nd then t s lso possble to deduce the formulton for y, subect to the constrnt tht the resultng formulton s probblty. Agn, for the cse Q = d + dn (nother wy of syng the query ncluded ll uthors, nd mtched exctly the set of ll uthors of d, we hve tht ll 0. If we defne p(q = yes = yes = p(q = no = no = some constnt c for ll, then we wnt c ( x y = c ( x + y = df ( nd therefore = + = x d y dn d x + ( A d y = df ( [4] c Now we need to defne x. A possble defnton s x df( sup erset d d =, so tht d = x = (sup df erset d where df( sup erset = d number of document tht hve t lest the ll the uthors n set d

8 ths defnton s useful becuse t reltes the probblty of document hvng set of uthors to the se of the set of document uthored by ll those uthors, wth possbly other uthors. Note lso tht 0 df(superset d df(d. Wth these defntons of x nd we cn clculte df ( d d x c d y = therefore y df ( d d x c =. ( d 2 snce 0 x, y, for these defntons to be vld they need to stsfy these two condtons: df ( d d x 0, nd c 2 df ( d d x A d c df ( d Condton mples df (sup erset d = x. Snce 0 c t must be tht c df ( d df (sup erset d, whch s only possble to gurntee when c =. Therefore we defne c = p(q = yes = yes = p(q = no = no =. Wth these vlues for, to probe tht the defnton of x stsfes condton 2 s to prove tht x A + d + df (d. We hve four possble cses: Cse d = nd df(d = / D Here x A + d + df ( d = A + +. Snce df(d = / D nd becuse / D D df(superset d df(d, t must be tht df(superset d = df(d = / D. Snce - A + 0, t follows tht x = A + +. D D

9 Cse 2 d = A nd df(d = / D Followng the resonng n cse, t must be tht df(superset d = df(d=/ D. Furthermore n ths cse - A + d = 0, then x =. D D Cse 3 d = A nd df(d = Now we need to prove A + d + df (d = df(d. Snce d = A mens ll the uthors re x ncluded n d, nd df(d = mens there s only one document mtchng, t follows tht there must be only one document n the collecton. Therefore D =, nd snce d cnnot be empty (we hve to be mtchng t lest one uthor, the only possble nswer s df(superset d =df( s needed. Cse 4 d = nd df(d = x A + d + df (d then x A + + df ( d = A + 2. We need to loo t two subcses, nmely A =, nd A >. If A =, then df(=df(superset d, becuse then the sme uthor s the only uthor of ll documents, nd condton 2 becomes = x A + + df ( d = df ( d. If A > then condton 2, for cse 4 becomes A + + df ( d = 0, whch s lwys true. x Therefore we hve proven cse 4. Ths s ll we need to clculte p(d AG. For ll structurl nformton, s References, Journl where the document ws publshed, or Authors, ll of the bove pples. For content nformton (set KG, we follow the sme pproch s n [Be99] nd we defne p(kg d, =, where d, = df ( tf ( f term =yes n document d, or 0 otherwse, nd d p(q q =, where Q q = weght of term n query Q, ff q = n d, or 0 otherwse.

10 These defntons of p(kg nd p(q, when ppled to equton [2], me the result of equton [2] to be the cosne smlrty between the query nd the document, tmes constnt p(. 2.7 Cost of clcultng p(d Q. Under the bove defnton of the probbltes n the belef networ, t s esy to see tht f vlues le df(d nd df(superset d re precomputed whle the collecton s ndexed, the cost of clcultng p(d Q s O( totl number of uthors + number of Keywords + number of references for ech document d nd query Q, snce the cost of clcultng ech probblty becomes constnt, nd clcultng the mportnce of source s clcultng set of dot-products. 2.8 Document/Cluster probblty. For two documents d nd d 2, we defne p(d d 2 = p(d Q = KG of d 2 ;Q = AG of d 2 ;Q r = RG of d 2 If for moment we ssume we now p(d Cn ; then the probblty of cluster n gven document, p(d Cn p(cn s p(c n d = by Byes Rule. p( d If we ssume the exstence of D clusters (s mny clusters s documents, nd -pror probbltes p(c = p(d constnt for ll D, then p(c n d = p(d C n, whch we cn defne n mny wys, for exmple p ( d Cn = p( d d (verge smlrty between the document nd ll other documents for C n d C n whch p(d C n 0: To vod the problem of ll documents dependng on ll the other documents' probbltes, t the begnnng we cn ntle p(d C n = ff = n; nd 0 otherwse (t the begnnng, ech document belongs to ts own cluster, whch s consstent wth the ntl stte of mny clusterng lgorthms. 3. REFERENCES [Perl88] Perl, J. Probblstc Resonng n Intellgent Systems: Networs of Plusble Inference. Morgn Kufmnn, Sn Mteo, CA, 988. [Buntne94] Buntne, W. Opertons for lernng wth grphcl models. Journl of Artfcl Intellgence Reserch, 994. [Jensen200] Jensen, F. Byesn Networs nd Decson Grphs. Sprnger-Verlg, 200.

11 [Fung95] Fung, R., nd Fvero, B. Applyng Byesn Networs to Informton Retrevl. Communctons of the ACM, Vol 38, No 3, 995. [Turtle90] Turtle, H., Croft, W. Inference networs for document retrevl. In Proceedngs of SIGIR 90, pp.-24, 990. [Slv2000] Slv, I., Rbero-Neto, B., Cldo, P., Mour, E., Zvn, N. Ln-bsed nd content-bsed evdentl nformton n belef networ model. In Proceedngs of SIGIR 2000, pp , [Rbero-Neto96] Rbero-Neto, B., Munt, R. A belef networ for IR. In Proceedngs of SIGIR 96. pp , 996. [Hnes93] Hnes, D., Croft, B. Relevnce feedbc nd nference networs. In Proceedngs of SIGIR 93, pp. 2-, 993. [Klenberg98] Klenberg, J. Authorttve Sources n Hyperlned Envronment. In Proceedngs of ACM-SIAM Symposum on Dscrete Algorthms, pp , 998. [Be99] Be-Ytes, R, Rbero-Neto, B. Modern Informton Retrevl. ACM Press, 999.