Learning Similarity Measures in Non-orthogonal Space*

Transcription

1 Learig Similarity Measures i N-rthgal Space* Nig Liu, Beyu Zhag, Ju Ya 3, Qiag Yag 4, Shuicheg Ya, Zheg Che, Fegsha Bai, Wei-Yig Ma Departmet Mathematical Sciece, sighua Uiversity, Beiig, 00084, PR Chia liu0@mailstsighuaeduc bai@mathtsighuaeduc Micrst Research Asia, 49 Zhichu Rad, Beiig 00080, PR Chia {byzhag, zhegc, wyma}@micrstcm v-scya@msrchiaresearchmicrstcm 3 LMAM, Departmet Irmati Sciece, Schl Mathematical Sciece, Peig Uiversity, Beiig 0087, PR Chia yau@mathpueduc 4 Departmet Cmputer Sciece, Hg Kg Uiversity Sciece echlgy, Hg Kg qyag@csusth ABSRAC May machie learig data miig algrithms crucially rely the similarity metrics he Csie similarity, which calculates the ier prduct tw rmalized eature vectrs, is e the mst cmmly used similarity measures Hwever, i may practical tass such as text categrizati dcumet clusterig, the Csie similarity is calculated uder the assumpti that the iput space is a rthgal space which usually culd t be satisied due t syymy plysemy Varius algrithms such as Latet Sematic Idexig (LSI were used t slve this prblem by prectig the rigial data it a rthgal space Hwever LSI als suered rm the high cmputatial cst data sparseess hese shrtcmigs led t icreases i cmputati time strage requiremets r large scale realistic data I this paper, we prpse a vel eective similarity metric i the -rthgal iput space he basic idea ur prpsed metric is that the similarity eatures shuld aect the similarity bects, vice versa A vel iterative algrithm r cmputig -rthgal space similarity measures is the prpsed Experimetal results a sythetic data set, a real MSN search clic-thru lgs, 0NG dataset shw that ur algrithm utperrms the traditial Csie similarity is superir t LSI Permissi t mae digital r hard cpies all r part this wr r persal r classrm use is grated withut ee prvided that cpies are t made r distributed r prit r cmmercial advatage that cpies bear this tice the ull citati the irst page cpy therwise, r republish, t pst servers r t redistribute t lists, requires prir speciic permissi /r a ee CIKM 04, Nvember 8-3, 004, Washigt DC, USA Cpyright 004 ACM /04/00 $500 *his wr cducted at Micrst Research Asia Categries Subect Descriptrs I53 [Patter Recgiti]: Clusterig Applicatis similarity measures, text prcessig Geeral erms Algrithms, Measuremet Keywrds Similarity Measures (SM, Vectr Space Mdel (VSM, N- Orthgal Space (NOS, Latet Sematic Idexig (LSI INRODUCION he perrmace may data miig algrithms such as dcumet clusterig text categrizati critically depeds a gd metric that relects the relatiship betwee the data bects i the iput space [] [5] It is therere imprtat t calculate the similarity as eectively as pssible [] I the classical Vectr Space Mdel (VSM [], queries dcumets are represeted as vectrs terms hese vectrs deie a iput space where each distict term represets a axis that space he the similarity tw dcumets r tw queries equals t the csie the agle betwee the high dimesi vectrs idexed by the terms i crpus [9] his apprach is a eective apprximati, but it is evertheless a versimpliicati he mar limitati is that it assumes that the terms are idepedet, ie the dimesis the iput space are rthgal Hwever, i text applicati, the iput space is usually -rthgal due t the llwig issues here are tw cmm prblems with the Vectr Space Mdel [], [4] he irst is syymy Fr example, the wrd buildig ca als be represeted by huse r cstructi he secd is plysemy which meas that mst wrds have mre tha e meaig Fr example, paper reers t a material made 334

2 cellulse pulp, a rmal writte cmpsiti iteded t be published, r eve a icial dcumet hese acts shw that the terms are t idepedet, ie the space is a -rthgal e hus the similarity measured by csie r ier prduct based Euclidea s distace ca t exactly describe the relatiship betwee bects Recetly, attempts have bee made t icrprate sematic wledge with the vectr space represetati Latet Sematic Idexig (LSI [4] [5] [6] []is a well-w apprach amg them LSI attempts t capture the term-term statistical relatiships I LSI, the dcumet space i which each dimesi is a actual term ccurrig i the cllecti is replaced by a much lwer dimesial dcumet space called the LSI space i which each dimesi is a derived ccept Dcumets are represeted by LSI space vectr similarity ca be calculated i the same way i LSI space as the traditial VSM Nevertheless, LSI has its w weaess Fr istace, the ccept space LSI is hard t be explaied ituitively Ather prblem is that the cmputatial cmplexity SVD algrithm is t high, which is O(N 3, where N is less value the umber terms dcumets, which maes LSI a time-csumig prcess All these prblems mae the LSI algrithm ieasible r large scale, sparse data sets Mrever, hw t determie the ptimal reduced dimesiality is still t slved prperly Amg the varius appraches used t deal with -rthgal space prblem, which gaied mre iterest recetly, the distace metric learig apprach [] [3] is based psig metric learig as a cvex ptimizati prblem ther appraches [7] used the Mahalabis distace t describe the similarity Hwever, all them are supervised learig algrithms which lear the similarity metric r distace matrix that critically depeds traiig samples Due t this reas, such methds are limited i lexibility due t the lac eugh traiig data which te ccurs i mst real tass I this paper, we prpse a vel iterative similarity learig apprach t measure the similarity amg the bects i the rthgal eature space Our prpsed algrithm is based a ituitive assumpti that the similarity eatures shuld aect the similarity bects vice versa Cmpared with the traditial algrithms such as LSI, ur methd has the llwig advatages: ( it ca lear the similarity i the rigial eature space hus, it preserves the sparse structure the rigial data csequetly the strage cmplexity is lwer; ( the time cmplexity is much lwer tha SVD Experimetal results shw that ur algrithm utperrms Csie similarity, is wrse te is better tha LSI he rest the paper is rgaized as llws I Secti, we itrduce sme bacgrud wledge similarity measuremet learig, such as Csie similarity, LSI algrithms supervised algrithms Fllwig that, we preset the prblem rmulati the detailed algrithm i Secti 3 he experimetal results the sythetic datasets the real data are demstrated i Secti 4 Cclusi uture wr are preseted i Secti 5 BACKGROUND May irmati miig prblems, such as text classiicati text clusterig are suerig rm the prblems t uderst the represetati dcumets t capture the relatiship betwee dcumets I this secti, we will itrduce three ppular appraches relative t these prblems: Csie similarity, Latet Sematic Idexig supervised learig algrithms Csie Similarity i Vectr Space Mdel Vectr Space Mdel (VSM is the irst apprach t represet dcumets as a set terms [3] [9] his set terms deies a space such that each distict term represets the etries i that space Sice we are represetig the dcumets as a set terms, we ca view this space as a dcumet space We ca the assig a umeric weight t each term i a give dcumet, represetig a estimate the useuless the give term as a descriptr the give dcumets [8] he weights assiged t the terms i a give dcumets ca the be iterpreted as the crdiates the dcumet i the dcumets space Fr istace, B is the matrix term by dcumets i the case text data, t d where bi, is the term weightig he, a vectr similarity ucti, such as the csie the agle betwee the high dimesi vectrs idexed by the terms i crpus, ca be used t cmpute the similarity matrix amg dcumets Fr tw vectrs b b the Csie similarity is give by: i sim b, cs i b ( bi, b cs( θ bi b where θ is the agle betwee < > = = ( bi b Fr the term by dcumet matrix B, i the etire clum vectrs i B have bee rmalized, the similarity matrix will be: cs (, sim dc dc = B B ( Fr a ideal similarity measure, the maximum similarity is e, crrespdig t the tw dcumet vectrs beig idetical (agle betwee them is zer he miimum similarity is zer, crrespdig t the tw vectrs havig terms i cmm (agle betwee them is 90 degree Others shuld betwee zer e sice ay csie value a acute agle shuld betwee zer e he mar limitati Csie similarity is that it assumes the terms are idepedet, rthgal dimesis the space, but i act it is usually t rthgal hus mistaes ccur uder this assumpti Fr istace, csider the llwig tw seteces: C: Huma machie iterace r Lab ABC cmputer applicatis C5: Relati user-perceived respse time t errr measuremet Usig the simplest cutig strategy t establish a vectr space mdel, the traspsed bect matrix B culd be shw i able able VSM Example cmputer Huma iterace respse time user C C Fllwig the example abve, c c5 are bth talig abut the huma-cmputer iteracti [4], thus similar i sematic space Hwever, the Csie similarity c c5 by rmula ( are zer which meas that they are t similar at all his mistae ccurs due t ur assumpti that the iput space is rthgal while it is t he terms are t rthgal due t there are 335

3 crrelatis amg dieret wrds, r example, user i c5 huma i c have strg relatiship i sematic space, but we assume that they are rthgal basis the vectr space Figure gives the ituitive iterpretati Csie similarity i rthgal space Figure Csie similarity i -rthgal space Sice the basis vectrs are t rthgal, the prectis the cmpets vectrs are t the same Fr simplicity, we csider a tw dimesial case Suppse that e e are basis vectrs (ie, terms a vectr space mdel, a b are tw vectrs (ie dcumets i this space Sice the basis vectrs are t rthgal, the prectis (pictured by bre lie with square ed the cmpets (pictured by dt lie these vectrs are t the same he Csie similarity a b which csiders the cmpets as prectis will certaily lead t the wrg sluti hus, the Csie similarity is less suitable r -rthgal vectr space Latet Sematic Idexig Latet Sematic Idexig (LSI [6] [4] is a irmati retrieval methd desiged t vercme tw cmm prblems i irmati retrieval: syymy plysemy I ther wrds, LSI aims at prectig rigial data i vectr space t a rthgal space i where the Csie similarity will t lead t mistaes Frm a very high level, LSI tries t tae advatage the cceptual ctet dcumets A techique w as Sigular Value Decmpsiti (SVD is used t create this ccept space Belw is a simplistic verview what happes i the preprcessig stage LSI hw SVD is used [4] LSI taes the rigial term by dcumet matrix B m as iput he, the SVD precti is cmputed by decmpsig matrix B it the prduct three matrices B = S D, m m m m where N = mi( m,, D have rthrmal clums S is diagal By restrictig the matrixes, D S t their irst < rws e btais the matrix B = S D B is m m the best square apprximati B by a matrix ra he ier prduct betwee tw clum vectrs B relects the extet t which tw dcumets have a similar prile terms hus the matrix B B ctais the dcumet-t-dcumet similarity: LSI sim ( dc, dc = B B = DS D (3 he research it LSI s ar has bee ecuragig Hwever, LSI has sme shrtcmigs i perrmace Oe is the cmputatial cmplexity SVD algrithm is O(N 3 where N is the mir value betwee the umber terms dcumets It maes the LSI algrithm ueasible r large sparse dataset Meawhile, LSI has e b a e the strage space prblem Ater perrmig a SVD, the apprximate matrix B is t a sparse matrix Furthermre, the prblem with LSI is that the ccept space is hard t uderst by humas the similarity betwee dcumets will appear egative values Besides, a parameter uder the user s ctrl ca aect the irmati reservati, ie the chice parameter is still a peig issue dieret chice may be aect the ial similarity greatly 3 Supervised Similarity Learig Recetly, researchers have csidered usig distace metrics t measure the similarity bects hey preset sme algrithms that ca lear a distace metric t icrease the accuracy irmati retrieval [] [3] he Mahalabis distace is a very csiderable way determiig the similarity a dataset Mahalabis distace uses B AB t replace the rigial ier prduct, where A is a parameterized amily distace metrics he learig distace algrithms discrimiately searches r the parameters that best ulill the traiig data hese methds were shw t be very eective; hwever, measurig the similarity idividual web bects may t be precise whe usig their algrithms due t eugh traiig data t lear the parameters 3 LEARNING SIMLARIY IN NON- ORHOGONAL SPACE I this secti, we preset the prblem rmulati give the detailed similarity learig algrithm i -rthgal space [0] We derivate ur algrithm rm a simple ituitive assumpti: the similarity eatures shuld aect the similarity bects vice versa Fllwig that is the cvergece pr this algrithm he, the parameter chsig strategy is give at the ed this secti 3 Prblem Frmulati Sice we are iterested i t ly the prblem terms dcumets, but als ther irmati retrieval prblems such as measurig the similarity amg queries pages s, i the iterest geerality, we use bect eature t represet the detailed prblems such as dcumets terms r queries pages i the sequel he classical Csie similarity is i act the ier prduct tw rmalized vectrs i a rthgal vectr space Mistaes ccur sice we apply it i the -rthgal space his mtivates us t give a vel similarity measure which is suitable r a -rthgal space Suppse that x space, y are bects i a -dimesial vectr H he i th etry x is deted as x i which is the value x the i th eature Let x be the traspse vectr x Matrix is represeted by capital letter B whse clums are bects i H b, i is the value the th i bect the th eature space Sice these bag wrds lie vectrs have very sparse structure, this sparse structure culd save strage requiremet greatly perrm quic algrithm s as t save time requiremet As discussed abve, ur prblem is learig the similarity based the eature-bect matrix B which the eatures may t be idepedet, ie the space is a rthgal e, preservig the sparse structure matrix B 336

4 3 Equatis r Similarity Learig here are may appraches such as erel based algrithms [7] measure the similarity i -rthgal space by, i i S(, = P, where i are tw bects P is a sematic prximity matrix satisies the symmetric psitive semi-deiite Let m B R be a eature by bect matrix, which culd be led as bects i eature space rm clums r eatures i bect space rm rws Suppse S S are the similarity matrix betwee bects the similarity matrix betwee eatures respectively Firstly, let us iterpret ur basic assumpti, the similarity eatures shuld aect the similarity bects vice versa I ther wrds, tw bects shuld be mre similar i their eatures are mre similar; the ther h, tw eatures shuld be mre similar i their crrespdig bects are mre similar, the tw actrs shuld aect each ther culd t be csidered idividually util cverge herere, uder this assumpti, we culd adapt the similarity measure i a rthgal space t a -rthgal rmulati I ctrast t the Csie similarity which is mre suitable r rthgal space, we itrduce ur iteractive similarity measuremet i a -rthgal space, S = B S B S = BS B We assume that the similarity measuremet is rmalized I ther wrds, the btaied similarity values shuld betwee zer e hat is, i tw bects are t similar at all, the the similarity betwee them shuld be zer; i tw bects are the same, the the similarity betwee them shuld be e; therwise, the similarity amg bects shuld t less tha zer t larger tha e I rder t satisy this cstrai r cveiece, we rmalize ur iteractive similarity by tw psitive real parameters λ, λ slve these similarity iteratively, S + = S λ BS B + = B S B λ We will prve that i λ < B, λ < B, the etries similarity matrices will betwee zer e i the pr lemma i the appedix Hwever, sice the similarity the same bect shuld be e, we ca assig the diagal the similarity matrix a scre he we rewrite the recursive equatis as: + = λ + S B S B L (4 = BS B + L (5 λ where L = I diag( λb S B, L = I diag( λbs B λ, λ are psitive real parameters which satisy λ < B, λ < B I this paper, we call (4 (5 the basic Similarity equatis i the N-Orthgal Space (SNOS We chse iitial value 0 i i = Si = 0 i i r the iteractive iterati prcess I ther wrds, we tae 0 0 S = S = I t iitialize this iterati algrithm I act, there are may ther variatis ur algrithm i we relax cstrais the parameters r give ther iitial values Frm the algrithm equati, it culd be see that the vectr space, ie matrix B, has t bee chaged rm the begiig t the ed durig the iterati prcess I ther wrds, we preserved the sparse structure matrix B thus save a lt strage space cmpare with LSI Furthermre the time cmplexity ur algrithm is O( t m, where m is the umber eatures, is the umber bects t is the umber iterati steps he average iterati steps bere cverge is abut 8 by ur experimets I ctrast, i rder t get the similarity matrix eatures bects, the time cmplexity LSI 3 is O( + m which is much higher tha ur prpsed apprach 33 Cvergece Pr We give a pr summary the existece uiqueess r the basic SNOS equatis (4 (5 he detailed pr culd be ud i the appedix Deiiti suppse matrices Krecer Prduct A B is, m A R, p q B R ab a B a B ab ab ab A B = a B a B a B m m m mp q, the their m Deiiti the Rw-First Vectrizati a matrix A R, deted as A, culd be represeted as A = ( a, a,, a m, where a R, i =,,, m are rw vectrs A i m Lemma suppses A R B R are matrices, the ( ABA m R, mrever ( ABA = ( A A B Lemma the similarity matrices S SNOS equatis are buded Lemma 3 the etries similarity matrices S the basic SNOS equatis are -decreasig S deied i the basic S deied i herem the iteractive iterati basic SNOS equatis cverge t a uique sluti Pr: rm lemma 3 we w that S S are buded -decreasig, s they cverge t sme sluti Let s prve the uiqueess the sluti Suppse ( S, S,( S, S are tw dieret grup slutis the basic SNOS equatis he, 337

5 = λ + S B S B L S = λbs B + L S = λ B S B + L S = λbs B + L Fr all the -diagal elemets, chage the represetati by lemma Etries culd be deted as S + = = s ( l, s ( g, l,,,, g,,, m Suppse the elemet i S crrespd t s ( l is s ( i,, we have s ( l s ( l ( b b S ( b b S = λ i λ i λ ( b b ( S S λ i Fr all the diagal elemets s ( l s ( l = = 0 ( b b ( S S he O the ther h i ( S S ( S S < ( S S ( S S < his leads t the cclusi that S = S, S = S 34 Parameter Selecti As metied abve, several parameters are used i the rigial algrithm Fr rmalizati purpses, we chage the diagal elemets similarity matrices it at each iterati step thrugh matrices L L Althugh this rugh revisi will t aect the cvergece algrithm, ituiti tells us that t much this perati will lead t reduced eectiveess ur algrithm Whe chsig a parameter, we wish t miimize the rm L L ater cvergece ccurs I ther wrds, we suggest t slve the parameters thrugh the ptimizati prblems listed bellw: λ = arg mi L = arg mi I λdiag( B S B 0 λ < / B 0 λ < / B λ = arg mi L = arg mi I λ diag( BS B 0 λ < / B 0 λ < / B Nte that the etries matrices diag( BS B diag( BS B are w buded Furthermre, I diag( BS B 0 I diag( BS B 0, the ptimizati prblem culd be chaged it, λ = arg max λdiag( B S B 0 λ < / B λ = arg max λ diag( BS B 0 λ < / B his implicates that we shuld chse the parameters as large as pssible uder the cstrais 0 λ < / B, 0 λ < / B Fr cveiece, we chse a same parameter i ur experimets It must satisies 0 λ < / max{ B, B }, as large as pssible uder this cstrai S we chse: i all ur experimets λ = λ = 09 / max{ B, B } 4 EXPERIMENS I this secti, we discuss the experimetal data set, evaluati metric, the experimetal results based csie similarity, LSI, ur prpsed SNOS he irst experimet is cducted a sythetic data t demstrate the drawbacs csie similarity LSI, which are metied i the previus sectis he secd experimet is perrmed a real MSN clic-thrugh lg data t id similar queries LSI ailed t iish this experimet due t the large scale the data set Our prpsed SNOS achieves 806% imprvemet the precisi similar queries he third experimet is demstrated the perrmace the prpsed algrithm r classiicati 4 he Sythetic Data We cduct the irst experimet a sample dataset csistig the titles 9 techical memra [4] his dataset cmes rm the paper i which LSI was prpsed erms ccurrig i mre tha e title are italicized here are tw classes dcumets - ive abut huma-cmputer iteracti (c-c5 ur abut graphs (m-m4 his dataset ca be described by meas a term by dcumet matrix B, where each cell etry idicates the requecy with a term i a dcumet (able c c c3 c4 c5 m m m3 m4 able echical Mem Example Huma machie iterace r Lab ABC cmputer applicatis A survey user pii cmputer system respse time he EPS user iterace maagemet system System huma system egieerig testig EPS Relati user-perceived respse time t errr measuremet he geerati rm, biary, urdered trees he itersecti graph paths i trees Graph mirs IV: Widths trees well-quasi-rderig Graph mirs: A survey I this simple ituitive example, the Csie similarity i VSM, LSI ( =,3, 8 SNOS are used t cmpute the similarity betwee dcumets Withut lse geerality, the similarity betwee dcumet a all the cllecti is deted as sim( a, dc We use sim( c, dc t shw the sluti dieret appraches, where the bld etries are abrmal (Eg COS the 5th etry vectr sim ( c, dc detes the csie similarity betwee c c5 338

6 LSI ( = sim c dc LSI ( = sim c dc LSI ( = 3 sim c dc ( ( ( COS sim ( c, dc = (, = (, = (, = LSI ( = 4 sim ( c, dc = LSI ( = 5 sim c dc LSI ( = 6 sim c dc LSI ( = 7 sim c dc LSI ( = 8 sim c dc (, = (, = (, = (, = SNOS sim ( c, dc = It ca be see that: ( the Csie similarity betwee c c5 is zer althugh they are i the same predeied class his is due t the -rthgal eatures the iput space; ( there exists sme egative values amg the similarity slved by LSI, the meaig which culd t be ituitively uderstd; (3 the perrmace LSI is crucially depedet the parameter Hwever, the selecti is still a pe issue i LSI I ctrast, ur prpsed SNOS ca desig the similarity eectually, i ther wrds, the similarity values slved by SNOS are all betwee zer e, truly relectig the similarity values the itra class bects are larger tha thse iter class bects 4 he MSN Search Clic-thru Lg Data I this secti, we cmpare SNOS with csie similarity the real MSN clic-thrugh lgs data the tas idig similar queries It is ticed that LSI ailed this data set because SVD ca t deal with the large scale, sparse matrices his is a predmiace ur methd tha LSI 4 Dataset I rder t study the eectiveess SNOS r measurig the similarity web bects, experimets are cducted a real user query clic-thrugh lg cllected by the MSN Web search egie i December, 003 It ctais abut 4 milli query requests recrded sampled rm a perid six hurs he lg we btaied has already bee prcessed it a predeied rmat, ie each query request is assciated with the URL e cliced web page A sigle query (r web page URL ca ccur multiple times i the query clic-thrugh lg Bere ruig the experimets, sme preprcessig steps are applied t the queries web page URLs All queries are cverted t lwer-cases, stemmed by the Prter algrithm he stp-wrds i the queries are remved Ater these steps, the average query legth is abut 7 wrds All URLs are cverted it caical rm by perrmig such tass as replacig usae characters with their escape sequeces cllapsig sequeces lie \ Each URL is csidered as a eature, while each query is treated as a bect (We ca als treat URLs as bects, treat queries as eatures Our prpsed algrithm ca slve similarities betwee bects betwee eatures at the same iterati he weight r a query a URL is the requecy the query leadig t the URL 4 Evaluati Metrics Sice ur prpsed algrithm aims t id better similarity betwee bects, we develped a peratial measure precisi t evaluate the perrmace Give a bect as iput, we as 0 vluteers t idetiy the crrect similar bects rm the tp N retured results by each algrithm he precisi is deied as M Precisi = (6 N where N is the umber tp N similar bects t be evaluated, M is the umber crrect similar bects tagged by the vluteers he ial relevace udgmet r each bect is decided by marity vte I ur experimet, N is set as 0 43 Fidig Similar Queries I this experimet, the vluteers were ased t evaluate the precisi results r the selected 0 queries (which are air ticets, aut trader, ba America, ca cameras, Disey, mapquest, ms ctet, Presari 00, uited airlies, weather reprt Figure shws the cmparis the SNOS apprach with csie similarity We ud that SNOS utperrms the csie similarity i precisi by 806% hrugh careul study the query Presari 00 which was a ppular laptp mdel, we ud that ur prpsed algrithm t ly ca ilter sme u-related queries, but als ca id sme clse-related laptp mdels I able 3 the tag Y represets that the query is similar t the give query Presari 00 ; N idicates t similar Althugh the csie similarity returs sme similar queries (the st 5 th results, it suers rm the tpic drit issue (the 6 th 0 th results, eg Presari 00 Liux Cmpaq 00 share sme cliced web pages, hwever, thse web pages discuss hw t istall Liux i Presari 00, therere, thse cmm cliced web pages is actually a id ise eature r Presari 00, which causes Liux Cmpaq 00 t be retured as a similar query by csie similarity O the ther h, SNOS ids ut similar mdels which are t revealed by the csie similarity Althugh dieret mdels have may dieret cliced web pages, thse cliced web pages are cmputed as similar sice they are queried by similar queries, such as Cmpaq Presari, Cmpaq teb, Cmpaq laptp, etc Hece, dieret mdels have higher similarity based SNOS tha based csie similarity Figure Precisi similarity betwee queries 339

7 able 3 Similar queries r "Presari 00" Csie Similarity SNOS Cmpaq Presari Y Cmpaq Presari Y Cmpaq teb Y Cmpaq teb Y 3 Cmpaq laptp Y Cmpaq laptp Y 4 lie Cmpaq Presari Y lie Cmpaq Presari Y 5 Cmpaq Presari laptp Y Cmpaq Presari laptp Y 6 cmpaque N Cmpaq Presari 964 Y 7 Nteb price cmpare N Presari 800 sale Y 8 Cmpaq supprt N Cmpaq 575us Y 9 Liux Cmpaq 00 N Presari 964 Y 0 Used Cmpaq reseller N Cmpaq batteries N 43 he 0NG Data demstrate the classiicati perrmace the prpsed algrithm, the cmmly used 0NG data was used here ( ml We chse the ive classes abut cmputer which ctais 488 dcumets altgether he Csie similarity i rthgal space was used as the baselie We use the simple Nearest Neighbr classiier t assess the perrmace SNOS 0NG data Figure 3 shws the errr rate Csie similarity i ctrast t ur SNOS he X-axis detes the prprti data used r traiig Figure 3 he classiicati errr rate 0NG dataset by the earest eighbr classiier 5 CONCLUSIONS AND FUURE WORKS I this paper, we prpse a vel apprach t calculate the similarity metric i the -rthgal space (SNOS I ctrast t LSI, which aims at slvig the -rthgal space similarity prblem, ur prpsed apprach has the llwig advatages: ( lwer cst bth i strage cmputati ( preserves the sparse structure VSM by avidig rthgalizig the iput space We demstrated the disadvatage classical Csie similarity i the -rthgal space shwed the perrmace SNOS by experimets a sythetic dataset as well as a large sampled MSN search clic-thru lg I the ext step, we will explre the eiciecy, eectiveess, geerality the SNOS apprach We tice that there is csiderable rm r imprvemet i the quality clusterig bth Web pages queries We believe that SNOS culd be successully applied t imprve the eectiveess clusterig Fially, we pla t apply ur apprach t mre cmplex settigs, such as i digital libraries, where there are mre types bects, s as t demstrate urther the geerality SNOS Besides, we will give sme variatis SNOS i ur uture wr 6 ACKNOWLEDGEMEN Ju Ya thas the supprt the Lab Mathematics Applied Mathematics Peig Uiversity Qiag Yag wuld lie t tha Hg Kg RGC r their supprt 7 REFERENCES [] Ad, RK, Latet Sematic Space: Iterative Scalig Imprves Precisi Iter-dcumet Similarity Measuremet I Prceedigs the SIGIR, (Athes, Greece, 000, 6--3 [] Atau, S, Bruie, L Ksch, H, Similarity-Based Operatrs Query Optimizati r Multimedia Database Systems I Prceedigs the Iteratial Database Egieerig Applicati Sympsium, (Greble, Frace, 00, [3] Baeza-Yates, R Ribeir-Net, B Mder Irmati Retrieval Addis Wesley Lgma, 999 [4] Deerwester, S, Dumais, S, Furas, GW, Lauer, K Harshma, RA Idexig by Latet Sematic Aalysis Jural the America Sciety r Irmati Sciece, 4( [5] Dumais, S, Letsche, A, Littma, ML Lauer, K, Autmatic crss-laguage retrieval usig latet sematic idexig I Prceedigs the AAAI Sprig Sympsuim Crss-Laguage ext Speech Retrieval, (997 [6] Dumais, S, Platt, J, Hecerma, D Sahami, M, Iductive Learig Algrithms Represetatis r ext Categrizati I Prceedigs the 7th Iteratial Cerece Irmati Kwledge Maagemet, (Bethesda, Maryl, 998 [7] Kla, J, aylr, JS, Cristiaii, N Davis, Learig Sematic Similarity I Prceedigs the Neural Irmati Prcessig Systems (NIPS, (Whistler, BC, 003 [8] Lepld, E Kiderma, J ext Categrizati with Supprt Vectr Machies Hw t Represet exts i Iput Space? Machie Learig, [9] Salt, G McGill, MJ Itrducti t Mder Retrieval McGraw-Hill B Cmpay, 983 [0] Sers, I Key, L, Heuristics r placig rthgal axial lies t crss the adacecies betwee 340

8 rthgal rectagles I Prceedigs the 3th Caadia Cerece Cmputatial Gemetry, (00, [] Schultz, M Jachims,, Learig a Distace Metric rm Relative Cmparis I Prceedigs the Neural Irmati Prcessig Systems, (Whistler, BC, 003 [] su, AS Ferhatsmaglu, H, Vulerabilities i Similarity Search Based Systems I Prceedigs the th ACM Iteratial Cerece Irmati Kwledge Maagemet (CIKM, (McLea,VA, 00 [3] Xig, E, Y, A, Jrda, M Russell, S, Distace Metric Learig, with Applicati t Clusterig with Side- Irmati I Prceedigs the Neural Irmati Prcessig Systems (NIPS, (Whistler, BC, 003 [4] Zelivitz, S Hirsh, H, Usig LSI r ext Classiicati i the Presece Bacgrud ext I Prceedigs the 0th ACM Iteratial Cerece Irmati Kwledge Maagemet (CIKM, (Atlata, US, 00, ACM Press, New Yr, US, 3--8 [5] Zhai, C Laerty, J, Mdel-based Feedbac i the Laguage Mdelig Apprach t Irmati Retrieval I Prceedigs the 0th ACM Iteratial Cerece Irmati Kwledge Maagemet (CIKM, (Atlata, US, 00, APPENDIX m Deiiti 3: the -rm matrix A = ( a R, is A = max{ i= ai } m m Deiiti 4: the iiite-rm matrix A = ( a R, is A = max{ = ai } i m m Lemma r matrices A R, B R, the Lie-First m Vectrizati matrix ABA equal t a vectr ( A A B R Pr: a A a A a A b aa a A a A b ( A A B = ama ama am A b m a, ia, bi, i= = m a, ia, bi, = i= = = ( ABA m a a b m, i m, i, i= = mm Lemma the similarity matrices S, S deied i ( ( are buded i i + Pr: rm lemma, S ( B B = λ S + L Etries as S + = = (4 culd be deted s ( l, s ( g, l,,,, g,,, m Frm the iitial value, we w that i crrespd t s s ( l Suppse the elemet ( g is s + ( i, + Nte that s ( g = whe its crrespdig elemet i matrix is a diagal elemet, ie i =, due t L Otherwise, whe i, i 0 s ( l, l =,,,, 0 s ( g = ( b b S ( b b S + λ i i B ( bi b S b b i where bi is the i th lie matrix B bi is the umber zer elemets i b i he we culd draw the cclusi that the etries belg t [0,] by iducti I the same way, 0 r all Lemma 3 the etries similarity matrices S, S deied i ( ( are -decreasig Pr: he same as the pr lemma, we trasrm the algrithm it a equal rmulati: + S = λ ( B B S + L + S = λ ( B B L It is bviusly that s ( l = i it crrespds t a diagal etry matrix S due t the eectiveess L O the ther h, r all the -diagal etries, s ( l 0 = s ( l sice the iitial value 0, 0 S s 0 ( l is zer We culd draw the cclusi that S I S S, r all the -diagal etries s ( i, wh crrespd t s ( l, we have, s ( l s ( l ( b b S ( b b S + = λ i λ i = λ ( bi b ( S S 0 Fr all the crrespdig diagal etries have s ( l = Frm the discussi taled abve, the etries S S, we are decreasig Mrever, the etries S are -decreasig due t the same reas [3] 34