Incremental Collaborative Filtering for Highly- Scalable Recommendation Algorithms

Size: px

Start display at page:

Download "Incremental Collaborative Filtering for Highly- Scalable Recommendation Algorithms"

Bryce Foster
6 years ago
Views:

1 Incementl Collbotive Filteing fo Highl- Sclble Recommendtion Algoithms Mnos Ppgelis Ionnis Rousidis Dimitis Plexouskis Elis Theohopoulos 3* Institute of Compute Science FORTH Heklion Geece {ppggel Compute Science Deptment Univesit of Cete Heklion Geece 3 School of Infomtics Univesit of Edinbugh Edinbugh Scotlnd Abstct. Most ecommendtion sstems emplo vitions of Collbotive Filteing CF fo fomulting suggestions of items elevnt to uses inteests. Howeve CF equies expensive computtions tht gow polnomill with the numbe of uses nd items in the dtbse. Methods poposed fo hndling this sclbilit poblem nd speeding up ecommendtion fomultion e bsed on ppoximtion mechnisms nd even if the impove pefomnce most of the time esult in ccuc degdtion. We popose method fo ddessing the sclbilit poblem bsed on incementl updtes of use-to-use similities. Ou Incementl Collbotive Filteing ICF lgoithm i is not bsed on n ppoximtion method nd gives the potentil fo high-qulit ecommendtion fomultion ii povides ecommendtions odes of mgnitude fste thn clssic CF nd thus is suitble fo online ppliction. Intoduction Recommendtion lgoithms e extensivel dopted b both esech nd e- commece pplictions in ode to povide n intelligent mechnism to filte out the excess of infomtion vilble in domin []. Collbotive filteing CF [3] lmost cetinl is the ke method to effotlessl find out items tht uses will pobbl like ccoding to thei logged histo of pio tnsctions. Howeve CF equies computtions tht e ve expensive nd gow polnomill with the numbe of uses nd items in dtbse. Theefoe in ode to bing ecommendtion lgoithms effectivel on the web nd succeed in poviding ecommendtions with high ccuc nd cceptble pefomnce sophisticted dt stuctues nd dvnced sclble chitectues e equied. To ddess this sclbilit poblem we pesent n incementl CF method bsed on incementl * Wok conducted t ICS-FORTH

2 updtes of use-to-use similities tht is ble to ecommend items odes of mgnitude fste thn clssic CF while mintining the ecommendtion qulit. The eminde of the ppe is ognized s follows: Section elbotes on the sclbilit chllenge nd explins the weknesses of led poposed methods fo deling with it. Section 3 pesents ou Incementl CF method. Section 4 gues bout complexit issues of the lgoithms while Section 5 pesents ou expeimentl evlution. Section 6 concludes ou wok nd discusses futue esech diections. The Sclbilit Chllenge fo Collbotive Filteing Clssic CF lgoithm genetes ecommendtions bsed on subset of uses tht e most simil to the ctive use. Ech time ecommendtion is equested the lgoithm needs to compute the similit between the ctive use nd ll othe uses bsed on thei co-ted items so s to pick the ones with simil behvio. Subsequentl the lgoithm ecommends items to the ctive use tht e highl ted b his/he most simil uses. In ode to compute the similities between uses viet of similit mesues hve been poposed such s Peson coeltion cosine vecto similit Spemn coeltion entop-bsed uncetint mesue nd mensque diffeence. Howeve Beese et l. [4] nd Helocke et Al. [5] suggest tht Peson coeltion pefoms bette thn ll the est. If we define the use-item mtix s the mtix hving s elements the tings of uses to items then use s model is epesented in this mtix s n n-dimensionl vecto whee n is the numbe of items in the dtbse. This vecto is extemel spse fo most uses since even ones tht e ve ctive esult in ting just few of the totl numbe of items vilble in dtbse. If we define the subset of items tht uses u x nd u hve co-ted s I ={i x : x= n nd n n} whee n is the totl numbe of items in the dtbse u i x h s the ting of use u x to item i h nd u x s the u vege tings of uses u x nd u espectivel then the similit between two uses is defined s the Peson coeltion of thei ssocited ows in the use-item mtix nd is given b eqution [4]. i i x i i n u n u x i h u x u i h u u x x xx x xn sim ux u = n n u x i h ux u i h u u x - u m CF fils seiousl to scle up its computtion with the gowth of both the numbe of uses nd items in the dtbse. To del with the sclbilit poblem Beese et l [4] nd Ung et l [8] utilize Besin netwok nd clusteing ppoches while Sw et l [6 ] ppl folding in Singul Vlue Decomposition SVD to educe the

3 dimensionlit of the use-item mtix. It is lso possible to ddess these scling issues b dt eduction o dt focusing techniques. Yu et l [] nd Zeng et l [9] dopt instnce selection fo emoving the ielevnt nd edundnt instnces. Moeove content-boosted CF ppoches educe the numbe of items exmined b ptitioning the item spce ccoding to item ctego o subject clssifiction [7]. Finll moe geed ppoches concentte on ndoml smpling uses discding uses with few tings o discding ve popul o unpopul items. Unfotuntel ll these methods even if the esult in impoved pefomnce lso educe ecommendtion qulit in sevel ws. Besin netwoks m pove pcticl fo envionments in which knowledge of use pefeences chnges slowl with espect to the time needed to build the model but e not suitble fo envionments in which use pefeence models must be updted fequentl. Clusteing bsed methods e suffeing fom poo ccuc. It is possible to impove qulit b using numeous fine-gined segments [3] but then online use segment clssifiction becomes lmost s expensive s finding simil uses using the clssic CF. SVD bsed wok focuses minl on ccuc the thn efficienc. Dt focusing nd eduction ppoches such s instnce selection o item-spce ptitioning expeience educed ccuc due to the loss of infomtion. If n lgoithm discds the most popul o unpopul items thee m be items tht will neve be ecommended to some uses. It is obvious tht to gin in computtion one needs to lose ecommendtion qulit nd vice ves. Appopite tde-offs must be consideed. 3 Incementl Collbotive Filteing In this section we pesent method to del with the sclbilit chllenge without compomising ecommendtion qulit. We efe to this method s Incementl Collbotive Filteing ICF becuse it is bsed on incementl updtes of the useto-use similities. ICF cn be emploed to effectivel bing on the Web highl sclble nd ccute ecommendtion lgoithms. 3. Methodolog The similit between use u x nd u fo the subset of items the hve co-ted is given b eqution. Wheneve use u x submits new ting o updtes the vlue of n led submitted ting similit vlues between he/him nd the est of the uses m need to be e-computed. Ou objective is to expess the new similit vlues between the two uses in eltion to the old similit vlues. This descibes n incementl updte of thei ssocited similit. To smoothen the pogess of this tsk we dopt the following nottion fo the Peson Coeltion similit mesue of eqution :

4 B A = A= sim u u B= C = D = C D = = = Actull we split the similit mesue into thee fctos B C D independentl clculte the new vlues of ech fcto B C D nd then combine these vlues so s to ield the vlue of the new similit A s shown below: x ux x u ux x u h i i B' B+ e A' = A = B' = B+ e C' = C+ f D' = D+ g C' D' C+ f D+ g whee e f g e incements tht need to be computed fte eithe the submission of new o the updte of n existing ting. Next we split ou stud so s to conside the slightl diffeent computtions needed fo the two specil cses. Tble shows the incements tht need to be computed nd Appendix povides poof of equtions Cse : Submission of new ting To clculte the similit of u nd u when the ctive use u submits new ting fo the ctive item i we need to distinguish between two cses: i. u hd ted i : B C D e updted due to the new vege of u the new ting of u to i nd the new numbe of co-ted items ii. u hd not ted i : B C e updted due to the new vege of u. 3.. Cse : Updte of n existing ting To clculte the similit of u nd u when the ctive use u updtes n existing ting fo the ctive item i we need to distinguish between two cses: i. u hd ted i : B C e updted due to the new vege of u nd the new ting of u to i ii. u hd not ted i : B C e updted due to the new vege of u Tble. Summ of the incements tht need to be clculted Submission of new ting Updte of n existing ting u hd ted i e f g e = ' d e = du i u i u du u i h u 8 u i u u i u u u u i u u f = ' + d d u u g 3 d ' u i u i u i u u f = d + d ' + d ' = u i u 4 0 u u 9 g = 0 u hd not ted i e f u u 5 e = d u u u 6 f = d d u u e = d u u u f = d ' d ' g g = 0 7 g = 0 3

5 3. Cching In the pevious pgph we mnged to expess B C nd D using the fome vlues of B C nd D nd the espective incements e f g. Howeve to compute the incements with tivil opetions we need to cche the vlues of B C nd D fo ll pis of uses the vege ting of ech use nd the numbe of items tht ech use hs ted. Pt of the cched infomtion needs to be updted fte the submission of new o the updte of n existing ting. Tble explins how ech fcto tht ppes in incements e f g is computed. Tble. Computtion of fctos tht ppe in incements e f nd g Fctos Clcultion B C D Cched Infomtion Fo ll pis of uses m u Cched Infomtion The numbe of the items tht use hs ted Cched infomtion Avege tings of ll uses in dtbse u u ih u ' Cched Infomtion Fo ech pi of uses the sum of thei tings to co-ted items is cched New vege ting of ctive use: Submission of new ting: u i m ' = u u m+ + m+ Updte of existing ting: du ih u ' = + u m Intefce Actul ting of the ctive use u to the ctive item i u i du = ' ' u u u = u + d u d u The diffeence of use s pevious nd cuent vege ting u i Dtbse que. The ting of the use u to the item i 4 Complexit Issues In this section we discuss the computtionl complexit of the clssic CF nd ICF lgoithms. We initill pesent the wost cses nd then t to give ppoximtions of the lgoithms unde el conditions. Fo ech cse ou stud spns in two diections the one efes to the complexit of mintining the use similities mtix nd the othe efes to the complexit of fomulting single ecommendtion to n ctive use.

6 4. Wost Cse Complexities 4.. Clssic Collbotive Filteing The most expensive computtion of the clssic CF is the computtion of use-to-use similities. In ode to del with this mjo e-commece sstems pefe to c out expensive computtions offline nd feed the dtbse with updted infomtion peiodicll []. In this w the succeed to povide quick ecommendtions to uses bsed on pe-computed similities. These ecommendtions howeve e not poduced with the highest possible degee of confidence becuse tings submitted between two offline computtions e not consideed. The computtion complexit of mintining the use similities mtix in wost cse is Om n s explined below: Fo ech use m x Fo ech use m Fo the set of n items tht hve been co-ted b m x nd m Compute similit between m x nd m Altentivel if use similities e not pe-computed offline the need to be computed t the time ecommendtion is equested. In this cse thee is no need fo computing the whole use similities mtix but onl similities between the ctive use nd ll the est o set of tining uses. The cost of this computtion is of the ode Omn. Geneting single ecommendtion fo n ctive use is two-step computtion. Fist we need to find the most simil uses to the ctive use nd then scn items to find the ones tht bette mtch with the use s inteests ccoding to simil uses. In the wost cse this computtion costs On when similities e pe-computed offline o Omn bsed on Omn+On when similities e not pe-computed. 4.. Incementl Collbotive Filteing In the cse of ICF lgoithm use-to-use similities e computed incementll t the time of ting ctivit nd not t the time tht ecommendtion is equested. The complexit of this opetion is Omn t wost s t most m- similities need to be updted nd t most n items need to be exmined fo ech use. Since use similities e consideed pe-computed the cost of geneting single ecommendtion using the ICF is of the ode of On in the wost cse s n items need to be exmined. 4. Appoximtion Complexities Since spsit levels e ve high in ecommendtion sstems it is essentil to lso conside ppoximtions of the complexities in ode to estimte the expected pefomnce unde el conditions. In ode to compute the ppoximtion complexities we define:

7 m whee m <<m: the numbe of uses with whom the ctive use hs t lest one coted item m >0 is pecondition fo computing similities between u nd othe uses. n whee n <<n: the numbe of items tht hve not been ted b the ctive use nd hve been ted b t lest one of its simil uses n >0 is pecondition fo being ble to ecommend t lest one item to the ctive use. n whee n <<n: the numbe of co-ted items of the ctive use nd nothe use n >0 is pecondition fo the similit between the two uses to be computble. Accoding to these definitions we cn set up the ppoximtions of the complexities following the discussion of the pevious pgph. Wost cse nd ppoximtion complexities fo mintining the similit mtix o fomulting single ecommendtion with Clssic CF o Incementl CF e summized in Tble 3. Tble 3. Wost cse nd Appoximtion complexities of Clssic CF nd ICF Complexit fo mintining the Similit Mtix Complexit fo poviding ecommendtion to ctive use Clssic CF Incementl CF Wost Appoximtion Wost Appoximtion Ο mn mmn Ο Ο mn Ο m n Ο mn Ο m n +Ο n Pe-computed Offline Ο n Ο n Ο n Ο n As complexit computtion fils to give el time pefomnce nd behvio of the lgoithms descibed we set up n expeimentl scenio fo evluting the pefomnce of ou ICF lgoithm s opposed to the Clssic CF. 5 Expeimentl Evlution In this section we descibe the expeimentl evlution of incementl collbotive filteing. We pesent the evlution metics used descibe the expeimentl scenio nd discuss the esults. 5. Evlution Metics We evlute the pefomnce of the ecommendtion lgoithms pesented ccoding to esponse time nd ccuc metics s defined below: esponse time: Time equied b the lgoithm to find out the items to ecommend. Accuc: The fction of the numbe of items n lgoithm ecommends to the numbe of items tht e ecommended b n lgoithm tht tkes into considetion the whole dtset vilble.

8 The ssumption mde hee is tht ecommendtions bsed on the whole dtset e of the highest qulit which is not necessil tue. Indeed we define this to demonstte the potentil tht ICF gives fo fomulting ecommendtions bsed on the complete infomtion in dtbse nd not onl pt of it. 5. Expeimentl Scenio nd Results We compe the pefomnce of clssic CF ginst ou ICF in tems of esponse time nd ccuc fo diffeent use-item mtix sizes. The scenio is set up so s to depict the level of sclbilit tht both lgoithms demonstte when the ctive use equests single ecommendtion. We emplo spsit level of 9% in the use-item mtix which mens tht 9% of the mtix cells e empt nd thee e onl vlues fo 8% of it. We conside use to be simil to the ctive use if thei ssocited Peson coeltion coefficient is gete thn 0.65 in nge of - to with expessing highest mtch nd - expessing highest mismtch between the two uses nd lso tht n item is suitble fo ecommendtion if the vege tings of simil uses to this item is gete thn 8 in nge of -0. The vlues selected epesent tpicl vlues fo ecommendtion sstems nd do not influence the esults of the expeiments. Tble 4 pesents the esults of ou expeiments fo use-item mtix of size 00x00 nd 000x000 espectivel. Expeiments hve been cied out on.80 Mhz G RAM PC. Tble 4. Pefomnce compison of Clssic CF nd ICF Use-item Clssic CF Bsed on smpling Incementl CF mtix size Smples #uses Time sec Accuc Time sec Accuc 00 uses x 00 items 000 uses x 000 items % % % % % % % % % % % % % The following emks deive fom Tble 4 bout the pefomnce of CF nd ICF. The tde-off between pefomnce nd ccuc in cse of Clssic CF is confimed. Indeed Clssic CF is ve sensitive to the size of smples used. As the smple size inceses ccuc is impoved but the esponse time lso inceses nd vice ves. Lge smple sizes e impcticl fo online

9 pplictions due to the slow esponse time while smll smple sizes e impcticl due to ccuc degdtion The ccuc of ICF is lws s high s 00% since it is lws pplied to the totl infomtion in the dtbse ICF poves to be highl-sclble s its esponse time emins cceptble even fo ve lge dt set. E.g. it povides ecommendtion in 0.46 seconds fo mtix size of 000x000 Clssic CF equies extemel dispopotionl time to ech stisfcto ccuc level fo lge mtix sizes. E.g. when n ccuc level of 66.8% is intended using smple of 500 uses in 000x000 mtix Clssic CF pefoms 7 times slowe thn ICF ICF s pefomnce gows linel onl with the numbe of items in dtbse. In cses of ve lge numbe of items ICF will pobbl need to emplo some ppoximtion methods 6 Conclusions nd Futue Wok High dimensionlit seems to be the Achilles heel fo most of the CF-bsed ecommendtion sstems. Fo deling with this sclbilit poblem we poposed n incementl method tht eplces expensive vecto opetions with scl opetion ble to speed-up computtions of high dimensionl use-item mtices. We nmed this method Incementl Collbotive Filteing ICF. ICF is not bsed on n ppoximtion method nd thus povides the potentil of fomulting high-qulit ecommendtions. Moeove pe-computed use to use similities pemit fo ecommendtions to be deliveed odes of times fste thn with clssic CF. ICF ppes to be suitble fo online pplictions while the methodolog descibed is genel nd m pobbl be esil dopted to develop incementl collbotive filteing with the utiliztion of similit mesues othe thn Peson coeltion. As futue diections of ou esech we see the identifiction of tusted pths mong uses fo deling with the cold-stt nd spsit poblems. Unde the ssumption tht similit mesue cn somehow excessivel be consideed s computtionl metic fo expessing the ssocited tust between two uses it is possible to define eltion between two uses tht hve no common items t ll b emploing theoeticl wok of tust popgtion in smll netwoks. Refeences. Sw B. Kpis G. Konstn J. Riedl J.: Anlsis of ecommendtion lgoithms fo e-commece. Poc. of ACM Electonic Commece 000

10 . Linden G. Smith B. Yok J.: Amzon.com Recommendtions: Item-to-Item Collbotive Filteing. IEEE Intenet Computing Jnu Helocke J. L. Konstn J. A. Riedl J.: Explining Collbotive Filteing Recommendtions. Poc. of the ACM Conf.on CSCW Beese J. S. Heckemn D. Kdie C.: Empiicl nlsis of pedictive lgoithms fo collbotive filteing. Poc. of the UAI Helocke J. L. Konstn J. A. Boches A. Riedl J.: An Algoithmic Fmewok fo Pefoming Collbotive Filteing. Poc. of ACM SIGIR Sw B. M. Kpis G. Konstn J. A. Riedl J. T.: Appliction of Dimensionlit Reduction in Recommende Sstem: A Cse Stud. Poc. of ACM SIGKDD Popescul A. Ung L. H. Pennock D.M. Lwence S.: Pobbilistic Models fo Unified Collbotive nd Content-Bsed Recommendtion in Spse-Dt Envionments. Poc. of UAI Ung L. Foste D.: Clusteing Methods fo Collbotive Filteing. Poc. of Wokshop on Recommendtion Sstems AAAI Pess Zeng C. Xing C. Zhou L.: Similit Mesue nd Instnce Selection fo Collbotive Filteing. Poc. of WWW Sw B.M. Kpis G. Konstn J. Riedl J.: Incementl SVD-Bsed Algoithms fo Highl Scleble Recommende Sstems. Poc.of ICCIT 00.. Deeweste S. Dumis S. T. Funs G. W. Lndue T. K. Hshmn R.: Indexing b Ltent Semntic Anlsis. JASIS Yu K. Xu X. To J. Este M. Kiegel H.: Instnce Selection Techniques fo Memo-Bsed CF. Poc. of SDM Jung S. Y. Kim T.: An Incementl Similit Computtion Method in Agglometive Hiechicl Clusteing. Poc. of ISAIS Peson K.: Mthemticl contibution to the theo of evolution: VII on the coeltion of chctes not quntittivel mesuble. Phil. Tns. R. Soc. Lond. A Appendix: Poof of Equtions -3 Poof of Eqution ' B' = ' B' = ' + ' u u u i u u i u u u B ' = ' + B d e= ' d u i u u i u u u u i u u i u u u

11 Poof of Eqution 3 ' u u i u u u i u u u u u i u u u u C' = ' C' = ' + ' C ' = ' + C+ d d f = ' + d d Poof of Eqution 4 ' u u i u u u i u u i u D ' = D' = + D' = + D g = Poof of Eqution In the cse tht use u hs not ted the item i the vlues of B C nd D e poved in w simil to equtions 3 nd 4 espectivel. In this cse the incements e f nd g equl to: u u u u u e= d f = d d g = 0 Poof of Eqution 8 u u u i u u i u u u u i u i u u i u u i u u u u i u i u + u i h u u i h u B = d u i u i u + B du u i h u u i u i u u u B' = ' B' = ' + ' B' = d + ' + ' B' = d ' ' e= d d Poof of Eqution 9 u u i u u u i u i u i u u i u u u i u i u i u u u i + d ' u i u i u + C + d u du u i h u u ' i u i u i u u u u C' = ' C' = ' + ' C' = d + d ' + ' + ' C' = d + d ' + ' C' = d f = d + d + d d Poof of Eqution 0 ' = u i h u ' ' = = 0 D D D g Poof of Eqution 3 In the cse tht use u hs not ted the item i the vlues of B C nd D e poved in w simil to equtions 8 9 nd 0 espectivel. In this cse the incements e f nd g equl to: u u u u u e= d f = d ' d ' g = 0

9.4 The response of equilibrium to temperature (continued)

9.4 The response of equilibrium to temperature (continued) 9.4 The esponse of equilibium to tempetue (continued) In the lst lectue, we studied how the chemicl equilibium esponds to the vition of pessue nd tempetue. At the end, we deived the vn t off eqution: d