Location-Sensitive Resources Recommendation in Social Tagging Systems

Transcription

1 Location-Sensitive Resources Recommendation in Social Tagging Systems Chang Wan Ben Kao David W. Cheung Deartment of Comuter Science, The University of Hong Kong, Pokfulam, Hong Kong {cwan, kao, ABSTRACT In social tagging systems, resources such as images and videos are annotated with descritive words called tags. It has been shown that tag-based resource searching and retrieval is much more effective than content-based retrieval. With the advances in mobile technology, many resources are also geo-tagged with location information. We observe that a traditional tag (word) can carry different semantics at different locations. We study how location information can be used to hel distinguish the different semantics of a resource s tags and thus to imrove retrieval accuracy. Given a search query, we roose a location-artitioning method that artitions all locations into regions such that the user query carries distinguishing semantics in each region. Based on the identified regions, we utilize location information in estimating the ranking scores of resources for the given query. These ranking scores are learned using the Bayesian Personalized Ranking (BPR) framework. Two algorithms, namely, L and, which aly Tucker Decomosition and Pairwise Interaction Tensor Factorization, resectively for modeling the ranking score tensor are roosed. Through exeriments on real datasets, we show that L and outerform other tag-based resource retrieval methods. Categories and Subject Descritors H.3.3 [Information Search and Retrieval]: Retrieval models General Terms Algorithms, Exerimentation, Performance Keywords ranking, resources recommendation, location-sensitive 1. ITRODUCTIO In a social tagging system (e.g., Flickr and YouTube), resources (e.g., hotos and videos) are often annotated with descritive words called tags. Previous studies have shown that searching resources based on their tags leads to very effective and accurate resource retrieval [2, 6]. However, in some cases, tags are ambiguous. For Permission to make digital or hard coies of all or art of this work for ersonal or classroom use is granted without fee rovided that coies are not made or distributed for rofit or commercial advantage and that coies bear this notice and the full citation on the first age. To coy otherwise, to reublish, to ost on servers or to redistribute to lists, requires rior secific ermission and/or a fee. CIKM 12, October 29 ovember 2, 2012, Maui, HI, USA. Coyright 2012 ACM /12/10...$ examle, the word mouse could refer to an animal, a cartoon character (Mickey), or a comuter eriheral device. Tag ambiguity lowers the effectiveness of tag-based search. Over the years, researchers have sent much effort in roosing techniques to deal with the tag-ambiguity roblem [3, 5]. With the advances in mobile technology and the global ositioning system (GPS), resources, esecially images and videos, are geo-tagged with location information. Our observation is that resources location information often rovides useful hints in identifying the semantics of their tags. As an examle, consider a set of hotos that are tagged with the tag sand. Deending on the locations at which these hotos were taken, the semantics of the tag could be inferred. For examle, the tag sand of a hoto taken in the Sahara very likely refers to the concet desert ; for hotos taken in Hawaii and Singaore, the same tag robably refers to beach and the casino hotel Marina Bay Sands, resectively. The semantics of a user query can likewise be inferred. For examle, a user located in the Sahara who issues the query sand is robably looking for hotos of the desert. In the above examle, we suggested that the semantics of a tag or a query could be deduced by the region in which the resource was obtained or the query issuer was located. Intuitively, a region is a set of locations within which a tag or a query carries a unique distinguishing meaning. An interesting question is how these regions (such as The Sahara, Hawaii, and Singaore) be automatically identified. We remark that different tags induce different artitionings of the world into regions: while there are multile regions for the tag sand, there are two regions for the tag football (American football in orth America and soccer anywhere else in the world), and there is only one region (the whole world) for a tag that refers to some globally known concet, such as iphone. Another interesting question is if regions with resect to tags and queries could be identified, how could this information be leveraged in order to achieve good resource retrieval results. The objective of this aer is to address the above two questions. First, given a tag t, we identify the regions induced by t by using machine learning techniques to train a function f t(l), which mas a location l into a region. Second, given a query q issued by a user located at location l, we estimate the ranking scores of resources with resect to q. We extend the BPR framework [9] to include region information in the score estimation. As we will see later in Section 5, our solutions lead to very good resource retrieval results. Here, we summarize our contributions: We roose an algorithm for comuting the region maing function f t(l). We roose algorithms for estimating the ranking scores of resources with resect to a query. 1960

2 We erform an extensive exeriment using real datasets to evaluate the erformance of our algorithms in terms of resource retrieval effectiveness. 2. RELATED WORK is a oular semantic analysis technique that has roven to be very successful in IR. It attemts to overcome the word ambiguity roblem by extracting concets from words in an unstructured text collection, which is realized by erforming Singular Value Decomosition (SVD) on a term-document matrix and by measuring the tag-air similarities so derived. With the extracted concets, one can rank resources w.r.t. a given query by measuring the similarities between resources and the query on the concet level. Bi et al. [3] observes that user information can imrove resource retrieval results. They extend to the Cube method by alying three-dimensional Tucker Decomosition () on a third-order tensor whose dimensions are resources, users, and tags. Besides resource retrieval, tag recommendation is also a oular toic in the study of social tagging systems. Factorization models are a oular method. Rendle et al. [10] model the likelihood score that a user u would tag a resource r with a tag t as a 3D tensor, which is aroximated using. By using AUC (area under the ROC-curve) as the ranking criteria, [10] otimizes the model arameters of to obtain otimal values of the entries in the tensor. [9] extends the work of [10] by alying the Bayesian Personalized Ranking (BPR) framework in model arameter otimization. In [11], the Pairwise Interaction Tensor Factorization () model is roosed, which is shown to be more efficient than the model used in [10]. These techniques for tag recommendation can also be emloyed for solving the resources retrieval roblem. Some studies [8, 4] utilize location information in resource retrieval. Lu, Lu, and Cong [8] resent a hybrid index tree called IUR-tree, which combines location roximity with textual similarity. They further design a branch-and-bound search algorithm to retrieve resources that are the closest and the most relevant to a query. Our method is significantly different from theirs. In articular, we roose a query-driven location artitioning algorithm to transform the location dimension to a region dimension, which significantly imroves searching erformance. 3. PROBLEM STATEMET Our aroach to integrating location information in solving the tag-based resource retrieval roblem consists of two comonents: (1) A location-artitioning scheme that derives a region-maing function f t(l) for each tag t, and (2) a score function that ranks resources in terms of their relevancy to a given query q. In this section we formally define these two roblems. We consider four attributes of a social tagging system: a set of users (taggers) U, asetoftagst, a set of resources R, andaset of locations L 1. We assume that each location l L is reresented by a longitude-latitude air (x l,y l ). Resources in the systems are tagged by users. We assume that the tagging information is reresented by a set S U T R L of tagging records. Each tagging record is a quadrule (u, t, r, l) S, which indicates that auseru assigns a tag t to a resource r. l is the geo-tag location of resource r. Although l is deendent on r, to simlify our discussion, we exlicitly include l in a tagging record. A user query q =(u q,t q,l q) is a trilet which indicates that a user u q,whois located at location l q, issues a query with the tag t q. For simlicity, 1 L includes all ossible locations in the world, not only the locations that are associated with the resources in the social tagging system. we assume that each query consists of only one tag. Our technique can be easily extended to handle multi-tag queries, for examle, by considering all the tags in a multi-tag query as a single comlex tag. Given a tag t,theroblem of location artitioning is to artition L into a set of k t regions H t = {H 1,...,H kt } such that tag t refers to the same concet within each region H i and dissimilar concets in different regions. This artitioning can be reresented by a region maing function (RMF) f t : L [1...k t], which mas a location l L to the region H ft(l) H t. We say that tag t induces the region set H t. Given a query q, we first obtain the RMF f tq.leth q be the set of regions induced by t q. ote that while the query tag t q (such as sand ) can assume different meanings at different locations of the world, it refers to a unique concet within each region H i H q. For notational convenience, we use H q to denote H ftq (l q), which is the region that encomasses the location (l q) of the query. The resource ranking roblem is to derive a score function ŷ(u, t, H, r): U T H q R R, which gives a relevancy score of resource r with resect to a user query issued by user u, who is located in the region H, with the query tag t. Aswewillseelater,wesolve this roblem by reresenting the score function ŷ by a 4D tensor Ŷ,called thescore tensor, such that ŷ(u, t, H, r) is reresented by the entry ŷ of the score tensor. Finally, the to- resources with the highest relevancy scores are returned as the answer set of query q. T(q, ) :=arg max ŷuq,tq,hq,r. (1) r R 4. ALGORITHMS In this section we describe the algorithms for solving the location artitioning roblem and the resource ranking roblem. For the former, we show how to train an RMF f tq for the query tag t q.for the latter, we show how to extend the BPR framework to include region information in estimating the 4D score tensor Ŷ. 4.1 Location Partitioning Our objective is to derive f tq based on the tagging record set S. Our aroach is based on two intuitive assumtions: (1) Users in close roximity are likely to have the same understanding of a tag. (2) If t q s associations (e.g., co-occurrences) with other tags in resources at a certain location l 1 is similar to those at another location l 2,thenl 1 and l 2 should belong to the same region. Procedurally, we first retrieve all the tagging records that involve t q into a set S q, i.e., S q = {(u, t q,r,l) S}. ext, we create a set of training instances X in the following way. Given any tag t and a resource r that is tagged by t q, we define the distance of t from t q w.r.t. r by c(t, r) c(t q,r) d q(t, r) = max{ c(t, r) c(t, (2) i i,r) } where c(t, r) is the number of times resource r is tagged by t. Essentially, d q(t, r) measures the difference in the tagging frequencies of tags t q and t for the resource r, scaled to the range [-1,1]. ext, for each tule (u, t q,r,l) S q, we generate an instance x =(d q(t 1,r),...,d q(t T,r),l) and ut it in X. Based on these instances (observations), we create artitions of X to maximize a osteriori (MAP) estimate of arameters as follows. Suose the instances in X are observations of the mixture of k multivariate normal distributions of T +2dimensions, let z =(z 1,z 2,...,z ) be the latent variables that determine which distributions the observations come from. We have: X (z j = i) d (μ i, Σ i), 1 i k. (3) 1961

3 Let U =[μ i ], Σ =[Σ i],andp (z j = i) =ϕ i such that P k 1. Let ϕ =[ϕ i] R k 1 +. The model arameters to be estimated is denoted by Θ=(ϕ, U, Σ). The likelihood function is: ϕi = Y kx L(Θ; X, z) =P (X, z Θ) = I(z j = i)ϕ i f P (x j ; μ i, Σ i ), (4) j=1 where I is the indicator function and f P is the robability density function of a multivariate normal. We aly exectation-maximization (EM) algorithm to find MAP estimates of Θ with the following equations to udate ϕ, U, Σ: ϕ (t+1) =argmax ϕ Q(Θ Θ(t) ) = arg max ϕ ( X kx j=1 ji log ϕ t i ), (5) μ (t+1) j=1 i = ji x j j=1, (6) ji Σ (t+1) j=1 i = ji (x j μ (t+1) i )(x j μ (t+1) i ) j=1, (7) ji where P ji is the conditional robability of x j belonging to the i-th distribution, given by: ji = P (z j = i x j ;Θ (t) )= ϕ (t) i f P (x j ; μ (t) i, Σ (t) i ) f P (x j ; μ (t) i, Σ (t) i ). (8) P k ϕ(t) i Let L q = {l (u, t q,r,l) S q}. After the EM algorithm is done, for each location l L q, the RMF value f tq ( l) is determined by f tq ( l) = arg max ϕ i if P (x; μ i, Σ i). (9) For other locations l L L q, we first determine the location l L q that gives the smallest Euclidean distance D(l, l ) from l (i.e., l =argmind(l, l)). The region of l is taken as the region l Lq of l, i.e., f tq (l) =f tq ( l ). So far, we have assumed that the number of regions, k, is known. We can determine the value of k by gradually increasing its value from 1 until the likelihood value (Equation 4) stos increasing. Algorithm 1 summarizes the location artitioning algorithm LP. Algorithm 1 LP IPUT t q, S q, L, L q. OUTPUT The RMF f tq. 1: Generate the set of observations X; 2: k =0; 3: reeat 4: k = k +1; 5: Initialize Θ=(ϕ, U, Σ); 6: reeat 7: Udate P ji, ϕ, U, Σ according to Equations 5-8; 8: until convergence 9: until L(Θ; X, z) does not increase 10: l L q, f tq ( l) = arg maxϕ if P (x; μ i, Σ i); i 11: l L L q, l =argmind(l, l); f tq (l) =f tq ( l ); l L q 12: RETUR f tq 4.2 Estimating the Score Tensor Our next task is to estimate the score tensor Ŷ. We generate the values of the elements in Ŷ by assuming a certain model. Let Θ reresents the model s arameters. We determine Θ by first constructing a set of observations Y and then use these observations tofindthe otimal Θ following the Bayesian Personalized Ranking (BPR) framework. In the following, we first give the details of how the training set Y is constructed and how BPR is alied. Then, we show how Ŷ is determined using the Tucker Decomosition () and as two ossible models of First, for each record (u, t, l, r) Ŷ. S, we relace the location l of the record by the region H H q to which l belongs. Let S H = {(u, t, H, r) ((u, t, l, r) S) (H = H ftq (l))}. Suose auseru tags a resource r 1, which is located in region H, with the tag t, then(u, t, H, r 1) S H.Sinceumakes such a tagging, resource r is relevant to the tag t from the ersective of u. On the other hand, if the same user does not tag another resource r 2 with t, thenr 2 is not relevant to t according to u. Inotherwords, ŷ 1 should be large and ŷ 2 should be small. Define Δŷ(u, t, H, r 1,r 2)=ŷ 1 ŷ 2. We construct the set Y as Y = {(u, t, H, r 1,r 2 ) ((u, t, H, r 1 ) S H ) ((u, t, H, r 2 ) / S H )}. (10) Given a model and its arameters Θ, we follow [9] in estimating Θ. Secifically, based on Bayes theorem, (Ŷ Y) (Y Ŷ )(Ŷ ). (11) For each observation w =(u, t, H, r 1,r 2) Y, the robability (w Ŷ ) is estimated by, ((u, t, H, r 1,r 2) Ŷ )=σ(δŷ(u, t, H, r1,r2)), (12) 1 where σ is the logistic function σ(x) =. We assume 1+e ( x) that model arameters are drawn from a normal distribution Θ (0,σΘI). 2 Letλ Θ be the regularization constant of σ Θ and α be the learning rate of BPR. Algorithm 2 shows the model otimization rocedure BPR-OPT for training model arameters Θ. Algorithm 2 Model Otimization Using Bayesian Theorem and MAP (BPR-OPT) IPUT Y, λ Θ, α OUTPUT the redictive model arameters Θ. 1: Initialize Θ; 2: reeat 3: Draw (u, t, H, r 1,r 2) from Y; 4: Θ Θ+ α((1 σ(δŷ(u, t, H, r 1,r 2))) λ ΘΘ); 5: until convergence 6: RETUR Θ Θ Δŷ(u, t, H, r1,r2) Factorization models are very oular models in recommendation systems and resource searching systems. We use and as two models for redicting the score tensor Ŷ. We show how these models arameters can be learned using BPR-OPT. [Tucker Decomosition ()] factorizes a high-order cube Ŷ into a core tensor Ĉ Ru t h r and four factor matrices Û R U u, ˆT R T t, Ĥ R Hq h and ˆR R R r, where u, t, h,and r are arameters which control the sizes of Ĉ s dimensions. Each entry in Ŷ can be formulated as: ŷ = ĉũ,,,ûu,ũˆt t,ĥh, ˆrr,. (13) ũ 1962

4 The model arameters Θ include all the entries of Ĉ, Û, ˆT, Ĥ, and ˆR. For learning the model arameters using BPR-OPT, the gradients used in Algorithm 2 are: θŷ ŷ ĉũ,,, ŷ = X û u,ũ ŷ = X ˆt t, ũ ŷ ĤH, =û u,ũˆt t,ĥh, ˆrr, = X ũ ŷ = X ˆr r, ũ ĉũ,,,ˆt t,ĥh, ˆrr, ĉũ,,,ûu,ũĥh, ˆrr, ĉũ,,,ûu,ũˆt t,ˆr r, ĉũ,,,ûu,ũˆt t,ĥh,. (14) We call our solution of estimating Ŷ using, L.Thecomlexity of L is O( u t h r). [Pairwise Interaction Tensor Factorization] () The time comlexity of is high. To imrove efficiency, is roosed in [11], which only considers two-way interactions between r and other dimensions. We extend to a 4D version to include the region dimension H. We factorize Ŷ into four factor matrices Û R U, ˆT R T, Ĥ R Hq and ˆR R R,where is a arameter that controls the size of the matrices. Each entry in Ŷ can be formulated as: ŷ = X û u, ˆr r, U + X ˆt t, ˆr r, T + X Ĥ H, ˆr r, H (15) BPR-OPT can then be alied in a way similar to that of, with the following gradients: ŷ û u, ŷ ˆr U r, =ˆr U r,, ŷ ˆt t, =û u,, ŷ ˆr T r, =ˆr r,, T ŷ ĤH, =ˆr H r,, (16) = ˆt t,, ŷ = ˆr ĤH,. r, H We call the resulting solution, whose comlexity is O(). 5. EXPERIMETS In this section we comare the effectiveness of our algorithms in ranking resources for answering search queries. 5.1 Datasets We conducted exeriments on data collected from two social tagging systems: Flickr and Picasa. They are hoto sharing systems that allow users to annotate hotos with tags. Since the raw data is very noisy and sarse, we erformed some cleaning and rerocessing. First, we removed system-generated tags such as "uloaded:by=instagram". Also, to eliminate outliers, any user, tag, or resource that has aeared in less than 10 records of S are removed. Table 1 shows some statistics of the two datasets after cleaning. 5.2 Metrics Let Q be a set of queries. For each query q Q, a ranking algorithm returns a ranked list of resources. We evaluate the effectiveness of a ranking algorithm by three metrics, namely, Mean Recirocal Rank (MRR), Precision (P@), and ormalized Discounted Cumulative Gain (GCG@). Dataset U T R S Flickr 8,199 28,049 26, ,527 Picasa 509 1,545 1,041 18,135 Table 1: Data Statistics The recirocal rank of a ranked list is the multilicative inverse of the rank of the first relevant resource in the list. The MRR score of an algorithm A is the average recirocal rank obtained by the ranked lists given by A w.r.t. the query set Q. Formally, MRR = 1 X 1, (17) rank i where rank i is the rank of the first relevant resource in the ranked list for the i-th query. The Precision of a ranked list is the fraction of the resources in the list that are relevant to the corresonding query. The Precision score of an algorithm is the average recision of its ranked lists, which is given by: = 1 X q Q P rel (q, i), (18) where rel (q, i) is an indicator function whose value is 1 if the i th - ranked resource in the list for query q is relevant and 0 otherwise. Finally, the DCG score [7] is comuted as follows: DCG@ = 1 X q Q X Z q (2 rel (q,i) 1) log(i +1)!, where Z q is a normalization factor (see [7] for details). We invited 10 judges to articiate in the exeriments. We randomly selected 100 tagging records in S to generate the query set Q. For each selected record (u, t, r, l), we created a query q =(u, t, l). We then alied the various ranking algorithms to return their ranked lists for each query. Each judge was given 10 queries and was asked to judge whether each returned resource was relevant to the corresonding query. The judge was not told which algorithm s ranked list did a resource aear. The results were then used to comute the scores for the various metrics. 5.3 Algorithms and Settings Besides L and, we evaluate four other algorithms for ranking resources. They are [3], Cube [3], [9] and [11]. Due to sace limitations, readers are referred to the references for details. In the exeriments, the BPR-OPT arameters are α = 0.05 and λ θ = For LP s initialization ste (Algorithm 1, Line 5), we first erform k-means to cluster the instances in X into k clusters 2. This result is used to initialize U and Σ. Also,weset ϕ i = cluster(i),wherecluster(i) is the i-th cluster of the k-means result. The model arameters of,, L, and are all initialized with (0, 0.01). Also,weset = u = t = h = r = Results Figures 1-3 show the scores DCG@, P@, and MRR, resectively, of the algorithms when they are alied to the two datasets. From the figures, we see that, which considers only the dimensions tag and resource erforms worst. The three algorithms Cube,, and, which consider the user dimension as 2 The l dimension of x is reresented by a longitude dimension and a latitude dimension, both of them are normalized to the [-1,1] range. 1963

5 5 5 L Cube 5 5 L Cube DCG@ 5 5 L Figure 1: DCG@ P@ Cube 5 5 Figure 2: L Cube well erform better than. These three algorithms give comarable erformances among themselves. By considering the location dimension and extending the 3D tensor to 4D, L and clearly outerform the other four algorithms. The erformance of L and is very similar, desite the fact that uses a simler factorization model. Given that is much more efficient than L, is the algorithm of choice for solving the resource ranking roblem. 5.5 Parameter Sensitivity The arameters, u, t, h,and r control the sizes of the core tensor and the factorization matrices of and. These values affect the accuracy of our aroximation of the score tensor Ŷ.We have conducted an exeriment varying the values of these arameters. We observe that in general, setting them to 64 is enough for Ŷ to converge. Therefore, in our erformance exeriment, we set those arameters to COCLUSIO In this aer we studied the imact of location information on resource recommendation. We addressed two roblems, namely, the location artitioning roblem and the resource ranking roblem. We roosed the LP algorithm which comutes a region maing function to solve the location artitioning roblem. We also roosed the L algorithm and the algorithm for estimating the score tensor for ranking resources. Through an exerimental study on real datasets, we showed that our location-sensitive algorithms outerform other cometitors. Acknowledgements This roject is suorted by HKU Research Grant REFERECES [1] SIGMOD 2011, Athens, Greece, June 12-16, ACM, [2] S.Bao,G.-R.Xue,X.Wu,Y.Yu,B.Fei,andZ.Su. Otimizing web search using social annotations. In WWW, ages ACM, [3] B. Bi, S. D. Lee, B. Kao, and R. Cheng. Cube: An effective and efficient method for searching resources in social tagging systems. In ICDE, ages IEEE Comuter Society, [4] X. Cao, G. Cong, C. S. Jensen, and B. C. Ooi. Collective satial keyword querying. In SIGMOD Conference [1], ages [5] S.C.Deerwester,S.T.Dumais,T.K.Landauer,G.W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. JASIS, 41(6): , [6] P. Heymann, G. Koutrika, and H. Garcia-Molina. Can social bookmarking imrove web search? In WSDM, ages ACM, [7] K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst., 20(4): , [8] J. Lu, Y. Lu, and G. Cong. Reverse satial and textual k nearest neighbor search. In SIGMOD Conference [1], ages [9] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. BPR: Bayesian Personalized Ranking from Imlicit Feedback. In UAI, ages AUAI Press, [10] S. Rendle, L. B. Marinho, A. anooulos, and L. Schmidt-Thieme. Learning otimal ranking with tensor factorization for tag recommendation. In KDD, ages ACM, [11] S. Rendle and L. Schmidt-Thieme. Pairwise interaction tensor factorization for ersonalized tag recommendation. In WSDM, ages ACM, Figure 3: MRR 1964