Early Online Identification of Attention Gathering Items in Social Media

Transcription

1 Early Online Identification of Attention Gathering Items in Social Media Michael Mathiodakis Compter Science University of oronto Nick Kodas Compter Science University of oronto Peter Marbach Compter Science University of oronto ABSRAC Activity in social media sch as blogs, micro-blogs, social networks, etc is manifested via interaction that involves text, images, links and other information items. Natrally, some items attract more attention than others, expressed with large volmes of linking, commenting or tagging activity, to name a few examples. Moreover, high attention can be indicative of emerging events, breaking news or generally indicate information items of interest to a vast set of people. he nmbers associated with digital social activity are astonishing: in excess of millions of blog posts, tweets and forms pdates per day, millions of tags in photos, news articles or blogs. Being able to identify information items that gather mch attention in sch a real time information collective is a challenging task. In this paper, we consider the problem of early online identification of items that gather a lot of attention in social media. We model social media activity sing ISIS, a stochastic model for Interacting Streaming Information Sorces, that intitively captres the concept of attention gathering information items. Given the challenge of the information overload characterizing digital social activity, we present seqential statistical tests that enable early identification of attention gathering items. his effectively redces the set of items one has to monitor in real time in order to identify pieces of information attracting a lot of attention. Experiments on real data demonstrate the tility of or model, as well as the efficiency and effectiveness of the proposed seqential statistical tests. Categories and Sbject Descriptors J.7 [Compter Applications]: COMPUERS IN OHER SYS- EMS Real time; H.m [Information Systems]: MISCELLA- NEOUS General erms Measrement Keywords Social media analysis, User activity modeling and exploitation Permission to make digital or hard copies of all or part of this work for personal or classroom se is granted withot fee provided that copies are not made or distribted for profit or commercial advantage and that copies bear this notice and the fll citation on the first page. o copy otherwise, to repblish, to post on servers or to redistribte to lists, reqires prior specific permission and/or a fee. WSDM 10, Febrary 4 6, 2010, New York City, New York, USA. Copyright 2010 ACM /10/02...$ INRODUCION Activity in social media sch as blogs and micro-blogs (hosted by e.g., Blogger, Wordpress, LiveSpace, witter, Jaik), social networks (e.g., Facebook, MySpace, Friendster), mltimedia sharing services (e.g. Yotbe, Flickr) or online newspapers and magazines has been increasing at a phenomenal pace. Millions of individals participate daily in a social process of information exchange, generating information items sch as blog posts, images, videos or stats messages, as well as engaging with each other s generated items, e.g. by leaving comments or sharing them with friends. Indicative of the participation in social media are the 300 million sers of MySpace and Facebook [1, 9], the more than 30 million reglarly pdated blogs [2], millions of sers of witter, Yotbe, Flickr, etc. At an abstract level, individals participating in social media can be thoght of as information sorces that emit nits of information in a streaming fashion. Digital items sch as blog posts, videos, pictres and short stats messages are all examples of information nits. Besides acting as information sorces, individals also interact with each other. For instance, friends in a social network sch as Facebook or Friendster visit each other s profiles to view the newly pdated stats messages or posted pictres and possibly engage with them. Engaging with an item involves performing actions sch as leaving a comment, rating it or recommending it to others who might find it interesting. Natrally, some generated items gather more attention than others. For example, blog posts, pictres or videos related to important emerging events often attract significant nmber of links and comments in a few hors. Distingishing those items among the plethora of items generated in social media necessitates the definition of a measre for attention gathering potential, i.e. the ability of items to attract their adience s attention and stimlate their reactions. In the case of blogs, for example, common measres inclde the total nmber of attracted links or comments, the nmber of distinct linkers (as it is the case with echnorati [3]), etc. Sch measres, however, fail to captre significant temporal aspects of social media activity. For instance, consider a blog post p 1 that attracts 10 links after remaining on the front page of its hosting blog for 1 week. Consider, as well, a blog post p 2 that also attracts 10 links, bt only after remaining on the front page of its hosting blog for 1 hor. aking into accont the time each post remained visible on a blog webpage, it is reasonable to claim that post p 2 is associated with higher potential in attracting links than post p 1, even thogh the total nmber of links is the same. As another example, consider different blogs that are visited with varying rates by their readers. A post p 3 pblished on a blog that is rarely visited by its readers is less probable to attract the same nmber of actions (links or comments) with a post p 4 pblished on a freqently visited blog. herefore, in case p 3 attracts the same nmber of actions 301

2 with p 4, that fact shold be interpreted as p 3 having larger attention gathering potential than p 4. hese examples indicate that it is more intitive to measre the attention gathering potential of items by taking into accont not only the total nmber of actions they attract, bt also temporal aspects of social media activity. o captre sch temporal aspects, we propose a novel measre of attention gathering potential that encompasses the temporal dimension. he measre is derived from the analysis of ISIS, a general stochastic model for Interacting Streaming Information Sorces that is careflly defined to intitively follow the way individals in social media generate and engage with each other s items. In what follows, items with large attention gathering potential will be referred to as attention gathering items. Activity in social media is a dynamic process, with a large nmber of new items generated continosly and attention constantly shifting among items. In sch a dynamic setting, it is important that attention gathering items are identified in real time, as social media activity evolves. For example, if a recently pblished blog post reports an interesting story that attracts a significant nmber of links from other sites and comments from its viewers, it is preferable to report it in real time, as it might correspond to important emerging news (a crisis, accident, annoncement, etc). herefore, identification of attention gathering items is best sited as an online rather than offline task. Also, given the large volme of data sch a task needs to process in real time, it is more efficient to prne from consideration as early as possible items that do not appear likely to attract mch attention and focs on monitoring a smaller candidate set of items with larger attention gathering potential. A heristic way to identify attention gathering items in online fashion is the following: Maintain the nmber of actions each item attracts over time and report as attention gathering the ones that exceed a threshold k. Also, discard items that do not exceed the threshold after dt time from their creation. However, setting the parameters k and dt in a meaningfl way that takes into accont temporal aspects of social media activity, is a non-trivial isse: How wold k be set for sorces that interact at different rates with other sorces? If we wish to prne items that do not gather attention, what wold be the right vale for dt so that we discard them early, bt also avoid missing items that exceed threshold k later? o address the aforementioned isses, we present a principled approach that ses seqential statistical tests in order to achieve early online identification of attention gathering items. he tests are based on the assmptions of the ISIS model and allow for the exploration of trade-offs between early reporting of reslts and qality. Experiments over real data from social media activity demonstrate that this approach can achieve significantly early identification of attention gathering items, compromising little qality in its reslts. o smmarize, we make the following contribtions: We propose and analyze ISIS, a general stochastic model for interacting streaming information sorces. Under ISIS, we derive a measre for the attention gathering potential of information nits, that incorporates temporal aspects of social media activity in an intitive way. We present seqential statistical tests for early online identification of items with large attention gathering potential. We present experimental reslts on real data collected from a period of blogging activity. he experiments demonstrate the application of the model in real-world scenarios and attest to the efficiency and effectiveness of the proposed statistical tests for early identification of attention gathering items. o the best of or knowledge, this is the first work in the context of social media that formalizes and addresses the problem of early online identification of attention gathering items. 1.1 Roadmap he paper is organized as follows. Connection with previos work is discssed in section 2. he technical part of the paper is covered by sections 3 and 4. In section 3, we describe ISIS, a model that intitively follows the way social media activity evolves and we propose a measre for attention gathering potential. Sbseqently, based on the model and its analysis, section 4 describes how seqential tests are sed in order to achieve early, online identification of items with large attention gathering potential. Section 5 provides experimental reslts from the analysis of a blogging activity period that demonstrate the trade-offs in the performance of the seqential tests over real data. he paper concldes with section RELAED WORK Link analysis has been widely sed to obtain measres for the importance of webpages [7, 8, 17, 20]. Conventionally, webpages are modeled as nodes of a graph, with directed edges between nodes corresponding to hyperlinks between webpages [18]. Importance vales are then obtained for each webpage sing graphrelated measres for example, the PageRank of a webpage is sch a graph-based measre ([7] provides an in-depth smmary of link analysis approaches). Yet, the way social media activity evolves sggests a departre from the traditional web model. For instance, linking in social media is explicitly associated with individal docments, pictres, news articles, etc and not jst with the webpages that host those items. herefore, it is reasonable to have separate measres for the importance or attention gathering potential of different items. Moreover, linking activity in social media is the prodct of continos interaction between participating individals. Dynamic aspects of this process (sch as the rate with which content is generated or interactions occr) are not captred by the graph model, since it only considers the total nmber of links between webpages. Finally, linking is not the only action by which strctre arises in social media, as individals also interact by commenting, sharing, recommending or rating items they enconter online. In smmary, in the case of social media, individal webpages are better modeled as information sorces that emit information nits in a streaming fashion and interact by dynamically performing different types of actions pon each other s nits. In this work, we provide a first formal definition and analysis of sch a model and se it as a basis to identify attention gathering items in online fashion. Since attention gathering items possibly point to emerging events, or work has some affinity to event detection [4, 10, 11, 16, 19, 23]. Note, however, that there are strong dissimilarities between the two. As described in [4], the goal of event detection is to identify stories over a collection or stream of docments. ext analysis is applied towards that end, possibly taking into accont linking activity or an nderlying social network strctre [23]. On the contrary, or work identifies individal items that attract a significant nmber of actions and its main focs is early identification of sch items i.e. given a definition of what constittes an attention gathering item, identify it as early as possible. 3. HE ISIS MODEL In this section, we present ISIS, a model for interacting streaming information sorces, that intitively follows the way social media activity evolves and captres the concept of attention gathering 302

3 information items. he formal definition of the model is given in section 3.1 and its analysis is provided in section 3.2. Notation sed in this section and throghot the paper is smmarized in table 1. U p P P t p d p λ Λ x t x w p W X p X X time period nder stdy streaming information sorce the set of sorces information nit set of nits emitted by sorce U set of all nits generated dring creation time of nit p validity period of nit p the rate at which other sorces interact with sorce the set of interaction rates for all sorces in U action timestamp of action x interaction weight of nit p set of interaction weights of all nits in P the set of actions attracted by nit p the set of actions attracted by nits of sorce the set of all actions attracted by nits in P able 1: Notation 3.1 Model Definition he prpose of the ISIS model is to serve as an abstraction of social media activity. Information sorces (or sorces, for simplicity) in the model correspond to individals contribting information. A sorce is assmed to participate in two sets of stochastic processes: 1. he process of emitting information nits in a streaming fashion. 2. Processes of interaction with other sorces. Information nits (or, simply, nits ) emitted by sorces conceptally correspond to items sch as blog posts, stats messages, photos, etc that appear in the social media stream. Interaction between sorces corresponds to individals engaging with each other s items. hs, interaction between sorces is assmed to involve sorces performing actions pon nits emitted by other sorces. In the ISIS model, the notion of action is sed to represent different forms of engagement with items (sch as linking, commenting, recommending, etc). he two sets of processes are sbseqently described in detail (Sbsections and 3.1.2) Emission of Information Units Consider a set U of streaming information sorces. A sorce U emits nits according to a stochastic process, with every arrival of the process corresponding to the emission of a nit p (Figre 1). For example, nits emitted by a sorce might correspond to blog posts pblished on a blog. Each nit p is associated with two time vales, a timestamp t p and a validity period d p, both of which are known (observed variables). imestamp t p denotes the time when nit p is emitted. For example, posts pblished on a blog or stats pdates on micro-blogging websites (e.g. witter) are accompanied by a timestamp declaring the time the post or stats pdate was generated. Validity period d p is sed to model the temporary natre of social media activity, i.e. the fact that items generated in social media do not remain relevant, interesting or available to the pblic for an infinite amont of time. For example, readers of a blog are not expected to read and comment on posts that were created a long time ago or have been removed from the front page of that blog. In practice, we consider the validity period to be eqal to the time interval for which a post, news article, stats pdate or other item remains on the front page of the related blog, news portal, social network profile, etc. Dring the validity period of a nit, we refer to the nit as valid. For the prposes of the analysis that follows in section 3.2, assme that all qantities refer to social media activity that takes place dring a time period. In particlar, let P denote the set of all nits p emitted by sorce U and P denote the set of all nits. P = [ P U Figre 1: Information sorce. Each nit is associated with a timestamp t p and a validity period d p. Notice that validity periods of nits emitted by the same sorce might overlap Interactions between Streaming Information Sorces Besides emitting information nits, sorces also interact with each other e.g., friends in a social network interact by visiting each other s profile webpage. Moreover, dring interactions of a sorce with another sorce, there is a probability that performs an action pon valid nits of. For example, while individals interact with their friends in a social network by visiting their profiles, they sometimes perform an action (e.g. leave a comment) pon their friends posted items (pictres, stats pdates, etc). he time t x an action x occrs is known (observed variable). For instance, when a person leaves a comment on an item, the comment is accompanied by a timestamp that declares the time the action took place. In general, it is not possible to know when interactions occr, nless they involve an action. For example, it is not possible to determine when friends in a social network visit each other s profiles, as browsing history information is only available to the administrator of the social network website. Conseqently, the time interval δt between sccessive interactions of sorce with another sorce is a latent (nobserved) variable. In interest of simplicity, the process by which interactions occr is assmed memoryless. Specifically, a sorce i U is assmed to interact with sorce j U \{ i} according to a Poisson process I i, j (λ j ) of rate λ j [12], with every arrival of the process corresponding to a single interaction of i with j. Eqivalently, for any two sccessive interactions of i with j at times t k and t k+1, the inter-arrival interval δt = t k+1 t k of process I i, j (λ j ) is the vale of a random variable Δt that follows an exponential distribtion with parameter λ j. Pr(Δt = δt) =Exp(λ j )=λ j e λ j δt Interaction rate λ is a latent variable, the vale of which can be estimated based on the vales of observed variables, as explained in detail in section

4 Notice that interactions between sorces are not assmed to be symmetric, i.e. the process according to which a sorce U interacts with another sorce U is assmed distinct and independent from the process according to which interacts with. Notice also that the rate λ j with which sorce i interacts with sorce j is assmed to depend on j only and it will be referred to as the interaction rate of sorce 1 j. In what follows, Λ will be sed to denote the interaction rates of all sorces. Λ={λ U} Figre 2: Sorce interaction. Dring interaction of sorce with sorce, might perform an action pon a valid nit p of. More specifically, it is assmed that each valid nit p emitted by a sorce is associated with an interaction weight w p that determines the probability that a sorce which interacts with sorce performs an action pon p. For example, when a blog post p is viewed by a reader for the first time, the reader leaves a comment to p with probability w p. Interaction weight w p is not known a-priori (it is a latent variable). However, it can be estimated, given the nmber of actions nit p attracts, its validity period d p and the interaction rate λ of sorce. At a high level, the vales of d p and λ determine the nmber of interactions of sorces with sorce that occr while p is valid and therefore how many chances nit p has to attract an action. he smaller the vales of d p and λ, the smaller is the expected nmber of sch interactions. herefore, for a given nmber of actions attracted by nit p, the smaller its validity period d p and/or interaction rate λ, the larger the estimated vale of w p. On the other hand, for fixed vales of d p and λ, the larger the nmber of actions attracted by nit p, the larger its estimated w p. his connection between interaction weight w p and other variables (nmber of actions, d p and λ ) is shown analytically in section 3.2 and experimentally in section 5.2. We propose and se the estimated vale of w p as a measre for the attention gathering potential of items. In contrast with measres based solely on the nmber of actions an item attracts, the estimated vale of w p is not only a fnction of the nmber of actions, bt it also depends on temporal aspects of social media activity captred by d p and λ and has an intitive interpretation as a probability vale in the ISIS model. Estimation of w p throgh maximm likelihood analysis will be the sbject of section 3.2. In formal terms, if an arrival of process I i, j (λ j ) occrs at time t, sorce i performs an action x pon nit p emitted by sorce j with probability Pr action (p) =w p 1 One cold consider a more general model with a distinct interaction rate λ i, j for each pair of sorces i, j. However, in order to keep the presentation and analysis of the ISIS model as simple as possible, we make the assmption that all sorces i interact with j at the same rate λ j. as long as nit p satisfies the following two constraints: (1) p is valid at time t and (2) t is the first time i interacts with j while nit p is valid. he two constraints are imposed to model in a simple manner the temporary natre of social media activity, i.e. the fact that items do not attract actions for an infinite amont of time. If any of these two constraints is not satisfied, i performs no action pon nit p. Each action x is associated with a timestamp t x that denotes the time it occrred and which coincides with the time of the corresponding arrival of process I i, j (λ j ). In the example of figre 2, each arrow corresponds to an arrival of process I i, j (λ j ) and ths to an interaction of i with nits emitted by j. According to this specific example, interactions occr at times t 1,t 2,...,t 5 and according to ISIS, sorce i might perform an action pon nits p 1, p 2, p 3 at times t 1, t 3, t 5, respectively, each time with probability w pi, i = 1, 2, 3. However, it cannot perform an action pon any item at time t 2, since there is no valid nit emitted by sorce j at that time, nor at time t 4,since i had already interacted with j at time t 3, while nit p 2 was still valid. In principle, one can model different types of actions with different w s associated with each of them (i.e. se different w s for the actions of linking, commenting and so on). In interest of simplicity, a single type of action is assmed in this work; however extension of the model to more than one types of actions is straightforward. In what follows, W will be sed to denote the set of interaction weights for all emitted nits p P W = {w p emitted nit p P }. In addition, let X p denote the set of actions x along with their associated timestamp t x attracted by a single nit p and X denote all actions (together with their timestamps) attracted by nits p of sorce. (Notation p will be sed to denote that nit p has been emitted by sorce ). X will denote the entire set of actions created dring. X = [ X p, X = [ X p U 3.2 Analysis In this section, a maximm likelihood analysis for ISIS is provided. he prpose of the analysis is to estimate the vales of latent variables W and Λ given the vales of the observed variables. Consider the two examples shown in figre 3. Both depict sorce emitting a nit p at time t p, with the nit remaining valid for period d p. However, in figre 3(a) nit p attracts a small nmber of actions in total, while in 3(b) nit p ends p attracting many actions. If an estimate has to be derived for the respective interaction weights w 0, w 1 of nit p for the two cases, then, since the nmber of actions attracted by nit p in fig. 3(b) is more than in fig. 3(a) in the same time period, interaction weight w 1 will be larger than w 0. w 0 <w 1 In other words, since nder ISIS a higher vale of w p implies a larger expected nmber of attracted actions X p for nit p, then maximm likelihood analysis retrns higher estimates for w p when a larger nmber of actions X p is observed. However, the nmber of actions X p attracted by a nit p emitted by sorce does not depend only on w p. In fact, besides w p, X p also depends on the validity interval d p of nit p and the interaction rate λ of sorce. Specifically, a larger validity interval d p or interaction rate λ implies a larger expected nmber of actions X p nder ISIS. herefore, the same vale of X p might 304

5 nit p sorce nit p sorce t p t p (a) (b) t p +d p t p +d p actions actions Figre 3: Small and Large w p. ime ime lead to different (smaller or larger) estimates for w p, depending on the vale of d p or λ. For example, if two nits p 1, p 2 have the same nmber of attracted actions bt different validity periods X p1 = X p2 d p1 <d p2 then, nit p 1 will have a larger estimated interaction weight than p 2, since it attracted the same nmber of actions in less time. w p1 >w p Maximm Likelihood Estimation Assme that the sets of observed variables P and X are available for a period. Based on their vales, the maximm likelihood vales for the sets of latent variables Λ and W are compted. Let U be a sorce that emits a seqence of nits P = [p 1,p 2,...] dring time period and let X p =[x 1,x 2,...] be the set of actions attracted by nit p. hen, the log-likelihood fnction of the latent variables is given by the formla L(W, Λ) = log Pr(X P ; W, Λ) = X X L p(w p,λ j ) j U p P j (1) with L p(w p,λ j )=logpr(x p t p,d p; w p,λ j ), p j (2) being the log-likelihood of a nit p attracting actions X p, given its time vales t p, d p, its interaction weight w p and the rate λ j of interactions with sorce j. Calclating the r.h.s. expression of eqation 2 (details omitted de to space restrictions), we get X L p(w p,λ j )= X p log(λ j w p) λ j (t x t p)+ x X p (N X p )log(1 q pw p) (3) with N = U being the total nmber of sorces participating and q p being the likelihood that sorce i has interacted with j while nit p was valid. q p =1 e λ j dp he first two terms of eqation 3 represent the probability that a sorce i interacts with item p while it is valid and performs an action x at time t x, while the third term expresses the probability this does not happen. Maximizing L(W, Λ) reqires W, Λ sch that 8 < : L(W, Λ) = 0 0 w W 1 λ Λ 0. System 4 can be solved sing well known nmerical methods [13, 21, 15]. A special case of the model that helps obtain better intition pon the soltions of system 4 is analyzed in the following section A special case According to the definition of ISIS (Section 3.1), nits p are assmed to be associated with an interaction weight w p,that determines the probability they attract an action from sorces interacting with. No frther assmption is made abot weights w p, apart from the fact that they take vales in the range [0, 1]. o gain some intition into soltions of system 4, assme that all nits p emitted by sorce U share the same interaction weight w (4) w p = w, p P (5) and that the nits are emitted and remain valid niformly over time, i.e. that d p = d = k,p P, k= constant (6) P where k denotes the nmber of items of sorce that are valid at the same time. Assme also that all sorces share the same interaction rate λ which will be referred to as the global interaction rate. λ = λ, U (7) As it is easy to verify, for a fixed vale of λ, the interaction weights w p = w of nits p emitted by sorce are given by the formla w p = w = X N P q p = X N P (1 e kλ / P ) (8) where q p =1 e λdp =1 e λd =1 e kλ / P is the probability a sorce interacts with sorce while a particlar nit p is valid. Let s consider two extreme cases for λ in relation with d. Case 1: λk P = λd 0. hen, and q p =1 e λd 1 (1 λd )=λd = kλ / P w = w p X N P λ d = X λ X Case 2: λk P = λd. hen, X N P λ k q p =1 e λd 1 0=1 P and X w = w p N P X 1 P he first case demonstrates that when the rate λ at which other sorces interact with sorce is small compared to the rate P / = at which sorce emits nits, then the estimated vale of the d 1 305

6 interaction weight w is determined by the average nmber of actions per interaction X and ths by the total nmber X of λ attracted actions, since all sorces share a global interaction rate λ = λ. On the other extreme, when rate λ is large in comparison with P / = d 1, then the vale of w is determined by the average nmber of actions per nit emitted. he vale w, estimated nder the assmptions of eqations 5, 6 and 7, will be referred to as the aggregate interaction weight of sorce U. As it is also the case with interaction weight w p of individal items p, w has a simple and intitive interpretation as a probability in ISIS and characterizes the overall ability of a sorce to attract actions from other sorces with its nits. hs, jst as interaction weights w p are sed to measre and compare the attention gathering potential of individal items, aggregate interaction weights w are sed to perform a similar comparison at the sorce level. 4. EARLY ONLINE IDENIFICAION OF AENION GAHERING IEMS he ISIS model provides a formal framework for the estimation of interaction weights w p of nits p, as well as interaction rates λ of sorces, based on activity observed dring a time period. According to the analysis of section 3.2, estimation is achieved by solving system 4andisperformed inoffline fashion, after data from period is collected. In the context of social media monitoring, nits p with large estimated interaction weight w p correspond to items with high attention gathering potential. In this section, we explain how ISIS is sed when identification of items with high attention gathering potential needs to be performed in online fashion and retrn reslts as early as possible. he motivation for early, online identification of sch items comes from the fact that they might contain information that refers to an evolving event (e.g. a crisis) or point to novel information that people deem important. When sch items are identified, they are reported as attention gathering items. At the same time, items that are not likely to be of large attention gathering potential are prned from consideration. In this way, identification focses on a smaller sbset of candidate items. For an illstrative introdction to the problem, consider the two cases depicted in figre 4. Similarly with the examples of figre 3, they involve sorce emitting a nit p at time t p that remains valid for a period of length d p. However, nlike figre 3, figre 4 provides a snapshot of the activity p to time t within the validity period of nit p. Let w 1, w 0 be the interaction weights sed in the examples of figres 3(b) and 3(a), respectively, with w 0 <w 1. he qestion addressed in this section is the following: having observed the interaction of sorces U \{} with sorce onlyptotimet, isit possible to decide, with high confidence, whether nit p has a large interaction weight w 1 or a small interaction weight w 0? In fig. 4(b), for example, nit p has attracted many actions p to time t. Under the assmptions of ISIS, it is reasonable to predict that nit p will indeed end p with a large nmber of attracted actions ntil the end (t p + d p) of validation period, jst as in fig.3(b). hs in the example of fig. 4(b), we have strong early indication that nit p has large interaction weight w p rather than small and that it is more likely to be w p = w 1 rather than w p = w 0. On the contrary, the example of figre 4(a) is more similar to that of figre 3(a), and the small nmber of actions attracted by nit p p to time t indicate that its interaction weight w p is more likely to be closer to w 0 than to w 1. More formally, consider a sorce with known interaction rate nit p t p sorce t nit p p sorce t actions t (a) actions (b) t p +d p t p +d p Figre 4: Early Identification. ime ime λ. In practice, λ is estimated offline according to the analysis of section 3.2, based on activity of sorce dring a recent time period.hevaleofλ is considered to remain relatively invariant over short time periods and we make sre that new estimates of its vale are obtained reglarly. For each nit p emitted by, we wish to resolve as early as possible and with high confidence whether the interaction weight w p associated with p is large or small, where small and large are qantified by two vales of interaction weight w0 <w1, that are given as inpt. Specifically, based on the actions X p nit p gathers with time, we attempt to determine which of the two vales, w0 or w1 is the more likely interaction weight of nit p. Ifw1 is decided to be the one, then nit p is reported, otherwise if w0 is the most likely vale, it is ignored. In both cases, we reqire that a decision is taken with high confidence, i.e. that the probability or decision is mistaken is less than an error parameter ɛ. he vales of w0, w1 can be specified in varios ways. One option is that a ser of a social media monitoring system who wishes to detect attention gathering items of sorce in real time sets w0 and w1, ths specifying what level of interaction weight constittes an attention gathering item or not. A second option is to set them atomatically. One way to do this is the following. If the interaction weight of nits p generated by sorce over a recent time period have an average of m = avg p (w p) and a standard deviation of s = std p (w p),thenw0 and w1 are set to w 0 = m w 1 = m +2 s. he rationale for this selection of vales is that if interaction weights w p follow a Gassian distribtion with mean m and standard deviation s, then the probability of interaction weight higher than m +2s is less than 5% and ths it wold be intitive to report p as attention gathering. Notice that, in this way, items of sorce that are identified as attention gathering are not necessarily ones with large interaction weight in absolte terms, bt rather ones that have significantly large interaction weight relatively to the average estimate obtained from period. 4.1 Early Identification In this section, we explain how seqential tests [14, 22] offer a principled way to address the problem nder discssion. For a sorce U, two interaction weight vales w 0, w 1 are specified. For every nit p emitted by, consider hypotheses H 0 and H 1. H p 0 : w p = w 0 H p 1 : w p = w 1 (9) 306

7 o decide which of the two hypotheses to accept, we apply seqential test S, which is smmarized in table 2. Specifically, we observe the actions X t p attracted by p in the time period [t p,t]. Every time t sch an observation is made, a decision abot which hypothesis to accept is based on the likelihood ratio r t = Lt 1 = Pr(Xt p t p; w1,λ ) L t 0 Pr(Xp t t p; w0,λ), where L t 0, L t 1 are the likelihoods that nit p has interaction weight w0 or w1, given that it attracts actions Xp t till time t. If r t is sfficiently large, then hypothesis H 1 is accepted. How large r t needs to be for H 1 to be accepted is specified by test S based on the error parameter ɛ. Similarly, if r t is sfficiently small, then hypothesis H 0 is accepted. In the case that r t is not small or large enogh to decide which hypothesis to accept, the same procedre is repeated after a small time interval δs. In case the validity period d p of nit p expires before a decision is made, a decision is immediately made in favor of the hypothesis that corresponds to the larger likelihood L t 0 or L t 1 at time t = t p + d p. At any point in time, we maintain a set C of nits that are candidates to be identified as attention gathering. All nits are added to set C as soon as they are generated and remain its members for as long as test S has not decided which of the two hypotheses H p 0 or H p 1 to accept. Units are removed from set C when test S terminates. If hypothesis H p 1 is accepted, they are reported as attention gathering, otherwise they are discarded. able 2: Seqential est S Condition Decision r t < ɛ w 1 ɛ p = w0 ɛ 1 ɛ t p t t p + d p rt no decision yet 1 ɛ ɛ 1 ɛ <r ɛ t w p = w1 t p + d p <t L t 1 <L t 0 w p = w0 L t 1 L t 0 w p = w1 Seqential test S allows for the exploration of a trade-off between error parameter ɛ (i.e. the probability a correct decision is reached before the test is trncated) and the nmber of observations the test collects before one of the two hypotheses is accepted. More specifically, the larger error ɛ, the smaller is the time needed for a decision to be reached. In section 5, this trade-off is displayed experimentally over real datasets and it is shown that early identification can be achieved by compromising little qality in the reslts. 5. EXPERIMENS We provide experimental reslts from the application of ISIS (Section 3) and sage of seqential tests [22] (Section 4) on real social media data. In particlar, section 5.1 presents real examples of attention gathering items that prove that ISIS is able to identify items related to emerging events and/or of increased interest to social media adience. Sbseqently, section 5.2 demonstrates the ability of interaction weight w to captre temporal aspects of social media activity. Finally, section 5.3 presents the trade-offs that arise from the sage of the seqential tests. he dataset sed in the experiments was collected from BlogScope [5, 6], a social media warehosing platform developed at the University of oronto and which crrently hosts a mlti-terabyte collection of data from social media activity. In particlar, experiments were performed over real data from a 15-day period of blogging activity ( = [May 1st May 15th 2008]). he dataset consists of the activity of the 1000 most active blogs in that period, i.e. the blogs that attracted the most links from their viewers. In total, the dataset contained 280k posts, as well as 180k links attracted from those posts. he correspondence between blogging activity and ISIS is displayed in table 3. According to that, the set of blogs that were active dring period act as streaming information sorces, with blog posts as their emitted nits. Moreover, blogs interact when blog owners visit the webpage of other blogs and they perform the action of linking pon each other s posts i.e. dring interaction of blog with blog,blog possibly creates a link towards a blog post p generated on blog. Given this correspondence, the notation sed previosly for the definition of ISIS (Section 3) will also be sed for the description of the experiments. able 3: Correspondence between Model & Data Blogging Activity ISIS Model Blog Streaming Information Sorce Post Emitted Information Unit Visit a Blog owner visits Interaction between Sorces another Blog Link Action from a Blog to a Post performed by a Sorce pon a Unit 5.1 Attention Gathering Items In this part of experiments, we give examples of blog posts that are identified as attention gathering items nder ISIS. Following section 4, for each post p generated by blog we compare two hypotheses, H p 0 and H p 1. H p 0 : w p = w 0 H p 1 : w p = w 1 If the average vale of interaction weight w p for posts p of blog is m = avg p (w p) dring a recent period of activity and standard deviation is s = std p (w p), thenw 0 and w 1 are set as w 0 = m w 1 = m +2s. est S is performed with error parameter ɛ =0and posts p for which hypothesis H p 1 gets accepted are reported. We present examples of posts that are reported as attention gathering items. he posts come from two specific blogs, i.e. engadget. com and techcrnch.com and the w0,w1 vales that were sed for each blog dring test S are mentioned in table 4. For illstration prposes, figre 5 contains the vales of interaction weights w p of posts belonging to the two blogs, as estimated by solving system 4. he bottom horizontal line in each plot inside figre 5 corresponds to w0 while the top horizontal line corresponds to w1. he posts that were identified as attention gathering from test S are the ones with interaction weight w p that was closer to w1 than to w0. A sample of these posts (i.e. ones that were identified as attention gathering items) is shown in table 5. For example, on March 12th engadget.com pblished a post titled iphone OS 3.0 is coming, preview on March 17th 2. he post reported on emerging news abot the release of a new operating system for iphone and attracted nearly 100 links from other blogs that also reported on the news and cited engadget.com as their sorce. Similarly, on March 2 iphone-os-3-0-is-coming-march-17th/ 307

8 4th techcrnch.com pblished a post with title Facebook s response to witter 3. he post commented on the jst annonced change in Facebook s design and attracted a large nmber of links from other blogs that also commented on the news. he examples indicate that ISIS sccessflly identifies items related to emerging events or draw mch attention from social media sers. able 5: Attention Gathering Posts Microsoft shows a glimpse at the ftre of compting Apple notebook in Q3 iphone OS 3.0 is coming March 17th Ubnt 9.04 ported to Nokias N8x0 internet tablets hird party ipod Shffle headphones will reqire license Apple planning a March 24 event Big msic will srrender bt not ntil at least 2011 GrandCentral to finally lanch as Google Voice Google privacy blnder shares yor docs withot permission Wolfram Alpha comptes answers to factal qestions Facebook s response to twitter It s time to start thinking of witter as a search engine (a) (b) Figre 5: Interaction weights of posts in (a) engadget.com (b) techcrnch.com. able 4: Blogs Blog w 0 w 1 engadget.com techcrnch.com Connection between Λ, W In previos sections of this paper, we propose and se interaction weight w p as an intitive measre for the attention gathering potential of items p, that is not based only on the nmber of actions attracted, bt also takes into accont the temporal dimension of social media activity. he prpose of this part of experiments is to demonstrate that dependence of interaction weights on the temporal dimension throgh real examples. owards that end, we apply the analysis of section 3.2 on the blogging activity dataset collected from BlogScope and exhibit the connection between latent variables Λ and W of ISIS. In fact, in order to keep the presentation simple, we focs on the connection between global interaction rate λ and aggregate interaction weight w of blogs U, defined in section Following section 3.2.2, we assme that all blogs U share the same 3 facebooks-response-to-twitter/ interaction rate λ λ = λ, U and that posts of the same blog U share the same length of validity period d p = d = k,p. P he vale of constant k is set to k =50, as it was experimentally fond that for this vale most (nearly 90%) of observed links were inclded in the validity periods of the corresponding posts. Based on these assmptions and solving system 4, we estimate the aggregate interaction weight w of blogs based on three different and explicitly set vales of λ (λ =10 5 h 1,λ =10 3 h 1,λ = 10 1 h 1, h = 1 hor). For each vale of λ, the 10 blogs with maximm w were compted and are reported in figres 6(a), 6(b) and 6(c), respectively. he reslts in figre 6 are consistent with the theoretical analysis of section More specifically, for small interaction rate λ, aggregate interaction weight w is determined by the total nmber of links. For this reason, the list of blogs with maximm w (Figre 6(a)) is dominated by the blogs with largest total nmber of links. On the other hand, for large λ vales, w is determined by the average nmber of links per post. Conseqently, the list of blogs with maximm w (Figre 6(c)) is dominated by blogs with the largest average nmber of links per post. Finally, when the vale of λ is neither too big nor too small, the list of blogs with maximm w is mixed both with blogs with large total nmber of links or large average nmber of links. he reslts demonstrate an intitive relationship between interaction weights and interaction rates. In an setting where sorces interact very freqently with each other, all generated nits have the chance to attract an action. herefore it is reasonable to estimate the ability of a sorce to attract actions by the average nmber of actions attracted per nit. On the other hand, in a setting where sorces rarely interact with each other, the average nmber of actions attracted per nit is not a reasonable measre anymore: it wold nderestimate sorces that emit many nits, most of which do not actally get a chance to attract an action while they are valid. In sch settings, it is more intitive to measre the ability of a sorce to attract actions by the nmber of attracted actions per interaction, or simply the total nmber of actions if the interaction rate is eqal for all sorces. his argmentation can be extended from the sorce level to the level of nits p and their interaction weight w p. Under the described rationale, interaction weight w p is an intitive measre for attention gathering potential of items, that 308

9 Rank Blog Links Posts 1 2 engadget.com gardian.co.k.com 3 thinkprogress.org hotair.com techcrnch.com icanhascheezbrger.com 7 michellemalkin.com lifehacker.com Rank Rank pajamasmedia.com xkcd.com (a) λ =10 5 h 1, d p =50 Blog engadget.com xkcd.com gardian.co.k thinkprogress.org techcrnch.com hotair.com icanhascheezbrger.com michellemalkin.com thestorybeginnings.blogspot.com lifehacker.com Links P Posts (b) λ =10 3 h 1, d p =50 Blog xkcd.com thestorybeginnings.blogspot.com stevenberlinjohnson grosgrainfablos.blogspot.com sethgodin.typepad.com pinktentacle.com asofterworld.com fnnyordie.com smashingmagazine.com lonelyheartscasino.com Links P (c) λ =10 1 h 1, d p =50 Posts P Links/Post Links/Post Links/Post Figre 6: Blogs with maximm aggregate interaction weight for different λ vales. takes into accont the temporal aspects of social media activity Qality vs Efficiency rade-offs he experiments presented in this section aim to demonstrate the performance benefits from tilizing the seqential tests described in section 4 as well as the arising trade-offs in efficiency and qality. At a high level, qality in the experiments is measred by the fraction of correct decisions made by the seqential test w.r.t. the interaction weight of a post. Since for real data it is impossible to know the real vales of interaction weights, the classical measres of precision and recall cannot be sed and we ths need to resort to measres that are based on experimentally defined grond trth. he following seems to be a reasonable choice: a post will be said to be tre w 0 ( tre w 1 ) when it is more likely to be of interaction weight w 0 (w 1) at the end of its validity period. Qality is measred throgh the following qantities: experimental type I and type II error, w 1 imprity and tre w 1 miss rate. Experimental type I error expresses the fraction of tre w 0 posts that are decided by test S to be of interaction weight w 1. #(tre w0 decided w1) type I error = #(tre w 0) Similarly, experimental type II error expresses the fraction of tre w 1 posts that are decided by the seqential test S to be of interaction weight w 0. #(tre w1 decided w0) type II error = #(tre w 1) Moreover, w 1 imprity measres the fraction of posts identified as w 1 that are tre w 0 posts. #(tre w0 decided w1) imprity = #( decided w 1) Finally, miss rate expresses the fraction of tre w 1 posts that are either decided by test S to be of interaction weight w 0 or are not decided ntil their validity period expires. = miss rate = #(tre w1 ((decided w0) (test trncated))) #(tre w 1) #(tre w1 (test trncated)) = type II error + #(tre w 1) Efficiency is measred throgh average workload, i.e. the average nmber of posts over time that are considered as candidate attention gathering items. If D p [0,d p] is the time it takes for a post to be decided either as attention gathering or be prned from consideration, then P p workload = Dp Similarly with the qalitative experiments for attention gathering items (Section 5.1), the hypotheses tested by seqential test S were H p 0 : w p = w 0 = avg p (w p) H p 1 : wp = w 1 = avg p (w p)+2 std p (w p) with avg p (w p) being the average interaction weight w p of posts p of blog in a recent period of activity and std p (w p),beingthe standard deviation. Figre 7 demonstrates the workload for different model errors ɛ. As shown in the figre, workload ranges from 50k posts when ɛ =0and seqential test S is always trncated, to 5k posts when test S is performed with ɛ =0.5. Notice that by performing test S with ɛ =0.05, we already have a 50% decrease in the workload. Figre 8 demonstrates the trade-off between the workload and the qality in its reslts, as measred by the experimental type I and type II errors (please note that workload is normalized w.r.t. its maximm vale). As shown in the figre, if a type I error and type II error of 5% is tolerated, a 50% redction in workload is achieved for ɛ =0.05. Finally, it is interesting to notice that for small vales of ɛ (ɛ < 0.275) there is a trade-off between miss rate and imprity (Figre 9). Recall that miss rate corresponds to tre w 1 posts that are either (a) identified as w 0 (type II error) or (b) posts that test S fails to identify before it is trncated. For small ɛ most missed tre w 1 posts correspond to the second case. he interpretation of this trade-off is that, in order to make sbstantial se of the seqential 309