Microblog Hot Spot Mining Based on PAM Probabilistic Topic Model

Transcription

1 MATEC Web of Conference 22, ( 2015) DOI: / matecconf/ C Owned by the author, publihed by EDP Science, 2015 Microblog Hot Spot Mining Baed on PAM Probabilitic Topic Model Yaxin Zheng & Liu Ling Chongqing Univerity, Chongqing, China ABSTRACT: Microblog are hort text carried with limited information, which will increae the difficulty of topic mining. Thi paper propoe the ue of PAM (Pachinko Allocation Model) probabilitic topic model to extract the generative model of text implicit theme for microblog hot pot mining. Firt, three categorie of microblog and the main contribution of thi paper are illutrated. Second, for there are four topic model which are repectively explained, the PAM model i introduced in detail in term of how to generate a document, the accuracy of document claification and the topic correlation in PAM. Finally, MapReduce i decribed. For the number of microblog i huge a well a the number of contactor, the totally number of word i relatively mall With MapReduce, microblog data are plit by contactor, document-topic count matrix and contactor-topic count matrix can be locally tored while the word-topic count matrix mut be globally tored. Thu, the hot pot mining can be achieved on the bai of PAM probabilitic topic model. Keyword: microblog; hot pot; PAM probabilitic topic model; MapReduce 1 INTRODUCTION Microblog i becoming a major ource for producing and preading hot pot on the internet, it provide a hort and convenient way for uer to expre and hare their attitude intantly. With the development of Web2.0, there i a continuou improvement in people participation and the method of uing the Internet ha been changed that people are no longer paive in acquiition of knowledge from the network, but to expre their own view initiatively or attitude toward other or event through a network. Microblogging update meage via hort 140 character, and achieve intant hare of multi-tool, making it an important new media on the Internet. The emergence of microblogging make the information preent in microblogging with characteritic of fragmentation, intant and mobility, rather than complete contextual information. Through the microblogging freedom, convenience and intant way to expre their feeling, it ha become fahion on the Internet, but alo make it an important place to generate and talk about hot pot, which mean the event, the topic or the information ha been widely concerned, debated and dicued within a certain time, o the tudy on hot pot finding, monitoring and management in microblogging platform will become increaingly important. MapReduce i a framework for proceing huge dataet with ditributed Map and Reduction operation (Figure 1). In Map tep, the mater node plit input data into partition, which can be proceed by uer-defined Map function, and produce (key, value) tuple a the intermediate output. In the reduce tep, the reduce function are ued to merge all intermediate Figure 1. MapReduce framework Thi i an Open Acce article ditributed under the term of the Creative Common Attribution Licene 4.0, which permit unretricted ue, ditribution, and reproduction in any medium, provided the original work i properly cited. Article available at or

2 MATEC Web of Conference tuple with the ame key, ort them and output the final (key, value) tuple. Both the map function and reduce function can be executed in parallel on no overlapping input and intermediate data. In MapReduce framework, microblog mut be plit into partition. Under the ituation of real world application, the number of microblog i huge (over million) a well a the number of contactor, while the total number of word i relatively mall. Through thi analyi on the inference proce of PAM, we find out if microblog data are plit by contactor, the document-topic count matrix and the contactor-topic count matrix can be locally tored while the word-topic count matrix mut be globally tored becaue only the word-topic count matrix need global update. At preent, the dometic microblogging reearch i in it infancy, and many reearch quetion are to be olved in the field. Today, the monitoring and management of hot pot mining i becoming important area of reearch in the huge information flow of microblogging. When a certain event occur in hot microblogging platform, people can ue microblog to expre their view or attitude toward the hot pot. A to the microblog hot pot mining, the PAM probabilitic topic model can be ued in microblogging platform. The probabilitic topic model i ued to regard the theme a a lexical item of probability ditribution, and the text i een a a random mix theme. Compared with the clutering method, uing the ditribution of the word, a text meage will be eaily tranformed into digital information modeling, with the ability to recognize the potential of large-cale centralized theme text meage. Theme can be more intuitive expreion, which greatly implify the complexity of the problem, and the application of topic model i increaingly widepread. A microblog i becoming more and more popular, microblog ervice have become information provider on a web cale, o reearche on microblog begin to focu more on it content mining than olely uer relationhip analyi before. Although traditional text mining method have been tudied well, no algorithm i epecially deigned for microblog data, which contain tructured information on ocial network beide plain text. With the continuou development of the theme model, it ha a very wide range of application in term of text claification, information retrieval and natural language proceing. The PAM probabilitic topic model ha become one of the popular model becaue of it tructure flexibility, it can acce to a wealth of emantic aociation and it difficult to produce the phenomenon of over-fitting and o on, and it currently ha a certain application in the image retrieval, the document claification and the object recognition and o on. The goal of topic modeling i to automatically dicover the topic from a collection of document. The document themelve are oberved, while the topic tructure, the topic, per-document topic ditribution, and the per-document per-word topic aignment are hidden tructure. 2 CATEGORIES OF MICROBLOG AND PAPER CONTRIBUTION Microblog can be divided into three categorie: broadcat, converation, and retweet meage. Broadcat meage are mot common and can be een by any uer; converation meage tarting with a pecial have pecific contactor to talk to; retweet meage which i identified by a pecial ymbol RT allow uer to repot other meage with their peronal comment. In the information exploion era, how to effectively dig out latent topic and internal emantic tructure from large cale data i an important iue. Microblog contain the tructured information on ocial network except the plain text, and the relationhip on ocial network can play a upporting role in topic mining. On the other hand, microblog are hort text carried with limited information (which i retricted to 140 character), which will increae the difficulty of the topic mining. Thee natural feature of microblog mentioned above can prevent the traditional text mining algorithm to be directly employed with their full potential. In thi paper, we make the following contribution: A novel model PAM i propoed, it i uitable for microblog data by taking both tructured information and untructured information into conideration. Ditributed PAM in MapReduce framework i propoed in order to meet the requirement of proceing large cale microblog with high calability. The ret of thi paper i organized a follow. Section 3 introduce the novel model PAM for microblog mining. Section 4 propoe the PAM probabilitic topic model in MapReduce framework for large cale ituation and it applie performance model to further optimize our approach. 3 PAM MODEL There are four topic model, the model tructure are repectively hown in Figure 2. (a) Dirichlet Multinomial: For each document, a multinomial ditribution on word i ampled from a ingle Dirichlet. (b) LDA (latent Dirichlet allocation): Thi model ample a multinomial on topic for each document, and then generate word from the topic. (c) Four-Level PAM: A four-level hierarchy conit of a root, a et of ub-topic, a et of uper-topic and a word vocabulary. The uper-topic and the root are both aociated with Dirichlet ditribution, from which we ample multinomial over their children for each document p.2

3 ICETA 2015 Figure 2. Model tructure for four topic model (d) PAM: An arbitrary DAG tructure i ued to encode the topic correlation. Each interior node i conidering a topic and it i aociated with a Dirichlet ditribution. In thi ection, we introduce the pachinko allocation model (PAM), which ue a directed acyclic graph (DAG) tructure to repreent and learn arbitrary-arity, neted, and poibly pare topic correlation. In PAM, the concept of topic i extended to be ditribution not only over word, but alo over other topic. The model tructure conit of an arbitrary DAG, in which each leaf node i aociated with a word in the vocabulary, and each non-leaf\interior node that correpond to a topic ha a ditribution over it children. An interior node whoe children are all leave would correpond to a traditional LDA topic. But ome interior node that may alo have children are other topic, thu they are repreented a a mixture over topic. With many node like thee, PAM therefore capture not only correlation among word (a in LDA), but alo correlation among topic themelve. The LDA and four-level PAM graphical model are hown in Figure 3. A we can ee, the major difference i that PAM ha one additional layer of uper-topic modeled with Dirichlet ditribution, which are the key component of capturing topic correlation here. where topic node occupy the interior level and the leave are word. To generate a document d, we follow a two-tep proce: (1) Sample,,, t 1 t 2 t from g ( ), g ( ),, g ( ), where ti i a multinomial ditribution of topic t i over it children. (2) For each word ω in the document, Sample a topic path Z of length L : Z1, Z2,, Z L,. Z 1 i alway the root and Z 2 to Z L are topic node in T. Z i i a ubet of Z ( i 1) and it i ampled according to the multinomial ditribution. Z ( i 1) Sample word ω from Z L. Following thi proce, the joint probability of generating a document d, the topic aignment z and the multinomial ditribution θ are hown a follow: P(d,z, ) P( ) P(z ) P( ) ti i i i1 i2 z( i1) Through integrating out θ and umming over z, we calculate the marginal probability of a document which i hown a follow: L P(d ) P( ti i) ( P(z i z ) P( ( 1) z ))d i L L i1 z i2 Finally, the probability of generating a whole corpu i the product of the probability for every document: P(D ) P( d ) d A to the accuracy of document claification, we can refer to [11], it i hown a follow: zl Figure 3. LDA and Four-Level PAM To generate a document, LDA ample a multinomial ditribution over topic from g (), then repeatedly ample a topic from thi multinomial, and a word from the topic. Now we introduce notation for the pachinko allocation model. PAM i ued to connect word in V and topic in T with an arbitrary DAG, Figure 4. Accuracy of document claification When we randomly chooe a ubet of abtract p.3

4 MATEC Web of Conference Figure 5. Topic correlation in PAM from it large collection, where there are 4000 document, word token and unique word. Baed on PAM, the topic correlation can be een in Figure 5. Each circle correpond to a uper-topic each box correpond to a ub-topic. One uper-topic can be ued to connect to everal ub-topic and capture their correlation. The number on the edge are the correponding value for the (uper-topic, ub-topic) pair. Figure 4 how a ubet of uper-topic in the data, and how they capture correlation among ub-topic. 4 SIMULATIONS AND RESULTS of Dirichlet procee. To validate the effectivene and feaibility of the propoed PAM probabilitic topic model method in microblog hot pot mining, we can make imulation of PAM, LDA, CTM and HDP that have been propoed in ome reference. Chooe the index of Log-likelihood a an evaluation index, the imulation reult can be een in Figure 6. Then, we can ee that for given microblog hot pot number and training data, the Log-likelihood of PAM i much bigger, which implie that the PAM probabilitic topic model i effective and feaible in the microblog hot pot mining. After enough iteration, the model reache the convergence and calculate out the final count matrice. With thee count matrice, we can obtain the related ditribution, which are ueful in topic mining for microblog. In MapReduce environment with a cluter reource, the mapper are mainly ued to conduct the ampling proce in parallel and the reducer mainly update the count matrice for next iteration, which make PAM effective and calable in topic mining for large cale microblog. Compared with the ame PAM-baed microblog hot pot mining method in [13], thi paper propoe the MapReduce decription, and plit microblog data into contactor o that document-topic count matrix and contactor-topic count matrix can be locally tored. PAM and LDA have been introduced in detail a mentioned above. A to CTM and HDP, we will introduce them in brief. Although CTM and PAM are both trying to model topic correlation directly, PAM take a more flexible approach that can capture the neted correlation. In fact, CTM i very imilar to a pecial-cae tructure of PAM, while HDP can capture different topic correlation within thee group by uing a neted hierarchy p.4

5 ICETA 2015 Figure 6. Simulation reult 5 CONCLUSIONS In thi paper, we have preented pachinko allocation, a mixture model which ue a DAG tructure to capture arbitrary topic correlation. The DAG tructure i completely general, and ome topic model like LDA can be repreented a pecial cae of PAM. Compared with other approache for microblog hot pot mining uch a hierarchical LDA and correlated topic model, PAM provide more expreive power to upport complicated topic tructure and adopt more realitic aumption for generating document. In addition, thi paper introduce the novel model PAM for microblog mining and propoe the PAM probabilitic topic model in MapReduce framework for large cale ituation and applie performance model to further optimize our approach. [7] Andrzejewki D, Zhu X, Craven M Incorporating domain knowledge into topic modeling via Dirichlet foret prior. Proceeding of the 26th Annual International Conference on Machine Learning. ACM, pp: [8] Liu C, Wang F In-orbit etimation of inertia parameter of target atellite after capturing the tracking atellite. Intelligent Control and Automation (WCICA), th World Congre on. IEEE, pp: [9] Mimno D, McCallum A. Topic model conditioned on arbitrary feature with dirichlet-multinomial regreion. arxiv preprint arxiv: , [10]Teh Y W, Jordan M I, Beal M J, et al Hierarchical dirichlet procee. Journal of the American Statitical Aociation, 101(476). [11]Yu M, Wang J, Zhao X, et al Reearch on PAM Probability topic model. Computer Science, 40(5): 1-7. [12]Liu C, et al Ma and ma center identification of target atellite after rendezvou and docking. Intelligent Control and Automation (WCICA), th World Congre on. IEEE, pp: [13]Zhang C, Sun J Large cale microblog mining uing ditributed MB-LDA. Proceeding of the 21t International Conference Companion on World Wide Web. ACM, pp: [14]Yu M, Zhou Z, et al PAM-baed microblog hot pot mining. Technique and Method, 32(15): REFERENCES [1] Blei D M, Lafferty J D Topic model. Text Mining: Claification, Clutering, and Application, 10: 71. [2] Ramage D, Hall D, Nallapati R, et al Labeled LDA: A upervied topic model for credit attribution in multi-labeled corpora. Proceeding of the 2009 Conference on Empirical Method in Natural Language Proceing. Aociation for Computational Linguitic, pp: [3] Mimno D, Li W, McCallum A Mixture of hierarchical topic with pachinko allocation. Proceeding of the 24th International Conference on Machine Learning. ACM, pp: [4] Li W., McCallum A Pachinko allocation: DAG-tructured mixture model of topic correlation. Proceeding of the 23rd International Conference on Machine Learning. ACM, pp: [5] Liu C, Wang F, Shi K, et al Robut H control for atellite attitude control ytem with uncertaintie and additive perturbation. International Journal of Science, 1(2): 1-9. [6] Blei D M Probabilitic topic model. Communication of the ACM, 55(4): p.5