Statistical Debugging with Latent Topic Models
|
|
|
- Isabel Cook
- 5 years ago
- Views:
Transcription
1 Statistical Debugging with Latent Topic Models David Andrzejewski, Anne Mulhern, Ben Liblit, Xiaojin Zhu Department of Computer Sciences University of Wisconsin Madison European Conference on Machine Learning, 2007 (UW Madison) Topic-based statistical debugging ECML / 21
2 Outline 1 Statistical Debugging 2 Our Approach 3 Our Results (UW Madison) Topic-based statistical debugging ECML / 21
3
4
5 Debugging Debugging as machine learning (UW Madison) Topic-based statistical debugging ECML / 21
6 Program Runs as Documents Predicates Vocabulary Predicate counts Word counts Program run Bag-of-words document Debugging Latent topic analysis (UW Madison) Topic-based statistical debugging ECML / 21
7 Program Runs as Documents Predicates Vocabulary Predicate counts Word counts Program run Bag-of-words document Debugging Latent topic analysis (UW Madison) Topic-based statistical debugging ECML / 21
8 Program Runs as Documents Predicates Vocabulary Predicate counts Word counts Program run Bag-of-words document Debugging Latent topic analysis (UW Madison) Topic-based statistical debugging ECML / 21
9 Program Runs as Documents Predicates Vocabulary Predicate counts Word counts Program run Bag-of-words document Debugging Latent topic analysis (UW Madison) Topic-based statistical debugging ECML / 21
10 Program Runs as Documents Predicates Vocabulary Predicate counts Word counts Program run Bag-of-words document Debugging Latent topic analysis (UW Madison) Topic-based statistical debugging ECML / 21
11 Hidden topics Word counts (observed) Weighted latent topics (hidden) doc 1 doc 2 doc 3 θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 (UW Madison) Topic-based statistical debugging ECML / 21
12 Hidden topics Word counts (observed) doc 1 doc 2 doc 3 Weighted latent topics (hidden) φ 3, φ 4 = bug topics θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 θ 1 φ 1 + θ 2 φ 2 + θ 3 φ 3 + θ 4 φ 4 (UW Madison) Topic-based statistical debugging ECML / 21
13 Uncover Hidden Bugs Goals Uncover hidden bugs NOT predicting whether a single run will fail or not It will do that all on its own... Assumptions Few hidden bugs many failing runs 1 bug per failing run (UW Madison) Topic-based statistical debugging ECML / 21
14 Uncover Hidden Bugs Goals Uncover hidden bugs NOT predicting whether a single run will fail or not It will do that all on its own... Assumptions Few hidden bugs many failing runs 1 bug per failing run (UW Madison) Topic-based statistical debugging ECML / 21
15 Latent Topic Modeling Debugging Predicates Program runs Bug patterns Active bug patterns Topic modeling Words Documents Topics Topic weights (UW Madison) Topic-based statistical debugging ECML / 21
16 Latent Topic Modeling Debugging Predicates Program runs Bug patterns Active bug patterns Topic modeling Words Documents Topics Topic weights (UW Madison) Topic-based statistical debugging ECML / 21
17 Latent Topic Modeling Debugging Predicates Program runs Bug patterns Active bug patterns Topic modeling Words Documents Topics Topic weights (UW Madison) Topic-based statistical debugging ECML / 21
18 Latent Dirichlet Allocation (LDA) β u φ u Nu w z Nd θ D α s For each topic t φ t Dir(β) For each doc d θ d Dir(α) For each word w Topic z θ d w φ z (UW Madison) Topic-based statistical debugging ECML / 21
19 Latent Dirichlet Allocation (LDA) β u φ u Nu w z Nd θ D α s For each topic t φ t Dir(β) For each doc d θ d Dir(α) For each word w Topic z θ d w φ z (UW Madison) Topic-based statistical debugging ECML / 21
20 Latent Dirichlet Allocation (LDA) β u φ u Nu w z Nd θ D α s For each topic t φ t Dir(β) For each doc d θ d Dir(α) For each word w Topic z θ d w φ z (UW Madison) Topic-based statistical debugging ECML / 21
21 Latent Dirichlet Allocation (LDA) β u φ u Nu w z Nd θ D α s For each topic t φ t Dir(β) For each doc d θ d Dir(α) For each word w Topic z θ d w φ z (UW Madison) Topic-based statistical debugging ECML / 21
22 Latent Dirichlet Allocation (LDA) β u φ u Nu w z Nd θ D α s For each topic t φ t Dir(β) For each doc d θ d Dir(α) For each word w Topic z θ d w φ z (UW Madison) Topic-based statistical debugging ECML / 21
23 LDA motivation LDA cannot recover bug patterns Strong non-bug patterns also present Most runs successful Usage patterns explain behavior Some runs fail Usage patterns mostly explain behavior Usage patterns overwhelm bug patterns (UW Madison) Topic-based statistical debugging ECML / 21
24 LDA motivation LDA cannot recover bug patterns Strong non-bug patterns also present Most runs successful Usage patterns explain behavior Some runs fail Usage patterns mostly explain behavior Usage patterns overwhelm bug patterns (UW Madison) Topic-based statistical debugging ECML / 21
25 LDA motivation LDA cannot recover bug patterns Strong non-bug patterns also present Most runs successful Usage patterns explain behavior Some runs fail Usage patterns mostly explain behavior Usage patterns overwhelm bug patterns (UW Madison) Topic-based statistical debugging ECML / 21
26 LDA motivation LDA cannot recover bug patterns Strong non-bug patterns also present Most runs successful Usage patterns explain behavior Some runs fail Usage patterns mostly explain behavior Usage patterns overwhelm bug patterns (UW Madison) Topic-based statistical debugging ECML / 21
27 Toy Dataset (UW Madison) Topic-based statistical debugging ECML / 21
28 Toy Dataset (UW Madison) Topic-based statistical debugging ECML / 21
29 Toy Dataset (UW Madison) Topic-based statistical debugging ECML / 21
30 Toy Dataset (UW Madison) Topic-based statistical debugging ECML / 21
31 LDA β u b β φ u Nu b φ N b w z Nd o θ D α s f α w z θ φ u β u, α s o φ b β b, α f words topics topic weights usage topic multinomials hyperparameters outcome flag bug topic multinomials hyperparameters (UW Madison) Topic-based statistical debugging ECML / 21
32 LDA β u b β φ u Nu b φ N b w z Nd o θ D α s f α w z θ φ u β u, α s o φ b β b, α f words topics topic weights usage topic multinomials hyperparameters outcome flag bug topic multinomials hyperparameters (UW Madison) Topic-based statistical debugging ECML / 21
33 LDA β u β b φ u Nu φ b N b w z Nd o θ D α s α f w z θ φ u β u, α s o φ b β b, α f words topics topic weights usage topic multinomials hyperparameters outcome flag bug topic multinomials hyperparameters (UW Madison) Topic-based statistical debugging ECML / 21
34 LDA β u β b φ u Nu φ b Nb w z Nd o θ D α s α f w z θ φ u β u, α s o φ b β b, α f words topics topic weights usage topic multinomials hyperparameters outcome flag bug topic multinomials hyperparameters (UW Madison) Topic-based statistical debugging ECML / 21
35 LDA Successful run α s = [ ] Failing run α f = [ ] (UW Madison) Topic-based statistical debugging ECML / 21
![LDA Successful run α s = [1 1 1 0 0] Failing run α f = [1 1 1 1 1] Successful](/docs-images/84/90860179/images/36-0.jpg "run p(θ α s ) (UW Madison) Topic-based statistical debugging ECML 2007 12 /")
36 LDA Successful run α s = [ ] Failing run α f = [ ] Successful run p(θ α s ) (UW Madison) Topic-based statistical debugging ECML / 21
37 LDA Successful run α s = [ ] Failing run α f = [ ] Failing run p(θ α f ) (UW Madison) Topic-based statistical debugging ECML / 21
38 Inference Need to calculate posterior p(z w, o) Use z to estimate Topic multinomials φ for each topic Topic weights θ for each document Collapsed Gibbs Sampling Uses easily obtainable counts Efficient p(z k = i z k, w, o) ( n i k,jk + β i j k n i k, + W j βi j ) ( n d k k,i + α o k i n d k k, + N u+n b i α o k i ) (UW Madison) Topic-based statistical debugging ECML / 21
39 Inference Need to calculate posterior p(z w, o) Use z to estimate Topic multinomials φ for each topic Topic weights θ for each document Collapsed Gibbs Sampling Uses easily obtainable counts Efficient p(z k = i z k, w, o) ( n i k,jk + β i j k n i k, + W j βi j ) ( n d k k,i + α o k i n d k k, + N u+n b i α o k i ) (UW Madison) Topic-based statistical debugging ECML / 21
40 Group failing runs by bug Run 1 θ.. Run n θ (UW Madison) Topic-based statistical debugging ECML / 21
41 Results grep moss bug1 runs 144 bug2 runs topic topic 6 PCA PCA PCA bug1 runs 106 bug3 runs 147 bug4 runs 329 bug5 runs 206 bug8 runs 186 other runs (UW Madison) Topic-based statistical debugging ECML / 21
42 Results Rand index wrt ground truth LDA [1] [2] exif grep gzip moss Baselines 1 Statistical Debugging: Simultaneous Isolation of Multiple Bugs, Zheng et al ICML 06 2 Scalable Statistical Bug Isolation, Liblit et al PLDI 05 (UW Madison) Topic-based statistical debugging ECML / 21
43 Top predicates for each bug 1 Use φ to calculate p(z w) 2 For each bug topic z, sort predicates w by p(z w) Top predicates for exif bug topic 9 Rank p(z w) Predicate w Expert opinion jpeg-data.c:434 Direct result jpeg_data_set_exif_data() jpeg-data.c:436 Direct result jpeg_data_set_exif_data() jpeg-data.c:207 Smoking gun jpeg_data_load_data() (UW Madison) Topic-based statistical debugging ECML / 21
44 Top predicates for each bug 1 Use φ to calculate p(z w) 2 For each bug topic z, sort predicates w by p(z w) Top predicates for exif bug topic 9 Rank p(z w) Predicate w Expert opinion jpeg-data.c:434 Direct result jpeg_data_set_exif_data() jpeg-data.c:436 Direct result jpeg_data_set_exif_data() jpeg-data.c:207 Smoking gun jpeg_data_load_data() (UW Madison) Topic-based statistical debugging ECML / 21
45 Conclusion Debugging can be a machine learning problem LDA overcomes problems with standard LDA LDA successful on real-world data Other possible uses of LDA (eg, sentiment analysis) Acknowledgements NLM Training Grant 5T15LM07359, NSF Grant CCF , AFOSR Grant FA (UW Madison) Topic-based statistical debugging ECML / 21
46 LDA Debugging LDA Words Documents Usage topics Bug topics Bug topic documents Topic weights Debugging Predicates Program runs Usage patterns Bug patterns Failing runs Active usage/bug patterns (UW Madison) Topic-based statistical debugging ECML / 21
47 Debugging Dataset Runs Program Lines of Code Bugs Successful Failing exif 10, grep 15, gzip 8, moss 35, Topics Program Word Types Usage Bug exif grep 2, gzip 3, moss 1, (UW Madison) Topic-based statistical debugging ECML / 21
48 Joint distribution p(w, z) = p(w z)p(z) p(w z) = p(z) = (UW Madison) Topic-based statistical debugging ECML / 21
49 Joint distribution p(w, z) = p(w z)p(z) N p(w z) = p(φ i β) i dφ i p(z) = (UW Madison) Topic-based statistical debugging ECML / 21
50 Joint distribution p(w, z) = p(w z)p(z) N p(w z) = p(φ i β) p(z) = i D d p(θ d α) dφ i dθ d (UW Madison) Topic-based statistical debugging ECML / 21
51 Joint distribution p(w, z) = p(w z)p(z) N p(w z) = p(φ i β) p(z) = i D d p(θ d α) N i θ di n d i dφ i dθ d (UW Madison) Topic-based statistical debugging ECML / 21
52 Joint distribution p(w, z) = p(w z)p(z) N W n p(w z) = p(φ i β) φ j i ij dφ i p(z) = i D p(θ d α) d i j N θ di n d i dθ d (UW Madison) Topic-based statistical debugging ECML / 21
Topic Models. Charles Elkan November 20, 2008
Topic Models Charles Elan elan@cs.ucsd.edu November 20, 2008 Suppose that we have a collection of documents, and we want to find an organization for these, i.e. we want to do unsupervised learning. One
Introduction to Bayesian inference
Introduction to Bayesian inference Thomas Alexander Brouwer University of Cambridge tab43@cam.ac.uk 17 November 2015 Probabilistic models Describe how data was generated using probability distributions
Topic Modelling and Latent Dirichlet Allocation
Topic Modelling and Latent Dirichlet Allocation Stephen Clark (with thanks to Mark Gales for some of the slides) Lent 2013 Machine Learning for Language Processing: Lecture 7 MPhil in Advanced Computer
Document and Topic Models: plsa and LDA
Document and Topic Models: plsa and LDA Andrew Levandoski and Jonathan Lobo CS 3750 Advanced Topics in Machine Learning 2 October 2018 Outline Topic Models plsa LSA Model Fitting via EM phits: link analysis
Latent Dirichlet Allocation (LDA)
Latent Dirichlet Allocation (LDA) D. Blei, A. Ng, and M. Jordan. Journal of Machine Learning Research, 3:993-1022, January 2003. Following slides borrowed ant then heavily modified from: Jonathan Huang
Evaluation Methods for Topic Models
University of Massachusetts Amherst wallach@cs.umass.edu April 13, 2009 Joint work with Iain Murray, Ruslan Salakhutdinov and David Mimno Statistical Topic Models Useful for analyzing large, unstructured
Improving Topic Models with Latent Feature Word Representations
Improving Topic Models with Latent Feature Word Representations Dat Quoc Nguyen Joint work with Richard Billingsley, Lan Du and Mark Johnson Department of Computing Macquarie University Sydney, Australia
Sparse Stochastic Inference for Latent Dirichlet Allocation
Sparse Stochastic Inference for Latent Dirichlet Allocation David Mimno 1, Matthew D. Hoffman 2, David M. Blei 1 1 Dept. of Computer Science, Princeton U. 2 Dept. of Statistics, Columbia U. Presentation
CS Lecture 18. Topic Models and LDA
CS 6347 Lecture 18 Topic Models and LDA (some slides by David Blei) Generative vs. Discriminative Models Recall that, in Bayesian networks, there could be many different, but equivalent models of the same
Study Notes on the Latent Dirichlet Allocation
Study Notes on the Latent Dirichlet Allocation Xugang Ye 1. Model Framework A word is an element of dictionary {1,,}. A document is represented by a sequence of words: =(,, ), {1,,}. A corpus is a collection
Topic Models. Material adapted from David Mimno University of Maryland INTRODUCTION. Material adapted from David Mimno UMD Topic Models 1 / 51
Topic Models Material adapted from David Mimno University of Maryland INTRODUCTION Material adapted from David Mimno UMD Topic Models 1 / 51 Why topic models? Suppose you have a huge number of documents
Topic Models. Advanced Machine Learning for NLP Jordan Boyd-Graber OVERVIEW. Advanced Machine Learning for NLP Boyd-Graber Topic Models 1 of 1
Topic Models Advanced Machine Learning for NLP Jordan Boyd-Graber OVERVIEW Advanced Machine Learning for NLP Boyd-Graber Topic Models 1 of 1 Low-Dimensional Space for Documents Last time: embedding space
Non-Parametric Bayes
Non-Parametric Bayes Mark Schmidt UBC Machine Learning Reading Group January 2016 Current Hot Topics in Machine Learning Bayesian learning includes: Gaussian processes. Approximate inference. Bayesian
Applying Latent Dirichlet Allocation to Group Discovery in Large Graphs
Lawrence Livermore National Laboratory Applying Latent Dirichlet Allocation to Group Discovery in Large Graphs Keith Henderson and Tina Eliassi-Rad keith@llnl.gov and eliassi@llnl.gov This work was performed
LDA with Amortized Inference
LDA with Amortied Inference Nanbo Sun Abstract This report describes how to frame Latent Dirichlet Allocation LDA as a Variational Auto- Encoder VAE and use the Amortied Variational Inference AVI to optimie
Note for plsa and LDA-Version 1.1
Note for plsa and LDA-Version 1.1 Wayne Xin Zhao March 2, 2011 1 Disclaimer In this part of PLSA, I refer to [4, 5, 1]. In LDA part, I refer to [3, 2]. Due to the limit of my English ability, in some place,
Topic Modeling: Beyond Bag-of-Words
University of Cambridge hmw26@cam.ac.uk June 26, 2006 Generative Probabilistic Models of Text Used in text compression, predictive text entry, information retrieval Estimate probability of a word in a
Topic Models. Brandon Malone. February 20, Latent Dirichlet Allocation Success Stories Wrap-up
Much of this material is adapted from Blei 2003. Many of the images were taken from the Internet February 20, 2014 Suppose we have a large number of books. Each is about several unknown topics. How can
Latent Dirichlet Bayesian Co-Clustering
Latent Dirichlet Bayesian Co-Clustering Pu Wang 1, Carlotta Domeniconi 1, and athryn Blackmond Laskey 1 Department of Computer Science Department of Systems Engineering and Operations Research George Mason
Introduction To Machine Learning
Introduction To Machine Learning David Sontag New York University Lecture 21, April 14, 2016 David Sontag (NYU) Introduction To Machine Learning Lecture 21, April 14, 2016 1 / 14 Expectation maximization
Distributed Gibbs Sampling of Latent Topic Models: The Gritty Details THIS IS AN EARLY DRAFT. YOUR FEEDBACKS ARE HIGHLY APPRECIATED.
Distributed Gibbs Sampling of Latent Topic Models: The Gritty Details THIS IS AN EARLY DRAFT. YOUR FEEDBACKS ARE HIGHLY APPRECIATED. Yi Wang yi.wang.2005@gmail.com August 2008 Contents Preface 2 2 Latent
Generative Clustering, Topic Modeling, & Bayesian Inference
Generative Clustering, Topic Modeling, & Bayesian Inference INFO-4604, Applied Machine Learning University of Colorado Boulder December 12-14, 2017 Prof. Michael Paul Unsupervised Naïve Bayes Last week
Lecture 8: Graphical models for Text
Lecture 8: Graphical models for Text 4F13: Machine Learning Joaquin Quiñonero-Candela and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/
Measuring Topic Quality in Latent Dirichlet Allocation
Measuring Topic Quality in Sergei Koltsov Olessia Koltsova Steklov Institute of Mathematics at St. Petersburg Laboratory for Internet Studies, National Research University Higher School of Economics, St.
Gibbs Sampling. Héctor Corrada Bravo. University of Maryland, College Park, USA CMSC 644:
Gibbs Sampling Héctor Corrada Bravo University of Maryland, College Park, USA CMSC 644: 2019 03 27 Latent semantic analysis Documents as mixtures of topics (Hoffman 1999) 1 / 60 Latent semantic analysis
Note 1: Varitional Methods for Latent Dirichlet Allocation
Technical Note Series Spring 2013 Note 1: Varitional Methods for Latent Dirichlet Allocation Version 1.0 Wayne Xin Zhao batmanfly@gmail.com Disclaimer: The focus of this note was to reorganie the content
Applying LDA topic model to a corpus of Italian Supreme Court decisions
Applying LDA topic model to a corpus of Italian Supreme Court decisions Paolo Fantini Statistical Service of the Ministry of Justice - Italy CESS Conference - Rome - November 25, 2014 Our goal finding
A Unified Posterior Regularized Topic Model with Maximum Margin for Learning-to-Rank
A Unified Posterior Regularized Topic Model with Maximum Margin for Learning-to-Rank Shoaib Jameel Shoaib Jameel 1, Wai Lam 2, Steven Schockaert 1, and Lidong Bing 3 1 School of Computer Science and Informatics,
Distributed ML for DOSNs: giving power back to users
Distributed ML for DOSNs: giving power back to users Amira Soliman KTH isocial Marie Curie Initial Training Networks Part1 Agenda DOSNs and Machine Learning DIVa: Decentralized Identity Validation for
Gaussian Mixture Model
Case Study : Document Retrieval MAP EM, Latent Dirichlet Allocation, Gibbs Sampling Machine Learning/Statistics for Big Data CSE599C/STAT59, University of Washington Emily Fox 0 Emily Fox February 5 th,
Latent Dirichlet Allocation
Outlines Advanced Artificial Intelligence October 1, 2009 Outlines Part I: Theoretical Background Part II: Application and Results 1 Motive Previous Research Exchangeability 2 Notation and Terminology
Topic Models and Applications to Short Documents
Topic Models and Applications to Short Documents Dieu-Thu Le Email: dieuthu.le@unitn.it Trento University April 6, 2011 1 / 43 Outline Introduction Latent Dirichlet Allocation Gibbs Sampling Short Text
Collapsed Variational Inference for HDP
Collapse Variational Inference for HDP Yee W. Teh Davi Newman an Max Welling Publishe on NIPS 2007 Discussion le by Iulian Pruteanu Outline Introuction Hierarchical Bayesian moel for LDA Collapse VB inference
Latent Dirichlet Allocation
Latent Dirichlet Allocation 1 Directed Graphical Models William W. Cohen Machine Learning 10-601 2 DGMs: The Burglar Alarm example Node ~ random variable Burglar Earthquake Arcs define form of probability
Modeling Environment
Topic Model Modeling Environment What does it mean to understand/ your environment? Ability to predict Two approaches to ing environment of words and text Latent Semantic Analysis (LSA) Topic Model LSA
Bayesian Inference for Dirichlet-Multinomials
Bayesian Inference for Dirichlet-Multinomials Mark Johnson Macquarie University Sydney, Australia MLSS Summer School 1 / 50 Random variables and distributed according to notation A probability distribution
Dirichlet Enhanced Latent Semantic Analysis
Dirichlet Enhanced Latent Semantic Analysis Kai Yu Siemens Corporate Technology D-81730 Munich, Germany Kai.Yu@siemens.com Shipeng Yu Institute for Computer Science University of Munich D-80538 Munich,
Latent Dirichlet Allocation Introduction/Overview
Latent Dirichlet Allocation Introduction/Overview David Meyer 03.10.2016 David Meyer http://www.1-4-5.net/~dmm/ml/lda_intro.pdf 03.10.2016 Agenda What is Topic Modeling? Parametric vs. Non-Parametric Models
Efficient Tree-Based Topic Modeling
Efficient Tree-Based Topic Modeling Yuening Hu Department of Computer Science University of Maryland, College Park ynhu@cs.umd.edu Abstract Topic modeling with a tree-based prior has been used for a variety
Latent Dirichlet Alloca/on
Latent Dirichlet Alloca/on Blei, Ng and Jordan ( 2002 ) Presented by Deepak Santhanam What is Latent Dirichlet Alloca/on? Genera/ve Model for collec/ons of discrete data Data generated by parameters which
Statistical Debugging. Ben Liblit, University of Wisconsin Madison
Statistical Debugging Ben Liblit, University of Wisconsin Madison Bug Isolation Architecture Program Source Predicates Sampler Compiler Shipping Application Top bugs with likely causes Statistical Debugging
Mixtures of Multinomials
Mixtures of Multinomials Jason D. M. Rennie jrennie@gmail.com September, 25 Abstract We consider two different types of multinomial mixtures, () a wordlevel mixture, and (2) a document-level mixture. We
Latent Dirichlet Allocation (LDA)
Latent Dirichlet Allocation (LDA) A review of topic modeling and customer interactions application 3/11/2015 1 Agenda Agenda Items 1 What is topic modeling? Intro Text Mining & Pre-Processing Natural Language
Another Walkthrough of Variational Bayes. Bevan Jones Machine Learning Reading Group Macquarie University
Another Walkthrough of Variational Bayes Bevan Jones Machine Learning Reading Group Macquarie University 2 Variational Bayes? Bayes Bayes Theorem But the integral is intractable! Sampling Gibbs, Metropolis
INTERPRETING THE PREDICTION PROCESS OF A DEEP NETWORK CONSTRUCTED FROM SUPERVISED TOPIC MODELS
INTERPRETING THE PREDICTION PROCESS OF A DEEP NETWORK CONSTRUCTED FROM SUPERVISED TOPIC MODELS Jianshu Chen, Ji He, Xiaodong He, Lin Xiao, Jianfeng Gao, and Li Deng Microsoft Research, Redmond, WA 9852,
Tutorial on Gaussian Processes and the Gaussian Process Latent Variable Model
Tutorial on Gaussian Processes and the Gaussian Process Latent Variable Model (& discussion on the GPLVM tech. report by Prof. N. Lawrence, 06) Andreas Damianou Department of Neuro- and Computer Science,
Language Information Processing, Advanced. Topic Models
Language Information Processing, Advanced Topic Models mcuturi@i.kyoto-u.ac.jp Kyoto University - LIP, Adv. - 2011 1 Today s talk Continue exploring the representation of text as histogram of words. Objective:
Fast Inference and Learning for Modeling Documents with a Deep Boltzmann Machine
Fast Inference and Learning for Modeling Documents with a Deep Boltzmann Machine Nitish Srivastava nitish@cs.toronto.edu Ruslan Salahutdinov rsalahu@cs.toronto.edu Geoffrey Hinton hinton@cs.toronto.edu
Incorporating Social Context and Domain Knowledge for Entity Recognition
Incorporating Social Context and Domain Knowledge for Entity Recognition Jie Tang, Zhanpeng Fang Department of Computer Science, Tsinghua University Jimeng Sun College of Computing, Georgia Institute of
Online Bayesian Passive-Agressive Learning
Online Bayesian Passive-Agressive Learning International Conference on Machine Learning, 2014 Tianlin Shi Jun Zhu Tsinghua University, China 21 August 2015 Presented by: Kyle Ulrich Introduction Online
Applying hlda to Practical Topic Modeling
Joseph Heng lengerfulluse@gmail.com CIST Lab of BUPT March 17, 2013 Outline 1 HLDA Discussion 2 the nested CRP GEM Distribution Dirichlet Distribution Posterior Inference Outline 1 HLDA Discussion 2 the
Dimension Reduction (PCA, ICA, CCA, FLD,
Dimension Reduction (PCA, ICA, CCA, FLD, Topic Models) Yi Zhang 10-701, Machine Learning, Spring 2011 April 6 th, 2011 Parts of the PCA slides are from previous 10-701 lectures 1 Outline Dimension reduction
16 : Approximate Inference: Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models 10-708, Spring 2017 16 : Approximate Inference: Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Yuan Yang, Chao-Ming Yen 1 Introduction As the target distribution
Robust Spectral Inference for Joint Stochastic Matrix Factorization
Robust Spectral Inference for Joint Stochastic Matrix Factorization Kun Dong Cornell University October 20, 2016 K. Dong (Cornell University) Robust Spectral Inference for Joint Stochastic Matrix Factorization
Time-Sensitive Dirichlet Process Mixture Models
Time-Sensitive Dirichlet Process Mixture Models Xiaojin Zhu Zoubin Ghahramani John Lafferty May 25 CMU-CALD-5-4 School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 Abstract We introduce
Web-Mining Agents Topic Analysis: plsi and LDA. Tanya Braun Ralf Möller Universität zu Lübeck Institut für Informationssysteme
Web-Mining Agents Topic Analysis: plsi and LDA Tanya Braun Ralf Möller Universität zu Lübeck Institut für Informationssysteme Acknowledgments Pilfered from: Ramesh M. Nallapati Machine Learning applied
LDA Collapsed Gibbs Sampler, VariaNonal Inference. Task 3: Mixed Membership Models. Case Study 5: Mixed Membership Modeling
Case Stuy 5: Mixe Membership Moeling LDA Collapse Gibbs Sampler, VariaNonal Inference Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox May 8 th, 05 Emily Fox 05 Task : Mixe
Scaling Neighbourhood Methods
Quick Recap Scaling Neighbourhood Methods Collaborative Filtering m = #items n = #users Complexity : m * m * n Comparative Scale of Signals ~50 M users ~25 M items Explicit Ratings ~ O(1M) (1 per billion)
Graphical Models for Collaborative Filtering
Graphical Models for Collaborative Filtering Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Sequence modeling HMM, Kalman Filter, etc.: Similarity: the same graphical model topology,
Understanding Comments Submitted to FCC on Net Neutrality. Kevin (Junhui) Mao, Jing Xia, Dennis (Woncheol) Jeong December 12, 2014
Understanding Comments Submitted to FCC on Net Neutrality Kevin (Junhui) Mao, Jing Xia, Dennis (Woncheol) Jeong December 12, 2014 Abstract We aim to understand and summarize themes in the 1.65 million
Latent variable models for discrete data
Latent variable models for discrete data Jianfei Chen Department of Computer Science and Technology Tsinghua University, Beijing 100084 chris.jianfei.chen@gmail.com Janurary 13, 2014 Murphy, Kevin P. Machine
Inference Methods for Latent Dirichlet Allocation
Inference Methods for Latent Dirichlet Allocation Chase Geigle University of Illinois at Urbana-Champaign Department of Computer Science geigle1@illinois.edu October 15, 2016 Abstract Latent Dirichlet
A Practical Algorithm for Topic Modeling with Provable Guarantees
1 A Practical Algorithm for Topic Modeling with Provable Guarantees Sanjeev Arora, Rong Ge, Yoni Halpern, David Mimno, Ankur Moitra, David Sontag, Yichen Wu, Michael Zhu Reviewed by Zhao Song December
Introduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
how to *do* computationally assisted research
how to *do* computationally assisted research digital literacy @ comwell Kristoffer L Nielbo knielbo@sdu.dk knielbo.github.io/ March 22, 2018 1/30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 class Person(object):
Probablistic Graphical Models, Spring 2007 Homework 4 Due at the beginning of class on 11/26/07
Probablistic Graphical odels, Spring 2007 Homework 4 Due at the beginning of class on 11/26/07 Instructions There are four questions in this homework. The last question involves some programming which
METHODS FOR IDENTIFYING PUBLIC HEALTH TRENDS. Mark Dredze Department of Computer Science Johns Hopkins University
METHODS FOR IDENTIFYING PUBLIC HEALTH TRENDS Mark Dredze Department of Computer Science Johns Hopkins University disease surveillance self medicating vaccination PUBLIC HEALTH The prevention of disease,
Collapsed Variational Bayesian Inference for Hidden Markov Models
Collapsed Variational Bayesian Inference for Hidden Markov Models Pengyu Wang, Phil Blunsom Department of Computer Science, University of Oxford International Conference on Artificial Intelligence and
CSCI 5822 Probabilistic Model of Human and Machine Learning. Mike Mozer University of Colorado
CSCI 5822 Probabilistic Model of Human and Machine Learning Mike Mozer University of Colorado Topics Language modeling Hierarchical processes Pitman-Yor processes Based on work of Teh (2006), A hierarchical
Lecture 19, November 19, 2012
Machine Learning 0-70/5-78, Fall 0 Latent Space Analysis SVD and Topic Models Eric Xing Lecture 9, November 9, 0 Reading: Tutorial on Topic Model @ ACL Eric Xing @ CMU, 006-0 We are inundated with data
Design of Text Mining Experiments. Matt Taddy, University of Chicago Booth School of Business faculty.chicagobooth.edu/matt.
Design of Text Mining Experiments Matt Taddy, University of Chicago Booth School of Business faculty.chicagobooth.edu/matt.taddy/research Active Learning: a flavor of design of experiments Optimal : consider
Gentle Introduction to Infinite Gaussian Mixture Modeling
Gentle Introduction to Infinite Gaussian Mixture Modeling with an application in neuroscience By Frank Wood Rasmussen, NIPS 1999 Neuroscience Application: Spike Sorting Important in neuroscience and for
Web Search and Text Mining. Lecture 16: Topics and Communities
Web Search and Tet Mining Lecture 16: Topics and Communities Outline Latent Dirichlet Allocation (LDA) Graphical models for social netorks Eploration, discovery, and query-ansering in the contet of the
Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework
HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for
Motif representation using position weight matrix
Motif representation using position weight matrix Xiaohui Xie University of California, Irvine Motif representation using position weight matrix p.1/31 Position weight matrix Position weight matrix representation
COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017
COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University TOPIC MODELING MODELS FOR TEXT DATA
Streaming Variational Bayes
Streaming Variational Bayes Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C. Wilson, Michael I. Jordan UC Berkeley Discussion led by Miao Liu September 13, 2013 Introduction The SDA-Bayes Framework
Lecture 6: Graphical Models: Learning
Lecture 6: Graphical Models: Learning 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering, University of Cambridge February 3rd, 2010 Ghahramani & Rasmussen (CUED)
Introduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process
Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process Chong Wang Computer Science Department Princeton University chongw@cs.princeton.edu David M. Blei Computer Science Department
Content-based Recommendation
Content-based Recommendation Suthee Chaidaroon June 13, 2016 Contents 1 Introduction 1 1.1 Matrix Factorization......................... 2 2 slda 2 2.1 Model................................. 3 3 flda 3
Hybrid Models for Text and Graphs. 10/23/2012 Analysis of Social Media
Hybrid Models for Text and Graphs 10/23/2012 Analysis of Social Media Newswire Text Formal Primary purpose: Inform typical reader about recent events Broad audience: Explicitly establish shared context
arxiv: v3 [stat.ml] 15 Aug 2017
![arxiv: v3 [stat.ml] 15 Aug 2017](/thumbs/93/111929332.jpg "arxiv: v3 [stat.ml] 15 Aug 2017")
SPARSE PARTIALLY COLLAPSED MCMC FOR PARALLEL INFERENCE IN TOPIC MODELS MÅNS MAGNUSSON, LEIF JONSSON, MATTIAS VILLANI AND DAVID BROMAN ABSTRACT. Topic models, and more specifically the class of Latent Dirichlet
\ fwf The Institute for Integrating Statistics in Decision Sciences
# \ fwf The Institute for Integrating Statistics in Decision Sciences Technical Report TR-2007-8 May 22, 2007 Advances in Bayesian Software Reliability Modelling Fabrizio Ruggeri CNR IMATI Milano, Italy
Advanced Introduction to Machine Learning
10-715 Advanced Introduction to Machine Learning Homework 3 Due Nov 12, 10.30 am Rules 1. Homework is due on the due date at 10.30 am. Please hand over your homework at the beginning of class. Please see
Using Both Latent and Supervised Shared Topics for Multitask Learning
Using Both Latent and Supervised Shared Topics for Multitask Learning Ayan Acharya, Aditya Rawal, Raymond J. Mooney, Eduardo R. Hruschka UT Austin, Dept. of ECE September 21, 2013 Problem Definition An
A prior distribution over model space p(m) (or hypothesis space ) can be updated to a posterior distribution after observing data y.
June 2nd 2011 A prior distribution over model space p(m) (or hypothesis space ) can be updated to a posterior distribution after observing data y. This is implemented using Bayes rule p(m y) = p(y m)p(m)
Text mining and natural language analysis. Jefrey Lijffijt
Text mining and natural language analysis Jefrey Lijffijt PART I: Introduction to Text Mining Why text mining The amount of text published on paper, on the web, and even within companies is inconceivably
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation Yee Whye Teh Gatsby Computational Neuroscience Unit University College London 17 Queen Square, London WC1N 3AR, UK ywteh@gatsby.ucl.ac.uk
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation
THIS IS A DRAFT VERSION. FINAL VERSION TO BE PUBLISHED AT NIPS 06 A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation Yee Whye Teh School of Computing National University
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Clustering: Part 2 Instructor: Yizhou Sun yzsun@ccs.neu.edu October 19, 2014 Methods to Learn Matrix Data Set Data Sequence Data Time Series Graph & Network
Context-Aware Statistical Debugging
Context-Aware Statistical Debugging From Bug Predictors to Faulty Control Flow Paths Lingxiao Jiang and Zhendong Su University of California at Davis Outline Introduction Context-Aware Statistical Debugging
Machine Learning Summer School
Machine Learning Summer School Lecture 3: Learning parameters and structure Zoubin Ghahramani zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Department of Engineering University of Cambridge,
topic modeling hanna m. wallach
university of massachusetts amherst wallach@cs.umass.edu Ramona Blei-Gantz Helen Moss (Dave's Grandma) The Next 30 Minutes Motivations and a brief history: Latent semantic analysis Probabilistic latent
arxiv: v2 [cs.lg] 27 Feb 2016
![arxiv: v2 [cs.lg] 27 Feb 2016](/thumbs/73/68289512.jpg "arxiv: v2 [cs.lg] 27 Feb 2016")
A Gibbs-Newton Technique for Enhanced Inference of Multivariate Polya Parameters and Topic Models arxiv:50.06646v2 [cs.lg] 27 Feb 206 Osama Khalifa David Wolfe Corne Mike Chantler Mathematical and Computer
Learning Bayesian network : Given structure and completely observed data
Learning Bayesian network : Given structure and completely observed data Probabilistic Graphical Models Sharif University of Technology Spring 2017 Soleymani Learning problem Target: true distribution
Infering the Number of State Clusters in Hidden Markov Model and its Extension
Infering the Number of State Clusters in Hidden Markov Model and its Extension Xugang Ye Department of Applied Mathematics and Statistics, Johns Hopkins University Elements of a Hidden Markov Model (HMM)
Clustering bi-partite networks using collapsed latent block models
Clustering bi-partite networks using collapsed latent block models Jason Wyse, Nial Friel & Pierre Latouche Insight at UCD Laboratoire SAMM, Université Paris 1 Mail: jason.wyse@ucd.ie Insight Latent Space
Graphical Models and Kernel Methods
Graphical Models and Kernel Methods Jerry Zhu Department of Computer Sciences University of Wisconsin Madison, USA MLSS June 17, 2014 1 / 123 Outline Graphical Models Probabilistic Inference Directed vs.
Clustering problems, mixture models and Bayesian nonparametrics
Clustering problems, mixture models and Bayesian nonparametrics Nguyễn Xuân Long Department of Statistics Department of Electrical Engineering and Computer Science University of Michigan Vietnam Institute