Resolving Entity Coreference in Croatian with a Constrained Mention-Pair Model. Goran Glavaš and Jan Šnajder TakeLab UNIZG
|
|
|
- Jessica Sullivan
- 5 years ago
- Views:
Transcription
1 Resolving Entity Coreference in Croatian with a Constrained Mention-Pair Model Goran Glavaš and Jan Šnajder TakeLab UNIZG BSNLP RANLP, Hissar 10 Sep 2015
2 Background & Motivation Entity coreference resolution (CR) Identifying different mentions of the same entity Important NLP task with numerous applications: relation extraction, question answering, summarization,... Easy to define but difficult to tackle External knowledge often required (e.g., U.S. President Barack Obama ) Glavaš & Šnajder: Coreference Resolution for Croatian 2/20
3 Existing Work Early, rule-based CR focused on theories of discourse such as focusing and centering (Sidner 1979; Grosz et al., 1983) Shift to machine-learning approaches occurred with appearance of manually annotated coreference data (MUC) Glavaš & Šnajder: Coreference Resolution for Croatian 3/20
4 Existing Work Early, rule-based CR focused on theories of discourse such as focusing and centering (Sidner 1979; Grosz et al., 1983) Shift to machine-learning approaches occurred with appearance of manually annotated coreference data (MUC) The mention-pair model is the most widely applied coreference resolution model (Aone and Bennett, 1995) A binary classifier for pairs of event mentions Fails to account for transitivity of the coreference relation Glavaš & Šnajder: Coreference Resolution for Croatian 3/20
5 Existing Work Early, rule-based CR focused on theories of discourse such as focusing and centering (Sidner 1979; Grosz et al., 1983) Shift to machine-learning approaches occurred with appearance of manually annotated coreference data (MUC) The mention-pair model is the most widely applied coreference resolution model (Aone and Bennett, 1995) A binary classifier for pairs of event mentions Fails to account for transitivity of the coreference relation More complex models failed to significantly outperform the mention-pair model Entity-mention models (Daume III and Marcu, 2005) Ranking models (Yang et al., 2008) Glavaš & Šnajder: Coreference Resolution for Croatian 3/20
6 Existing Work Besides large body of work for English, much work has been done for other major languages as well Spanish (Palomar et al., 2001; Sapena et al., 2010) Italian (Kobdani and Schütze 2010; Poesio et al., 2010) German (Versley, 2006; Wunsch, 2010) Chinese (Converse, 2006; Kong and Zhou, 2010)... Glavaš & Šnajder: Coreference Resolution for Croatian 4/20
7 Existing Work Besides large body of work for English, much work has been done for other major languages as well Spanish (Palomar et al., 2001; Sapena et al., 2010) Italian (Kobdani and Schütze 2010; Poesio et al., 2010) German (Versley, 2006; Wunsch, 2010) Chinese (Converse, 2006; Kong and Zhou, 2010)... Research for Slavic languages has been quite limited Substantial research for Polish (Marciniak, 2002; Matysiak, 2007; Kopec and Ogrodniczuk, 2012) Czech (Linh et al., 2009) Bulgarian (Zhikov et al., 2013) Glavaš & Šnajder: Coreference Resolution for Croatian 4/20
8 Coreference Resolution for Croatian 1 Data Annotation 2 Constrained Mention-Pair Model Mention-Pair Model Enforcing Transitivity via ILP 3 Experimental Setup and Results 4 Conclusion Glavaš & Šnajder: Coreference Resolution for Croatian 5/20
9 Data Annotation We adopt the CR type scheme for Polish (Ogrodniczuk et al., 2013) CR type Identity Hyper-hypo Meronymy Metonymy -Anaphora Example Premijer je izjavio da on nije odobrio taj zahtjev. (The Prime Minister said he didn t grant that request.) Ivan je kupio novi automobil. Taj Mercedes je čudo od auta. (Ivan bought a new car. That Mercedes is an amazing car.) Od jedanaestorice rukometaša danas je igralo samo njih osam. (Only eight out of eleven handball players played today.) Dinamo je jučer pobijedio Cibaliju. Zagrepčani su postigli tri pogotka. (Dinamo defeated Cibalia yesterday. Zagreb boys scored three goals.) Marko je išao u trgovinu. Kupio je banane. (Marko went to the store. [He] bought bananas.) Glavaš & Šnajder: Coreference Resolution for Croatian 6/20
10 Data Annotation News articles corpus of 285 documents Six trained annotators Detailed annotation guidelines In-house developed annotation tool Glavaš & Šnajder: Coreference Resolution for Croatian 7/20
11 Data Annotation News articles corpus of 285 documents Six trained annotators Detailed annotation guidelines In-house developed annotation tool Workflow: 1 Calibration round on 15 documents + discussion + consenzus 2 Round 1 Three pairs of annotators, each working on 45 documents Each annotator annotated the data independently 3 Round 2 Same as Round 1, but with reshuffled annotator pairs 4 Estimate of the average pairwise IAA 70% agreement 5 Resolving the disagreements (one person) Glavaš & Šnajder: Coreference Resolution for Croatian 7/20
12 Data Annotation News articles corpus of 285 documents Six trained annotators Detailed annotation guidelines In-house developed annotation tool Workflow: 1 Calibration round on 15 documents + discussion + consenzus 2 Round 1 Three pairs of annotators, each working on 45 documents Each annotator annotated the data independently 3 Round 2 Same as Round 1, but with reshuffled annotator pairs 4 Estimate of the average pairwise IAA 70% agreement 5 Resolving the disagreements (one person) Final dataset: 270 documents with 13K CR relations Glavaš & Šnajder: Coreference Resolution for Croatian 7/20
13 Our Focus 1 We don t consider the mention detection but instead work on gold mentions 2 We consider only the Identity relation, which accounts for 87% CR relations 3 Identity is an equivalence relation, thus we want clusters Glavaš & Šnajder: Coreference Resolution for Croatian 8/20
14 Constrained Mention-Pair Model A mention-pair model is a binary classifier Predicts whether two given mentions refer to the same entity To produce clusters of coreferent mentions, we need to couple the mention-pair model with 1 A heuristic for creating mention-pair instances 2 A method for ensuring the transitivity of coreference relations (i.e., coherence of pairwise decisions) Glavaš & Šnajder: Coreference Resolution for Croatian 9/20
15 Creating Mention-Pair Instances Considering all possible mention pairs is not feasible Too many instances, the vast majority of which are negative We follow the approach by Ng and Cardie (2002) for creating training instances A positive instance between a mention m j and its closest preceding non-pronomial coreferent mention m i Negative instances by pairing m j with all mentions in between m j and its closest preceding coreferent mention m i (i.e., with m i+1,..., m j 1 ) Glavaš & Šnajder: Coreference Resolution for Croatian 10/20
16 The Mention-Pair Model A non-linear SVM (RBF) with 16 binary/numerical features: 1 String-matching features compare two mentions at the superficial string level strings identical, mention containment, longest common subsequence length, edit (Levenshtein) distance 2 Overlap features quantify the overlap in tokens at least one matching word/lemma/stem between mentions, number of common content (N/A/V/R) lemmas 3 Grammatical features aim to indicate the grammatical compatibility of the mentions pronominal mentions, gender match, number match 4 Distance-based features measure how close are the mentions distance in number of sentences/tokens, same sentence, adjacent mentions, number of mentions in between Glavaš & Šnajder: Coreference Resolution for Croatian 11/20
17 Enforcing Transitivity By making only pairwise predictions, the mention-pair model does not guarantee document-level coherence of coreference We employ constrained optimization via integer linear programming (ILP) to ensure that document-level coreference transitivity holds Glavaš & Šnajder: Coreference Resolution for Croatian 12/20
18 Enforcing Transitivity By making only pairwise predictions, the mention-pair model does not guarantee document-level coherence of coreference We employ constrained optimization via integer linear programming (ILP) to ensure that document-level coreference transitivity holds Objective function (to be maximized): x ij r(m i, m j ) C(m i, m j ) (m i,m j ) P r(m i, m j ) { 1, 1} is the mention-pair classifier s decision for mentions m i and m j C(m i, m j ) [0.5, 1] is the confidence of the binary mention-pair classifier x ij {0, 1} is the final decision for mentions m i and m j Glavaš & Šnajder: Coreference Resolution for Croatian 12/20
19 Enforcing Transitivity Transitivity property is encoded via linear constraints x ij + x jk x ik 1, x ij + x ik x jk 1, x jk + x ik x ij 1, {(m i, m j ), (m j, m k ), (m i, m k )} P After optimization, we obtain coreference clusters by simply computing the transitive closure over coherent pairwise decisions x ij Glavaš & Šnajder: Coreference Resolution for Croatian 13/20
20 Experimental Setup Dataset split: 220 training documents, 50 test documents SVM model selection (C and γ optimization) using 10-fold CV on the train set Glavaš & Šnajder: Coreference Resolution for Croatian 14/20
21 Experimental Setup Dataset split: 220 training documents, 50 test documents SVM model selection (C and γ optimization) using 10-fold CV on the train set Two baselines: 1 Overlap baseline classifies mentions as coreferent if they share at least one content word 2 GendNum baseline links each mention to the closest preceding mention of matching gender and number Standard closest-first clustering (Soon et al., 2001) is applied for both baselines Glavaš & Šnajder: Coreference Resolution for Croatian 14/20
22 Experimental Setup Standard coreference evaluation metrics: MUC and B 3 Models: MP Morph the binary mention pair model without grammatical features MP the binary mention-pair model MP+ILP the constrained mention-pair model (global coherence) Glavaš & Šnajder: Coreference Resolution for Croatian 15/20
23 Results MUC B 3 Model P R F 1 P R F 1 Overlap GendNum MP Morph MP MP+ILP Glavaš & Šnajder: Coreference Resolution for Croatian 16/20
24 Results MUC B 3 Model P R F 1 P R F 1 Overlap GendNum MP Morph MP MP+ILP MP significantly outperforms the baselines Removing morphological features lowers F1 by 2 points Enforcing transitivity significantly increases B 3 by 2 points Glavaš & Šnajder: Coreference Resolution for Croatian 16/20
25 Error Analysis Most false negatives due to cases where external knowledge is needed željezni kancelar (iron chancellor) Bismarck Most false positives due to non-coreferent mentions with significant lexical overlap Društvo hrvatskih književnika (Croatian Writers Association) svečanosti u Društvu hrvatskih književnika (ceremonies at the Croatian Writers Association) Glavaš & Šnajder: Coreference Resolution for Croatian 17/20
26 Conclusion The first coreference resolution model for Croatian A supervised mention-pair model is coupled with constrained optimization (using ILP) to enforce transitivity of coreference relations Most errors originate from lack of external knowledge needed to infer coreference Glavaš & Šnajder: Coreference Resolution for Croatian 18/20
27 Future Work Knowledge-based features from external sources like Wikipedia Detection of near-identity coreference relations (e.g., meronymy and zero anaphora) Building an end-to-end coreference resolution system A model for automated mention detection Glavaš & Šnajder: Coreference Resolution for Croatian 19/20
28 Thanks! Test data: Glavaš & Šnajder: Coreference Resolution for Croatian 20/20
Multiple Aspect Ranking Using the Good Grief Algorithm. Benjamin Snyder and Regina Barzilay MIT
Multiple Aspect Ranking Using the Good Grief Algorithm Benjamin Snyder and Regina Barzilay MIT From One Opinion To Many Much previous work assumes one opinion per text. (Turney 2002; Pang et al 2002; Pang
Semantics and Pragmatics of NLP Pronouns
Semantics and Pragmatics of NLP Pronouns School of Informatics University of Edinburgh Outline Observations About Data 1 Observations of what factors influence the way pronouns get resolved 2 Some algorithms
Automatically Evaluating Text Coherence using Anaphora and Coreference Resolution
1 1 Barzilay 1) Automatically Evaluating Text Coherence using Anaphora and Coreference Resolution Ryu Iida 1 and Takenobu Tokunaga 1 We propose a metric for automatically evaluating discourse coherence
Latent Variable Models in NLP
Latent Variable Models in NLP Aria Haghighi with Slav Petrov, John DeNero, and Dan Klein UC Berkeley, CS Division Latent Variable Models Latent Variable Models Latent Variable Models Observed Latent Variable
From Binary to Multiclass Classification. CS 6961: Structured Prediction Spring 2018
From Binary to Multiclass Classification CS 6961: Structured Prediction Spring 2018 1 So far: Binary Classification We have seen linear models Learning algorithms Perceptron SVM Logistic Regression Prediction
CLRG Biocreative V
CLRG ChemTMiner @ Biocreative V Sobha Lalitha Devi., Sindhuja Gopalan., Vijay Sundar Ram R., Malarkodi C.S., Lakshmi S., Pattabhi RK Rao Computational Linguistics Research Group, AU-KBC Research Centre
Task Oriented Dialog, Classic Models
Task Oriented Dialog, Classic Models Marilyn Walker. CS245. April 1 st, 2010 Natural Language and Dialogue Systems Lab Look at Syllabus Need to do sign ups for papers starting next week Will finish post
Multi-Task Structured Prediction for Entity Analysis: Search Based Learning Algorithms
Multi-Task Structured Prediction for Entity Analysis: Search Based Learning Algorithms Chao Ma, Janardhan Rao Doppa, Prasad Tadepalli, Hamed Shahbazi, Xiaoli Fern Oregon State University Washington State
Discriminative Training
Discriminative Training February 19, 2013 Noisy Channels Again p(e) source English Noisy Channels Again p(e) p(g e) source English German Noisy Channels Again p(e) p(g e) source English German decoder
Chinese Zero Pronoun Resolution: A Joint Unsupervised Discourse-Aware Model Rivaling State-of-the-Art Resolvers
Chinese Zero Pronoun Resolution: A Joint Unsupervised Discourse-Aware Model Rivaling State-of-the-Art Resolvers Chen Chen and Vincent Ng Human Language Technology Research Institute University of Texas
Evaluation Strategies
Evaluation Intrinsic Evaluation Comparison with an ideal output: Challenges: Requires a large testing set Intrinsic subjectivity of some discourse related judgments Hard to find corpora for training/testing
A Probabilistic Model for Canonicalizing Named Entity Mentions. Dani Yogatama Yanchuan Sim Noah A. Smith
A Probabilistic Model for Canonicalizing Named Entity Mentions Dani Yogatama Yanchuan Sim Noah A. Smith Introduction Model Experiments Conclusions Outline Introduction Model Experiments Conclusions Outline
Unsupervised Coreference Resolution in a Nonparametric Bayesian Model
Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan Klein Computer Science Division UC Berkeley {aria42, klein}@cs.berkeley.edu Abstract We present an unsupervised,
Unsupervised Vocabulary Induction
Infant Language Acquisition Unsupervised Vocabulary Induction MIT (Saffran et al., 1997) 8 month-old babies exposed to stream of syllables Stream composed of synthetic words (pabikumalikiwabufa) After
NEAL: A Neurally Enhanced Approach to Linking Citation and Reference
NEAL: A Neurally Enhanced Approach to Linking Citation and Reference Tadashi Nomoto 1 National Institute of Japanese Literature 2 The Graduate University of Advanced Studies (SOKENDAI) nomoto@acm.org Abstract.
Jeff Howbert Introduction to Machine Learning Winter
Classification / Regression Support Vector Machines Jeff Howbert Introduction to Machine Learning Winter 2012 1 Topics SVM classifiers for linearly separable classes SVM classifiers for non-linearly separable
CPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining Linear Classifiers: multi-class Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due in a week Midterm:
Chunking with Support Vector Machines
NAACL2001 Chunking with Support Vector Machines Graduate School of Information Science, Nara Institute of Science and Technology, JAPAN Taku Kudo, Yuji Matsumoto {taku-ku,matsu}@is.aist-nara.ac.jp Chunking
Evaluation. Brian Thompson slides by Philipp Koehn. 25 September 2018
Evaluation Brian Thompson slides by Philipp Koehn 25 September 2018 Evaluation 1 How good is a given machine translation system? Hard problem, since many different translations acceptable semantic equivalence
Preference-Based Rank Elicitation using Statistical Models: The Case of Mallows
Preference-Based Rank Elicitation using Statistical Models: The Case of Mallows Robert Busa-Fekete 1 Eyke Huellermeier 2 Balzs Szrnyi 3 1 MTA-SZTE Research Group on Artificial Intelligence, Tisza Lajos
Bayesian Estimation of Bipartite Matchings for Record Linkage
Bayesian Estimation of Bipartite Matchings for Record Linkage Mauricio Sadinle msadinle@stat.duke.edu Duke University Supported by NSF grants SES-11-30706 to Carnegie Mellon University and SES-11-31897
CSE 546 Final Exam, Autumn 2013
CSE 546 Final Exam, Autumn 0. Personal info: Name: Student ID: E-mail address:. There should be 5 numbered pages in this exam (including this cover sheet).. You can use any material you brought: any book,
Discourse Structure and Co-Reference: An Empirical Study
Discourse Structure and Co-Reference: An Empirical Study Dan Cristea Department of Computer Science University A.I. Cuza Iaşi, România dcristea@infoiasi.ro Nancy Ide Department of Computer Science Vassar
Final Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
A Discriminative Model for Semantics-to-String Translation
A Discriminative Model for Semantics-to-String Translation Aleš Tamchyna 1 and Chris Quirk 2 and Michel Galley 2 1 Charles University in Prague 2 Microsoft Research July 30, 2015 Tamchyna, Quirk, Galley
An Empirical Investigation of the Relation Between Discourse Structure and Co-Reference
An Empirical Investigation of the Relation Between Discourse Structure and Co-Reference Dan Cristea University A.I. Cuza Iaşi, România dcristea@infoiasi.ro Nancy Ide Vassar College Poughkeepsie, NY, USA
Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Previous lectures: Sparse vectors recap How to represent
ANLP Lecture 22 Lexical Semantics with Dense Vectors
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Henry S. Thompson ANLP Lecture 22 5 November 2018 Previous
Google s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Google s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Y. Wu, M. Schuster, Z. Chen, Q.V. Le, M. Norouzi, et al. Google arxiv:1609.08144v2 Reviewed by : Bill
Word Alignment via Submodular Maximization over Matroids
Word Alignment via Submodular Maximization over Matroids Hui Lin, Jeff Bilmes University of Washington, Seattle Dept. of Electrical Engineering June 21, 2011 Lin and Bilmes Submodular Word Alignment June
A Support Vector Method for Multivariate Performance Measures
A Support Vector Method for Multivariate Performance Measures Thorsten Joachims Cornell University Department of Computer Science Thanks to Rich Caruana, Alexandru Niculescu-Mizil, Pierre Dupont, Jérôme
CS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 5: Multi-class and preference learning Juho Rousu 11. October, 2017 Juho Rousu 11. October, 2017 1 / 37 Agenda from now on: This week s theme: going
Discriminative Training. March 4, 2014
Discriminative Training March 4, 2014 Noisy Channels Again p(e) source English Noisy Channels Again p(e) p(g e) source English German Noisy Channels Again p(e) p(g e) source English German decoder e =
Support Vector Machines. CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington
Support Vector Machines CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 A Linearly Separable Problem Consider the binary classification
Low-Dimensional Discriminative Reranking. Jagadeesh Jagarlamudi and Hal Daume III University of Maryland, College Park
Low-Dimensional Discriminative Reranking Jagadeesh Jagarlamudi and Hal Daume III University of Maryland, College Park Discriminative Reranking Useful for many NLP tasks Enables us to use arbitrary features
A Bayesian Partitioning Approach to Duplicate Detection and Record Linkage
A Bayesian Partitioning Approach to Duplicate Detection and Record Linkage Mauricio Sadinle msadinle@stat.duke.edu Duke University Supported by NSF grants SES-11-30706 to Carnegie Mellon University and
Joint Emotion Analysis via Multi-task Gaussian Processes
Joint Emotion Analysis via Multi-task Gaussian Processes Daniel Beck, Trevor Cohn, Lucia Specia October 28, 2014 1 Introduction 2 Multi-task Gaussian Process Regression 3 Experiments and Discussion 4 Conclusions
Natural Language Processing
Natural Language Processing Word vectors Many slides borrowed from Richard Socher and Chris Manning Lecture plan Word representations Word vectors (embeddings) skip-gram algorithm Relation to matrix factorization
Modeling Biological Processes for Reading Comprehension
Modeling Biological Processes for Reading Comprehension Vivek Srikumar University of Utah (Previously, Stanford University) Jonathan Berant, Pei- Chun Chen, Abby Vander Linden, BriEany Harding, Brad Huang,
Statistical Models of the Annotation Process
Bob Carpenter 1 Massimo Poesio 2 1 Alias-I 2 Università di Trento LREC 2010 Tutorial 17th May 2010 Many slides due to Ron Artstein Annotated corpora Annotated corpora are needed for: Supervised learning
Optical Character Recognition of Jutakshars within Devanagari Script
Optical Character Recognition of Jutakshars within Devanagari Script Sheallika Singh Shreesh Ladha Supervised by : Dr. Harish Karnick, Dr. Amit Mitra UGP Presentation, 10 April 2016 OCR of Jutakshars within
Annotating Spatial Containment Relations Between Events. Kirk Roberts, Travis Goodwin, and Sanda Harabagiu
Annotating Spatial Containment Relations Between Events Kirk Roberts, Travis Goodwin, and Sanda Harabagiu Problem Previous Work Schema Corpus Future Work Problem Natural language documents contain a large
Tuning as Linear Regression
Tuning as Linear Regression Marzieh Bazrafshan, Tagyoung Chung and Daniel Gildea Department of Computer Science University of Rochester Rochester, NY 14627 Abstract We propose a tuning method for statistical
Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287
Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287 Review: Neural Networks One-layer multi-layer perceptron architecture, NN MLP1 (x) = g(xw 1 + b 1 )W 2 + b 2 xw + b; perceptron x is the
Machine Translation Evaluation
Machine Translation Evaluation Sara Stymne 2017-03-29 Partly based on Philipp Koehn s slides for chapter 8 Why Evaluation? How good is a given machine translation system? Which one is the best system for
Learning Features from Co-occurrences: A Theoretical Analysis
Learning Features from Co-occurrences: A Theoretical Analysis Yanpeng Li IBM T. J. Watson Research Center Yorktown Heights, New York 10598 liyanpeng.lyp@gmail.com Abstract Representing a word by its co-occurrences
Multiclass Classification-1
CS 446 Machine Learning Fall 2016 Oct 27, 2016 Multiclass Classification Professor: Dan Roth Scribe: C. Cheng Overview Binary to multiclass Multiclass SVM Constraint classification 1 Introduction Multiclass
Machine Learning and Data Mining. Support Vector Machines. Kalev Kask
Machine Learning and Data Mining Support Vector Machines Kalev Kask Linear classifiers Which decision boundary is better? Both have zero training error (perfect training accuracy) But, one of them seems
Quasi-Synchronous Phrase Dependency Grammars for Machine Translation. lti
Quasi-Synchronous Phrase Dependency Grammars for Machine Translation Kevin Gimpel Noah A. Smith 1 Introduction MT using dependency grammars on phrases Phrases capture local reordering and idiomatic translations
Driving Semantic Parsing from the World s Response
Driving Semantic Parsing from the World s Response James Clarke, Dan Goldwasser, Ming-Wei Chang, Dan Roth Cognitive Computation Group University of Illinois at Urbana-Champaign CoNLL 2010 Clarke, Goldwasser,
Deep Learning for Natural Language Processing. Sidharth Mudgal April 4, 2017
Deep Learning for Natural Language Processing Sidharth Mudgal April 4, 2017 Table of contents 1. Intro 2. Word Vectors 3. Word2Vec 4. Char Level Word Embeddings 5. Application: Entity Matching 6. Conclusion
Machine Learning. Lecture 6: Support Vector Machine. Feng Li.
Machine Learning Lecture 6: Support Vector Machine Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Warm Up 2 / 80 Warm Up (Contd.)
TnT Part of Speech Tagger
TnT Part of Speech Tagger By Thorsten Brants Presented By Arghya Roy Chaudhuri Kevin Patel Satyam July 29, 2014 1 / 31 Outline 1 Why Then? Why Now? 2 Underlying Model Other technicalities 3 Evaluation
Applied Natural Language Processing
Applied Natural Language Processing Info 256 Lecture 7: Testing (Feb 12, 2019) David Bamman, UC Berkeley Significance in NLP You develop a new method for text classification; is it better than what comes
Information Extraction from Text
Information Extraction from Text Jing Jiang Chapter 2 from Mining Text Data (2012) Presented by Andrew Landgraf, September 13, 2013 1 What is Information Extraction? Goal is to discover structured information
Prenominal Modifier Ordering via MSA. Alignment
Introduction Prenominal Modifier Ordering via Multiple Sequence Alignment Aaron Dunlop Margaret Mitchell 2 Brian Roark Oregon Health & Science University Portland, OR 2 University of Aberdeen Aberdeen,
Algorithms for NLP. Classification II. Taylor Berg-Kirkpatrick CMU Slides: Dan Klein UC Berkeley
Algorithms for NLP Classification II Taylor Berg-Kirkpatrick CMU Slides: Dan Klein UC Berkeley Minimize Training Error? A loss function declares how costly each mistake is E.g. 0 loss for correct label,
Performance Comparison of K-Means and Expectation Maximization with Gaussian Mixture Models for Clustering EE6540 Final Project
Performance Comparison of K-Means and Expectation Maximization with Gaussian Mixture Models for Clustering EE6540 Final Project Devin Cornell & Sushruth Sastry May 2015 1 Abstract In this article, we explore
Multiword Expression Identification with Tree Substitution Grammars
Multiword Expression Identification with Tree Substitution Grammars Spence Green, Marie-Catherine de Marneffe, John Bauer, and Christopher D. Manning Stanford University EMNLP 2011 Main Idea Use syntactic
Fast Logistic Regression for Text Categorization with Variable-Length N-grams
Fast Logistic Regression for Text Categorization with Variable-Length N-grams Georgiana Ifrim *, Gökhan Bakır +, Gerhard Weikum * * Max-Planck Institute for Informatics Saarbrücken, Germany + Google Switzerland
Joint Inference for Event Timeline Construction
Joint Inference for Event Timeline Construction Quang Xuan Do Wei Lu Dan Roth Department of Computer Science University of Illinois at Urbana-Champaign Urbana, IL 61801, USA {quangdo2,luwei,danr}@illinois.edu
Lecture 5 Neural models for NLP
CS546: Machine Learning in NLP (Spring 2018) http://courses.engr.illinois.edu/cs546/ Lecture 5 Neural models for NLP Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Office hours: Tue/Thu 2pm-3pm
ML4NLP Multiclass Classification
ML4NLP Multiclass Classification CS 590NLP Dan Goldwasser Purdue University dgoldwas@purdue.edu Social NLP Last week we discussed the speed-dates paper. Interesting perspective on NLP problems- Can we
Generating Sentences by Editing Prototypes
Generating Sentences by Editing Prototypes K. Guu 2, T.B. Hashimoto 1,2, Y. Oren 1, P. Liang 1,2 1 Department of Computer Science Stanford University 2 Department of Statistics Stanford University arxiv
Learning a Degree-Augmented Distance Metric from a Network. Bert Huang, U of Maryland Blake Shaw, Foursquare Tony Jebara, Columbia U
Learning a Degree-Augmented Distance Metric from a Network Bert Huang, U of Maryland Blake Shaw, Foursquare Tony Jebara, Columbia U Beyond Mahalanobis: Supervised Large-Scale Learning of Similarity NIPS
Support Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2016 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
Natural Language Processing. Slides from Andreas Vlachos, Chris Manning, Mihai Surdeanu
Natural Language Processing Slides from Andreas Vlachos, Chris Manning, Mihai Surdeanu סקר הערכת הוראה Fill it! Presentations next week 21 projects 20/6: 12 presentations 27/6: 9 presentations 10 minutes
Capturing Salience with a Trainable Cache Model for Zero-anaphora Resolution
Capturing Salience with a Trainable Cache Model for Zero-anaphora Resolution Ryu Iida Department of Computer Science Tokyo Institute of Technology 2-12-1, Ôokayama, Meguro, Tokyo 152-8552, Japan ryu-i@cl.cs.titech.ac.jp
Data Intensive Computing Handout 11 Spark: Transitive Closure
Data Intensive Computing Handout 11 Spark: Transitive Closure from random import Random spark/transitive_closure.py num edges = 3000 num vertices = 500 rand = Random(42) def generate g raph ( ) : edges
Pattern Recognition and Machine Learning. Learning and Evaluation of Pattern Recognition Processes
Pattern Recognition and Machine Learning James L. Crowley ENSIMAG 3 - MMIS Fall Semester 2016 Lesson 1 5 October 2016 Learning and Evaluation of Pattern Recognition Processes Outline Notation...2 1. The
Unsupervised Rank Aggregation with Distance-Based Models
Unsupervised Rank Aggregation with Distance-Based Models Kevin Small Tufts University Collaborators: Alex Klementiev (Johns Hopkins University) Ivan Titov (Saarland University) Dan Roth (University of
Mining coreference relations between formulas and text using Wikipedia
Mining coreference relations between formulas and text using Wikipedia Minh Nghiem Quoc 1, Keisuke Yokoi 2, Yuichiroh Matsubayashi 3 Akiko Aizawa 1 2 3 1 Department of Informatics, The Graduate University
Online Videos FERPA. Sign waiver or sit on the sides or in the back. Off camera question time before and after lecture. Questions?
Online Videos FERPA Sign waiver or sit on the sides or in the back Off camera question time before and after lecture Questions? Lecture 1, Slide 1 CS224d Deep NLP Lecture 4: Word Window Classification
Machine Learning Linear Models
Machine Learning Linear Models Outline II - Linear Models 1. Linear Regression (a) Linear regression: History (b) Linear regression with Least Squares (c) Matrix representation and Normal Equation Method
What is Optimization? Optimization is the process of minimizing or maximizing a function subject to several constraints on its variables
Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø What is Optimization? Optimization is the process of minimizing or maximizing a function subject to several constraints on its variables Optimization In our every day
Sound Recognition in Mixtures
Sound Recognition in Mixtures Juhan Nam, Gautham J. Mysore 2, and Paris Smaragdis 2,3 Center for Computer Research in Music and Acoustics, Stanford University, 2 Advanced Technology Labs, Adobe Systems
Large Scale Semi-supervised Linear SVM with Stochastic Gradient Descent
Journal of Computational Information Systems 9: 15 (2013) 6251 6258 Available at http://www.jofcis.com Large Scale Semi-supervised Linear SVM with Stochastic Gradient Descent Xin ZHOU, Conghui ZHU, Sheng
On a New Model for Automatic Text Categorization Based on Vector Space Model
On a New Model for Automatic Text Categorization Based on Vector Space Model Makoto Suzuki, Naohide Yamagishi, Takashi Ishida, Masayuki Goto and Shigeichi Hirasawa Faculty of Information Science, Shonan
Spectral Unsupervised Parsing with Additive Tree Metrics
Spectral Unsupervised Parsing with Additive Tree Metrics Ankur Parikh, Shay Cohen, Eric P. Xing Carnegie Mellon, University of Edinburgh Ankur Parikh 2014 1 Overview Model: We present a novel approach
Real Estate Price Prediction with Regression and Classification CS 229 Autumn 2016 Project Final Report
Real Estate Price Prediction with Regression and Classification CS 229 Autumn 2016 Project Final Report Hujia Yu, Jiafu Wu [hujiay, jiafuwu]@stanford.edu 1. Introduction Housing prices are an important
Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Linear classifier Which classifier? x 2 x 1 2 Linear classifier Margin concept x 2
SVMs: Non-Separable Data, Convex Surrogate Loss, Multi-Class Classification, Kernels
SVMs: Non-Separable Data, Convex Surrogate Loss, Multi-Class Classification, Kernels Karl Stratos June 21, 2018 1 / 33 Tangent: Some Loose Ends in Logistic Regression Polynomial feature expansion in logistic
Observing Dark Worlds (Final Report)
Observing Dark Worlds (Final Report) Bingrui Joel Li (0009) Abstract Dark matter is hypothesized to account for a large proportion of the universe s total mass. It does not emit or absorb light, making
FINAL EXAM: FALL 2013 CS 6375 INSTRUCTOR: VIBHAV GOGATE
FINAL EXAM: FALL 2013 CS 6375 INSTRUCTOR: VIBHAV GOGATE You are allowed a two-page cheat sheet. You are also allowed to use a calculator. Answer the questions in the spaces provided on the question sheets.
Extracting emotion from twitter
Extracting emotion from twitter John S. Lewin Alexis Pribula INTRODUCTION Problem Being Investigate Our projects consider tweets or status updates of www.twitter.com users. Based on the information contained
Cross-Lingual Language Modeling for Automatic Speech Recogntion
GBO Presentation Cross-Lingual Language Modeling for Automatic Speech Recogntion November 14, 2003 Woosung Kim woosung@cs.jhu.edu Center for Language and Speech Processing Dept. of Computer Science The
CS4495/6495 Introduction to Computer Vision. 8C-L3 Support Vector Machines
CS4495/6495 Introduction to Computer Vision 8C-L3 Support Vector Machines Discriminative classifiers Discriminative classifiers find a division (surface) in feature space that separates the classes Several
Semantic Role Labeling and Constrained Conditional Models
Semantic Role Labeling and Constrained Conditional Models Mausam Slides by Ming-Wei Chang, Nick Rizzolo, Dan Roth, Dan Jurafsky Page 1 Nice to Meet You 0: 2 ILP & Constraints Conditional Models (CCMs)
Information Extraction and GATE. Valentin Tablan University of Sheffield Department of Computer Science NLP Group
Information Extraction and GATE Valentin Tablan University of Sheffield Department of Computer Science NLP Group Information Extraction Information Extraction (IE) pulls facts and structured information
LING 473: Day 13. START THE RECORDING Evaluation. Lecture 13: Evaluation
LING 473: Day 13 START THE RECORDING Slides sourced from Glenn Slayden, Will Lewis 1 Reminders Project 4 is due today. Writing assignment is due next Tuesday. Project 5 is due a week from today. Course
Overview of Statistical Tools. Statistical Inference. Bayesian Framework. Modeling. Very simple case. Things are usually more complicated
Fall 3 Computer Vision Overview of Statistical Tools Statistical Inference Haibin Ling Observation inference Decision Prior knowledge http://www.dabi.temple.edu/~hbling/teaching/3f_5543/index.html Bayesian
Natural Language Processing. Classification. Features. Some Definitions. Classification. Feature Vectors. Classification I. Dan Klein UC Berkeley
Natural Language Processing Classification Classification I Dan Klein UC Berkeley Classification Automatically make a decision about inputs Example: document category Example: image of digit digit Example:
Chapter 2: Introduction to Propositional Logic
Chapter 2: Introduction to Propositional Logic PART ONE: History and Motivation Origins: Stoic school of philosophy (3rd century B.C.), with the most eminent representative was Chryssipus. Modern Origins:
Augmented Statistical Models for Speech Recognition
Augmented Statistical Models for Speech Recognition Mark Gales & Martin Layton 31 August 2005 Trajectory Models For Speech Processing Workshop Overview Dependency Modelling in Speech Recognition: latent
Conditional Random Fields
Conditional Random Fields Micha Elsner February 14, 2013 2 Sums of logs Issue: computing α forward probabilities can undeflow Normally we d fix this using logs But α requires a sum of probabilities Not
Ad Placement Strategies
Case Study 1: Estimating Click Probabilities Tackling an Unknown Number of Features with Sketching Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox 2014 Emily Fox January
CMPS 6630: Introduction to Computational Biology and Bioinformatics. Structure Comparison
CMPS 6630: Introduction to Computational Biology and Bioinformatics Structure Comparison Protein Structure Comparison Motivation Understand sequence and structure variability Understand Domain architecture
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: August 30, 2018, 14.00 19.00 RESPONSIBLE TEACHER: Niklas Wahlström NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical
Constraint Satisfaction [RN2] Sec [RN3] Sec Outline
![Constraint Satisfaction [RN2] Sec [RN3] Sec Outline](/thumbs/85/93044691.jpg "Constraint Satisfaction [RN2] Sec [RN3] Sec Outline")
Constraint Satisfaction [RN2] Sec 5.1-5.2 [RN3] Sec 6.1-6.3 CS 486/686 Lecture 4: Jan 12, 2012 University of Waterloo 1 Outline What are CSPs? Standard search and CSPs Improvements Backtracking Backtracking
Beyond the Point Cloud: From Transductive to Semi-Supervised Learning
Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Vikas Sindhwani, Partha Niyogi, Mikhail Belkin Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of