Calculating Semantic Relatedness with GermaNet

Size: px
Start display at page:

Download "Calculating Semantic Relatedness with GermaNet"

Transcription

1 Organismus, Lebewesen organism, being Katze cat... Haustier pet Hund dog Baum tree Calculating Semantic Relatedness with GermaNet Verena Henrich, Düsseldorf, 19. Februar 2015

2 Semantic Relatedness Measures Overview Path-based measures - Leacock and Chodorow (1998) - Wu and Palmer (1994) - Hirst and St-Onge (1998) - Path Information-content-based measures - Resnik (1995) - Jiang and Conrath (1997) - Lin (1998) Gloss-based measures: several variants of Lesk (1986) - Glosses from GermaNet and/or Wiktionary - Lexical fields from GermaNet and/or Wiktionary

3 Semantic Relatedness Measures Terminology Relatedness/similarity is calculated between the two semantic concepts (synset nodes) s1 and s2 GNROOT B A GermaNet s root node C D Haustier pet LCS(s1, s2) = lowest common subsumer of s1 and s2 synset s1 Katze cat Hund dog synset s2

4 Semantic Relatedness Measures Terminology GNROOT B A D LCS Haustier pet C depth(s2) = 6 length of the shortest path from synset s2 to GNROOT s1 Katze cat s2 Hund dog

5 Semantic Relatedness Measures Terminology GNROOT B A C D LCS Haustier pet length(s1, s2) = 2 shortest path between s1 and s2 s1 Katze cat s2 Hund dog

6 Path-based Relatedness Measures GNROOT A C B Similarity by Wu and Palmer (1994): sim wup (s1, s2) = 2 depth(lcs(s1, s2)) depth(s1) + depth(s2) s1 Katze cat D LCS Haustier pet s2 Hund dog sim wup (cat, dog) = 0.83 where depth(lcs) = 5 depth(cat) = 6 depth(dog) = 6

7 Path-based Relatedness Measures GNROOT A s1 Katze cat B LCS Organismus, Lebewesen organism, being D Haustier pet Hund dog E F s2 Baum tree sim wup (cat, tree) = 0.5 where depth(lcs) = 3 depth(cat) = 6 depth(tree) = 6

8 Path-based Relatedness Measures Similarity by Leacock and Chodorow (1998): - The negative logarithm of the length of the shortest path between s1 and s2 (limited to hypernymy/hyponymy) over the path length of the overall depth of GermaNet sim lch (s1, s2) = -log where length(s1, s2) 2 MAXDEPTH MAXDEPTH = maximal distance of all synsets s from GNROOT

9 Path-based Relatedness Measures Semantic relatedness by Hirst and St-Onge (1998): - The length of the shortest path between the concepts (not limited to hypernymy/hyponymy) and the change of direction (i.e., the relations in GermaNet can be grouped into upwards, downwards, and horizontal) are considered - Not bound to nouns sim hso (s1, s2) = C - length(s1, s2) - k d where C = maximum value for medium strength relations (preset to 10) k = factor for the number of direction changes (preset to 1) d = the number of changes of direction in the shortest path

10 Path-based Relatedness Measures Path relatedness: - A function of the distance between the two synsets s1 and s2 and the longest possible shortest path between any two nodes in GermaNet sim path (s1, s2) = MAXSHORTESTPATH - length(s1, s2) MAXSHORTESTPATH where MAXSHORTESTPATH = maximum of all shortest paths in GermaNet

11 Information Content (IC) cumfreq(s) = cumulated frequency of synset s (freq. of s and cumfreqs of all hyponyms) GNROOT Frequencies from TüPP-D/Z, a German newspaper corpus of B 200 Mio. Words Organismus, Lebewesen organism, being A cumfreq(gnroot) = 145,959,901 cumfreq(being) = 7,230,927 D E cumfreq(pet) = 26,441 Katze cat Haustier pet cumfreq(cat) = 7,012 15,648 Hund dog F Baum tree... cumfreq(tree) = 13,353

12 Information Content (IC) Concept probabilities (relative frequency): p(s) = w W(s) where W(s) = the set of words w in synset s cumfreq(w) cumfreq(gnroot) GNROOT B A Organismus, Lebewesen organism, being p(gnroot) = 1 p(being) = 0.05 D E p(pet) = Katze cat Haustier pet Hund dog F Baum tree... p(tree) =

13 Information Content (IC) IC(s) = -log( p(s) ) The more specific a concept, the higher its IC GNROOT B A IC(GNROOT) = 0 Organismus, Lebewesen organism, being IC(being) = -log(0.05) = 4.3 D E IC(pet) = 12.3 Katze cat Haustier pet Hund dog F Baum tree... IC(tree) = 13.4

14 Information-content-based Relatedness Measures Similarity by Resnik (1995): sim res (s1, s2) = IC(LCS(s1, s2) sim res (cat, dog) = 12.3 GNROOT B A where LCS(cat, dog) = pet Organismus, Lebewesen organism, being IC(pet) = 12.3 s1 Katze cat D LCS Haustier pet s2 Hund dog E F Baum tree...

15 Information-content-based Relatedness Measures Similarity by Resnik (1995): sim res (s1, s2) = IC(LCS(s1, s2) GNROOT A sim res (cat, tree) = 4.3 where LCS(cat, tree) = being B LCS Organismus, Lebewesen organism, being IC(being) = -log(0.05) = 4.3 D E s1 Katze cat Haustier pet Hund dog F s2 Baum tree...

16 Information-content-based Relatedness Measures Similarity by Jiang and Conrath (1997): - The inverse of the distance between the two concepts (measured as the sum of the ICs of the concepts minus double the IC of their LCS) sim jcn (s1, s2) = IC(s1) + IC(s2) - 2 IC(LCS(s1, s2)) Similarity by Lin (1998): - The IC of the concept s LCS (multiplied by two) over the sum of the concept s ICs sim lin (s1, s2) = 2 IC(LCS(s1, s2)) IC(s1) + IC(s2)

17 Gloss-Based Measures: Variants of Lesk Algorithm The Lesk algorithm (Lesk 1986) is one of the first algorithms for the semantic disambiguation of all words Required resource: a dictionary with definitions for each possible word sense Idea: disambiguate words by computing the word overlap among their sense definitions, i.e., word sense whose definition has maximum overlap with the definitions of each of the context words is chosen

18 Lesk Algorithm: Pseudocode for disambiguating two words for each sense i of word W 1 - for each sense j of word W 2 compute overlap(i,j), the number of (nonstop) words in common between the definition of sense i and sense j find i and j for which overlap(i,j) is maximized assign sense i to W 1 and j to W 2

19 Lesk Algorithm Example Example context:...pine cone... 2 word senses of pine #1 kinds of evergreen tree with needle-shaped leaves #2 waste away through sorrow or illness 3 word senses of cone #1 solid body which narrows to a point #2 something of this shape whether solid or hollow #3 fruit of a certain evergreen tree Sense inventory contains 2 word senses for pine and 3 word senses for cone

20 Lesk Algorithm Example Example context:...pine cone... 2 word senses of pine #1 kinds of evergreen tree with needle-shaped leaves #2 waste away through sorrow or illness 3 word senses of cone #1 solid body which narrows to a point #2 something of this shape whether solid or hollow #3 fruit of a certain evergreen tree overlap(pine#1, cone#1) = overlap(pine#2, cone#1) = overlap(pine#1, cone#2) = overlap(pine#2, cone#2) = overlap(pine#2, cone#3) = 0 overlap(pine#1, cone#3) = {evergreen + tree} = 2 à pine#1 and cone#3 are selected

21 Problem with the Original Lesk Algorithm Problem: Combinatorial explosion of possible word sense combinations when more than two words are considered Example: "I saw a man who is 98 years old and can still walk and tell jokes." - Several possible senses for ambiguous words: see(26), man(11), year(4), old(8), can(5), still(4), walk(10), tell(8), joke(3) - 43,929,600 sense combinations Solution: simplified version of the Lesk algorithm

22 Simplified Lesk Algorithm Tries to solve the combinatorial explosion of word sense combinations by disambiguating each ambiguous word in the input text individually, regardless of the meaning of the other words occurring in the same context Correct meaning of each word is determined by finding the sense that leads to the highest overlap between its dictionary definition and the current context Simplified Lesk can significantly outperform the original algorithm in terms of precision and efficiency

23 Simplified Lesk Algorithm: Pseudocode to disambiguate a target word for each sense i of W - determine overlap(i,c), the number of (non-stop) words in common between the definition of sense i and current sentential context c find sense i for which overlap(i,c) is maximized assign sense i to W

24 Simplified Lesk Algorithm Example 2 word senses of bank #1 sloping land (especially the slope beside a body of water) #2 a financial institution that accepts deposits and channels the money into lending activities Target word: bank Example context: The bank profits from the difference between the level of interest it pays for deposits, and the level of interest it charges in its lending activities.

25 Simplified Lesk Algorithm Example 2 word senses of bank #1 sloping land (especially the slope beside a body of water) #2 a financial institution that accepts deposits and channels the money into lending activities overlap(bank#1, context) = 0 overlap(bank#2, context) = {deposits + lending + activities} = 3 à bank#2 is selected Example context: The bank profits from the difference between the level of interest it pays for deposits, and the level of interest it charges in its lending activities.

26 Simplified Lesk Algorithm for GermaNet Problem: GermaNet does not contain comprehensive sense definitions for all word senses Solution: - Represent a word sense by a bag of words (BOW) that includes all synonyms and related words - Calculate overlap by counting the words in common between the BOW of a word sense and the current sentential context BOW: can contain all words that are linked to the word sense in question, e.g., all synonyms, antonyms, hypernyms, hyponyms, meronyms, holonyms, etc.

27 Simplified Lesk Algorithm for GermaNet Example Example context: Der Verlag hatte sein eigenes Kapital in eine feste Geldanlage umgewandelt, während die Kohle anderer Unterstützer für laufende Kosten verwendet wurde. 2 word senses of Kohle in GermaNet ( money and coal ) #1 synonyms: Geld, Kies, Knete, Moneten; hypernyms: Zahlungsmittel, Finanzmittel; holonyms: Geldvermögen, Geldmittel, Mittel, Kapital, Finanzausstattung, Geldkapital, Geldanlage, Anlage, Asset #2 synonyms: Kohlegestein; hypernyms: Rückstandsgestein; hyponyms: Importkohle, Rohkohle, Aktivkohle, Torf, Braunkohle, Steinkohle, Koks, Kokskohle, Anthrazit, Grafit, Glanzkohle, Xylit

28 Simplified Lesk Algorithm for GermaNet Example Example context: Der Verlag hatte sein eigenes Kapital in eine feste Geldanlage umgewandelt, während die Kohle anderer Unterstützer für laufende Kosten verwendet wurde. 2 word senses of Kohle in GermaNet ( money and coal ) #1 synonyms: Geld, Kies, Knete, Moneten; hypernyms: Zahlungsmittel, Finanzmittel; holonyms: Geldvermögen, Geldmittel, Mittel, Kapital, Finanzausstattung, Geldkapital, Geldanlage, Anlage, Asset #2 synonyms: Kohlegestein; hypernyms: Rückstandsgestein; hyponyms: Importkohle, Rohkohle, Aktivkohle, Torf, Braunkohle, Steinkohle, Koks, Kokskohle, Anthrazit, Grafit, Glanzkohle, Xylit

29 Semantic Relatedness References Hirst, G. and D. St-Onge: Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms. In C. Fellbaum (ed.), WordNet: An Electronic Lexical Database, The MIT Press, Jiang, J.J. and D.W. Conrath: Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy. In Proceedings of ROCLING 1997, Taiwan, Leacock, C. and M. Chodorow: Combining Local Context and WordNet Similarity for Word Sense Identification. In C. Fellbaum (ed.), WordNet: An Electronic Lexical Database, The MIT Press, Lesk, M.: Automatic Sense Disambiguation Using Machine Readable Dictionaries: How to Tell a Pine Cone from an Ice Cream Cone. Proceedings of SIGDOC 86, New York, NY, USA, Lin, D.: An Information-Theoretic Definition of Similarity. Proceedings of ICML 98, San Francisco, CA, USA, Resnik, P.: Using Information Content to Evaluate Semantic Similarity in a Taxonomy. Proceedings of the 14th international joint conference on Artificial intelligence - Volume 1, San Francisco, CA, USA, Wu, Z. and M. Palmer: Verb Semantics And Lexical Selection. Proceedings of ACL 94, Stroudsburg, PA, USA, 1994.

MATHEMATICAL AND EXPERIMENTAL INVESTIGATION OF ONTOLOGICAL SIMILARITY MEASURES AND THEIR USE IN BIOMEDICAL DOMAINS

MATHEMATICAL AND EXPERIMENTAL INVESTIGATION OF ONTOLOGICAL SIMILARITY MEASURES AND THEIR USE IN BIOMEDICAL DOMAINS ABSTRACT MATHEMATICAL AND EXPERIMENTAL INVESTIGATION OF ONTOLOGICAL SIMILARITY MEASURES AND THEIR USE IN BIOMEDICAL DOMAINS by Xinran Yu Similarity measurement is an important notion. In the context of

More information

Semantic distances & LSA

Semantic distances & LSA Semantic istances & LSA Stefan rausan-matu Politehnica" University of Bucharest an Romanian Acaemy Research Institute for Artificial Intelligence Bucharest, Romania stefan.trausan@cs.pub.ro http://www.racai.ro/~trausan

More information

A Generalized Vector Space Model for Text Retrieval Based on Semantic Relatedness

A Generalized Vector Space Model for Text Retrieval Based on Semantic Relatedness A Generalized Vector Space Model for Text Retrieval Based on Semantic Relatedness George Tsatsaronis and Vicky Panagiotopoulou Department of Informatics Athens University of Economics and Business, 76,

More information

WEST: WEIGHTED-EDGE BASED SIMILARITY MEASUREMENT TOOLS FOR WORD SEMANTICS

WEST: WEIGHTED-EDGE BASED SIMILARITY MEASUREMENT TOOLS FOR WORD SEMANTICS WEST: WEIGHTED-EDGE BASED SIMILARITY MEASUREMENT TOOLS FOR WORD SEMANTICS Liang Dong, Pradip K. Srimani, James Z. Wang School of Computing, Clemson University Web Intelligence 2010, September 1, 2010 Outline

More information

The OntoNL Semantic Relatedness Measure for OWL Ontologies

The OntoNL Semantic Relatedness Measure for OWL Ontologies The OntoNL Semantic Relatedness Measure for OWL Ontologies Anastasia Karanastasi and Stavros hristodoulakis Laboratory of Distributed Multimedia Information Systems and Applications Technical University

More information

Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction

Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction Alexander Panchenko alexander.panchenko@student.uclouvain.be Université catholique de Louvain & Bauman Moscow State

More information

Semantic Similarity and Relatedness

Semantic Similarity and Relatedness Semantic Relatedness Semantic Similarity and Relatedness (Based on Budanitsky, Hirst 2006 and Chapter 20 of Jurafsky/Martin 2 nd. Ed. - Most figures taken from either source.) Many applications require

More information

Toponym Disambiguation using Ontology-based Semantic Similarity

Toponym Disambiguation using Ontology-based Semantic Similarity Toponym Disambiguation using Ontology-based Semantic Similarity David S Batista 1, João D Ferreira 2, Francisco M Couto 2, and Mário J Silva 1 1 IST/INESC-ID Lisbon, Portugal {dsbatista,msilva}@inesc-id.pt

More information

Natural Language Processing SoSe Words and Language Model

Natural Language Processing SoSe Words and Language Model Natural Language Processing SoSe 2016 Words and Language Model Dr. Mariana Neves May 2nd, 2016 Outline 2 Words Language Model Outline 3 Words Language Model Tokenization Separation of words in a sentence

More information

WordNet Lesk Algorithm Preprocessing. WordNet. Marina Sedinkina - Folien von Desislava Zhekova - CIS, LMU.

WordNet Lesk Algorithm Preprocessing. WordNet. Marina Sedinkina - Folien von Desislava Zhekova - CIS, LMU. Marina Sedinkina - Folien von Desislava Zhekova - CIS, LMU marina.sedinkina@campus.lmu.de January 9, 2018 Marina Sedinkina- Folien von Desislava Zhekova - Language Processing and Python 1/67 Outline 1

More information

Francisco M. Couto Mário J. Silva Pedro Coutinho

Francisco M. Couto Mário J. Silva Pedro Coutinho Francisco M. Couto Mário J. Silva Pedro Coutinho DI FCUL TR 03 29 Departamento de Informática Faculdade de Ciências da Universidade de Lisboa Campo Grande, 1749 016 Lisboa Portugal Technical reports are

More information

A Game-Theoretic Approach to Graph Transduction: An Experimental Study

A Game-Theoretic Approach to Graph Transduction: An Experimental Study MSc (ex D.M. 270/2004) in Computer Science Dissertation A Game-Theoretic Approach to Graph Transduction: An Experimental Study Supervisor Prof. Marcello Pelillo Candidate Michele Schiavinato Id 810469

More information

Similarity Measures for Query Expansion in TopX

Similarity Measures for Query Expansion in TopX Universität des Saarlandes Naturwissenschaftlich-Technische Fak. I Fachrichtung 6.2 - Informatik Max-Planck-Institut für Informatik AG 5 - Datenbanken und Informationssysteme Prof. Dr. Gerhard Weikum Diplom-Studiengang

More information

Coarse to Fine Grained Sense Disambiguation in Wikipedia

Coarse to Fine Grained Sense Disambiguation in Wikipedia Coarse to Fine Grained Sense Disambiguation in Wikipedia Hui Shen [Ohio University] Razvan Bunescu [Ohio University] Rada Mihalcea [University of North Texas] *SEM, June 13, 2013 Word Senses A word sense

More information

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University Text Mining Dr. Yanjun Li Associate Professor Department of Computer and Information Sciences Fordham University Outline Introduction: Data Mining Part One: Text Mining Part Two: Preprocessing Text Data

More information

IBM Model 1 for Machine Translation

IBM Model 1 for Machine Translation IBM Model 1 for Machine Translation Micha Elsner March 28, 2014 2 Machine translation A key area of computational linguistics Bar-Hillel points out that human-like translation requires understanding of

More information

Exploiting WordNet as Background Knowledge

Exploiting WordNet as Background Knowledge Exploiting WordNet as Background Knowledge Chantal Reynaud, Brigitte Safar LRI, Université Paris-Sud, Bât. G, INRIA Futurs Parc Club Orsay-Université - 2-4 rue Jacques Monod, F-91893 Orsay, France {chantal.reynaud,

More information

Natural Language Processing SoSe Language Modelling. (based on the slides of Dr. Saeedeh Momtazi)

Natural Language Processing SoSe Language Modelling. (based on the slides of Dr. Saeedeh Momtazi) Natural Language Processing SoSe 2015 Language Modelling Dr. Mariana Neves April 20th, 2015 (based on the slides of Dr. Saeedeh Momtazi) Outline 2 Motivation Estimation Evaluation Smoothing Outline 3 Motivation

More information

Automated Slogan Production Using a Genetic Algorithm

Automated Slogan Production Using a Genetic Algorithm Automated Slogan Production Using a Genetic Algorithm Polona Tomašič* XLAB d.o.o. Pot za Brdom 100 1000 Ljubljana, Slovenia (polona.tomasic@gmail.com) Gregor Papa* Jožef Stefan Institute Jamova cesta 39

More information

Discriminative Training

Discriminative Training Discriminative Training February 19, 2013 Noisy Channels Again p(e) source English Noisy Channels Again p(e) p(g e) source English German Noisy Channels Again p(e) p(g e) source English German decoder

More information

OSS: A Semantic Similarity Function based on Hierarchical Ontologies

OSS: A Semantic Similarity Function based on Hierarchical Ontologies OSS: A Semantic Similarity Function based on Hierarchical Ontologies Vincent Schickel-Zuber and Boi Faltings Swiss Federal Institute of Technology - EPFL Artificial Intelligence Laboratory vincent.schickel-zuber@epfl.ch,

More information

2 GENE FUNCTIONAL SIMILARITY. 2.1 Semantic values of GO terms

2 GENE FUNCTIONAL SIMILARITY. 2.1 Semantic values of GO terms Bioinformatics Advance Access published March 7, 2007 The Author (2007). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org

More information

Natural Language Processing

Natural Language Processing Natural Language Processing Word vectors Many slides borrowed from Richard Socher and Chris Manning Lecture plan Word representations Word vectors (embeddings) skip-gram algorithm Relation to matrix factorization

More information

Syntax versus Semantics:

Syntax versus Semantics: Syntax versus Semantics: Analysis of Enriched Vector Space Models Benno Stein and Sven Meyer zu Eissen and Martin Potthast Bauhaus University Weimar Relevance Computation Information retrieval aims at

More information

A Context-Free Grammar

A Context-Free Grammar Statistical Parsing A Context-Free Grammar S VP VP Vi VP Vt VP VP PP DT NN PP PP P Vi sleeps Vt saw NN man NN dog NN telescope DT the IN with IN in Ambiguity A sentence of reasonable length can easily

More information

Methodology for geospatial data source discovery in ontologydriven geo-information integration architectures

Methodology for geospatial data source discovery in ontologydriven geo-information integration architectures Manuscript Click here to view linked References Methodology for geospatial data source discovery in ontologydriven geo-information integration architectures 1. Introduction Because of the advances in information,

More information

Word Sense Disambiguation (WSD) Based on Foundations of Statistical NLP by C. Manning & H. Schütze, ch. 7 MIT Press, 2002

Word Sense Disambiguation (WSD) Based on Foundations of Statistical NLP by C. Manning & H. Schütze, ch. 7 MIT Press, 2002 0. Word Sense Disambiguation (WSD) Based on Foundations of Statistical NLP by C. Manning & H. Schütze, ch. 7 MIT Press, 2002 WSD Examples 1. They have the right to bear arms. (drept) The sign on the right

More information

CS 662 Sample Midterm

CS 662 Sample Midterm Name: 1 True/False, plus corrections CS 662 Sample Midterm 35 pts, 5 pts each Each of the following statements is either true or false. If it is true, mark it true. If it is false, correct the statement

More information

Formal Semantics Of Verbs For Knowledge Inference

Formal Semantics Of Verbs For Knowledge Inference Formal Semantics Of Verbs For Knowledge Inference Igor Boyko, Ph.D. Logical Properties Inc., Montreal, Canada igor_m_boyko@hotmail.com Abstract This short paper is focused on the formal semantic model:

More information

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Language Models & Hidden Markov Models

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Language Models & Hidden Markov Models 1 University of Oslo : Department of Informatics INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Language Models & Hidden Markov Models Stephan Oepen & Erik Velldal Language

More information

Latent Semantic Analysis. Hongning Wang

Latent Semantic Analysis. Hongning Wang Latent Semantic Analysis Hongning Wang CS@UVa Recap: vector space model Represent both doc and query by concept vectors Each concept defines one dimension K concepts define a high-dimensional space Element

More information

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Hidden Markov Models

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Hidden Markov Models INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Hidden Markov Models Murhaf Fares & Stephan Oepen Language Technology Group (LTG) October 27, 2016 Recap: Probabilistic Language

More information

LSI vs. Wordnet Ontology in Dimension Reduction for Information Retrieval

LSI vs. Wordnet Ontology in Dimension Reduction for Information Retrieval LSI vs. Wordnet Ontology in Dimension Reduction for Information Retrieval Pavel Moravec, Michal Kolovrat, and Václav Snášel Department of Computer Science, FEI, VŠB - Technical University of Ostrava, 17.

More information

Data Compression Techniques

Data Compression Techniques Data Compression Techniques Part 2: Text Compression Lecture 5: Context-Based Compression Juha Kärkkäinen 14.11.2017 1 / 19 Text Compression We will now look at techniques for text compression. These techniques

More information

Algorithmic Computation and Approximation of Semantic Similarity

Algorithmic Computation and Approximation of Semantic Similarity Algorithmic Computation and Approximation of Semantic Similarity Ana G. Maguitman, Filippo Menczer, Fulya Erdinc, Heather Roinestad, Alessandro Vespignani Department of Computer Science School of Informatics

More information

Features of Statistical Parsers

Features of Statistical Parsers Features of tatistical Parsers Preliminary results Mark Johnson Brown University TTI, October 2003 Joint work with Michael Collins (MIT) upported by NF grants LI 9720368 and II0095940 1 Talk outline tatistical

More information

Latent Semantic Analysis. Hongning Wang

Latent Semantic Analysis. Hongning Wang Latent Semantic Analysis Hongning Wang CS@UVa VS model in practice Document and query are represented by term vectors Terms are not necessarily orthogonal to each other Synonymy: car v.s. automobile Polysemy:

More information

PORT-BASED ONTOLOGY SEMANTIC SIMILARITIES FOR MODULE CONCEPT CREATION. 1 Copyright 2009 by ASME

PORT-BASED ONTOLOGY SEMANTIC SIMILARITIES FOR MODULE CONCEPT CREATION. 1 Copyright 2009 by ASME Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2009 August 30 - September 2, 2009, San Diego, California,

More information

LECTURER: BURCU CAN Spring

LECTURER: BURCU CAN Spring LECTURER: BURCU CAN 2017-2018 Spring Regular Language Hidden Markov Model (HMM) Context Free Language Context Sensitive Language Probabilistic Context Free Grammar (PCFG) Unrestricted Language PCFGs can

More information

Structured Output Prediction: Generative Models

Structured Output Prediction: Generative Models Structured Output Prediction: Generative Models CS6780 Advanced Machine Learning Spring 2015 Thorsten Joachims Cornell University Reading: Murphy 17.3, 17.4, 17.5.1 Structured Output Prediction Supervised

More information

Exercise 1: Basics of probability calculus

Exercise 1: Basics of probability calculus : Basics of probability calculus Stig-Arne Grönroos Department of Signal Processing and Acoustics Aalto University, School of Electrical Engineering stig-arne.gronroos@aalto.fi [21.01.2016] Ex 1.1: Conditional

More information

Lecture 12: Algorithms for HMMs

Lecture 12: Algorithms for HMMs Lecture 12: Algorithms for HMMs Nathan Schneider (some slides from Sharon Goldwater; thanks to Jonathan May for bug fixes) ENLP 17 October 2016 updated 9 September 2017 Recap: tagging POS tagging is a

More information

Recognizing Implicit Discourse Relations through Abductive Reasoning with Large-scale Lexical Knowledge

Recognizing Implicit Discourse Relations through Abductive Reasoning with Large-scale Lexical Knowledge Recognizing Implicit Discourse Relations through Abductive Reasoning with Large-scale Lexical Knowledge Jun Sugiura, Naoya Inoue, and Kentaro Inui Tohoku University, 6-3-09 Aoba, Aramaki, Aoba-ku, Sendai,

More information

III.6 Advanced Query Types

III.6 Advanced Query Types III.6 Advanced Query Types 1. Query Expansion 2. Relevance Feedback 3. Novelty & Diversity Based on MRS Chapter 9, BY Chapter 5, [Carbonell and Goldstein 98] [Agrawal et al 09] 123 1. Query Expansion Query

More information

Collapsed Variational Bayesian Inference for Hidden Markov Models

Collapsed Variational Bayesian Inference for Hidden Markov Models Collapsed Variational Bayesian Inference for Hidden Markov Models Pengyu Wang, Phil Blunsom Department of Computer Science, University of Oxford International Conference on Artificial Intelligence and

More information

Connotation: A Dash of Sentiment Beneath the Surface Meaning

Connotation: A Dash of Sentiment Beneath the Surface Meaning Connotation: A Dash of Sentiment Beneath the Surface Meaning Con notation com- ( together or with ) notare ( to mark ) 1 Con notation com- ( together or with ) notare ( to mark ) Commonly understood cultural

More information

John Pavlopoulos and Ion Androutsopoulos NLP Group, Department of Informatics Athens University of Economics and Business, Greece

John Pavlopoulos and Ion Androutsopoulos NLP Group, Department of Informatics Athens University of Economics and Business, Greece John Pavlopoulos and Ion Androutsopoulos NLP Group, Department of Informatics Athens University of Economics and Business, Greece http://nlp.cs.aueb.gr/ A laptop with great design, but the service was

More information

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Hidden Markov Models

INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Hidden Markov Models INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Hidden Markov Models Murhaf Fares & Stephan Oepen Language Technology Group (LTG) October 18, 2017 Recap: Probabilistic Language

More information

Lecture 12: Algorithms for HMMs

Lecture 12: Algorithms for HMMs Lecture 12: Algorithms for HMMs Nathan Schneider (some slides from Sharon Goldwater; thanks to Jonathan May for bug fixes) ENLP 26 February 2018 Recap: tagging POS tagging is a sequence labelling task.

More information

Introduction to Probablistic Natural Language Processing

Introduction to Probablistic Natural Language Processing Introduction to Probablistic Natural Language Processing Alexis Nasr Laboratoire d Informatique Fondamentale de Marseille Natural Language Processing Use computers to process human languages Machine Translation

More information

HMM Expanded to Multiple Interleaved Chains as a Model for Word Sense Disambiguation

HMM Expanded to Multiple Interleaved Chains as a Model for Word Sense Disambiguation HMM Expanded to Multiple Interleaved Chains as a Model for Word Sense Disambiguation Denis Turdakov and Dmitry Lizorkin Institute for System Programming of the Russian Academy of Sciences, 25 Solzhenitsina

More information

Maxent Models and Discriminative Estimation

Maxent Models and Discriminative Estimation Maxent Models and Discriminative Estimation Generative vs. Discriminative models (Reading: J+M Ch6) Introduction So far we ve looked at generative models Language models, Naive Bayes But there is now much

More information

A HYBRID SEMANTIC SIMILARITY MEASURING APPROACH FOR ANNOTATING WSDL DOCUMENTS WITH ONTOLOGY CONCEPTS. Received February 2017; revised May 2017

A HYBRID SEMANTIC SIMILARITY MEASURING APPROACH FOR ANNOTATING WSDL DOCUMENTS WITH ONTOLOGY CONCEPTS. Received February 2017; revised May 2017 International Journal of Innovative Computing, Information and Control ICIC International c 2017 ISSN 1349-4198 Volume 13, Number 4, August 2017 pp. 1221 1242 A HYBRID SEMANTIC SIMILARITY MEASURING APPROACH

More information

Discriminative Training. March 4, 2014

Discriminative Training. March 4, 2014 Discriminative Training March 4, 2014 Noisy Channels Again p(e) source English Noisy Channels Again p(e) p(g e) source English German Noisy Channels Again p(e) p(g e) source English German decoder e =

More information

FROM QUERIES TO TOP-K RESULTS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS

FROM QUERIES TO TOP-K RESULTS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS FROM QUERIES TO TOP-K RESULTS Dr. Gjergji Kasneci Introduction to Information Retrieval WS 2012-13 1 Outline Intro Basics of probability and information theory Retrieval models Retrieval evaluation Link

More information

Synthetic logic characterizations of meanings extracted from large corpora

Synthetic logic characterizations of meanings extracted from large corpora Synthetic logic characterizations of meanings extracted from large corpora Alex Djalali and Christopher Potts Stanford Linguistics Workshop on Semantics for Textual Inference LSA Institute, University

More information

A Tutorial on Learning with Bayesian Networks

A Tutorial on Learning with Bayesian Networks A utorial on Learning with Bayesian Networks David Heckerman Presented by: Krishna V Chengavalli April 21 2003 Outline Introduction Different Approaches Bayesian Networks Learning Probabilities and Structure

More information

Multi-Component Word Sense Disambiguation

Multi-Component Word Sense Disambiguation Multi-Component Word Sense Disambiguation Massimiliano Ciaramita and Mark Johnson Brown University BLLIP: http://www.cog.brown.edu/research/nlp Ciaramita and Johnson 1 Outline Pattern classification for

More information

Extraction of Opposite Sentiments in Classified Free Format Text Reviews

Extraction of Opposite Sentiments in Classified Free Format Text Reviews Extraction of Opposite Sentiments in Classified Free Format Text Reviews Dong (Haoyuan) Li 1, Anne Laurent 2, Mathieu Roche 2, and Pascal Poncelet 1 1 LGI2P - École des Mines d Alès, Parc Scientifique

More information

Cross-Lingual Language Modeling for Automatic Speech Recogntion

Cross-Lingual Language Modeling for Automatic Speech Recogntion GBO Presentation Cross-Lingual Language Modeling for Automatic Speech Recogntion November 14, 2003 Woosung Kim woosung@cs.jhu.edu Center for Language and Speech Processing Dept. of Computer Science The

More information

Chapter 4. Introduction to Mathematical Modeling. Types of Modeling. 1) Linear Modeling 2) Quadratic Modeling 3) Exponential Modeling

Chapter 4. Introduction to Mathematical Modeling. Types of Modeling. 1) Linear Modeling 2) Quadratic Modeling 3) Exponential Modeling Chapter 4 Introduction to Mathematical Modeling Tpes of Modeling 1) Linear Modeling ) Quadratic Modeling ) Eponential Modeling Each tpe of modeling in mathematics is determined b the graph of equation

More information

TEXT SIMILARITY MEASUREMENT WITH SEMANTIC ANALYSIS. Xu Li, Na Liu, Chunlong Yao and Fenglong Fan. Received February 2017; revised June 2017

TEXT SIMILARITY MEASUREMENT WITH SEMANTIC ANALYSIS. Xu Li, Na Liu, Chunlong Yao and Fenglong Fan. Received February 2017; revised June 2017 International Journal of Innovative Computing, Information and Control ICIC International c 2017 ISSN 1349-4198 Volume 13, Number 5, October 2017 pp. 1693 1708 TEXT SIMILARITY MEASUREMENT WITH SEMANTIC

More information

ICDM Pisa, December 16, Motivation

ICDM Pisa, December 16, Motivation Using Wikipedia for Co-clustering Based Cross-domain Text Classification Pu Wang and Carlotta Domeniconi George Mason University Jian Hu Microsoft Research Asia ICDM Pisa, December 16, 2008 Motivation

More information

Parsing with Context-Free Grammars

Parsing with Context-Free Grammars Parsing with Context-Free Grammars Berlin Chen 2005 References: 1. Natural Language Understanding, chapter 3 (3.1~3.4, 3.6) 2. Speech and Language Processing, chapters 9, 10 NLP-Berlin Chen 1 Grammars

More information

Maximum Entropy Markov Models

Maximum Entropy Markov Models Wi nøt trei a høliday in Sweden this yër? September 19th 26 Background Preliminary CRF work. Published in 2. Authors: McCallum, Freitag and Pereira. Key concepts: Maximum entropy / Overlapping features.

More information

Machine Translation. CL1: Jordan Boyd-Graber. University of Maryland. November 11, 2013

Machine Translation. CL1: Jordan Boyd-Graber. University of Maryland. November 11, 2013 Machine Translation CL1: Jordan Boyd-Graber University of Maryland November 11, 2013 Adapted from material by Philipp Koehn CL1: Jordan Boyd-Graber (UMD) Machine Translation November 11, 2013 1 / 48 Roadmap

More information

DT2118 Speech and Speaker Recognition

DT2118 Speech and Speaker Recognition DT2118 Speech and Speaker Recognition Language Modelling Giampiero Salvi KTH/CSC/TMH giampi@kth.se VT 2015 1 / 56 Outline Introduction Formal Language Theory Stochastic Language Models (SLM) N-gram Language

More information

Feature Engineering. Knowledge Discovery and Data Mining 1. Roman Kern. ISDS, TU Graz

Feature Engineering. Knowledge Discovery and Data Mining 1. Roman Kern. ISDS, TU Graz Feature Engineering Knowledge Discovery and Data Mining 1 Roman Kern ISDS, TU Graz 2017-11-09 Roman Kern (ISDS, TU Graz) Feature Engineering 2017-11-09 1 / 66 Big picture: KDDM Probability Theory Linear

More information

Meaning as Sense (Frege 1892)

Meaning as Sense (Frege 1892) Meaning as Sense (Frege 1892) The meaning of an expression is its sense, an abstract object that determines its reference. Senses are functions that determine reference. 1/ 39 Functions: a Reminder A function

More information

Natural Language Processing with Deep Learning CS224N/Ling284. Christopher Manning Lecture 1: Introduction and Word Vectors

Natural Language Processing with Deep Learning CS224N/Ling284. Christopher Manning Lecture 1: Introduction and Word Vectors Natural Language Processing with Deep Learning CS224N/Ling284 Christopher Manning Lecture 1: Introduction and Word Vectors Lecture Plan Lecture 1: Introduction and Word Vectors 1. The course (10 mins)

More information

The speed the speed of light is 30,000,000,000 m/s. Write this number in scientific notation.

The speed the speed of light is 30,000,000,000 m/s. Write this number in scientific notation. Chapter 1 Section 1.1 Scientific Notation Powers of Ten 1 1 1.1.1.1.1 Standard Scientific Notation N n where 1 N and n is an integers Eamples of numbers in scientific notation. 8.17 11 Using Scientific

More information

Lesson Objectives. Core Content Objectives. Language Arts Objectives

Lesson Objectives. Core Content Objectives. Language Arts Objectives Evergreen Trees 9 Lesson Objectives Core Content Objectives Students will: Explain that evergreen trees are one type of plant that stays green all year and does not become dormant in the winter Compare

More information

An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition

An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition Yu-Seop Kim 1, Jeong-Ho Chang 2, and Byoung-Tak Zhang 2 1 Division of Information and Telecommunication

More information

Unterspezifikation in der Semantik Scope Semantics in Lexicalized Tree Adjoining Grammars

Unterspezifikation in der Semantik Scope Semantics in Lexicalized Tree Adjoining Grammars in der emantik cope emantics in Lexicalized Tree Adjoining Grammars Laura Heinrich-Heine-Universität Düsseldorf Wintersemester 2011/2012 LTAG: The Formalism (1) Tree Adjoining Grammars (TAG): Tree-rewriting

More information

Appendix A. Common Mathematical Operations in Chemistry

Appendix A. Common Mathematical Operations in Chemistry Appendix A Common Mathematical Operations in Chemistry In addition to basic arithmetic and algebra, four mathematical operations are used frequently in general chemistry: manipulating logarithms, using

More information

Learning Word Representations by Jointly Modeling Syntagmatic and Paradigmatic Relations

Learning Word Representations by Jointly Modeling Syntagmatic and Paradigmatic Relations Learning Word Representations by Jointly Modeling Syntagmatic and Paradigmatic Relations Fei Sun, Jiafeng Guo, Yanyan Lan, Jun Xu, and Xueqi Cheng CAS Key Lab of Network Data Science and Technology Institute

More information

arxiv: v3 [cs.ir] 26 Sep 2018

arxiv: v3 [cs.ir] 26 Sep 2018 A Wikipedia-based approach to profiling activities on social media Extended Abstract Christian Torrero, Carlo Caprini and Daniele Miorandi U-Hopper srl Trento, Italy {name.surname}@u-hopper.com arxiv:1804.0224v3

More information

7.5: Conditional Probability and Independent Events

7.5: Conditional Probability and Independent Events c Dr Oksana Shatalov, Spring 2012 1 7.5: Conditional Probability and Independent Events EXAMPLE 1. Two cards are drawn from a deck of 52 without replacement. (a) What is the probability of that the first

More information

Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul

Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul Generative Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 29, 2018 Prof. Michael Paul Generative vs Discriminative The classification algorithms we have seen so far

More information

IM1 Summative Practice #1 Show all your Work

IM1 Summative Practice #1 Show all your Work IM1 Summative Practice #1 Name: Show all your Work Period: Simplify each expression below. 1. 5x (7 3x) 2. 5(x 12) + 15(x + y) 3. 8 2(x 2 3x) (9 3x 2 ) Solve each equation. Justify each step with a property

More information

Section Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence

Section Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence Section 2.4 Section Summary Sequences. Examples: Geometric Progression, Arithmetic Progression Recurrence Relations Example: Fibonacci Sequence Summations Introduction Sequences are ordered lists of elements.

More information

3.3 Linear Equations in Standard Form

3.3 Linear Equations in Standard Form 3.3 Linear Equations in Standard Form Learning Objectives Write equivalent equations in standard form. Find the slope and y intercept from an equation in standard form. Write equations in standard form

More information

3-3 Using Tables and Equations of Lines

3-3 Using Tables and Equations of Lines 3-3 Using Tables and Equations of Lines Objectives Students will understand that linear models are appropriate when the situation has a constant increase/decrease. slope is the rate of change. the rate

More information

The Language of Motion

The Language of Motion The Language of Motion Textbook pages 344 361 Section 8.1 Summary Before You Read What does the term uniform mean to you? If motion is uniform, how does it behave? Write your ideas in the lines below.

More information

Learning Random Walk Models for Inducing Word Dependency Distributions

Learning Random Walk Models for Inducing Word Dependency Distributions Learning Random Walk Models for Inducing Word Dependency Distributions Kristina Toutanova Christopher D. Manning Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA Abstract

More information

Bringing machine learning & compositional semantics together: central concepts

Bringing machine learning & compositional semantics together: central concepts Bringing machine learning & compositional semantics together: central concepts https://githubcom/cgpotts/annualreview-complearning Chris Potts Stanford Linguistics CS 244U: Natural language understanding

More information

Deep Learning. Language Models and Word Embeddings. Christof Monz

Deep Learning. Language Models and Word Embeddings. Christof Monz Deep Learning Today s Class N-gram language modeling Feed-forward neural language model Architecture Final layer computations Word embeddings Continuous bag-of-words model Skip-gram Negative sampling 1

More information

Moreno Mitrović. The Saarland Lectures on Formal Semantics

Moreno Mitrović. The Saarland Lectures on Formal Semantics ,, 3 Moreno Mitrović The Saarland Lectures on Formal Semantics λ- λ- λ- ( λ- ) Before we move onto this, let's recall our f -notation for intransitive verbs 1/33 λ- ( λ- ) Before we move onto this, let's

More information

The Elements for Logic In Compositional Distributional Models of Meaning

The Elements for Logic In Compositional Distributional Models of Meaning The Elements for Logic In Compositional Distributional Models of Meaning Joshua Steves: 1005061 St Peter s College University of Oxford A thesis submitted for the degree of MSc Mathematics and Foundations

More information

Lesson 26 MA Nick Egbert

Lesson 26 MA Nick Egbert Overview There is no new material for this lesson. Here, we explore a multitude of fun word problems that we can now do using the method of Lagrange multipliers from the previous lesson. Examples Example

More information

Textual Entailment as a Directional Relation

Textual Entailment as a Directional Relation Textual Entailment as a Directional Relation Doina Tătar, Gabriela Şerban, Mihiş Andreea Department of Computer Science University Babes-Bolyai, Cluj-Napoca, Romania dtatar@cs.ubbcluj.ro, gabis@cs.ubbcluj.ro,

More information

Ontology-Based Information Extraction on PubMed abstracts using the OMIT ontology to discover inconsistencies

Ontology-Based Information Extraction on PubMed abstracts using the OMIT ontology to discover inconsistencies Ontology-Based Information Extraction on PubMed abstracts using the OMIT ontology to discover inconsistencies Nisansa de Silva Department of Computer and Information Science University of Oregon Email:

More information

Neural Networks Language Models

Neural Networks Language Models Neural Networks Language Models Philipp Koehn 10 October 2017 N-Gram Backoff Language Model 1 Previously, we approximated... by applying the chain rule p(w ) = p(w 1, w 2,..., w n ) p(w ) = i p(w i w 1,...,

More information

Computerlinguistik II / Sprachtechnologie

Computerlinguistik II / Sprachtechnologie Computerlinguistik II / Sprachtechnologie Vorlesung im SS 2018 (M-GSW-10) Prof. Dr. Udo Hahn Lehrstuhl für Computerlinguistik Institut für Germanistische Sprachwissenschaft Friedrich-Schiller-Universität

More information

Parsing with CFGs L445 / L545 / B659. Dept. of Linguistics, Indiana University Spring Parsing with CFGs. Direction of processing

Parsing with CFGs L445 / L545 / B659. Dept. of Linguistics, Indiana University Spring Parsing with CFGs. Direction of processing L445 / L545 / B659 Dept. of Linguistics, Indiana University Spring 2016 1 / 46 : Overview Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the

More information

Parsing with CFGs. Direction of processing. Top-down. Bottom-up. Left-corner parsing. Chart parsing CYK. Earley 1 / 46.

Parsing with CFGs. Direction of processing. Top-down. Bottom-up. Left-corner parsing. Chart parsing CYK. Earley 1 / 46. : Overview L545 Dept. of Linguistics, Indiana University Spring 2013 Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the problem as searching

More information

Toponym Disambiguation by Arborescent Relationships

Toponym Disambiguation by Arborescent Relationships Journal of Computer Science 6 (6): 653-659, 2010 ISSN 1549-3636 2010 Science Publications Toponym Disambiguation by Arborescent Relationships Imene Bensalem and Mohamed-Khireddine Kholladi Department of

More information

Maschinelle Sprachverarbeitung

Maschinelle Sprachverarbeitung Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other

More information

Developing Tools and Models for Evaluating Geospatial Data Integration of Official and VGI Data Sources

Developing Tools and Models for Evaluating Geospatial Data Integration of Official and VGI Data Sources Developing Tools and Models for Evaluating Geospatial Data Integration of Official and VGI Data Sources Maythm M Sharky Al-Bakri B.Sc. Surveying Engineering M.Sc. Surveying Engineering Thesis submitted

More information

Maschinelle Sprachverarbeitung

Maschinelle Sprachverarbeitung Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other

More information