Semantic distances & LSA

Size: px

Start display at page:

Download "Semantic distances & LSA"

Samuel Dickerson
5 years ago
Views:

1 Semantic istances & LSA Stefan rausan-matu Politehnica" University of Bucharest an Romanian Acaemy Research Institute for Artificial Intelligence Bucharest, Romania 1

2 Lexical chains Venture capitalists have become culture heroes in the New Economy. hey're the visionary investors who sink millions of ollars into risky start-up companies, take seats on their boars to mentor the tech ners--an then cash out at a huge profit. he name "San Hill Roa," the street in Menlo Park where Kleiner Perkins an some of the other famous venture capitalists have their offices, has assume an almost magical quality, as if the partners were ispensing fairy ust rather than cash. 29-Apr-11 S. rausan-matu 2 2

3 Applications of lexical chains Verification of text cohesion ext segmentation Summarization Wor sense isambiguation Determination of iscourse structure Automatic hypertext generation Intelligent spelling checking Information retrieval 29-Apr-11 S. rausan-matu 3 3

4 Builing lexical chains ext scanning an etection of semantically relate wors small istance Constructing a set of potential lexical chains Computational problems 29-Apr-11 S. rausan-matu 4 4

5 Low semantic istance = high similarity bank money bank river apple fruit orange-fruit pen paper pen pencil men-crow re-blue man-human woman-human man-men men-women man-bike wheel bike sweet-bitter sweet-essert esert-storm esert-efect singer-song hacker-soft 29-Apr-11 S. rausan-matu 5 5

6 Closeness relations Synonym Hyponym/hypernim Meronym/Holonym Antonym Entailment ypicality 29-Apr-11 S. rausan-matu 6 6

7 High semantic istance bike-cat software-og og-amoeba iea-sleep runner-paint 29-Apr-11 S. rausan-matu 7 7

8 Semantic istance accoring to: Dictionaries (Kozima an Ferugori, Kozima an Ito) hesauri (e.g. Roget Morris an Hirst; Bunrui goihyo Japanese thesaurus Okamura an Hona) Semantic networks (e.g. MeSH Meical Subject Heaings Raa) Ontologies WorNet, FrameNet 29-Apr-11 S. rausan-matu 8 8

9 Chains of Wornet senses [venture capitalist(1), hero(1), visionary(1), investor(1), mentor(1), ner(1), profit(2), venture capitalist(1), quality(1), partner(1), fairy(1)] 29-Apr-11 S. rausan-matu 9 9

10 Raa et al. ist R (c1,c2)=min nr of eges between c1 an c2 29-Apr-11 S. rausan-matu 10 10

11 Hirst an St Onge Morris an Hirst applie for WorNet Directions: ownwar (cause, hyponym, holonym, entailment) upwar (hypernym, meronym) horizontal (similar, participle_of, see_also, antonyme, attribute) 29-Apr-11 S. rausan-matu 11 11

12 Hirst an St Onge Relations: extra-strong wor repetition strong same synset (esert efect) antonyms (col-hot) sub-phrase (school elementary school) meium-strength 29-Apr-11 S. rausan-matu 12 12

13 rel HS = Meium strength 3C, for extra-strong relations 2C, for strong relations C path_length (k * # changes_in_irection), for meium strength relations 0 otherwise 29-Apr-11 S. rausan-matu 13 13

14 Forbien sequences No other irection can precee an upwar irection Only one irection change is allowe Exception: upwar horizontal - ownwar 29-Apr-11 S. rausan-matu 14 14

15 Forbien sequences 29-Apr-11 S. rausan-matu 15 15

16 Allowe sequences 29-Apr-11 S. rausan-matu 16 16

17 Sussna Weight of ege epens on fanout nr: the number of arcs leaving c. w(c1 c2) = 2 1 / nr(c1) If c1 an c2 are ajacent: Dist S (c1,c2)=(w(c1 c2)+w(c2 c1)) / 2 = the epth of the ege in the ontology If not aiacent a istances to shortest paths 29-Apr-11 S. rausan-matu 17 17

18 Wu si Palmer c = lso(c1;c2), lso=lowest super orinate sim WP (c1,c2)=(2xn)/(n1+n2+2xn), Ni the path from ci to c N the path from c to the root ist WP (c1,c2)=(n1+n2)/(n1+n2+2xn), 29-Apr-11 S. rausan-matu 18 18

19 Leacock si Choorow sim LC (c1,c2)=-log(1+length(c1,c2))/(2xd)) length(c1,c2) is Raa s shortest path D is the height of the ontology 29-Apr-11 S. rausan-matu 19 19

20 Resnik he similarity of two concepts is their share information information content of the lowest superorinate Information content of c is given by -log p(c) the less frequent it is, the more information it contains. sim R (c1,c2) = -log p(lso(c1;c2)) 29-Apr-11 S. rausan-matu 20 20

21 Lin he similarity between arbitrary objects A an B is measure by the ratio between the amount of information neee to state their commonality an that neee to fully escribe what they are. 29-Apr-11 S. rausan-matu 21 21

22 Lin sim L (c1,c2)=2log p(lso(c1,c2))/ (log p(c1)+log p(c2)) 29-Apr-11 S. rausan-matu 22 22

23 Jiang an Conrath ist JC (c1,c2)=2log p(lso(c1,c2)) - (log p(c1)+log p(c2)) 29-Apr-11 S. rausan-matu 23 23

24 Problems in etecting lexical chains Uner-chaining the ifficulty of fining semantically relate wors Over-chaining wrong etermination of semantically unrelate wors 29-Apr-11 S. rausan-matu 24 24

25 Uner-chaining Name entitities recognition Venture capitalists have become culture heroes in the New Economy. hey're the visionary investors who sink millions of ollars into risky start-up companies, take seats on their boars to mentor the tech ners--an then cash out at a huge profit. he name "San Hill Roa," the street in Menlo Park where Kleiner Perkins an some of the other famous venture capitalists have their offices, has assume an almost magical quality, as if the partners were ispensing fairy ust rather than cash. 29-Apr-11 S. rausan-matu 25 25

26 Uner-chaining Coreference resolution Venture capitalists have become culture heroes in the New Economy. hey're the visionary investors who sink millions of ollars into risky start-up companies, take seats on their boars to mentor the tech ners--an then cash out at a huge profit. he name "San Hill Roa," the street in Menlo Park where Kleiner Perkins an some of the other famous venture capitalists have their offices, has assume an almost magical quality, as if the partners were ispensing fairy ust rather than cash. 29-Apr-11 S. rausan-matu 26 26

27 Over-chaining Venture capitalists have become culture heroes in the New Economy. hey're the visionary investors who sink millions of ollars into risky start-up companies, take seats on their boars to mentor the tech ners--an then cash out at a huge profit. he name "San Hill Roa," the street in Menlo Park where Kleiner Perkins an some of the other famous venture capitalists have their offices, has assume an almost magical quality, as if the partners were ispensing fairy ust rather than cash. 29-Apr-11 S. rausan-matu 27 27

28 Semantic spaces in Latent Semantic Inexing (LSI) 29-Apr-11 S. rausan-matu 28 28

29 Vector space moel 29-Apr-11 S. rausan-matu 29 29

30 he LSI iea Reucing the imensionality of the vector space, similarly to the least squares metho he effect is the creation of semantic spaces containing semantically relate wors 29-Apr-11 S. rausan-matu 30 30

31 31 29-Apr-11 S. rausan-matu 31 erms-ocuments array (ex. from Manning an Schutze, 1999) = cos truck car moon astronaut monaut A

32 Singular value ecomposition (SVD) A = tx txn S nxn D xn n=min(t,) 29-Apr-11 S. rausan-matu 32 32

33 33 29-Apr-11 S. rausan-matu 33 = im im im im im1 cos truck car moon astronaut monaut

34 34 29-Apr-11 S. rausan-matu 34 S = S

35 35 29-Apr-11 S. rausan-matu 35 D = im im im im im D

36 Properties of SVD SVD is unique, D are orthonormal: = S values are sorte D D = I 29-Apr-11 S. rausan-matu 36 36

37 Reuce A = A A ) 2 By SVD on maps the n-imension space on a k-imension one, with n >>k Common values for k are 100 an Apr-11 S. rausan-matu 37 37

38 B B = S 2x2D x 2 B = im1 im Apr-11 S. rausan-matu 38 38

39 39 29-Apr-11 S. rausan-matu 39 Document correlation (Manning an Schutze, 1999) B B SD SD DS DS SD DS SD SD A A = = = = = ) ( ) ( ) )( ( ) (

40 erm correlation AA = SD ( SD ) = SD DS = ( S)( S) 29-Apr-11 S. rausan-matu 40 40

Semantic Similarity and Relatedness

Semantic Relatedness Semantic Similarity and Relatedness (Based on Budanitsky, Hirst 2006 and Chapter 20 of Jurafsky/Martin 2 nd. Ed. - Most figures taken from either source.) Many applications require