Connotation: A Dash of Sentiment Beneath the Surface Meaning

Transcription

1 Connotation: A Dash of Sentiment Beneath the Surface Meaning

2 Con notation com- ( together or with ) notare ( to mark ) 1

3 Con notation com- ( together or with ) notare ( to mark ) Commonly understood cultural or emotional association that some word carries, in addition to its explicit or literal meaning (denotation). Generally described as positive or negative. 2

4 Good News? Bad News? Decrease in deforestation drives the trend, but emissions from energy and agriculture grow. 3

5 Good News? Bad News? the removal of trees Decrease in deforestation drives the trend, but emissions from energy and agriculture grow. The production and discharge of something, esp. gas or 4

6 Good News? Bad News? the removal of trees Decrease in deforestation drives the trend, but emissions from energy and agriculture grow. The production and discharge of something 5

7 Good News? Bad News? the removal of trees Decrease in deforestation drives the trend, but emissions from energy and agriculture grow. The production and discharge of something,,esp. gas or radiation 6

8 Learning the General Connotation Decrease in deforestation drives the trend, but emissions from energy and agriculture grow. Negatively connotative in general 7

9 Con notation com- ( together or with ) notare ( to mark ) Commonly understood cultural or emotional association that some word carries, in addition to its explicit or literal meaning (denotation). Generally described as positive or negative. 8

10 Sentiment vs. Connotation 9

11 Sentiment vs. Connotation joy sick 10

12 Sentiment vs. Connotation Neutral joy Sentiment sick surfing scientist grid rose header blister emission salt bedbug 11

13 Sentiment vs. Connotation Neutral joy scientist music surfing rose Sentiment sick flu Connotation deforestation emission surfing scientist grid rose header blister emission salt bedbug 12

14 Sentiment vs. Connotation Neutral joy scientist music surfing rose sick Connotation flu bedbug deforestation emission surfing scientist grid rose header blister emission salt bedbug 13

15 Learning the General Connotation } Data } Linguistic insights } Graph representation } Inference algorithms } Evaluations 14

16 Data } Web-driven data Google Web 1T (Brants and Franz (2006)) } N-grams (1 <= n <= 5) } Frequency of occurrences } Example: prevent financial malware 4130 } Dictionary-drive data WordNet (George A. Miller (1995)) } Synsets: synonyms, antonyms 15

17 Learning the General Connotation } Data } Linguistic insights } Graph representation } Inference algorithms } Evaluations 16

18 Diverse Linguistic Insights } Semantic prosody } Semantic parallelism of coordination } Distributional similarity } Semantic relations 17

19 Diverse Linguistic Insights } Semantic prosody } Semantic parallelism of coordination } Distributional similarity } Semantic relations 18

20 Semantic Prosody ptimization: ILP / LP Sinclair (1991, 2004), louw (1993) 930,000,000 2,650,000 19

21 Selectional Preference on Connotation accident ß avoid accident, cause accident, avoid à avoid bedbugs, avoid danger, enjoy Pred à [enjoy] Pred [music] Arg oid bedbugs, oid danger, art ß enjoy the art, appreciate the art, à enjoy the wine, enjoy life, prevent Pred à [prevent] Pred [bedbugs] Arg 20

22 Selectional Preference on Connotation accident ß avoid accident, cause accident, avoid à avoid bedbugs, avoid danger, enjoy Pred à [enjoy] Pred [music] Arg oid bedbugs, oid danger, Candidates art ß enjoy the art, appreciate the art, y à enjoy the wine, enjoy life, prevent Pred à [prevent] Pred [bedbugs] Arg 21

23 Connotative Predicate: Feng et al A predicate that has selectional preference on the connotative polarity of some of its semantic arguments. 22

24 Connotative Predicate: Feng et al A predicate that has selectional preference on the connotative polarity of some of its semantic arguments. Connotative Predicates Sentiment of predicate Preference on arguments Examples suffer negative negative suffering from cough cure positive negative cure backache cause neutral negative caused CO 2 emissions 23

25 Connotative Predicate: Feng et al A predicate that has selectional preference on the connotative polarity of some of its semantic arguments. Connotative Predicates Sentiment of predicate Preference on arguments Examples suffer negative negative suffering from cough cure positive negative cure backache cause neutral negative caused CO 2 emissions 24

26 Accomplish Achieve Advance Advocate Admire Applaud Appreciate Compliment Congratulate Develop 20 Positive Connotative Predicates Desire Enhance Enjoy Improve Praise Promote Respect Save Support Win Alleviate Accuse Avert Avoid Cause Complain Condemn Criticize Detect Eliminate 20 Negative Connotative Predicates Eradicate Mitigate Overcome Prevent Prohibit Protest Refrain Suffer Tolerate Withstand Feng et al

27 Diverse Linguistic Insights } Semantic prosody [Corpus: GoogleNgram] } [enjoy] Pred [music] Arg,S[prevent] Pred [begbugs] Arg antic parallelism of coordination (Google) Pattern * and *, e.g., music and wine Distributional similarity (Google) findings potential > findings corrections Semantic relations (WordNet) Synonyms Antonyms 26

28 Diverse Linguistic Insights } Semantic prosody [Corpus: GoogleNgram] } enjoy * ; prevent * } Semantic parallelism of coordination [Corpus: GoogleNgram] } Pattern * and *, e.g., music and wine Semantic relations (WorldNet) Synonyms Antonyms 27

29 Diverse Linguistic Insights } Semantic prosody [Corpus: GoogleNgram] } enjoy * ; prevent * } Semantic parallelism of coordination [Corpus: GoogleNgram] } Pattern * and *, e.g., music and wine } Distributional similarity [Corpus : GoogleNgram] } findings potentials > findings modifications Semantic relations (WorldNet) Synonyms Antonyms 28

30 Diverse Linguistic Insights } Semantic prosody [Corpus: GoogleNgram] } enjoy * ; prevent * } Semantic parallelism of coordination [Corpus: GoogleNgram] } Pattern * and *, e.g., music and wine } Distributional similarity [Corpus : GoogleNgram] } findings potentials > findings modifications } Semantic relations [Corpus: WordNet] } Synonyms } Antonyms relations (WorldNet) 29

31 Learning the General Connotation } Data } Linguistic insights } Graph representation } Inference algorithms } Evaluations 30

32 Graph Representation } Graph G = (V, E) } V = {Pred} {Arg} } E1: Pred Arg Pred-Arg enjoy thank avoid prevent writing profit help risk 31

33 Graph Representation } Graph G = (V, E) } V = {Pred} {Arg} } E1: Pred Arg Pred-Arg enjoy thank writing profit Arg-Arg reading investment } E2: Arg Arg avoid prevent help risk aid 32

34 Graph Representation } Graph G = (V, E) } V = {Pred} {Arg} } E1: Pred Arg Pred-Arg enjoy thank writing profit Arg-Arg reading investment } E2: Arg Arg avoid prevent help risk aid a 1 and w 1 à PMI(a 1, w 1 ) a 1 and w 2 à PMI(a 1, w 2 ) a 1 and w 3 à PMI(a 1, w n ) 33 a 1 := PMI(a 1, w 1 ) PMI(a 1, w 2 ) PMI(a 1, w n )

35 Graph Representation prevent suffer enjoy thank tax loss writing profit preventing gain bonus investment cold flu prosody coordination synonyms antonyms 34

36 Learning the General Connotation } Data } Linguistic insights } Graph representation } Inference algorithms } Evaluations 35

37 Inference Algorithm Algorithms Linguistic Insights HITS/PageRank Semantic Prosody Label Propagation Integer Linear Programing Semantic Parallelism of Coordination Distributional Similarity Belief Propagation Semantic Relations W 36

38 Inference Algorithm Algorithms Linguistic Insights HITS/PageRank Semantic Prosody Label Propagation Integer Linear Programing Semantic Parallelism of Coordination Distributional Similarity Belief Propagation Semantic Relations W 37

39 ILP: Problem Formulation } For each unlabeled word i, solve x i, y i, z i {0, 1}, x i à positive, y i à negative, z i à neutral; x i + y i + z i = 1. 38

40 ILP: Problem Formulation } For each unlabeled word i, solve x i, y i, z i {0, 1}, x i à positive, y i à negative, z i à neutral; x i + y i + z i = 1. } Initialization (Hard constraints) } Positive seed predicates (e.g. achieve ) à x i = 1 } Negative seed predicates (e.g. prevent ) à y i = 1 39

41 ILP: Objective Function Maximize 40

42 ILP: Objective Function Maximize prevent suffer enjoy thank tax loss writing profit preventing bonus flu cold 41

43 ILP: Objective Function Maximize prevent suffer enjoy thank tax loss writing profit preventing bonus flu cold 42

44 ILP: Objective Function 43

45 ILP: Objective Function 44

46 ILP: Soft Constraints } Predicate Argument } Argument Argument 45

47 ILP: Hard Constraints Semantic relations Antonym pairs will not have the same positive or negative polarity. Synonym pairs will not have the opposite polarity. 46

48 Inference Algorithm Algorithms Linguistic Insights HITS/PageRank Semantic Prosody Label Propagation Integer Linear Programing Semantic Parallelism of Coordination Distributional Similarity Belief Propagation Semantic Relations W 47

49 Graph LEMMA SENSE pain ache wound -injury -wound prevent suffer loss accident injure lose -lose -decrease enjoy life profit win gain -win -profits achieve success investment -gain -acquire put on -gain, -put on prosody (Pred-Arg) coordination memberships antonyms 48

50 Graph LEMMA SENSE pain ache wound -injury -wound prevent suffer loss accident injure lose -lose -decrease enjoy life profit win gain -win -profits achieve success investment -gain -acquire put on -gain, -put on prosody (Pred-Arg) coordination memberships antonyms 49

51 Graph Types of Nodes Lemmas prevent (115K) Synsets (63K) suffer pain loss LEMMA ache accident wound injure lose life Types of Edges win enjoy 1. Predicate-argument profit (179K) gain 2. Argumentachieve argument (144K) 3. Argument-synset (126K) success 4. Synset-synset (34K) investment put on prosody (Pred-Arg) coordination SENSE -injury -wound -lose -decrease -win -profits -gain -acquire -gain, -put on memberships antonyms 50

52 Problem Formulation } G Lemma + Sense = (V, E) } Nodes (random variables) } V = {v 1, v 2,, v n } } Unobserved variables: } Typed edges } E = {e(v i, v j, v k ) } where v i, v j V, T k T } T = {pre-arg, arg-arg, arg-syn, syn-syn} } Neighborhood function } N v = {u e(u, v) E} } Labels ϒ = {Y 1,Y 2,...,Y n }, y i } L = {+, -}, y i denotes the label of Y i. 51

53 Pairwise Markov Random Fields 52

54 Objective Function P(y x) = 1 Z(x) Yi ϒ ψ i (y i ) t ψ ij (y i, y j ) 53

55 Objective Function An assignment to all the unobserved variables P(y x) = 1 Z(x) Yi ϒ ψ i (y i ) t ψ ij (y i, y j ) 54

56 Objective Function An assignment to all the unobserved variables P(y x) = 1 Z(x) Yi ϒ Variables with known labels ψ i (y i ) t ψ ij (y i, y j ) 55

57 Objective Function P(y x) = 1 Z(x) Yi ϒ ψ i (y i ) t ψ ij (y i, y j ) Prior mapping: y i refers to Y i s label ψ i is prior mapping. 56

58 Objective Function P(y x) = 1 Z(x) Yi ϒ ψ i (y i ) t ψ ij (y i, y j ) Prior mapping: L à R 0 Compatibility mapping: L x L à R 0 57

59 Objective Function An assignment to all the unobserved variables 58

60 Loopy Belief Propagation } Message passing m i j (y i ) = α (ψ ij y i L t (y i, y j )ψ i (y i ) m k i (y i )), y j L Y k Ni ϒ \Y i } Belief b i (y i ) = βψ i (y i ) m j i (y i ), y i L Y j Ni ϒ 59

61 Loopy Belief Propagation } Initialize message between all node pairs connected by an edge. } Initialize priors for all nodes } Integrative message passing until all messages stop changing m i j (y i ) = α (ψ ij y i L } Compute beliefs. } Assign the label t (y i, y j )ψ i (y i ) m k i (y i )), y j L Y k Ni ϒ \Y i Y j Ni ϒ b i (y i ) = βψ i (y i ) m j i (y i ), y i L L i max i b i (y i ) 60