Conditional Language Modeling. Chris Dyer

Transcription

1 Conditional Language Modeling Chris Dyer

2 Unconditional LMs A language model assigns probabilities to sequences of words,. w =(w 1,w 2,...,w`) It is convenient to decompose this probability using the chain rule, as follows: p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) p(w` w 1,...,w` 1 ) = w Y t=1 p(w t w 1,...,w t 1 ) This reduces the language modeling problem to modeling the probability of the next word, given the history of preceding words.

3 Evaluating unconditional LMs How good is our unconditional language model? 1. Held-out per-word cross entropy or perplexity H = ppl = b 1 w i=1 P w i=1 log b p(w i w <i ) Same as training criterion. How uncertain is the model at each time position, an average? 2. Task-based evaluation 1 w w X log 2 p(w i w <i ) (units: bits per word) (units: uncertainty per word) Use in a task-model that uses a language model in place of some other language model. Does it improve?

4 History-based LMs A common strategy is to make a Markov assumption, which is a conditional independence assumption. p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) p(w 4 w 1,w 2,w 3 )...

5 History-based LMs A common strategy is to make a Markov assumption, which is a conditional independence assumption. p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) Markov: forget the distant past. Is this valid for language? No Is it practical? Often! p(w 4 w 1,w 2,w 3 )...

6 History-based LMs A common strategy is to make a Markov assumption, which is a conditional independence assumption. p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) Markov: forget the distant past. Is this valid for language? No Is it practical? Often! p(w 4 w 1,w 2,w 3 )... Why RNNs are great for language: no more Markov assumptions!

7 History-based LMs with RNNs RNN hidden state random variable vector, length= vocab p(w 5 w 1,w 2,w 3,w 4 ) z } { h 1 h 2 h 3 h 4 h 0 w 1 w 2 w 3 w 4 observed context word w 1 w 2 w 3 w 4 vector (word embedding)

8 History-based LMs with RNNs RNN hidden state random variable vector, length= vocab p(w 5 w 1,w 2,w 3,w 4 ) z } { h 1 h 2 h 3 h 4 h 0 w 1 w 2 w 3 w 4 observed context word w 1 w 2 w 3 w 4 vector (word embedding)

9 Bridle. (1990) Probabilistic interpretation of feedforward classification Distributions over words Each dimension corresponds to a word in a closed vocabulary, V. h the a and cat dog horse runs says walked walks walking pig Lisbon sardines u = Wh + b p i = exp u i P j exp u j The pi s form a distribution, i.e. p i > 0 8i, X i p i =1

10 Distributions over words h the a and cat dog horse runs says walked walks walking pig Lisbon sardines u = Wh + b p i = exp u i P j exp u j p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) p(w 4 w 1,w 2,w 3 )...

11 Distributions over words h the a and cat dog horse runs says walked walks walking pig Lisbon sardines u = Wh + b p i = exp u i P j exp u j p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) histories are sequences of words p(w 4 w 1,w 2,w 3 )...

12 Distributions over words h the a and cat dog horse runs says walked walks walking pig Lisbon sardines u = Wh + b p i = exp u i P j exp u j h 2 R d V = 100, 000 What are the dimensions of b? p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) histories are sequences of words p(w 4 w 1,w 2,w 3 )...

13 Distributions over words h the a and cat dog horse runs says walked walks walking pig Lisbon sardines u = Wh + b p i = exp u i P j exp u j h 2 R d V = 100, 000 What are the dimensions of W? p(w) =p(w 1 ) p(w 2 w 1 ) p(w 3 w 1,w 2 ) histories are sequences of words p(w 4 w 1,w 2,w 3 )...

14 RNN language models

15 RNN language models h 1 h 0 x 1 <s>

16 RNN language models ˆp 1 h 1 h 0 x 1 <s>

17 RNN language models p(tom hsi) tom ˆp 1 h 1 h 0 x 1 <s>

18 RNN language models p(tom hsi) tom ˆp 1 h 1 h 0 x 1 x 2 <s>

19 RNN language models p(tom hsi) tom ˆp 1 h 1 h 2 h 0 x 1 x 2 <s>

20 RNN language models p(tom hsi) p(likes hsi, tom) tom likes ˆp 1 h 1 h 2 h 0 x 1 x 2 <s>

21 RNN language models p(tom hsi) p(likes hsi, tom) p(beer hsi, tom, likes) tom likes beer ˆp 1 h 1 h 2 h 3 h 0 x 1 x 2 x 3 <s>

22 RNN language models p(tom hsi) p(likes hsi, tom) p(beer hsi, tom, likes) p(h/si hsi, tom, likes, beer) tom likes beer </s> ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

23 Training RNN language models tom likes beer </s> ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

24 Training RNN language models tom likes beer </s> cost 1 cost 2 cost 3 cost 4 ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

25 Training RNN language models log loss/ cross entropy tom likes beer </s> {cost 1 cost 2 cost 3 cost 4 ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

26 Training RNN language models F log loss/ cross entropy { tom likes beer </s> cost 1 cost 2 cost 3 cost 4 ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

27 Training RNN language models F log loss/ cross entropy { tom likes beer </s> cost 1 cost 2 cost 3 cost 4 ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

28 Training RNN language models The cross-entropy objective seeks the maximum likelihood (MLE) objective. Find the parameters that make the training data most likely.

29 Training RNN language models The cross-entropy objective seeks the maximum likelihood (MLE) objective. Find the parameters that make the training data most likely. You will overfit. 1. Stop training early, based on a validation set 2. Weight decay / other regularizers 3. Dropout during training. In contrast to count-based models, zeroes aren t a problem.

30 RNN language models Unlike Markov (n-gram) models, RNNs never forget However, they don t always remember so well (recall Felix s lectures on RNNs vs. LSTMs) Algorithms Sample a sequence from the probability distribution defined by the RNN Train the RNN to minimize cross entropy (aka MLE) What about: what is the most probable sequence?

31 How well do RNN LMs do? perplexity Word Error Rate (WER) order=5 Markov Kneser-Ney freq. est RNN 400 hidden xRNN interpolation Mikolov et al. (2010 Interspeech) Recurrent neural network based language model

32 How well do RNN LMs do? perplexity Word Error Rate (WER) order=5 Markov Kneser-Ney freq. est RNN 400 hidden xRNN interpolation Mikolov et al. (2010 Interspeech) Recurrent neural network based language model

33 Conditional LMs A conditional language model assigns probabilities to sequences of words, w =(w 1,w 2,...,w`), given some conditioning context, x. As with unconditional models, it is again helpful to use the chain rule to decompose this probability: Ỳ p(w x) = p(w t x,w 1,w 2,...,w t 1 ) t=1 What is the probability of the next word, given the history of previously generated words and conditioning context x?

34 Conditional LMs x input text output An author A document written by that author A topic label An article about that topic {SPAM, NOT_SPAM} A sentence in French A sentence in English A sentence in English An image A document A document Meterological measurements Acoustic signal Conversational history + database A question + a document A question + an image w An Its English translation Its French translation Its Chinese translation A text description of the image Its summary Its translation A weather report Transcription of speech Dialogue system response Its answer Its answer

35 Conditional LMs x input text output An author A document written by that author A topic label An article about that topic {SPAM, NOT_SPAM} A sentence in French A sentence in English A sentence in English An image A document A document Meterological measurements Acoustic signal Conversational history + database A question + a document A question + an image w An Its English translation Its French translation Its Chinese translation A text description of the image Its summary Its translation A weather report Transcription of speech Dialogue system response Its answer Its answer

36 Conditional LMs x input text output An author A document written by that author A topic label An article about that topic {SPAM, NOT_SPAM} A sentence in French A sentence in English A sentence in English An image A document A document Meterological measurements Acoustic signal Conversational history + database A question + a document A question + an image w An Its English translation Its French translation Its Chinese translation A text description of the image Its summary Its translation A weather report Transcription of speech Dialogue system response Its answer Its answer

37 Data for training conditional LMs To train conditional language models, we need paired samples, {(x i, w i )} N i=1. Data availability varies. It s easy to think of tasks that could be solved by conditional language models, but the data just doesn t exist. Relatively large amounts of data for: Translation, summarisation, caption generation, speech recognition

38 Evaluating conditional LMs How good is our conditional language model? These are language models, we can use cross-entropy or perplexity. okay to implement, hard to interpret Task-specific evaluation. Compare the model s most likely output to human-generated expected output using a task-specific evaluation metric L. w = arg max w p(w x) L(w, w ref ) Examples of L: BLEU, METEOR, WER, ROUGE. easy to implement, okay to interpret Human evaluation. hard to implement, easy to interpret

39 Evaluating conditional LMs How good is our conditional language model? These are language models, we can use cross-entropy or perplexity. okay to implement, hard to interpret Task-specific evaluation. Compare the model s most likely output to human-generated expected output using a task-specific evaluation metric L. w = arg max w p(w x) L(w, w ref ) Examples of L: BLEU, METEOR, WER, ROUGE. easy to implement, okay to interpret Human evaluation. hard to implement, easy to interpret

40 Lecture overview The rest of this lecture will look at encoder-decoder models that learn a function that maps into a fixed-size vector and then uses a language model to decode that vector into a sequence of words, w. x x Kunst kann nicht gelehrt werden encoder representation decoder w Artistry can t be taught

41 Lecture overview The rest of this lecture will look at encoder-decoder models that learn a function that maps into a fixed-size vector and then uses a language model to decode that vector into a sequence of words, w. x x encoder representation decoder w A dog is playing on the beach.

42 Lecture overview Two questions How do we encode x as a fixed-size vector, c? - Problem (or at least modality) specific - Think about assumptions How do we condition on c in the decoding model? - Less problem specific - We will review one standard solution: RNNs

43 Kalchbrenner and Blunsom 2013 Encoder c =embed(x) s = Vc

44 Kalchbrenner and Blunsom 2013 Encoder c =embed(x) s = Vc Recurrent decoder Recurrent connection Embedding of w t 1 h t = g(w[h t 1 ; w t 1 ] + s + b]) u t = Ph t + b 0 p(w t x, w <t ) = (u t ) Learnt bias Source sentence

45 Kalchbrenner and Blunsom 2013 Encoder c =embed(x) s = Vc Recurrent decoder Recurrent connection Embedding of w t 1 h t = g(w[h t 1 ; w t 1 ] + s + b]) u t = Ph t + b 0 p(w t x, w <t ) = (u t ) Learnt bias Source sentence Recall unconditional RNN h t = g(w[h t 1 ; w t 1 ]+b])

46 K&B 2013: Encoder How should we define c =embed(x)? The simplest model possible: c = X i x i x 1 x 2 x 3 x 4 x 5 x 6 x 1 x 2 x 3 x 4 x 5 x 6 What do you think of this model?

47 K&B 2013: RNN Decoder Encoder c =embed(x) s = Vc Recurrent decoder Recurrent connection Embedding of w t 1 h t = g(w[h t 1 ; w t 1 ] + s + b]) u t = Ph t + b 0 p(w t x, w <t ) = (u t ) Learnt bias Source sentence Recall unconditional RNN h t = g(w[h t 1 ; w t 1 ]+b])

48 K&B 2013: RNN Decoder s h 1 h 0 x 1 <s>

49 K&B 2013: RNN Decoder s ˆp 1 h 1 h 0 x 1 <s>

50 K&B 2013: RNN Decoder p(tom s, hsi) s tom ˆp 1 h 1 h 0 x 1 <s>

51 K&B 2013: RNN Decoder p(tom s, hsi) p(likes s, hsi, tom) s tom likes ˆp 1 h 1 h 2 h 0 x 1 x 2 <s>

52 K&B 2013: RNN Decoder p(tom s, hsi) p(likes s, hsi, tom) p(beer s, hsi, tom, likes) s tom likes beer ˆp 1 h 1 h 2 h 3 h 0 x 1 x 2 x 3 <s>

53 K&B 2013: RNN Decoder p(tom s, hsi) s p(likes s, hsi, tom) p(beer s, hsi, tom, likes) p(h\si s, hsi, tom, likes, beer) tom likes beer </s> ˆp 1 h 1 h 2 h 3 h 4 h 0 x 1 x 2 x 3 x 4 <s>

54 A word about decoding In general, we want to find the most probable (MAP) output given the input, i.e. w = arg max w = arg max w p(w x) w X t=1 log p(w t x, w <t )

55 A word about decoding In general, we want to find the most probable (MAP) output given the input, i.e. w = arg max w = arg max w p(w x) w X t=1 undecidable :( log p(w t x, w <t ) This is, for general RNNs, a hard problem. We therefore approximate it with a greedy search: w 1 arg max w 1 p(w 1 x) w 2 arg max w 2 p(w 2 x,w 1).. w t arg max w t p(w t x, w <t)

56 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I hsi logprob=0 w 0 w 1 w 2 w 3

57 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 w 0 w 1 w 2 w 3

58 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 w 0 w 1 w 2 w 3

59 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 beer logprob=-8.66 drink logprob=-2.87 w 0 w 1 w 2 w 3

60 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 beer logprob=-8.66 drink logprob=-2.87 w 0 w 1 w 2 w 3

61 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 drink logprob=-6.28 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 beer logprob=-8.66 like logprob=-7.31 beer logprob=-3.04 drink logprob=-2.87 wine logprob=-5.12 w 0 w 1 w 2 w 3

62 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 drink logprob=-6.28 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 beer logprob=-8.66 like logprob=-7.31 beer logprob=-3.04 drink logprob=-2.87 wine logprob=-5.12 w 0 w 1 w 2 w 3

63 A word about decoding A slightly better approximation is to use a beam search with beam size b. Key idea: keep track of top b hypothesis. E.g., for b=2: x = Bier trinke ich beer drink I drink logprob=-6.93 drink logprob=-6.28 hsi logprob=0 beer logprob=-1.82 I logprob=-2.11 I logprob=-5.80 beer logprob=-8.66 like logprob=-7.31 beer logprob=-3.04 drink logprob=-2.87 wine logprob=-5.12 w 0 w 1 w 2 w 3

64 How well does this model do? perplexity (2011) perplexity (2012) order=5 Markov Kneser-Ney freq. est RNN LM RNN LM + x

65 How well does this model do?

66 How well does this model do?

67 How well does this model do?

68 How well does this model do?

69 How well does this model do?

70 How well does this model do?

71 How well does this model do? (Literal: I will feel bad if you do not find a solution.)

72 How well does this model do? (Literal: I will feel bad if you do not find a solution.)

73 Summary Conditional language modeling provides a convenient formulation for a lot of practical applications Two big problems: Model expressivity Decoding difficulties Next time A better encoder for vector to sequence models Attention for better learning Lots of results on machine translation

74 Questions?