Gated RNN & Sequence Generation. Hung-yi Lee 李宏毅

Transcription

1 Gated RNN & Sequence Generation Hung-yi Lee 李宏毅

2 Outline RNN with Gated Mechanism Sequence Generation Conditional Sequence Generation Tips for Generation

3 RNN with Gated Mechanism

4 Recurrent Neural Network Given function f: h, y = f h, x h and h are vectors with the same dimension y 1 y 2 y 3 h 0 f h 1 f h 2 f h 3 x 1 x 2 x 3 No matter how long the input/output sequence is, we only need one function f

5 Deep RNN h, y = f 1 h, x b, c = f 2 b, y c 1 c 2 c 3 b 0 f 2 b 1 f 2 b 2 f 2 b 3 y 1 y 2 y 3 h 0 f 1 h 1 f 1 h 2 f 1 h 3 x 1 x 2 x 3

6 idirectional RNN h, a = f 1 h, x b, c = f 2 b, x x 1 x 2 x 3 b 0 f 2 b 1 f 2 b 2 f 2 b 3 c 1 c 2 c 3 y = f 3 a, c f 3 y 1 f 3 y 2 f 3 y 3 a 1 a 2 a 3 h 0 f 1 h 1 f 1 h 2 f 1 h 3 x 1 x 2 x 3

7 Naïve RNN Given function f: h, y = f h, x y h ' + = σ W h h W i x h f h ' x y = σ softmax W o h ' Ignore bias here

8 LSTM y t h t-1 y t Naive h t c t-1 h t-1 LSTM c t h t x t x t c changes slowly h changes faster c t is c t-1 added by something h t and h t-1 can be very different

9 z = tanh W x t h t-1 c t-1 z i = σ W i x t h t-1 z f z i z z o z f = σ W f x t h t-1 h t-1 x t z o = σ W o x t h t-1

10 x t z = tanh W h t-1 c t-1 c t-1 diagonal peephole z o z f z i obtained by the same way z f z i z z o h t-1 x t

11 y t c t-1 c t + c t = z f c t 1 +z i z h t = z o tanh c t z f z i z z o y t = σ W h t h t-1 x t h t

12 LSTM y t y t+1 c t-1 c t c t z f z i z z o z f z i z z o h t h t-1 x t h t x t+1

13 GRU h t = z h t z h y t h t-1 + h t 1- reset r update z h' h t-1 x t x t

14 LSTM: Search Space Odyssey

15 LSTM: Search Space Odyssey Standard LSTM works well Simply LSTM: coupling input and forget gate, removing peephole Forget gate is critical for performance Output gate activation function is critical

16 n Empirical Exploration of Recurrent Network rchitectures LSTM-f/i/o: removing forget/input/output gates LSTM-b: large bias Importance: forget > input > output Large bias for forget gate is helpful

17 n Empirical Exploration of Recurrent Network rchitectures

18 Neural rchitecture Search with Reinforcement Learning LSTM From Reinforcement Learning

19 Sequence Generation

20 Generation x: y: 你 我 他 是 很 Sentences are composed of characters/words Generating a character/word at each time by RNN h y f h ' Distribution over the token (sampling from the distribution to generate a token) x The token generated at the last time step. (represented by 1-of-N encoding)

21 Generation y 1 : P(w <OS>) y 2 : P(w <OS>, 床 ) y 3 : P(w <OS>, 床, 前 ) Sentences are composed of characters/words Generating a character/word at each time by RNN 床前明 ~ y 1 sample ~ ~ y 2 y 3 Until <EOS> is generated h 0 f h 1 f h 2 f h 3 x 1 <OS> x 2 床 x 3 前

22 Generation : minimizing cross-entropy Training Training data: 春眠不覺曉 春眠不 y 1 y 2 y 3 h 0 f h 1 f h 2 f h 3 x 1 <OS> x 2 春 x 3 眠

23 Generation Images are composed of pixels Generating a pixel at each time by RNN red blue green ~ y 1 sample ~ y 2 Consider as a sentence blue red yellow gray Train a RNN based on the sentences ~ y 3 h 0 f h 1 f h 2 f h 3 x 1 <OS> x 2 red x 3 blue

24 Generation 3 x 3 images Images are composed of pixels

25 Conditional Sequence Generation

26 Conditional Generation We don t want to simply generate some random sentences. Generate sentences based on conditions: Caption Generation Given condition: young girl is dancing. Chat-bot Given condition: Hello Hello. Nice to see you.

27 Conditional Generation Represent the input condition as a vector, and consider the vector as the input of RNN generator Image Caption Generation vector woman. (period) CNN <OS> Input image

28 Conditional Generation Sequence-tosequence learning Represent the input condition as a vector, and consider the vector as the input of RNN generator E.g. Machine translation / Chat-bot Information of the whole sentences machine learning. (period) 機 器 學 習 Encoder Jointly train Decoder

29 Conditional Generation M: Hello U: Hi M: Hi Need to consider longer context during chatting M: Hi M: Hello U: Hi Serban, Iulian V., lessandro Sordoni, Yoshua engio, aron Courville, and Joelle Pineau, 2015 "uilding End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models.

30 Dynamic Conditional Generation c 1 c 2 machine learning. (period) h 1 h 2 h 3 h 4 機 器 學 習 c 1 c 2 c 3 Encoder Decoder

31 Dynamic Conditional Generation Encoder Decoder Information of the whole sentences machine learning. (period) 機 器 學 習 c 1 c 2 c 3

32 Machine Translation ttention-based model α 0 1 match h 1 h 2 h 3 h 4 機 器 學 習 z 0 Jointly learned with other part of the network What is? match Design by yourself Cosine similarity of z and h Small NN whose input is z and h, output a scalar α = h T Wz match h α z

33 Machine Translation ttention-based model c α α α α 0 softmax 1 α 0 2 α 0 3 α 0 4 α 0 h 1 h 2 h 3 h 4 machine z 0 z 1 c 0 c 0 = α 0 i h i Decoder input 機 器 學 習 = 0.5h h 2

34 Machine Translation ttention-based model α 1 1 match h 1 h 2 h 3 h 4 z 0 machine z 1 c 0 機 器 學 習

35 Machine Translation ttention-based model c 1 machine learning α 1 α α α 1 softmax z 0 z 1 z 2 α 1 1 α 1 2 α 1 3 α 1 4 c 0 c 1 h 1 h 2 h 3 h 4 c 1 = α 1 i h i 機 器 學 習 = 0.5h h 4

36 Machine Translation ttention-based model machine learning α 2 1 match z 0 z 1 z 2 c 0 c 1 h 1 h 2 h 3 h 4 機 器 學 習 The same process repeat until generating <EOS>

37 Speech Recognition William Chan, Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, Listen, ttend and Spell, ICSSP, 2016

38 Image Caption Generation vector for each region z 0 match 0.7 CNN filter filter filter filter filter filter filter filter filter filter filter filter

39 Image Caption Generation vector for each region z 0 Word 1 z 1 CNN filter filter filter filter filter filter weighted sum filter filter filter filter filter filter

40 Image Caption Generation vector for each region z 0 Word 1 z 1 Word 2 z 2 CNN filter filter filter filter filter filter weighted sum filter filter filter filter filter filter

41 Image Caption Generation Kelvin Xu, Jimmy a, Ryan Kiros, Kyunghyun Cho, aron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua engio, Show, ttend and Tell: Neural Image Caption Generation with Visual ttention, ICML, 2015

42 Image Caption Generation Kelvin Xu, Jimmy a, Ryan Kiros, Kyunghyun Cho, aron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua engio, Show, ttend and Tell: Neural Image Caption Generation with Visual ttention, ICML, 2015

43 Li Yao, tousa Torabi, Kyunghyun Cho, Nicolas allas, Christopher Pal, Hugo Larochelle, aron Courville, Describing Videos by Exploiting Temporal Structure, ICCV, 2015

44 Question nswering Given a document and a query, output an answer bbi: the answer is a word SQuD: the answer is a sequence of words (in the input document) MS MRCO: the answer is a sequence of words MovieQ: Multiple choice question (output a number)

45 Memory Network nswer Sentence to vector can be jointly trained. Extracted Information N = n=1 α n x n DNN Document α 1 x 1 α 2 α 3 α N x 2 x 3 x N vector Match q Query Sainbayar Sukhbaatar, rthur Szlam, Jason Weston, Rob Fergus, End-To-End Memory Networks, NIPS, 2015

46 Memory Network Extracted Information N = n=1 α n h n nswer DNN Jointly learned h 1 h 2 h 3 h N α 1 α 2 α 3 α N Hopping Document x 1 x 2 x 3 x N Match q Query

47 Memory Network Extract information Compute attention DNN a Extract information Compute attention q

48 Reading Comprehension End-To-End Memory Networks. S. Sukhbaatar,. Szlam, J. Weston, R. Fergus. NIPS, The position of reading head: Keras has example: bi_memnn.py

49 Wei Fang, Juei-Yang Hsu, Hungyi Lee, Lin-Shan Lee, "Hierarchical ttention Model for Improved Machine Comprehension of Spoken Content", SLT, 2016

50 Neural Turing Machine von Neumann architecture Neural Turing Machine not only read from memory lso modify the memory through attention

51 Neural Turing Machine y 1 h 0 f r 0 = α 0 i m 0 i α 0 1 α 0 2 α 0 3 α 0 4 x 1 Retrieval process r 0 m 0 1 m 0 2 m 0 3 m 0 4

52 Neural Turing Machine y 1 h 0 f r 0 = α 0 i m 0 i x 1 r 0 k 1 e 1 a 1 α 0 1 α 0 2 α 0 3 α 0 4 m 0 1 m 0 2 m 0 3 m 0 4 α 1 1 α 2 1 α α 1 softmax α 1 1 α 2 1 α α 1 α i 1 = cos m i 0, k 1

53 Neural Turing Machine m 1 i i i = m 0 α e 1 i i + α a 1 1 m 0 1 (element-wise) k 1 e 1 a 1 α 0 1 α 0 2 α 0 3 α 0 4 m 0 1 m 0 2 m 0 3 m 0 4 α 1 1 α 1 2 α 1 3 α 1 4 m 1 1 m 1 2 m 1 3 m ~ 1

54 Neural Turing Machine y 1 y 2 h 0 f h 1 f x 1 r 0 x 2 r 1 α 0 1 α 0 2 α 0 3 α 0 4 α 1 1 α 1 2 α 1 3 α 1 4 α 2 1 α 2 2 α 2 3 α 2 4 m 0 1 m 0 2 m 0 3 m 0 4 m 1 1 m 1 2 m 1 3 m 1 4 m 2 1 m 2 2 m 2 3 m 2 4

55 Neural Turing Machine Wei-Jen Ko, o-hsiang Tseng, Hung-yi Lee, Recurrent Neural Network based Language Modeling with Controllable External Memory, ICSSP, 2017

56 rmand Joulin, Tomas Mikolov, Inferring lgorithmic Patterns with Stack-ugmented Recurrent Nets, arxiv Pre-Print, 2015 Stack RNN y t Push Pop Nothing stack Information to store Push, Pop, Nothing f + + X0.7 X0.2 X0.1 x t

57 Tips for Generation

58 α t i ttention component time Kelvin Xu, Jimmy a, Ryan Kiros, Kyunghyun Cho, aron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua engio, Show, ttend and Tell: Neural Image Caption Generation with Visual ttention, ICML, 2015 ad ttention α 1 1 α 2 1 α 3 1 α 4 1 α 1 2 α 2 2 α 3 2 α 4 2 α 1 3 α 2 3 α 3 3 α 4 3 α 1 4 α 2 4 α 3 4 α 4 4 w 1 w 2 (woman) w 3 w 4 (woman) no cooking Good ttention: each input component has approximately the same attention weight 2 E.g. Regularization term: i τ α t i t For each component Over the generation

59 Mismatch between Train and Test Training Reference: C = t C t Minimizing cross-entropy of each component : condition <OS>

60 Mismatch between Train and Test Generation We do not know the reference Testing: The inputs are the outputs of the last time step. Training: The inputs are reference. <OS> Exposure ias

61 One step wrong May be totally wrong Never explore 一步錯, 步步錯

62 Modifying Training Process? When we try to decrease the loss for both steps 1 and 2.. Reference Training is matched to testing. In practice, it is hard to train in this way.

63 Scheduled Sampling Reference from model From reference from model From reference

64 Scheduled Sampling Caption generation on MSCOCO LEU-4 METEOR CIDER lways from reference lways from model Scheduled Sampling Samy engio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer, Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks, arxiv preprint, 2015

65 eam Search The green path has higher score. Not possible to check all the paths

66 eam Search Keep several best path at each step eam size = 2

67 eam Search The size of beam is 3 in this example.

68 etter Idea? I am You are I are You am I you you are high score you am are <OS> <OS> you I you

69 Object level v.s. Component level Minimizing the error defined on component level is not equivalent to improving the generated objects Ref: The dog is running fast C = t C t Cross-entropy of each step cat a a a The dog is is fast The dog is running fast Optimize object-level criterion instead of component-level crossentropy. object-level criterion: R y, y Gradient Descent? y: generated utterance, y: ground truth

70 Reinforcement learning? Start with observation s 1 Observation s 2 Observation s 3 Obtain reward r 1 = 0 ction a 1 : right Obtain reward r 2 = 5 ction a 2 : fire (kill an alien)

71 Reinforcement learning? reward: R(, reference) ction taken r = 0 r = 0 ctions set observation <OS> Marc'urelio Ranzato, Sumit Chopra, Michael uli, Wojciech Zaremba, Sequence Level Training with Recurrent Neural Networks, ICLR, 2016 The action we take influence the observation in the next step

72 Concluding Remarks RNN with Gated Mechanism Sequence Generation Conditional Sequence Generation Tips for Generation