Data-Intensive Computing with MapReduce

Data-Intensive Computing with MapReduce Session 8: Sequence Labeling Jimmy Lin University of Maryland Thursday, March 14, 2013 This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States See http://creativecommons.org/licenses/by-nc-sa/3.0/us/ for details

Source: Wikipedia (The Scream)

Today s Agenda Sequence labeling Midterm Final projects

Sequences Everywhere! Examples of sequences l Waveforms in utterances l Letters in words l Words in sentences l Bases in DNA l Proteins in amino acids l Actions in video l Closing prices in the stock market All share the property of having structural dependencies

Sequence Labeling Given a sequence of observations, assign a label to each observation l Observations can be discrete or continuous, scalar or vector l Assume labels are drawn from a discrete finite set Sample applications: l Speech recognition l POS tagging l Handwriting recognition l Video analysis l Protein secondary structure detection

Approaches Treat as a classification problem? l Make decisions about each observation independently Can we do better? As with before, lots of background material before we get to MapReduce!

MapReduce Implementation: Mapper 1: class Mapper 2: method Initialize(integer iteration) P T 3: hs, Oi ReadModel i=1\o 4: ha, B, i ReadModelParams(iteration) ˆbj (v k )= t =v k P T 5: method Map(sample id, sequence x) i=1 6: Forward(x, ). cf. Section P6.2.2 T 1 7: Backward(x, ). âcf. ij Section = P 6.2.4 8: I new AssociativeArray. Initial state expectations T 1 9: for all q 2S do. Loop over states 10: I{q} 1 (q) 1(q) 11: O new AssociativeArray of AssociativeArray. Emissions 12: for t =1to x do. Loop over observations 13: for all q 2S do. Loop over states 14: O{q}{x t } O{q}{x t } + t (q) t(q) 15: t t +1 16: T new AssociativeArray of AssociativeArray. Transitions 17: for t =1to x 1 do. Loop over observations 18: for all q 2S do. Loop over states 19: for all r 2S do. Loop over states 20: T {q}{r} T {q}{r} + t (q) A q (r) B r (x t+1 ) t+1(r) 21: t t +1 22: Emit(string initial, stripe I) 23: for all q 2S do. Loop over states 24: Emit(string emit from + q, stripe O{q}) 25: Emit(string transit from + q, stripe T {q}) t(j) t(j) t=1 t(i, j) t=1 P N j=1 t(i, j) t(j) = t(j) t (j) P (O ) t (i, j) = t(i)a ij b j (o t+1 ) t+1 (j) P (O )

MapReduce Implementation: Reducer 1: class Combiner 2: method Combine(string t, stripes [C 1,C 2,...]) 3: C f new AssociativeArray 4: for all stripe C 2 stripes [C 1,C 2,...] do 5: Sum(C f,c) 6: Emit(string t, stripe C f ) 1: class Reducer 2: method Reduce(string t, stripes [C 1,C 2,...]) 3: C f new AssociativeArray 4: for all stripe C 2 stripes [C 1,C 2,...] do 5: Sum(C f,c) 6: z 0 7: for all hk, vi 2C f do 8: z z + v 9: P f new AssociativeArray. Final parameters vector 10: for all hk, vi 2C f do 11: P f {k} v/z 12: Emit(string t, stripe P f ) ˆbj (v k )= â ij = P T i=1\o t =v k P T i=1 t(j) t(j) P T 1 t=1 t(i, j) P T 1 t=1 P N j=1 t(i, j) t(j) = t(j) t (j) P (O ) t (i, j) = t(i)a ij b j (o t+1 ) t+1 (j) P (O )

Finite State Machines Q: a finite set of N states l Q = {q 0, q 1, q 2, q 3, } l The start state: q 0 l The set of final states: q F Σ: a finite input alphabet of symbols δ(q,i): transition function l Given state q and input symbol i, transition to new state q'

Finite number of states

Transitions

Input alphabet

Start state

Final state(s)

What can we do with FSMs? Generate valid sequences Accept valid sequences

Weighted FSMs FSMs treat all transitions as equally likely What if we know more about state transitions? l a is twice as likely to be seen in state 1 as b or c l c is three times as likely to be seen in state 2 as a 2 3 1 1 1 1 What do we get out of it? l score( ab ) = 2 l score( bc ) = 3

Probabilistic FSMs = Markov Chains Let s replace weights with probabilities What if we know more about state transitions? l a is twice as likely to be seen in state 1 as b or c l c is three times as likely to be seen in state 2 as a 0.5 0.75 0.25 0.25 1.0 0.25 What do we get out of it? l P( ab ) = 0.5 l P( bc ) = 0.1875

Specifying Markov Chains Q: a finite set of N states l Q = {q 0, q 1, q 2, q 3, } The start state l An explicit start state: q 0 l Alternatively, a probability distribution over start states: {π 1, π 2, π 3, }, Σ π i = 1 The set of final states: q F N N Transition probability matrix A = [a ij ] l a ij = P(q j q i ), Σ a ij = 1 i 0.5 0.75 0.25 0.25 0.25 1.0

Let s model the stock market 0.2 Each state corresponds to a physical state in the world What s missing? Add priors 0.5 0.3 What s special about this FSM? l Present state only depends on the previous state! The (1st order) Markov assumption P (q i q 1...q i 1 )=P (q i q i 1 )

Are states always observable? Day: 1 2 3 4 5 6 Bull Bear S Bear Not observable! S Bull Here s what you actually observe: Bull: Bull Market Bear: Bear Market S: Static Market : Market is up : Market is down : Market hasn t changed

Hidden Markov Models Markov chains aren t enough! l What if you can t directly observe the states? l We need to model problems where observations don t directly correspond to states Solution: Hidden Markov Model (HMM) l Assume two probabilistic processes l Underlying process (state transition) is hidden l Second process generates sequence of observed events

Specifying HMMs An HMM =(A, B, ) is characterized by: l N states: Q = {q 1,q 2,...q N } l N x N Transition probability matrix A =[a ij ] X a ij = p(q j q i ) a ij =1 8i l V observation symbols: O = {o 1,o 2,...o V } l N x V Emission probability matrix B =[b iv ] j b iv = b i (o v )=p(o v q i ) l Prior probabilities vector NX i =1 i=1 =[ i, 2,... N ]

Stock Market HMM States? Transitions? Vocabulary? Emissions? Priors?

Stock Market HMM π 1 =0.5 π 2 =0.2 π 3 =0.3 States? Transitions? Vocabulary? Emissions? Priors?

Properties of HMMs Markov assumption: P (q i q 1...q i 1 )=P (q i q i 1 ) Output independence: P (o i q 1...q i...q T,o 1...o i...o T )=P (o i q i ) Remember, these are distinct: l Number of states (N) l The number of distinct observation symbols (V) l The length of the sequence (T)

Running an HMM 1. Set t = 1, select initial state based on 2. Select an observation to emit based on 3. If t = T, then done =[ i, 2,... N ] B =[b iv ] 4. Select transition into a new state based on A =[a ij ] 5. Set t = t + 1 6. Goto step 2

HMMs: Three Problems Likelihood: Given an HMM λ and a sequence of observed events O, find the likelihood of observing the sequence Decoding: Given an HMM λ and an observation sequence O, find the most likely (hidden) state sequence Learning: Given a set of observation sequences, learn the parameters A and B for λ Okay, but where did the structure of the HMM come from?

HMM Problem #1: Likelihood

Computing Likelihood π 1 =0.5 π 2 =0.2 π 3 =0.3 t 1 2 3 4 5 6 O Given this model of the stock market, how likely are we to observe the sequence of outputs?

Computing Likelihood Easy, right? l Sum over all possible ways in which we could generate O from λ l What s the problem? Right idea, wrong algorithm! Takes O(N T ) time to compute!

Computing Likelihood What are we doing wrong? l State sequences may have a lot of overlap l We re recomputing the shared subsequences every time l Let s store intermediate results and reuse them Sounds like a job for dynamic programming!

Forward Algorithm Forward probability l Probability of being in state j after t observations t (j) =P (o 1,o 2...o t,q t = j ) Build an N T trellis l Intuition: forward probability only depends on the previous state and observation

Forward Algorithm t (j) =P (o 1,o 2...o t,q t = j ) Initialization 1 (j) = j b j (o i ) 1 apple j apple N Recursion t (j) = Termination P (O )= NX t 1 (i)a ij b j (o t ) 1 apple j apple N,2 apple t apple T i=1 NX T (i) i=1 q 1 a 1j q 2 a 2j q 3 a 3j q j.... q N t 1 (i) a Nj t (j)

Forward Algorithm O = find P (O )

Forward Algorithm Static states Bear Bull t=1 t=2 t=3 time

Forward Algorithm: Initialization 1 (j) = j b j (o i ) 1 apple j apple N Static α 1 (Static) 0.3 0.3 =0.09 states Bear α 1 (Bear) 0.5 0.1 =0.05 Bull α 1 (Bull) 0.2 0.7=0.14 t=1 t=2 t=3 time

Forward Algorithm: Recursion t (j) = NX t 1 (i)a ij b j (o t ) 1 apple j apple N,2 apple t apple T i=1 Static 0.3 0.3 =0.09 states Bear 0.5 0.1 =0.05 X... and so on Bull 0.2 0.7=0.14 0.14 0.6 0.1=0.0084 0.0145 α 1 (Bull) a BullBull b Bull ( ) t=1 t=2 t=3 time

Forward Algorithm: Recursion Static 0.3 0.3 =0.09 0.0249 0.006477 states Bear 0.5 0.1 =0.05 0.0312 0.001475 Bull 0.2 0.7=0.14 0.0145 0.024 t=1 t=2 t=3 time

Forward Algorithm: Termination P (O )= NX T (i) i=1 Static 0.3 0.3 =0.09 0.0249 0.006477 states Bear 0.5 0.1 =0.05 0.0312 X 0.001475 Bull 0.2 0.7=0.14 0.0145 0.024 t=1 t=2 t=3 time P(O) = 0.03195

HMM Problem #2: Decoding

Decoding π 1 =0.5 π 2 =0.2 π 3 =0.3 t 1 2 3 4 5 6 O Given this model of the stock market, what are the most likely states the market went through to produce O?

Decoding Decoding because states are hidden First try: l Compute P(O) for all possible state sequences, then choose sequence with highest probability l What s the problem here? Second try: l For each possible hidden state sequence, compute P(O) using the forward algorithm l What s the problem here?

Viterbi Algorithm Decoding = computing most likely state sequence l Another dynamic programming algorithm l Efficient: polynomial vs. exponential Same idea as the forward algorithm l Store intermediate computation results in a trellis l Build new cells from existing cells

Viterbi Algorithm Viterbi probability l Probability of being in state j after seeing t observations and passing through the most likely state sequence so far v t (j) = Build an N T trellis max P (q 0,q 1...q t 1,o 1,o 2...o t,q t = j ) q 0,q 1,...q t 1 l Intuition: forward probability only depends on the previous state and observation

Viterbi vs. Forward Maximization instead of summation over previous paths This algorithm is still missing something! l In forward algorithm, we only care about the probabilities l What s different here? We need to store the most likely path (transition): l Use backpointers to keep track of most likely transition l At the end, follow the chain of backpointers to recover the most likely state sequence

Viterbi Algorithm: Formal Definition Initialization v 1 (j) = j b j (o 1 ) Bt 1 (j) =; 1 apple j apple N 1 apple j apple N Recursion v t (j) = Bt t (j) = arg Termination P = q T = arg N max i=1 [v t 1(i)a ij ]b j (o t ) 1 apple j apple N,2 apple t apple T N max j=1 v T (j) N max i=1 [v t 1(i)a ij ] 1 apple j apple N,2 apple t apple T N max j=1 v T (j) Why no b()?

Viterbi Algorithm O = find most likely state sequence

Viterbi Algorithm Static states Bear Bull t=1 t=2 t=3 time

Viterbi Algorithm: Initialization v 1 (j) = j b j (o 1 ) 1 apple j apple N Bt 1 (j) =; 1 apple j apple N Static v 1 (Static) 0.3 0.3 =0.09 states Bear v 1 (Bear) 0.5 0.1 =0.05 Bull v 1 (Bull) 0.2 0.7=0.14 t=1 t=2 t=3 time

Viterbi Algorithm: Recursion v t (j) = Bt t (j) = arg N max i=1 [v t 1(i)a ij ]b j (o t ) 1 apple j apple N,2 apple t apple T N max i=1 [v t 1(i)a ij ] 1 apple j apple N,2 apple t apple T Static 0.3 0.3 =0.09 states Bear 0.5 0.1 =0.05 Max Bull 0.2 0.7=0.14 0.14 0.6 0.1=0.0084 0.0084 v 1 (Bull) a BullBull b Bull ( ) t=1 t=2 t=3 time

Viterbi Algorithm: Recursion v t (j) = Bt t (j) = arg N max i=1 [v t 1(i)a ij ]b j (o t ) 1 apple j apple N,2 apple t apple T N max i=1 [v t 1(i)a ij ] 1 apple j apple N,2 apple t apple T Static 0.3 0.3 =0.09 states Bear 0.5 0.1 =0.05 store backpointer... and so on Bull 0.2 0.7=0.14 0.0084 t=1 t=2 t=3 time

Viterbi Algorithm: Recursion Static 0.3 0.3 =0.09 0.0135 0.00202 states Bear 0.5 0.1 =0.05 0.0168 0.000504 Bull 0.2 0.7=0.14 0.0084 0.00588 t=1 t=2 t=3 time

Viterbi Algorithm: Termination P = q T = arg N max j=1 v T (j) N max j=1 v T (j) Static 0.3 0.3 =0.09 0.0135 0.00202 states Bear 0.5 0.1 =0.05 0.0168 0.000504 Bull 0.2 0.7=0.14 0.0084 0.00588 t=1 t=2 t=3 time

Viterbi Algorithm: Termination Static 0.3 0.3 =0.09 0.0135 Most likely state sequence: [ Bull, Bear, Bull ], P = 0.00588 0.00202 states Bear 0.5 0.1 =0.05 0.0168 0.000504 Bull 0.2 0.7=0.14 0.0084 0.00588 t=1 t=2 t=3 time

HMM Problem #3: Learning

The Problem Given training data, learn parameters for the HMM =(A, B, ) l A: transition probabilities l B: emission probabilities Two cases: l Supervised training: sequences where observations are paired with states l Unsupervised training: sequences with observations only

Supervised Training Easy! Just like relative frequency counting Compute maximum likelihood estimates directly from data Transition probabilities: E[a ij ]= C(i! j) P j 0 C(i! j 0 ) Emission probabilities: E[b j (v)] = C(j " v) P v 0 C(j " v 0 )

Supervised Training Easy! Just like relative frequency counting Remember the usual MapReduce tricks: l Pairs vs. stripes for keep track of joint counts l Combiners and in-mapper combining apply

Unsupervised Training Hard! Actually, it s not that bad l Pseudo-counts instead of counts l More complex bookkeeping

Unsupervised Training What s the fundamental problem? Chick-and-egg problem: l If we knew what the hidden variables were, then computing the model parameters would be easy l If we knew the model parameters, we d be able to compute the hidden variables (but of course, that s the whole point) Where have we seen this before?

EM to the Rescue! Start with initial parameters Iterate until convergence: l E-step: Compute posterior distribution over latent (hidden) variables given the model parameters l M-step: Update model parameters using posterior distribution computed in the E-step How does this work for HMMs? First, we need one more definition

Backward Algorithm Backward probability l Probability of seeing observations from time t+1 to T given that we re in state j at time t t(j) =P (o t+1,o t+2...o T q t = i, ) Guess what? Build an N T trellis

Backward Algorithm t(j) =P (o t+1,o t+2...o T q t = j, ) Initialization T (j) =1 1apple j apple N Recursion t(j) = Termination P (O )= NX a ji b i (o t+1 ) t+1 (i) i=1 NX j b j (o 1 ) 1 (j) j=1 1 apple j apple N,1 apple t<t a j1 a j2 a j3 q j q 1 q 2 q 3 a jn.... q N t(j) t+1(i)

Backward Algorithm T (j) =1 1apple j apple N Static states Bear Bull t=1 t=2 t=3 time

Backward Algorithm: Initialization Static 1.0 states Bear 1.0 Bull 1.0 t=1 t=2 t=3 time

Backward Algorithm: Recursion t(j) = NX a ji b i (o t+1 ) t+1 (i) i=1 1 apple j apple N,1 apple t<t Static 0.0135 1.0 states Bear... and so on 1.0 Bull 1.0 t=1 t=2 t=3 time

Backward Algorithm: Recursion Static 0.1124 0.44 1.0 states Bear 0.1306 0.44 1.0 Bull 0.1092 0.5 1.0 t=1 t=2 t=3 time

Forward-Backward Forward probability l Probability of being in state j after t observations t (j) =P (o 1,o 2...o t,q t = j ) Backward probability l Probability of seeing observations from time t+1 to T given that we re in state j at time t t(j) =P (o t+1,o t+2...o T q t = i, ) Neat Observation: P (O )= NX t (j) t (j) j=1

Forward-Backward P (O )= NX t (j) t (j) j=1 Static 0.09 0.1124 0.0249 0.44 0.006477 1.0 states Bear 0.05 0.1306 0.0312 0.44 0.001475 1.0 Bull 0.14 0.1092 0.0145 0.5 0.024 1.0 P (O )= t=1 t=2 t=3 NX j b j (o 1 ) 1 (j) j=1 time

Forward-Backward t (j) =P (o 1,o 2...o t,q t = j ) t(j) =P (o t+1,o t+2...o T q t = i, ).... q j.... t (j) t(j) o t 1 o t o t+1

Estimating Emissions Probabilities Basic idea: b j (v k ) = expected number of times in state j and observing symbol v k expected number of times in state j Let s define: Thus: t(j) = P (q t = j, O ) P (O ) ˆbj (v k )= P T i=1\o t =v k P T i=1 t(j) = t(j) t (j) P (O ) t(j)

Forward-Backward a ij b j (o t+1 ).... q i q j.... t (i) t+1(j) o t 1 o t o t+1 o t+2

Estimating Transition Probabilities Basic idea: a ij = expected number of transitions from state i to state j expected number of transitions from state i Let s define: Thus: t (i, j) = t(i)a ij b j (o t+1 ) t+1 (j) P (O ) â ij = P T 1 t=1 t(i, j) P T 1 t=1 P N j=1 t(i, j)

Implementation Notes Standard setup of iterative MapReduce algorithms l Driver program sets up MapReduce job l Waits for completion l Checks for convergence l Repeats if necessary Additional details: l Must load and serialize model parameters at each iteration l Reducer receives 2N + 1 keys, so limits to reducer parallelization l Iteration overhead less than other iterative algorithms

Questions? Source: Wikipedia (Japanese rock garden)

Midterm

Final Projects Anything in the big data space Teams of 2-3 people Available data resources Available cluster resources Project Ideas?