Introduction to Finite Automaton
|
|
- Ashley Sharp
- 5 years ago
- Views:
Transcription
1 Lecture 1 Introduction to Finite Automaton IIP-TL@NTU Lim Zhi Hao 2015
2 Lecture 1 Introduction to Finite Automata (FA) Intuition of FA Informally, it is a collection of a finite set of states and state transitions, where each transition has at least one label. Figure: Some examples of Finite Automata
3 Lecture 1 Introduction to Finite Automata (FA) Applications of FA Applications of FA Finding all occurrences of pattern 10 in a binary string start x 0 1 x 1 0 x 2 0 Consider 3 binary strings: X X
4 Lecture 1 Introduction to Finite Automata (FA) Applications of FA Applications of FA A Bot is a character in a game that is controlled by the computer (Not NPC!). Coding Bot behaviours in games Powerful enemy start Weak enemy flee roam fight Escaped Enemy escapes/dies And many other applications as well.
5 Lecture 1 Introduction to Finite Automata (FA) Definition of FA Finite Automata (Definition) 5Tuplewrittenas(Q, A, E, I, F ) Where Q: Finite set of states A: Alphabet, or a formal language E: Finite set of state transitions I : Set of Initial states F : Set of Final states
6 Lecture 1 Introduction to Finite Automata (FA) Definition of FA Worked Example Let C =(Q, A, E, I, F ) Where Q = {1, 2}, I = {1}, F = {2}, A = {a, b} and E = {(1, a, 1), (1, a, 2), (2, b, 1), (2, b, 2)} Solution:
7 Lecture 1 Introduction to Finite Automata (FA) Paths in FA Intuition of Path Note that this is a series of directed edges, = e 1 e n An Example 1 =(1, a, 1), (1, a, 2), (2, b, 2), (2, b, 1), (1, a, 2), (2, b, 2) is a path
8 Lecture 1 Introduction to Finite Automata (FA) Paths in FA Intuition of Path Another Example: (Bot Behaviour) Powerful enemy start Weak enemy flee roam fight Escaped Enemy escapes/dies 1 = (roam, Powerful Enemy, flee), (flee, Escaped, roam) is a path
9 Lecture 1 Introduction to Finite Automata (FA) Paths in FA Definition of Paths in FA Given a Path: = e 1 e n Origin or Previous state: p[ ], e.g p[ ] =e 1 Destination or Next state: n[ ], e.g n[ ] =e n Input Label: i( ) Output Label: o( ) Set of Paths (Definition) P(R 1, R 2 ): Set of paths from R 1 Q to R 2 Q P(R 1, x, R 2 ): With Input label x 2 P(R 1, x, y, R 2 ): With Output label y 2
10 Lecture 1 Introduction to Finite Automata (FA) Paths in FA Summary/Recap What is a Finite Automaton? Definition of Finite Automaton. Definition of Paths and Sets of Paths
11 Lecture 1 Appendix References References I Jean-Eric Pin Mathematical Foundations of Automata Theory. Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers.
12 Lecture 2 Introduction to Finite State Transducers IIP-TL@NTU Lim Zhi Hao 2015
13 Lecture 2 Introduction to FSTs and WFSTs Introduction to FST Intuition of FST FST is an extension of FA, where it has both an input and output Eg: Input abbcd, thefstwillreturnzyyxw. Why?
14 Lecture 2 Introduction to FSTs and WFSTs Introduction to FST FST: Formal Definition Finite State Transducer (Definition): 6 Tuple written as (,, Q, E, I, F ) : Set of input labels : Set of output labels Q: Finite set of states E: Finite set of state transitions E Q { [ "} { [ "} Q I : Set of Initial states F : Set of Final states
15 Lecture 2 Introduction to FSTs and WFSTs Introduction to FST What is Q { [ "} { [ "} Q? Consider a walk, that goes from 0! 1! 2! 5 Formally, the path is written as: 1 =(0, a, z, 1), (1, c, x, 2), (2, d, w, 5) Notice each 4 tuple has a format of Q { [ "} { [ "} Q?
16 Lecture 2 Introduction to FSTs and WFSTs Introduction to WFST Intuition of WFST WFST is a weighted FST, each transition, initial state and final state has an additional weight associated to them. Eg: Input abbcd, thewfstwillreturnzyyxw and the weight of the path,
17 Lecture 2 Introduction to FSTs and WFSTs Introduction to WFST Weighted Finite State Transducer (Definition): 8 Tuple written as (,, Q, E, I, F,, ) : Set of input labels : Set of output labels Q: Finite set of states E: Finite set of state transitions E Q { [ "} { [ "} K Q I : Set of Initial states F : Set of Final states : Initial weight function ( : I! K) : Final weight function ( : F! K)
18 Lecture 2 Introduction to FSTs and WFSTs Introduction to WFST What is Q { [ "} { [ "} K Q? Consider the same walk as before: 0! 1! 2! 5 Formally, the path is written as: 2 =(0, a, z, 1.2, 1), (1, c, x, 3, 2), (2, d, w, 2, 5) Each 5 tuple has a format of Q { [ "} { [ "} K Q
19 Lecture 2 Introduction to FSTs and WFSTs Introduction to WFST Di erences in FST and WFST
20 Lecture 2 Introduction to FSTs and WFSTs Introduction to WFST Summary on Types of FA
21 Lecture 2 Introduction to FSTs and WFSTs Basic Properties of Transducers Basic Properties of Transducers Given T :(,, Q, E, I, F,, ) andx 2, y 2 Deterministic: T has no two transitions leaving the same state sharing the same input label Sequential: T has a unique initial state Functional: T has a unique y for any x Epsilon-Transitions: 9 a state transition that can be made without any input (") Stochastic WFST: For every state, the sum of all transitions leaving that state is EXACTLY 1
22 Lecture 2 Introduction to FSTs and WFSTs Basic Properties of Transducers Summary/Recap What is a Finite State Transducer? Definition of Weighted Finite State Transducers. Di erent types of Finite Automaton. Basic Properties of Transducers
23 Lecture 2 Appendix References References I Jean-Eric Pin Mathematical Foundations of Automata Theory. Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers.
24 Lecture 3 Introduction to Semirings I IIP-TL@NTU Lim Zhi Hao 2015
25 Lecture 3 Introduction to Semirings Why study Semirings? Motivation and Importance I Semirings are a type of Algebraic Structure In order to manipulate WFSTs, we need to learn the concept of semirings. In WFST-based speech recognition, the tropical semiring is mainly used In some optimization steps for WFSTs, the log semiring is also used The operations on WFSTs will be introduced in detail in the following sections.
26 Lecture 3 Introduction to Semirings Why study Semirings? Motivation and Importance II Reasons Computer Scientists are interested in semirings: Reduce problem complexity Enable Generic Problem Solving Reasons Mathematicians are interested in semirings: Semirings are fundamentally di erent from fields New areas of research thanks to applications in CS
27 Lecture 3 Introduction to Semirings What are Semirings? What are Semirings: Algebra revisited I Base on your own knowledge, these should look familiar to you: 1 + (2 + 3) = (1 + 2) = =3 1+( 1) = 0 2 (3 + 4) = (2 3) + (2 4) (3 + 4) 2=(3 2) + (4 2) 2 (3 4) = (2 3) 4 2 5= = =1 In pure mathematics, there is a rigorous study and extension of such algebraic structures, known as Abstract Algebra
28 Lecture 3 Introduction to Semirings What are Semirings? What are Semirings: Algebra revisited II Rewriting into classical algebra, given x, y, z 2 R: x +(y + z) =(x + y)+z x + y = y + x x +0=x x (y z) =(x y) z x y = y x x 1=x x +( x) =0 x x 1 =1, x 6= 0 z (x + y) =(z x)+(z y) (x + y) z =(x z)+(y z) All of the properties each have their unique names.
29 Lecture 3 Introduction to Semirings What are Semirings? What are Semirings: Algebra revisited III Addition: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 0 : x 0 =x Inverse, x : x ( x) = 0 Distributivity of Multiplication over Addition: z (x y) =(z x) (z y) (x y) z =(x z) (y z) Multiplication: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 1 : x 1 =x Inverse, x 1 : x x 1 = 1 The algebraic structure in the above case is known as a FIELD. Recall POLYNOMIALS
30 Lecture 3 Introduction to Semirings What are Semirings? What are Semirings: Algebra revisited IV Addition: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 0 : x 0 =x Inverse, x : x ( x) = 0 Multiplication: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 1 : x 1 =x Inverse Distributivity of Multiplication over Addition: z (x y) =(z x)+(z y) (x y) z =(x z) (y z) The algebraic structure in the above case is known as a RING. Recall SQUARE MATRICES
31 Lecture 3 Introduction to Semirings What are Semirings? What are Semirings: Algebra revisited V Addition: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 0 : x 0 =x Inverse Inverse Distributivity of Multiplication over Addition: z (x y) =(z x)+(z y) (x y) z =(x z) (y z) Multiplication: Associativity: x (y z) =(x y) z Commutativity: x y = y x Identity, 1 : x 1 =x The algebraic structure above is known as a SEMIRING. Recall PROBABILITY
32 Lecture 3 Introduction to Semirings What are Semirings? Summary/Recap Informally: Fields Rings Semirings
33 Lecture 3 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers.
34 Lecture 4 Introduction to Semirings II IIP-TL@NTU Lim Zhi Hao 2015
35 Lecture 4 Formal Definition of Semirings Semiring (Definition): 5Tuplewrittenas(K,,, 0, 1) Where K: A set containing some mathematical objects : Addition operation : Multiplication operation 0: Additive Identity. Note: 0 2 K 1: Multiplicative Identity. Note: 1 2 K
36 Lecture 4 Why study Semirings? -Addition: Compute the weight of a sequence Associativity: x (y z) =(x y) z Commutativity: x y = y x Additive Identity: x 0 =x -Multiplication: Compute the weight of a path Associativity: x (y z) =(x y) z Distributivity: z (x y) =(z x) (z y) (x y) z =(x z) (y z) Multiplicative Identity: x 1 = x
37 Lecture 4 Why study Semirings? Why study Semirings? In most areas that uses mathematics, understanding how a field works is more than enough. However, when it comes to manipulating WFSTs, a knowledge of semirings is required. Also useful for Generic problem solving
38 Lecture 4 Why study Semirings? Shortest Path from S to T Step 1, Compute: = = = 11 Step 2, Then: min{13, 12, 11} = 11
39 Lecture 4 Why study Semirings? Maximum Reliability from S to T Step 1, Compute: = = =0.567 Step 2, Then: max{0.32, 0.126, 0.567} = 0.567
40 Lecture 4 Why study Semirings? Language leading from S to T in the Automaton Step 1, Compute: {a} {t} = {at} {t} {h} {e} = {the} {t} {i} {m} {e} = {time} Step 2, Then: S {{at}, {the}, {time}} = {at, the, time}
41 Lecture 4 Why study Semirings? Generic Problem Solving These problems were vastly di erent, but at the core, they are actually very much the same problem. Consider this: min{(9 + 4), ( ), ( )} max{( ), ( ), ( )} S {{a} {t}, {t} {h} {e}, {t} {i} {m} {e}} Is there a more general way of looking at the expressions above?
42 Lecture 4 Why study Semirings? Generic Problem Solving If we were to replace the inner operations with and the outer operation with L {{w1 w 2, {w {z } 3 w 4 w 5, {w {z } 3 w 6 w 7 w 5 } {z } w( 1 ) w( 2 ) w( 3 ) Even more compactly: L 2P(S,T ) w( )
43 Lecture 4 Why study Semirings? Summary/Recap Formal Definition Of Semiring? Why Study Semirings? Generic Problem Solving.
44 Lecture 4 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers.
45 Lecture 5 Introduction to Semirings III IIP-TL@NTU Lim Zhi Hao 2015
46 Lecture 5 Examples Of Semirings Common Examples of Semirings Probability: ([0, 1], +,, 0, 1) Strings: ( [ s 1, ^,, s 1,"), where : A finite set of strings s 1 :InfiniteString ^: Lowest common prefix [{abcde} ^{bcf } = {bc}] : Concatenation of 2 strings Tropical: (R, min, +, 1, 0), where min: The minimum function Log: (R, LOG, +, 1, 0), where x LOG y = log(e x + e y )
47 Lecture 5 Tropical Semirings Proof of Tropical Semiring Tropical Semiring (Definition): 5Tuplewrittenas(R, min, +, 1, 0) (K,,, 0, 1) Addition: Associativity: x (y z) =min(x, min(y, z)) = min(min(x, y), z) =(x y) z Commutativity: Additive Identity: x y = min(x, y) =min(y, x) =y x x 0 =min(x, 1) =x
48 Lecture 5 Tropical Semirings Proof of Tropical Semiring Tropical Semiring (Definition): 5Tuplewrittenas(R, min, +, 1, 0) (K,,, 0, 1) Multiplication: Associativity: x (y z) =x +(y + z) =(x + y)+z =(x y) z Multiplicative Identity: x 1 =x +0=x
49 Lecture 5 Tropical Semirings Proof of Tropical Semiring Tropical Semiring (Definition): 5Tuplewrittenas(R, min, +, 1, 0) (K,,, 0, 1) Left Distributivity of Multiplication, WLOG suppose x apple y: z (x y) =z + min(x, y) =z + x (z x) (z y) =min(z + x, z + y) =z + x = z (x y) Right Distributivity of Multiplication (x y) z = min(x, y)+z = x + z (x z) (y z) =min(x + z, y + z) =x + z =(x y) z
50 Lecture 5 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers.
51 Lecture 6 Introduction to Semirings IV IIP-TL@NTU Lim Zhi Hao 2015
52 Lecture 6 Log Semirings Proof of Log Semiring Log Semiring (Definition): 5Tuplewrittenas(R, LOG, +, 1, 0) (K,,, 0, 1) Associativity of Addition ( x LOG y = log(e x + e y )) x LOG (y LOG z) = log(e x + e (y LOG z) ) = log(e x + e ( log(e y +e z )) ) = log(e x + e log(e y +e z ) ) = log(e x + e y + e z ) = log(e log(e x +e y ) + e z ) = log(e (x LOG y) + e z ) =(x LOG y) LOG z
53 Lecture 6 Log Semirings Proof of Log Semiring Log Semiring (Definition): 5Tuplewrittenas(R, LOG, +, 1, 0) (K,,, 0, 1) Addition Commutativity: x LOG y = log(e x + e y )= log(e y + e x )=y LOG x Additive Identity: x LOG 0 = log(e x + e 1 )= log(e x )=x Multiplication: Associativity: x (y z) =x +(y + z) =(x + y)+z =(x y) z Additive Identity: x 1 =x +0=x
54 Lecture 6 Log Semirings Proof of Log Semiring Log Semiring (Definition): 5Tuplewrittenas(R, LOG, +, 1, 0) (K,,, 0, 1) Left Distributivity of Multiplication: z (x LOG y) =z log(e x + e y ) = log(e z ) log(e x + e y ) = log(e z e x + e z e y ) = log(e (z+x) + e (z+y) ) =(z + x) LOG (z + y) =(z x) LOG (z y) Proof of Right Distributive property is similar
55 Lecture 6 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers.
56 Lecture 7 Introduction to Semirings V IIP-TL@NTU Lim Zhi Hao 2015
57 Lecture 7 Special Cases of Semirings Special Cases of Semirings Recall that in some optimization steps for WFSTs, the addition and multiplication used are defined by the semirings Hence, it is good to know some additional types of semirings, so as to simplified some of the computations (Short-cuts) Commutative Semirings Idempotent Semirings k-closed Semirings Weakly Left Divisible Semirings ZeroSum Free Semirings
58 Lecture 7 Special Cases of Semirings Special Cases of Semirings Commutative Semiring A Semiring is a Commutative Semiring if is commutative as well Tropical and Log are Commutative Semirings The commutative property holds for classical addition Idempotent Semiring A Semiring is an Idempotent Semiring if x x = x, 8x 2 K Tropical and String are Idempotent Semirings Tropical: x x = min(x, x) =x String: x x = x ^ x = x
59 Lecture 7 Special Cases of Semirings Special Cases of Semirings k-closed Semiring 8a 2 K: Leta n = a a {z } and a 0 = 1 ntimes ASemiringisak-closed Semiring if L k n=0 an = L k+1 n=0 an Or less compactly: 1 a a k = 1 a a k a k+1 Positive Tropical Semirings are 0-closed: 1 = 1 1 =0=min{0, a} = 1 a, 8a 2 R + String Semirings are also 0-closed 1 =" = " ^ a = 1 a, 8a 2 a
60 Lecture 7 Special Cases of Semirings Special Cases of Semirings Weakly Left Divisible Semiring ASemiringisaWeaklyLeftDivisibleSemiringif 8(x y) 6= 0, 9z 2 K such that x =(x y) z Tropical and Log Semirings are also Weakly Left Divisible Semirings Tropical Semiring (R, min, +, 1, 0): z = x min(x, y) Log Semirings (R, LOG, +, 1, 0): z = log(1 + e x y )
61 Lecture 7 Special Cases of Semirings ZeroSum Free Semiring A Semiring is a ZeroSum Free Semiring if x x = y = 0 y = 0 implies Probability Semiring is a ZeroSum Free Semiring Proof: Suppose not. 9x, y 2 [0, 1] such that x + y =0but x 6= 0andy 6= 0. Then x = y with x 2 [0, 1]. This implies that y /2 (0, 1]. Contradiction These special cases will come into play in the following section, where we discuss the operations on WFSTs.
62 Lecture 7 Special Cases of Semirings Summary Of Special Cases When manupilating WFST s, some of these properties are used
63 Lecture 7 Special Cases of Semirings Sneak Peek on Operations Most WFST s are defined over a Semiring, that is, the operations are defined by the Semiring. and The next section will utilize some of the above mentioned special properties.
64 Lecture 7 Special Cases of Semirings Summary/Recap Tropical Semirings? Log Semirings? Special Cases of Semirings.
65 Lecture 7 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers.
66 Lecture 8 Introduction to Basic Operations for Finite Automaton IIP-TL@NTU Lim Zhi Hao 2015
67 Lecture 8 Operations for FA Basic Operations of FA Basic Operations WFST over semiring S =(K,,, 1, 0) (Definition): L T (x, y) = (p[ ]) w[ ] (n[ ]) 2P(I,x,y,F ) Recall that: L 2P(S,T ) w( ) The expression above is analogous to the definition of WFST over a Semiring. Figure: WFSA from Lecture 4 The extra (p[ ]) and (n[ ]) are the weights of the initial and final states respectively.
68 Lecture 8 Operations for FA Basic Operations of FA Given a WFSTs over a Tropical Semiring: (R, min, +, 1, 0) Consider 0 =(0, a, A, 1, 1), (1, c, B, 1, 3), x = {ac}, y = {AB} M T (x, y) = (p[ ]) w[ ] (n[ ]) 2P(I,x,y,F ) = (p[ 0 ]) w[ 0 ] (n[ 0 ]) = (0) (1 1) (3) = =2.5
69 Lecture 8 Operations for FA Basic Operations of FA Basic Operations WFST over semiring S =(K,, ) (Definition): L T (x, y) = (p[ ]) w[ ] (n[ ]) 2P(I,x,y,F ) Kleene closure: Union: Modifies the automaton so that the set of symbol sequences or transductions is sequentially repeated zero or more times Combines two automata in parallel Concatenation: Combines two automata in series
70 Lecture 8 Operations for FA Basic Operations of FA Basic Operations WFST over semiring S =(K,, ) (Definition): L T (x, y) = (p[ ]) w[ ] (n[ ]) Inverse: 2P(I,x,y,F ) Exchanging input and output symbols Reversal: Assigns to each pair of strings (x, y) what T assigns to its mirror images ( x, ỹ) Projection: Converting a transducer into an acceptor by omitting input or output labels
71 Lecture 8 Operations for FA Basic Operations of FA An example of a Kleene Closure Definition: T L (x, y) = 1 T n (x, y) n=0
72 Lecture 8 Operations for FA Basic Operations of FA An example of a Union Definition: (T A T B )(x, y) =T A (x, y) T B (x, y)
73 Lecture 8 Operations for FA Basic Operations of FA An example of a Concatenation Definition: L (T A T B )(x, y) = T A (x A, y A ) T B (x B, y B ) x=x A, x B ;y=y A, y B
74 Lecture 8 Operations for FA Basic Operations of FA An example of an Inverse Definition: T 1 (x, y) =T (y, x)
75 Lecture 8 Operations for FA Basic Operations of FA An example of a Reversal Definition: T (x, y) =T (ỹ, x)
76 Lecture 8 Operations for FA Basic Operations of FA An example of a Projection Definition: # T (x) = L y T (y, x)
77 Lecture 8 Operations for FA Basic Operations of FA Summary/Recap What is meant by WFST over a Semiring. Kleene Closure Union Concatenation Inverse Reversal Projection
78 Lecture 8 Operations for FA 2mainOperationsofWFSTs 2mainOperationsofWFSTs WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
79 Lecture 8 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers. Mehryar Mohri Weighted Automata Algorithms.
80 Lecture 9 Introduction to 5 Basic Operations for WFST: Composition IIP-TL@NTU Lim Zhi Hao 2015
81 Lecture 9 Composition WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
82 Lecture 9 Composition Rationale for Composition, an example T A converts a sequence of letters so that they are all uppercase letters T B converts the sequence of uppercase letters to a sequence of specific words that matches the letters T = T A T B converts a sequence of letters into a sequence of specific words The resulting transducer may be harder to construct from scratch So given a complicated transducer, it can be broken down into many simple transducers Composition simplifies construction of complicated transducers
83 Lecture 9 Composition Composition Combining two related transducers into one single transducer that represents a set of transductions cascaded with the original two transducers (Takaaki Hori,2013). Given 2 WFST over the Probability Semiring: We obtain:
84 Lecture 9 Composition Given 2 transducers: T A :(,, Q 1, E 1, I 1, F 1, 1, 1 ), T B :(,, Q 2, E 2, I 2, F 2, 2, 2 ) (T A T B )(x, y) = L T A (x, z) T B (z, y) z2
85 Lecture 9 Composition Composition I First iteration, consider (0 A, 0 B )
86 Lecture 9 Composition Composition II Second iteration, consider (1 A, 1 B )
87 Lecture 9 Composition Composition III Third iteration, consider (0 A, 1 B )
88 Lecture 9 Composition Composition IV Fourth iteration, consider (2 A, 1 B )
89 Lecture 9 Composition Composition V Fifth iteration, consider (3 A, 1 B )
90 Lecture 9 Composition Composition VI Sixth iteration, consider (3 A, 2 B )
91 Lecture 9 Composition Composition VII Seventh iteration, consider (3 A, 3 B )
92 Lecture 9 Composition Composition VIII We obtain the new transducer, T = T A node (3, 2) T B with an extra Dead
93 Lecture 9 Composition Composition IX Trimming away the node, we arrive at our previous result: (T A T B )(x, y) :(,, Q, E, I, F,, ) WhereQ Q 1 Q 2
94 Lecture 9 Composition Composition X May produce incorrect results for non-idempotent semirings as path-weights may be counted more than once Hence the Semiring defined necessarily needs to be idempotent. "-transitions create redundant paths during simple composition Solution is to utilize an intermediate filter transducer to eliminate the redundant paths
95 Lecture 9 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers. Mehryar Mohri Weighted Automata Algorithms. Josef R. Novak Weighted Finite-State Transducers Important Algorithms University of Tokyo, Minematsu Lab
96 Lecture 10 Introduction to 5 Basic Operations for WFST: Epsilon Removal IIP-TL@NTU Lim Zhi Hao 2015
97 Lecture 10 Epsilon-Removal WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
98 Lecture 10 Epsilon-Removal Introduction to Epsilon-Removal Removes "-transitions from the input transducer and produces a new, epsilon-free transducer equivalent to the original Figure: Before "-Removal Figure: After "-Removal
99 Lecture 10 Epsilon-Removal Rationale for Epsilon-Removal Various automata or transducer operations such as rational operations generate "-transitions These transitions cause some delay in the use of the resulting transducers Since the search requires reading some sequences of " s first To make these weighted transducers more e cient to use, it may be preferable to remove all "-transitions Figure: Before "-Removal Figure: After "-Removal
100 Lecture 10 Epsilon-Removal Intuition of Algorithm Figure: Part of Original FA Consider the following paths: 1 0! 1! 2! 4 Requires input {a} only 2 0! 1! 2! 5 Requires input {b} only 3 0! 1! 3 Requires input {c} only
101 Lecture 10 Epsilon-Removal Intuition of Algorithm Figure: "-Closure of FA Remove redundant "-transitions: 1 0! 1! 2! 4 0! 4 2 0! 1! 2! 5 0! 5 3 0! 1! 3 0! 3 This step is to calculate the "-closure of each state.
102 Lecture 10 Epsilon-Removal Description of Epsilon-Removal Algorithm Compute the weighted "-closure for each state in the FST. Weighted "-closure (Definition) C(p) ={(q, w) :P(p,",q) 6= ;, w = " [p, q]} L Where "[p, q] = w( ) 2P(p,",q) 8p 2 Q Remove all "-transitions and 8p 2 Q and each (q, w) 2 C(p), the transition set of p is augmented to: Augmenting all transitions associated at state p 1 {(p, a, b, "[p, q] w, r) :(q, a, b, w, r) 2 E, (a, b) 6= (", ")} 2 If 9(q, w) s.tq 2 F : w(p) w(p) ( " [p, q] (q))
103 Lecture 10 Epsilon-Removal Epsilon-Removal Worked Example I Weighted "-closure (Definition) C(p) ={(q, w) :P(p,",q) 6= ;, w = " [p, q]} L Where "[p, q] = w( ) 2P(p,",q) 8p 2 Q Figure: WFSA over a Tropical SR C(0) contains (1, w 1 )and(2, w 2 ) w 1 = " [0, 1] = 1 w 2 = " [0, 2] = 1 1=2 Thus C(0) = {(1, 1), (2, 2)}
104 Lecture 10 Epsilon-Removal Epsilon-Removal Worked Example II Augmenting all transitions associated at state 0 1 {(p, a, b, "[p, q] w, r) :(q, a, b, w, r) 2 E, (a, b) 6= (", ")} 2 If 9(q, w) s.tq 2 F : w(p) w(p) ( " [p, q] (q)) Figure: "-Closure of FA Consider all transitions that begin with 1 and 2, that DO NOT contain any "-transitions (2,a,2,4)! (0,a, " [0, 2] 2,4) (2,b,3,5)! (0,b, " [0, 2] 3,5) (1,c,4,3)! (0,c, " [0, 1] 4,3)
105 Lecture 10 Epsilon-Removal Epsilon-Removal Worked Example III Augmenting all transitions associated at state 0 1 {(p, a, b, "[p, q] w, r) :(q, a, b, w, r) 2 E, (a, b) 6= (", ")} 2 "[0, 2] =2and " [0, 1] =1 Replacing the 3 transitions with: (2,a,2,4)! (0,a,4,4) (2,b,3,5)! (0,b,5,5) (1,c,4,3)! (0,c,5,3) Figure: "-Closure of FA
106 Lecture 10 Epsilon-Removal Epsilon-Removal Worked Example IV Removing the states 1 and 2 yields: Figure: Before "-Removal Figure: After "-Removal The new transitions: (0,a,4,4), (0,b,5,5) and(0,a,4,3)
107 Lecture 10 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Speech Recognition With Weighted Finite-State Transducers. Mehryar Mohri Weighted Automata Algorithms. Josef R. Novak Weighted Finite-State Transducers Important Algorithms University of Tokyo, Minematsu Lab
108 Lecture 11 Introduction to 5 Basic Operations for WFST: Determinization IIP-TL@NTU Lim Zhi Hao 2015
109 Lecture 11 Determinization WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
110 Lecture 11 Determinization Introduction to Determinization NFA (Informal): At certain states, cannot determine where it will go to, hence the destination states is a powerset, instead of a single state For example: Given the input ab there are 2 possible states 4 or 6.
111 Lecture 11 Determinization Rationale of Determinization Availability of determinization is a big advantage as regards solving sequence recognition or transduction problems using FAs A deterministic FA (DFA) that is equivalent to the original FA Accelerates run-time speed as it yields the most e terms of speed cient FA in
112 Lecture 11 Determinization Description of Algorithm States are grouped based on the transitions input symbols Thus, the resulting transducer have UNIQUE inputs and UNIQUE paths Multiple final states are compressed into a single, weighted final state final emission function in each final state is replaced with epsilon transitions
113 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring First label initial state with weight 1. Then start by focusing on all the transitions with the same input. Compute total weight of given input a : w(e 1 )= 1 1=0+1=1 w(e 2 )= 1 2=0+2=2 w(e 1 ) w(e 2 )=min(1, 2) = 1 This is the new arc weight w(e 0 1 )=1
114 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Start by computing residual weights: w r (e i )=w(e 0 j ) 1 w(p[e i ]) w(e i ) w r (e 1 )= 1+0+1=0 w r (e 2 )= 1+0+2=1 n[e 1 ] = 1 and n[e 2 ] = 2 Group using a new notation: ((1, 0), (2, 1)), the new state Then the new transition is: {(0, 0), a, 1, ((1, 0), (2, 1))}
115 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Figure: After first update
116 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Now consider ((1, 0), (2, 1)). State 1 and 2 has residual weights 0 and 1 respectively. Compute total weight of given input b : w(e 3 )=0 3=0+3=3 w(e 4 )=1 3=1+3=4 w(e 3 ) w(e 4 )=min(3, 4) = 3 This is the new arc weight w(e 0 2 )=3
117 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Start by computing residual weights: w r (e i )=w(e 0 j ) 1 w(p[e i ]) w(e i ) w r (e 3 )= 3+0+3=0 w r (e 4 )= 3+1+3=1 n[e 3 ] = 1 and n[e 4 ] = 2 Group using a new notation: ((1, 0), (2, 1)), the new state Then the next transition is: {((1, 0), (2, 1)), b, 3, ((1, 0), (2, 1))}
118 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Figure: After second update
119 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Now consider ((1, 0), (2, 1)). State 1 and 2 has residual weights 0 and 1 respectively. Compute total weight of given input c : w(e 5 )=0 5=0+5=5 w(e 5 ) 0 =min(5, 1) =5 This is the new arc weight w(e 0 3 )=5
120 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Start by computing residual weights: w r (e i )=w(e 0 j ) 1 w(p[e i ]) w(e i ) w r (e 5 )= 5+0+5=0 n[e 5 ] = 3 Group using a new notation: (3, 0), the new state Then the next transition is: {((1, 0), (2, 1)), c, 5, (3, 0)}
121 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Figure: After third update
122 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Now consider ((1, 0), (2, 1)). State 1 and 2 has residual weights 0 and 1 respectively. Compute total weight of given input d : w(e 6 )=1 6=1+6=7 w(e 6 ) 0 =min(7, 1) =7 This is the new arc weight w(e 0 3 )=7
123 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Start by computing residual weights: w r (e i )=w(e 0 j ) 1 w(p[e i ]) w(e i ) w r (e 6 )= 7+1+6=0 n[e 6 ] = 3 Group using a new notation: (3, 0), the new state Then the next transition is: {((1, 0), (2, 1)), d, 7, (3, 0)}
124 Lecture 11 Determinization Determinization: Worked Example Figure: Original WFSA over Tropical Semiring Figure: After fourth update
125 Lecture 11 Determinization Determinization: Worked Example Figure: Determinized WFSA Some issues with weighted determinization: May not terminate for some transducers May not finish for some inputs, even if they are valid However, all FSAs are determinizable
126 Lecture 11 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Weighted Automata Algorithms.
127 Lecture 12 Introduction to 5 Basic Operations for WFST: Weight-Pushing IIP-TL@NTU Lim Zhi Hao 2015
128 Lecture 12 Weight-Pushing WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
129 Lecture 12 Weight-Pushing Introduction to Weight-Pushing Moves the weights distributed over all the paths to the initial states, without changing the initial WFST/WFSA Example: Given a WFST over a Tropical Semiring Figure: Original WFST, T Figure: Weight-Pushed WFST in Tropical Semiring, push(t )
130 Lecture 12 Weight-Pushing Rationale of Weight-Pushing Many sequence recognition or transduction problems reduces to finding the minimal cost path. As the unpromising paths can be eliminated in the early stage of the search, and therefore reducing the total search time. Figure: Weight-Pushed WFST in Tropical Semiring, push(t )
131 Lecture 12 Weight-Pushing Description Of Weight-Pushing Algorithm Given a WFST T :(,, Q, E, I, F,, ) over semiring S =(K,,, 0, 1) 1 Compute potential of each state, given by V [q] = L w( ) (n[ ]) 2 (q,f ) Where (q, F ) is the set of paths originating from state q In the event of an infinite loop, employ the k-closed property 2 Re-weighting the transition weights, initial weights and final weights in the following way: 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 8q 2 I, (q) (q) V [q] 8q 2 F, (q) V [q] 1 (q)
132 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) First compute potential of each state: V [q] = L w( ) (n[ ]) 2 (q,f )
133 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall Potential of state: V [q] = L 2 (q,f ) w( ) (n[ ]) Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) Step 1.1: State 0 has 2 paths going to the final state 3 1 = e 1, e 3 and 2 = e 2, e 4 w( 1 ) (n[ 1 ]) = (1 1) (0.5) = =2.5 w( 2 ) (n[ 2 ]) = (0 3) (0.5) = =3.5 Using the potential formula V [0] = =min(2.5, 3.5) = 2.5 Based on this Semiring, what are we doing?
134 Lecture 12 Weight-Pushing Recall: Shortest Path from S to T Step 1, Compute: = = = 11 Step 2, Then: min{13, 12, 11} = 11
135 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall Potential of state: V [q] = L 2 (q,f ) w( ) (n[ ]) Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) Step 1.2: State 1 has 1 path going to the final state 3 3 = e 3 w( 3 ) (n[ 3 ]) = (1) (0.5) = =1.5 Using the potential formula V [1] = =min(1.5, 1) =1.5
136 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall Potential of state: V [q] = L 2 (q,f ) w( ) (n[ ]) Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) Step 1.3: State 2 has 1 path going to the final state 3 4 = e 4 w( 4 ) (n[ 4 ]) = (3) (0.5) = =3.5 Using the potential formula V [2] = =min(3.5, 1) =3.5
137 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall Potential of state: V [q] = L w( ) (n[ ]) 2 (q,f ) Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) Step 1.4: State 3 is the final state, initialize V [3] = (3) = 0.5 Hence we obtain all the potentials in the transducer: V [0] = 2.5 V [2] = 3.5 V [1] = 1.5 V [3] = 0.5
138 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall: Re-weighting 1 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 2 8q 2 I, (q) (q) V [q] 3 8q 2 F, (q) V [q] 1 (q) 4 Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) V [0] = 2.5 V [1] = 1.5 V [2] = 3.5 V [3] = 0.5 Step 2.1: Using (2) and (3), re-weight initial and final states 0 and 3 (0) (0) V [0] = =0+2.5 =2.5 (3) V [3] 1 (3) = V [3] 1 V [3] = 1 =0
139 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Update Figure: After Update
140 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall: Re-weighting 1 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 2 8q 2 I, (q) (q) V [q] 3 8q 2 F, (q) V [q] 1 (q) 4 Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) V [0] = 2.5 V [1] = 1.5 V [2] = 3.5 V [3] = 0.5 Step 2.2: Using (1), re-weight the transition e 1 =(0, 1, 1) w[e 1 ] V [p[e 1 ]] 1 w[e 1 ] V [n[e 1 ]] V [0] 1 w[e 1 ] V [1] = =0 w[e 1 ] 0
141 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Update Figure: After Update
142 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall: Re-weighting 1 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 2 8q 2 I, (q) (q) V [q] 3 8q 2 F, (q) V [q] 1 (q) 4 Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) V [0] = 2.5 V [1] = 1.5 V [2] = 3.5 V [3] = 0.5 Step 2.3: Using (1), re-weight the transition e 2 =(0, 0, 2) w[e 2 ] V [p[e 2 ]] 1 w[e 2 ] V [n[e 2 ]] V [0] 1 w[e 2 ] V [2] = =1 w[e 2 ] 1
143 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Update Figure: After Update
144 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall: Re-weighting 1 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 2 8q 2 I, (q) (q) V [q] 3 8q 2 F, (q) V [q] 1 (q) 4 Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) V [0] = 2.5 V [1] = 1.5 V [2] = 3.5 V [3] = 0.5 Step 2.4: Using (1), re-weight the transition e 3 =(1, 1, 3) w[e 3 ] V [p[e 3 ]] 1 w[e 3 ] V [n[e 3 ]] V [1] 1 w[e 3 ] V [3] = =0 w[e 3 ] 0
145 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Update Figure: After Update
146 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Recall: Re-weighting 1 8e 2 E s.t V [p[e]] 6= 0, w[e] V [p[e]] 1 w[e] V [n[e]] 2 8q 2 I, (q) (q) V [q] 3 8q 2 F, (q) V [q] 1 (q) 4 Tropical Semiring: (R, min, +, 1, 0) (K,,, 0, 1) V [0] = 2.5 V [1] = 1.5 V [2] = 3.5 V [3] = 0.5 Step 2.5: Using (1), re-weight the transition e 4 =(2, 3, 3) w[e 4 ] V [p[e 4 ]] 1 w[e 4 ] V [n[e 4 ]] V [2] 1 w[e 4 ] V [3] = =0 w[e 4 ] 0
147 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Update Figure: After Update
148 Lecture 12 Weight-Pushing Worked Example (T over Tropical Semiring) Figure: Before Weight-Pushing, T Figure: After Weight-Pushing, push(t ) over a Tropical Semiring
149 Lecture 12 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers. Mehryar Mohri Weighted Automata Algorithms.
150 Lecture 13 Introduction to 5 Basic Operations for WFST: Minimization IIP-TL@NTU Lim Zhi Hao 2015
151 Lecture 13 Minimization WFST s Operations Composition Optimization Epsilon-Removal Determinization Weight-Pushing Minimization
152 Lecture 13 Minimization Introduction to Minimization Minimize the number of states for any DFA. Hence, given a transducer T, usually a determinization operation is performed, resulting in det(t) First, push weights and output labels to the initial states in the WFST Then, minimize the WFST using a classical minimization algorithm assuming that the triplet input:output/weight on each transition as one single label.
153 Lecture 13 Minimization Minimization Algorithm I For example, given a determinized WFST det(t ):
154 Lecture 13 Minimization Minimization Algorithm II Perform weight-pushing on det(t ): to obtain push(det(t ))
155 Lecture 13 Minimization Minimization Algorithm III Finally, apply any minimization algorithm on push(det(t )): And obtain min(push(det(t )))
156 Lecture 13 Minimization Rationale for Minimization Hopcrofts algorithm is an algorithm for e ciently minimizing DFAs. If WFST is acyclic, a more e cient algorithm, Revuzs algorithm, is used. By reducing the number of states, the search paths are greatly reduced, shortening search time.
157 Lecture 13 Minimization Recall Properties Required
158 Lecture 13 Applications Some Applications of WFSTs Speech Recognition Optical Character Recognition Text Classification Machine translation
159 Lecture 13 Appendix References References I Takaaki Hori Speech Recognition Algorithms Using Weighted Finite StateTransducers.
Weighted Finite-State Transducer Algorithms An Overview
Weighted Finite-State Transducer Algorithms An Overview Mehryar Mohri AT&T Labs Research Shannon Laboratory 80 Park Avenue, Florham Park, NJ 0793, USA mohri@research.att.com May 4, 004 Abstract Weighted
More informationAxioms of Kleene Algebra
Introduction to Kleene Algebra Lecture 2 CS786 Spring 2004 January 28, 2004 Axioms of Kleene Algebra In this lecture we give the formal definition of a Kleene algebra and derive some basic consequences.
More informationFinite-State Transducers
Finite-State Transducers - Seminar on Natural Language Processing - Michael Pradel July 6, 2007 Finite-state transducers play an important role in natural language processing. They provide a model for
More informationOpenFst: An Open-Source, Weighted Finite-State Transducer Library and its Applications to Speech and Language. Part I. Theory and Algorithms
OpenFst: An Open-Source, Weighted Finite-State Transducer Library and its Applications to Speech and Language Part I. Theory and Algorithms Overview. Preliminaries Semirings Weighted Automata and Transducers.
More informationKleene Algebras and Algebraic Path Problems
Kleene Algebras and Algebraic Path Problems Davis Foote May 8, 015 1 Regular Languages 1.1 Deterministic Finite Automata A deterministic finite automaton (DFA) is a model of computation that can simulate
More informationIntroduction to Finite-State Automata
Introduction to Finite-State Automata John McDonough Language Technologies Institute, Machine Learning for Signal Processing Group, Carnegie Mellon University March 26, 2012 Introduction In this lecture,
More informationFinite Automata. Finite Automata
Finite Automata Finite Automata Formal Specification of Languages Generators Grammars Context-free Regular Regular Expressions Recognizers Parsers, Push-down Automata Context Free Grammar Finite State
More informationFinite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018 Lecture 9 Ana Bove April 19th 2018 Recap: Regular Expressions Algebraic representation of (regular) languages; R, S ::= a R + S RS R......
More informationPS2 - Comments. University of Virginia - cs3102: Theory of Computation Spring 2010
University of Virginia - cs3102: Theory of Computation Spring 2010 PS2 - Comments Average: 77.4 (full credit for each question is 100 points) Distribution (of 54 submissions): 90, 12; 80 89, 11; 70-79,
More informationSri vidya college of engineering and technology
Unit I FINITE AUTOMATA 1. Define hypothesis. The formal proof can be using deductive proof and inductive proof. The deductive proof consists of sequence of statements given with logical reasoning in order
More informationGeneric ǫ-removal and Input ǫ-normalization Algorithms for Weighted Transducers
International Journal of Foundations of Computer Science c World Scientific Publishing Company Generic ǫ-removal and Input ǫ-normalization Algorithms for Weighted Transducers Mehryar Mohri mohri@research.att.com
More informationCOMP4141 Theory of Computation
COMP4141 Theory of Computation Lecture 4 Regular Languages cont. Ron van der Meyden CSE, UNSW Revision: 2013/03/14 (Credits: David Dill, Thomas Wilke, Kai Engelhardt, Peter Höfner, Rob van Glabbeek) Regular
More informationCS:4330 Theory of Computation Spring Regular Languages. Finite Automata and Regular Expressions. Haniel Barbosa
CS:4330 Theory of Computation Spring 2018 Regular Languages Finite Automata and Regular Expressions Haniel Barbosa Readings for this lecture Chapter 1 of [Sipser 1996], 3rd edition. Sections 1.1 and 1.3.
More informationFinite Automata and Formal Languages
Finite Automata and Formal Languages TMV26/DIT32 LP4 2 Lecture 6 April 5th 2 Regular expressions (RE) are an algebraic way to denote languages. Given a RE R, it defines the language L(R). Actually, they
More informationCMSC 330: Organization of Programming Languages. Theory of Regular Expressions Finite Automata
: Organization of Programming Languages Theory of Regular Expressions Finite Automata Previous Course Review {s s defined} means the set of string s such that s is chosen or defined as given s A means
More informationA Disambiguation Algorithm for Weighted Automata
A Disambiguation Algorithm for Weighted Automata Mehryar Mohri a,b and Michael D. Riley b a Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, NY 10012. b Google Research, 76 Ninth
More informationSpeech Recognition Lecture 4: Weighted Transducer Software Library. Mehryar Mohri Courant Institute of Mathematical Sciences
Speech Recognition Lecture 4: Weighted Transducer Software Library Mehryar Mohri Courant Institute of Mathematical Sciences mohri@cims.nyu.edu Software Libraries FSM Library: Finite-State Machine Library.
More informationLecture 3: Nondeterministic Finite Automata
Lecture 3: Nondeterministic Finite Automata September 5, 206 CS 00 Theory of Computation As a recap of last lecture, recall that a deterministic finite automaton (DFA) consists of (Q, Σ, δ, q 0, F ) where
More informationHKN CS/ECE 374 Midterm 1 Review. Nathan Bleier and Mahir Morshed
HKN CS/ECE 374 Midterm 1 Review Nathan Bleier and Mahir Morshed For the most part, all about strings! String induction (to some extent) Regular languages Regular expressions (regexps) Deterministic finite
More informationCOM364 Automata Theory Lecture Note 2 - Nondeterminism
COM364 Automata Theory Lecture Note 2 - Nondeterminism Kurtuluş Küllü March 2018 The FA we saw until now were deterministic FA (DFA) in the sense that for each state and input symbol there was exactly
More informationEFFICIENT ALGORITHMS FOR TESTING THE TWINS PROPERTY
Journal of Automata, Languages and Combinatorics u (v) w, x y c Otto-von-Guericke-Universität Magdeburg EFFICIENT ALGORITHMS FOR TESTING THE TWINS PROPERTY Cyril Allauzen AT&T Labs Research 180 Park Avenue
More informationDeterministic Finite Automaton (DFA)
1 Lecture Overview Deterministic Finite Automata (DFA) o accepting a string o defining a language Nondeterministic Finite Automata (NFA) o converting to DFA (subset construction) o constructed from a regular
More informationFinite-state Machines: Theory and Applications
Finite-state Machines: Theory and Applications Unweighted Finite-state Automata Thomas Hanneforth Institut für Linguistik Universität Potsdam December 10, 2008 Thomas Hanneforth (Universität Potsdam) Finite-state
More informationFS Properties and FSTs
FS Properties and FSTs Chris Dyer Algorithms for NLP 11-711 Announcements HW1 has been posted; due in class in 2 weeks Goals of Today s Lecture Understand properties of regular languages Understand Brzozowski
More informationNondeterministic Finite Automata
Nondeterministic Finite Automata Not A DFA Does not have exactly one transition from every state on every symbol: Two transitions from q 0 on a No transition from q 1 (on either a or b) Though not a DFA,
More informationUNIT-II. NONDETERMINISTIC FINITE AUTOMATA WITH ε TRANSITIONS: SIGNIFICANCE. Use of ε-transitions. s t a r t. ε r. e g u l a r
Syllabus R9 Regulation UNIT-II NONDETERMINISTIC FINITE AUTOMATA WITH ε TRANSITIONS: In the automata theory, a nondeterministic finite automaton (NFA) or nondeterministic finite state machine is a finite
More informationP(t w) = arg maxp(t, w) (5.1) P(t,w) = P(t)P(w t). (5.2) The first term, P(t), can be described using a language model, for example, a bigram model:
Chapter 5 Text Input 5.1 Problem In the last two chapters we looked at language models, and in your first homework you are building language models for English and Chinese to enable the computer to guess
More informationCISC 4090: Theory of Computation Chapter 1 Regular Languages. Section 1.1: Finite Automata. What is a computer? Finite automata
CISC 4090: Theory of Computation Chapter Regular Languages Xiaolan Zhang, adapted from slides by Prof. Werschulz Section.: Finite Automata Fordham University Department of Computer and Information Sciences
More informationIntroduction: Computer Science is a cluster of related scientific and engineering disciplines concerned with the study and application of computations. These disciplines range from the pure and basic scientific
More informationUses of finite automata
Chapter 2 :Finite Automata 2.1 Finite Automata Automata are computational devices to solve language recognition problems. Language recognition problem is to determine whether a word belongs to a language.
More informationTheory of Computation Lecture 1. Dr. Nahla Belal
Theory of Computation Lecture 1 Dr. Nahla Belal Book The primary textbook is: Introduction to the Theory of Computation by Michael Sipser. Grading 10%: Weekly Homework. 30%: Two quizzes and one exam. 20%:
More informationFormal Models in NLP
Formal Models in NLP Finite-State Automata Nina Seemann Universität Stuttgart Institut für Maschinelle Sprachverarbeitung Pfaffenwaldring 5b 70569 Stuttgart May 15, 2012 Nina Seemann (IMS) Formal Models
More informationInf2A: Converting from NFAs to DFAs and Closure Properties
1/43 Inf2A: Converting from NFAs to DFAs and Stuart Anderson School of Informatics University of Edinburgh October 13, 2009 Starter Questions 2/43 1 Can you devise a way of testing for any FSM M whether
More informationCS 154, Lecture 2: Finite Automata, Closure Properties Nondeterminism,
CS 54, Lecture 2: Finite Automata, Closure Properties Nondeterminism, Why so Many Models? Streaming Algorithms 0 42 Deterministic Finite Automata Anatomy of Deterministic Finite Automata transition: for
More informationAuthor: Vivek Kulkarni ( )
Author: Vivek Kulkarni ( vivek_kulkarni@yahoo.com ) Chapter-2: Finite State Machines Solutions for Review Questions @ Oxford University Press 2013. All rights reserved. 1 Q.1 Construct Mealy and Moore
More informationFinite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018 Lecture 4 Ana Bove March 23rd 2018 Recap: Formal Proofs How formal should a proof be? Depends on its purpose...... but should be convincing......
More informationClarifications from last time. This Lecture. Last Lecture. CMSC 330: Organization of Programming Languages. Finite Automata.
CMSC 330: Organization of Programming Languages Last Lecture Languages Sets of strings Operations on languages Finite Automata Regular expressions Constants Operators Precedence CMSC 330 2 Clarifications
More informationKleene Algebra and Arden s Theorem. Anshul Kumar Inzemamul Haque
Kleene Algebra and Arden s Theorem Anshul Kumar Inzemamul Haque Motivation Regular Expression is a Kleene Algebra. We can use the properties and theorems of Kleene Algebra to simplify regular expressions
More informationRegular Expressions and Language Properties
Regular Expressions and Language Properties Mridul Aanjaneya Stanford University July 3, 2012 Mridul Aanjaneya Automata Theory 1/ 47 Tentative Schedule HW #1: Out (07/03), Due (07/11) HW #2: Out (07/10),
More informationRegular Expressions. Definitions Equivalence to Finite Automata
Regular Expressions Definitions Equivalence to Finite Automata 1 RE s: Introduction Regular expressions are an algebraic way to describe languages. They describe exactly the regular languages. If E is
More informationFooling Sets and. Lecture 5
Fooling Sets and Introduction to Nondeterministic Finite Automata Lecture 5 Proving that a language is not regular Given a language, we saw how to prove it is regular (union, intersection, concatenation,
More informationFinite Automata and Formal Languages TMV026/DIT321 LP4 2012
Finite Automata and Formal Languages TMV26/DIT32 LP4 22 Lecture 7 Ana Bove March 27th 22 Overview of today s lecture: Regular Expressions From FA to RE Regular Expressions Regular expressions (RE) are
More informationFinite Automata Theory and Formal Languages TMV027/DIT321 LP4 2017
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2017 Lecture 4 Ana Bove March 24th 2017 Structural induction; Concepts of automata theory. Overview of today s lecture: Recap: Formal Proofs
More informationMethods for the specification and verification of business processes MPB (6 cfu, 295AA)
Methods for the specification and verification of business processes MPB (6 cfu, 295AA) Roberto Bruni http://www.di.unipi.it/~bruni 07 - Introduction to nets 1 Object Overview of the basic concepts of
More informationMethods for the specification and verification of business processes MPB (6 cfu, 295AA)
Methods for the specification and verification of business processes MPB (6 cfu, 295AA) Roberto Bruni http://www.di.unipi.it/~bruni 07 - Introduction to nets 1 Object Overview of the basic concepts of
More informationCS 455/555: Finite automata
CS 455/555: Finite automata Stefan D. Bruda Winter 2019 AUTOMATA (FINITE OR NOT) Generally any automaton Has a finite-state control Scans the input one symbol at a time Takes an action based on the currently
More informationBusiness Processes Modelling MPB (6 cfu, 295AA)
Business Processes Modelling MPB (6 cfu, 295AA) Roberto Bruni http://www.di.unipi.it/~bruni 07 - Introduction to nets!1 Object Overview of the basic concepts of Petri nets Free Choice Nets (book, optional
More informationIntroduction to Kleene Algebras
Introduction to Kleene Algebras Riccardo Pucella Basic Notions Seminar December 1, 2005 Introduction to Kleene Algebras p.1 Idempotent Semirings An idempotent semiring is a structure S = (S, +,, 1, 0)
More informationTheory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
More informationClasses and conversions
Classes and conversions Regular expressions Syntax: r = ε a r r r + r r Semantics: The language L r of a regular expression r is inductively defined as follows: L =, L ε = {ε}, L a = a L r r = L r L r
More informationAn algebraic characterization of unary two-way transducers
An algebraic characterization of unary two-way transducers (Extended Abstract) Christian Choffrut 1 and Bruno Guillon 1 LIAFA, CNRS and Université Paris 7 Denis Diderot, France. Abstract. Two-way transducers
More informationFoundations of
91.304 Foundations of (Theoretical) Computer Science Chapter 1 Lecture Notes (Section 1.3: Regular Expressions) David Martin dm@cs.uml.edu d with some modifications by Prof. Karen Daniels, Spring 2012
More informationComputational Models #1
Computational Models #1 Handout Mode Nachum Dershowitz & Yishay Mansour March 13-15, 2017 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 1 / 41 Lecture Outline I Motivation
More informationFINITE STATE AUTOMATA
FINITE STATE AUTOMATA States An FSA has a finite set of states A system has a limited number of configurations Examples {On, Off}, {1,2,3,4,,k} {TV channels} States can be graphically represented as follows:
More informationHow do regular expressions work? CMSC 330: Organization of Programming Languages
How do regular expressions work? CMSC 330: Organization of Programming Languages Regular Expressions and Finite Automata What we ve learned What regular expressions are What they can express, and cannot
More informationFinite Automata and Regular languages
Finite Automata and Regular languages Huan Long Shanghai Jiao Tong University Acknowledgements Part of the slides comes from a similar course in Fudan University given by Prof. Yijia Chen. http://basics.sjtu.edu.cn/
More informationClosure under the Regular Operations
Closure under the Regular Operations Application of NFA Now we use the NFA to show that collection of regular languages is closed under regular operations union, concatenation, and star Earlier we have
More informationAutomata: a short introduction
ILIAS, University of Luxembourg Discrete Mathematics II May 2012 What is a computer? Real computers are complicated; We abstract up to an essential model of computation; We begin with the simplest possible
More informationComputational Models - Lecture 4
Computational Models - Lecture 4 Regular languages: The Myhill-Nerode Theorem Context-free Grammars Chomsky Normal Form Pumping Lemma for context free languages Non context-free languages: Examples Push
More informationcse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska
cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 6 CHAPTER 2 FINITE AUTOMATA 2. Nondeterministic Finite Automata NFA 3. Finite Automata and Regular Expressions 4. Languages
More informationRegular expressions and Kleene s theorem
and Informatics 2A: Lecture 5 Alex Simpson School of Informatics University of Edinburgh als@inf.ed.ac.uk 25 September, 2014 1 / 26 1 More closure properties of regular languages Operations on languages
More informationSeptember 11, Second Part of Regular Expressions Equivalence with Finite Aut
Second Part of Regular Expressions Equivalence with Finite Automata September 11, 2013 Lemma 1.60 If a language is regular then it is specified by a regular expression Proof idea: For a given regular language
More informationTheory of Computation p.1/?? Theory of Computation p.2/?? Unknown: Implicitly a Boolean variable: true if a word is
Abstraction of Problems Data: abstracted as a word in a given alphabet. Σ: alphabet, a finite, non-empty set of symbols. Σ : all the words of finite length built up using Σ: Conditions: abstracted as a
More information1 More finite deterministic automata
CS 125 Section #6 Finite automata October 18, 2016 1 More finite deterministic automata Exercise. Consider the following game with two players: Repeatedly flip a coin. On heads, player 1 gets a point.
More informationProbabilistic Aspects of Computer Science: Probabilistic Automata
Probabilistic Aspects of Computer Science: Probabilistic Automata Serge Haddad LSV, ENS Paris-Saclay & CNRS & Inria M Jacques Herbrand Presentation 2 Properties of Stochastic Languages 3 Decidability Results
More informationRegular expressions and automata
Regular expressions and automata Introduction Finite State Automaton (FSA) Finite State Transducers (FST) Regular expressions(i) (Res) Standard notation for characterizing text sequences Specifying text
More informationDecision, Computation and Language
Decision, Computation and Language Non-Deterministic Finite Automata (NFA) Dr. Muhammad S Khan (mskhan@liv.ac.uk) Ashton Building, Room G22 http://www.csc.liv.ac.uk/~khan/comp218 Finite State Automata
More informationChapter 6. Properties of Regular Languages
Chapter 6 Properties of Regular Languages Regular Sets and Languages Claim(1). The family of languages accepted by FSAs consists of precisely the regular sets over a given alphabet. Every regular set is
More informationTheory of Computation
Thomas Zeugmann Hokkaido University Laboratory for Algorithmics http://www-alg.ist.hokudai.ac.jp/ thomas/toc/ Lecture 3: Finite State Automata Motivation In the previous lecture we learned how to formalize
More informationLanguage Technology. Unit 1: Sequence Models. CUNY Graduate Center. Lecture 4a: Probabilities and Estimations
Language Technology CUNY Graduate Center Unit 1: Sequence Models Lecture 4a: Probabilities and Estimations Lecture 4b: Weighted Finite-State Machines required hard optional Liang Huang Probabilities experiment
More informationWarshall s algorithm
Regular Expressions [1] Warshall s algorithm See Floyd-Warshall algorithm on Wikipedia The Floyd-Warshall algorithm is a graph analysis algorithm for finding shortest paths in a weigthed, directed graph
More informationDeterministic Finite Automata. Non deterministic finite automata. Non-Deterministic Finite Automata (NFA) Non-Deterministic Finite Automata (NFA)
Deterministic Finite Automata Non deterministic finite automata Automata we ve been dealing with have been deterministic For every state and every alphabet symbol there is exactly one move that the machine
More informationDeterministic Finite Automata (DFAs)
Algorithms & Models of Computation CS/ECE 374, Fall 27 Deterministic Finite Automata (DFAs) Lecture 3 Tuesday, September 5, 27 Sariel Har-Peled (UIUC) CS374 Fall 27 / 36 Part I DFA Introduction Sariel
More informationCS 154, Lecture 3: DFA NFA, Regular Expressions
CS 154, Lecture 3: DFA NFA, Regular Expressions Homework 1 is coming out Deterministic Finite Automata Computation with finite memory Non-Deterministic Finite Automata Computation with finite memory and
More informationLet us first give some intuitive idea about a state of a system and state transitions before describing finite automata.
Finite Automata Automata (singular: automation) are a particularly simple, but useful, model of computation. They were initially proposed as a simple model for the behavior of neurons. The concept of a
More informationLanguages, regular languages, finite automata
Notes on Computer Theory Last updated: January, 2018 Languages, regular languages, finite automata Content largely taken from Richards [1] and Sipser [2] 1 Languages An alphabet is a finite set of characters,
More informationComputer Sciences Department
1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER 3 objectives Finite automaton Infinite automaton Formal definition State diagram Regular and Non-regular
More informationLecture 17: Language Recognition
Lecture 17: Language Recognition Finite State Automata Deterministic and Non-Deterministic Finite Automata Regular Expressions Push-Down Automata Turing Machines Modeling Computation When attempting to
More informationNotes on generating functions in automata theory
Notes on generating functions in automata theory Benjamin Steinberg December 5, 2009 Contents Introduction: Calculus can count 2 Formal power series 5 3 Rational power series 9 3. Rational power series
More informationFinite Automata - Deterministic Finite Automata. Deterministic Finite Automaton (DFA) (or Finite State Machine)
Finite Automata - Deterministic Finite Automata Deterministic Finite Automaton (DFA) (or Finite State Machine) M = (K, Σ, δ, s, A), where K is a finite set of states Σ is an input alphabet s K is a distinguished
More informationSome useful tasks involving language. Finite-State Machines and Regular Languages. More useful tasks involving language. Regular expressions
Some useful tasks involving language Finite-State Machines and Regular Languages Find all phone numbers in a text, e.g., occurrences such as When you call (614) 292-8833, you reach the fax machine. Find
More informationComputational Models Lecture 2 1
Computational Models Lecture 2 1 Handout Mode Iftach Haitner. Tel Aviv University. October 30, 2017 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.
More informationOn-Line Learning with Path Experts and Non-Additive Losses
On-Line Learning with Path Experts and Non-Additive Losses Joint work with Corinna Cortes (Google Research) Vitaly Kuznetsov (Courant Institute) Manfred Warmuth (UC Santa Cruz) MEHRYAR MOHRI MOHRI@ COURANT
More informationCS243, Logic and Computation Nondeterministic finite automata
CS243, Prof. Alvarez NONDETERMINISTIC FINITE AUTOMATA (NFA) Prof. Sergio A. Alvarez http://www.cs.bc.edu/ alvarez/ Maloney Hall, room 569 alvarez@cs.bc.edu Computer Science Department voice: (67) 552-4333
More informationFinite Automata and Regular Languages
Finite Automata and Regular Languages Topics to be covered in Chapters 1-4 include: deterministic vs. nondeterministic FA, regular expressions, one-way vs. two-way FA, minimization, pumping lemma for regular
More informationTasks of lexer. CISC 5920: Compiler Construction Chapter 2 Lexical Analysis. Tokens and lexemes. Buffering
Tasks of lexer CISC 5920: Compiler Construction Chapter 2 Lexical Analysis Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Copyright Arthur G. Werschulz, 2017. All
More information3515ICT: Theory of Computation. Regular languages
3515ICT: Theory of Computation Regular languages Notation and concepts concerning alphabets, strings and languages, and identification of languages with problems (H, 1.5). Regular expressions (H, 3.1,
More informationT (s, xa) = T (T (s, x), a). The language recognized by M, denoted L(M), is the set of strings accepted by M. That is,
Recall A deterministic finite automaton is a five-tuple where S is a finite set of states, M = (S, Σ, T, s 0, F ) Σ is an alphabet the input alphabet, T : S Σ S is the transition function, s 0 S is the
More informationA Weak Bisimulation for Weighted Automata
Weak Bisimulation for Weighted utomata Peter Kemper College of William and Mary Weighted utomata and Semirings here focus on commutative & idempotent semirings Weak Bisimulation Composition operators Congruence
More informationNondeterminism and Epsilon Transitions
Nondeterminism and Epsilon Transitions Mridul Aanjaneya Stanford University June 28, 22 Mridul Aanjaneya Automata Theory / 3 Challenge Problem Question Prove that any square with side length a power of
More informationTuring Machines (TM) Deterministic Turing Machine (DTM) Nondeterministic Turing Machine (NDTM)
Turing Machines (TM) Deterministic Turing Machine (DTM) Nondeterministic Turing Machine (NDTM) 1 Deterministic Turing Machine (DTM).. B B 0 1 1 0 0 B B.. Finite Control Two-way, infinite tape, broken into
More informationFormal Languages. We ll use the English language as a running example.
Formal Languages We ll use the English language as a running example. Definitions. A string is a finite set of symbols, where each symbol belongs to an alphabet denoted by. Examples. The set of all strings
More informationCompiler Design. Spring Lexical Analysis. Sample Exercises and Solutions. Prof. Pedro C. Diniz
Compiler Design Spring 2011 Lexical Analysis Sample Exercises and Solutions Prof. Pedro C. Diniz USC / Information Sciences Institute 4676 Admiralty Way, Suite 1001 Marina del Rey, California 90292 pedro@isi.edu
More informationCOMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages
COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed
More informationUnit 6. Non Regular Languages The Pumping Lemma. Reading: Sipser, chapter 1
Unit 6 Non Regular Languages The Pumping Lemma Reading: Sipser, chapter 1 1 Are all languages regular? No! Most of the languages are not regular! Why? A finite automaton has limited memory. How can we
More informationWe define the multi-step transition function T : S Σ S as follows. 1. For any s S, T (s,λ) = s. 2. For any s S, x Σ and a Σ,
Distinguishability Recall A deterministic finite automaton is a five-tuple M = (S,Σ,T,s 0,F) where S is a finite set of states, Σ is an alphabet the input alphabet, T : S Σ S is the transition function,
More informationComputational Models Lecture 2 1
Computational Models Lecture 2 1 Handout Mode Ronitt Rubinfeld and Iftach Haitner. Tel Aviv University. March 16/18, 2015 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice
More informationWhat we have done so far
What we have done so far DFAs and regular languages NFAs and their equivalence to DFAs Regular expressions. Regular expressions capture exactly regular languages: Construct a NFA from a regular expression.
More informationIntroduction to Theory of Computing
CSCI 2670, Fall 2012 Introduction to Theory of Computing Department of Computer Science University of Georgia Athens, GA 30602 Instructor: Liming Cai www.cs.uga.edu/ cai 0 Lecture Note 3 Context-Free Languages
More informationTheory of Computation (II) Yijia Chen Fudan University
Theory of Computation (II) Yijia Chen Fudan University Review A language L is a subset of strings over an alphabet Σ. Our goal is to identify those languages that can be recognized by one of the simplest
More information