Computability and Complexity

Computability and Complexity Sequences and Automata CAS 705 Ryszard Janicki Department of Computing and Software McMaster University Hamilton, Ontario, Canada janicki@mcmaster.ca Ryszard Janicki Computability and Complexity 1 / 33

Outline Languages and Machines: sets of sequences/languages (nite/innite sequences, closure properties, Nerode equivalence, regular languages, automata, transition systems), dening sets by expressions and equations (regular expressions, x-point theory), machines, computational models and structures (Turing machines, RAM, PRAM, non-standard automata, etc.). Rewriting Systems: Grammars, Post systems, Chomsky hierarchy, Grammars vs automata Ryszard Janicki Computability and Complexity 2 / 33

Outline Computability: Functions and sets, Primitive recursive functions and sets, recursive functions and sets, recursively enumerable sets, what does computable and decidable mean (Halting problem), limitation of formal systems (Gödel's theorem), Church thesis. Complexity: denition, independence from a computing platform, time/space hierarchy, P vs NP, NP-completeness, reduction problem. Ryszard Janicki Computability and Complexity 3 / 33

Sequences Alphabet: an arbitrary (usually nite) set of elements, often denoted by the symbol Σ. Sequence: an element x = (a 1, a 2,..., a k ) Σ k, where Σ k is a Cartesian product of Σ's. For convenience we write x = a 1 a 2... a k. a function φ : {1,..., k} Σ, such that φ(1) = a 1,..., φ(k) = a k. The two above denitions are in a sense identical since: Σ }. {{.. Σ } {f f : {1,..., k} Σ}. n Frequently a sequence is considered as a primitive undened concept that is understood and does not need any explanation. Ryszard Janicki Computability and Complexity 4 / 33

Sequences and strings If the elements of Σ are symbols, then a nite sequence of symbols is often called a string or a word. The length of a sequence x, denoted x, is the number of elements composing the sequence. The empty sequence, ε, is the sequence consisting of zero symbols, i.e. ε = 0. A prex of a sequence is any number of leading symbols of that sequence, and a sux is any number of trailing symbols (any number means `zero included'). Ryszard Janicki Computability and Complexity 5 / 33

Concatenation Concatenation (operation) Let x = a 1... a k, y = b 1... b l. Then x y = a 1... a k b 1... b l. We usually write xy instead of x y. Properties of concatenation: 1 x(yz) = (xy)z 2 εx = xε = x Fact. A triple (Σ,, ε) is a monoid. Power operator: x 0 = ε, x 1 = x and x k = x }.{{.. x}. k Recursive denition of power: x 0 = ε x k+1 = x k x. Ryszard Janicki Computability and Complexity 6 / 33

Σ and Formal Language Let Σ be a nite alphabet. Then we dene Σ as: Σ = {a 1... a k a i Σ k 0}, i.e. the set of all sequences, including ε, built from the elements of Σ. A (formal) language over Σ is any subset of Σ, including the empty set and Σ. Ryszard Janicki Computability and Complexity 7 / 33

Two Important Models of Sequence Monoid 1 Algebra of Strings elements of Σ are symbols, elements of Σ are words or strings, concatenation is just a juxtaposition. 2 Algebra of Relations Ryszard Janicki Computability and Complexity 8 / 33

Algebra of Relations Let X be a set and Σ = {R 1,..., R k }, where each R i X X is a relation. Now concatenation corresponds to composition of relations, i.e. x(r S)y t Σ. xrt tsy, and ε is the identity relation, i.e. ε = {(x, x) x X }. Let A = {r 1,..., r n } be the set of names of relations, i.e. name(r i ) = r i, rel(r i ) = R i, where name : Σ A and rel : A Σ. Let extend rel : A Σ to rel : A Σ, where Σ is the set of all relations on X that could be composed from the elements of Σ, plus identity relation, by rel(r i1... r ik ) = rel(r i1 )... rel(r ik ) = R i1... R ik. All properties of the names algebra hold for the relation algebra. Ryszard Janicki Computability and Complexity 9 / 33

Automata or State Machines There is a set of states Q. Q may be nite, then we have nite state machines. There is a set of actions/operations that allow to move from one state to another state. There is a transition function/relation that allow movement form one state to another state. The last two statements are equivalent. There is an initial state. There might be nal states. The concept of a current state may easily be introduced. The set of actions/operations maybe innite, but usually is nite. There is a concept of nondeterministic choice. Ryszard Janicki Computability and Complexity 10 / 33

Automata or State Machines: an example Ryszard Janicki Computability and Complexity 11 / 33

Automata: Non-determinism Ryszard Janicki Computability and Complexity 12 / 33

Deterministic Automata: Classical Denition Denition (Automaton - 1) A deterministic (nite) automaton (state machine) is a 5-tuple: M = (Σ, Q, δ, s 0, F ), where: Σ is the alphabet (nite) (input alphabet), Q is the set of states (nite), δ : Q Σ Q is the transition function, s 0 Q is the initial state, F Q is the set of nal states. Denition (ˆδ function) We extend the function δ to ˆδ : Q Σ Q as follows: q Q. ˆδ(q, ε) = q q Q. x Σ. a Σ. ˆδ(q, xa) = δ(ˆδ(q, x), a). Ryszard Janicki Computability and Complexity 13 / 33

Language Accepted/Generated by an Automaton Denition (Language - 1) For every automaton M, the set L(M) = {x ˆδ(s 0, x) F } is called the language accepted/generated by M. We will write δ instead of ˆδ if it will not lead to any misunderstanding. Ryszard Janicki Computability and Complexity 14 / 33

Deterministic Automata: `Local' Denition Denition (Automaton - 2) A deterministic (nite) automaton (state machine) is a 4-tuple: M = (Σ, Q, s 0, F ), where: Σ is the set of actions, i.e. each a Σ is a function Q is the set of states (nite), s 0 Q is the initial state, F Q is the set of nal states. a : Q Q, The relationship between Denition 1 and Denition 2: δ(s, a) = q a(s) = q. Ryszard Janicki Computability and Complexity 15 / 33

Language with Denition 2 We dene Σ as the set of all functions that can be constructed from the elements of Σ and the function composition. For example the sequence x = a 1... a k Σ denes the function x : Q Q, x(s) = a 1... a k (s) = a k (... a 2 (a 1 (s))...). Denition (Language - 2) For every automaton M, the set L(M) = {x x(s 0 ) F } is called the language accepted/generated by M. RULE: s a p δ(s, a) = p a(s) = p. Ryszard Janicki Computability and Complexity 16 / 33

Deterministic Automaton: an Example (two versions) Consider the following deterministic automaton In both models: Σ = {a, b}, Q = {s 0, s 1, s 2, s 3 }, F = {s 2, s 3 } and the below table shows both transition function δ (classical model) and actions a(...), b(...) (`local' model). s a(s) b(s) `local' model δ a b classical model s 0 s 1 s 2 s 1 s 3 s 1 s 2 s 1 s 3 s 3 s 2 s 1 Ryszard Janicki Computability and Complexity 17 / 33

Problems! (1) We cannot specify: The meaning is pretty simple: rst execute a and next execute any number of b, including none. However, for the rst denition we have δ(s 0, a) = s 1 and δ(s 1, b) = s 1, but what about δ(s 0, b) =? and δ(s 1, a) =?. For the second denition we have a(s 0 ) = s 1 and b(s 1 ) = s 1 but still b(s 0 ) =? and a(s 1 ) =?. UGLY SOLUTION Ryszard Janicki Computability and Complexity 18 / 33

Ugly Solution UGLY SOLUTION This solution is good for illustration, problematics for real systems as we have to introduce entities that may not exist in the real system! Ryszard Janicki Computability and Complexity 19 / 33

Solutions A solution with Denition 2: for each a Σ, assume that a : Q Q is a partial function. The solution is elegant only on the surface as dealing with partial functions is tricky and often problematic. When Denition 1 is used we have to introduce the concept of Non-determinism. This concept can also be used for Denition 2. Ryszard Janicki Computability and Complexity 20 / 33

Non-determinism: Problem #1 Notation for `power set': 2 Q = P(Q) = {X X Q}, and clearly 2 Q. Problem #1: How to model the below situation? Functional solutions: (1) δ(s, a) = {s 1,..., s k }, which implies δ : Q Σ 2 Q, (2) a(s) = {s 1,..., s k }, which implies a Σ. a : Q 2 Q. Relational solutions: (3) (s, a, s i ) δ for i = 1,..., k, = δ Q Σ Q, (4) (s, s i ) a for i = 1,..., k, = a Σ. a Q Q. Solutions (1) and (4) are the most popular. Ryszard Janicki Computability and Complexity 21 / 33

Non-determinism: Problem #2 Consider the following automaton: Which is true? ab L(M) or ab / L(M)? Ryszard Janicki Computability and Complexity 22 / 33

Non-determinism: Problem #2 Consider the following automaton: Which is true? ab L(M) or ab / L(M)? Usually it is assumed that ab L(M). It is called angelic semantics. Ryszard Janicki Computability and Complexity 22 / 33

Non-determinism: Problem #2 Consider the following automaton: Which is true? ab L(M) or ab / L(M)? Usually it is assumed that ab L(M). It is called angelic semantics. Ryszard Janicki Computability and Complexity 22 / 33

Angelic vs Demonic Semantics Angelic: At each state an angel will tell you where to go, so if there is a good choice you will make it. The only bad case is when all choices are bad. Demonic: At each state a demon will tell you where to go, so if there is a bad choice you will make it. The only good case is when all choices are good. How does this distinction aects computability and complexity? How does it aect theory of Turing Machines? The `angelic' approach is the classical one, the `demonic' is about 25 years old. Ryszard Janicki Computability and Complexity 23 / 33

Angelic vs Demonic Semantics - An Example Consider the three automata below: Let L A (M 1 ), i = 1, 2, 3 denote a language dened by M i under angelic semantics, and let L D (M 1 ), i = 1, 2, 3 denote a language dened by M i under demonic semantics. Note that L A (M 1 ) = L D (M 1 ) =, L A (M 2 ) = {ab}, L D (M 2 ) = and L A (M 3 ) = L D (M 3 ) = {ab}. Ryszard Janicki Computability and Complexity 24 / 33

Denition (Non-deterministic Automaton - 1) A non-deterministic (nite) automaton (state machine) is a 5-tuple: M = (Σ, Q, δ, s 0, F ), where: Σ is the alphabet (nite) (input alphabet), Q is the set of states (nite), δ : Q Σ 2 Q is the transition function, s 0 Q is the initial state, F Q is the set of nal states. Denition (non-deterministic ˆδ function) We extend the function δ to ˆδ : Q Σ 2 Q as follows: q Q. ˆδ(q, ε) = {q} q Q. x Σ. a Σ. ˆδ(q, xa) = s ˆδ(q,x) δ(s, a). Sometimes, by a small abuse of notation, we write ˆδ(q, xa) = δ(ˆδ(q, x), a). Ryszard Janicki Computability and Complexity 25 / 33

Language Dened by a Non-deterministic Automaton Denition (Angelic semantics) For every automaton M, the set L(M) = {x ˆδ(s 0, x) F } is called the language accepted/generated by M. Denition (Demonic semantics) For every automaton M, the set L(M) = {x ˆδ(s 0, x) F } is called the language accepted/generated by M. We will again write δ instead of ˆδ if it will not lead to any misunderstanding. Ryszard Janicki Computability and Complexity 26 / 33

Non-deterministic Automata: `Local' Denition Denition (Non-deterministic Automaton - 2) A non-deterministic (nite) automaton (state machine) is a 4-tuple: M = (Σ, Q, s 0, F ), where: Σ is the set of actions, i.e. each a Σ is a relation Q is the set of states (nite), s 0 Q is the initial state, F Q is the set of nal states. The relationship is the following: a Q Q, q δ(s, a) (s, q) a. Ryszard Janicki Computability and Complexity 27 / 33

Here Σ is the set of all relations that can be built form the elements of Σ, i.e. a 1... a k = a 1 a 2... a k, there is the composition of relations given on page 9 of this lecture notes. Denition (Angelic Semantics) For every automaton M, the set L(M) = {x Σ {s (s 0, s) x} F } is called the language accepted/generated by M. Denition (Demonic Semantics) For every automaton M, the set L(M) = {x Σ {s (s 0, s) x} F } is called the language accepted/generated by M. RULE: s a p p δ(s, a) (s, p) a. Ryszard Janicki Computability and Complexity 28 / 33

Non-determinism: Example 1 Consider the following example: Classical model: Σ = {a, b}, Q = {s 0, s}, F = {s 1 }. δ(s 0, a) = {s 1 }, δ(s 0, b) =, δ(s 1, a) =, δ(s 1, b) = {s 1 }. `Local' model: Σ = {a, b}, Q = {s 0, s}, F = {s 1 }. a = {(s 0, s 1 )}, b = {(s 1, s 1 )}. Ryszard Janicki Computability and Complexity 29 / 33

Non-determinism: Example 2 Consider the following example: Classical model: Σ = {a, b, c}, Q = {q 0, q 1, q 2, q 3 }, F = {q 3 }. δ(q 0, a) = {q 1, q 2 }, δ(q 0, b) = δ(q 0, c) =, δ(q 1, a) = {q 3 }, δ(q 1, b) =, δ(q 1, c) = {q 1 }, δ(q 2, a) = {q 3 }, δ(q 2, b) = {q 2 }, δ(q 2, c) =, δ(q 3, a) = δ(q 3, b) = δ(q 3, c) = `Local' model: Σ = {a, b, c}, Q = {q 0, q 1, q 2, q 3 }, F = {q 3 }. a = {(q 0, q 1 ), (q 0, q 2 ), (q 1, q 3 ), (q 2, q 3 )}, b = {(q 2, q 2 )}, c = {(q 1, q 1 )}. Ryszard Janicki Computability and Complexity 30 / 33

Another Approach to Non-determinism Denition An automaton (state machine) is a 5-tuple: M = (Σ, Q, δ, s 0, F ), where: Σ is the alphabet (nite), Q is the set of states (nite), δ : Q Σ 2 Q is the transition function, s 0 Q is the initial state, F Q is the set of nal states. Denition M is deterministic i q Q. a Σ. δ(q, a) 1 M is strictly deterministic i q Q. a Σ. δ(q, a) = 1 Ryszard Janicki Computability and Complexity 31 / 33

Transition Systems Transition (from Collins Dictionary): a passing or change from one place, state, condition, etc., to another. Consider the case: s a 1 s 2 a s 3 Can a be called a transition? Transitions, state and labels: Transitions are unique: t 1 s a 1 s 2 Ryszard Janicki Computability and Complexity 32 / 33

Petri Nets Transitions are usually needed for modelling concurrency. Consider the following very simple Petri Net: There is a structural dierence between the label a attached to the transition t 1 and the same label a attached to the transition t 3. Ryszard Janicki Computability and Complexity 33 / 33