CDT314 FABE Formal anguages, Automata and Models of Computation ecture 12 Mälardalen University 2013 1
Content Chomsky s anguage Hierarchy Turing Machines Determinism Halting TM Examples Standard TM Computing Functions with TM Combining TMs Turing Thesis 2
{! : n a n n n { a n b c : n 0} { ww : w { a, b }} ecursively enumerable languages Turing Machines 0} n n { a b : n Context-free languages Push-down Automata t 0} n 2 n { a b : n { ww : w { a, b}*} 0} egular anguages Finite Automata a *b a *b* 3
Turing Machines Theoretical Aspects Based on C Busch, PI, Models of Computation 4
TUING MACHINES Unlike egular anguages (with Finite Automata, egular Expressions and egular Grammars) and Context Free anguages (with Pushdown Automata and Context Free Grammars) with many practical applications, estriction Free anguages (and corresponding Turing Machines) are primarily important for Theoretical Computer Science as a means of study of the mechanisms and limits of computability. 5
Alan Turing Who was Alan Turing? Founder of Computability Theory, Mathematician, Codebreaker, Natural philosopher, Visionary man before his time. http://www.cs.usfca.edu/www.alanturing.net/turing_archive/index.html Alan Turing archive - Jack Copeland and Diane Proudfoot http://www.turing.org.uk/turing/ The Alan Turing Home Page Andrew Hodges 6
Alan Turing 1912 (23 June): Birth, ondon 1926-31: Sherborne School 1930: Death of friend Christopher Morcom 1931-34: 34: Undergraduate at King's College, Cambridge University 1932-35: Quantum mechanics, probability, logic 1935: Elected fellow of King's College, Cambridge 1936: The Turing machine, computability, universal machine 1936-38: Princeton University. Ph.D. ogic, algebra, number theory 1938-39: eturn to Cambridge. Introduced to German Enigma cipher machine 1939-40: The Bombe, machine for Enigma decryption 1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic 7
Alan Turing 1943-45: Chief Anglo-American crypto consultant. Electronic work. 1945: National Physical aboratory, ondon 1946: Computer and software design leading the world. 1947-48: Programming, neural nets, and artificial intelligence 1948: Manchester University 1949: First serious mathematical use of a computer 1950: The Turing Test for machine intelligence 1951: Elected FS. Non-linear theory of biological growth 1952: Arrested as a homosexual, loss of security clearance 1953-54: Unfinished work in biology and physics 1954 (7 June): Death (suicide) id by cyanide poisoning, i Wilmslow, l Cheshire. hi 8
Turing's Work and Scientific egacy The Mathematics of Emergence: The Mysteries of Morphogenesis Possibility of Building a Brain: Intelligent Machines, Practice and Theory Nature of Information: Complexity, andomness, Hiddenness of Information How should we compute? New Models of ogic and Computation http://www.mathcomp.leeds.ac.uk/turing2012/give-page.php?408 9
Hilbert s Program - formalization of mathematics, 1900 Hilbert s hope was that mathematics would be reducible to finding proofs (syntactic manipulating the strings of symbols) from a fixed system of axioms. Can all of mathematics be made algorithmic, or will there always be new problems that surpass any given algorithm, and so require creative acts of mind to solve? 10
Turing Machines and Computability The question Hilbert asked was: is there a general method or process by which one could decide whether a mathematical proposition could be proved? But what exactly was meant by a 'method' or 'process'? People had already used the concept of a 'mechanical' process, and Turing had an idea which made this quite precise: computation. 11
Turing Machines and Computability The Turing machine concept involves specifying a very restricted set of logical operations, but Turing showed how other more complex mathematical procedures could be built out of these atomic components. Turing argued that his formalism was sufficiently i general to encompass anything that a human being could do when carrying out a definite computation method. Question: How about geometric methods? There is a way to translate geometry into algebra algebraic geometry. 12
Turing Machines and Computability Turing's famous 1936-7 paper On computable numbers, with an application to the Entscheidungsproblem, which worked out the theory of Turing machines and the definition of computability, is available as a PDF file on-line: http://www.turingarchive.org/browse.php/b/12 http://www.cs.ox.ac.uk/activities/ieg/e-library/sources/tp2-ie.pdf 13
Turing Machines Turing s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing) 14
Turing Machine Examples Based on C Busch, PI, Models of Computation 15
Tape... Turing Machine... Control Unit ead-write head 16
... The Tape No boundaries -- infinite length... ead-write head The head moves eft or ight 17
...... ead-write head The head at each time step: 1. eads a symbol 2. Writes a symbol 3. Moves eft or ight 18
Example Step 0... a b a c... Step 1... a b k c... 1. eads a 2. Writes k 3. Moves eft 19
Step 1... a b k c... Step 2... a f k c... 1. eads b 2. Writes f 3. Moves ight 20
The Input String Input string Blank symbol... # # a b a c # # #... head Head starts t at the leftmost t position of the input string 21
States & Transitions ead Write Move eft a b, q q2 1 Move ight a b, q q2 1 22
Example Step 1... # # a b a c # # #... q 1 current state a b, q q 2 1 23
Step 1... # # a b a c # # #... q 1 Step 2...... # # a b b c # # # q 2 a b, q q2 1 24
Example Step 1... # # a b a c # # #... q 1 Step 2...... # # a b b c # # # q 2 a b, q q2 1 25
Example Step 1... # # a b a c # # #... q 1 Step 2... g # # a b b c g # #... # g, q q2 1 q 2 26
Determinism Turing Machines are deterministic Allowed Not Allowed q 1 a b, a b, q2 q 2 q d 3 q 3 a d, b d, q 1 No lambda transitions allowed in TM! 27
Determinism Note the difference between state indeterminism when not even possible future states are known in advance and choice indeterminism when possible future states are known, but we do not know which state will be taken. 28
Example Partial Transition Function... # # a b a c # # #... q 1 q 1 a b, b d, q 2 q 3 Allowed No transition for input symbol c 29
Halting The machine halts if there are no possible transitions to follow 30
Example... # # a b a c # # #... q 1 a b, b d, q 2 q 3 q 1 No possible transition HAT! 31
Final States q q 2 1 Allowed q1 q2 Not Allowed Final states have no outgoing transitions In a final state the machine halts 32
Acceptance Accept Input If machine halts in a final state eject Input If machine halts in a non-final state or If machine enters an infinite loop 33
Turing Machine Example A TM that accepts the language aa * a a, # #, q 0 q 1 34
Step 0 # # a a a # # q 0 a a, # #, q 0 q 1 35
Step 1 # # a a a # # q 0 a a, # #, q0 q 1 36
Step 2 # # a a a # # q 0 a a, # #, q0 q 1 37
Step 3 # # a a a # # q 0 a a, # #, q0 q 1 38
Step 4 # # a a a # # q 1 a a, Halt & Accept # #, q 0 q 1 39
ejection Example Step 0 # # a b a # # q 0 a a, # #, q0 q 1 40
Step 1 # # a b a # # q 0 a a, No possible Transition Halt & eject # #, q0 q 1 41
Infinite oop Example Another TM for language aa * b b, a a, # #, q0 q 1 42
Step 0 # # a b a # # q 0 b b, a a, q # #, 0 q 1 43
Step 1 # # a b a # # q 0 b b, a a, # #,# q0 q 1 44
Step 2 # # a b a # # q 0 b b, a a, # #, q0 q 1 45
Step 2 # # a b a # # q 0 Step 3 # # a b a # # q 0 Step 4 # # a b a # # q 0 Step 5 # # a b a # #... infinite loop q 0 46
Because of the infinite loop: The final state cannot be reached The machine never halts The input is not accepted 47
Another Turing Machine Example Turing machine for the language { a n b n } y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 48
Step 0 # a a b b # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a, x, b y,, q0 1 q q2 x x, 49
Step 1 # x a b b # # q 1 { a n b n } y y, q 4 # #,# y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 50
Step 2 # x a b b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 51
Step 3 # x a y b # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 52
Step 4 # x a y b # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 53
Step 5 # x a y b # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 54
Step 6 # x x y b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 55
Step 7 # x x y b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 56
Step 8 # x x y y # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 57
Step 9 # x x y y # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 58
Step 10 # x x y y # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 59
Step 11 # x x y y # # { a n b n } q 3 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 60
Step 12 # x x y y # # { a n b n } q 3 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 61
Step 13 # x x y y # # { a n b n } q 4 Halt & Accept y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 62
Observation If we modify the n b n machine for the language g { a } we can easily construct t a machine for the language { a b c } n n n 63
Formal Definitions for Turing Machines 64
Transition Function a b, q q 2 1 δ ( 2 q 1, a) = ( q, b, ) 65
Transition Function c d, q q2 1 δ ( q 1, c ) = ( q2, d, ) 66
Turing Machine States Input alphabet Tape alphabet M = ( Q, Σ, Γ, δ, q, #, F 0 ) Transition Final function Initial state blank states 67
Configuration # # c a b a # # q 1 Instantaneous description: ca q 1 ba 68
Step 4 Step 5 # x a y b # # # x a y b # # q2 q0 A move: q2 xayb a x q0 ayb 69
Step 4 Step 5 # x a y b # # # x a y b # # q 2 q q 0 Step 6 Step 7 # x x y b # # # x x y b # # q1 q 1 q2 xayb a x q0 ayb a xx q1 yb a xxy q1 70 b
q2xayb a xq0ayb a xxq1 yb a xxq1 b Equivalent notation: q 2 xayb a xxy q 1 b 71
Initial configuration: q w q 0 Input string w # a a b b # # q 0 72
The Accepted anguage For any Turing Machine M ( M ) = { w : q0 w a x1 q f x2} Initial state Final state 73
Standard Turing Machine The machine we described is the standard: Deterministic Infinite tape in both directions Tape is the input/output file 74
Computing Functions with Turing Machines 75
Af function f (w ) Domain D (domän): has: w D ange S f ( w ) S (värdemängd): 76
A function may have many parameters: Example: Addition function f ( x, y) = x + y 77
Integer Domain Decimal: 5 Binary: 101 Unary: 11111 We prefer unary representation: easier to manipulate 78
Definition f A function is computable if M there is a Turing Machine such that: Initial configuration Final configuration # w # # f (w) # q0 initial state q f final state For all w D Domain 79
In other words f A function is computable if there is a Turing Machine M such that a q0 w a q f f (w) w Initial Configuration Final Configuration For all w D Domain 80
Example (Addition) The function f ( x, y) = x + y is computable x, y are integers Turing Machine: Input string: x00 y unary Output string: xy0 unary 81
x y Start 0 # 1 1 1 1 1 # q 0 initial state x + y Finishi # 1 1 1 1 0 # q f final state 82
Turing machine for function f ( x, y) = x + y 1 1, 1 1, 1 1, 0 1, 1 q0 q 1 2 # #, q 1 0, q3 q 4 # #, 83
Execution Example: Time 0 x y =11 x (2) # 1 1 0 1 1 # y =11 (2) q0 Final esult x + y # 1 1 1 1 0 # q 4 84
f ( x, y) = x + y Step 0 0 # 1 1 1 1 # q 0 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 85
f ( x, y) = x + y Step 1 # 1 1 0 1 1 # q 0 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 86
Step 2 # 1 1 0 1 1 # q 0 f ( x, y) = x + y 1 1, 1 1, 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 87
f ( x, y) = x + y Step 3 # 1 1 1 1 1 # q 1 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 88
f ( x, y) = x + y Step 4 # 1 1 1 1 1 # q 1 1 1, 1 1, 1 1 1, 1 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 89
f ( x, y) = x + y Step 5 # 1 1 1 1 1 # q 1 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 90
f ( x, y) = x + y Step 6 # 1 1 1 1 1 # q 2 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 91
f ( x, y) = x + y Step 7 # 1 1 1 1 0 # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 92
f ( x, y) = x + y Step 8 # 1 1 1 1 0 # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 93
f ( x, y) = x + y Step 9 # 1 1 1 1 0 # q 3 1 1, 1 1, 1 1 1, 1 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 94
f ( x, y) = x + y Step 10 # 1 1 1 1 0 # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 95
f ( x, y) = x + y Step 11 # 1 1 1 1 0 # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 96
f ( x, y) = x + y Step 12 # 1 1 1 1 0 # q 4 1 1, 1 1, 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 HAT & accept q 4 # #, 97
Another Example (Multiplication) The function f ( x ) = 2x is computable x is integer Turing Machine Input string: x unary Output string: xx unary 98
x Start 1 # 1 1 # q0 initial state 2x Finishi # 1 1 1 1 1 # q f final state 99
Turing Machine Pseudocode for f ( x ) = 2x eplace every 1 with $ epeat: Find rightmost $, replace it with 1 Go to right end, insert 1 Until no more $ remain 100
Turing Machine for f ( x ) = 2x 1 $, 1 1, 1 1, # #, q 0 1 # #, q3 q $ 1, q2 # 1, 101
Start f ( x) = 2x Finish # 1 1 # # 1 1 1 1 # q 0 1 $, 1 1, q 3 1 1, # #, q $ 1, 0 q 1 q2 # #, q3 # 1, 102
Step 0 # 1 # # # 1 f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 103
Step 1 # $ 1 # # # f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 104
Step 2 # $ $ # # # f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 105
Step 3 # # # # $ $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 106
Step 4 # 1 # # # $ f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 107
Step 5 # 1 1 # # $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 108
Step 6 # 1 1 # # $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 109
Step 7 # 1 1 1 # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 110
Step 8 # 1 1 1 # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 111
Step 9 # 1 1 1 # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 112
Step 10 # 1 1 1 1 # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 113
Step 11 # 1 1 1 1 # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 114
Step 12 # 1 1 1 1 # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 115
Step 13 # 1 1 1 1 # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 116
Step 14 # 1 1 1 1 # f ( x) = 2x q 3 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 117
Yet Another Example Turing Machine for f ( x, y ) = 1 if x > y 0 if x y The function is computable Input: x0y or Output: 1 0 118
Turing Machine Pseudocode epeat Match a 1 from x with a 1 from y Until all of x or y are matched x If a 1 from is not matched erase tape, write 1 else erase tape, write 0 119
Combining Turing Machines 120
Block Diagram input Turing Machine output 121
Example x + y if x > y f ( x, y) = 0 x y if Adder x + y x, y x > y Comparer x y 0 Eraser 122
Turing s Thesis 123
Question: Do Turing machines have the same power as a digital computer? Intuitive answer: Yes There is no formal proof! Among others because computers develop all the time. http://pub.ist.ac.at/~udiboker/files/arbitrarydomains.pdf The Church-Turing Thesis over Arbitrary Domains Udi Boker and Nachum Dershowitz 124
Turing s thesis (1930) Any computation carried out by mechanical means can be performed by a Turing Machine. http://www.engr.uconn.edu/~dqg/papers/myth.pdf p p p The Origins of the Turing Thesis Myth Goldin & Wegner 125
And, the other way round: A computation is mechanical if and only if it can be performed by a Turing Machine. Strong Turing Thesis: There is no known model of computation more powerful than Turing Machine. Wrong. Asynchronous parallel computation is more powerful than TM model, and cannot be reduced to TM. 126
Definition of an Algorithm An algorithm for function f (w) is a Turing Machine which computes f (w) 127
Algorithms are Turing Machines When we say: There exists an algorithm It means: There exists a Turing Machine. 128
anguages, Grammars, Machines properly inclusive anguage Grammar Machine Chomsky Hierarchy Non-computable?? ecursively enumerable (E) ecursive Unrestricted Turing machines 0 Turing machines that always halt 0 Context-sensitive Context-sensitive grammar inear-bounded automata 1 Context-free Context-free grammar Non-deterministic pushdown automata 2 egular egular expressions Finite state automata 3 129
{ a n! : n 0} { a b : n 0} n n n { a b c : n 0} { ww : w { a, b }} n Unrestricted grammar languages TUING MACHINES Context-free languages PUSH-DOWN AUTOMATA { a n n b : n 0} { ww : w { a, b }*} 2 n egular anguages FINITE AUTOMATA a *b* 130
Turing Machine Video Examples http://aturingmachine.com/ http://www.youtube.com/watch?v=cyw2ewoo6c4 EGO Turing Machine http://www2.math.uu.se/~salling/automata_dv/media/2011/turin g_machines/turing_machines.mov Salling on Turing Machines http://www.youtube.com/watch?v=e3keemwfhy Overview Turing Machines http://www.youtube.com/watch?v=abtoqfjrl4 Subtraction 131
Interesting Further eading http://www.mathcomp.leeds.ac.uk/turing2012/give-page.php?408 2012 The Alan Turing World http://www.ams.org/notices/200610/fea-feferman.pdf Turing s thesis Computational and evolutionary aspects of language http://www.nature.com/nature/journal/v417/n6889/full/nature00771.html https://sites.google.com/site/naturalcomputingaisbiacap2012 Symposium on Natural/Unconventional Computing AISB/IACAP World Congress 132