and Models of Computation

Transcription

1 CDT314 FABE Formal anguages, Automata and Models of Computation ecture 12 Mälardalen University

2 Content Chomsky s anguage Hierarchy Turing Machines Determinism Halting TM Examples Standard TM Computing Functions with TM Combining TMs Turing Thesis 2

3 {! : n a n n n { a n b c : n 0} { ww : w { a, b }} ecursively enumerable languages Turing Machines 0} n n { a b : n Context-free languages Push-down Automata t 0} n 2 n { a b : n { ww : w { a, b}*} 0} egular anguages Finite Automata a *b a *b* 3

4 Turing Machines Theoretical Aspects Based on C Busch, PI, Models of Computation 4

5 TUING MACHINES Unlike egular anguages (with Finite Automata, egular Expressions and egular Grammars) and Context Free anguages (with Pushdown Automata and Context Free Grammars) with many practical applications, estriction Free anguages (and corresponding Turing Machines) are primarily important for Theoretical Computer Science as a means of study of the mechanisms and limits of computability. 5

6 Alan Turing Who was Alan Turing? Founder of Computability Theory, Mathematician, Codebreaker, Natural philosopher, Visionary man before his time. Alan Turing archive - Jack Copeland and Diane Proudfoot The Alan Turing Home Page Andrew Hodges 6

7 Alan Turing 1912 (23 June): Birth, ondon : Sherborne School 1930: Death of friend Christopher Morcom : 34: Undergraduate at King's College, Cambridge University : Quantum mechanics, probability, logic 1935: Elected fellow of King's College, Cambridge 1936: The Turing machine, computability, universal machine : Princeton University. Ph.D. ogic, algebra, number theory : eturn to Cambridge. Introduced to German Enigma cipher machine : The Bombe, machine for Enigma decryption : Breaking of U-boat Enigma, saving battle of the Atlantic 7

8 Alan Turing : Chief Anglo-American crypto consultant. Electronic work. 1945: National Physical aboratory, ondon 1946: Computer and software design leading the world : Programming, neural nets, and artificial intelligence 1948: Manchester University 1949: First serious mathematical use of a computer 1950: The Turing Test for machine intelligence 1951: Elected FS. Non-linear theory of biological growth 1952: Arrested as a homosexual, loss of security clearance : Unfinished work in biology and physics 1954 (7 June): Death (suicide) id by cyanide poisoning, i Wilmslow, l Cheshire. hi 8

9 Turing's Work and Scientific egacy The Mathematics of Emergence: The Mysteries of Morphogenesis Possibility of Building a Brain: Intelligent Machines, Practice and Theory Nature of Information: Complexity, andomness, Hiddenness of Information How should we compute? New Models of ogic and Computation 9

10 Hilbert s Program - formalization of mathematics, 1900 Hilbert s hope was that mathematics would be reducible to finding proofs (syntactic manipulating the strings of symbols) from a fixed system of axioms. Can all of mathematics be made algorithmic, or will there always be new problems that surpass any given algorithm, and so require creative acts of mind to solve? 10

11 Turing Machines and Computability The question Hilbert asked was: is there a general method or process by which one could decide whether a mathematical proposition could be proved? But what exactly was meant by a 'method' or 'process'? People had already used the concept of a 'mechanical' process, and Turing had an idea which made this quite precise: computation. 11

12 Turing Machines and Computability The Turing machine concept involves specifying a very restricted set of logical operations, but Turing showed how other more complex mathematical procedures could be built out of these atomic components. Turing argued that his formalism was sufficiently i general to encompass anything that a human being could do when carrying out a definite computation method. Question: How about geometric methods? There is a way to translate geometry into algebra algebraic geometry. 12

13 Turing Machines and Computability Turing's famous paper On computable numbers, with an application to the Entscheidungsproblem, which worked out the theory of Turing machines and the definition of computability, is available as a PDF file on-line:

14 Turing Machines Turing s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing) 14

15 Turing Machine Examples Based on C Busch, PI, Models of Computation 15

16 Tape... Turing Machine... Control Unit ead-write head 16

17 ... The Tape No boundaries -- infinite length... ead-write head The head moves eft or ight 17

18 ead-write head The head at each time step: 1. eads a symbol 2. Writes a symbol 3. Moves eft or ight 18

19 Example Step 0... a b a c... Step 1... a b k c eads a 2. Writes k 3. Moves eft 19

20 Step 1... a b k c... Step 2... a f k c eads b 2. Writes f 3. Moves ight 20

21 The Input String Input string Blank symbol... # # a b a c # # #... head Head starts t at the leftmost t position of the input string 21

22 States & Transitions ead Write Move eft a b, q q2 1 Move ight a b, q q2 1 22

23 Example Step 1... # # a b a c # # #... q 1 current state a b, q q

24 Step 1... # # a b a c # # #... q 1 Step # # a b b c # # # q 2 a b, q q2 1 24

25 Example Step 1... # # a b a c # # #... q 1 Step # # a b b c # # # q 2 a b, q q2 1 25

26 Example Step 1... # # a b a c # # #... q 1 Step 2... g # # a b b c g # #... # g, q q2 1 q 2 26

27 Determinism Turing Machines are deterministic Allowed Not Allowed q 1 a b, a b, q2 q 2 q d 3 q 3 a d, b d, q 1 No lambda transitions allowed in TM! 27

28 Determinism Note the difference between state indeterminism when not even possible future states are known in advance and choice indeterminism when possible future states are known, but we do not know which state will be taken. 28

29 Example Partial Transition Function... # # a b a c # # #... q 1 q 1 a b, b d, q 2 q 3 Allowed No transition for input symbol c 29

30 Halting The machine halts if there are no possible transitions to follow 30

31 Example... # # a b a c # # #... q 1 a b, b d, q 2 q 3 q 1 No possible transition HAT! 31

32 Final States q q 2 1 Allowed q1 q2 Not Allowed Final states have no outgoing transitions In a final state the machine halts 32

33 Acceptance Accept Input If machine halts in a final state eject Input If machine halts in a non-final state or If machine enters an infinite loop 33

34 Turing Machine Example A TM that accepts the language aa * a a, # #, q 0 q 1 34

35 Step 0 # # a a a # # q 0 a a, # #, q 0 q 1 35

36 Step 1 # # a a a # # q 0 a a, # #, q0 q 1 36

37 Step 2 # # a a a # # q 0 a a, # #, q0 q 1 37

38 Step 3 # # a a a # # q 0 a a, # #, q0 q 1 38

39 Step 4 # # a a a # # q 1 a a, Halt & Accept # #, q 0 q 1 39

40 ejection Example Step 0 # # a b a # # q 0 a a, # #, q0 q 1 40

41 Step 1 # # a b a # # q 0 a a, No possible Transition Halt & eject # #, q0 q 1 41

42 Infinite oop Example Another TM for language aa * b b, a a, # #, q0 q 1 42

43 Step 0 # # a b a # # q 0 b b, a a, q # #, 0 q 1 43

44 Step 1 # # a b a # # q 0 b b, a a, # #,# q0 q 1 44

45 Step 2 # # a b a # # q 0 b b, a a, # #, q0 q 1 45

46 Step 2 # # a b a # # q 0 Step 3 # # a b a # # q 0 Step 4 # # a b a # # q 0 Step 5 # # a b a # #... infinite loop q 0 46

47 Because of the infinite loop: The final state cannot be reached The machine never halts The input is not accepted 47

48 Another Turing Machine Example Turing machine for the language { a n b n } y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 48

49 Step 0 # a a b b # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a, x, b y,, q0 1 q q2 x x, 49

50 Step 1 # x a b b # # q 1 { a n b n } y y, q 4 # #,# y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 50

51 Step 2 # x a b b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 51

52 Step 3 # x a y b # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q0 1 q b y, q2 x x, 52

53 Step 4 # x a y b # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 53

54 Step 5 # x a y b # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 54

55 Step 6 # x x y b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 55

56 Step 7 # x x y b # # { a n b n } q 1 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 56

57 Step 8 # x x y y # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 57

58 Step 9 # x x y y # # { a n b n } q 2 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 58

59 Step 10 # x x y y # # { a n b n } q 0 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 59

60 Step 11 # x x y y # # { a n b n } q 3 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 60

61 Step 12 # x x y y # # { a n b n } q 3 y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 61

62 Step 13 # x x y y # # { a n b n } q 4 Halt & Accept y y, q 4 # #, y y, a a, y y, a a, q 3 y y, a x, q 0 1 q b y, q 2 x x, 62

63 Observation If we modify the n b n machine for the language g { a } we can easily construct t a machine for the language { a b c } n n n 63

64 Formal Definitions for Turing Machines 64

65 Transition Function a b, q q 2 1 δ ( 2 q 1, a) = ( q, b, ) 65

66 Transition Function c d, q q2 1 δ ( q 1, c ) = ( q2, d, ) 66

67 Turing Machine States Input alphabet Tape alphabet M = ( Q, Σ, Γ, δ, q, #, F 0 ) Transition Final function Initial state blank states 67

68 Configuration # # c a b a # # q 1 Instantaneous description: ca q 1 ba 68

69 Step 4 Step 5 # x a y b # # # x a y b # # q2 q0 A move: q2 xayb a x q0 ayb 69

70 Step 4 Step 5 # x a y b # # # x a y b # # q 2 q q 0 Step 6 Step 7 # x x y b # # # x x y b # # q1 q 1 q2 xayb a x q0 ayb a xx q1 yb a xxy q1 70 b

71 q2xayb a xq0ayb a xxq1 yb a xxq1 b Equivalent notation: q 2 xayb a xxy q 1 b 71

72 Initial configuration: q w q 0 Input string w # a a b b # # q 0 72

73 The Accepted anguage For any Turing Machine M ( M ) = { w : q0 w a x1 q f x2} Initial state Final state 73

74 Standard Turing Machine The machine we described is the standard: Deterministic Infinite tape in both directions Tape is the input/output file 74

75 Computing Functions with Turing Machines 75

76 Af function f (w ) Domain D (domän): has: w D ange S f ( w ) S (värdemängd): 76

77 A function may have many parameters: Example: Addition function f ( x, y) = x + y 77

78 Integer Domain Decimal: 5 Binary: 101 Unary: We prefer unary representation: easier to manipulate 78

79 Definition f A function is computable if M there is a Turing Machine such that: Initial configuration Final configuration # w # # f (w) # q0 initial state q f final state For all w D Domain 79

80 In other words f A function is computable if there is a Turing Machine M such that a q0 w a q f f (w) w Initial Configuration Final Configuration For all w D Domain 80

81 Example (Addition) The function f ( x, y) = x + y is computable x, y are integers Turing Machine: Input string: x00 y unary Output string: xy0 unary 81

82 x y Start 0 # # q 0 initial state x + y Finishi # # q f final state 82

83 Turing machine for function f ( x, y) = x + y 1 1, 1 1, 1 1, 0 1, 1 q0 q 1 2 # #, q 1 0, q3 q 4 # #, 83

84 Execution Example: Time 0 x y =11 x (2) # # y =11 (2) q0 Final esult x + y # # q 4 84

85 f ( x, y) = x + y Step 0 0 # # q 0 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 85

86 f ( x, y) = x + y Step 1 # # q 0 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 86

87 Step 2 # # q 0 f ( x, y) = x + y 1 1, 1 1, 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 87

88 f ( x, y) = x + y Step 3 # # q 1 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 88

89 f ( x, y) = x + y Step 4 # # q 1 1 1, 1 1, 1 1 1, 1 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 89

90 f ( x, y) = x + y Step 5 # # q 1 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 90

91 f ( x, y) = x + y Step 6 # # q 2 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 91

92 f ( x, y) = x + y Step 7 # # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 92

93 f ( x, y) = x + y Step 8 # # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 93

94 f ( x, y) = x + y Step 9 # # q 3 1 1, 1 1, 1 1 1, 1 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 94

95 f ( x, y) = x + y Step 10 # # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 q 4 # #, 95

96 f ( x, y) = x + y Step 11 # # q 3 1 1, 1 1, 1 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 # #, q 4 96

97 f ( x, y) = x + y Step 12 # # q 4 1 1, 1 1, 1 1, 0 1, # #, q0 q 1 2 q 1 0, q3 HAT & accept q 4 # #, 97

98 Another Example (Multiplication) The function f ( x ) = 2x is computable x is integer Turing Machine Input string: x unary Output string: xx unary 98

99 x Start 1 # 1 1 # q0 initial state 2x Finishi # # q f final state 99

100 Turing Machine Pseudocode for f ( x ) = 2x eplace every 1 with $ epeat: Find rightmost $, replace it with 1 Go to right end, insert 1 Until no more $ remain 100

101 Turing Machine for f ( x ) = 2x 1 $, 1 1, 1 1, # #, q 0 1 # #, q3 q $ 1, q2 # 1, 101

102 Start f ( x) = 2x Finish # 1 1 # # # q 0 1 $, 1 1, q 3 1 1, # #, q $ 1, 0 q 1 q2 # #, q3 # 1, 102

103 Step 0 # 1 # # # 1 f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 103

104 Step 1 # $ 1 # # # f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 104

105 Step 2 # $ $ # # # f ( x) = 2x q 0 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 105

106 Step 3 # # # # $ $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 106

107 Step 4 # 1 # # # $ f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 107

108 Step 5 # 1 1 # # $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 108

109 Step 6 # 1 1 # # $ f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 109

110 Step 7 # # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 110

111 Step 8 # # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 111

112 Step 9 # # # f ( x) = 2x q 2 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 112

113 Step 10 # # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 113

114 Step 11 # # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 114

115 Step 12 # # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 115

116 Step 13 # # f ( x) = 2x q 1 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 116

117 Step 14 # # f ( x) = 2x q 3 1 $, 1 1, 1 1, # #, $ 1, q q 1 q2 q 0 # #, q3 # 1, 117

118 Yet Another Example Turing Machine for f ( x, y ) = 1 if x > y 0 if x y The function is computable Input: x0y or Output:

119 Turing Machine Pseudocode epeat Match a 1 from x with a 1 from y Until all of x or y are matched x If a 1 from is not matched erase tape, write 1 else erase tape, write 0 119

120 Combining Turing Machines 120

121 Block Diagram input Turing Machine output 121

122 Example x + y if x > y f ( x, y) = 0 x y if Adder x + y x, y x > y Comparer x y 0 Eraser 122

123 Turing s Thesis 123

124 Question: Do Turing machines have the same power as a digital computer? Intuitive answer: Yes There is no formal proof! Among others because computers develop all the time. The Church-Turing Thesis over Arbitrary Domains Udi Boker and Nachum Dershowitz 124

125 Turing s thesis (1930) Any computation carried out by mechanical means can be performed by a Turing Machine. p p p The Origins of the Turing Thesis Myth Goldin & Wegner 125

126 And, the other way round: A computation is mechanical if and only if it can be performed by a Turing Machine. Strong Turing Thesis: There is no known model of computation more powerful than Turing Machine. Wrong. Asynchronous parallel computation is more powerful than TM model, and cannot be reduced to TM. 126

127 Definition of an Algorithm An algorithm for function f (w) is a Turing Machine which computes f (w) 127

128 Algorithms are Turing Machines When we say: There exists an algorithm It means: There exists a Turing Machine. 128

129 anguages, Grammars, Machines properly inclusive anguage Grammar Machine Chomsky Hierarchy Non-computable?? ecursively enumerable (E) ecursive Unrestricted Turing machines 0 Turing machines that always halt 0 Context-sensitive Context-sensitive grammar inear-bounded automata 1 Context-free Context-free grammar Non-deterministic pushdown automata 2 egular egular expressions Finite state automata 3 129

130 { a n! : n 0} { a b : n 0} n n n { a b c : n 0} { ww : w { a, b }} n Unrestricted grammar languages TUING MACHINES Context-free languages PUSH-DOWN AUTOMATA { a n n b : n 0} { ww : w { a, b }*} 2 n egular anguages FINITE AUTOMATA a *b* 130

131 Turing Machine Video Examples EGO Turing Machine g_machines/turing_machines.mov Salling on Turing Machines Overview Turing Machines Subtraction 131

132 Interesting Further eading The Alan Turing World Turing s thesis Computational and evolutionary aspects of language Symposium on Natural/Unconventional Computing AISB/IACAP World Congress 132