CS6902 Theory of Computation and Algorithms
|
|
- Constance Willis
- 5 years ago
- Views:
Transcription
1 CS6902 Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational tool - procedure/analysis
2 Problem descriptions Formalize a problem
3 Sort the names of this class into alphabetic order lexicographically. - Abstract version of the problem: Instance: A set of names (last name follows middle name follows given name) Question: Find a list of these names in their lexicographical order - Decision version: of the problem (Yes/No) Instance: A set of names Question: Does the output list in lexicographical order? (Yes/No)
4 -Concrete version: by a reasonable encoding method, convert the decision version of the problem into binary string, say 01 string. - Problem and algorithm -> program in high level languages (C++, JAVA etc.) (programmer) -> assemble language (compiler) -> machine codes: 01-strings (assembler). This is machine acceptable.
5 A language L over alphabet E (a finite set of symbols, say {0,1}), is any set of string made up of symbols from E. - L={10, 11, 101, 111, 1011, 1101,10001, } is the language of binary representations of prime numbers. - L = {xx x in L and x in L } is that the concatenation of two languages L and L is the language L.
6 Problems can be formalized as Languages. - The decision problem PATH: Instance: A graph G=(V,E), vertices u and v, and a nonnegative integer k. Question: Does a path exist in G between u and v whose length is at most k? - The formal language version of PATH: PATH={<G, u, v, k> G=(V,E) is a graph, u, v in V, k in I+, and there exists a path from u to v in G whose length is at most k}.
7 Tools -Human hand with pen and paper, -Calculator -Computer The mathematical abstract of computational Tools correspond the computational models. The power of computational tools reflected by their corresponding models.
8 What is a computer? the computational power of a computer? -Turing Machine resources: storage (space) and time
9 Models of machines we will study are -Finite state automata (FSA): finite amount of storage and states. -linear bounded automata : on the size of the input of storage, finite states -Turing machine (TM): array -tape- with an unlimited number of entries, finite states
10 -Regarding the analysis of computational power of machine (algorithms) question... practical limitations (complexity) logical limitations (computability) we approach both issues formally, i.e., mathematically
11 Complexity: Algorithms / Problems Hierarchy of problems according to the complexity of algorithms to solve these problems. Undecidable (unsolvable) problems. Decidable (solvable) problems. NP-hard, NP-complete problems. Polynomial time solvable problems.
12 Figure 1: An simple illustration of complexity of problems
13 Undecidable (unsolvable) problems: (No algorithm exists) The Halting problem: Does there exist a program (algorithm/turing machine) Q to determine, for arbitrary given program P and input data D, whether or not P will halt on D?
14 Post's correspondence problem A correspondence system is a finite set P of ordered pairs of nonempty strings. A match of P is any string w such that for some n>0 and some pairs (u 1, v 1 ), (u 2, v 2 ),...,(u n, v n ) P, w = u 1 u 2...u n = v 1 v 2...v n.
15 For example If P = {(a, ab), (b, ca), (ca, a), (abc, c)}, then w=abcaaabc is a match of P, since if the following sequence of five pairs is chosen, the concatenation of the first components and the concatenation of the second components are both equal to w= abcaaabc : (a, ab), (b, ca), (ca, a), (a, ab), (abc, c). The post's correspondence problem is to determine given a correspondence system whether that system has a match.
16 Hilbert s tenth problem To devise an `algorithm that test whether or not a polynomial has an integral root. A polynomial is a sum of terms. For example, 6x 3 yz 2 + 3xy 2 - x 3 10 An integral root is a set of integer values setting to the polynomial so that it will be zero.
17 For example, the above polynomial has an Integral root: x=5, y=3, z=0 ( =0). Let D denote a set of polynomials so that D = {p p is a polynomial with an integral root} Hilbert s tenth problem becomes Is D decidable? The answer is that it is not decidable.
18 A brief idea of proof Let D be a set of special polynomials that p has one variable. I.e., D ={p p is polynomial over x with an integral root}. For example, 4x 3-2x 2 + x 7 Let M be a Turing Machine that input: a polynomial p over the variable x. Program: Evaluate p with x set successively to the values 0, 1, -1, 2, -2, 3, -3, If at any point the polynomial evaluated to zero, accept.
19 M recognizes D. For general polynomial, we can devise a Turing machine M to recognize D similarly. To set successively the values of all combinations to the variables, if the polynomial evaluated to zero, accept. For example, x y Can you set the value pattern as x y ?
20 M can be modified as decidable with D. The bound of the value of the single variable can be calculated. For example, 4x 3-2x 2 + x 7 has a bound of x < 7. Since the bound is finite, if the value has exceeded the bound and the polynomial is not evaluated to zero, then stop. Thus, M can decide D. The general bound for D is k C max /C 1. Matijasevic proved no such a bound for D.
21 An intuitive proof of Halting problem Let us assume there exits an algorithm Q with such a property, i.e., Q(P(D)) will be run forever if arbitrary algorithm P with input data D, P(D) run forever. halt if P(D) halts.
22 New algorithm B Note that Algorithm P is a string and data D is a string too. Thus, Q(P(P)) is also a legal input to Q, regarding P itself as data. Design an new Algorithm B(X) for any algorithm X such that B(X) Halts if Q(X(X)) runs forever Runs forever if Q(X(X)) halts
23 The construction of B Note that B can be constructed because Q can be constructed. For example, we may build B on Q as follows: When Q detects P(D) stops and Q shall stop, but we modify Q (called B) and let B run forever; while Q detects P(D) runs forever and Q shall run forever, but we modify Q (called B) and let B stop.
24 Contradiction Let B run with input data B, then B(B) will either halt or will run forever, and this can be detected by Q(B(B)). If B(B) stops, hence Q(B(B)) stops and forces B(B) runs forever by the construction of B. --- B(B) enters both stop and run forever.
25 continuous If B(B) runs forever, then Q(B,B) runs forever and forces B(B) stops. --- B(B) enters both stop and run forever. --- All statements are logically followed the assumption --- assumption is wrong --- there cannot exist such a program Q.
26 The diagonalization method This method is due Georg Cantor in Definitions: one-to-one function f : A to B if it never maps two different elements to the same place. f is onto if it hits every element of B. f is correspondence if it is both one-to-one and onto. Correspondence is to pair the elements in A and B.
27 The correspondence can be used to compare the size of two sets. Cantor extended this idea to infinite set. Definition: A set A is countable if either it is finite or it has the same size as natural numbers N. For example, N = {1,2,3, }, E={2,4,6, }, O={1,3,5, } are same size and hence countable. Q be set of rational numbers: Q={m/n m, n in N}
28 1/1 1/2 1/3 1/4 1/5 2/1 2/2 2/3 2/4 2/5 3/1 3/2 3/3 3/4 3/5 4/1 4/2 4/3 4/4 4/5.. 5/1 5/2 5/3 5/4 5/5.. Q is countable.
29 Real numbers R is uncountable. Let f make correspondence between N and R. n f(n) Construct a real number x by giving its decimal representation such that x is not belong to any f(n).
30 To do that, we let the first digit of the first real be different from the first digi of x, say x=.2; then let the second digit of the second real be different from the second digit of x, say x=.34; so on so forth. The new real number x=.34. is different from any real in the table by at least one digit difference. Therefore, x does not correspondence to any natural number and is uncountable. (can we choose 0 and 9 as digits in x?)
31 More example Is * for ={a,b} countable? ὲ a b aa bb ab ba aaa abb aba aab bbb baa bab bba Yes. aaaa aabb aaba aaab abbb abaa abab abba baaa babb baba baab bbbb bbaa bbab bbba 2 L. number of stings with length L
32 The correspondence of natural number N and the string with length L can be N = i=0 to L 2 L for the first string with length L. Arbitrarily assign a integer between 2 L and 2 L+1 1 to the rest of strings with length L. For example, string with length of 2 are aa ab bb ba.
33 Diagonalization for Halting problem Let M1, M2, M3,. be all Turing machines listed in rows of an infinite table. Obviously, they include those machines: P,Q,B. (Algorithm regarded as a machine.) Let (M1), (M2), (M3), be their descriptions (as strings) listed in columns. Let entry (i,j) represent the result of the i-th machine runs on the j-th description as input.
34 (M1) (M2) (M3) M1 accept rej/nstop accept M2 accept accept rej/nstop M3 rej/nstop accept rej/nstop.... When a machine M runs on a description as input, it either accept or reject or nonstop.
35 When a machine Q runs on a description (machine M runs on input D), it either accept or reject. (M1) (M2) (M3) M1 accept reject accept M2 accept accept reject M3 reject accept reject...
36 When a machine B runs on a description (machine B runs on input B), it is both accept and reject. (M1) (M2) (M3) (B) M1 accept reject accept M2 accept accept reject M3 reject accept reject.. B reject reject accept?. More problems than machine can solve.
37 Polynomial-time decidable problems: (Algorithms exist and relatively efficient) Sorting a set of elements. Find the maximum, minimum, and median of a set of elements, Matrix multiplication. Matrixchain multiplication, Single source shortest path. Convex hull of a set of points. Voronoi diagrams. Delaunay triangulations.
38 NP-hard, NP-complete problems: (Algorithms exist, but not efficient) Boolean Satisfiability problem, vertex cover problem. Hamiltonian-cycle problem. A hamiltonian cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. Does a graph G have a hamiltonian cycle? Traveling salespersons problem (Ω(m!), where m is the number of vertices in V.)
39 The measurement of the efficiency of algorithms (1) The worst-case time (and space). Insertion sort O(n 2 ) in worst-case time. (2) The average-case time. Quick sort O(n log n) time in average-case, O(n 2 ) in worst-case time. Other analysis methods:
40 The amortized analysis The randomized analysis In an amortized analysis, the time complexity is obtained by taking the average over all the operations performed. Even though a single operation in the sequence of operations may be very expensive, but the average cost of all operations in the sequence may be low. Example: Incrementing a binary counter. We shall count the number of flips in the counter when we keep adding a one from its lowest bit.
41
42 Increment(A) i 0 While i < length[a] and A[i]=1 Do A[i] 0 i i+1 If i < length[a] Then A[i] 1
43 In the conventional worst-case analysis, consider all k bits in the counter are `1's. any further increasing a `1' will cause k flips. Therefore, n incremental will cause O(kn) flips. Note that A[1] flip every time, A[2] flip every other time, A[3] flip every foutrth time,..., A[i] flip every 2 i th time. Thus, we have that log n i =0 n/2 i < n i=01/2 i = 2n. The average cost of each incremental is O(1), not O(k).
44 Optimal Algorithms Upper bound of a problem (1) the number of basic operations sufficient to solving a problem (2) the minimum time complexity among all known algorithms for solving a problem (3) upper bound can be established by illustrating an algorithm.
45 lower bound of a problem (1) the number of basic operations necessary to solving a problem (2) the maximum time complexity necessary by any algorithm for solving a problem (3) lower bound is much more difficult to establish.
46 An algorithm is optimal if its time complexity (i.e., its upper bound) matches with the lower bound of the problem. For example, the problem of sort n elements by comparisons. Lower bound = log 2 n! as there are n! different outcomes (permutations) and any decision tree which has n! leaves must be of height >= log 2 n!. Clearly, Merge-sort algorithm is optimal and insertion sort is not.
47 While you may already learn some methods of lower bound establishment such as decision tree and adversary (oracle), we shall also introduce a very useful method: Establish upper and lower bounds by transformable problems. Decision tree. Adversary. Transformation.
48 Figure 2Transfer of upper and lower bounds between transformable problems.
49 Suppose we have two problems, problem α and problem β, which are related so that problem α can be solved as follows: 1. The input to problem α is converted into a suitable input to problem β. 2. Problem β is solved. 3. The output of problem β is transformed into a correct solution to problem α We then say that problem α has been transformed to problem β. If the above transformation of step 1 and step 3 together can be done in O(τ(N)) time, where N is the size of problem α, then we say that α is τ(n)- transformable to β
50 Proposition 1 (Lower bound via transformability). If problem α is known to require at least T(N) time and α is τ(n)-transformable to problem α τ ( N ) β, then β requires at least T(N) - O(τ(N)) time. Proposition 2 (upper bound via transformability). If problem β can be solved in T(N) time and problem α is τ(n)-transformable to problem α τ ( N ) β, then α can be solved in at most T(N) + O(τ(N)) time.
51 For example. Element Uniqueness: Given N real numbers, decide if any two are unique. (Denote this problem as α.) This problem is known to have a lower bound. In the algebraic decision tree model any algorithm that determines whether the member of a set of N real numbers are distinct requires Ω(N log N) tests. Now, we have another problem, Closest Pair: Given N points in the Euclidean plane, find the closest pair of points (the shortest Euclidean distance). Denote this as β.
52 We want to find the lower bound of this problem. (Can we use decision tree method or adversary method to this problem?)
53 We transfer Element Uniqueness problem to Closest Pair problem. Given a set of real numbers (x 1, x 2,..., x N ) (INPUT to α), treat them as points in the y=0 line in the xycoordinate system (convert them into a suitable input of β). Apply any algorithm to solve β. The solution is the closest pair. If the distance between this pair is nonzero, then the points are distinct, otherwise it is not. (Convert the solution of β to the solution of α.) τ N = O(N). By Proposition 1, β takes at least Ω(N log N) - O(N) time, which is the lower bound.
54 Using the same method, we can prove that the lower bound of sorting by comparison operations is Ω(n log n), by transferring the Element Uniqueness to Sorting. The lower bounds of a chain of problems can be proved in this manner.
55 Reduction for intractability The above transformation method can be used for proving a problem is intractable or tractable if the cost of transformation is bounded by a polynomial. For example, CLIQUE: Instance: A graph G=(V,E) and a positive integer J <= V.
56 Question: Does G contain a clique of size J or more? That is, a subset V in V such that V >= J and every two vertices in V are joined by an edge in E. VERTEX COVER (VC): Instance: A graph G=(V,E) and a positive integer k <= V. Question: Is there a vertex cover of size k or less for G? That is, a subset V in V such that V <=k and for each edge in E at least one of the endpoints is in V.
57 Let A be VC and B be clique For every instance of A, we can convert it to an instance of B in polynomial time. Let G=(V,E) and k <= V be an instance of VC. The corresponding instance of Clique is G c and the integer j= V -k. For covert the output of B to an output of A in polynomial time (constant time yes/no). => If A is intractable, then B is intractable.
58 For every instance of B, we can convert it to an instance of A in polynomial time. Let G=(V,E) and j <= V be an instance of Clique. The corresponding instance of VC is G c and the integer k= V -j. => If B is tractable, then A is tractable.
59 clique V =(1,4,5),k=3 (a) complete graph of G (b) vertex cover V"=(2,3), k=n k=2 (c)
60 Reduction for decidability Mapping reducibility: If there is a computable function f: for every w in A, there is an f(w) in B. f is called the reduction of A to B. If A < m B, and B is decidable, then A is decidable. If A < m B, and A is undecidable, then B is undecidable.
61 Post correspondence problem Some instance has no match obviously. (abc, ab) (ca, a) (acc, ba) since the first element in an order pair is always larger than the second. Let us define PCP more precisely: PCP={[P] P is an instance of the Post correspondence problem with a match}
62 Where P={t 1 /b 1, t 2 /b 2, t k /b k }, and a match is a sequence i 1, i 2,, i s such that t i1 t i2 t is = b i1 b i2 b is. Proof idea: To show that for any TM M and input w, we can construct an instance P such that a match is an accepting computation history for M on w. Thus, if we can determine whether the instance P has a match, we can determine whether M accepts w (halting problem).
63 Let us call [t i /b i ] a domino. In the construction of P, we choose the dominos so that a match forces a simulation of M to accept ω. Let us consider a simpler case that M on ω does not move its head off the left-hand end of the tape and the PCP requires a match always starts with [t 1 /b 1. ] Call it MPCP. MPCP={[P] P is an instance of the Post correspondence problem with a match starting at [t 1 /b 1 ]}.
64 Proof. Let TM R decide the PCP and construct TM S to decide A TM. M =(Q,Σ, Γ, δ, q o, q accept, q reject,), where Q is the set of states, Σ is the input alphabet, Γ is the tape alphabet, δ is the transition function of M. S constructs an instance P of PCP such that P has a match if and only if M accepts ω. The construction of P of MCPC consists of 7 parts.
65 1. Let [# # q o w 1 w w w n #] be the first domino in P, [t 1 /b 1 ]. where C 1 =q o w = q o w 1 w w w n is the first configuration of M and # is the separator. The current P will force the extension of the top string in order to form a match.
66 To do so, we shall provide additional dominos to allow this, but at the same time these dominos causes a single step simulation of M, as shown in the bottom part of the domino. Parts 2, 3, and 4 are as follows: 2. For every a, b in Γ and every q, r in Q, where q =q reject if δ( q, a) = (r,b,r), put [qa/br] into P. (head moves right)
67 3. For every a, b, c in Γ and every q, r in Q, where q = q reject. if δ( q, a) = (r,b,l), put [cqa /rcb] into P. (head moves left) 4. For every a in Γ, put [a/a] into P. (head is not on symbol a) What do the above construction parts mean? Consider the following example: Let Γ ={0, 1, 2, e}, where e is empty, w= , and the start state of M is q o.
68 Part 1 puts the first domino as follows. [# ] [# q o ], then start to match. Suppose M in q o reads 0 and enters q 7, write a 2 on the tape and moves head to R. That is δ(q 0,0)=(q 7,2, R). Part 2 puts domino [q o 0 / 2 q 7 ], [# q o 0 ] [# q o # 2 q 7 ], Part 3 puts nothing, and Part 4 puts [0/0], [1/1], [2,2], and [e/e], [# q o # ] [# q o # 2 q #].
69 Part 5 for copy the # symbol to separate different configuration of M. I.e., put [#/#] and [#/e#] into P. The second domino allow us to add an empty symbol e to represent infinite number of blanks to the right. Thus, the current P has two configurations separated by #. [# q o # ] [# q o # 2 q #].
70 Now, suppose M in q 7 reads 1 and enters q 5, write a 0 on the tape and moves head to R. That is, δ(q 7,1)=(q 5,0, R). With [2/2], [0,0], [0,0], we have that [# q o # 2 q # ] [# q o 0100 # 2 q # 2 0 q #].
71 Then, suppose M in q 5 reads 0 and enters q 9, write a 2 on the tape and moves head to L. That is, δ(q 5,0)=(q 9,2, L). We have dominos: [0q 5 0 / q 9 02], [1q 5 0 / q 9 12], [2q 5 0 / q 9 22], and [eq 5 0 / q 9 e2]. Only the first domino fits. [# q o 0100 # 2 q # 20 q 5 00 # # q o 0100 # 2 q # 20 q 5 00 # 2 q # ]. This process of match and simulation M on ω continue until q accept has been reached.
72 We need to make a catch up for the top part of the current P. To do so, we have part For every a in Γ, put [a q accept / q accept ] and [q accept a / q accept ] into P. This is to add pseudo-steps to M after halted as the head eats the adjacent symbols until no symbol left. Suppose that M in q 9 reads 0 and enters q accept.
73 [# q o 0100 # 2 q # 20 q 5 00 # ] # q o 0100 # 2 q # 20 q 5 00 # 2 q # ]. [ #20q 5 00#2q # ] [ #20q 5 00#2q #2q accept 20#] [ #20q 5 00#2q #2q accept 20# ] [ #20q 5 00#2q #2q accept 20#2q accept 0#] [ #2q #2q accept 20#2q accept 0# ] [ #2q #2q accept 20#2q accept 0#2q accept #]
74 7. Finally, we add domino [q accept ##/#] to complete the match. [ #2q accept 20#2q accept 0#2q accept # ] [ #2q accept 20#2q accept 0#2q accept #q accept #] [ #2q accept 20#2q accept 0#2q accept #q accept ##] [ #2q accept 20#2q accept 0#2q accept #q accept ##]
75 To remove the restriction on P that must start at the first domino, we add some symbols to every element of P. Then, it must start at the first domino. If P = {[t 1 /b 1 ],[ t 2 /b 2 ],, [t k /b k ]} is a match, then we let P={[*t 1 /*b 1 *],[ *t 2 /b 2 *],, [*t k /b k *], [*o/o]}. Clearly, PCP must start at the first domino since only *b 1 * has left symbol *. [*o/o] for allowing the top of P to add #.
76
CS3719 Theory of Computation and Algorithms
CS3719 Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational tool - analysis
More informationCS6901: review of Theory of Computation and Algorithms
CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational
More informationReducability. Sipser, pages
Reducability Sipser, pages 187-214 Reduction Reduction encodes (transforms) one problem as a second problem. A solution to the second, can be transformed into a solution to the first. We expect both transformations
More informationTheory of Computation
Theory of Computation Lecture #2 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 1 Lecture 2: Overview Recall some basic definitions from Automata Theory.
More informationA Universal Turing Machine
A Universal Turing Machine A limitation of Turing Machines: Turing Machines are hardwired they execute only one program Real Computers are re-programmable Solution: Universal Turing Machine Attributes:
More informationLecture 12: Mapping Reductions
Lecture 12: Mapping Reductions October 18, 2016 CS 1010 Theory of Computation Topics Covered 1. The Language EQ T M 2. Mapping Reducibility 3. The Post Correspondence Problem 1 The Language EQ T M The
More information4.2 The Halting Problem
172 4.2 The Halting Problem The technique of diagonalization was discovered in 1873 by Georg Cantor who was concerned with the problem of measuring the sizes of infinite sets For finite sets we can simply
More informationUndecidable Problems and Reducibility
University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.
More informationUndecidable Problems. Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, / 65
Undecidable Problems Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, 2018 1/ 65 Algorithmically Solvable Problems Let us assume we have a problem P. If there is an algorithm solving
More informationUndecidability COMS Ashley Montanaro 4 April Department of Computer Science, University of Bristol Bristol, UK
COMS11700 Undecidability Department of Computer Science, University of Bristol Bristol, UK 4 April 2014 COMS11700: Undecidability Slide 1/29 Decidability We are particularly interested in Turing machines
More informationComputational Models Lecture 8 1
Computational Models Lecture 8 1 Handout Mode Ronitt Rubinfeld and Iftach Haitner. Tel Aviv University. May 11/13, 2015 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice
More informationCS Automata, Computability and Formal Languages
Automata, Computability and Formal Languages Luc Longpré faculty.utep.edu/longpre 1 - Pg 1 Slides : version 3.1 version 1 A. Tapp version 2 P. McKenzie, L. Longpré version 2.1 D. Gehl version 2.2 M. Csűrös,
More informationNotes for Lecture Notes 2
Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationCS5371 Theory of Computation. Lecture 15: Computability VI (Post s Problem, Reducibility)
CS5371 Theory of Computation Lecture 15: Computability VI (Post s Problem, Reducibility) Objectives In this lecture, we introduce Post s correspondence problem (playing with a special type of domino) We
More informationComputational Models Lecture 8 1
Computational Models Lecture 8 1 Handout Mode Nachum Dershowitz & Yishay Mansour. Tel Aviv University. May 17 22, 2017 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice
More information1 Computational Problems
Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationComputational Models Lecture 8 1
Computational Models Lecture 8 1 Handout Mode Ronitt Rubinfeld and Iftach Haitner. Tel Aviv University. April 18/ May 2, 2016 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice
More informationTheory of Computation - Module 4
Theory of Computation - Module 4 Syllabus Turing Machines Formal definition Language acceptability by TM TM as acceptors, Transducers - designing of TM- Two way infinite TM- Multi tape TM - Universal Turing
More informationThe Binomial Theorem.
The Binomial Theorem RajeshRathod42@gmail.com The Problem Evaluate (A+B) N as a polynomial in powers of A and B Where N is a positive integer A and B are numbers Example: (A+B) 5 = A 5 +5A 4 B+10A 3 B
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTATION
FORMAL LANGUAGES, AUTOMATA AND COMPUTATION DECIDABILITY ( LECTURE 15) SLIDES FOR 15-453 SPRING 2011 1 / 34 TURING MACHINES-SYNOPSIS The most general model of computation Computations of a TM are described
More informationNP-completeness. Chapter 34. Sergey Bereg
NP-completeness Chapter 34 Sergey Bereg Oct 2017 Examples Some problems admit polynomial time algorithms, i.e. O(n k ) running time where n is the input size. We will study a class of NP-complete problems
More informationUndecibability. Hilbert's 10th Problem: Give an algorithm that given a polynomial decides if the polynomial has integer roots or not.
Undecibability Hilbert's 10th Problem: Give an algorithm that given a polynomial decides if the polynomial has integer roots or not. The problem was posed in 1900. In 1970 it was proved that there can
More informationCSE 105 Homework 1 Due: Monday October 9, Instructions. should be on each page of the submission.
CSE 5 Homework Due: Monday October 9, 7 Instructions Upload a single file to Gradescope for each group. should be on each page of the submission. All group members names and PIDs Your assignments in this
More informationPart I: Definitions and Properties
Turing Machines Part I: Definitions and Properties Finite State Automata Deterministic Automata (DFSA) M = {Q, Σ, δ, q 0, F} -- Σ = Symbols -- Q = States -- q 0 = Initial State -- F = Accepting States
More informationCPSC 421: Tutorial #1
CPSC 421: Tutorial #1 October 14, 2016 Set Theory. 1. Let A be an arbitrary set, and let B = {x A : x / x}. That is, B contains all sets in A that do not contain themselves: For all y, ( ) y B if and only
More informationWeek 3: Reductions and Completeness
Computational Complexity Theory Summer HSSP 2018 Week 3: Reductions and Completeness Dylan Hendrickson MIT Educational Studies Program 3.1 Reductions Suppose I know how to solve some problem quickly. How
More informationACS2: Decidability Decidability
Decidability Bernhard Nebel and Christian Becker-Asano 1 Overview An investigation into the solvable/decidable Decidable languages The halting problem (undecidable) 2 Decidable problems? Acceptance problem
More informationChapter 4. Regular Expressions. 4.1 Some Definitions
Chapter 4 Regular Expressions 4.1 Some Definitions Definition: If S and T are sets of strings of letters (whether they are finite or infinite sets), we define the product set of strings of letters to be
More informationComputability and Complexity Theory
Discrete Math for Bioinformatics WS 09/10:, by A Bockmayr/K Reinert, January 27, 2010, 18:39 9001 Computability and Complexity Theory Computability and complexity Computability theory What problems can
More informationThe Turing Machine. Computability. The Church-Turing Thesis (1936) Theory Hall of Fame. Theory Hall of Fame. Undecidability
The Turing Machine Computability Motivating idea Build a theoretical a human computer Likened to a human with a paper and pencil that can solve problems in an algorithmic way The theoretical provides a
More informationNon-emptiness Testing for TMs
180 5. Reducibility The proof of unsolvability of the halting problem is an example of a reduction: a way of converting problem A to problem B in such a way that a solution to problem B can be used to
More information6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, Class 8 Nancy Lynch
6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 8 Nancy Lynch Today More undecidable problems: About Turing machines: Emptiness, etc. About
More informationCOMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages
COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed
More informationUNIT-VIII COMPUTABILITY THEORY
CONTEXT SENSITIVE LANGUAGE UNIT-VIII COMPUTABILITY THEORY A Context Sensitive Grammar is a 4-tuple, G = (N, Σ P, S) where: N Set of non terminal symbols Σ Set of terminal symbols S Start symbol of the
More informationSection 14.1 Computability then else
Section 14.1 Computability Some problems cannot be solved by any machine/algorithm. To prove such statements we need to effectively describe all possible algorithms. Example (Turing machines). Associate
More informationA canonical semi-deterministic transducer
A canonical semi-deterministic transducer Achilles A. Beros Joint work with Colin de la Higuera Laboratoire d Informatique de Nantes Atlantique, Université de Nantes September 18, 2014 The Results There
More informationIntroduction to Languages and Computation
Introduction to Languages and Computation George Voutsadakis 1 1 Mathematics and Computer Science Lake Superior State University LSSU Math 400 George Voutsadakis (LSSU) Languages and Computation July 2014
More informationDecidability (What, stuff is unsolvable?)
University of Georgia Fall 2014 Outline Decidability Decidable Problems for Regular Languages Decidable Problems for Context Free Languages The Halting Problem Countable and Uncountable Sets Diagonalization
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a
More informationComputational Theory
Computational Theory Finite Automata and Regular Languages Curtis Larsen Dixie State University Computing and Design Fall 2018 Adapted from notes by Russ Ross Adapted from notes by Harry Lewis Curtis Larsen
More informationTheory of Computation
Theory of Computation Lecture #6 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 39 Lecture 6: Overview Prove the equivalence of enumerators and TMs. Dovetailing
More informationCS151 Complexity Theory. Lecture 1 April 3, 2017
CS151 Complexity Theory Lecture 1 April 3, 2017 Complexity Theory Classify problems according to the computational resources required running time storage space parallelism randomness rounds of interaction,
More informationTheory of Computation p.1/?? Theory of Computation p.2/?? We develop examples of languages that are decidable
Decidable Languages We use languages to represent various computational problems because we have a terminology for dealing with languages Definition: A language is decidable if there is an algorithm (i.e.
More informationFriday Four Square! Today at 4:15PM, Outside Gates
P and NP Friday Four Square! Today at 4:15PM, Outside Gates Recap from Last Time Regular Languages DCFLs CFLs Efficiently Decidable Languages R Undecidable Languages Time Complexity A step of a Turing
More informationTheory of Computation (Classroom Practice Booklet Solutions)
Theory of Computation (Classroom Practice Booklet Solutions) 1. Finite Automata & Regular Sets 01. Ans: (a) & (c) Sol: (a) The reversal of a regular set is regular as the reversal of a regular expression
More informationECS 120 Lesson 20 The Post Correspondence Problem
ECS 120 Lesson 20 The Post Correspondence Problem Oliver Kreylos Wednesday, May 16th, 2001 Today we will continue yesterday s discussion of reduction techniques by looking at another common strategy, reduction
More informationAutomata & languages. A primer on the Theory of Computation. Laurent Vanbever. ETH Zürich (D-ITET) October,
Automata & languages A primer on the Theory of Computation Laurent Vanbever www.vanbever.eu ETH Zürich (D-ITET) October, 19 2017 Part 5 out of 5 Last week was all about Context-Free Languages Context-Free
More informationCSCE 551: Chin-Tser Huang. University of South Carolina
CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan
More informationP,NP, NP-Hard and NP-Complete
P,NP, NP-Hard and NP-Complete We can categorize the problem space into two parts Solvable Problems Unsolvable problems 7/11/2011 1 Halting Problem Given a description of a program and a finite input, decide
More information6.5.3 An NP-complete domino game
26 Chapter 6. Complexity Theory 3SAT NP. We know from Theorem 6.5.7 that this is true. A P 3SAT, for every language A NP. Hence, we have to show this for languages A such as kcolor, HC, SOS, NPrim, KS,
More informationLimitations of Algorithm Power
Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying
More information1 Showing Recognizability
CSCC63 Worksheet Recognizability and Decidability 1 1 Showing Recognizability 1.1 An Example - take 1 Let Σ be an alphabet. L = { M M is a T M and L(M) }, i.e., that M accepts some string from Σ. Prove
More informationComputational Complexity and Intractability: An Introduction to the Theory of NP. Chapter 9
1 Computational Complexity and Intractability: An Introduction to the Theory of NP Chapter 9 2 Objectives Classify problems as tractable or intractable Define decision problems Define the class P Define
More informationDecidability: Church-Turing Thesis
Decidability: Church-Turing Thesis While there are a countably infinite number of languages that are described by TMs over some alphabet Σ, there are an uncountably infinite number that are not Are there
More informationCP405 Theory of Computation
CP405 Theory of Computation BB(3) q 0 q 1 q 2 0 q 1 1R q 2 0R q 2 1L 1 H1R q 1 1R q 0 1L Growing Fast BB(3) = 6 BB(4) = 13 BB(5) = 4098 BB(6) = 3.515 x 10 18267 (known) (known) (possible) (possible) Language:
More informationCSCI3390-Lecture 6: An Undecidable Problem
CSCI3390-Lecture 6: An Undecidable Problem September 21, 2018 1 Summary The language L T M recognized by the universal Turing machine is not decidable. Thus there is no algorithm that determines, yes or
More informationTuring s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine
Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine There is no known model of computation more powerful than Turing Machines Definition of Algorithm:
More informationTuring Machines and Time Complexity
Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move
More informationPreliminaries and Complexity Theory
Preliminaries and Complexity Theory Oleksandr Romanko CAS 746 - Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006 Introduction Book structure: 2 Part I Linear Algebra
More information6.8 The Post Correspondence Problem
6.8. THE POST CORRESPONDENCE PROBLEM 423 6.8 The Post Correspondence Problem The Post correspondence problem (due to Emil Post) is another undecidable problem that turns out to be a very helpful tool for
More informationDecision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012
Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM
More informationECS 120 Lesson 18 Decidable Problems, the Halting Problem
ECS 120 Lesson 18 Decidable Problems, the Halting Problem Oliver Kreylos Friday, May 11th, 2001 In the last lecture, we had a look at a problem that we claimed was not solvable by an algorithm the problem
More information1 Reals are Uncountable
CS 30: Discrete Math in CS (Winter 2019): Lecture 6 Date: 11th January, 2019 (Friday) Topic: Uncountability and Undecidability Disclaimer: These notes have not gone through scrutiny and in all probability
More informationCSE 311: Foundations of Computing. Lecture 26: More on Limits of FSMs, Cardinality
CSE 311: Foundations of Computing Lecture 26: More on Limits of FSMs, Cardinality Last time: Languages and Representations All Context-Free??? Prove there is Regular a context-free DFA language 0* NFA
More informationFinish K-Complexity, Start Time Complexity
6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts
More informationCombinatorial Optimization
Combinatorial Optimization Problem set 8: solutions 1. Fix constants a R and b > 1. For n N, let f(n) = n a and g(n) = b n. Prove that f(n) = o ( g(n) ). Solution. First we observe that g(n) 0 for all
More informationacs-07: Decidability Decidability Andreas Karwath und Malte Helmert Informatik Theorie II (A) WS2009/10
Decidability Andreas Karwath und Malte Helmert 1 Overview An investigation into the solvable/decidable Decidable languages The halting problem (undecidable) 2 Decidable problems? Acceptance problem : decide
More informationFABER Formal Languages, Automata. Lecture 2. Mälardalen University
CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 2 Mälardalen University 2010 1 Content Languages, g Alphabets and Strings Strings & String Operations Languages & Language Operations
More informationTheory of Computation
Theory of Computation Unit 4-6: Turing Machines and Computability Decidability and Encoding Turing Machines Complexity and NP Completeness Syedur Rahman syedurrahman@gmail.com Turing Machines Q The set
More informationCS 125 Section #10 (Un)decidability and Probability November 1, 2016
CS 125 Section #10 (Un)decidability and Probability November 1, 2016 1 Countability Recall that a set S is countable (either finite or countably infinite) if and only if there exists a surjective mapping
More informationComplexity (Pre Lecture)
Complexity (Pre Lecture) Dr. Neil T. Dantam CSCI-561, Colorado School of Mines Fall 2018 Dantam (Mines CSCI-561) Complexity (Pre Lecture) Fall 2018 1 / 70 Why? What can we always compute efficiently? What
More informationCSCI 340: Computational Models. Regular Expressions. Department of Computer Science
CSCI 340: Computational Models Regular Expressions Chapter 4 Department of Computer Science Yet Another New Method for Defining Languages Given the Language: L 1 = {x n for n = 1 2 3...} We could easily
More informationCSC 8301 Design & Analysis of Algorithms: Lower Bounds
CSC 8301 Design & Analysis of Algorithms: Lower Bounds Professor Henry Carter Fall 2016 Recap Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized Simplex
More information} Some languages are Turing-decidable A Turing Machine will halt on all inputs (either accepting or rejecting). No infinite loops.
and their languages } Some languages are Turing-decidable A Turing Machine will halt on all inputs (either accepting or rejecting). No infinite loops. } Some languages are Turing-recognizable, but not
More informationComplexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler
Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard
More informationTuring Machines Part III
Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's
More informationOutline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.
Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität
More informationTuring Machines, diagonalization, the halting problem, reducibility
Notes on Computer Theory Last updated: September, 015 Turing Machines, diagonalization, the halting problem, reducibility 1 Turing Machines A Turing machine is a state machine, similar to the ones we have
More informationFurther discussion of Turing machines
Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will
More informationIn English, there are at least three different types of entities: letters, words, sentences.
Chapter 2 Languages 2.1 Introduction In English, there are at least three different types of entities: letters, words, sentences. letters are from a finite alphabet { a, b, c,..., z } words are made up
More information15-251: Great Theoretical Ideas in Computer Science Fall 2016 Lecture 6 September 15, Turing & the Uncomputable
15-251: Great Theoretical Ideas in Computer Science Fall 2016 Lecture 6 September 15, 2016 Turing & the Uncomputable Comparing the cardinality of sets A B if there is an injection (one-to-one map) from
More informationAuthor: Vivek Kulkarni ( )
Author: Vivek Kulkarni ( vivek_kulkarni@yahoo.com ) Chapter-3: Regular Expressions Solutions for Review Questions @ Oxford University Press 2013. All rights reserved. 1 Q.1 Define the following and give
More informationArtificial Intelligence. 3 Problem Complexity. Prof. Dr. Jana Koehler Fall 2016 HSLU - JK
Artificial Intelligence 3 Problem Complexity Prof. Dr. Jana Koehler Fall 2016 Agenda Computability and Turing Machines Tractable and Intractable Problems P vs. NP Decision Problems Optimization problems
More informationCSCI3390-Assignment 2 Solutions
CSCI3390-Assignment 2 Solutions due February 3, 2016 1 TMs for Deciding Languages Write the specification of a Turing machine recognizing one of the following three languages. Do one of these problems.
More informationNODIA AND COMPANY. GATE SOLVED PAPER Computer Science Engineering Theory of Computation. Copyright By NODIA & COMPANY
No part of this publication may be reproduced or distributed in any form or any means, electronic, mechanical, photocopying, or otherwise without the prior permission of the author. GATE SOLVED PAPER Computer
More informationTuring Machines. Lecture 8
Turing Machines Lecture 8 1 Course Trajectory We will see algorithms, what can be done. But what cannot be done? 2 Computation Problem: To compute a function F that maps each input (a string) to an output
More informationComputability Theory. CS215, Lecture 6,
Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David
More informationA non-turing-recognizable language
CS 360: Introduction to the Theory of Computing John Watrous, University of Waterloo A non-turing-recognizable language 1 OVERVIEW Thus far in the course we have seen many examples of decidable languages
More informationTheory of Computer Science
Theory of Computer Science C1. Formal Languages and Grammars Malte Helmert University of Basel March 14, 2016 Introduction Example: Propositional Formulas from the logic part: Definition (Syntax of Propositional
More informationStream Codes. 6.1 The guessing game
About Chapter 6 Before reading Chapter 6, you should have read the previous chapter and worked on most of the exercises in it. We ll also make use of some Bayesian modelling ideas that arrived in the vicinity
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/ Logistics HW7 due tonight Thursday's class: REVIEW Final exam on Thursday Dec 8, 8am-11am, LEDDN AUD Note card allowed
More informationNP-Complete Problems. Complexity Class P. .. Cal Poly CSC 349: Design and Analyis of Algorithms Alexander Dekhtyar..
.. Cal Poly CSC 349: Design and Analyis of Algorithms Alexander Dekhtyar.. Complexity Class P NP-Complete Problems Abstract Problems. An abstract problem Q is a binary relation on sets I of input instances
More informationCS 350 Algorithms and Complexity
CS 350 Algorithms and Complexity Winter 2019 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower
More informationTuring Machine Recap
Turing Machine Recap DFA with (infinite) tape. One move: read, write, move, change state. High-level Points Church-Turing thesis: TMs are the most general computing devices. So far no counter example Every
More informationSet Theory. CSE 215, Foundations of Computer Science Stony Brook University
Set Theory CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.edu/~cse215 Set theory Abstract set theory is one of the foundations of mathematical thought Most mathematical
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2017 http://cseweb.ucsd.edu/classes/sp17/cse105-ab/ Today's learning goals Summarize key concepts, ideas, themes from CSE 105. Approach your final exam studying with
More informationCS 350 Algorithms and Complexity
1 CS 350 Algorithms and Complexity Fall 2015 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower
More informationCS154, Lecture 17: conp, Oracles again, Space Complexity
CS154, Lecture 17: conp, Oracles again, Space Complexity Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string
More informationCSE 105 THEORY OF COMPUTATION. Spring 2018 review class
CSE 105 THEORY OF COMPUTATION Spring 2018 review class Today's learning goals Summarize key concepts, ideas, themes from CSE 105. Approach your final exam studying with confidence. Identify areas to focus
More informationSection Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence
Section 2.4 1 Section Summary Sequences. Examples: Geometric Progression, Arithmetic Progression Recurrence Relations Example: Fibonacci Sequence Summations 2 Introduction Sequences are ordered lists of
More information