Introduction to Computational Complexity

Size: px

Start display at page:

Download "Introduction to Computational Complexity"

Roger Spencer
5 years ago
Views:

1 Introduction to Computational Complexity Jiyou Li Department of Mathematics, Shanghai Jiao Tong University Sep. 24th, 2013

2 Computation is everywhere Mathematics: addition; multiplication; functions; integrals; solving Diophantine or differential equations; proving theorems...

3 Computation is everywhere Mathematics: addition; multiplication; functions; integrals; solving Diophantine or differential equations; proving theorems... Computer Science: scientific computing; algorithms and data structures; information processing; artificial intelligence...

4 Computation is everywhere Mathematics: addition; multiplication; functions; integrals; solving Diophantine or differential equations; proving theorems... Computer Science: scientific computing; algorithms and data structures; information processing; artificial intelligence... Physics & Biology: dynamical systems such as weather evolution and SARS infectio; fetal development; emotional reactions (such as music listening)...

5 Math Theory of Computing Computer Science Biology, Physics...

6 Fundamental Problems in Computation 1. Will a virus in an evolution system spread, or die out? 2. What are the principles behind the development of life? 3. Is there a program to solve all equations? 4. Is there a program to prove all provable theorems? 5. Which problems can be computed? 6. Given any C-program, can we check if it will halt?

7 Algorithms in Mathematics 1. Inputs and outputs; 2. Step-by-step; 3. Finite language; 4. Compute on every possible input.

8 History of Algorithms B.C. 300: Euclid [proofs and algorithms: GCD algorithm ] A.D. 900 : al-khwãrizmĩ [arithmetic] A.D. 1940: Turing [defined algorithms]

9 Alan Turing ( )

10 Father of Computing 1936: /On computable numbers, with an application to the entscheindungsproblem0, Foundations of Computer Science; : Blechley Park, breaking Enigma; 1950: /Computing machinery and intelligence0.

11 Turing memorial statue plaque

12 Formal Languages 1. Σ = {σ 1, σ 2,..., σ k } is a finite set of symbols; 2. Σ is the set of all words over Σ; 3. A language L is a subset of Σ.

13 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where

14 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where Q is a finite set of states;

15 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where Q is a finite set of states; Γ is the tape alphabet including the blank Γ;

16 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where Q is a finite set of states; Γ is the tape alphabet including the blank Γ; q 0 Q is the initial state;

17 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where Q is a finite set of states; Γ is the tape alphabet including the blank Γ; q 0 Q is the initial state; F Q is the set of final states;

18 Definition of the Turing Machine A Turing machine M is a five-tuple M = (Q; Γ; δ; q 0 ; F) where Q is a finite set of states; Γ is the tape alphabet including the blank Γ; q 0 Q is the initial state; F Q is the set of final states; δ is the transition function: δ : (Q F ) Γ Q Γ {R, N, L}.

19 Configurations Start configuration: ( ; q 0 ; w) where w is the input; End configuration: (v; q; z) for q F where z is the output.

20 A Configuration

21 A state table Current Scanned Print Action Next 5-tuples A 0 1 R B (A, 0, 1, R, B) A 1 1 L C (A, 1, 1, L, C) B 0 1 L A (B, 0, 1, L, A) B 1 1 R B (B, 1, 1, R, B) C 0 1 L B (C, 0, 1, L, B) C 1 1 N z (C, 1, 1, N, z) Table: Here Q = {A, B, C, z}, Γ = {0, 1} and z F.

22 The Turing Machine and Insights Demo:

23 The Turing Machine and Insights Demo: Makes the concept of computation clear; Duality of program and input; The model is simple and can be easily simulated by a human; Turing machines are finite objects; Universality; The power of computation: computable functions; Church-Turing Thesis: Every function computable by any reasonable device, is computable by a Turing machine; The limits of computation: uncomputable functions.

24 The Turing Machine and Insights Algorithm: A Turing machine which halts in finite time on every possible (finite) input. Machine M on input x computes M(x) Duality: A Turing Machine can be input to another Turing Machine.

25 The limits of computation Does a given computer program P halt on all inputs?

26 The limits of computation Does a given computer program P halt on all inputs? Is a given computer program bug-free?

27 The limits of computation Does a given computer program P halt on all inputs? Is a given computer program bug-free? Is a math statement provable?

28 The limits of computation Does a given computer program P halt on all inputs? Is a given computer program bug-free? Is a math statement provable? Is a given equation solvable?

29 Universal Turing Machine Theorem There exists a universal Turing machine U, such that for all Turing machines M and all words w Σ, U(M; w) = M(w). In particular, U does not halt iff M does not halt.

30 Computable functions Interpret a Turing Machine M as a function f : Γ Γ. All such functions are called computable functions.

31 Acceptors For decision problem L, suppose M halt in F (finite states), either state q yes for positive instances w L or state q no for negative instances w L. We say M accepts the language L(M): L(M) := {w Σ a, b Σ, (ɛ; q 0 ; w) (a; q yes ; b)}.

32 Acceptors For decision problem L, suppose M halt in F (finite states), either state q yes for positive instances w L or state q no for negative instances w L. We say M accepts the language L(M): L(M) := {w Σ a, b Σ, (ɛ; q 0 ; w) (a; q yes ; b)}. L(M) := {w Σ M accepts w}.

33 Decidability and Undecidability A language L is called decidable, if there exists a Turing Machine with L(M) = L that halts on all inputs.

34 Decidability and Undecidability A language L is called decidable, if there exists a Turing Machine with L(M) = L that halts on all inputs. 1. Turing: Halting Problem is undecidable; 2. Turing: The question of if a math statement is provable is undecidable; 3. Mattiasevich: The solvability of a Diophantine is undecidable; 4. Conway: The problem of if a given epidemic will spread or die is undecidable.

35 Non-deterministic Turing Machine δ : (Q F ) Γ P(Q Γ {R, N, L}), where P(X) is the power set of X.

36 Non-deterministic Turing Machine δ : (Q F ) Γ P(Q Γ {R, N, L}), where P(X) is the power set of X. Given a non-deterministic Turing Machine N, one can construct a deterministic Turing Machine M with L(M) = L(N). Further, if N(w) accepts after t(w) steps, then there is c such that M(w) accepts after at most c t(w) steps.

37 Complexity What can algorithms do with restricted resources? How long can we solve a problem? How little disk-space can we use to solve a problem? What can we compute fast?

38 Computational Complexity Runtime and required space; Hard and easy problems; The importance of efficient algorithms; The P vs. NP problem; The ubiquity of NP-complete problems; Proving vs. Verifying; Reductions, Hardness, Completeness.

39 P and NP P: There is a deterministic Turing Machine M and a polynomial p(n) such that for each input w Σ the running time p( w ); NP: There is a nondeterministic Turing Machine M and a polynomial p(n) such that for each input w Σ the running time p( w );

40 Complexity Classes for Deterministic Turing Machine 1. LINTIME: Linear time; 2. P: Polynomial time: 3. EXP: Exponential time; 4. L: Logarithmic space; 5. PSPACE: Polynomial space; 6. EXPSPACE: Exponential space.

41 Complexity Classes for Non-deterministic Turing Machine 1. NLINTIME: Linear time; 2. NP: Polynomial time: 3. NEXP: Exponential time; 4. NL: Logarithmic space; 5. NPSPACE: Polynomial space; 6. NEXPSPACE: Exponential space.

42 Complexity Classes Turing Machine Theorem LINTIME P EXP. NLINTIME NP NEXP. P NP PSPACE = NPSPACE.

43 Complexity Classes LN P Solvable NP NP-complete

44 The equivalent definition of NP 1. A checking relation R Σ Σ 1 ; 2. Define a language L R = {w#y R(w, y)}; 3. R is polynomial-time iff L R P; 4. A language L over Σ is in NP iff there is k N and a polynomial-time checking relation R such that for all w Σ, w L y( y ( w ) k and R(w, y)).

45 P vs NP Problems in P: solutions can be efficiently found; Problems in NP: solutions can be efficiently checked; Problems in NP-complete: The "hardest" NP problems.

46 Clay Math Institute Millennium Problems Birch and Swinnerton-Dyer Conjecture Hodge Conjecture Navier-Stokes Equations P=NP? Poincaré Conjecture Riemann Hypothesis Yang-Mills Theory

47 Examples in P Matrix Multiplication (2.236); Determinant; Linear Programming; Primality test; Many Graph Problems (Connectivity, Perfect Matching, Shortest Path)...

48 Examples in NP, not known to be in P Factorization; Discrete Logarithm; Polynomial Identity Testing; Graph Isomorphism...

49 Theorem (Cook and Levin, 1970s) 3-SAT is NP-complete.

50 Examples in NP-complete 3-SAT (The Satisfiability Problem); Subset Sum Problem; SuDoku; Theorem proving; Integer Linear Programming; Many Graph Problems (TSP, Independent Set, Covering Set, Chromatic number)...

51 Exercises 1. Write a Turing Machine to compute the addition of two natural numbers; 2. Write a Turing Machine to compute the function f (x) = 3x; 3. Prove that both 1-SAT and 2-SAT are in P; 4. Find more interesting examples according to complexity classes.

Turing Machines and Time Complexity

Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move