CSE 105 THEORY OF COMPUTATION

Transcription

1 CSE 105 THEORY OF COMPUTATION Fall

2 This Week Beyond Theory of Computation: Computational Complexity Reading: Sipser Chapter 7 Not covered in final exam Revisit familiar topics Diagonalization Reduction NP-complenetess

3 What we studied so far Decidable RE Context-free Regular co-re

4 Theory of Computation Question: What computational problems admit an algorithmic solution? Decidable problems: Yes, they can be solved by a TM Undecidable problems: No, no algorithmic solution Classification of computational problems based on restricted models of computation: DFA, NFA: Regular languages PDA, CFG: Context Free Languages Decidable: TM

5 Computational Complexity Goal: better understanding of decidable problems Computational Model: Turing Machine (or equivalent) Questions: How much time is needed to solve a problem? How much memory is needed to solve a problem? Are there problems that intrinsically require a lot of time or memory?

6 Measuring Time and Space Assume a TM M is run on input w Computation: Cinit C 1 C 2 C halt Time(M(w)): Number of steps in the computation Space(M(w)): Number of tape cells used during the computation. Equivalently, size of largest C k

7 Running time as a function Fix TM/program M Running time Time(M(w)) depends on input w It is natural to expect computation to take longer as w gets larger We are not insterested in exact running time We don t care if the running time (number of steps) is even or odd It is usually enough to know an upper bound on Time(M(w)) Time(n) = max { Time(M(w)) w =n} Maximum running time over all inputs of size n

8 Asymptotic notation Time(n) = O(f(n)) There are constant a,b (independent of n) such that T < a*f(n) + b Significance of a and b: You can think of b as some fixed startup cost You can think of a as using a slower or faster computer Theoretical goal Get bounds on running time that are independent of details of computational model Ignore low level details, for the sake of generality

9 Some example problems Addition: on input x and y, output x+y Multiplication: on input x and y, output x*y How can we model them as decision problems? Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. How fast can you add up or multiply numbers?

10 Some example problems What is the time complexity of Addition? Addition: on input x and y, output x+y A) T=O(n): You need to add all digits one by one B) T=O(3n): You need to readn 2n input digits and output an n-digit result C) T=1: Addition is a single machine instruction. D) It depends on how clock-speed, etc. Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 Multiplication: on input x and y, output x*y How can we model them as decision problems? Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. How fast can you add up or multiply numbers?

11 Some example problems What is the time complexity of Multiplication? Addition: on input x and y, output x+y A) T=O(n) because output is 2n digits long B) T=O(n 2 ) as taught in elemntary school C) Somewhere between O(n) and O(n 2 ), exact answer Multiplication: on input x and y, output x*y How can we model is them not known as decision problems? D) It depends on how clock-speed, etc. Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. How fast can you add up or multiply numbers?

12 Complexity of Multiplication Schoolbook algorithm: O(n 2 ) What is the complexity of multiplication? 1960: Karatsuba algorithm O(n ) 1971: Schonhage-Strassen algorithm O(n * log(n) * log(log(n))) 2007: Furer algorithm O(n * log(n) * exp(log*(n))) A) O(n / log n) B) O(n) C) O(n * log(n)) D) O(n * log(n) * exp(log*(n)))

13 How hard can it be? Even simple problems may require more than O(n) computation Parsing: decide L(G) for a CFG G Can be solved in Time = O(n 3 ) for any G Can it be solved in O(n 2 ) for any G? Can it be solved in O(n) for any G? Are there problems that require T=O(n100 )? Are there problems that require T=O(exp(n))?

14 Computationally hard problems Is there a decidable problem that Cannot be solved in time O(n 100 )? Cannot be solved in time O(exp(n))? Cannot be solved in time O(exp(exp(exp( exp(n) )))? How can we come up with such a problem? How do we show it cannot be solved in time T(n)?

15 An undecidable language We want a language L that is different from L(M 1 ) different from L(M 2 ) different from L(M 3 ).. different from L(M k )..

16 Diagonalization We want a language L that is different from L(M 1 ) at <M 1 > different from L(M 2 ) at <M 2 > different from L(M 3 ) at <M 3 >.. different from L(M k ) at <M k >.. Diag = { <M> M is a TM s.t. <M> is not in L(M) }

17 Diagonalization We want a language L that is different from L(M 1 ) at <M 1 > A) Diag is in RE, but not in core different from B) Diag L(Mis 2 core, ) at <Mbut 2 > not in RE different from C) Diag L(Mis 3 neither ) at <Min 3 > RE nor in core.. different from E) I don t L(Mknow k ) at <M k >.. Question: What can you tell about Diag? D) Diag is decidable Diag = { <M> M is a TM s.t. <M> is not in L(M) }

18 Twist on Diag Diag ={ <M> M is a TM s.t. <M> is not in L(M) } Diag2={<M> M is a TM s.t. M(<M>) does not accept in less than 2 <M> steps} <M> is in Diag2 if M(<M>) rejects, or M(<M>) takes more than 2 n steps

19 Twist on Diag Diag ={ <M> M is a TM s.t. <M> is not in L(M) } Diag2={<M> M is a TM s.t. M(<M>) does not accept in less than 2 <M> steps} <M> is in Diag2 if Question: What can you say about Diag2? 1. M(<M>) rejects, or A) Diag2 is in RE, but not in core 2. M(<M>) takes more than 2 n steps B) Diag2 is core, but not in RE C) Diag2 is neither in RE nor in core D) Diag2 is decidable

20 Twist on Diag Diag ={ <M> M is a TM s.t. <M> is not in L(M) } Diag2={<M> M is a TM s.t. M(<M>) does not accept in less than 2 <M> steps} <M> is in Diag2 if Question: How Diag2 compare to Diag? 1. M(<M>) rejects, or A) Diag is a subset of Diag2 2. M(<M>) takes more than 2 n steps B) Diag is a proper subset of Diag2 C) Diag is a superset of Diag2 D) Diag is a proper superset of Diag2

21 The complexity of Diag2 Diag2={<M> M is a TM s.t. M(<M>) does not accept in less than 2 <M> steps} So, Diag2 is decidable But, how fast can you solve Diag2? Is there an algorithm deciding Diag2 in time O(n 3 )? Is there an algorithm deciding Diag2 in time O(2 n )?

22 Diag2 cannot be solved in T<2 n Diag2={<M> M is a TM s.t. M(<M>) does not accept in less than 2 <M> steps} Assume there is a TM P such that L(P) = Diag2 Time(P(w))<2 n for every input of length w =n Does P accept w=<p>? Yes w is in Diag2 either P(w) rejects of T(P(w))>2 n Contradiction! No w is not in Diag2 P(w) accepts (in time <2 n ). Contradiction!