Theory of omputation F 101 Fundamental nformation Technology ssistant Prof. Dr. Turgay ĐBRĐKÇĐ - What can be computed? - an a computer solve any problem, given enough time and disk-space? - How fast can we solve a problem? - How little disk-space can we use to solve a problem -What problems can we solve given really very little space? (constant space) ourse slides are adapted from slides provided by ddison-wesley omputing Fundamentals of nformation Technology Theory of omputation What problems can a computer solve? ot all problems!!! g. iven a -program, we cannot check if it will not crash! Verification of correctness of programs is hence impossible! (The woe of icrosoft!) Theory of omputation What problems can a computer solve? ven checking whether a -program will halt/terminate is not possible! input n; assume n>1; while (n!=1) { if (n is even) n := n/2; else n := 3*n+1; } o one knows whether this terminates on on all inputs! 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1. Fundamentals of nformation Technology Fundamentals of nformation Technology Theory of omputation How fast can we compute a function? How much space do we require? Polynomial time computable on-det Poly Time (P) pproximation, Randomization Functions that cannot be computed fast: pplications to security ncrypt fast, Decryption cannot be done fast RS cryptography, web applications R S O P L X T Y Theory of omputation What can we compute? -- ost general notions of computability -- Uncomputable functions What can we compute fast? -- Faster algorithms, polynomial time -- Problems that cannot be solved fast: * ryptography What can we compute with very little space? -- onstant space (+stack) * String searching, language parsing, hardware verification, etc. Fundamentals of nformation Technology Fundamentals of nformation Technology 1
R S O P L X T Y Theory of omputation : --- Foundations of computing --- athematical methods of argument --- Simple setting R S O P L X T Y Theory of omputation ontext-free languages --- rammars, parsing --- achines with stack --- Still a simple setting; but infinite state Fundamentals of nformation Technology Fundamentals of nformation Technology R S Theory of omputation Turing machines (1940s): -- The most general notion of computing -- The hurch-turing thesis -- Limits to computing: Uncomputable functions Figure 11.1: n attempt to display the function that converts measurements in yards into meters O P L X T Y otivation from mathematics: an we solve any mathematical question methodically? odel s theorem: O! ven the most powerful machines cannot solve some problems. Fundamentals of nformation Technology Fundamentals of nformation Technology 10 Figure 11.2: The components of a Turing machine Figure 11.3: Turing machine for incrementing a value 1: tape which is divided into cells, one next to the other. 2: head that can read and write symbols on the tape and move left and right. 3: state register that stores the state of the Turing machine 4: n action table (or transition function) Fundamentals of nformation Technology 11 Fundamentals of nformation Technology 12 2
HURH-TUR THSS ny well-defined procedure that can be grasped and performed by the human mind and pencil/paper, can be performed on a conventional digital computer with no bound on memory. Figure 11.4: Bare Bones program for computing X Y Z = X * Y; The hurch-turing Thesis is OT a theorem. t is a statement of belief concerning the universe we live in. Fundamentals of nformation Technology 13 Fundamentals of nformation Technology 14 Figure 11.5: Bare Bones implementation of the instruction copy Today to Tomorrow Tomorrow = Today; tmp = 0; Tomorrow = 0; while(today) { tmp++; Today--; } while (tmp) { Today++; Tomorrow+; tmp--; } lan Turing (1912-1954) Published On omputable umbers (1936) ntroduced the Halting Problem Formal model of computation (now known as Turing achine ) odebreaker at Bletchley Park Broke nigma ipher Perhaps more important than Lorenz fter the war: convicted of homosexuality (then a crime in Britain), committed suicide eating cyanide apple Fundamentals of nformation Technology 15 Fundamentals of nformation Technology 16 Bletchley Park During World War the erman armed forces top secret codes were broken at Bletchley Park, providing the allies with vital information towards their war effort. Situated 50 miles orth-west of London, the site played host to a diverse group of code breakers, including lan Turing and Dilly Knox. mong the ciphers that were broken were nigma and Lorenz. The aesar ipher One of the simplest examples of a substitution cipher is the aesar cipher, which is said to have been used by Julius aesar to communicate with his army. aesar is considered to be one of the first persons to have ever employed encryption for the sake of securing messages. aesar decided that shifting each letter in the message would be his standard algorithm, and so he informed all of his generals of his decision, and was then able to send them secured messages. Using the aesar Shift (3 to the right), the message, would be encrypted as: "RTUR TO RO" "UHWXU WR URPH" Fundamentals of nformation Technology 17 Fundamentals of nformation Technology 18 3
The nigma ipher The nigma cipher is most well known for it's contributions to World War on the ermans' side. They developed what came to be known as The nigma achine. The machine was based on a system of three rotors that substituted cipher text letters for plain text letters. The rotors would spin in conjunction with each other, thus performing varying substitutions much like the aeser Shift. Theory The theory of what can and can t be computed by an ideal computer is called Theory or Recursion Theory. Fundamentals of nformation Technology 19 Fundamentals of nformation Technology 20 omputable Function Fix any finite set of symbols, Σ. Fix any precise programming language, e.x., Java. program is a finite string of characters that is syntactically valid. function f: Σ * -> Σ* is computable if there is a program P that when executed on an ideal computer, computes f. That is, for all strings x belongs to Σ *, P(x) = f(x). Fundamentals of nformation Technology 21 Decidability problem P is decidable if some Turing machine decides (solves) the problem. onsider problems with answer YS or O Decidable problems: Does achine have three states? s string w a binary number? Does DF accept any input? Fundamentals of nformation Technology 22 Decision problems decision problem is a computational problem with a yes or no answer. xample: s the number n prime? Why focus on decision problems? Decision problems are simple: This makes it easy to develop a rigorous mathematical theory. Decision problems are surprisingly general: any other problems can be recast in terms of decision problems that are essentially equivalent. Fundamentals of nformation Technology 23 Recasting other problems as decision problems ultiplication problem: What is the product of m and n? ultiplication decision problem: s the kth bit of the product of m and n a one? Time required to solve one of these problems is the same (to within a small overhead) as the time required to solve the other. Factoring problem: What is the smallest non-trivial factor of n? Factoring decision problem: Does n have a non-trivial factor smaller than k? Time required to solve one of these problems is the same (to within a small overhead) as the time required to solve the other. Fundamentals of nformation Technology 24 4
The machine that decides (solves) a problem: The Turing machine that decides (solves) a problem answers YS or O for each instance of the problem f the answer is YS then halts in a yes state nput problem instance Turing achine YS O f the answer is O then halts in a no state These states may not be final states Fundamentals of nformation Technology 25 Fundamentals of nformation Technology 26 Turing achine that decides a problem YS states O states Some problems are undecidable: which means: there is no Turing achine that solves all instances of the problem problem is undecidable if it cannot be solved by any Turing machine that halts on all inputs. simple undecidable problem: YS and O states are halting states Fundamentals of nformation Technology 27 The Halting problem Fundamentals of nformation Technology 28 The Halting problem Figure 11.7: Proving the unsolvability of the halting program (1/3) P(x) means the output that arises from running program P on input x P(P) means the output obtained when we run P on the text of its own source code. Write a program HLT such that: HLT(P) = yes, if P(P) halts HLT(P) = no, if P(P) does not halt Fundamentals of nformation Technology 29 Fundamentals of nformation Technology 30 5
Figure 11.7: Proving the unsolvability of the halting program (2/3) Figure 11.7: Proving the unsolvability of the halting program (3/3) Fundamentals of nformation Technology 31 Fundamentals of nformation Technology 32 Does program number x halt on input of x? 0 if program x halts on input x h( x ) 1 otherwise s there an algorithm to solve the halting problem, that is, to compute h(x)? PROR: TUR(x) Suppose such an algorithm exists. Let T be the program number for TUR. h(t) = 0 The halting problem again ontradiction! F h(x) = 1 TH HLT LS loop forever TUR(T) halts h(t) = 1 Fundamentals of nformation Technology 33 ondeterministic Turing achine States 0 nitial 1 Record 2 Look for 0 3 Look for 1 4 Scan Left 5+Rest of program urrent State Read Symbol B 0 1 0 1,B,R 1 accept,b,r 2,B,R 3,B,R 2 2,0,R 4,2,L 2,1,R 3 3,0,R 3,1,R 4,2,L 4 5,B,R 4,0,L 4,1,L ondeterministic T: 2 possible actions from single point means no possible action from this point Deterministic T: t most one possible action at any point Fundamentals of nformation Technology 34 Figure 11.12: graphic summation of problem classification Reducibility lgorithm for Problem B s P P? (ondeterministic Polynomial time) US$1,000,000 dollars for a solution!!!!!!!!!! gone : Dr. rigoriy Perelman http://www.claymath.org/millennium/ Fundamentals of nformation Technology 35 x nput for Problem B Reduction from B to R(x) lgorithm for Yes/o Output for Problem B P-Hard: problem Π is P-hard if every problem in P has a polynomial-time reduction to Π. f Π is in ΝP and P-hard then Π is P-complete. oral: t least as hard as any other problem in P Fundamentals of nformation Technology 36 6
P-omplete What, intuitively, does it mean if we can reduce problem P to problem Q? P is no harder than Q How do we reduce P to Q? Transform instances of P to instances of Q in polynomial time s.t. Q: yes iff P: yes What does it mean if Q is P-Hard? very problem P P p Q What does it mean if Q is P-omplete? Q is P-Hard and Q P Boolean ST problem iven a proposition logic formula, decide if it is satisfiable. formula is satisfiable if there exists a variable assignment such that the formula becomes true. Boolean ST is P complete (ook 1971) Fundamentals of nformation Technology 37 Fundamentals of nformation Technology 38 ook s Theorem ST is P-complete an generate Boolean formula that checks whether DT accepts string in polynomial time Translation Procedure iven DT Polynomial function p x Translation nput string x enerate formula F p F is satisfiable iff accepts x in time p( x ) Size of F is polynomial in x Procedure generates F in (deterministic) time polynomial in x F Figure 11.13: ncrypting a bit pattern as a knapsack problem Fundamentals of nformation Technology 39 Fundamentals of nformation Technology 40 Figure 11.14: Public key encryption using knapsack problems Figure 11.15: onstructing a public key encryption system Fundamentals of nformation Technology 41 Fundamentals of nformation Technology 42 7
Public-Key ryptographic lgorithms The RS lgorithm Key eneration RS and Diffie-Hellman RS - Ron Rives, di Shamir and Len dleman at T, in 1977. RS is a block cipher The most widely implemented Diffie-Hellman in 1976 change a secret key securely ompute discrete logarithms (64Bytes) 1. Select p,q p and q both prime 2. alculate n = p x q 3. alculate Φ( n) = ( p 1)( q 1) 4. Select integer e gcd( Φ( n), e) = 1;1 < e < Φ( n) 5. alculate d 1 d = e mod Φ ( n) 6. Public Key KU = {e,n} 7. Private key KR = {d,n} Block cipher vs. Stream cipher Fundamentals of nformation Technology 43 Fundamentals of nformation Technology 44 xample of RS lgorithm xample of RS lgorithm (cont.) 1. Select p,q p =7, q =17 2. alculate n = p x q =7 x 17 = 119 3. alculate = 96 Φ( n) = ( p 1)( q 1) 4. Select integer e=5 gcd( Φ( n), e) = 1;1 < e < Φ( n) 5. alculate d =77 1 d = e mod Φ( n) 6. Public Key KU = {e,n} = {5, 119} 7. Private key KR = {d,n} = {77, 119} 77 x 5 = 385 = 4 x 96 + 1 Fundamentals of nformation Technology 45 Fundamentals of nformation Technology 46 Diffie-Hellman Key change hecksumming: yclic Redundancy heck(r) view data bits, D, as a binary number choose r+1 bit pattern (generator), goal: choose r R bits, R, such that <D,R> exactly divisible by (modulo 2) receiver knows, divides <D,R> by. f non-zero remainder: error detected! can detect all burst errors less than r+1 bits widely used in practice (T, HDL) Fundamentals of nformation Technology 47 Fundamentals of nformation Technology 48 8
R xample Want: D. 2 r XOR R = n equivalently: D. 2 r = n XOR R equivalently: if we divide D. 2 r by, want remainder R D R. 2 r = remainder[ ] Fundamentals of nformation Technology 49 Fundamentals of nformation Technology 50 9