Limits of Computation

Size: px

Start display at page:

Download "Limits of Computation"

Augusta Hart
5 years ago
Views:

1 The real danger is not that computers will begin to think like men, but that men will begin to think like computers Limits of Computation - Sydney J. Harris What makes you believe now that I am just talking nonsense? - Eliza

2 2-Flavors of Processing Elements Combinational Logic: Table look-up Recall that there are precisely 2 2 i, i-input digital combinational functions. Finite State Machines: Tables with State i i Addr Data o o Is there any more to a computer than tables and state? Thus far, we know of nothing more powerful than an FSM s Addr Data

3 FSMs as Programmable Machines What is an FSM s table s size? Given i, s, and o, we need a table organized as: 2 i+s rows x (o+s) bits So how many possible i-input, o-output, FSMs with s-state bits exist? Recall how we were able to enumerate or name every 2-input gate? Can we do the same for FSMs? s The number of bits in the Table i 2 i+s An FSM s behavior is completely determined by the contents of its lookup table. i inputs s N outputs s N+1 o All possible settings of the table to: 1 or 0 2 (o+s)2i+s (some may be equivalent) o

4 GOAL: List all possible FSMs in some canonical order. INFINITE list, but Every FSM has an entry in and an associated index. FSM Enumeration These FSMs have 2 inputs and 2 outputs, or 8-bits in their ROM. 2 8 FSMs i inputs s N o outputs s N+1 18,446,744,073,709,551, x Every possible FSM can be associated with a unique number. This requires a few wasteful simplifications. First, given an i-input, s-state-bit, and o-output FSM, we ll replace it with its equivalent n-input, n-state-bit and n-output FSM, where n is the greatest of i, s, and o. We can always ignore the extra input-bits, and set the extra output bits to 0. This allows us to discuss the i th FSM

5 FSM 837 FSM 1077 FSM 1537 FSM Some Perennial Favorites... modulo 3 counter 4-bit counter Combination lock Cheap digital watch FSM Intel Pentium CPU rev 1 FSM Intel Pentium CPU rev 2 FSM Intel Pentium II CPU

6 Are FSMs the Ultimate Digital Device? Nope! There exist many simple problems that cannot be computed by FSMs. For instance: Checking for balanced parenthesis (()(()())) - Okay (()())) - No good! PROBLEM: Requires ARBITRARILY many states, depending on input. Must "COUNT" unmatched LEFT parens. But, an FSM can only keep track of a finite number of objects. Is there a machine that can solve this problem?

7 Unbounded-Space Computation S 1 0,(1,R) 1,(1,L) 0,(1,L) 1,Halt S 2 DURING 1920s & 1930s, much of the science part of computer science was being developed (long before actual electronic computers existed). Many different Models of Computation were proposed, and the classes of functions that each could compute were analyzed. One of these models was the TURING MACHINE, named after Alan Turing. A Turing Machine is just an FSM which receives its inputs and writes outputs onto an infinite tape... Alan Turing This simple addition solves "FINITE" problem of FSMs.

8 A Turing Machine Example Turing Machine Specification Doubly-infinite tape Discrete symbol positions Finite alphabet say {0, 1} Control FSM INPUTS: Current symbol OUTPUTS: write 0/1 move Left/Right Initial Starting State {S0} Halt State {Halt} A Turing machine, like an FSM, can be specified with a truth table. The following Turing Machine implements a unary (base 1) incrementer

9 Turing Machine Tapes as Integers Canonical names for bounded tape configurations: b 8 b 6 b 4 b 2 b 0 b 1 b 3 b 5 b FSM i Look, it s just FSM i operating on tape j

10 TMs as Discrete Functions Turing Machine T i operating on Tape x, where x = b 8 b 7 b 6 b 5 b 4 b 3 b 2 b 1 b 0 y = T [x] i x: input tape configuration y: output tape when TM halts I wonder if a TM can compute EVERY integer function...

11 Alternative Models of Computation Turing Machines [Turing] Hardware head Recursive Functions [Kleene] Theory head FSM i (define (fact n) (... (fact (- n 1))...) Kleene Turing Lambda calculus [Church, Curry, Rosser...] Math head λx.λy.xxy Production Systems [Post, Markov] α β Language head IF pulse=0 THEN patient=dead (lambda(x)(lambda(y)(x (x y)))) Post Church

12 The 1 st Computer Industry Shakeout Here s a TM that computes SQUARE ROOT! FSM

13 And the Battles Raged Here s a Lambda Expression that does the same thing... (λ(x)...)... and here s one that computes the n th root for ANY n! (λ(x n)...)

14 Fundamental Result: Each model is capable of computing exactly the same set of integer functions! Proof Technique: BIG IDEA: Constructions that translate between models Computability, independent of computation scheme chosen Does,this mean that we know of no computer that is more powerful than a Turing machine? Church's Thesis: Every discrete function computable by ANY realizable machine is computable by some Turing machine.

15 Computable Functions f(x) computable <=> for some k, all x: f(x) = T K [x] f K (x) Representation tricks: to compute f k (x,y) <x,y> integer whose even bits come from x, and whose odd bits come from y; whence f K (x, y) T K [<x, y>] f (x,y) = x * y f (x) = 1 iff x is prime, else 0

16 Enumeration of Computable functions Conceptual table of TM behaviors... VERTICAL AXIS: Enumeration of TMs. HORIZONTAL AXIS: Enumeration of input tapes. (j, k) entry = result of TM k [j] -- integer, or * if never halts. f i f 1 f k f i (0) f i (1) f i (2) f i (j) fx X 1 23 X0 X 1 62X0 * * f k (j) Is every Integer Function that I can precisely specify computable? The Halting Problem: Given j, k: Does TM k Halt with input j?

17 The Halting Function: T H [k, j] = 1 iff TM k [j] halts, else 0 The Halting Problem Can a Turing machine compute this function? Replace the Halt state of TH with this. Suppose, for a moment, T H exists: 1 iff Tx[y] HALTS 0 otherwise Then we can build a T Nasty : T H x y If T H is computable then so is T Nasty 1,(1,L) -,(0,R) N 2 N 1 0,(0,L) LOOP HALT 1 0? T H k N H T Nasty [k] LOOP if T k [k] = 1 (halts) HALT if T k [k] = 0 (loops)

18 What does T Nasty [Nasty] do? Answer: T Nasty [Nasty] loops if T Nasty [Nasty] halts T Nasty [Nasty] halts if T Nasty [Nasty] loops That s a contradiction. Thus, T H is uncomputable by a Turing Machine! Net Result: There are some questions that Turing Machines simply cannot answer. Since, we know of no better model of computation than a Turing machine, this implies that there are some questions that defy computation.

19 Reality: Limits of Turing Machines A Turing machine is formal abstraction that addresses Fundamental Limits of Computability What is means to compute. The existence of uncomputable functions. We know of no machine more powerful than a Turing machine in terms of the *functions* that it can compute.

20 Do Computers Think? What does it mean to think as people think? Is thought mechanistic? A combination of memory and strategies? Computers are invaluable tools for simulating complex systems. Is thought in the brain merely a by-product of electrical impulses and biochemistry? If so, might we simulate these processes using a computer program? The Turing Test Described by Alan Turing in a 1950 paper "Computing machinery and intelligence" A judge engages in conversations with a human and a machine, each of which tries to appear human; if the judge cannot reliably distinguish which is which, then the machine is said to pass the test. Eliza ( In 1966, Joseph Weizenbaum, an MIT researcher with combined interests in artificial intelligence and natural language processing, wrote Eliza, to parody a computer therapist. A surprising number of people have been momentarily fooled by it.

Are Computer s Intelligent? Intelligence is a trait the we attribute- Its definition has changed over time. In the Middle Ages, Reading and Reckoning were the hallmarks of intelligence.

21 Are Computer s Intelligent? Intelligence is a trait the we attribute- Its definition has changed over time. In the Middle Ages, Reading and Reckoning were the hallmarks of intelligence. Today intelligence is is defined in terms of an ability to reason, plan, and think abstractly. It might include charateristics such as creativity, personality, character, and wisdom. Artificial Intelligence- A branch of computer science focused on developing algorithms for reasoning, planning, learning, and acquiring knowledge. Computer Chess In 1997, a computer program running on a supercomputer beat the reigning world chess champion, Gary Kasparov. For the first time ever, a non-human was the best chess player in the world.

Brute Force vs. Finesse In 1950, Claude Shannon, published one of the first papers on how a computer might play chess. He hypothesized two possible strategies.

22 Brute Force vs. Finesse In 1950, Claude Shannon, published one of the first papers on how a computer might play chess. He hypothesized two possible strategies. The first, was a brute force approach involving enumerating all possible moves in a given position, and all possible replies to that move, for as many plies as time and memory would allow. Then every final position would be evaluated an this tree, and a search would select the best MinMax solution (Find the path that would maximize your position, while minimizing your opponents while assuming both of you make the best choice). He argued that such an approach would be impractical. His second strategy was to analyze the current position first, and then to consider only a small number of moves on each ply. He predicted this would be the most fruitful approach. In the end, computers succeeded using brute force.

23 Summary A simple state machine that can write on an infinite tape is the most powerful model of computing currently known. Powerful in the sense of the functions it is able to compute (not how fast or efficiently that it computes them) We know of functions that no known computer is able to compute. In asking the questions of does a computer think, we need to develop a definition of what thought is, and how we do it. If it is merely an algorithm operating on stored memories, or the by product of a physical process then it is possible that machines might someday think, via either simulation or computation.

State Machines. Example FSM: Roboant

State Machines. Example FSM: Roboant page 1 State Machines 1) State Machine Design 2) How can we improve on FSMs? 3) Turing Machines 4) Computability Oh genie, will you now tell me what it means to compute? Doctor, I think you ve built a