Week 3: More Processing, Bits and Bytes. Blinky, Logic Gates, Digital Representations, Huffman Coding. Natural Language and Dialogue Systems Lab

Size: px

Start display at page:

Download "Week 3: More Processing, Bits and Bytes. Blinky, Logic Gates, Digital Representations, Huffman Coding. Natural Language and Dialogue Systems Lab"

Zoe Melinda Horton
6 years ago
Views:

1 Week 3: More Processing, Bits and Bytes. Blinky, Logic Gates, Digital Representations, Huffman Coding Natural Language and Dialogue Systems Lab

2 Robot Homework. Can keep playing with it

3 Announcements Privacy essay and Robot homework were due today. If you have a question on your homework please ask Gabby or Chao if you want a v. timely response Ask the person whose section you signed up for: Gabby T/Th ghalberg@soe.ucsc.edu Chao M/W zhu@soe.ucsc.edu Questions about grading: ask Gabby but CC me. 3 homeworks this week: 2 processing homeworks, one easy (Thurs), one harder (Tues), One principles homework, due next Thursday Practice the kinds of problems that will be on the midterm/ quizzes.

4 More Announcements Invited talk next week: Lifelogging Winter12/01/pages/syllabus Testing on invited talks: No homeworks on them Questions on Midterm/Final that would demonstrate That you were there That you were paying attention That you got something out of it

5 Next Processing Homework: DUE THURS! Write your own code from scratch, not just change existing code. But lots of hints and instructions in the homework PDF

6 How do computers compute? Natural Language and Dialogue Systems Lab

7 Remember Lec 2. McCulloch and Pitts 1943 A Logical Calculus of the Ideas Immanent in Nervous Activity Simple model of human brain, neuron Influenced the design of the first computers Starts with Logic Ideas from logicians: Carnap, Russell and Whitehead.

8 V. technical paper. Can provide if anyone wants

9 The Neuron Soma is the body of neuron Attached to soma are long filaments: dendrites. Dendrites act as connections through which all the inputs to the neuron arrive. Axons serve as output channel

10 McCulloch & Pitts Neuron INPUT X1 W1 INPUT X2 W2 Threshhold unit Does weighted sum of inputs pass threshhold Θ OUTPUT INPUT Xn Wn Input = (W1 * X1) + (W2 * X2). + (Wn * Xn) Output = If Input > Θ Threshhold then 1, otherwise 0

11 Neuron can model most Logical Functions McCulloch and Pitts then proceeded to show (prove mathematically) that such a model of the neuron could calculate most logical functions Today will show you AND, OR, NOT A well known result from logic that we can construct any logical function from these three operations Example from Lecture 2. Lets say P = Today is a sunny day Lets say Q = I am happy When is (P and Q) true?

12 Fundamental units of computers: Logic Gates

13 Truth Table for And (using True and False) P Q P and Q True True True True False False False True False False False False

14 Truth Table for And (using 0 and 1) P Q P and Q

15 McCulloch & Pitts Neuron INPUT X1 W1 INPUT X2 W2 Threshhold unit Does weighted sum of inputs pass threshhold Θ OUTPUT INPUT Xn Wn Input = (W1 * X1) + (W2 * X2). + (Wn * Xn) Output = If Input > Θ Threshhold then 1, otherwise 0

16 MP Neuron for LOGICAL AND X +1 Y +1 =2 IS 1*X + 1*Y 2?? X AND Y

17 How the neuron computes logical AND X Y IS 1* X + 1* Y Output 1 1 YES NO NO NO 0

18 McCulloch Pitts Neuron for OR X +1 Y =1 +1 Is X + Y 1? X OR Y

19 How MP OR Neuron computes logical OR X Y IS X+Y Output 1 1 X+Y= X+Y= X+Y = X+Y = 0 0

20 McCulloch Pitts Neuron for NOT X -1 X OUTPUT 2 BIAS INPUT ALWAYS IS 2 =2 1 IS -X + BIAS 2 NOT X

21 Fundamental units of computers: Logic Gates

22 Can get complex! Cascades of these things.

23 Truth Table for Not And (using 0 and 1) P Q P AND Q NOT (P AND Q)

24 (NOT P) OR (NOT Q) vs. NOT (P AND Q) P Q NOT P NOT Q P AND Q NOT (P AND Q) (NOT P) OR (NOT Q)

25 (NOT P) OR (NOT Q) vs. NOT (P AND Q) P Q NOT P NOT Q P AND Q NOT (P AND Q) (NOT P) OR (NOT Q) NOT (P NOT P NOT Q This is DeMorgan s Law of Boolean Algebra

26 Syntactic Manipulation on Symbolic Expressions Allow new facts to be derived from existing facts We don t have to do it via truth tables but we can! Homework! NOT (NOT P) equals P Commutative: (P AND Q) equals (Q AND P) (P OR Q) equals (NOT P IMPLIES Q) (P IMPLIES Q) equals (NOT Q IMPLIES NOT P) etc

27 Fundamental units of computers: Logic Gates

28 Exclusive-OR == XOR Consider two propositions, either of which may be true or false Exclusive-or is the relationship between them when JUST ONE OF THEM is true. It EXCLUDES the case when both are true,so exclusive-or of the two is False when both are true or both are false, and true in the other two cases.

29 Truth Table for XOR (using 0 and 1) P Q P xor Q

30 MP Neuron for XOR? (EITHER X OR Y BUT NOT BOTH TRUE OR BOTH FALSE) X?? Y?? =? X XOR Y

31 Logical XOR Neuron exists? How long do we keep looking for a solution? We need to be able to calculate the weights, not just keep looking for the answer by trial and error. Each possible pair of inputs corresponds to an equation (a linear inequality) for the output in terms of the inputs, the weights and the threshhold. E.g. for AND it was IS 1*X + 1*Y 2?? These can be used to compute the weights and thresholds.

32 We can read it off the truth Table for XOR P Q P xor Q

33 Truth Table gives the conditions on w1 w2 for XOR w1* 1 + w2* 1 < threshhold (line 1 of TT) w1* > threshhold (line 2 of TT, w2 is negative?) 0 + w2* 1 > threshhold (line 3, oops w2 is positive?, well maybe threshhold is negative? ) < threshhold, (line 4, i.e. threshhold must be positive

34 Linear Separability THERE IS NO SOLUTION => A single layer MP Neuron (perceptron) cannot compute this function (Minsky & Papert 1969) Need to be able to draw a single LINE to distinguish the cases

35 Summary: It All Works Because of Digital Key principle: information is represented as simply the presence or absence of a phenomenon at a given place and time! Phenomenon in computers: Electrical output on a line Hole in punch card, early example. Neuron example: Phenomenon activation on the axon Present action potential greater than threshhold, neuron fires Absent action potential not greater Logic Gates Charge on line No charge

36 Robot Homework. Can keep playing with it

37 Next Processing Homework: DUE THURS! Write your own code from scratch, not just change existing code. But lots of hints and instructions in the homework PDF

Reification of Boolean Logic

526 U1180 neural networks 1 Chapter 1 Reification of Boolean Logic The modern era of neural networks began with the pioneer work of McCulloch and Pitts (1943). McCulloch was a psychiatrist and neuroanatomist;