IST 4 Information and Logic

Transcription

1 IST 4 Information and Logic

2 T = today x= hw#x out x= hw#x due mon tue wed thr fri 30 M 6 oh M oh 3 oh oh 2M2M 20 oh oh 2 27 oh M2 oh midterms Students MQ oh = office hours Mx= MQx out 4 3 oh 3 4 oh oh oh presentations Mx= MQx due 8 oh oh T oh 5 oh oh oh

3 MQs. Everyone has a gift! (Tuesday) 2. Memory (Thursday)

4 Tuesday, 6/2, 2:30pm. Christopher Haack: The gift of resilience 2. Joon Lee: Settling is not an option 3. Spencer Strumwasser: The gift of dyslexia 4. Richard Zhu: The gift of memory 5. Ah Ashwin Hari: The gift of musical composition 6. Jessica Nassimi: Evolution a gift in disguise 7. Serena Delgadillo: The gift of self-expression 8. Megan Keehan: Gift of motherliness 9. Zane Murphy: Grandmother and the piano

5 Thursday, 6/4, 2:30pm. Connor Lee: Memory is a fickle thing blessing or curse 2. Pallavi Aggarwal: The wonders of human memory 3. Peter Kundzicz and Anshul Ramachandran: Muscle memories 4. Siva Gangavarapu: A cultural retrospection 5. Philip Liu: The light of other days 6. Jason Simon: Math and Broadway 7. Yujie Xu: Memory v.s. ESL 8. Celia Zhang: When memory sours

6 Last Lecture Gates and circuits AON: AND, OR, Not LT: Linear Threshold LT: Linear Threshold > > a b c a b > > a b c a >b > > > b c a b c a b c > > > > b c a b c a b c

7 Last Lecture AON: AND, OR, Not LT: Linear Threshold General construction for symmetric functions AON 5 LT-l 4 LT-nl 2 * Exponential gap in size What are the symmetric functions that can be computed by a single LT gate? * = it is optimal * *

8 Linear Threshold and SYM

9 LT: Linear Threshold

10 Symmetric Functions and LT Circuits Q: Which class has more functions? Q: How is SYM related to LT?? Definitions: () SYM = the class of Boolean symmetric functions (2) LT = the class of Boolean functions that can be (2) LT the class of Boolean functions that can be realized by a single LT gate.

11 AND, OR, XOR and MAJ are symmetric functions Q: Which h symmetric functions are in LT?? X AND OR XOR MAJ LT not LT LT LT LT = the class of Boolean functions that can be realized by a single LT gate.

12 Definition: A symmetric ti Boolean function is in TH if it has at most a single transition in the symmetric function table = a transition X AND OR XOR MAJ In TH Not in TH

13 The Class TH is in LT X TH0 TH TH2 TH3 TH0 TH TH2 TH

14 Q: How is TH related to SYM and LT?? We know that: We Proved that: SYM TH LT

15 TH is exactly in the intersection of SYM and LT Theorem: Proof: Not today... you might want to try and prove it... Q: What are the 4 functions? SYM TH LT???

16 LT Function that is not Symmetric

17 Linear Threshold l Circuits for symmetric functions

18 AON 5 LT-l 4 LT-nl 2 General construction for symmetric functions

19 X XOR Q: compute XOR with TH gates? X TH TH2 TH+TH

20 LT Depth-2 Circuits TH - + TH2 X TH TH2 TH+TH

21 Generalization X f(x)

22 Generalization X f(x)

23 Generalization X f(x) TH

24 Generalization X f(x) TH TH

25 Generalization X f(x) TH TH3 Σ

26 X f(x) TH TH3 Σ

27 Generalization to EQ 0 0 n

28 Generalization to EQ

29 Generalization to SYM - + Q: What is the generalization to arbitrary symmetric functions?

30 Generalization to SYM Q: What is the generalization to arbitrary symmetric functions? A: Consider the symmetric function table, it is a sum of non-overlapping -intervals 0 0 Sum of two TH functions

31 Back to XOR n TH gates for XOR of n variables

32 LT-l Circuit Design Algorithm for SYM f(x) Subtract for every isolated -block

33 The Layered Construction for SYM -Some History Saburo Muroga Was born in Japan Majority Decision PhD in 958 from Tokyo U, Japan : Researcher at IBM Research, NY : professor at the University of Illinois, Urbana-Champaign

34 Saburo Muroga HW#5 problem 2a

35 neural circuits and logic some more history...

36 Being Homeless and Interdisciplinary Research Warren McCulloch Walter Pitts Neurophysiologist, MD Logician, Autodidact Warren McCulloch arrived in early 942 to the University of Chicago, invited Pitts, who was homeless, to live with his family In the evenings McCulloch and Pitts collaborated. Pitts was familiar with the work of Leibniz on computing. They considered the question of whether the nervous system is a kind of universal computing device as described by Leibniz This led to their 943 seminal neural networks paper: A Logical Calculus of Ideas Immanent in Nervous Activity

37 Impact Warren McCulloch Walter Pitts Neurophysiologist, MD Logician, Autodidact This led to their 943 seminal neural networks paper: p A Logical Calculus of Ideas Immanent in Nervous Activity Neural networks and Logic Time Memory Threshold Logic and Learning State Machines

38 neural circuits and memory m computing with dynamics

39 Linear Threshold Some Adjustments Linear Threshold (LT) gate -t threshold -t -

40 AND Function with {0,}

41 AND Function with {-,} The AND function of two variables with {-, }:???

42 AND Function with {-,} The AND function of two variables with {-, }:???

43 Linear Threshold with Memory Elephants are symbols of wisdom in Asian cultures and are famed for their exceptional memory A memory nose Remembers the last f(x)

44 Feedback Networks Example weights thresholds Th t t f th t k th t th t d The state of the network: the vector that corresponds to the states (noses ) of the gates

45 Feedback Networks Example Label the gates

46 Feedback Networks Example

47 Feedback Networks Example

48 Feedback Networks Example is a stable state

49 Feedback Networks Example

50 Feedback Networks Example

51 Feedback Networks Example

52 Feedback Networks Example is a stable state

53 Feedback Networks Example state The node that computes State transition diagram (state space) Q: Is -- a stable state?

54 Feedback Networks Example Answer: No Q: Is -- a stable state?

55 Feedback Networks Example

56 Feedback Networks Example

57 Feedback Networks Example

58 Feedback Networks Example stable states

59 neural circuits and memory m associative memory

60 Feedback Networks Computing with Dynamics stable states Input: initial state Feedback Network Output: stable state -

61 Feedback Networks Computing with Dynamics stable states Input: initial state Feedback Network Output: stable state -

62 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

63 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

64 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

65 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

66 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

67 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

68 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

69 Input: initial state Feedback Networks Computing with Dynamics Associative Memory The Leibniz-Boole Machine Output: stable state Feedback Network

70 Who is this person?????

71 John Hopfield Feedback Networks Hopfield Model (Caltech 982)

72 John Hopfield Feedback Networks Hopfield Model (Caltech 982) i = node i = threshold ti = state vi - = weight of edge (i,j)

73 The matrix description

74 Feedback Networks The Vector/Matrix Description An n node feedback network can be specified by: W an nxn matrix of weights T an n vector of thresholds V an n vector of states

75 The Matrix Description Example An n node feedback network can be specified by: W an nxn matrix of weights T an n vector of thresholds V an n vector of states

76 The Matrix Description Computation Computation ti in N= (W,T) by column 5

77 Order of computation serial and parallel

78 Modes of Operation Q: when do the nodes compute? Serial mode: one node at a time (arbitrary order) - - 2

79 Modes of Operation Q: when do the nodes compute? Serial mode: one node at a time (arbitrary order) - - 2

80 Modes of Operation Q: when do the nodes compute? Serial mode: one node at a time (arbitrary order) Fully-Parallel mode: all nodes at the same time - - 2

81 Three examples

82 Example Serial Mode Symmetric Weight Matrix The state space: stable states

83 Example 2 Fully-Parallel (FP) Mode Symmetric Weight Matrix Q: how does the state space look? start with It s a cycle!

84 Example 2 Fully-Parallel (FP) Mode Symmetric Weight Matrix The state space: stable states cycle of length 2

85 Example 23 Fully-Parallel Mode Antisymmetric Symmetric Weight Matrix W T = WW Q: how does the state space look?

86 Example 3 Fully-Parallel Mode Antisymmetric Weight Matrix - Q: how does the state space look? 2 cycle of length 4

87 Example 3 Fully-Parallel Mode Antisymmetric Weight Matrix - 2 The state space: - cycle of length

88 The Three Cases Cycle lengths mode W symmetric antisymmetric Example # serial? fully-parallel,

89 The Three Cases Cycle lengths Example # mode W symmetric antisymmetric serial? fully-parallel, Hopfield Goles Goles 986

90 Proof Ideas Cycle lengths W symmetric antisymmetric mode Example # serial? fully-parallel, The proofs of these three results use the concept of an energy function For the serial mode: Show that: t Namely, stable states are local max of the energy E

91 Questions on Convergence Posted on the class web site Cycle lengths mode W symmetric antisymmetric Example # serial? Hopfield 982 fully-parallel, Goles 985 Q: Are the three cases distinct? 3 Goles 986 Q2: Elementary proof? (wo/energy)

92