IST 4 Information and Logic

Transcription

1 IST 4 Information and Logic

2 mon tue wed thr fri sun T = today 3 M oh x= hw#x out 0 oh M 7 oh oh 2 M2 oh oh x= hw#x due 24 oh oh 2 oh = office hours oh oh M2 8 3 oh midterms oh oh Mx= MQx out 5 oh 3 4 oh Mx= MQx due 22 oh oh 4 T 5 CP CP = challenge problem 29 CP2 5 oh 5 oh oh CP oh oh

3 Last Lecture: languages for biology - Stochastic chemical networks Stochastic logic design The B- algorithm Duality - Molecular switches DNA strand displacement - Stochastic flow networks Feedback helps!

4 A relay circuit is a physical system for syntax manipulation Relay circuits are not the only option! AND gate OR gate Linear Threshold (LT) gate a t (deep) learning: adjusting the weights b

5 Circuits with Gates functional gates syntax boxes AON: AND, OR, Not LT: Linear Threshold

6 Questions about building blocks? Feasibility Given a set of building blocks: What can/cannot be constructed? Efficiency and complexity If feasible, how many blocks are needed? Algorizm?

7 AND, OR and NOT (AON) What is the function computed by this circuit? a a b 3 total number of gates in the circuit b a b 2 longest path from input to output counting the number of gates

8 Every 0- Boolean Function Can be Implemented Using A Depth Two AON Circuit Implement the DNF representation: OR of many ANDs

9 XOR of 3 Variables abc XOR(a,b,c)

10 XOR of 3 Variables Depth = 2 Size = 5 a b c a b c a b c a b c > > > > > is the complement

11 XOR of More Variables? How many gates in a depth-2 circuit for XOR of n variables with AON? Surprisingly, this is the optimal size for depth-2

12 Depth-2 AON Circuit for XOR Theorem: An optimal size depth-2 AON circuit for has gates??? Proof: The construction follows from the DNF representation: a normal terms + one OR gate b c The lower bound: WLOG (i) Every AND gate must have all n inputs (ii) Every AND gate computes a normal term DNF is a representation, hence, Without Loss Of there are AND gates Generality?? a b c a b c a b c > > > > >

13 Proof (cont): Depth-2 AON Circuit for XOR Need to prove: (i) Every AND gate must have all n inputs By contradiction: Assume that there is a gate G with n- Making G=? inputs. Say x is missing from G Assume that: 0 0 a b c Theorem: An optimal size depth-2 AON circuit for has gates > Hence, the output of the circuit is OR gate has input of a b c a b c a b c a b c > > > > set a variable to and a complement to 0 >

14 Depth-2 AON Circuit for XOR Theorem: An optimal size depth-2 AON circuit for has gates Proof (cont): Assume that: Hence, the output of the circuit is (OR gate has input of ) Note that the following two assignments force the output of the circuit to be : So what? Q Those assignments have different parities Contradiction!!

15 How many gates in a depth 2 circuit for XOR of n variables with AON? It is optimal size for depth-2 n=4, depth 2, size 9 Q: for n=4, arbitrary depth, suggest a circuit for XOR with size less than 9?

16 Size 8 AON Circuit for XOR of Four Variables size 5 size 3 a b c XOR(x,y,z) XOR(x,y) XOR(a,b,c,d) d Arbitrary depth circuit for XOR of n variables with AON? Idea: Compute a large XOR by using a circuit of small XOR gates

17 AON Circuit for XOR Idea: Compute a large XOR by using a circuit of small XOR gates 8 variables in-degree = 2 Tree leaf = input edge edge = wire node = XOR gate XOR

18 Q: Can we do better for 8 variables? Idea: Compute a large XOR by using a circuit of small XOR gates 8 variables Circuit size in AON gates? Size = Node size X number of nodes 3 X 7 = 2 Note that we need size 29 in depth-2 XOR

19 Q: Can we do better for 8 variables? Idea: Use a larger in-degree? 9 variables Size 8 for 8 variables Size = Node size X number of nodes 5 X 4 = 20 Note that we need size 2 with in-degree 2 XOR

20 In general, we can prove that degree-3 XOR trees are the best! Size is Idea: Use a larger in-degree? 9 variables Size 8 for 8 variables Size = Node size X number of nodes 5 X 4 = 20 Note that we need size 2 with in-degree 2 XOR

21 AON Constructions for XOR n=4 circuit kind size 9 8 AON, d-2 AON optimal lower bound: not optimal

22 AON Circuit for XOR We have a construction of size we know how to prove a lower bound of 2n

23 AON Circuit for XOR We have a construction of size we know how to prove a lower bound of 2n- Matt Cook proved (2005) that an AON circuit of size 7 for XOR does not exist he used a computer search next gap

24 A circuit for n=6 with 2 AND gates (7n 4)/3 A recent result by Kombarov, 205 An upper bound! next gap

25 Ingo Wegener, 99 (will be posted on the class web site) The complexity of the parity function in unbounded fan-in, unbounded depth circuits. Prove a matching upper/lower bounds and get an MSc in CS New upper bound: (7n-4)/3

26 The problem: Most functions require a large circuit size - in the number of inputs

27 4x(3-)=8

28 The circuit complexity problem: While most functions require a large circuit size - in the number of inputs Currently we can only prove lower bounds... Show a function that requires Size: total number of gates in the circuit circuit size!

29 Circuits with Gates LT: Linear Threshold

30 Neuron Neural Gate LT: Linear Threshold

31 LT: Linear Threshold What is the function computed by this gate?

32 Neural Circuits feasibility

33 LT: Linear Threshold Q: Are LT gates magical? 2 input Linear Threshold (LT) gate

34 LT: Linear Threshold Q: Are LT gates magical? Idea: A Linear Threshold is Magical Can compute AND, OR and NOT

35 We showed that we can compute the AND function with an LT gate

36 Can We Compute an OR Function with an LT Gate?

37 Can We Compute a NOT with an LT Gate? -2 Can we compute NOT without sgn?

38 More Variables for AND? Hence is an AND

39 More Variables for OR? Hence is an OR

40 Circuits Efficiency and complexity

41 The Functions of the Adder d d2 c 2 symbol adder c s sum carry

42 c d d2 2 symbol adder c XOR with a Single LT Gate s Is it possible to compute with a single LT gate? Idea: Find weights w0, w and w2 such that:

43 c d d2 2 symbol adder c XOR with a Single LT Gate s Is it possible to compute with a single LT gate? Answer : NO Proof: By contradiction assume it is possible and reach a contradiction Q

44 d d2 c 2 symbol adder c XOR with More Variables? s Is it possible to compute with a single LT gate? Need LT circuits for XOR! Idea: suppose that it is possible, and reach a contradiction However, And, Contradiction

45 c d d2 2 symbol adder c MAJ with a Single LT Gate s Is it possible to compute with a single LT gate? X MAJ

46 AND, OR, XOR and MAJ are symmetric functions Q: Which symmetric functions are in LT? X AND OR XOR MAJ LT LT LT not LT LT = the class of Boolean functions that can be realized by a single LT gate.

47 Definition: A symmetric Boolean function is in TH if it has at most a single transition in the symmetric function table = a transition X AND OR XOR MAJ In TH Not in TH

48 The Class TH

49 The Class TH - Single Transition = a transition Q: what is TH? the number TH functions... A: 2n+2 X TH0 TH TH2 TH3 TH0 TH TH2 TH

50 Claim: Proof: 0 Q

51 The Class TH is in LT X TH0 TH TH2 TH3 TH0 TH TH2 TH

52 Need LT circuits for XOR! AON and Linear Threshold Circuits XOR example

53 XOR of Three Variables Size 5 is optimal for AON depth 2 Depth = 2 Size = 5 a b c a b c a b c a b c > > > > > is the complement

54 LT gates are MORE Powerful FOR XOR: Size 5 is optimal for AON depth 2 - Size 4 LT depth

55 LT-l = LT layered inputs go to first layer only LT gates are MORE Powerful - A TH functions A B C A+B+C -2+A+B+C B C Can take the sgn or add

56 XOR Function: Size of LT vs AON in Depth 2 AON LT-l 5 4 * * * = it is optimal Exponential gap in size

57 AON LT-l 5 4 General construction for symmetric functions

58 Linear Threshold Circuits symmetric functions

59 LT Depth-2 Circuits TH - +??? TH2 X TH TH2 TH+TH

60 Generalization X f(x)

61 Generalization X f(x)

62 Generalization X f(x) TH

63 Generalization X f(x) TH TH

64 Generalization X f(x) TH TH3 Σ

65 X f(x) TH TH3 Σ

66 Generalization to SYM - + Q: What is the generalization to arbitrary symmetric functions?

67 Generalization to SYM Q: What is the generalization to arbitrary symmetric functions? A: Consider the symmetric function table, it is a sum of non-overlapping -intervals 0 0 Sum of two TH functions

68 Back to XOR n TH gates for XOR of n variables

69 LT-l Circuit Design Algorithm for SYM f(x) Subtract for every isolated -block

70 The Layered Construction for SYM Some History Saburo Muroga Was born in Japan Majority Decision PhD in 958 from Tokyo U, Japan : Researcher at IBM Research, NY : professor at the University of Illinois, Urbana-Champaign

71

72

73

74

75

76 (4,2,2,3) (6,2,0,2) LT = Can be computed by a single LT gate In LT show a construction Not in LT show a proof