Valiant s Neuroidal Model

Transcription

1 Valiant s Neuroidal Model Leslie Valiant CSE Department, UCSD Presented by: Max Hopkins November 13, 2018 Leslie Valiant (Harvard University) November 13, / 15

2 Roadmap 1 Modeling Memory and Association in the Brain Brain Model Join Link Shared Representations Noise 2 Computational Results Equation Solutions Model Refinements Algorithms Leslie Valiant (Harvard University) November 13, / 15

3 Model Weighted Directed Random Graph G(n, p) n vertices Each edge exists with probability p Vertices model neurons Edges model synapses Edge direction models pre v.s. post-synaptic weights model edge strength Leslie Valiant (Harvard University) November 13, / 15

4 Model (Continued) Memory Representation Collection of r neurons Considered to be active if r/2 neurons fire Neuron Firing Neurons have a threshold firing value T Synapses have strength T /k Let I (x) be the indicator that neuron x is firing Neuron j fires if: I (i)w ij > T (i,j) E Modes Neurons and Synapses can be in modes m i, s ij respectively Leslie Valiant (Harvard University) November 13, / 15

5 Join (Equation 1) Memory Formation Given memories A and B, create new memory C C fires if and only if A and B are firing Graph Theoretical Implications Given a Bernoulli distribution B(p), let [ r ] B(r, p, k) = Pr (x1...x r ) B(p) r x i k T (r, p, j) = ( r j B(r, p, k) = r j=k ) p j (1 p) r j T (r, p, j) i=1 Memory Formation (Join) requires: nb(r, p, k) 2 = r (1) Leslie Valiant (Harvard University) November 13, / 15

6 Stability of Join Building new memories from C The size of C is r in expectation Is building chains of memories stable? Stability To understand stability, consider standard deviation Standard deviation is r 1 r as a fraction of the mean Implies stability for large r Leslie Valiant (Harvard University) November 13, / 15

7 Misfiring We consider two cases to prevent C from misfiring: C should not fire if only one of A or B is firing Since C can vary, we demand fewer than r/10 nodes of C fire. B(n, B(r/2, p, k)b(r, p, k), r/10) 0 (2) C should not fire from a different Join A + D B(n, (1 B(r, p, k))b(r, p, k) 2, 2r/3) 1 (3) Leslie Valiant (Harvard University) November 13, / 15

8 Link (Equation 1) Link associates pre-formed memories A and B If A and B are linked, when A fires, B fires as well. Implemented through relay nodes Relay nodes have at least k pre-synaptic edges from A, and k post-synaptic edges to B This implies the relation: B(n, pb(r, p, k), k) 1 (4) Leslie Valiant (Harvard University) November 13, / 15

9 Misfiring (Part 2!) We consider two cases to prevent B from misfiring: B should not be caused to fire by less than half of A B(r, B(n, pb(r/2, p, k), k), r/2) 0 (5) We also assure B cannot fire due to an unlinked memory C Assume B has been linked to t nodes A 1,..., A t: B(r, B(n, p(1 (1 B(r, p, k)) t )B(r, p, k), k), r/2) 0 (6) Leslie Valiant (Harvard University) November 13, / 15

10 Shared Representations Previously, we assumed memories were disjoint Instead, we can model a memory as a collection of r randomly chosen neurons In this model, memories can share neurons The intersection between two memories is r 2 /n in expectation Equations (1),(2),(3), and (6) need to be updated in this model Updates follow the general form: k 1 B(r, p, k) 2 B(r 2 /n, p, k) + T (r 2 /n, p, i)b(r r 2 /n, p, k i) 2 i=0 Leslie Valiant (Harvard University) November 13, / 15

11 Noise and Interference Model: A constant fraction σ of nodes misfire σ is called the noise rate Equation updates (2),(3),(5),(6),(2 ),(3 ),(6 ) updated r r + σn No guarantees outside of robustness and misfiring Seizures σ >.0001 Guarantees are probabilistic, and computed approximately In simpler models, we compute capacity This model bounds interference instead Leslie Valiant (Harvard University) November 13, / 15

12 Equation Solutions Table values reflect measurements in human brain Table entries give r d is graph degree p(n 1) ˆ and relate to noise Leslie Valiant (Harvard University) November 13, / 15

13 Model Refinements Neurons Can be in one of a number of modes m i Mode specifies threshold T i and state q i Synapses Can be in one of a number of modes s ij Mode specifies weight w ij, and state q ij Leslie Valiant (Harvard University) November 13, / 15

14 Algorithms (Disjoint Representations) Join Set initial states to q 1, Threshold to T, and weights to T /k Fire A and then B C := {x V : x fired twice} Set the states of C to be q 2 Set edge weights w AC = T /x, w BC T /y where x is the number of edges from A to C, and y is analogous. Link Set relay nodes threshold to T, and post-synaptic weights 0 Fire A and B, then set relay post-synaptic weights to T /k Leslie Valiant (Harvard University) November 13, / 15

15 Thank You Leslie Valiant (Harvard University) November 13, / 15