Leo Kadanoff and 2d XY Models with Symmetry-Breaking Fields. renormalization group study of higher order gradients, cosines and vortices

Transcription

1 Leo Kadanoff and d XY Models with Symmetry-Breaking Fields renormalization group study of higher order gradients, cosines and vortices

2 Leo Kadanoff and Random Matrix Theory

3 Non-Hermitian Localization in d Neural Networks Non-Hermitian matrices, with complex eigenvalue spectra, arise naturally in simple models of sparse neural networks. Striking departures from the conventional wisdom about localization arise in the onedimensional non-hermitian random matrices Ariel Amir Harvard/SEAS An intricate eigenvalue spectrum controls the spontaneous activity and induced response. Directed rings of neurons lead to a hole centered on in the density of states in the complex plane. All states are extended on the rim of this hole, while the states outside the hole are localized. Naomichi Hatano University of Tokyo

4 Random Matrices in Neuroscience Spike rate r(t) depends on orientation of bar moving across the visual field Visual stimulus s(t) transferred from retinal neurons LGN V region of the visual coretex spike rate r(t) Pathway from the retina through the lateral geniculate nucleus (LGN) to the primary visual cortex signal s(t): orientation in degrees Dayan and Abbott: Theoretical Neuroscience

5 Random matrix models of the brain (H. Sompolinsky, L. Abbott et al.) Random neural connections can be formed during development, with many stochastic attachments of axons and dendrites to other neurons. Over time, pruning and strengthening/weakening of connections allow neural circuits to "learn" various functions. The spectra and eigenfunctions of completely random neural networks with a mixture of inhibitory and excitatory connections, can describe neural activity during the early stages of development. Girko s Law Re λ=0 K. Raan, 009 Spontaneous and Stimulus driven Network Dynamics. Doctoral Dissertation, Columbia University.

6 Random matrix model of a sparse neural network Sensory Inputs Processing N M (,) M (, ) Sensory inputs, possibly after a processing step, are sent via feed forward couplings into a circular ring of N neurons Note that M(,) and M(,) can not only be unequal, but also of opposite sign, if one direction is excitatory and the other inhibitory. u i th firing rate deviation from background of the i neuron in recurrent network th input firing rate of the neuron in the input (feed forward) network dv dt i dv dt i v tanh N M v i i N v M v h i i i N h, h W u i i i (linear approximation)

7 Non-Hermitian neural networks with random excitatory (M(i,) > 0) and inhibitory (M(i,)< 0) connections N g g M se + s e g provides a systematic clockwise (g > 0) or counterclockwise (g < 0) directional bias Study eigenvalues and eigenvectors of directed, banded non-hermitian random matrices s, s, indep. random variables; Set g = 0 for now random sign model of J. Feinberg and A. Zee, PRE (999) M 0 se 0... se se g g N g g 0 se 0 0 g se 0 g : 0 sn e g g N... 0 N 0 s e s e

8 Eigenvalue distribution in the complex plane λ = λ + iλ What would Leo say... Fractals where s the Physics?? (Physics Today ) Result of exact diagonalization of 0,000 N x N matrices with N = 5000 and g = 0 What are the localization properties of this spectrum??

9 What does localization mean? Eigenfunctions within circle on right side are highly localized w/real eigenvalues N = 000 g = 0.0 Eigenfunctions in an annulus closer to the origin are more extended localization length diverges near the origin (, ) const. ( )

10 What is the effect of the bias parameter g? M 0 se 0... se se g g N g g 0 se 0 0 g se 0 g : 0 sn e g g N... 0 N 0 s e s e 5 g> 0 excitatory connection inhibitory connection 4 3 s, s with equal probability 0 g (no Dale's law for now) As g increases from 0, it tunes down the amount of back propagation in a feed clockwise recurrent network δ ξ δ ξ 0 g δ 3 J. Hertz et al. g Similar layered neural nets used for image classification, etc. in machine learning algorithms. Many layers deep learning

11 Effect of a directional bias around the chain (g > 0) N = 5000, g = 0.0

12 Effect of a directional bias around the chain (g > 0) N = 5000, g = 0.00

13 Effect of a directional bias around the chain (g > 0) N = 5000, g =

14 Effect of a directional bias around the chain (g > 0) N = 5000, g = 0.05 A gap or hole appears in the eigenvalue spectrum in the complex plane

15 Effect of a directional bias around the chain (g > 0) N = 5000, g = 0.

16 Effect of a directional bias around the chain (g > 0) N = 5000, g = 0.

17 Effect of a directional bias around the chain (g > 0) N = 5000, g =

18 Localization lengths and effect of boundary conditions Define inverse participation ratio IPR IPR 4 / ~ inverse localization length extended state: ~ / N, IPR 4 / ~/ N localized state, ~ exp[- x x / ] IPR 0 loc 4 / O() Eigenvalue spectrum for g = 0 (or, for any g with open boundary conditions!) Eigenvalue spectrum for g = 0. with periodic boundary conditions Localization length diverges on the rim of the hole when g > 0 extended states

19 Large g limit: Plane wave states, all eigenfunctions delocalized Traectories of eigenvalues for N=00 and values of g decreasing from down to zero. Eigenvalues "flow" in the complex plane. Motion stops once eigenvalues localize

20 Energy gap and rings of extended states also appear for coupled neural clusters g = 0. g= triangular neural clusters, obeying Dale s law, and coupled together to form a ring extended states Band theory for neural networks? Layered neural network with tunable back propagation

21 Thank you! Photo: Tom Witten