Do Neurons Process Information Efficiently?

Transcription

1 Do Neurons Process Information Efficiently? James V Stone, University of Sheffield Claude Shannon,

2 Nothing in biology makes sense except in the light of evolution. Theodosius Dobzhansky, Question What is the main driver in the evolution of brains: 1 Energy? 2 Information? 3 Information per Joule? 4 Something else? 2

3 How expensive are neurons? Structure What is information? How does the cost of information vary with information rate? Do neurons maximise information/s or information/joule? (eg firing rate, axon diameter, synaptic efficacy) Colour aftereffects how and why. Do ganglion cells transmit information efficiently? Conclusion 3

4 Entropy and Information are two sides of the same coin In terms of Shannon s information theory (1948), entropy = uncertainty. The uncertainty (entropy) you have about which side a coin will land on is 1 bit. If I tell you which side it landed on then: I have reduced the entropy you have about the coin by 1 bit, and (equivalently) I have given you 1 bit of information. 4

5 Energetic Brains Child Body Brain 50% 50% In a child, the brain uses 50% of the body s energy at rest (the adult brain uses 20%). Lennie (2003). Maintenance Action potentials Energy budget 5

6 Energy Efficiency (with or without neurons) How does information transmission vary with signal power in general? Shannon s model communication channel. Axon = output line The neuron as a communication channel. 6

7 Diminishing returns on signal power Given a signal with power S picojoules per second (pj/s) in a communication channel (eg neuron) which has noise with power N pj/s, the maximum rate at which information can be transmitted is MI = log S + N N bits/s MI = mutual information. Doubling power does not double information rate. Power (pw or pj/s) 7

8 Energy efficiency with no baseline power cost bits/s energy efficiency = bits/pj pj/s (bits/s = mutual information) Efficiency (bits/pj) Efficiency Information Information rate Energy efficiency Information (bits/s) Energy efficiency peaks at a power cost of 0 J/s! Power, E (pj/s) 8

9 Efficiency with a baseline power cost = 1pJ/s bits/s energy efficiency = bits/pj baseline + pj/s Efficiency (bits/pj) Energy efficiency Power, E (pj/s) Baseline cost Information rate Efficiency Information Information (bits/s) With a nonzero baseline cost, energy efficiency peaks above 0 pj/s (here, at power cost of 3 pj/s) 9

10 Optimal Firing Rate When we see, we are not interpreting the pattern of light intensity that falls on our retina; we are interpreting the pattern of spikes that the million cells of our optic nerve send to the brain. Rieke, Warland, de Ruyter van Steveninck, and Bialek,

11 The visual system Ganglion cell axons (1 million) 11

12 Coding Efficiency (Barlow) Optimal Firing Rate Information transmitted / total information capacity Energy Efficiency Information transmitted / energy (= bits/joule). Question: Does the mean firing rate maximise 1) Coding Efficiency or 2) Energy Efficiency? But before we can answer this, we need to know about how information varies with energy in general... 12

13 Measuring firing rates We can estimate the firing rate of a neuron by dividing time into small intervals dt... the firing rate is then y = N/T spikes/s 13

14 Diminishing returns in Neurons For a neuron with a firing rate of y spikes/s, the MI is approximately MI = ky log 1 ydt bits/s k=constant=0.7 Axon = output line Neuron Based on Harris

15 Optimal firing rate to maximise coding efficiency Maximum MI at 50 spikes/s (ie half max firing rate) Mutual information (bits/s) 15

16 Optimal firing rate to maximise energy efficiency Maximum energy efficiency at 2 spikes/s Ganglion cells (for example) have mean firing rates that maximise energy efficiency (Levy and Baxter, 1996). 16

17 Optimal Axon Diameter 17

18 Axon Diameter MI increases approximately as a logarithmic function of diameter (Perge et al 2009). So it is always better to use many thin axons than one thick axon if the objective is to maximise information per second (ie MI). Effect of sharing fixed axon cross-section area between 1, 2 and 3 axons. So, thinner is better. But how thin is best? 18

19 Optimal Axon Diameter Thus coding efficiency would be maximised by many axons with small diameters. The smallest (MY) diameter ganglion cell axon is dmin=0.46 micrometres. If the baseline cost=0 then d=dmin would maximise energy efficiency and coding efficiency. But baseline>0, and the mean diameter is not d=dmin. The mean diameter is d=0.7micrometres, which maximises energy efficiency, NOT coding efficiency (Perge et al 2009). Maximum energy efficiency is at d=0.7=the mean diameter observed in ganglion axons. Maximum coding efficiency is at d=0.46. Energy efficiency (bits/pj) Myelinated axons Unmyelinated axons 19

20 Optimal synaptic efficacy at the LGN 20

21 No Yellow Receptors Here Not just how but why... 21

22

23

24 24

25 25

26 Cone cells responsive to blue dots, green and red light in the retina Very few blue cones, and... NO YELLOW CONES.

27 Retinal Output The data from 126 million photoreceptors (rods and cones) leaves the eye via 1 million axons of retinal ganglion cells. 27

28 Constructing opponent pairs Blue-yellow channel Red+green channel Red-green channel Sum Ganglion cells Difference Difference Blue cone Green cone Red cone This tells us how to make a yellow channel, but not why it is made.

29 Colour Aftereffects How and Why The following refers to red-green aftereffect, but the same logic applies to blue-yellow. 29

30 HOW: Colour aftereffect: A neuronal model Stimulus Perception a) Pre-adaptation White Red channel Green channel Compare White b) Adaptation Red Red channel Green channel Compare Red c) Post-adaptation White Red channel Compare Green Green channel This tells us how colour aftereffects could occur, but not why they should occur.

31 Not HOW but WHY aftereffect? Red and green cone have similar tuning curves, so the responses of red and green cones are correlated. Without recoding in the retina, these two correlated signals would be sent along two ganglion cell axons. Cone outputs Cone tuning curves Green cone output Output r G r r Red cone output Wavelength (nm) 31

32 Why aftereffect? Recoding red and green cone outputs as two new signals that are the sum g R+G =(r R + r G ) (yellow, luminance) and difference g R-G =(r R - r G ) ensures that these new signals are uncorrelated. GC difference output Ganglion cell outputs g R+G UNcorrelated with g R-G. So there is little wasted capacity in sending these two recoded signals down two ganglion cell axons. GC sum output 32

33 Ganglion cells Red+Green Red-Green b Ganglion cell outputs are uncorrelated. Cones Green Red a Cone outputs are correlated.

34 Optimal ganglion cell (GC) receptive field (RF) size If RF is small => mutual information lost If RF large => correlations between GC outputs increase. In the limit, all GCs convey the same information, but the total information is small. Optimal RF size = 2sigma, as observed in the retina. Maximum MI at 2*sigma Mutual information (bits) Borghuis et al (2008) Receptive field spacing 34

35 GC receptive fields adapt to maximise information Data points are from human subjects in different lighting conditions Curves are information theoretic predictions. This model has no free parameters. Atick

36 Correlation between ganglion cell outputs in salamander 0.5 Correlation between image luminances Correlation Median correlation between GC outputs Distance on retina (micrometres) Pitkow et al,

37 Conclusion Information theory explains not just how, but why. It does not specify how evolution builds brains, but it does place severe limits on the cost per bit of what can be built. To return to the question posed at start... A plausible candidate for the major driver in the evolution of brains is: not minimising energy expended (Joules), not maximising information received (bits), but maximising bits received per Joule expended. To understand life, one has to understand not just the flow of energy, but also the flow of information. William Bialek,

38 38