Design principles for contrast gain control from an information theoretic perspective

Transcription

1 Design principles for contrast gain control from an information theoretic perspective Yuguo Yu Brian Potetz Tai Sing Lee Center for the Neural Computer Science Computer Science Basis of Cognition Department Department Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University Abstract Contrast gain control is an important and common mechanism underlying the visual system s adaptation to the statistics of the visual scenes. In this paper, we first showed that the threshold and saturation determine the preferred contrast sensitivity as well as the maimum information coding capacity of the neuronal model. Then we investigated the design principles underlying adaptation behavior in contrast gain control phenomena by an adaptive linear-nonlinear model. We found that an adaptive rescaling mechanism predicted by information transmission maimization can eplain a variety of observed contrast gain control phenomena in neurophysiological eperiments, including the divisive input-output relations, and the inverse power law relation between response gain and input contrast. Our results suggest that contrast gain control in visual systems might be designed for information maimization. 1 Introduction The visual systems ehibit great fleibility in adapting their input-output functions to the mean [1] and the contrast [,3] of luminance intensity in the visual environment. The amplitude gains of the transfer functions of visual neurons were found to decrease with input variance [4-7]. The relationship between the kernel gain and the input variance has been found to follow an inverse power law relationship [6,7]. In addition, the contrast response functions of visual cortical neurons were found to adapt to the mean contrast by shifting along the log contrast ais to match the range of the prevailing input signals [3,8,9]. These phenomena are called contrast gain control and have been observed in many different types of neurons in the sensory systems of many species, such as neurons in the retina [,4,5,7], striate [3,6,8] and etrastriate visual corte [9] of mammals, and fly H1 neurons [10,11]. Recently, a number of biophysical and neural models have been advanced to account for contrast gain control, including the normalization model [1], the synaptic depression model [13] and a more recent model based on background ecitatory and inhibitory synaptic modulation [14,15]. Various biophysical factors that have been

2 implicated in gain control include threshold [1], synaptic depression [13], synaptic noise [14], dendritic saturation [15], long-term slow adaptation [3,4], and active ionic channels in the spike generation [5,8]. While it is possible that these multiple biological factors and mechanisms can co-eist to affect various aspects of contrast gain adaptation [16], the rules by which the various factors are adjusted to mediate gain control and the principles governing the determination of these factors remain unclear. In this paper, we will investigate the basic biophysical causes and the computational principles underlying contrast gain control by studying a cascade model of an adaptive linear kernel followed by a static nonlinearity. Model and Analysis Fig.1. The adaptive linear-nonlinear (LN) model consists of an adaptive linear filter h(t) followed by nonlinearity g(.). The amplitude of h(t) is scaled by, which acts as an adaptive mechanism. (t) is the response of the linear filter h(t). y(t) is the output. Recent studies [5-7] suggested that the adaptive behaviors in contrast gain control eperiments can be modeled by an adaptive linear kernel cascaded with a static nonlinearity (see Fig.1). Here, we set the adaptive linear function h( t) = β ( σ ) sin( πt/ τ ) ep( t/ τ ) (1) with a = 80 ms and b = 100 ms. We assume here that there eists an adaptive mechanism which can maimize the mutual information between each input signal s(t) and the output y(t) of the neuron by adjusting the adaptive rescaling factor (). The linear response (t) is given by 0 a ( t) = h( τ ) s( t τ ) dτ. The nonlinearity is given by 0, if ( t) < θ, y( t) = g( ( t)) = ( t) θ, if θ ( t) < η, η θ, if ( t) η. where is the response threshold, is the saturation level, and y(t) is the response of the neuron. We use a Gaussian white noise stimulus s(t) with zero mean and SD as the input signal. Its probability density function (PDF) is given by The linear response (t) also has a Gaussian distribution with PDF where is given by σ ( ) σ ( τ ) τ t h d average. This is the adaptive LN model. 3 Gain Analysis 0 b () s 1 e σ πσ p( s) =. 1 e σ πσ p( ) =, =< >=, where <... > denotes time First we fied adaptive factor () = 1 and only studied the role of the nonlinearity in the sensory coding process. In eperimental studies, Wiener Kernel method [17,18] was typically used to recover the linear transfer function h (t) of the investigated

3 system based on the input s(t) and the response y(t). What is the relationship between the real linear function h(t) and the recovered linear kernel h (t) in the presence of the static nonlinearity? According to Bussgang s theorem [19], for any memoryless nonlinear system y = g() with an input signal drawn from a Gaussian distribution, K(f), the Fourier transform of the optimal linear transfer function, specifying the input-output relationship of the static nonlinearity g() is given by Y ( f ) X ( f ) < g( ) > K( f ) = =, (3) X ( f ) X ( f ) σ where Y(f) is the Fourier transform of the output y(t), and * stands for conjugate. For the entire cascade model, the optimal linear transfer function T(f), the Fourier transform of the resultant linear kernel h (t) for the entire cascade, is given by Y ( f ) S( f ) T ( f ) =, (4) S( f ) S( f ) where S(f) is the Fourier transform of input signal s(t). Combining the equations, we have < g( ) > T ( f ) = H ( f ) K( f ) = H ( f ) (5) σ This indicates the entire effect of the static nonlinearity on the recovered Wiener g ( ) Kernel simply introduces a gain scaling factor α = < > to the original linear kernel. Therefore, the recovered linear kernel h (t) is given by h (t) = h(t), where gain factor quantifies how the recovered linear kernel h (t) is affected by the threshold, saturation and the standard deviation of the stimulus. The gain factor can be determined by ( θ ) pd + ( η θ ) pd < g( ) > θ η α( σ ) = =. σ η σ σ h ( τ ) dτ Performing the integrations and simplifying yields 1 η θ α( σ ) = ( erf ( ) erf ( )) = P[ ( t) [ θ, η]] (7) σ σ The basic conclusion of this analysis is that the gain of the measured effective transfer function will change with input variance due to the effect of the static nonlinearity, even though the parameters of the model,, and, are fied. To illustrate this phenomenon, we fi = 1, = 5 and = 40, and plot the gain for signals with different s according to the analytical equation. Fig.a shows the resultant linear kernel h (t) (the inverse Fourier transform of T(f)) is heavily dependent on the value of. Interestingly, is not monotonic, it increases with in the small range, reaches a maimum, and decreases with a further increase in (circles in Fig.b). 0 (6) To confirm these analytical results, we applied the standard Wiener kernel technique [18] to recover the linear wiener kernel for the whole cascade model with = 1 based on the input s( t ) and the output y( t ). The amplitude gain of the recovered kernel (triangles in Fig.b) in this computational study is shown to match well with the

4 theoretical prediction. The recovered kernel h ˆ ( t) only ehibited gain scaling relative to the linear kernel h(t). There is no temporal dilation or contraction of the linear kernel. The computational study therefore confirms the correctness of our theoretical results. The optimal opt in which gain is maimum can be obtained by differentiating Eq.[7], θ η σ =. (8) opt (ln ln ) h ( ) η θ τ dτ 0 The obtained opt is a function of saturation and threshold. This might provide a mechanism and rules for a neuron to adjust its transfer function and gain tuning curve according to the statistical contet of the input signals. However, the range of adjustment of the optimal opt by changing and is rather limited. 4 Information Analysis Fig.. (a) Recovered kernels for = 3 and = 50 for various input stimuli of different. (b) Gain is a function of, ehibiting a tuning curve. This tuning curve is predicted by the theoretical analysis, and is confirmed by the simulation result. (c) I m(,) as a function of for various and. (d) I m(,) as a function of (, ) for various. The distortion or gain tuning effect due to the nonlinearity indeed can affect the information encoding process of the neuron. Now we use Shannon s information theory [0] to quantify the information transmission of the LN model. For a system with input s( t ) and output y( t ), the total output entropy H ( y) = p( y) log p( y) (9) y quantifies the system s theoretical limit on information transfer capacity, while the mutual information in discrete form [1] is given by, I = H ( y) H ( y s) = p( y) log p( y) + p( s) p( y s) log p( y s), m y (10) s, y measures how much of that capacity is utilized to transmit and encode the input signal. H(y s) is noise entropy, accounting for the variability in the response that is not due to variations in the stimulus, but comes from noise sources. For simplicity, we consider the noiseless case, where H(y s) = 0. In this case, the mutual information is set to be equal to the output entropy I m = H(y). The probability distribution of the output response y(t) can be derived from Eq.[1] and []. We can compute the entropy of y(t) directly from this distribution using Eq.[10].

5 We fi = 1, and we compute the mutual information I m as a function of stimulus to eamine the effect of nonlinearity. Fig.c shows that mutual information, in a way similar to effective gain, varies nonlinearly with input, ehibiting a tuning curve, with maimum at an intermediate. This optimal is denoted by opt, corresponding the signals that can introduce maimum information transmission of the system. For a fied, I m increases with an increase in saturation value or with a decrease in the threshold value. This suggests that any nonlinear system with threshold and saturation properties can best encode or transmit signals of a particular range of, and will not encode adequately signals outside this range without adaptation of its parameters. In fact, mutual information I m is roughly proportional to the gain factor (see Fig.d), suggesting that efficient information encoding and gain maimization are tightly correlated. 5 Information maimization in the adaptive LN model Fig.c shows that for the static model (i.e., = 1 is fied for various input), there eists an optimal input distribution with opt that can induce maimal information transmission (recall that opt maimizes gain). To maintain maimal information rate for any given input, we propose an adaptive mechanism that rescales the amplitude of the linear kernel in the LN cascade so that the output of the linear kernel (t) is effectively adjusted to operate at the optimal regime of the given static nonlinearity. Let the rescaling factor be adapt (), then the linear kernel is h ( t) = β ( σ ) sin( π t/ τ ) ep( t/ τ ), (11) A adapt a b where adapt () is determined as the appropriate scaling factor necessary for maimizing information transmission for each input variance. The precise biophysical mechanism for mediating this effect is not known at presence, and presumably can be mediated by a variety of biophysical or network feedback mechanisms. Our primary task here is to elucidate the rules underlying the choice of the scaling factor, and the ramification of such a choice on the contrast gain control phenomena. We propose that the adaptive rescaling mechanism essentially chooses adapt () = opt / for each so that the maimum of the information transmission capacity I ma of the system can be reached. Fig.3a shows that the information transmission for such an adaptive LN model is maintained at the highest level I ma independent of the variance of the signal input. Note that for the static LN model with = 1, I m varies with, with only one global maimum I ma at a particular opt (see Fig.3a). The adaptive model thus ensures that the capacity of the system be fully utilized in different statistical contets of the environment. The maimum of information transmission is constrained only by the threshold and the saturation level (Fig.3b). The lower (or higher) is the threshold (saturation), the higher is the maimum information rate. Therefore, the total gain of the adaptive LN model, i.e., the amplitude of the recovered linear kernel from input s(t) and output y(t) comes from two effects: gain due to the nonlinearity effect (see Eq.[6]), i.e., α, and gain due to the true adjustment adapt. Thus, the total gain factor for the adaptive LN model is γ ( σ ) = β α( σ ) = α ( σ ) σ / σ. (1) adapt opt opt opt Fig.3c demonstrates the inverse power-law relationship between input and the total gain of the adaptive LN model. This inverse power law relationship in the gain-variance curve has been observed in several recent eperimental studies [6-7] (see Fig.3d). It is important to note that without the information maimizing adaptive

6 rescaling, the relationship between the response gain and is a bell-shape curve (as shown in Fig.) rather than an inverse power-law. Our analytical results therefore provide the connection between the empirical inverse power-law observed (Fig.3d) and the principle of information maimization. comparison as the dash line with slope of -1). Fig. 3. I m varies with input in the static (fied ) LN model ( = 0 and = 50) but is kept at maimum rate in adaptive model due to adapting adapt for each. (b) I m is maintained at the maimum level for various and. (c) The amplitude of the recovered kernel follows an inverse power-law relationship with input. (d) Eperimental data by Truchard et al. [6] shows that monocular gain decreases with stimulus contrast (from.5% to 50%) for two recorded cells. They are close to the inverse power-law relationship (shown for 6 Adaptation of the contrast response functions To recover the input-output relationship, eperimenters typically use a stimulus that keeps an input attribute constant for a period of time t, and obtain the output by averaging the response of the neuron during that period [3]. We now proceed to investigate the contrast response function for the adaptive LN model using similar signals. Can adaptive rescaling by adapt in the adaptive LN model eplain the observed adaptive shift in the contrast response curve as a function of the mean contrast? To answer this question, we simulated this eperiment with our cascade LN model, using one-dimensional temporal sinewave gratings of different contrasts (see the black line in Fig.4a) as input to the neuron. Here, a sinewave grating with a temporal frequency of 10 Hz can be considered the carrier signal (see gray line in Fig.4a), amplitude modulated by the input contrast signal c(t). Signal modulated by each contrast value c( t ) is presented for t = 4 seconds. The contrast values are drawn from a Gaussian white distribution with standard deviation. c determines the mean contrast level of the contrast signal in each sequence, which lasts for 1000 seconds. The input-output curves are obtained from sequences of four different mean contrast levels, with c = 1, 5, 10, and 0 respectively, To plot the input-output curve, the model s response for each time bin t is averaged to get a mean output value for each contrast value. The contrast response functions (I/O curves) change their slopes for four different mean contrast levels ( c = 1, 5, 10, and 0 respectively). In a log-contrast plot, this change in slope is manifested as a horizontal shift in the contrast response function (Fig.4b). This behavior is qualitatively similar to the neurophysiological observations [3,8,9]. When the input contrast is divided by the mean contrast c, the contrast response functions become superimposed on top of each other, demonstrating that the

7 adaptation is a divisive effect (see Fig.4c). Thus, the predicted rescaling of the linear kernel based on information maimization can eplain the divisive contrast gain adaptation observed in the neurophysiological eperiments [3,8,9]. A similar rescaling of input-output relations has also been observed in recent eperimental works on H1 neurons of blowfly [10-11], which provided direct evidence that the scaling of the input-output function is set to maimize information transmission for each distribution of signals. Our theoretical results thus demonstrate the underlying connection of these eperimental findings on contrast gain control from an information theoretical prospective. Figure 4. (a) An eample of an input contrast signal with sinewave modulation (temporal sine frequency is 10 Hz). The contrast c(t) (magnitude of the sinewave) changes every 4 seconds. The standard deviation of the contrast levels for this sequence is c = 1. (b) In log-log plot, the contrast response functions i.e., c(t) ~ y(t), recovered from four classes of input contrast signals with c = 1, 3, 5, and 10 respectively. (c) This adaptive shift is a divisive effect, as the contrast response functions collapse together when the input contrast is divided by mean contrast c. 7 Discussion In summary, we first isolated the effect of nonlinearity on contrast gain tuning. We found that the threshold and saturation determines the selectivity and sensitivity of the neuron to the statistics of the input signals. Input signals with optimal variance can maimize the sensitivity of the system and results in the maimal information transmission. Net we studied the relationship between the adaptive linearity and contrast gain control phenomena by employing the principle of information maimization. For any signal with a given variance, the linear kernel amplitude can be adjusted to an optimal level by an adaptive mechanism to maimize the information transmission is maimized. This model is successful in reproducing three important phenomena observed in earlier eperiments related to contrast gain control: 1) the logarithmic decay of the linear kernel gain with the input contrast [6,7], see Fig.3c; ) the divisive adjustment of the contrast response functions in adaptation to different mean contrast levels [3,8,9], see Fig.4b; 3) the rescaling input/output relationship for maimal information transmission [10,11] see Fig.4c. Our theoretical work therefore provides a coherent framework for understanding why the various eperimental observations listed above are in fact evidence in support of the proposal that contrast gain control is a mechanism for information maimization. Further eperimental investigations are needed to clarify the underlying biological factors and mechanisms for the optimizing adaptation of the linear kernel. Acknowledgments This research is supported by NSF CAREER , a NIH P41PR for biomedical supercomputing and NIH MH64445.

8 References [1] Creutzfeldt, O.D. (197). Transfer function of the retina. Electroencephalogr. clin. Neurophysiol. Suppl. 31: [] Shapley, R.M. & Victor, J.D. (1979). The contrast gain control of the cat retina. Vision Res. 19: [3] Ohzawa, I., Sclar, G. & Freeman, R.D. (1985). Contrast gain control in the cat s visual system. J. Neurophysiol. 54: [4] Smirnakis, S.M., Berry, M.J., Warland, D.K., Bialek, W. & Meister, M. (1997). Adaptation of retinal processing to image contrast and spatial scale. Nature 386: [5] Kim, K.J. & Rieke, F. (001). Temporal contrast adaptation in the input and output signals of salamander retinal ganglion cells. J. Neurosci. 1: [6] Truchard, A.M., Ohzawa, I. & Freeman, R.D. (000). Contrast Gain Control in the Visual Corte: Monocular Versus Binocular Mechanisms. J. Neurosci. 0: [7] Chander, D. & Chichilnisky, E.J. (001). Adaptation to temporal contrast in primate and salamander retina. J. Neurosci. 1: [8] Sanchez-Vives, M.V., Nowak, L.G. & McCormick, D.A. (000). Membrane mechanisms underlying contrast adaptation in cat area 17 in vivo. J. Neurosci. 0: [9] Kohn, A. & Movshon, J.A. (003). Neuronal adaptation to visual motion in area MT of the macaque. Neuron 39: [10] Brenner, N., Bialek, W. & de Ruyter van Steveninck, R. (000). Adaptive rescaling maimizes information transmission. Neuron 6: [11] Fairhall, A.L., Lewen, G.D., Bialek, W. & de Ruyter van Steveninck, R. (001). Efficiency and ambiguity in an adaptive neural code. Nature 41: [1] Heeger, D.J. (199). Normalization of cell responses in cat striate corte. Vis. Neuro. 9: [13] Abbott, L.F., Varela, J.A., Sen, K. & Nelson, S.B. (1997). Synaptic Depression and Cortical Gain Control. Science 75:0-3. [14] Chance, F.S., Abbott, L.F. & Reyes, A.D. (00). Gain modulation from background synaptic input. Neuron 35: [15] Prescott, S.A. & De Koninck, Y. (003). Gain control of firing rate by shunting inhibition: Roles of synaptic noise and dendritic saturation. P. Natl. Acad. Sci. USA. 100: [16] Demb, J.B. (00). Multiple mechanisms for contrast adaptation in the retina. Neuron 36: [17] Lee, Y.W. & Schetzen, M. (1965). Measurement of the Wiener kernels of a non-linear system by cross-correlation. Int. J. Control. : [18] Marmarelis, V.Z. (1993). Identification of nonlinear biological systems using Laguerre epansions of kernels. Annals of Biomedical Engineering 1: [19] Bendat, J.S. Nonlinear system analysis and identification from random data. John Wiley and Sons, New York, [0] Shannon, C.E. & Weaver, W. The Mathematical Theory of Communication. Univ. of Illinois Press, Ur-bana, IL, [1] Dayan, P. & Abbott, L.F. Theoretical Neuroscience. MIT Press, Cambridge, 001, Chap.4, pp.19.