LINEAR SYSTEMS
Linear Systems 2 Neural coding and cognitive neuroscience in general concerns input-output relationships. Inputs Light intensity Pre-synaptic action potentials Number of items in display Outputs Number of isomerizations post-synaptic conductance Search time
Linear Systems 3 To a first approximation, we can often model these relationships as linear systems. This makes it much easier to Identify these systems (next topic) Make predictions e.g., given a new input x, the output will be y given an observed output y, the input must have been x
Don t get carried away 4 Clearly the brain cannot be one big linear system: We do not simply resonate with our environment. We have to make decisions: this involves the nonlinear mapping of observations to categorical variables. However, many subsystems can be approximated as linear.
What is a linear system? 5 Consider a system T that maps inputs x to outputs y. The system is linear if it satisfies the principle of superposition: Additivity T ( α x + βx ) 2 = αt ( x ) + βt ( x ) 2 Homogeneity
Shift Invariance 6 We will typically be concerned with inputs x defined over time t and/or space u. A system T is shift-invariant with respect to one of these variables if a shift in the input along the variable produces an identical shift in the output: y(t) = T x(t) y(t s) = T x(t s)
Continuous and Discrete Signals 7 Most signals are discrete at a fine scale, and our measurements are always discrete. However, we can often approximate these as continuous, and this is sometimes convenient mathematically.
Pulses and Steps 8 Δ Discrete pulse δ Δ (t) = Δ Δ if t < Δ otherwise lim δ Δ Δ (t) Dirac delta function δ(t) = if t = otherwise Paul Dirac t u(t) = t δ(s) ds Unit step u(t) = if t < if t
Representing a signal with impulses 9 = + +... + +... x(t) k = x ( kδ)δ ( Δ t kδ) Δ x(t) = lim Δ x ( kδ)δ ( Δ t kδ) Δ = x(s)δ(t s)ds k =
Convolution and the Impulse Response Function The superposition principle is very powerful. It means that we can characterize a linear system T completely by its response to a unit impulse. In particular, we can use the impulse response function to predict the response of the system to any input. Impulse T Impulse Response
Convolution and the Impulse Response Function y(t) = T[x(t)] = T x(s)δ(t s)ds = T lim Δ k = x( kδ)δ Δ t kδ ( ) Δ = lim Δ k = T x( kδ)δ Δ t kδ ( ) Δ (Additivity) = T x(s)δ(t s) ds = x(s)t δ(t s) ds (Homogeneity)
2 Convolution and the Impulse Response Function y(t) = x(s)t δ(t s) ds Let h(t) = T δ(t) represent the impulse response function. Then y(t) = x(s)h(t s)ds x(t) h(t) where signifies convolution. Impulse Impulses Impulse Response Each Impulse Creates a Scaled and Shifted Impulse Response For example The sum of all the impulse responses is the final system response
Properties of Convolution 3 x y = y x ( x y ) z = x y z (commutative) ( ) (associative) ( x z) + ( y z) = ( x + y ) z (distributive)
Linear Shift-Invariant Systems 4 A system is linear and shift-invariant if and only if the output is a weighted sum of the input. past present future /8 /4 /2 /2 /4 /8 input (impulse) weights output (impulse response) input (step) /8 /4 /2 weights /2 3/4 7/8 7/8 7/8 7/8 output (step response)
Fourier Series 5 We have already seen that any signal x(t) can be expressed exactly as an infinite sum of impulses. It turns out that any signal can alternatively be expressed exactly as an infinite sum of sinusoids. x(t) = A f sin( 2πft + φ f )df = A ω sin( ωt + φ ω )dω This is known as a Fourier series. Joseph Fourier Fourier Series Approximations Original Squarewave 4 Term Approximation 2 - -2 2 4 6 8.5 -.5-2 4 6 8.5 -.5 2 - - 2 4 6 8-2 2 4 6 8 8 Term Approximation 6 Term Approximation
Fourier Transforms 6 The expansion of a signal in terms of sinusoids can be more neatly expressed using complex numbers: x(t) = X(f )e j 2πft df where e j 2πft = cos 2πft + j sin 2πft (Euler's equation) Inverse Fourier transform x(t) = F X(f ) X(f) can be computed from x(t) using X(f ) = x(t)e j 2πft dt Fourier transform X(f ) = F x(t)
Fourier Transforms 7 x(t) Signal Amplitude of Fourier Transform X(f) real part cosine sine -8 8-8 8-4 8 imaginary part -8 8-8 8 x(t) X(f) amplitude -8 8-8 8 - Time (sec) 4 Frequency (Hz) 8 phase Frequency (Hz) Frequency (Hz) π/2
Some Properties of Fourier Transforms 8 The Fourier transform is a linear system. Thus F ( α x (t) + βx 2 (t)) = α F ( x (t)) + β F ( x 2 (t)) Convolution property: F ( h(t) x(t) ) = F ( h(t) ) F ( x(t) ) = H(f ) X(f ) Differentiation property: F d dt x(t) = 2πf F ( x(t) )
Why Fourier transforms? 9 When we feed an impulse into a shiftinvariant linear system, we get out a potentially complicated function (the impulse response). Impulse Impulse Response In contrast, when we feed a sinusoid into a shift-invariant linear system, we always get out another sinusoid of the same frequency, though generally different amplitude and phase. Thus to identify the system, all we need to do is stimulate with different sinusoids of known frequency, amplitude and phase, and record the amplitude and phase of each corresponding output sinusoid. Sinusoidal Inputs Scaled and Shifted sinusoidal outputs scaling shifting Frequency Description of the system x x x frequency x frequency x x
Linear Systems Identification 2 Linear Systems Logic Measure the impulse response Space/time method Input Stimulus Express as sum of scaled and shifted impulses Calculate the response to each impulse Sum the impulse responses to determine the output Express as sum of scaled and shifted sinusoids Calculate the response to each sinusoid Sum the sinusoidal responses to determine the output Measure the sinusoidal responses Frequency method
Example: retinal ganglion cell model 2 - + - - - + - - - + - - Response amplitude... 3 3 Spatial frequency (cpd)
Different forms of linear system 22 x(ω) A h(ω) y(ω) x(ω) h (ω) h (ω) 2 : y (ω) y (ω) 2 B h (ω) N y (ω) N x(ω) + y(ω) f(ω) C