Linear systems, small signals, and integrators

Linear systems, small signals, and integrators CNS WS05 Class Giacomo Indiveri Institute of Neuroinformatics University ETH Zurich Zurich, December 2005

Outline 1 Linear Systems Crash Course Linear Time-Invariant Systems Convolution & Impulse response Impulse and step response Complex numbers and Transfer functions 2 RC integrators and differentiators Resistor-Capacitor circuits RC circuit frequency analysis 3 Integrators in VLSI The follower integrator A log-domain integrator

Linear Systems x(t) SYSTEM y(t) Input signal x(t) Output signal y(t) Transfer function F[ ] y(t) = F[x(t)] If a linear time-invariant system has no internally stored energy, then the output of the system y(t) is the forced response due entirely to its input x(t): y(t) = F[x(t)]. A system is said to be linear if it obeys the two fundamental principles of homogeneity and additivity.

Homogeneity and Additivity Homogeneity If then ˆx(t) = αx(t) ŷ(t) = F[αx(t)] = αf[x(t)] Additivity If x(t) = a k x k (t) k with a k constant k then y(t) = a k F[x k (t)] k

Homogeneity Input Output

Additivity Input Output

Time invariance A time-invariant system obeys the following time-shift invariance property: If the input signal ˆx(t) is then for any real constant τ, ˆx(t) = x(t τ) ŷ(t) = F[x(t τ)] = y(t τ)

Time invariance Input Output

Convolution It is a useful mathematical tool for analyzing linear systems. v(t) w(t). = = + + v(λ)w(t λ)dλ v(t λ)w(λ)dλ Note that in the integral t is the independent variable, and λ is the integration variable. Insofar as the integral is concerned, t is constant.

Convolution Graphical example v(t) w(t) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 1 0.5 0 0.5 1 1.5 2 0 1 0.5 0 0.5 1 1.5 2

Convolution Graphical example 1 0.8 0.6 0.4 0.2 0 1 0.5 0 0.5 1 1.5 2

Convolution Graphical example 50 v(t) w(t) 45 40 35 30 25 20 15 10 5 0 1 0.5 0 0.5 1 1.5 2 2.5 3

Impulses The Dirac-delta function, or unit impulse δ(t) is not a strict function in the mathematical sense. It is only defined by a set of assignment rules.

Impulses The Dirac-delta function, or unit impulse δ(t) is not a strict function in the mathematical sense. It is only defined by a set of assignment rules. If v(t) is continuous at time t = 0, { t2 v(0) if t 1 < 0 < t 2 v(t)δ(t)dt = t 1 0 otherwise + +ε δ(t)dt = δ(t)dt = 1 ε The Dirac-delta function, δ(t) has unit area at t = 0, is null for t 0, and has no mathematical or physical meaning, unless it appears under the operation of integration.

Impulse integration properties Replication operation v(t) δ(t τ) = v(t τ)

Impulse integration properties Replication operation Sampling operation v(t) δ(t τ) = v(t τ) + v(t)δ(t τ)dt = v(τ)

Linear system s impulse response We define the impulse response of a system characterized by y(t) = F[x(t)] as: h(t). = F[δ(t)] If x(t) is a continuous function, x(t) = x(t) δ(t).

Linear system s impulse response We define the impulse response of a system characterized by y(t) = F[x(t)] as: h(t). = F[δ(t)] If x(t) is a continuous function, x(t) = x(t) δ(t). [ + ] y(t) = F[x(t)] = F x(λ)δ(t λ)dλ If the system is linear + y(t) = x(λ)f[δ(t λ)]dλ

Step and impulse response If u(t) is a step function such that { 1 if t 0 u(t) = 0 otherwise the system s step response is defined as g(t). = F[u(t)].

Step and impulse response If u(t) is a step function such that { 1 if t 0 u(t) = 0 otherwise the system s step response is defined as g(t). = F[u(t)]. Exploiting the system s impulse response properties: g(t) = h(t) u(t) d dt g(t) = h(t) d u(t) = h(t) δ(t) dt

Complex exponentials Linear systems are typically described by a set of linear differential equations. A first order linear system is characterized by a first order linear differential equation. A linear system of order n is characterized by a linear differential equation of order n. Time-domain analysis becomes increasingly difficult for higher order systems. Fortunately there is a unified representation in which any solution to a linear system can be expressed: Exponentials with complex arguments.

Complex exponentials All solutions to linear homogeneous equations are of the form e st where s. = σ + jω = M cos(φ) + jm sin(φ), and j = 1. σ represents the real part of the complex number, ω its imaginary part, M represents its magnitude and φ its phase. Magnitude and phase of a complex number obey to the following relationships: M = σ 2 + ω 2 e jφ = cos(φ) + j sin(φ) φ = ( ) ω arctan σ e jφ = cos(φ) j sin(φ) It follows that s can be also written as s = Me jφ

Measured responses In practice, we measure responses of the type V = e st with real instruments, so if e st were a complex exponential, we would be able to measure only { Re e st} } = Re {e (σ+jω)t = { e σt Re e jωt} So the measured response of the system would be V ω V meas = e σt cosωt V V t V V t t V t t 0 t σ

Heaviside-Laplace Transform Oliver Heaviside, in analyzing analog circuits made the following observation: d n dt n est = s n e st we can consider s as an operator meaning derivative with respect to time. Similarly, we can view 1 as the operator for s integration with respect to time. This observation was formalized by mathematicians, when they realized that Heaviside s method was using polynomials in s that were Laplace transforms. The Laplace transform is an operator that allows to link functions that operate in the time domain with functions of complex variables: L [y(t)] = Y (s) y(t)e st dt

Impulse response and transfer functions The Transfer Function is defined as H(s) Y (s) X(s) If x(t) = δ(t), then X(s) = δ(t)e st dt = 1 and Y (s) = y(t)e st dt = h(t)e st dt Hence, H(s) = h(λ)e λ s dλ = L [h(t)]

Resistor-Capacitor circuits x(t) + - R C + - y(t) x(t) + - C R + - y(t) First order, linear, time-invariant systems. The input signal is x(t), the output signal is y(t). RC integrator circuit CR differentiator circuit

RC integrator circuit R x(t) + - C + - y(t) Time domain: RC d y(t) + y(t) = x(t) dt Solving for x(t) = δ(t): h(t) = 1 RC e t/rc u(t) Laplace domain: RC sy (s) + Y (s) = X(s) Solving for X(s) = 1: 1 H(s) = 1 + RC s

RC integrator circuit impulse response h(t) 1/RC τ = RC The circuit s time constant τ = RC is the time required to discharge the capacitor, through the resistor, to 36.8% (1/e) of its final steady state value. t

CR differentiator circuit x(t) + - C R + - y(t) Time domain: τ d dt y(t) + y(t) = τ d dt x(t) For a step input, the derivative of the input is an impulse at t = 0. Thus, the capacitor reaches full charge very quickly and becomes an open circuit Laplace domain: τ sy (s) + Y (s) = τ sx(s) Solving for X(s) = 1: H(s) = τ s 1 + τ s

Frequency domain analysis To analyze the system s response in the frequency domain, we have to apply as input signals sinusoids. Sinusoids have a very special relationship to shift-invariant linear systems: when we apply a sinusoidal signal as input to a shift-invariant linear system, the system s response will be an other sinusoidal signal, with probably a different amplitude and a different phase, but with the exact same frequency! If the input is x(t) = sin(ωt) the output we will be y(t) = Asin(ωt + φ) where A and φ determine the amount of scaling and shift.

RC-integrator frequency domain analysis In this domain s = jω and the RC-circuit s transfer function is: H(jω) = ωτ 1 Y (jω) X(jω) ωτ 1 Y (jω) 1 jωτ X(jω) 1 1 + jωτ

RC-integrator frequency domain analysis In this domain s = jω and the RC-circuit s transfer function is: H(jω) = 1 1 + jωτ ωτ 1 Y (jω) X(jω) ωτ 1 Y (jω) 1 jωτ X(jω) The frequency f c = ω 2π = 1 2π τ is defined to be the cutoff frequency. In electronics, cutoff frequency (fc) is the frequency either above which or below which the power output of a circuit, such as a line, amplifier, or filter, is reduced to 1/2 of the passband power; the half-power point. This is equivalent to a voltage (or amplitude) reduction to 70.7% of the passband, because voltage V 2 is proportional to power P. This happens to be close to -3 decibels, and the cutoff frequency is frequently referred to as the -3 db point. Wikipedia, http://en.wikipedia.org/wiki/cutoff_frequency

Magnitude, phase, and Bode plots The magnitude of the RC-integrator s transfer function is: Its phase is H(jω) = 1 1 + (ωτ) 2 φ = arctan( ωτ) The plots of a transfer function s magnitude and phase versus input frequency are called Bode plots (after Hendrik Wade Bode). 10 0 0 10 20 Magnitude Phase (deg) 30 40 50 60 10 0 10 1 Angular frequency (Hz) 70 80 90 10 0 10 1 Angular frequency (Hz)

Low pass filters Integrators are low-pass filters: they allow low frequencies to pass unchanged, and attenuate, or suppress high frequencies.

Low pass filters Integrators are low-pass filters: they allow low frequencies to pass unchanged, and attenuate, or suppress high frequencies. I inj-1 I inj I inj+1 G V j-1 I j-1 G V j I j G V j+1 I j+1 G I outj-1 I outj I outj+1 R R R

RC-integrator frequency response Normalized RC responce 1.2 1 0.8 0.6 0.4 0.2 Input 50 Hz 100 Hz 200 Hz 400 Hz 0 0 0.2 0.4 0.6 0.8 1 Normalized time units Responses of an RC lowpass filter with R = 10MΩ and C = 1nF. All data is plotted on a normalized scale. The signals amplitudes have been normalized with respect to the input and the time-base has been normalized to unity.

Linear Passive Filters RC and CR circuits are oldest and the most basic form of passive filters. Typically filters are designed using active components, and can be of several types: A low-pass filter passes low frequencies. A high-pass filter passes high frequencies. A band-pass filter passes a limited range of frequencies. A band-stop filter passes all frequencies except a limited range. In the real world, filter design is a very complicated art. We just scratched the surface. To know more about filter design: ETH course (227-0468-00) ( ASF) Analoge Signalverarbeitung und Filterugn, by Dr. Hanspeter Schmid Williams, Arthur B & Taylor, Fred J (1995). Electronic Filter Design Handbook, McGraw-Hill. ISBN 0-07-070441-4.

The subthreshold follower-integrator circuit V in + V b C V out Its composed of a unity-gain follower biased in subthreshold, connected to a capacitor. Transconductance amplifier: Transconductance: g m = I out (V 1 V 2 ) = I bκ 2U T Output conductance: g d = I out V out I b V E Input conductance: g in = I in V out 0 Voltage gain: A = V out (V 1 V 2 ) = g m g d Unity-gain follower: Transfer function: dv out dv in = A A+1 1. Input impedance: Z in = V in I in Output impedance: Z out = V out I out = 1 g d V E Ib

The follower-integrator circuit V out V in + V b C C d dt V out = I b tanh In the small-signal regime: ( ) κ(vin V out ) 2U T C d dt V out = G(V in V out ) where G = κi b 2U T is the amplifier s transconductance.

The follower-integrator circuit In the small signal regime, the follower-integrator has a transfer function that is identical to the one of the R-C integrator, with a time constant τ = C G : V out = 1 V in 1 + τs The amplifier will operate in its linear range as long the voltage V in does not change too rapidly: d dt V in < 4U T τ

Delay Lines What happens when we combine RC-integrator and follower-integrator circuits? R R R R V in V out C C C C V in V out + C + C + C V b V b V b

The follower-integrator composition property Each section in the follower-integrator delay line is modular (from a functional point of view) and independent of other sections. If the number of elements in the delay line is n, we can write ( ) n V out 1 = V in 1 + τs We can write the delay-line transfer function explicitly in terms of magnitude and phase: V out V in 1 1 + n e jnωτ (jωτ)2 2

Follower-integrator large signal behavior For very large variations of V in, the output current of the transconductance amplifier saturates to ± I b. As long as V out V in > 4U T,V out will change linearly with time. As V out V in 4U T, the amplifier starts behaving as a linear conductance and V out starts increasing (or decreasing) exponentially.

Current-Mode Low pass filter circuit Vdd I in I out Vdd M2 Vdd M3 M1 V b V c M4 I b I b C lp

Current-Mode Low pass filter circuit Vdd I in I out I M1 ds I M2 ds = I M3 ds I M4 ds Vdd Vdd I M3 ds = I b + C lp d dt V c M1 M2 V b I b M3 I b V c C lp M4 d dt I out = κ d I out U T dt V c I in I b = (I b + C lpu T κ I out I out ) I out V M1 gs + V M2 gs V M3 gs V M4 gs = 0 κvg Vs U I ds = I 0 e T τ I out + I out = I in with τ = C lpu T κi b

Synapses and Neurons Synapses integrate 160 Neurons differentiate 14 4 Excitatory Synaptic Current (na) 140 120 100 80 60 40 20 Instantaneous firing rate (Hz) 12 10 8 6 4 3 2 1 0 0 1 2 3 4 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Time (s) 2 0 5 10 15 20 Spike count

Next week Silicon Neurons and then... Silicon Synapses