EE-295 Image Processing, Spring 2008 Lecture 1 Material presented here is from the course 6.003, Signals & Systems offered by MIT faculty member, Prof. Alan Willsky, Copyright c 2003. This material is subject to the MIT OpenCourseWare Creative Commons license, http://ocw.mit.edu/ocwweb/web/terms/terms/index.htm#cc. 1
Signals and Systems Fall 2003 Lecture #1 Prof. Alan S. Willsky 4 September 2003 1) Administrative details 2) Signals 3) Systems 4) For examples... Figures and images used in these lecture notes by permission, copyright 1997 by Alan V. Oppenheim and Alan S. Willsky 1
SIGNALS Signals are functions of independent variables that carry information. For example: Electrical signals --- voltages and currents in a circuit Acoustic signals --- audio or speech signals (analog or digital) Video signals --- intensity variations in an image (e.g. a CAT scan) Biological signals --- sequence of bases in a gene... 2
THE INDEPENDENT VARIABLES Can be continuous Trajectory of a space shuttle Mass density in a cross-section of a brain Can be discrete DNA base sequence Digital image pixels Can be 1-D, 2-D, N-D For this course: Focus on a single (1-D) independent variable which we call time. Continuous-Time (CT) signals: x(t), t continuous values Discrete-Time (DT) signals: x[n], n integer values only 3
CT Signals Most of the signals in the physical world are CT signals E.g. voltage & current, pressure, temperature, velocity, etc. 4
DT Signals x[n], n integer, time varies discretely Examples of DT signals in nature: DNA base sequence Population of the nth generation of certain species 5
Many human-made DT Signals Ex.#1 Weekly Dow-Jones industrial average Ex.#2 digital image Courtesy of Jason Oppenheim. Used with permission. Why DT? Can be processed by modern digital computers and digital signal processors (DSPs). 6
SYSTEMS For the most part, our view of systems will be from an input-output perspective: A system responds to applied input signals, and its response is described in terms of one or more output signals x(t) CT System y(t) x[n] DT System y[n] 7
An RLC circuit EXAMPLES OF SYSTEMS Dynamics of an aircraft or space vehicle An algorithm for analyzing financial and economic factors to predict bond prices An algorithm for post-flight analysis of a space launch An edge detection algorithm for medical images 8
SYSTEM INTERCONNECTIOINS An important concept is that of interconnecting systems To build more complex systems by interconnecting simpler subsystems To modify response of a system Signal flow (Block) diagram Cascade Parallel + Feedback + 9
Signals and Systems Fall 2003 Lecture #10 7 October 2003 1. Examples of the DT Fourier Transform 2. Properties of the DT Fourier Transform 3. The Convolution Property and its Implications and Uses
DT Fourier Transform Pair Analysis Equation FT Synthesis Equation Inverse FT
Convergence Issues Synthesis Equation: Analysis Equation: None, since integrating over a finite interval Need conditions analogous to CTFT, e.g. Finite energy Absolutely summable
Examples Parallel with the CT examples in Lecture #8
More Examples Infinite sum formula
Still More 4) DT Rectangular pulse (Drawn for N 1 = 2)
5)
DTFTs of Sums of Complex Exponentials Recall CT result: What about DT: a) We expect an impulse (of area 2π) at ω = ω o b) But X(e jω ) must be periodic with period 2π In fact Note: The integration in the synthesis equation is over 2π period, only need X(e jω ) in one 2π period. Thus,
DTFT of Periodic Signals DTFS synthesis eq. Linearity of DTFT
Example #1: DT sine function
Example #2: DT periodic impulse train Also periodic impulse train in the frequency domain!
Properties of the DT Fourier Transform Different from CTFT
More Properties Example Important implications in DT because of periodicity
Still More Properties
7) Time Expansion Recall CT property: But in DT: Yet Still More Properties x[n/2] makes no sense x[2n] misses odd values of x[n] Time scale in CT is infinitely fine But we can slow a DT signal down by inserting zeros: k an integer 1 x (k) [n] insert (k - 1) zeros between successive values Insert two zeros in this example (k=3)
Time Expansion (continued) Stretched by a factor of k in time domain -compressed by a factor of k in frequency domain
Is There No End to These Properties? 8) Differentiation in Frequency Multiplication by n 9) Parseval s Relation Differentiation in frequency Total energy in time domain Total energy in frequency domain
Example #1: The Convolution Property
Example #2: Ideal Lowpass Filter
Example #3:
Signals and Systems Fall 2003 Lecture #11 9 October 2003 1. DTFT Properties and Examples 2. Duality in FS & FT 3. Magnitude/Phase of Transforms and Frequency Responses
Convolution Property Example
DT LTI System Described by LCCDE s Rational function of e -jω, use PFE to get h[n]
Example: First-order recursive system with the condition of initial rest causal
DTFT Multiplication Property
Calculating Periodic Convolutions
Example:
Duality in Fourier Analysis Fourier Transform is highly symmetric CTFT: Both time and frequency are continuous and in general aperiodic Same except for these differences Suppose f( ) and g( ) are two functions related by Then
Example of CTFT duality Square pulse in either time or frequency domain
DTFS Duality in DTFS Then
Duality between CTFS and DTFT CTFS DTFT
CTFS-DTFT Duality
Magnitude and Phase of FT, and Parseval Relation CT: Parseval Relation: Energy density in ω DT: Parseval Relation:
Effects of Phase Not on signal energy distribution as a function of frequency Can have dramatic effect on signal shape/character Constructive/Destructive interference Is that important? Depends on the signal and the context
Demo: 1) Effect of phase on Fourier Series 2) Effect of phase on image processing
Log-Magnitude and Phase Easy to add
Plotting Log-Magnitude and Phase a) For real-valued signals and systems Plot for ω 0, often with a logarithmic scale for frequency in CT b) In DT, need only plot for 0 ω π (with linear scale) c) For historical reasons, log-magnitude is usually plotted in units of decibels (db): power magnitude So 20 db or 2 bels: = 10 amplitude gain = 100 power gain
A Typical Bode plot for a second-order CT system 20 log H(jω) and H(jω) vs. log ω 40 db/decade Changes by -π
A typical plot of the magnitude and phase of a secondorder DT frequency response 20log H(e jω ) and H(e jω ) vs. ω For real signals, 0 to π is enough
Signals and Systems Fall 2003 Lecture #12 16 October 2003 1. Linear and Nonlinear Phase 2. Ideal and Nonideal Frequency-Selective Filters 3. CT & DT Rational Frequency Responses 4. DT First- and Second-Order Systems
Linear Phase CT Result: Linear phase simply a rigid shift in time, no distortion Nonlinear phase distortion as well as shift DT Question:
All-Pass Systems CT DT
Demo: Impulse response and output of an all-pass system with nonlinear phase
φ How do we think about signal delay when the phase is nonlinear? Group Delay
Ideal Lowpass Filter CT Noncausal h(t <0) 0 Oscillatory Response e.g. step response Overshoot by 9%, Gibbs phenomenon
Nonideal Lowpass Filter Sometimes we don t want a sharp cutoff, e.g. Often have specifications in time and frequency domain Trade-offs Freq. Response Step response
CT Rational Frequency Responses CT: If the system is described by LCCDEs, then Prototypical Systems First-order system, has only one energy storing element, e.g. L or C Second-order system, has two energy storing elements, e.g. L and C
DT Rational Frequency Responses If the system is described by LCCDE s (Linear-Constant-Coefficient Difference Equations), then
DT First-Order Systems
Demo: Unit-sample, unit-step, and frequency response of DT first-order systems
DT Second-Order System decaying oscillations
Demo: Unit-sample, unit-step, and frequency response of DT second-order systems
Signals and Systems Fall 2003 1. The Concept and Representation of Periodic Sampling of a CT Signal 2. Analysis of Sampling in the Frequency Domain 3. The Sampling Theorem the Nyquist Rate 4. In the Time Domain: Interpolation 5. Undersampling and Aliasing Lecture #13 21 October 2003
SAMPLING We live in a continuous-time world: most of the signals we encounter are CT signals, e.g. x(t). How do we convert them into DT signals x[n]? Sampling, taking snap shots of x(t) every T seconds. T sampling period x[n] x(nt), n =..., -1, 0, 1, 2,... regularly spaced samples Applications and Examples Digital Processing of Signals Strobe Images in Newspapers Sampling Oscilloscope How do we perform sampling?
Why/When Would a Set of Samples Be Adequate? Observation: Lots of signals have the same samples By sampling we throw out lots of information all values of x(t) between sampling points are lost. Key Question for Sampling: Under what conditions can we reconstruct the original CT signal x(t) from its samples?
Impulse Sampling Multiplying x(t) by the sampling function
Analysis of Sampling in the Frequency Domain Important to note: ω s 1/T
Illustration of sampling in the frequency-domain for a band-limited (X(jω)=0 for ω > ω M ) signal No overlap between shifted spectra
Reconstruction of x(t) from sampled signals If there is no overlap between shifted spectra, a LPF can reproduce x(t) from x p (t)
The Sampling Theorem Suppose x(t) is bandlimited, so that Then x(t) is uniquely determined by its samples {x(nt)} if
(1) In practice, we obviously don t sample with impulses or implement ideal lowpass filters. One practical example: The Zero-Order Hold Observations on Sampling
Observations (Continued) (2) Sampling is fundamentally a time-varying operation, since we multiply x(t) with a time-varying function p(t). However, is the identity system (which is TI) for bandlimited x(t) satisfying the sampling theorem (ω s > 2ω M ). (3) What if ω s 2ω M? Something different: more later.
Time-Domain Interpretation of Reconstruction of Sampled Signals Band-Limited Interpolation The lowpass filter interpolates the samples assuming x(t) contains no energy at frequencies ω c
Original CT signal Graphic Illustration of Time-Domain Interpolation h(t) After sampling T After passing the LPF
Interpolation Methods Bandlimited Interpolation Zero-Order Hold First-Order Hold Linear interpolation
When ω s 2 ω M Undersampling Undersampling and Aliasing
Undersampling and Aliasing (continued) X r (jω) X(jω) Distortion because of aliasing Higher frequencies of x(t) are folded back and take on the aliases of lower frequencies Note that at the sample times, x r (nt) = x(nt)
A Simple Example Picture would be Modified Demo: Sampling and reconstruction of cosω o t
Signals and Systems Fall 2003 Lecture #14 23 October 2003 1. Review/Examples of Sampling/Aliasing 2. DT Processing of CT Signals
Sampling Review Demo: Effect of aliasing on music.
Strobe Demo > 0, strobed image moves forward, but at a slower pace = 0, strobed image still < 0, strobed image moves backward. Applications of the strobe effect (aliasing can be useful sometimes): E.g., Sampling oscilloscope
DT Processing of Band-Limited CT Signals Why do this? Inexpensive, versatile, and higher noise margin. How do we analyze this system? We will need to do it in the frequency domain in both CT and DT In order to avoid confusion about notations, specify ω CT frequency variable Ω DT frequency variable (Ω = ωτ) Step 1: Find the relation between x c (t) and x d [n], or X c (jω) and X d (e jω )
Note: Not full analog/digital (A/D) conversion not quantizing the x[n] values Time-Domain Interpretation of C/D Conversion
Frequency-Domain Interpretation of C/D Conversion Note: ω s 2π CT DT
Illustration of C/D Conversion in the Frequency-Domain X d (e jω ) X d (e jω ) Ω = ωt 1 Ω = ωt2
D/C Conversion y d [n] y c (t) Reverse of the process of C/D conversion
Now the whole picture Overall system is time-varying if sampling theorem is not satisfied It is LTI if the sampling theorem is satisfied, i.e. for bandlimited inputs x c (t), with ω M < ω s 2 When the input x c (t) is band-limited (X(jω) = 0 at ω > ω Μ ) and the sampling theorem is satisfied (ω s > 2ω M ), then DT omege needs to changed
Frequency-Domain Illustration of DT Processing of CT Signals DT filter Sampling DT freq CT freq CT freq DT freq Interpolate (LPF) equivalent CT filter
Assuming No Aliasing In practice, first specify the desired H c (jω), then design H d (e jω ).
Example: Digital Differentiator Applications: Edge Enhancement Courtesy of Jason Oppenheim. Used with permission. Courtesy of Jason Oppenheim. Used with permission.
Bandlimited Differentiator Construction of Digital Differentiator
Band-Limited Digital Differentiator (continued) CT DT
Signals and Systems Fall 2003 Lecture #2 9 September 2003 1) Some examples of systems 2) System properties and examples a) Causality b) Linearity c) Time invariance
SYSTEM EXAMPLES x(t) CT System y(t) x[n] DT System y[n] Ex. #1 RLC circuit
Ex. #2 Mechanical system Force Balance: Observation: Very different physical systems may be modeled mathematically in very similar ways.
Ex. #3 Thermal system Cooling Fin in Steady State
Ex. #3 (Continued) Observations Independent variable can be something other than time, such as space. Such systems may, more naturally, have boundary conditions, rather than initial conditions.
Ex. #4 Financial system Fluctuations in the price of zero-coupon bonds t = 0 Time of purchase at price y 0 t = T Time of maturity at value y T y(t) = Values of bond at time t x(t) = Influence of external factors on fluctuations in bond price Observation: Even if the independent variable is time, there are interesting and important systems which have boundary conditions.
Ex. #5 A rudimentary edge detector This system detects changes in signal slope 0 1 2 3
Observations 1) A very rich class of systems (but by no means all systems of interest to us) are described by differential and difference equations. 2) Such an equation, by itself, does not completely describe the input-output behavior of a system: we need auxiliary conditions (initial conditions, boundary conditions). 3) In some cases the system of interest has time as the natural independent variable and is causal. However, that is not always the case. 4) Very different physical systems may have very similar mathematical descriptions.
WHY? SYSTEM PROPERTIES (Causality, Linearity, Time-invariance, etc.) Important practical/physical implications They provide us with insight and structure that we can exploit both to analyze and understand systems more deeply.
CAUSALITY A system is causal if the output does not anticipate future values of the input, i.e., if the output at any time depends only on values of the input up to that time. Allreal-time physical systems are causal, because time only moves forward. Effect occurs after cause. (Imagine if you own a noncausal system whose output depends on tomorrow s stock price.) Causality does not apply to spatially varying signals. (We can move both left and right, up and down.) Causality does not apply to systems processing recorded signals, e.g. taped sports games vs. live broadcast.
CAUSALITY (continued) Mathematically (in CT): A system x(t) y(t) is causal if when and x 1 (t) y 1 (t) x 2 (t) y 2 (t) x 1 (t) = x 2 (t) for all t t o Then y 1 (t) = y 2 (t) for all t t o
CAUSAL OR NONCAUSAL
TIME-INVARIANCE (TI) Informally, a system is time-invariant (TI) if its behavior does not depend on what time it is. Mathematically (in DT): A system x[n] y[n] is TI if for any input x[n] and any time shift n 0, If x[n] y[n] then x[n - n 0 ] y[n - n 0 ]. Similarly for a CT time-invariant system, If x(t) y(t) then x(t - t o ) y(t - t o ).
TIME-INVARIANT OR TIME-VARYING? TI Time-varying (NOT time-invariant)
NOW WE CAN DEDUCE SOMETHING! Fact: If the input to a TI System is periodic, then the output is periodic with the same period. Proof : Suppose x(t + T) = x(t) and x(t) y(t) Then by TI x(t + T) y(t + T). These are the same input! So these must be the same output, i.e., y(t) = y(t + T).
LINEAR AND NONLINEAR SYSTEMS Many systems are nonlinear. For example: many circuit elements (e.g., diodes), dynamics of aircraft, econometric models, However, in 6.003 we focus exclusively on linear systems. Why? Linear models represent accurate representations of behavior of many systems (e.g., linear resistors, capacitors, other examples given previously, ) Can often linearize models to examine small signal perturbations around operating points Linear systems are analytically tractable, providing basis for important tools and considerable insight
LINEARITY A (CT) system is linear if it has the superposition property: If x 1 (t) y 1 (t) and x 2 (t) y 2 (t) then ax 1 (t) + bx 2 (t) ay 1 (t) + by 2 (t) y[n] = x 2 [n] y(t) = x(2t) Nonlinear, TI, Causal Linear, notti, Noncausal Can you find systems with other combinations? - e.g. Linear, TI, Noncausal Linear, not TI, Causal
PROPERTIES OF LINEAR SYSTEMS Superposition If Then For linear systems, zero input zero output "Proof" 0 = 0 x[n] 0 y[n] = 0
Properties of Linear Systems (Continued) A linear system is causal if and only if it satisfies the condition of initial rest: Proof a) Suppose system is causal. Show that (*) holds. b) Suppose (*) holds. Show that the system is causal.
LINEAR TIME-INVARIANT (LTI) SYSTEMS Focus of most of this course - Practical importance (Eg. #1-3 earlier this lecture are all LTI systems.) - The powerful analysis tools associated with LTI systems A basic fact: If we know the response of an LTI system to some inputs, we actually know the response to many inputs
Example: DT LTI System
Signals and Systems Fall 2003 Lecture #21 25 November 2003 1. Feedback a) Root Locus b) Tracking c) Disturbance Rejection d) The Inverted Pendulum 2. Introduction to the Z-Transform
The Concept of a Root Locus C(s),G(s) Designed with one or more free parameters Question: How do the closed-loop poles move as we vary these parameters? Root locus of 1+ C(s)G(s)H(s)
The Classical Root Locus Problem C(s) = K a simple linear amplifier Closed-loop poles are the same.
A Simple Example In either case, pole is at s o = -2 - K Sketch where pole moves as K increases... Becomes more stable Becomes less stable
What Happens More Generally? For simplicity, suppose there is no pole-zero cancellation in G(s)H(s) Closed-loop poles are the solutions of That is Difficult to solve explicitly for solutions given any specific value of K, unless G(s)H(s) is second-order or lower. Much easier to plot the root locus, the values of s that are solutions for some value of K, because: 1) It is easier to find the roots in the limiting cases for K = 0, ±. 2) There are rules on how to connect between these limiting points.
Rules for Plotting Root Locus End points At K = 0, G(s o )H(s o ) = s o are poles of the open-loop system function G(s)H(s). At K =, G(s o )H(s o ) = 0 s o are zeros of the open-loop system function G(s)H(s). Thus: Rule #1: A root locus starts (at K = 0) from a pole of G(s)H(s) and ends (at K = ) at a zero of G(s)H(s). Question: Answer: What if the number of poles the number of zeros? Start or end at ±.
Rule #2: Angle criterion of the root locus Thus, s 0 is a pole for some positive value of K if: In this case, s 0 is a pole if K = 1/ G(s 0 ) H(s 0 ). Similarly s 0 is a pole for some negative value of K if: In this case, s 0 is a pole if K = -1/ G(s 0 ) H(s 0 ).
Example of Root Locus. One zero at -2, two poles at 0, -1.
In addition to stability, we may want good tracking behavior, i.e. for at least some set of input signals. Tracking + = ) ( ) ( ) ( 1 1 ) ( s X s H s C s E ) ( ) ( ) ( 1 1 ) ( ω ω ω ω j X j H j C j E + = We want to be large in frequency bands in which we want good tracking ) ( ) ( ω ω j P j C
Tracking (continued) Using the final-value theorem Basic example: Tracking error for a step input
Disturbance Rejection There may be other objectives in feedback controls due to unavoidable disturbances. Clearly, sensitivities to the disturbances D 1 (s) and D 2 (s) are much reduced when the amplitude of the loop gain
Internal Instabilities Due to Pole-Zero Cancellation H w(t) ) ( 3 3 1 ) ( ) ( ) ( 1 ) ( ) ( ) ( 2 ) ( 1) ( 1 ) ( Stable 2 s X s s s X s H s C s H s C s Y s s s H, s s s C + + = + = + = + = However ) ( 3) 3 ( 2 ) ( ) ( ) ( 1 ) ( ) ( Unstable 2 s X s s s s s X s H s C s C s W + + + = + =
Inverted Pendulum Unstable!
Feedback System to Stabilize the Pendulum a PI feedback stabilizes θ Subtle problem: internal instability in x(t)! Additional PD feedback around motor / amplifier centers the pendulum
Root Locus & the Inverted Pendulum Attempt #1: Negative feedback driving the motor Root locus of M(s)G(s) Remains unstable! after K. Lundberg
Root Locus & the Inverted Pendulum Attempt #2: Proportional/Integral Compensator Root locus of K(s)M(s)G(s) Stable for large enough K after K. Lundberg
Root Locus & the Inverted Pendulum BUT x(t) unstable: System subject to drift... Solution: add PD feedback around motor and compensator: after K. Lundberg
The z-transform Motivation: Analogous to Laplace Transform in CT We now do not restrict ourselves just to z = e jω The (Bilateral) z-transform
The ROC and the Relation Between zt and DTFT, r = z depends only on r = z, just like the ROC in s-plane only depends on Re(s) Unit circle (r = 1) in the ROC DTFT X(e jω ) exists
Example #1 This form for PFE and inverse z- transform = 1 1 az 1 = z z a That is, ROC z > a, outside a circle This form to find pole and zero locations
Example #2: Same X(z) as in Ex #1, but different ROC.
Rational z-transforms x[n] = linear combination of exponentials for n > 0 and for n < 0 Polynomials in z characterized (except for a gain) by its poles and zeros
Signals and Systems Fall 2003 Lecture #22 2 December 2003 1. Properties of the ROC of the z-transform 2. Inverse z-transform 3. Examples 4. Properties of the z-transform 5. System Functions of DT LTI Systems a. Causality b. Stability
The z-transform -depends only on r = z, just like the ROC in s-plane only depends on Re(s) Last time: Unit circle (r = 1) in the ROC DTFT X(e jω ) exists Rational transforms correspond to signals that are linear combinations of DT exponentials
Some Intuition on the Relation between zt and LT The (Bilateral) z-transform Can think of z-transform as DT version of Laplace transform with
More intuition on zt-lt, s-plane - z-plane relationship LHP in s-plane, Re(s) < 0 z = e st < 1, inside the z = 1 circle. Special case, Re(s) = - z = 0. RHP in s-plane, Re(s) > 0 z = e st > 1, outside the z = 1 circle. Special case, Re(s) = + z =. A vertical line in s-plane, Re(s) = constant e st = constant, a circle in z-plane.
Properties of the ROCs of z-transforms (1) The ROC of X(z) consists of a ring in the z-plane centered about the origin (equivalent to a vertical strip in the s-plane) (2) The ROC does not contain any poles (same as in LT).
More ROC Properties (3) If x[n] is of finite duration, then the ROC is the entire z-plane, except possibly at z = 0 and/or z =. Why? Examples: CT counterpart
ROC Properties Continued (4) If x[n] is a right-sided sequence, and if z = r o is in the ROC, then all finite values of z for which z > r o are also in the ROC.
Side by Side (5) If x[n] is a left-sided sequence, and if z = r o is in the ROC, then all finite values of z for which 0 < z < r o are also in the ROC. (6) If x[n] is two-sided, and if z = r o is in the ROC, then the ROC consists of a ring in the z-plane including the circle z = r o. What types of signals do the following ROC correspond to? right-sided left-sided two-sided
Example #1
Example #1 continued Clearly, ROC does not exist if b > 1 No z-transform for b n.
for fixed r: Inverse z-transforms
Example #2 Partial Fraction Expansion Algebra: A = 1, B = 2 Note, particular to z-transforms: 1) When finding poles and zeros, express X(z) as a function of z. 2) When doing inverse z-transform using PFE, express X(z) as a function of z -1.
ROC III: ROC II: ROC I:
Inversion by Identifying Coefficients in the Power Series Example #3: 3-1 2 0 for all other n s A finite-duration DT sequence
Example #4: (a) (b)
Properties of z-transforms (1) Time Shifting The rationality of X(z) unchanged, different from LT. ROC unchanged except for the possible addition or deletion of the origin or infinity n o > 0 ROC z 0 (maybe) n o < 0 ROC z (maybe) (2) z-domain Differentiation same ROC Derivation:
Convolution Property and System Functions Y(z) = H(z)X(z), ROC at least the intersection of the ROCs of H(z) and X(z), can be bigger if there is pole/zero cancellation. e.g. H(z) + ROC tells us everything about system
CAUSALITY (1) h[n] right-sided ROC is the exterior of a circle possibly including z = : A DT LTI system with system function H(z) is causal the ROC of H(z) is the exterior of a circle including z =
Causality for Systems with Rational System Functions A DT LTI system with rational system function H(z) is causal (a) the ROC is the exterior of a circle outside the outermost pole; and (b) if we write H(z) as a ratio of polynomials then
Stability LTI System Stable ROC of H(z) includes the unit circle z = 1 Frequency Response H(e jω ) (DTFT of h[n]) exists. A causal LTI system with rational system function is stable all poles are inside the unit circle, i.e. have magnitudes < 1
Signals and Systems Fall 2003 Lecture #23 4 December 2003 1. Geometric Evaluation of z-transforms and DT Frequency Responses 2. First- and Second-Order Systems 3. System Function Algebra and Block Diagrams 4. Unilateral z-transforms
Geometric Evaluation of a Rational z-transform Example #1: Example #2: Example #3: All same as in s-plane
Geometric Evaluation of DT Frequency Responses First-Order System one real pole
Second-Order System Two poles that are a complex conjugate pair (z 1 = re jθ =z 2* ) Clearly, H peaks near ω = ±θ
Demo: DT pole-zero diagrams, frequency response, vector diagrams, and impulse- & step-responses
DT LTI Systems Described by LCCDEs Use the time-shift property Rational ROC: Depends on Boundary Conditions, left-, right-, or two-sided. For Causal Systems ROC is outside the outermost pole
Feedback System (causal systems) System Function Algebra and Block Diagrams negative feedback configuration Example #1: z -1 D Delay
Example #2: Cascade of two systems
Unilateral z-transform Note: (1) If x[n] = 0 for n < 0, then (2) UZT of x[n] = BZT of x[n]u[n] ROC always outside a circle and includes z = (3) For causal LTI systems,
Properties of Unilateral z-transform Many properties are analogous to properties of the BZT e.g. Convolution property (for x 1 [n<0] = x 2 [n<0] = 0) But there are important differences. For example, time-shift Derivation: Initial condition
Use of UZTs in Solving Difference Equations with Initial Conditions UZT of Difference Equation ZIR Output purely due to the initial conditions, ZSR Output purely due to the input.
Example (continued) β = 0 System is initially at rest: ZSR α = 0 Get response to initial conditions ZIR
Signals and Systems Fall 2003 Lecture #3 11 September 2003 1) Representation of DT signals in terms of shifted unit samples 2) Convolution sum representation of DT LTI systems 3) Examples 4) The unit sample response and properties of DT LTI systems
Exploiting Superposition and Time-Invariance Question: Are there sets of basic signals so that: a) We can represent rich classes of signals as linear combinations of these building block signals. b) The response of LTI Systems to these basic signals are both simple and insightful. Fact: For LTI Systems (CT or DT) there are two natural choices for these building blocks Focus for now: DT Shifted unit samples CT Shifted unit impulses
Representation of DT Signals Using Unit Samples
That is... Coefficients Basic Signals The Sifting Property of the Unit Sample
x[n] DT System y[n] Suppose the system is linear, and define h k [n] as the response to δ[n - k]: From superposition:
x[n] DT System y[n] Now suppose the system is LTI, and define the unit sample response h[n]: From TI: From LTI:
Convolution Sum Representation of Response of LTI Systems Interpretation n n n n
Visualizing the calculation of Choose value of n and consider it fixed View as functions of k with n fixed y[0] = prod of overlap for n = 0 y[1] = prod of overlap for n = 1
Calculating Successive Values: Shift, Multiply, Sum -1 1 1 = 1 0 1 + 1 2 = 2 (-1) 1 + 0 2 + 1 (-1) = -2 (-1) 2 + 0 (-1) + 1 (-1) = -3 (-1) (-1) + 0 (-1) = 1 (-1) (-1) = 1 4
Properties of Convolution and DT LTI Systems 1) A DT LTI System is completely characterized by its unit sample response
Unit Sample response
The Commutative Property Ex: Step response s[n] of an LTI system step input input Unit Sample response of accumulator
Interpretation The Distributive Property
The Associative Property Implication (Very special to LTI Systems)
Properties of LTI Systems 1) Causality 2) Stability
Signals and Systems Fall 2003 Lecture #4 16 September 2003 1. Representation of CT Signals in terms of shifted unit impulses 2. Convolution integral representation of CT LTI systems 3. Properties and Examples 4. The unit impulse as an idealized pulse that is short enough : The operational definition of δ(t)
Representation of CT Signals Approximate any input x(t) as a sum of shifted, scaled pulses
The Sifting Property of the Unit Impulse has unit area
Response of a CT LTI System LTI
Operation of CT Convolution Example: CT convolution
-1-1 0 0 1 1 2 2
PROPERTIES AND EXAMPLES 1) Commutativity: 2) 3) An integrator: 4) Step response:
DISTRIBUTIVITY
ASSOCIATIVITY
The impulse as an idealized short pulse Consider response from initial rest to pulses of different shapes and durations, but with unit area. As the duration decreases, the responses become similar for different pulse shapes.
The Operational Definition of the Unit Impulse δ(t) δ(t) idealization of a unit-area pulse that is so short that, for any physical systems of interest to us, the system responds only to the area of the pulse and is insensitive to its duration Operationally: The unit impulse is the signal which when applied to any LTI system results in an output equal to the impulse response of the system. That is, δ(t) is defined by what it does under convolution.
The Unit Doublet Differentiator Impulse response = unit doublet The operational definition of the unit doublet:
Triplets and beyond! n is number of differentiations
Integrators -1 derivatives" = integral I.R. = unit step
Integrators (continued)
Notation Define Then E.g.
Sometimes Useful Tricks Differentiate first, then convolve, then integrate
Example 1 1 2 2
Example (continued)
Signals and Systems Fall 2003 Lecture #5 18 September 2003 1. Complex Exponentials as Eigenfunctions of LTI Systems 2. Fourier Series representation of CT periodic signals 3. How do we calculate the Fourier coefficients? 4. Convergence and Gibbs Phenomenon
Portrait of Jean Baptiste Joseph Fourier Image removed due to copyright considerations. Signals & Systems, 2nd ed. Upper Saddle River, N.J.: Prentice Hall, 1997, p. 179.
Desirable Characteristics of a Set of Basic Signals a. We can represent large and useful classes of signals using these building blocks b. The response of LTI systems to these basic signals is particularly simple, useful, and insightful Previous focus: Unit samples and impulses Focus now: Eigenfunctions of all LTI systems
The eigenfunctions φ k (t) and their properties (Focus on CT systems now, but results apply to DT systems as well.) eigenvalue eigenfunction Eigenfunction in same function out with a gain From the superposition property of LTI systems: Now the task of finding response of LTI systems is to determine λ k.
Complex Exponentials as the Eigenfunctions of any LTI Systems eigenvalue eigenfunction eigenvalue eigenfunction
DT:
What kinds of signals can we represent as sums of complex exponentials? For Now: Focus on restricted sets of complex exponentials CT: DT: Magnitude 1 CT & DT Fourier Series and Transforms Periodic Signals
Fourier Series Representation of CT Periodic Signals -smallest such T is the fundamental period - is the fundamental frequency ω o = 2π T - periodic with period T -{a k } are the Fourier (series) coefficients - k = 0 DC - k = ±1 first harmonic - k = ±2 second harmonic
Question #1: How do we find the Fourier coefficients? First, for simple periodic signals consisting of a few sinusoidal terms Euler's relation (memorize!) 0 no dc component 0 0
For real periodic signals, there are two other commonly used forms for CT Fourier series: Because of the eigenfunction property of e jωt, we will usually use the complex exponential form in 6.003. - A consequence of this is that we need to include terms for both positive and negative frequencies:
Now, the complete answer to Question #1
Ex: Periodic Square Wave DC component is just the average
Convergence of CT Fourier Series How can the Fourier series for the square wave possibly make sense? The key is: What do we mean by One useful notion for engineers: there is no energy in the difference (just need x(t) to have finite energy per period)
Under a different, but reasonable set of conditions (the Dirichlet conditions) Condition 1. x(t) is absolutely integrable over one period, i. e. And Condition 2. of Ex. In a finite time interval, x(t) has a finite number maxima and minima. An example that violates Condition 2. And Condition 3. Ex. In a finite time interval, x(t) has only a finite number of discontinuities. An example that violates Condition 3.
Dirichlet conditions are met for the signals we will encounter in the real world. Then - The Fourier series = x(t) at points where x(t) is continuous - The Fourier series = midpoint at points of discontinuity Still, convergence has some interesting characteristics: - As N, x N (t) exhibits Gibbs phenomenon at points of discontinuity Demo: Fourier Series for CT square wave (Gibbs phenomenon).
Signals and Systems Fall 2003 Lecture #6 23 September 2003 1. CT Fourier series reprise, properties, and examples 2. DT Fourier series 3. DT Fourier series examples and differences with CTFS
CT Fourier Series Pairs Skip it in future for shorthand
Another (important!) example: Periodic Impulse Train All components have: (1) the same amplitude, & (2) the same phase.
Linearity (A few of the) Properties of CT Fourier Series Conjugate Symmetry Time shift Introduces a linear phase shift t o
Example: Shift by half period
Parseval s Relation Energy is the same whether measured in the time-domain or the frequency-domain Multiplication Property
Periodic Convolution x(t), y(t) periodic with period T
Periodic Convolution (continued) Periodic convolution: Integrate over any one period (e.g. -T/2 to T/2)
Periodic Convolution (continued) Facts 1) z(t) is periodic with period T (why?) 2) Doesn t matter what period over which we choose to integrate: 3) Periodic convolution in time Multiplication in frequency!
Fourier Series Representation of DT Periodic Signals x[n] - periodic with fundamental period N, fundamental frequency Only e jω n which are periodic with period N will appear in the FS There are only N distinct signals of this form So we could just use However, it is often useful to allow the choice of N consecutive values of k to be arbitrary.
DT Fourier Series Representation k =<N > = Sum over any N consecutive values of k This is a finite series {a k } - Fourier (series) coefficients Questions: 1) What DT periodic signals have such a representation? 2) How do we find a k?
Answer to Question #1: Any DT periodic signal has a Fourier series representation
Finite geometric series A More Direct Way to Solve for a k
So, from
DT Fourier Series Pair Note: It is convenient to think of a k as being defined for all integers k. So: 1) a k+n = a k Special property of DT Fourier Coefficients. 2) We only use N consecutive values of a k in the synthesis equation. (Since x[n] is periodic, it is specified by N numbers, either in the time or frequency domain)
Example #1: Sum of a pair of sinusoids 0 1/2 1/2 e jπ/4 /2 e -jπ/4 /2 0 0 a -1+16 = a -1 = 1/2 a 2+4 16 = a 2 = e jπ/4 /2
Example #2: DT Square Wave Using n = m - N 1
Example #2: DT Square wave (continued)
Convergence Issues for DT Fourier Series: Not an issue, since all series are finite sums. Properties of DT Fourier Series: Lots, just as with CT Fourier Series Example:
Signals and Systems Fall 2003 Lecture #7 25 September 2003 1. Fourier Series and LTI Systems 2. Frequency Response and Filtering 3. Examples and Demos
The Eigenfunction Property of Complex Exponentials CT: CT "System Function" DT: DT "System Function"
Fourier Series: Periodic Signals and LTI Systems
The Frequency Response of an LTI System CT notation
Frequency Shaping and Filtering By choice of H(jω) (or H(e jω )) as a function of ω, we can shape the frequency composition of the output - Preferential amplification - Selective filtering of some frequencies Example #1: Audio System Adjustable Filter Equalizer Speaker Bass, Mid-range, Treble controls For audio signals, the amplitude is much more important than the phase.
Example #2: Frequency Selective Filters Filter out signals outside of the frequency range of interest Lowpass Filters: Only show amplitude here. low frequency low frequency
Highpass Filters Remember: high frequency high frequency
Bandpass Filters Demo: Filtering effects on audio signals
Idealized Filters CT ω c cutoff frequency DT Note: H = 1 and H = 0 for the ideal filters in the passbands, no need for the phase plot.
Highpass CT DT
Bandpass CT lower cut-off upper cut-off DT
Example #3: DT Averager/Smoother FIR (Finite Impulse Response) filters LPF
Example #4: Nonrecursive DT (FIR) filters Rolls off at lower ω as M+N+1 increases
Example #5: Simple DT Edge Detector DT 2-point differentiator Passes high-frequency components
Demo: DT filters, LP, HP, and BP applied to DJ Industrial average
Example #6: Edge enhancement using DT differentiator Courtesy of Jason Oppenheim. Used with permission. Courtesy of Jason Oppenheim. Used with permission.
Example #7: A Filter Bank
Demo: Apply different filters to two-dimensional image signals. Face of a monkey. HP Image removed do to copyright considerations LP LP BP HP BP Note: To really understand these examples, we need to understand frequency contents of aperiodic signals the Fourier Transform
Signals and Systems Fall 2003 Lecture #8 30 September 2003 1. Derivation of the CT Fourier Transform pair 2. Examples of Fourier Transforms 3. Fourier Transforms of Periodic Signals 4. Properties of the CT Fourier Transform
Fourier s Derivation of the CT Fourier Transform x(t) - an aperiodic signal - view it as the limit of a periodic signal as T For a periodic signal, the harmonic components are spaced ω 0 = 2π/T apart... As T, ω 0 0, and harmonic components are spaced closer and closer in frequency Fourier series Fourier integral
Motivating Example: Square wave increases kept fixed Discrete frequency points become denser in ω as T increases
So, on with the derivation... For simplicity, assume x(t) has a finite duration.
Derivation (continued)
Derivation (continued)
For what kinds of signals can we do this? (1) It works also even if x(t) is infinite duration, but satisfies: a) Finite energy In this case, there is zero energy in the error b) Dirichlet conditions c) By allowing impulses in x(t)or inx(jω), we can represent even more signals E.g. It allows us to consider FT for periodic signals
Example #1 (a) (b)
Example #2: Exponential function Even symmetry Odd symmetry
Example #3: A square pulse in the time-domain Note the inverse relation between the two widths Uncertainty principle Useful facts about CTFT s
Example #4: x(t) = e at 2 A Gaussian, important in probability, optics, etc. (Pulse width in t) (Pulse width in ω) t ω ~ (1/a 1/2 ) (a 1/2 ) = 1 Also a Gaussian! Uncertainty Principle! Cannot make both t and ω arbitrarily small.
CT Fourier Transforms of Periodic Signals periodic in t with frequency ω o All the energy is concentrated in one frequency ω o
Example #4: Line spectrum
Example #5: Sampling function Same function in the frequency-domain! Note: (period in t) T (period in ω) 2π/T Inverse relationship again!
1) Linearity Properties of the CT Fourier Transform 2) Time Shifting FT magnitude unchanged Linear change in FT phase
3) Conjugate Symmetry Properties (continued) Even Odd Even Odd
The Properties Keep on Coming... 4) Time-Scaling a) x(t) real and even b) x(t) real and odd c)
Signals and Systems Fall 2003 Lecture #9 2 October 2003 1. The Convolution Property of the CTFT 2. Frequency Response and LTI Systems Revisited 3. Multiplication Property and Parseval s Relation 4. The DT Fourier Transform
The CT Fourier Transform Pair (Analysis Equation) (Synthesis Equation) Last lecture: Today: some properties further exploration
Convolution Property A consequence of the eigenfunction property: Synthesis equation for y(t)
The Frequency Response Revisited impulse response The frequency response of a CT LTI system is simply the Fourier transform of its impulse response Example #1: frequency response
Example #2: A differentiator Differentiation property: 1) Amplifies high frequencies (enhances sharp edges) Larger at high ω o phase shift
Example #3: Impulse Response of an Ideal Lowpass Filter Questions: 1) Is this a causal system? 2) What is h(0)? No. 3) What is the steady-state value of the step response, i.e. s( )?
Example #4: Cascading filtering operations H(jω)
Example #5: Example #6: Gaussian Gaussian = Gaussian Gaussian Gaussian = Gaussian
Example #2 from last lecture
Example #7:
Example #8: LTI Systems Described by LCCDE s (Linear-constant-coefficient differential equations) Using the Differentiation Property 1) Rational, can use PFE to get h(t) 2) If X(jω) is rational e.g. then Y(jω) is also rational
Parseval s Relation FT is highly symmetric, Multiplication Property We already know that: Then it isn t a surprise that: Convolution in ω A consequence of Duality
Examples of the Multiplication Property For any s(t)...
Example (continued)
The Discrete-Time Fourier Transform
DTFT Derivation (Continued) DTFS synthesis eq. DTFS analysis eq. Define
DTFT Derivation (Home Stretch)