ECE 425. Image Science and Engineering

Size: px

Start display at page:

Download "ECE 425. Image Science and Engineering"

Darcy Hamilton
5 years ago
Views:

1 ECE 425 Image Science and Engineering Spring Semester 2000 Course Notes Robert A. Schowengerdt (520) (voice), (520) (fa) ECE402

2 DEFINITIONS 2

3 Image science The theory of optical image formation and detection Includes elements of: optics radiometry linear systems statistics vision 3

4 Image engineering The technologies of image acquisition, transmission, storage and display Includes elements of: detectors signal processing data compression image processing 4

5 OVERVIEW Electronic imaging systems pervade modern life Eamples Document processing (scanning, storage, printing, digital libraries, WWW) Consumer products (HDTV, digital cameras, photo scanners and printers) Machine vision (quality control inspection, robotics) Medical imaging (disease diagnosis, medication monitoring) Scientific visualization (comple mathematical models, interactive graphics) 5

6 Remote sensing (earth science, environmental monitoring, weather) Military (reconnaisance, surveillance, targeting) 6

7 THE SYSTEMS APPROACH A perceived image is the result of a chain of systems: optics detector coding/decoding display human vision Each can be considered a subsystem of the total electronic imaging system The engineering design goal is to optimize the 7

8 performance of each subsystem in relation to that of the others and the total system This course covers the tools used for imaging system analysis, design and evaluation 8

9 An imaging system consists of several subsystems light source scene image acquisition subsystem transmission subsystem display subsystem* human vision subsystem optics detector* coder electronics decoder optics neural network retina* brain * points of signal transduction, optical < > electronic 9

10 For optical components, we re concerned with: size and location of the image (geometrical optics) intensity of the image (radiometry) contrast and sharpness of the image (linear systems) 0

11 For electronic components, we re concerned with: image sampling and quantization (analog filters, A/D converters, coding) image processing (digital signal processing)

12 SECTION I MATHEMATICAL TOOLS Mathematics Background Convolution and Fourier Transforms Linear Filtering and Sampling Two-dimensional Functions and Operations Discrete Fourier Transform and Fast Fourier Transform 2

13 MATHEMATICS BACKGROUND 3

14 Comple Notation Comple arithmetic will be necessary for Fourier analysis and optics Comple numbers consist of two real numbers, joined by a phasor relationship where c is a comple number, a is the real part of c c = a+ jb b is the imaginary part of c j is Phasor relationship 4

15 imaginary part The amplitude A of c A = a 2 + b 2 The phase θ of c θ = atan( b a) b A θ a c real part Can write c as c = Ae jθ which, by Euler s Theorem, c = A( cosθ + jsinθ) = A a Ā -- j b + Ā -- = a+ jb 5

16 Using Euler s Theorem, we can derive the following relations: sin( θ) cos( θ) = = ---- ( e jθ e jθ ) 2 j -- ( e jθ + e jθ ) 2? Using Euler s Theorem, show that ( sinθ) 2 + ( cosθ) 2 = 6

17 Simple Functions Delta function and its relatives delta δ( 0 ) 0 = 0 0 NOTE: The delta function s amplitude is infinite and its area is. The amplitude is shown as for convenience in plots.? Write the equation that defines the area of a delta function as.? Review the definition of delta function in terms of the limit of conventional functions, such as the rectangle function even delta pair δδ = b [ δ( b 0 + b) + δ( 0 b) ] 7

18 0 = b b - b b 0 - b 0 + b odd delta pair δ 0 δ = b [ δ( b 0 + b) δ( 0 b) ] 0 = 0 b 0 0 b - b b 0 - b 0 + b - b - b 8

19 comb (shah) comb = b δ b ( 0 nb) n = 0 = 0 b - 2b - b 0 b 2b b 0-2b 0 -b b 0 +2b Even delta pair, odd delta pair and comb functions are all scaled by b 9

20 Use of the δ function sifting f ( α)δ α 0 = f( 0 ) = constant NOTE: Sifting is a convolution, evaluated for a particular shift Finds the value of a function at a specific value of the independent variable (similar to a look-up table) sampling f( )δ( 0 ) = f( 0 )δ( 0 ) 20

21 f() = f( 0 ) NOTE: Sampling is a mulitplication Output is a delta function, with area determined by the value of the function at the specified value of the independent variable. uniform sampling f( )comb = f b b ( 0 + nb)δ( 0 nb) n = 2

22 ... / b b 0 -b b 0 +2b b 0 -b b 0 +2b NOTE: Must divide comb function by b to retain amplitude of f(). NOTE: f() modulates the comb function. shifting g ( ) = f( ) δ( 0 ) = f ( α)δ 0 α = f( 0 ) f() = g() 0 0 replicating g ( ) = f b ( ) comb b 22

23 g() f()... / b... = b 0 -b b 0-2b 0 -b b NOTE: Must divide by b to retain amplitude of f() Other -D functions rectangle (square pulse) rect -- b = 0 b > 2 2 b = 2 b < 2 triangle tri -- b = 0 b b b < 23

24 .2 f() rect tri For a given b, the tri function is twice as wide as the rect function? What is the value of b in the above graph? sinc sinc( b ) = sin[ π b] π b sinc-squared sinc 2 ( b) 24

25 sinc sinc squared 0.6 f() ? What is the value of b in the above graph? gaus(sian) gaus( b) = e π b ( )2 25

26 .2 gaus f() ? What is the value of b in the above graph? cosine cos( 2π b) = e j2π( b) + e 2π j ( b) sine sin( 2π b) = e j2π( b) e 2π j ( b) j 26

27 .2 cos sin f() ? What is the value of b in the above graph? 27

28 CONVOLUTION AND FOURIER TRANSFORMS (-D) Convolution (-D) Why is it important? Describes the effect of a Linear Shift Invariant (LSI) system on input signals input signal f() system (operator L) output signal g() L is the system operator Can write a general description of any system as, g ( ) = L[ f( ) ] For an LSI system, L is a convolution, g ( ) = f input signal and the system impulse response, h() ( ) h( ), of the 28

29 More on this later Mathematical and graphical description g ( ) = f( ) h( ) = f ( α)h ( α) dα Eample 2 f() or f(α) h() or h(α) 0 3 or α or α 29

30 Convolution. write both as a function of α 2. flip h (or f) about α = 0 f(α) and h(α) h(-α) 3. shift h (or f) by an amount h( - α) 4. multiply the two functions 5. integrate the product function over all α f(α)h( -α) g() 6. repeat steps 3 through 5 until done 30

31 shift h(-α) multiply f(α)h(0 - α) integrate area = g(0) α α = 0 f(α)h( - α) h( -α) area = g() α α = f(α)h(2 - α) h(2 -α) area = g(2) α α = 2 f(α)h(3 - α) h(3 -α) area = g(3) = 3 h(4 -α) α f(α)h(4 - α) α = 4 α α area = g(4) 3

32 Plot g() g() The shifts in this eample are by integer steps, for illustration convenience 32

33 -D CONVOLUTION PROPERTIES property commutative distributive = f( ) h( ) = h ( ) f( ) f( ) [ h ( ) + h 2 ( ) ] f( ) h ( ) + f( ) h 2 ( ) associative = f( ) [ h ( ) h 2 ( ) ] [ f( ) h ( ) ] h 2 ( ) 33

34 -D CONVOLUTION EXAMPLES f() h() g() f() δ() f() f(- 0 ) h() g(- 0 ) f() h(- 0 ) g(- 0 ) rect() rect() tri() sinc() sinc() sinc() gaus() gaus() gaus

35 Fourier Transforms (-D) Why is it important? For an LSI system, the convolution operator becomes a multiplicative operator in the Fourier domain Taking the Fourier transfom of the system equation, 35

36 Gu ( ) = Fu ( )H( u) where G(u) is the spectrum of the output signal, F(u) is the spectrum of the input signal, and H(u) is the system transfer function In many cases, it is easier to analyze an LSI system in the Fourier domain Forward transform Fu ( ) = f( )e j2πu d Inverse transform f( ) = Fu ( )e j2πu du 36

37 Properties for special functions f() and F(u) are in general, comple functions If f() real F(u) = F*(-u) Hermitian: Re[F(u)] even, Im[F(u)] odd If f() real and even Im[F(u)] = 0 (F(u) is real) Forward transform is the analysis of f() into its spectrum F(u) of sines and cosines at different frequencies, in general, each with a different amplitude and phase. Inverse transform is the synthesis of f() from F(u). 37

38 -D FOURIER TRANSFORM PAIRS f() F(u) δ(u) δ() cos( 2πu 0 ) δδ u u u δδ cos( 2πu 0 ) 0 sin( 2πu 0 ) rect() sinc() comb() gaus() tri() j u δδ u u sinc(u) rect(u) comb(u) gaus(u) sinc 2 (u) 38

39 -D FOURIER TRANSFORM PROPERTIES name f() F(u) f(±) F(±) F(±u) scaling f(/b) b F(bu) f ( + u) shifting f( ± 0 ) ± e j2πu 0 f( ) ± e j2π 0u Fu ( ) Fu ( + u 0 ) derivative linearity convolution f ( k) ( ) ( j2πu) k Fu ( ) ( j2π) k f( ) F ( k) ( u) a f ( ) + a 2 f 2 ( ) a F ( u) + a 2 F 2 ( u) f ( ) f 2 ( ) F ( u)f 2 ( u) f ( )f 2 ( ) F ( u) F 2 ( u) 39

40 -D FOURIER TRANSFORM PROPERTIES name f() F(u) correlation f ( ) f 2 ( ) F ( u)f 2 ( u) f ( )f 2 ( ) F ( u) F 2 ( u) 40

41 LINEAR, SHIFT-INVARIANT (LSI) SYSTEMS (-D) Linear: output of a sum of inputs is equal to the sum of the individual outputs g ( ) = af ( ) h( ) g 2 ( ) = bf 2 ( ) h( ) g ( ) = [ af ( ) + bf 2 ( ) ] h( ) = = ( af ( ) h( ) + bf 2 ( ) h( ) ) g ( ) + g 2 ( ) shift-invariant: system response does not change over space Relation between input f() and output g() of an LSI system g ( ) = f( ) h( ) where h() is the system impulse response 4

42 Fourier transform of LSI system equation Gu ( ) = Fu ( )H( u) where H(u) is the system transfer function H(u) is a comple filter that modifies the spectrum F(u) of f() In optics, the amplitude of H(u) is called the Modulation Transfer Function (MTF) The properties of comple functions give, for an LSI system, ampl[ G( u) ] = ampl[ F( u) ]ampl[ H( u) ] phase[ G( u) ] = phase[ F( u) ] + phase[ H( u) ] 42

43 N cascaded LSI systems -D CASCADED SYSTEMS f() h ()... h N () g() g ( ) = {[ f( ) h ( ) ] h 2 ( ) } h N ( ) 43

44 Single system equivalent f() h net () g() h net ( ) = h ( ) h 2 ( ) h N ( ) where h net is the net system impulse response 44

45 FOURIER TRANSFORM EXAMPLES E. Find sinc(/2) sinc(/3) Convolution in this case is very difficult! Take the Fourier transform 2rect( 2u) 3rect( 3u) = 6rect( 3u) = -/4 /4 u -/6 /6 u -/6 /6 u Take the inverse Fourier transform 6 --sinc(/3) 3 = 2sinc(/3) 45

46 E 2. Find spectrum of square wave with a DC bias f() Write square wave as convolution f( ) = --rect comb 5 -- Take the Fourier transform Fu ( ) = = sinc( 2u) comb( 5u) 5 2sinc( 2u)comb( 5u) spectrum is comb function, modulated by sinc function sampled at frequency interval u = /5, i.e /period zeros at u = n/2, n = ±, ±2,... 46

47 if square wave period P = 2 pulse width, we have the classic square wave spectrum at u = 0, ±/P, ±3/P, ±5/P,...? Verify the above statement for an arbitrary period P sinc(2u) and comb(5u) /5-7/5-6/5-5/5-4/5-3/5-2/5-/5 /5 /5 2/5 3/5 4/5 5/5 6/5 7/5 8/5 u /2 - -/2 0 /2 3/2 47

48 sinc(2u) times comb(5u).2 2/5 0.8 comb, /5-7/5-6/5-5/5-4/5-3/5-2/5-/5 /5 2/5 3/5 4/5 5/5 6/5 7/5 8/5 modulated by sinc function /2 - -/2 0 /2 3/2 48

49 SCALING PROPERTY OF FOURIER TRANSFORMS 49

50 Width of F(u) is inversely proportional to width of f() rect() F sinc(u) -/2 /2 u rect(/2) 2sinc(2u) 2 - u rect(/3) 3sinc(3u) 3-3/2 3/2 u 50

51 δδ(2) -/2 /2 /2 F cos(πu) u δδ() 2cos(2πu) - u δδ(2/3) 3/2 3cos(3πu) -3/2 3/2 u 5

52 cos(2π) F δδ(u)/2 /2 - u cos(4π) δδ(u/2)/4 /2-2 2 u cos(6π) δδ(u/3)/6 /2-3 3 u 52

53 SUPERPOSITION PROPERTY OF FOURIER TRANSFORMS Fourier transform of sum of functions equals sum of their individual Fourier transforms + cos(2π) δ(u)+δδ(u)/2 F /2 - u cos(2π)+cos(4π) δδ(u)/2+δδ(u/2)/4 / u 53

54 SYSTEM ANALYSIS WITH THE FOURIER TRANSFORM Variety of applications LSI system equation: g ( ) = f( ) h( ) Application Given Find Spatial Domain Fourier Domain system output f(), h() g() g ( ) = f( ) h( ) Gu ( ) = Fu ( )H( u) g ( ) = F [ Gu ( )] system identification inversion f(), g() h(), g() h() NA H( u) = Gu ( ) Fu ( ) h ( ) = F [ H( u) ] ill-conditioned f() NA Fu ( ) = Gu ( ) H( u) f( ) = F [ Fu ( )] ill-conditioned 54

55 SAMPLING (-D) Two components to digitizing an analog signal Sample analog signal at discrete values of Quantize the sampled signal amplitude to digital values Sampling is either ideal (delta function samples) or non-ideal (time-integrated samples) 55

56 Sampled function f s () IDEAL SAMPLING b f() - b 0 b 2b... = b f s () - b 0 b 2b... Mathematically f s ( ) = f( )comb b b -- Sample interval = b, sample rate = /b 56

57 Fourier domain description Analog signal f( ) Fu ( ) F(u) 0 u -u B u B band limit of analog signal = ± u B (highest frequency component in signal) bandwidth of analog signal = 2u B Sampled signal f( )comb b b -- f s ( ) F s ( u) Fu ( ) comb( bu) 57

58 ... F(u-2u 0 ) F(u+u 0 ) F(u) F(u-u 0 ) F(u-2u 0 )... -2/b -/b 0 /b 2/b u -u f u f -u B -/b+u B /b-u B u B F s (u) -2/b -/b /b 2/b u sampling frequency (rate) = u s = /b folding frequency = u f = /2b 58

59 Aliasing occurs where the individual spectra overlap... F(u-2u 0 ) F(u+u 0 ) F(u) F(u-u 0 ) F(u-2u 0 )... -2/b -/b 0 /b 2/b u -u f u f -u B -/b+u B /b-u B u B Frequency components above u f appear to be lower frequency components below u f in the sampled signal Once aliasing occurs, it cannot be removed Sometimes, analog signal is pre-filtered (before sampling) by a low-pass filter so that the analog signal has a lower bandwidth and u B u f 59

60 Can avoid aliasing if: original analog signal is band-limited (u B is finite) and sample rate satisfies the following condition u s 2u B -- 2u b B or, the sample interval satisfies the condition, b u B 60

1-D MATH REVIEW CONTINUOUS 1-D FUNCTIONS. Kronecker delta function and its relatives. x 0 = 0

1-D MATH REVIEW CONTINUOUS 1-D FUNCTIONS. Kronecker delta function and its relatives. x 0 = 0 -D MATH REVIEW CONTINUOUS -D FUNCTIONS Kronecker delta function and its relatives delta function δ ( 0 ) 0 = 0 NOTE: The delta function s amplitude is infinite and its area is. The amplitude is shown as