Image Acquisition and Sampling Theory

Electromagnetic Spectrum The wavelength required to see an object must be the same size of smaller than the object 2

Image Sensors 3

Sensor Strips 4

Digital Image Acquisition Process 5

Digital Image Representation An image is a function defined on a 2D coordinate f(x,y). The value of f(x,y) is the intensity. 3 such functions can be defined for a color image, each represents one color component A digital image can be represented as a matrix. 6

Range Image acquisition 7

Image File Formats ultiple image formats are available. Some are compressed and some are raw format Examples:.jpg,.j2k,.gif,.bmp,.png,.ppm,.tiff, etc. For most image processing operations, an image file will need to be converted into raw data format Not all images may be represented in all formats. 8

Sampling Theory How accurate a digitized image is with respect to the original analog image? Shannon s sampling theory provides a classical answer. In short, it states that If the analog signal is band-limited with bandwidth f o /2 where f o = /T, then the analog signal can be reconstructed perfectly provided that The sampling frequency exceeds the Nyquist rate which is twice the bandwidth f 0 /2 and The amplitudes of the samples are not quantized. The reconstruction is accomplished via a kernel interpolation formula and can only be accomplished in ideal situation. 9

Sampling Theorem (D case) Let x a (t), to be a continuous time signal, sampled at a rate of one sample per T seconds, yielding a sampled signal x ( t) x ( t) t nt x nt t nt s a a n n The sequence {x(n); x(n) = x a (nt)} is a discrete time sequence obtained from the coefficients of x s (t). The Fourier transform of x s (t), X s (f) is the convolution of that of x a (t) and of the pulse train (comb filter): F xs t F xa t F t nt n ( ) ( ) m m X a f * f X a f T m T T m T 0

n where Also, t nt d e m Side Note j2 mt / T T /2 j2 mt / T dm t nt e dt T tt/2n j2 mt / T j 2 mt / T j 2 ft F e e e dt f : Fourier Series Representation T Thus, F t nt F e n T m m m T t j2 mt / T F e f j2 mt / T m T m T m T

Spectrum Interpretation X a (f) f o /2 0 f o /2 f X s (f) f o /2 0 f o /2 f When a band-limited signal is sampled at twice of its bandwidth, the sampled spectrum is a periodic extension of original spectrum with period equal to sampling frequency 2

Reconstruction X s (f) f o /2 0 f o /2 ** X f X f P f x t x t F P f a s f a s f where P f o f if fo / 2 f fo / 2; 0 otherwise. sin 2 t/ T t F P f e df sinc T 2 t / T T T fo /2 j2 ft f o f /2 o o t Thus, xa t x( n) t nt * sinc n T T t nt t xn ( )sinc x( n)sinc n T n T T n T o f 3

Z-Transform The z-transform is the discrete time version of Laplace transform. Given a sequence {x(n)}, its z-transform is: In particular, ( ) ( ) X ( z) x( n) x( n) z n n n x n x n z n n n n n n x( n) z x( n) z X ( z) n n (c) 2003-202 by Yu Hen Hu 4

Z-Transform and Fourier Transform Discrete time Fourier transform (DTFT): ( j jn X e ) X ( j) x( n) e X ( z) j n Discrete Fourier Transform: n0 ze N j2 kn / N j X ( k) x( n) e X ( z) j 2 kn / N X ( e ) ze N 2 k/ N (c) 2003-202 by Yu Hen Hu 5

Frequency Domain Representation Discrete time Fourier transform (DTFT) X ( e j ) X ( z ) x j ( n ) e ze n jn Im{z} Discrete Fourier Transform (DFT) Re{z} j 2 k / N N X ( k ) X ( z ) x ( n ) e ze n0 j2 kn / N DFT of a length N sequence can be evaluated at N points on the unitcircle on the Z-plane Z-plane (c) 2003-202 by Yu Hen Hu 6

Sub-sequence of a Finite Sequence Let x(n) = x(0) x() x(2) x(3) x(4) x(5) x(6) x 0 (n) = x(0) 0 x(2) 0 x(4) 0 x(6) x (n) = 0 x() 0 x(3) 0 x(5) 0 Then, clearly, x 0 (n) + x (n) = x(n), and x 0 (n) = [x(n) + ( ) n x(n)]/2, x (n) = [x(n) ( ) n x(n)]/2 Denote X(z) to be the Z-transform of x(n), then n x( n) x( n) X0( z) X ( z) X ( z) 2 2 n x( n) x( n) X( z) X ( z) X ( z) 2 2 (c) 2003-202 by Yu Hen Hu 7

Z transform of a Sub-sequence Define Let W =exp(j2/), then One may write ( ) 0 0 0 0 ( ) ( ) ( ) ( ) m n k k m mk mn m mk mn n m n mk m m X z W x n W W x n W W x n z W X zw. 0, 0 ) ( otherwise k n W m k n m., 0,,. 0 ) ( ) ( k otherwise k n n x n x k 0 ) ( ) ( ) ( m k n m k W n x n x (c) 2003-202 by Yu Hen Hu 8

Decimation (down-sample) -fold decimator y k (n) = x(n+k) = x k (n+k), 0 k - nk n ( k) / k ( ) k ( ) k ( ) n Y z x n k z x z k/ k / / z mk / m k m0 z x ( ) z W X ( z W ) Example. = 2. y 0 (n) = x(2n), y (n) = x(2n+), / 2 m / 2 / 2 0 2 2 m0 2 Y ( z) X ( z W ) X ( z ) X ( z ) / 2 / 2 z m m z 2 2 2 m0 2 / 2 / 2 Y ( z) W X ( zw ) X ( z ) X ( z ) (c) 2003-202 by Yu Hen Hu 9

Interpolation (up-sample) L-fold Expander z L ( n) x( n/ L) n/ L :integer, 0 Otherwise. n L L L n/ L Z ( z) z ( n) z ( n) z x( n / L) z L L L n m m x( m) z X z n Example. L = 2. {z L (n)} ={x(0), 0, x(), 0, x(2), 0, } and z 2 2 ( n) X z (c) 2003-202 by Yu Hen Hu 20

Up-sampling Up-sampling from samples/sec to L samples/sec: Insert L- 0s between successive samples of original sequence. The length of the sequence becomes L times of original. Interpolate the augmented sequence by passing it through a low pass filter or other types of interpolation filter 4 2 0 8 4 0 2 4 4 8 = 2 22

Frequency domain Interpretation For = 2, with decimation j j / 2 j / 2 Y0 ( e ) X ( e ) X ( e ) 2 Note that j /2 j e j / 2 j / 2 Y ( e ) X ( e ) X ( e ) 2 j j2 j j2 0 X ( e ) Y ( e ) e Y ( e ) For L = 2, with interpolation, j j2 Z2 ( e ) X ( e ) In general, -fold down-samples will stretch the spectrum -times followed by a weighted sum. This may cause the aliasing effect. jk / j e mk j/ m Yk ( e ) W X ( e W ) m0 k j2 m / j2 m / e X ( e ) m0 L-fold up-sample will compress the spectrum L times j Z ( e ) X e L jl (c) 2003-202 by Yu Hen Hu 23

Spatial and Gray Level Resolution When an analog picture is scanned into computer, it is digitized in two ways: Spatially, image is divided into small pixels The value of each pixel is represented with a quantized number with finite number of digits Spatial resolution: # of samples per unit length or area DPI, PPI: dots (pixels) per inch: size of an individual pixel If pixel size is kept constant, the size of an image will affect spatial resolution Gray level resolution: Number of bits per pixel 8 bits (usual gray scale) to bit (binary image) Color image has 3 image planes to yield 8 x 3 = 24 bits/pixel Too few levels may cause false contour 24

Same Pixel Size, different Image Sizes 25

Same Image Size, Different Pixel Sizes Pixel size is dependent on physical display 26

Varying Gray Level Resolution 27

Image Resizing Enlargement interpolation, Reduction sub-sampling Needed when the same image must be displayed on different displays (cell phone, tablet, TV screen, etc.) Typical Screen sizes: Smart phone: 960 x 640 HDTV: 920 x 080, (080p, 6:9 aspect ratio) HDTV: 280 x 720, (720p) Desk top monitors: Challenges: Non-power of two sizing ratios Aspect ratio changes (4:3 to and from 6:9, or arbitrary) Loss of quality after resizing 28

Image Resizing How to convert a 080p frame into a 720p frame? Vertical: 080 720, horizontal: 920 280 Answer: V: 080 260 720, 920 3840 280 Converting sampling rate N can be accomplished by Upsample LC(, N) Least common multiplier of, N Downsample LC(, N) N 29