Lecture 4: Convolution and Frequency Domain Filtering Harvey Rhody Chester F. Carlson Center for Imaging Science Rochester Institute of Technology rhody@cis.rit.edu October 7, 005 Abstract The impulse response of a filter describes its spatial processing behavior. This is closely tied to the global processing characteristics that are obtained using Fourier transforms. Here we focus on the relationship between the spatial and frequency domains and provide examples of alternative implementations of filters with various desirable characteristics. DIP Lecture 4
Filtering by Convolution We will first examine the relationship of convolution and filtering by frequency-domain multiplication with D sequences. Let f(n), 0 n L be a data record. Let h(n), 0 n K be the impulse response of a discrete filter. If the sequence f(n) is passed through the discrete filter then the output is g(m) = k k=k h(k)f(m k), < m < where k and k are appropriate limits on the index. DIP Lecture 4
Index Values The values of k and k depend upon m. Getting this dependence right is the detail that causes the most trouble in the implementation of convolution. Condition Restriction h(k) = 0 for k < 0 k 0 f(m k) = 0 for m k L k m L + h(k) = 0 for k K k K f(m k)=0 for m k < 0 k m These restrictions can be combined with the requirement that k k to yield m k k 0 m 0 () K k k m L + m K + L () Hence, g(m) = 0 outside the interval 0 m K + L. DIP Lecture 4
Convolution Sum g(m) = k (m) k=k (m) = 0, elsewhere h(k)f(m k), 0 m K + L where the limits are described by the following tables. Case K < L Range of m k k 0 m K 0 m K m L 0 K L m K + L m L + K Case K L Range of m k k 0 m L 0 m L m K m L + m K m K + L m L + K DIP Lecture 4 3
Periodic Convolution The complexity of the convolution calculation can be reduced by using periodic functions. Let f(n) and h(n) be padded with zeros to length P L + K and then repeated with period P. The resulting periodic functions will be called f P (n) and h P (n). The periodic convolution is defined as g P (m) = k=0 h P (k)f P (m k) The result is periodic with period P. g P (m + jp ) = k=0 h P (k)f P (m + jp k) = k=0 h P (k)f P (m k) DIP Lecture 4 4
Shown at the right are two functions that we would like to process by convolution. f(n) has length L = 6 and h(n) has length K = 5. The functions have been extended by zero-padding to length P = L + K = 0. Example 5 4 3.5 0.5-0.5-4 6 8 Function f(n). 4 6 8 Function h(n). DIP Lecture 4 5
Example The convolution of f P (n) with h P (n) is shown at the right. One period of f P (n) is shown at the top of each column. The cyclic shifts h P (n m) of the filter response are shown in rows -6. 5 4 3 4 6 8.5 0.5 g(0) = 0 4 6 8-0.5 -.5 0.5 g() = 4 6 8-0.5-5 4 3.5 0.5-0.5 -.5 0.5-0.5-4 6 8 4 6 8 4 6 8 g(5) = 5 g(6) =.5.5 The value of g P (m) is found by multiplying h P (m n) with f P (n) and summing. The result for each shift is listed beside the corresponding filter shift. 0.5 g() = 4 4 6 8-0.5 -.5 0.5 g(3) = 8 4 6 8-0.5 -.5 0.5 g(4) = 4 6 8-0.5-0.5-0.5 -.5 0.5-0.5 -.5 0.5-0.5-4 6 8 4 6 8 4 6 8 g(7) = g(8) = 4 g(9) = 5 DIP Lecture 4 6
It is readily demonstrated that Periodic Convolution g P (m) = g(m), 0 m P Thus, we can use circular convolution to compute the a finite convolution. The periodic functions f P (n) and h P (n) have the discrete Fourier transforms F (u) = P f P (n) = u=0 n=0 f P (n)e iπun/p H(u) = P F (u)e iπun/p h P (n) = u=0 n=0 h P (n)e iπun/p H(u)e iπun/p Note the dependence on P, rather than on L or K. Note also that F (u) and H(u) are periodic with period P. DIP Lecture 4 7
Example 5 5 4 3.5 0.5 0 5-0.5 0 0 30 40 50 0 0 30 40 50 0 0 30 40 50 - -5 Function f P (n). Function h P (n). Function g P (n). The functions involved in the periodic convolution of f and h are shown above. The period P = 0 is the minimum length for these functions. The result is a periodic function that is in agreement with g in the interval 0 n K + L. DIP Lecture 4 8
DFT The DFT of g P (m) can be computed by G(u) = P = P m=0 k=0 g P (m)e iπum/p m=0 h P (k)f P (m k)e iπum/p Make the change of index m = k + n and rearrange. G(u) = P = ( P k=0 k=0 h P (k) P k n= k h P (k)e iπuk/p ) f P (n)e iπu(n+k)/p P k n= k f P (n)e iπun/p The terms n = k,..., in the last sum can be replaced by n = P k,..., P because the summand is periodic. Hence... DIP Lecture 4 9
DFT The sum then simplifies to the product G(u) = P ( P k=0 = P H(u)F (u) ) ( h P (k)e iπuk/p P n=0 f P (n)e iπun/p ). By construction, f P (n), h P (n) and g P (n) are all periodic with period P.. It is required that P L + K. 3. F (u), H(u) and G(u) are all complex periodic functions with period P. 4. F (u) can be computed by padding f(n) with zeros to length P and then doing a DFT. 5. A similar statement is true for H(u) and G(u). 6. u takes on integer values. DIP Lecture 4 0
Example Function F (u). Function H(u). Function G(u). The transforms F (u), H(u) and G(u) can be displayed in the complex plane. The points are numbered with the value of the index u, 0 u P. Because the functions are periodic, they follow the same cycle repetitively. DIP Lecture 4
Power Relationships The mean-squared value of f(n) over the interval of length P is P n=0 f P (n)f P (n) = P = u=0 n=0 v=0 u=0 v=0 F (u)f (v)e iπ(u v)n/p ( F (u)f (v) P n=0 e iπ(u v)n/p ) The term in parentheses is δ u,v Hence, P n=0 f P (n) = u=0 F (u). Because f(n) = f P (n) in 0 n P, the above statement is also valid for f(n).. An equivalent statement is true for h P (n) and g P (n). DIP Lecture 4
Example The power calculations for the waveforms and their transforms are shown in the table below. In each case, the same result is obtained in both domains. P u=0 n=0 f P (n) = 5.5 F (u) = 5.5 P u=0 n=0 h P (n) = 0.7 H(u) = 0.7 P u=0 n=0 g P (n) = 63.9 G(u) = 63.9 DIP Lecture 4 3
Filtering of D Functions Let f(x, y) be a L L array that represents the brightness values of an image. Let h(x, y) be a K K array that represents the impulse response of a spatial filter. The result of filtering f with h is the array g(x, y). By analogy with the D example, we find that g(x, y) has size (L + K, L + K ). The convolution operation can be most conviently written in terms of functions that are periodic with period (P, P ), where P L + K P L + K DIP Lecture 4 4
Periodic Convolution The periodic convolution is defined as g P (x, y) = j=0 k=0 h P (j, k)f P (x j, y k) The result is periodic with period P, P. g P (x + ap, y + bp ) = j=0 k=0 h P (j, k)f P (x + ap j, y + bp k) = j=0 k=0 h P (j, k)f P (x j, y k) = g P (x, y) DIP Lecture 4 5
Periodic Convolution. Assume that f(x, y) is given only for 0 x L and 0 y L.. Assume that h(x, y) is given only for 0 x K and 0 y K. 3. Without loss of generality, we can assume any extension we wish outside the region of definition. 4. The convolution g = f h has size (L + K, L + K ). 5. We can construct periodic functions f P (x, y) and h P (x, y) that are defined on the primary frame (P L + K, P L + K ) by padding with zeros. In the examples we will use P = L + K, P = L + K 6. Periodic convolution of f P (x, y) with h P (x, y) produces a periodic function g P (x, y) that is equal to g(x, y) for 0 x P, 0 y P. DIP Lecture 4 6
Periodic Convolution 7. Periodic convolution is equivalent to multiplication of the transforms of the periodic spatial functions. 8. The discrete Fourier transforms are periodic with period (P, P ). F (u, v) = P P x=0 y=0 f P (x, y)e iπ xu P + vy P = P P L x=0 L y=0 f(x, y)e iπ xu P + vy P Similar expressions hold for the other variables. DIP Lecture 4 7
Algorithm. Read the image array f(x, y) and determine the dimensions (L, L ).. Define the filter h(x, y) with dimensions (K, K ). 3. Construct zero-padded arrays f P (x, y) and h P (x, y) with dimensions (P, P ). 4. Compute F (u, v) = FFT(f P (x, y)) and H(u, v) = FFT(h P (x, y)). 5. Compute G(u, v) = P P F (u, v)h(u, v) 6. Compute g P (x, y) = FFT (G(u, v)). DIP Lecture 4 8
Algorithm Example ;STEP : DEFINE THE IMAGE ARRAY WITH SIZE 0X0. N=0 & M=0 x=(findgen(n)-n/)#replicate(,m) y=(findgen(m)-m/)##replicate(,n) fxy=exp(-(xˆ/5+yˆ/5)) ;STEP : DEFINE THE FILTER ARRAY WITH SIZE 3X3. hxy=[[-,0,-],[0,4,0],[-,0,-]] ;STEP 3: PAD THE ARRAYS TO (,). hp=zeropad(hxy,,) fp=zeropad(fxy,,) ;STEP 4: DO THE FFT s ff=fft(fp) hf=fft(hp) ;STEP 5: COMPUTE G(u,v) gf=hf*ff** ;STEP 6: COMPUTE gp gp=float(fft(gf,/inverse)) DIP Lecture 4 9
Example Function f(x, y). Function h P (x, y). Function g P (x, y). Function F (u, v). Function H(u, v). Function G(u, v). DIP Lecture 4 0
Zeropad Function zeropad,a,n,m,center=center ;+ ;B=zeroPad(A,N,M) returns an array B of size NxM such that B[i,j]=A[i,j] ;where A is defined and B[i,j]=0 where A[i,j] is not defined. B is the same ;type as A. ; ;If A is a scalar or a row vector then zeropad can be called with a single ;dimension argument. B=zeroPad(A,N) returns a row vector of length N. ; ;B=zeropad(A) returns an array that has a single ring of zeros around A. ; ;It is an error if N or M is smaller than the corresponding dimension of A. ; ;KEYWORDS ; ;CENTER: Set the keyword center if the array should be centered in the ;field of zeros. ; ;HISTORY ;Written by H. Rhody September, 999 ;Modified to include additional error checks and CENTER keyword ;January 00. ;Modified to handle B=zeropad(A) case. 9.3.005 HR ;- DIP Lecture 4
Zeropad Examples IDL> print,zeropad(4) 0 0 0 0 4 0 0 0 0 IDL> print,zeropad([,,3]) 0 0 0 0 0 0 3 0 0 0 0 0 0 IDL> print,zeropad([,,3],7) 3 0 0 0 0 IDL> print,zeropad([,,3],7,/center) 0 0 3 0 0 IDL> print,zeropad(indgen(3,4),7,8,/center) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 4 5 0 0 0 0 6 7 8 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 DIP Lecture 4
Comparison with Convolution. Spatial filtering can be carried out by convolution.. IDL contains a fast convolution routine, CONVOL. 3. Care must be taken to do mathematical convolution with CONVOL. Steps to do convolution of arrays A and B with CONVOL.. Determine the dimensions (L, L ) and (K, K ) of A and B.. Construct A p by zero-padding A to size (L + K, L + K ). Ap=zeropad(A,L+K-,L+K-) 3. Compute CP=CONVOL(Ap,B,CENTER=0,/EDGE ZERO). This procedure replaces a more complicated method that was required before the EDGE ZERO keyword was introduced in IDL 6.. The old method is used in the CONVOLVE program below, which makes it compatible with older versions of IDL. DIP Lecture 4 3
CONVOLVE Program FUNCTION CONVOLVE,A,M,SCALE,_EXTRA=extra ;+ ;B=CONVOLVE(A,M,S) returns an array B that is the D ;convolution of the arrays A and M. The convolution calculation ;is actually carried out by the IDL CONVOL routine. ; ;An array AP is constructed by padding A, then ;C=CONVOL(AP,M,S,CENTER=0,/EDGE_TRUNCATE) is computed ;and then C is trimmed to provide the proper output. ; ;The size of the returned array is equal to ;Ac+Mc- by Ar+Mr-, where Ac x Ar is the size of ;A and Mc x Mr is the size of M. ; ;If S is provided then the results are scaled by ;passing S to the CONVOL routine. Otherwise, the ;scale value S= is used. ; ;CONVOL keywords are passed via _EXTRA. ; ;H. Rhody ;September, 00 ;- DIP Lecture 4 4
CONVOLVE (cont) IF N_PARAMS() LT THEN MESSAGE, Incorrect number of parameters. SA=Size(A) SM=Size(M) typea=sa[sa[0]+] typem=sm[sm[0]+] IF N_PARAMS() LT 3 THEN SCALE= ;Determine the width and height parameters. The case of a D array ;has to be handled differently. CASE SA[0] OF : BEGIN WA=SA[] & HA= END : BEGIN WA=SA[] & HA=SA[] END ENDCASE CASE SM[0] OF : BEGIN WM=SM[] & HM= END : BEGIN WM=SM[] & HM=SM[] END ENDCASE DIP Lecture 4 5
CONVOLVE (cont) ;Construct a zero-padded array to use in the convolution operation SAP=[,WA+*(WM-),HA+*(HM-),MAX([typeA,typeM]),(WA+*(WM-))*(HA+*(HM-))] AP=Make_Array(Value=0,Size=SAP) AP[WM-:WM+WA-,HM-:HM+HA-]=A ;Do the convolution of AP with M AC=CONVOL(AP,M,SCALE,CENTER=0,/EDGE_TRUNCATE,_EXTRA=extra) ;Trim and return the convol result. CSTART=WM- RSTART=HM- RETURN,AC[CSTART:CSTART+WM+WA-,RSTART:RSTART+HM+HA-] END DIP Lecture 4 6
Comparison of Transform and Convolution Methods Output Difference Shown above is the output array and the magnitude of the difference between the transform and convolution methods. All of the differences are less than 3 0 7. DIP Lecture 4 7
Power Calculations The mean-squared value can be computed in either the spatial or frequency domain. P P x=0 y=0 For the preceding example we have f P (x, y) = u=0 v=0 F (u, v) x=0 y=0 f P (x, y) = 0.08 u=0 v=0 F (u, v) = 0.08 DIP Lecture 4 8
Power Calculations (cont) The output image has the frequency-domain relationship G(u, v) = P P F (u, v)h(u, v) Hence, the power can be computed by P P x=0 y=0 g(x, y) = u=0 v=0 G(u, v) = (P P ) u=0 v=0 F (u, v) H(u, v) =.047 for the example above DIP Lecture 4 9
Image Filtering Example Image arrays can be processed by either spatial or transform filtering. Just substitute the image array for f(x, y) in the above techniques. You may need to rescale the amplitude of the final array to fit the range [0,55] for display. Original Image Transform Filtering Convolution Filtering DIP Lecture 4 30