Image preprocessing in spatial domain

Transcription

1 Image preprocessing in spatial domain Sampling theorem, aliasing, interpolation, geometrical transformations Revision: 1.4, dated: May 25, 2006 Tomáš Svoboda Czech Technical University, Faculty of Electrical Engineering Center for Machine Perception, Prague, Czech Republic

2 Reminder Convolution theorem The Fourier transform of a convolution is the product of the Fourier transforms. F{f(x, y) h(x, y)} = F (u, v)h(u, v) The Fourier transform of a product is the convolution of the Fourier transforms. F{f(x, y)h(x, y)} = F (u, v) H(u, v) 2/90

3 Impulse (delta) function 3/90 δ(x) = H(x) x where H(x) is the Heaviside step function 1 Sifting property scaling property 1 f(x)δ(x a)dx = f(a) δ(ax) = a δ(x) 1 Eric W. Weisstein. Heaviside Step Function. From MathWorld A Wolfram Web Resource.

4 Replication of delta function Shah function 4/90 III(x) = k= δ(x k)

5 Replication of delta function Shah function 4/90 III(x) = k= δ(x k) 2 2 Eric W. Weisstein. Shah Function. From MathWorld A Wolfram Web Resource.

6 2D Shah function bed and nails 5/90 y x III(x, y) = k= l= δ(x k, y l)

7 For unit intervals 2D Shah function properties 6/90 III(x, y) = k= l= δ(x k, y l)

8 For unit intervals 2D Shah function properties 6/90 III(x, y) = δ(x k, y l) k= l= If we need samples spaced X and Y III ( x X, y Y ) = XY δ(x kx, y ly ) k= l= because of the scaling property of the delta function δ(ax) = 1 a δ(x)

9 2D Shah function Fourier pair 7/90 F{III ( x X, y Y ) } = XY III (Xu, Y v) The Shah function is its own transform but the frequency inverses! This will have important consequences!

10 Sampling by using the Shah function Product of a function with the Shah function. 8/90 s(x, y) = III(x, y)f(x, y) We know from convolution theorem that S(u, v) = III(u, v) F (u, v)

11 Sampling theorem 9/ Image taken from [1]

12 Sampling theorem Aliasing 10/90

13 Sampling theorem Aliasing 11/90

14 Sampling theorem Aliasing 12/90

15 Sampling theorem Aliasing 13/90

16 From continuous to discrete and back again 14/ x x 10 4

17 From continuous to discrete and back again 15/90

18 From continuous to discrete and back again 16/90

19 From continuous to discrete and back again 17/90

20 From continuous to discrete and back again 18/90

21 From continuous to discrete and back again 19/90

22 From continuous to discrete and back again 20/90

23 From continuous to discrete and back again 21/90

24 From continuous to discrete and back again 22/90

25 From continuous to discrete and back again 23/90 Sum the sinc functions up.

26 From continuous to discrete and back again 24/90 Compare the reconstructed result with the original function.

27 1D sampling T vz = T max 8 25/ x x 10 4

28 1D sampling T vz = T max 2 26/ x x 10 4

29 1D sampling T vz = T max 27/ x x 10 4

30 1D sampling T vz = 2.1 T max 28/ x x 10 4

31 Image aliasing example 100% 29/90

32 Aliasing example 90% 30/90

33 Aliasing example 80% 31/90

34 Aliasing example 70% 32/90

35 Aliasing example 60% 33/90

36 Aliasing example 50% 34/90

37 Aliasing example 40% 35/90

38 Aliasing example 30% 36/90

39 Aliasing example 20% 37/90

40 Aliasing example 10% 38/90

41 Aliasing problem in digital photography 39/90

42 Aliasing example 90% 40/90

43 Aliasing example 80% 41/90

44 Aliasing example 70% 42/90

45 Aliasing example 60% 43/90

46 Aliasing example 50% 44/90

47 Aliasing example 40% 45/90

48 Aliasing example 30% 46/90

49 Aliasing example 20% 47/90

50 Aliasing example 10% 48/90

51 What can we do against aliasing? Increase sampling frequency number of pixels (but not only, optics is also important) Decrease the frequency blurring The same snapshots but the input image blurred (convolved) with 5 5 Gaussian with σ = 2: 49/90

52 Suppressed Aliasing 50/90

53 Suppressed Aliasing example 90% 51/90

54 Suppressed Aliasing example 80% 52/90

55 Suppressed Aliasing example 70% 53/90

56 Suppressed Aliasing example 60% 54/90

57 Suppressed Aliasing example 50% 55/90

58 Suppressed Aliasing example 40% 56/90

59 Suppressed Aliasing example 30% 57/90

60 Suppressed Aliasing example 20% 58/90

61 Suppressed Aliasing example 10% 59/90

62 Sampling, Aliasing Revisited Sampling is an important issue in images. 60/90 Aliasing is typically not wanted. Aliasing is ubiquitous. Toy www example4 4

63 Sampling, Aliasing Revisited Sampling is an important issue in images. 60/90 Aliasing is typically not wanted. Aliasing is ubiquitous. Toy www example4 Defeating Aliasing Low-pass filter before subsampling Subsampling by using image interpolation (will come to that later) 4

64 Geometrical Transformation Transformation of spatial coordinates 61/90

65 Geometrical Transformation What for? intentional image transformations (you know where to go) 62/90 resizing rotation shift warping, texture mapping correction of distortions (you know how it should look) projective skew non-linear distortion (fish-eyes) Techniques shared by Image processing, Computer graphics, even Robotics or Mechanics.

66 Example of texture mapping 63/90

67 Realization Rotation and shift 64/90 x = cos(α)x + sin(α)y + t x y = sin(α)x + cos(α)y + t y More elegant and efficient x y 1 = cos(α) sin(α) t x sin(α) cos(α) t y x y 1 in a matrix form x = T x where x and x are homogeneous coordinates: x = [λx, λy, λ] T, λ 0.

68 Identity 65/90 T = identity

69 Rotation 66/90 T = cos(α) sin(α) 0 sin(α) cos(α) rotation for α = 10 T =

70 Rotation + translation 67/90 T = cos(α) sin(α) t x sin(α) cos(α) t y Euclidean for α = 10 and t = [ 50, 30] T T =

71 Affine 68/90 T = affine in general T = a b c d e f degrees of freedom

72 Projective 69/90 T = projective in general T = a b c d e f g h degrees of freedom

73 Correction of converging lines I 70/90

74 Correction of converging lines II 71/90

75 Correction of converging lines III 72/90

76 x = T x; Forward mapping 73/90 I(x,y) I (x,y )

77 Forward mapping problems Maps outside the pixel locations. 74/90 I(x,y) I (x,y )

78 Forward mapping problems Solution: Spread out the effect of each pixel 75/90 I(x,y) I (x,y )

79 Forward mapping problems May produce holes in the output 76/90 I(x,y) I (x,y )

80 Forward mapping problems May produce holes in the output 77/90 I(x,y) I (x,y )

81 Forward mapping problems May produce holes in the output 78/90 I(x,y) I (x,y )

82 Forward mapping problems May produce holes in the output 79/90 I(x,y)? I (x,y )

83 x = T 1 x Backward mapping 80/90 I(x,y) I (x,y )

84 Backward mapping problems Does not always map from a pixel 81/90 I(x,y) I (x,y )

85 Backward mapping problems Solution: Interpolate between pixels 82/90 I(x,y) I (x,y )

86 Interpolation as 2D convolution We have sampled (possibly outside the regular grid) function s(x, y) instead of the wanted f(x, y). We want to find a good approximation by convolution 83/90 ˆf(x, y) = k= l= s(k, l)h(x k, y l)

87 Interpolation nearest neighbour 84/90

88 Interpolation bilinear 85/90 ˆf(x, y) = (1 a)(1 b) s(l, k) + a(1 b) s(l + 1, k) +b(1 a) s(l, k + 1) + ab s(l + 1, k + 1), where l = round (x), a = x l, k = round (y), b = y k.

89 Just 1D, for clarity Interpolation bicubic 86/90 ˆf = 1 2 x 2 + x 3 pro 0 x < 1, 4 8 x + 5 x 2 x 3 pro 1 x < 2, 0 elsewhere. Does not remind you the shape something?

90 Just 1D, for clarity Interpolation bicubic 86/90 ˆf = 1 2 x 2 + x 3 pro 0 x < 1, 4 8 x + 5 x 2 x 3 pro 1 x < 2, 0 elsewhere. Does not remind you the shape something? sinc(x)! The ideal reconstructor.

91 Interpolation nearest vs. bilinear 87/90 rotation rotation nearest bilinear

92 Interpolation nearest vs. bilinear close up 88/90 affine affine nearest bilinear

93 Geometrical Transformation Summary Closed form solution, all pixels undergo the same transformation 89/90 rotation, scaling, translation affine, perspective General, deformation meshes. Transformation depends on position (warping, morphing)

94 References 90/90 [1] Ronald N. Bracewell. Fourier analysis and imaging. Kluwer Academic/Plenum Publishers, New York, USA, 2003.

95

96

97 y x

98

99

100

101

102

103 x x 10 4

104

105

106

107

108

109

110

111

112

113

114 x x 10 4

115 x x 10 4

116 x x 10 4

117 x x 10 4

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150 identity

151 rotation

152 Euclidean

153 affine

154 projective

155

156

157

158

159

160

161 I(x,y) I (x,y )

162 I(x,y) I (x,y )

163 I(x,y) I (x,y )

164 I(x,y) I (x,y )

165 I(x,y) I (x,y )

166 I(x,y) I (x,y )

167 I(x,y)? I (x,y )

168 I(x,y) I (x,y )

169 I(x,y) I (x,y )

170 I(x,y) I (x,y )

171

172

173

174 rotation

175 rotation

176 affine

177 affine