Rotation Invariant Texture Description obtained by General Moment Invariants A. Dimai Communications Technology Laboratory Swiss Federal Institute of

Transcription

1 Rotation Invariant Texture Description obtained by General Moment Invariants. Dimai Communications Technology Laboratory Swiss Federal Institute of Technology, ETH CH Zurich, Switzerland adimaivision.ee.ethz.ch bstract Rotation invariant texture descriptors are required in several applications. The paper proposes a new rotation invariant texture descriptor which is based on scale and orientation tunable Gabor lters. The rotation invariant description is obtained by applying a framework for constructing general moment invariants. The novelty of this approach is that the coupling of dierent scales and statistical orders is included in the texture description as well. The eectiveness of the proposed descriptor is demonstrated experimentally. Introduction large number of texture descriptors or classiers has been proposed in the literature which are eective for classication or similarity assessment of textures. However, the vast majority of algorithms for texture feature extraction make an explicit or implicit assumption that all the images are captured under the same orientations. In many practical applications such as image retrieval, object recognition by texture, surface inspection, object character recognition etc, such an assumption is unrealistic, simply because controlling the environment to ensure a zero rotation angle between images is either costly, dicult or even impossible. Further more, for a given texture image, no matter how it is rotated, it is always perceived as the same texture by a human observer. Therefore, from both the practical and the theoretical point of view, rotation invariant texture descriptors are highly desirable. Despite its importance, work on rotation invariant texture analysis is very limited. n overview of existing work on the subject can be found in [2]. In this paper, a novel rotation invariant texture descriptor is proposed. The construction of this descriptor requires two steps. First, a non-invariant texture descriptor is extracted using a set of Gabor lters. Secondly, from the non-invariant representation a rotational invariant representation is obtained using the framework of the so called general invariant moments. The novelty of the proposed descriptor is that it uses texture characteristics on several scales and statistical order to describe the features and that the coupling of the dierent characteristics is not lost due to the rotation invariant description. This feature is not provided by other rotation invariant texture descriptors. The paper is organized as follows: The framework for constructing general moment invariants is discussed in section 2. Section 4 explains how the general moment invariants are used to build a rotation invariant texture description. Section 5 presents and discusses the results of the proposed texture de- The conclusion are summarized in section 6. 2 General Moment Invariants n often encountered problem in computer vision is scriptor. to nd a representation of a function that is invariant under transformations of the underlying space. ssume function f : IR n! IR and a one parametric transformation S g : IR n! IR n. Then a invariant M is searched for which the following equation holds: 2. Existing Work M (f (x)) = M (f (Sx)) () Several methods were proposed to tackle the problem of invariant pattern recognition. n important class of invariants is build by integral invariants. Commonly used are Fourier coecients, Mellin transform and the Fourier-Mellin [] transform. nother group are the cross or triple correlations [], which are invariant to ane transformations. The existence of such integral invariants and its extension for other problems in the plane are discussed in [3]. The second group of invariants can be summarized by the so called algebraic invariants, which are obtained by taking quotients and powers of moments.

2 moment is a weighted integral of function f (x), with weights equal to some polynomial in x. simple kind of moments are the regular moments. Based on these moments, Hu [7] derived seven moment invariants that are invariant to ane transformations. Other moments based on Legendre, complex moments and Zernike moments [5] [6] have been used, too. However, common to all these works is that they are suited only for functions in the plane and 3D space and ane transformations. Only a few work tackle the problem of nding invariants in a more general setting [4]. In the following, the notion of innitesimal generators is exploit to construct invariants. 3 Innitesimal Generators In the following, the denitions of the regular moments and the innitesimal generators are given. multiindex of dimension n denotes an ordered n-tuple = ( ; :::; n ) (2) of nonnegative integers i. The order of the multiindex is d = jj = nx i= n (3) Further, amonomial x of x 2 IR n and a multiindex is dened by x = ny i= x i i (4) Consider a distribution f (x) with IR n! IR. Distribution have the properties that they are integrable to any degree of polonomyal order d. Hence, the existence of the following integrals are guaranteed. The total volume m of f is dened as: m (f ) = Z IR n f (x)dx (5) Then, the regular moment m of a function is dened as: Z m (f ) = x f (x)dx (6) m IR n The moment has order d = jj. ll the moments up to the order d build a vector m dde embedded in the moment space IR e. For example, for n = 2 the moments up to order d = 2 are embedded in a e = 6 dimensional space m d2e = (m 2 ; m ; m 2 ; m ; m ; m ) T. ssume a one parametric transformation S g that acts on IR n, and that it induces a transformation T g : IR e! IR e on the space of the regular moment, i.e. T g (m dde(f (x))) = m dde (f (S(x))) (7) In many cases the transformation T can be easily computed because the regular moments are elements of tensor products of the coordinate dimensions of IR n. However, for other transformations S, e.g. projective transformations, no corresponding transformation T exist. For a one parametric transformation T the in- nitesimal transformation is given by: U g = T g j g= (8) If the transformation T acts on m then the innitesimal changes of the vector m are dm = U g m dg (9) This fact can be used for generating conditions for moment invariants. The goal is to nd moment invariants M which fulll the following conditions; M (m dde ) = M (T (m dde )) () which can also be expressed by dm =. Hence, dm = 5Mdm, where the term 5M is the gradient M=m of M along each dimension of m. On the other hand the changes of dm are given by equation (9). Therefore, dm = U g m5mdg, which has to equal zero. Using the standard scalar product h:; :i then the operator L = hu g m; m i () is called the innitesimal generator of T. The conditions for M in equation () can then be expressed by: LM = (2) The partial dierential equations for M as function of the regular moments m(f ) has to be solved. To summarize, three steps are required to construct general moment invariants. First, the transformation T g has to be calculated which is invoked by the transformation S g, i.e. T g (m dde (f (x))) = m dde (f (S g x)). Secondly, calculation of the innitesimal generator L. Thirdly, solving the partial dierential equation (2) for M. 4 Rotation Invariant Texture Description In the following the proposed framework is applied on the problem of rotation invariant texture descriptor. For the construction of a rotation invariant feature the signal responses of Gabor ltered images are used.

3 Gabor lters g s can be understand as scale s and orientation tunable edge detectors. Expanding a signal using this basis provides a localized frequency description. s texture description the signal responses of the image I(x; y) are used: w s (x; y) = Z Z I(x ; y )g s (x?x ; y?y )dx dy (3) Taking the moment of order t of the signal response w s yields the pattern functions: Q s () = s = R R jw s (x; y)jdxdy Q t s () =?R R (jw s (x; y)j? s ) t dxdy t (4) It was demonstrated in [9] that descriptors based on such pattern functions for moments t = ; 2 are eective for texture retrieval. In practise the pattern function is not evaluated for all angles and scales. To reduce the redundancy a tiling of the frequency plane as shown in gure is exploit [2]. Typical values for the number of scales are S = 2::4 and number of angles K = 4::8 (for details see [9]). v Figure : Tiling of the frequency planes by Gabor lters. The ellipses indicate the contour of the lters half band width. 4. Pattern Functions under Rotation Q t s Q t s The proposed feature descriptor with elements () is not rotation invariant. However, the patterns () have simple transformation laws. If for example the texture image is rotated by R(), where R is the rotation matrix in equation 7, then the extracted pattern Q t s () is only a shifted version of the original pattern, i.e. Q t s () = Q(? ). ccordingly, a rotation invariant texture representation can be found if the patterns are characterized rotational invariant. 4.2 Existing Work The authors in [3] [6] [8] and [5] were all using the pattern(s) Q() to dene rotation invariant texture descriptors. Mostly, the shape of pattern Q() was s θ u described by Fourier coecient and only patterns for one scale (or frequency) and the rst moment t = (or the mean value of lter responses) were used. The information of other lter response moments or scales were completely neglected. One possibility to include other scales and moments is to apply the invariant description to all patterns Q t s (). However, such a simple approach would completely neglect another important fact. If the texture image is rotated, then all the patterns Q t s () are rotated around the same angle. Therefore, if each pattern by itself is described by rotational invariants, then the coupling between the patterns on dierent scales and moments is lost. 4.3 Rotational Invariants The negligence of the coupling term is a shortcoming of existent rotation invariant texture descriptors. This is one of the important arguments why another rotation invariant descriptor is proposed. The framework of general moment invariants allows to construct a rotation invariant descriptor, based on the patterns Q t s () that includes the coupling between dierent patterns. nother reason why general moments invariant are suited for this application is that the patterns Q t s () are relatively smooth according changes of the angle. Hence, low regular moments already provide a good description of the shape of the patterns. The patterns Q t s () are only dened at discreet points n = n=k. To compute the regular moments of these patterns then a extrapolation between these points can be applied. On the other hand, just using these points in the 2D plane have proven to work sucient for this task. To gain invariants, the following steps have to be executed: First a function f has to be dened that combines all the dierent patterns Q t s (). Secondly, the transformation S acting on f has to be calculated if the image is rotated. Thirdly, the transformation T acting on the space of regular moments of function f have to be deduced from transformation S. Fourthly, the innitesimal transformations and the PDE for M are derived. These steps result in a large number of possible invariants M j and a feature selection is applied to nd the most eective invariants. 4.4 Feature Patterns First, the function f that consists of all patterns Q t s is constructed. For convenience, the patterns Q t s are numerated using only one index i, i.e. Q i with i = ::T S. In the following the pattern function Q i () is interpreted as pattern lying in a two dimensional plane where the pattern is given by the polar-coordinates

4 (; Q()), i.e. the patterns are dened in the Euclidean plane x i 2 IR 2. ll these patterns are combined to one function by: f (x) = X i Q i (x i ) (x ; :::; x i? ; x i+ ; :::; x T S ) (5) where is the Dirac distribution. The multiplication of each pattern with the Dirac distribution ensures that the regular moments dened for each pattern Q i in the plane is identical with the regular moments de- ned in the subspaces of function f. The regular moments up to order d build the vector m dde (f ) (see equation (6)). 4.5 Texture Rotation ssume that the texture image is rotated according R() in equation (7). This rotation induces a rotation of each pattern Q i around the same angel. Then the transformation S acting on x can be written as a one parametric transformation: S = B R() ::: R() ::: ::: ::: ::: ::: ::: R() where R() is the rotation of the plane: cos sin R() =? sin cos C (6) (7) The function f changes to f (x) = f (S (x)). 4.6 Transformation of Regular Moments The next step is to deduce the transformation T that acts on the space of regular moments m dde if the function f is rotated by S. The transformation T is calculated straightforward by using the fact that the regular moments are elements of tensor products of coordinate dimension of the underlying space of f. In the following, the derivation of T is done only for two planes, i.e. a four dimensional space (x ; x 2 ; x 3 ; x 4 ). Then, the result can be easily generalized by permutation of the indices. Further, the calculation is done only for regular moments up to order two, i.e. d = 2. With transformation S of equation (6) and the denition of the regular moments in equation 6, the second moments for one pattern Q i are transformed according (m ; 2 m ; m 2 )T = T (m 2 ; m ; m 2 ) T. It is straightforward to calculate that T takes the form: T = cos 2 2 cos sin sin 2? cos sin cos 2? sin 2 cos sin sin 2?2 cos sin cos 2 The second order moments between distinct planes are transformed as (m ; m ; m ; m ) = T c (m ; m ; m ; m ) where T is: B Tc = cos 2 cos sin cos sin sin 2? cos sin cos 2? sin 2 cos sin? cos sin? sin 2 cos 2 cos sin sin 2? cos sin? cos sin cos 2 The subscript c indicates that the transformation acts on regular moments which couple dierent planes. 4.7 Innitesimal Transformations and PDE From the transformation the innitesimal transformations U are gained by deriving each matrix element along and evaluating the derivative at the unity, i.e. the position =. The innitesimal transformation for T is: and for T c : U = U c = B?2? 2???? C Substitute these innitesimal transformations into equations () and (2) yields PDE's of the form?2m M + (?m m 2 + m 2 ) M + ::: 2 m 2m M = m 2 for the moments of one plane and (m + m ) M m? M m + ::: (?m + m ) M + m M = m (9) for the moments of the coupling terms. The rst PDE is solved by M = m 2 + m 2 and M 2 = m 2 m 2 + m 2. The second PDE is solved by M 3 = m m? m m, M 4 = m +m and M 5 = m?m. The invariants M 3 ; M 4 ; M 5 couple the dierent scales and moments of the Gabor texture description. The results do not consider any moments of degree one because the patterns Q i are all symmetric to the origin of their planes and are therefore equal to zero. (8) C

5 4.8 Feature Selection If high dimensional spaces and high order of regular moments are considered, than the number of possible invariants can be large. For example, if S = 4, T = 2 and order d = 2 then eight planes and possible invariants for M ; M 2 ; M 3 exist. Thereby the highest number steams from the coupling terms M 3 ; M 4 ; M 5. For this work moments up to order d = 3 were considered. From this large set of possible invariants the most discriminative invariants were selected by a principle component analysis. It turned out that the most discriminative invariants are M ; M 2 and some coupling terms M 3. Supported by the principle component analysis the following invariants were selected: for each plane the invariants M ; M 2, i.e. T S +T S terms, the coupling term M 3 on each scale for the different moments, i.e. T S, and the coupling term for the same moments on two consecutive scales, i.e. T (S? ). Selecting these invariants the total size of the rotation invariant descriptor is 4 T S? T, which is smaller than the non rotational Gabor texture descriptor of [9], which has K T S values. 4.9 Similarity ssessment The texture descriptor is used in a content based image retrieval system similar to [4], using the query by example paradigm. The system explores similarity assessment of image descriptors, but does not classify images by learning or user feedback. In this system the descriptor x for an image is extracted. Then, the distances to descriptors y of an image in the database is computed. common distance measure is the Mahalanobis distance. ssume that the covariance matrix of all descriptors in the database is then the Mahalanobis distance between two descriptors x; y is given by: d M (x; y) = q (x? y) T? (x? y) (2) For the image retrieval an image with descriptor x is used as query image. Then the distance d M (x; y) 2 is used for ranking retrieved images. Small values of the distance indicate a high degree of similarity and vice versa. It is noteworthy that a descriptor which performs good for linear similarity assessment is also well suited for linear classication. 5 Results The rotation-invariant texture descriptor was tested in a texture database. It was attempted to constitute a large and heterogeneous database, which can resemble a real work scenario. Most of these textures were taken from standard Brodatz database of textures [], the others have been collected by us from a variety or real-world texture sources, e.g. text, satellite images etc. The database comprises of 25 images of size 28x28 pixels. Some textures had two images in the database. From such images, query images were extracted. From this texture collections, dierent test databases were constructed. The rst database DB I was build to compare the eectiveness of the proposed descriptor for images for which rotation invariance is not required. It was used to test if in such a case the rotation invariant descriptor is comparable with a non-rotational descriptor. For each texture patch, a set of 4 non-overlapping 64 x 64 windows were extracted. These images build the rst database DB I with 2' images. In this database each of the used query images had 7 similar (or corresponding) images. The second database, DB II, was constructed to evaluate the ability to retrieve rotated textures. The original 28x28 image was rotated by 4 o increments, between o and 6 o. The increments of 4 o were chosen such, that it does not equal the 3 o steps in the construction of the Gabor lter set. Each rotated image was then cropped in such a way that the largest square with horizontal and vertical boarders remained. From the cropped image, four 64x64 images were extracted in such a way that the overlap of the images was as small as possible. Following this procedure, for each original image 2 x 5 x 4 = 4 images were extracted. The resulting texture database DB II consists of ' images. In this database each of the used query images had 79 corresponding (or corresponding) images. Four dierent type of texture descriptors were implemented and tested. First, the gabor based nonrotation invariant descriptor (GM). The parameters were K = 6; S = 4; T = 2 which results in a descriptor size of 48 values. The second descriptor was the rotation invariant descriptor (GI) with 3 values. Third, a rotation invariant rotation descriptor without the coupling terms M 3 (GIN) was investigated which consists of 6 values. Fourthly, a rotation invariant texture descriptor based on the Fourier coecient (FC) of the feature pattern proposed in [3] was used. It consists of 5 values. For each descriptor ten query images were submitted to the system. The system extracted a ranked list, according to the distance d M. The performance of each run was evaluated using precision and recall. Consider the number of relevant images (according the query image) as r and the number of retrieved (or con-

6 sidered images) as n, then precision is dened as the proportion of retrieved images that are relevant: P = r n (2) Recall is dened as the portion of relevant documents that are retrieved: R = r r + m (22) where m is the number of relevant document that were not retrieved. high value of the numbers indicates a good performance of the system. Because the similarity assessment generates a ranked list, only images are regarded with ranks lower than the cuto number n. In the presented experiments, the cuto number was chosen in such a way that it equals the number of total relevant images for a query image. This means, for the database DB I the cuto number was n = 7 and for DB II n = 79. In this case, both measure, recall and precision, are equal. Model no rotation (DB I) rotation (DB II) P=R P=R GI :9 :7 :85 :4 GIN :85 :9 :75 : GM :94 :7 :52 :8 FC :8 :3 :68 : Table : Precision of dierent texture models. The mean and the standard deviation of ten retrieval results are listed for dierent descriptor models. GI: proposed rotation invariant texture descriptor. GIN: invariant descriptor without coupling terms M 3. GM: no-rotational invariant texture descriptor using Gabor lters. FC: rotation invariant texture descriptor using Fourier coecients. For each descriptor the same set of ten query images was used. The mean and its standard deviation of precision for the ten test runs are listed in table for dierent descriptor models. The results clearly indicate the good performance of the rotation invariant descriptors. In the case of the database where the rotation invariance is irrelevant, the rotation invariant descriptor performs nearly as well as the non-rotation invariant Gabor descriptor. It seems that the introduction of rotation invariance does not diminish the discrimination power signicantly. Comparing GI and GIN shows that the coupling term M 3 of dierent dimension and signal moments improves the similarity assessment. Therefore, these experiments support the theoretical consideration to include the coupling of the pattern Q t s. In the second database, DB II, the performance differences are more signicant, due to the larger cuto number n. The proposed invariant descriptor GI performs best. nd like in the previous experiments, it is better then the rotation invariant descriptor without including the coupling terms. This dierence is mainly due to three query images, where the structures on different scales were very characteristic for the image and not only on one scale. The bad performance of GM could be expected. Only in the cases, where the texture is nearly isotropic the GM performed reasonable. The modest performance of the rotation invariant descriptor F I in both cases shows that the information of dierent scales and signal response moments should not be neglected. 6 Conclusion The paper proposes a novel rotation invariant texture descriptor. The descriptor is based on a texture descriptor using scale and orientation tunable Gabor lters. The texture characteristics for dierent angles, scale and statistical order are then combined to a rotation invariant representation. For the construction of the representation the concept of the innitesimal generator is applied, which yields a set of partial differential equations for the invariants. The proposed texture descriptor allows to encode the coupling between dierent texture characteristics on dierent scale and statistical orders. feature that is not provided by other rotation invariant texture descriptor. Experiments on a texture database were conducted. s expected, the rotation invariant descriptor outperforms the non-rotational invariant descriptor where rotation invariants is required. In the case of nonrotated textures the proposed method performs nearly as good as a non-rotation invariant descriptor. cknowledgments The research was supported partially by the ETH cooperative project on Integrated Image nalysis and Retrieval. References [] P. Brodatz. Textures: a photographic album for artists and designers. Dover,New York, US, 956. [2] J. G. Daugman. Complete discrete 2-D gabor transforms by neural networks for image analysis and compression. IEEE Trans. coustics, Speech and Signal Processing, 36(7):6979, 988.

7 Query Image Figure 2: Retrieval results for three query images (textile, remote sensing images of agriculture elds and mountains) using the proposed rotation invariant texture descriptor. The database contains 2' not rotated images (DB I). The results indicate that the rotation invariant descriptor performs nearly as good as the non-rotation invariant texture descriptor. [3] M. Ferraro and T. M. Caelli. Relationship between integral transform invariances and lie group theory. Journal Optical Society of merica, 5(5):738742, 988. [4] M. Flickner, H. Sawhney, W. Niblack, J. shley, B. Dom Q. Huang, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker. Query by image and video content: the QBIC system. IEEE Computer, 28(9):2332, 995. [5] H. Greenspan, S. Belongie, R. Goodman, and P. Perona. Rotation invariant texture recognition using a steerable pyramid. In Inter. Conf. on Pattern Recognition, volume B, pages 6267, Nov [6] G. M. Haley and B. S. Manjunath. Rotation invariant texture classication using modied gabor lters. In Inter. Conf. on Image Processing, pages , Oct [7] M. K. Hu. Visual pattern recognition by moment invariants. IRE Trans. Information Theory, 8(2):79 87, 962. [8] M. M. Leung and. M. Peterson. Scale and rotation invariant texture classication. In Proceedings 26th Silomar Conference Signals, Systems and Computers, volume, pages 46465, 992. [9] W.Y. Ma and B.S. Manjunath. Texture features and learning similarity. In IEEE Conf. on Computer Vision and Pattern Recognition, pages 42543, June 996. [] D. Pintsov. Invariant pattern recognition, symmetry and radon transforms. Journal Optical Society of merica, 6():544554, 989. [] Y. Sheng and H. H. rsenault. Experiments on pattern recognition using invariant fourier-mellin descriptors. Journal Optical Society of merica, 3:77 776, 986. [2] T. N. Tan. Texture feature extraction via cortical channel modelling. In Proceedings of th IPR International Conference on Pattern Recognition, pages C67C6. IEEE CS Press, 992. [3] T. N. Tan. Rotation invariant texture features and their use in automatic script identication. IEEE Trans. on Pattern nalysis and Machine Intelligence, 2(7):75756, 998. [4] G. Taubin and D. Cooper. Object recognition based on moment (or algebric) invariants. In J. Mundy and. Zisserman, editors, Geometric Invariance in Computer Vision, pages MIT Press, Cambridge, Mass, 992. [5] M.R. Teague. Image analysis via the general theory of moments. Journal Optical Society of merica, 7(8):9293, 98. [6]. Wallin and O. K bler. Complete sets of complex zernike moment invariants and the role of the pseudoinvariants. IEEE Trans. on Pattern nalysis and Machine Intelligence, 7():6, 995.