Multiscle Fourier Descriptor for Shpe Clssifiction Iivri Kunttu, een epistö, Juhni Ruhm 2, nd Ari Vis Tmpere University of Technology Institute of Signl Processing P. O. Box 553, FI-330 Tmpere, Finlnd Iivri.Kunttu@tut.fi 2 ABB Oy Pper, Printing, Metls & Minerls P. O. Box 94, FI-0038 Helsinki, Finlnd Abstrct The description of the object shpe is n importnt chrcteristic of the imge. In imge processing nd pttern recognition, severl different shpe descriptors re used. In humn visul perception, the shpes re processed in multiple resolutions. Therefore multiscle shpe representtion is essentil in the shpe bsed imge clssifiction nd retrievl. In the description of the object shpe, the multiresolution representtion provides lso dditionl ccurcy to the shpe clssifiction. In this pper we introduce new descriptor for shpe clssifiction. This descriptor is clled multiscle Fourier descriptor, nd it combines the benefits of Fourier descriptor nd multiscle shpe representtion. This descriptor is formed by pplying Fourier trnsform to the coefficients of wvelet trnsform of the object boundry. In this wy the Fourier descriptor cn be presented in multiple resolutions. We mke clssifiction experiments using three imge dtbses. The clssifiction results of our method re compred to those of Fourier descriptors.. Introduction The description of the object shpe is n importnt tsk in imge nlysis nd pttern recognition. The shpes occurring in the imges hve lso remrkble significnce in imge retrievl [4]. The bsic problem in shpe clssifiction is to define the similrity between two shpes. In mny cses, this similrity mesurement should obey the humn shpe perception. Imges cn be clssified bsed on their shpe content using different types of shpe descriptors [3]. In the field of psychophysics, it hs been found tht the humn visul system processes nd nlyzes imge informtion t different resolutions. Therefore multiscle shpe representtion is essentil in the clssifiction of the shpes occurring in the imges. Severl methods for shpe description hve been developed. The shpe description techniques cn be divided into two types, boundry bsed nd region bsed techniques [7]. The region bsed methods consider the whole re of the object wheres the boundry bsed methods concentrte merely on the object boundry line. In this pper we consider the boundry bsed methods. The most common boundry bsed shpe descriptors re chin codes [6] nd Fourier descriptors [5]. Also utoregressive (AR) [5],[2] models hve been used to represent the boundries of curves. During recent yers, curvture scle-spce (CSS) [4] shpe representtion hs lso been widely used. Kuppinen et l. [] mde comprison between utoregressive models nd Fourierbsed descriptors in shpe clssifiction. In this comprison, Fourier-descriptors proved to be the best in the clssifiction of different shpes. In the comprison mde by Mehtre et l. [3], retrievl bility of chin codes, Fourier-descriptors, nd different moments were compred in shpe similrity-bsed retrievl. In this cse, the best results were obtined using moments nd Fourier-descriptors, wheres the lowest retrievl results were given by chin codes. In this pper, we pply wvelet trnsform to the clssifiction of different shpes. Wvelet trnsform hs been widely used in multiscle imge nd signl nlysis. It is used for exmple in imge nd signl compression nd noise reduction. However, wvelet trnsform hs only few pplictions in the field of shpe description. In [] Chung nd Kuo used one-dimensionl discrete periodized wvelet trnsform (DPWT) to describe plnr curves. The sme trnsform ws used lso in [9], in which the method ws mde rottion invrint. In contrst to these studies, we use complex wvelet trnsform. In our pproch, the boundry of the object is presented in complex form like in the cse of Fourier descriptors. Using complex wvelet trnsform, the multiscle representtion of the shpe cn be chieved. The multiscle Fourier descriptor is obtined by pplying the Fourier trnsform to the coefficients of the multiscle wvelet trnsform. Using Fourier trnsform, the wvelet Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)
coefficients cn be presented in frequency domin, which mkes the descriptor invrint for rottion nd strting point of the boundry. The use of Fourier trnsform mkes it lso possible to present the descriptor in fixed length, independent on the length of the object boundry. In section two, the generl principle of Fourier descriptors nd complex wvelet trnsform re presented. In the sme section we show how the multiscle Fourier descriptor for object shpe cn be formed using complex wvelet trnsform. The clssifiction bility of the descriptors is tested in section three using three sets of testing imges. The results of the clssifiction re discussed in section four. 2. Shpe descriptors The use of Fourier descriptors is common in pttern recognition nd imge nlysis. The benefits of the Fourier descriptors re invrince to the strting point of the boundry nd rottion [3]. However, the use of Fourier-bsed multiscle representtion of shpe is new ppliction in shpe representtion. In this section we present multiscle shpe descriptor bsed on the complex wvelet trnsform nd Fourier trnsform. It is simple descriptor tht combines the benefits of Fourier representtion of the object shpe nd the multiresolution nture of wvelet trnsform. 2.. Representtion of the object boundry In this pper the shpe description methods re bsed on the boundry of the object. Therefore, the boundry of the object hs to be extrcted from the imge. In the presenttion of the object boundry, we use the complex coordinte function []. This function is simply the coordintes of the boundry pixels in n object centered coordinte system, represented s complex numbers: z k) = ( x x ) + j( y y ) () ( k c k c for k=0,,2,,-, in which is the length of the boundry nd (x c, y c ) is the centroid of the object. Using complex coordinte function, the boundry cn be represented independent on the loction of the object in the imge. In this wy the trnsltion invrince cn be chieved. 2.2. Fourier descriptors The shpe descriptor bsed on the object boundry cn be formed in severl wys. Fourier trnsform [5] is the most common method for this purpose. Fourier trnsformtion of boundry function genertes set of complex numbers which re clled Fourier descriptors. Fourier descriptors chrcterize the object shpe in frequency domin. The Fourier descriptors cn be formed for complex boundry using discrete Fourier trnsform (DFT) [7]. Fourier trnsform of z(k) is: F( n) = k = 0 z( k) e j 2πnk / for n =0,,2,,-. The generl shpe of the object is represented by the lower frequency descriptors, wheres high frequency descriptors represent the smll detils of the object shpe. A common pproch to shpe clssifiction is to use only subset of the descriptors. These subsets cn be formed in severl different wys. Kuppinen et l. [] hve compred Curvture Fourier, Rdius Fourier, Contour Fourier, nd A-invrint methods for Fourier-bsed shpe representtion. According to their experimentl results, Contour Fourier nd A-invrint methods were best pproches in shpe clssifiction. In this work, we selected Contour Fourier method for testing purposes. The Contour Fourier method mkes the Fourier trnsform directly for the complex coordinte function of the object boundry. In this method, the descriptors re tken both positive nd negtive frequency xis. The scling of the descriptors is mde by dividing the bsolute vlues of the selected descriptors by the bsolute vlue of the first non-zero component. The feture vector for this method is: F x = ( / 2 ) F (2) T F F F 2 / 2 (3) in which is constnt vlue tht defines the number of the smples selected from the Fourier coefficients. 2.3. Complex wvelet trnsform The multiscle representtion of the object boundry cn be chieved using wvelet trnsform. The boundry function is trnsformed using some wvelet Ψ. Complex wvelet trnsform is bsed on the continuous wvelet trnsform (CWT) [2]. In CWT, the wvelet coefficient of the boundry z(k) t scle nd position b is defined by: C b) = R k b z( k) ψ dk ( (4) As in the Fourier trnsform, lso in cse of CWT we obtin set of complex coefficients C (b) of scle. The coefficients re defined for ll the positions b=0,,2,...,-. Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)
2.4. Multiscle Fourier descriptor The problem with the coefficients obtined from the complex wvelet trnsform is the fct tht they re dependent on the strting point of the object boundry. Also the length of the feture vector depends on the length of the object boundry. Therefore, the coefficient vectors of different shpes cnnot directly be mtched in the imge clssifiction. The solution for this problem is to pply the Fourier trnsform to the coefficients obtined from the complex wvelet trnsform. In this wy the multiscle shpe representtion cn be trnsformed to the frequency domin. As result, multiscle Fourier descriptor is obtined. The descriptor is formed by pplying the discrete Fourier trnsform of eqution 2 to the set of complex coefficients C (b): F ( n) = b= 0 C ( b) e j 2πnb / The multiscle descriptor x of ech scle is then formed from coefficients F (n) using Contour Fourier method presented in eqution 3: x F = F ( / 2 ) T (5) F F F 2 / 2 (6) The multiscle representtion of the object shpe cn then be formed by defining the descriptor x using severl different scles, nd combining the descriptors into single feture vector, FW of length R. et the set of scles be A={, 2,..., r }. So the number of the scles in the descriptor is r. 3. Clssifiction experiments In this section, we mke clssifiction experiments using our method, multiscle Fourier descriptor. The clssifiction results re compred to those of Contour Fourier pproch. 3.. Testing dtbses For testing purposes, we used three imge dtbses. Two of these dtbses were industril defect imge dtbses, which re quite difficult to clssify. However, in these imges, shpe is one essentil clssifying feture nd therefore these dtbses re used in the experimentl prt of this pper. In ddition to these industril dtbses, we hd lso dtbse of very simple shpes. Using these three dtbses, we cn show tht our method cn be used in the shpe-bsed clssifiction of severl different imge dtbses. Testing dtbse I consisted of pper defect imges. The imges were tken from the pper mnufcturing process using pper inspection system [6]. The defects occurring in the pper cn be for exmple holes, wrinkles or different kinds of dirt spots. The test set consisted of 204 pper defects, which represented 4 defect clsses so tht ech clss consisted of 27-03 imges. An exmple imge of ech clss is presented in figure. Within the clsses, there were differences in the size nd orienttion of the defects. This fct cn be seen in figure 2, in which the vritions of the defect clss re presented. The second industril imge set, testing dtbse II, contined 943 metl defect imges. Also this dtbse contined 4 defect clsses. In ech clss, there were 00-65 imges. Figure 3 presents n exmple of ech clss nd the vritions in the defect clss re presented in figure 4. Different defect types in both industril dtbses cn be distinguished using their shpe or gry level. In this pper we concentrte on the shpe informtion of the defects. The clssifiction of the defect imges is demnding tsk, becuse in some clsses the shpes re very similr. In the cse of some defect clsses, the shpes re lso overlpping, which reduces the clssifiction. The defects cn be extrcted from their bckground using n imge segmenttion method presented in [0]. The third test set, testing dtbse III, consisted of 30 imge clsses selected from the MPEG-7 imge dtbse. Ech clss contined 20 imges, so tht the size of the whole testing dtbse ws 600 imges. The imges were silhouettes of some simple objects. An exmple of ech imge clss in testing dtbse III is presented in figure 5. In ech clss, the imges were vritions of the sme object. In these imges, shpe, size, nd orienttion re vrying. An exmple of the vritions within the clss deer is presented in figure 6. In ll imges, the object is deer, but the size, shpe, nd orienttion of the deer vries significntly. 3.2. Clssifiction The dtbse imges were indexed by clculting the feture vector x for them. The selected wvelet ψ ws complex gussin wvelet of order two. The multiscle presenttion ws chieved using set of three scles. The scle sets A were selected to be [0,5,20], [0,20,30], nd [50,80,0] for testing dtbses I, II, nd III, respectively. For comprison, lso the feture vector x of Contour Fourier method ws clculted for ech test set imge. Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)
Figure. The exmple imges of ech pper defect imge clss of testing dtbse I. Figure 2. 0 exmples of clss pper defect imges in the testing dtbse I. Figure 3. The exmple imges of ech metl defect imge clss of testing dtbse II. Figure 4. 0 exmples of clss metl defect imges in the testing dtbse II. The clssifiction ws mde using nerest neighbor lgorithm. The distnce mesure between the feture vectors ws selected to be Eucliden distnce. This distnce cn be clculted between the feture vectors (FW) of query imge Q nd dtbse imge D in the following wy: E R Q D ( ( i) ( i) ) D ( Q, D) = FW FW (7) i= The vlidtion of the shpe-bsed clssifiction ws mde using leving one out method [8]. In this method ech imge in turn is left out from the test set nd used s query imge, wheres the other imges in the test set form testing dtbse. The verge clssifiction rte ws mesured for both testing dtbses using three vlues for. The results re presented in tbles, 2, nd 3. 2 The computtionl chrcteristics of the clssifiction in both dtbses re presented in tble 4. The computtion ws mde using Mtlb on PC with 804 MHz Pentium III CPU nd 256 MB primry memory. Tble. The verge clssifiction rte of test set I. Contour Multiscle Fourier Fourier 6 37. % 43.7 % 32 39.0 % 45. % 64 40.8 % 43.9 % Tble 2. The verge clssifiction rte of test set II. Contour Multiscle Fourier Fourier 6 26.6 % 3.6 % 32 27.8 % 30.4 % 64 29.2 % 30.5 % Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)
Figure 5. The exmple imges of ech defect imge clss of testing dtbse II. Figure 6. 20 exmples of deer imges in the testing dtbse II. Tble 3. The verge clssifiction rte of test set III. Contour Multiscle Fourier Fourier 6 93.5 % 96.3 % 32 93.5 % 94.7 % 64 94.2 % 94.2 % 4. Results nd discussion In this pper we presented new shpe representtion method, multiscle Fourier descriptor, for shpe-bsed imge clssifiction. This descriptor combines wvelet trnsform nd Fourier trnsform. In this wy, the benefits of both trnsforms cn be utilized. Therefore, when the wvelet trnsform is pplied to the object boundry, the shpe description is obtined in multiple resolutions. This is remrkble becuse humn vision system uses multiresolution representtion of shpe. This representtion improves lso the clssifiction of the shpes occurring in the imges. Tble 4. The Computtionl chrcteristics of the methods. The computing times re presented for the clssifiction of the whole dtbses. FEATURE Multiscle Fourier =6 =32 =64 Contour Fourier =6 =32 =64 Vector Clssifiction time length DB I DB II DB III *r 48 96 92 6 32 64 56 sec 8 sec 28 sec 44 sec 47 sec 64 sec 67 sec 97 sec 59 sec 46 sec 56 sec 83 sec 28 sec 36 sec 49 sec 0 sec 25 sec 29 sec Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)
In our pproch, the obtined multiscle shpe representtion is trnsformed into frequency domin using Fourier trnsform. In this wy, our shpe description pproch is invrint for rottion nd the strting point of the boundry line. According to the results presented in tbles, 2, nd 3, our method, multiscle Fourier, gives better clssifiction results thn Contour Fourier method in ll testing dtbses. The clssifiction ccurcy ws very high in cse of the dtbse III, in which the object shpes were quite simple nd esy to distinguish from ech other. On the other hnd, the clssifiction rte ws reltively low in the industril imge dtbses I nd II. This is becuse the shpe clsses of these dtbses re much hrder to distinguish from ech other. In fct, the clssifiction of the defect imges is demnding tsk even to n expert. However, the results show tht our method is pplicble in severl types of imge dtbses. The computtionl cost of multiscle Fourier is lso resonble. Compred to the Contour Fourier, multiscle pproch demnds more computtion time due to the incresed feture vector length. On the other hnd, the difference between the clssifiction times is not remrkble, nd in the cse of smll vlues of, the whole dtbses cn be clssified in the less thn 00 sec. In conclusion, the multiscle Fourier descriptor proved to be n effective tool for clssifying different types of shpes. The clssifiction results show tht when multiscle representtion is combined to the commonly used Fourier-bsed shpe description, the clssifiction results cn be esily improved. 5. Acknowledgment The uthors wish to thnk the Technology Development Centre of Finlnd (TEKES s grnt 40397/0) for finncil support. 6. References [] G.C.-H. Chung nd C.-C.J. Kuo, Wvelet Descriptor of Plnr Curves: Theory nd Applictions, IEEE Trnsctions on Imge Processing, Vol. 5, o., Jn. 996, pp. 56-70. [2] C.K. Chui, An Introduction to Wvelets, Acdemic Press, Sn Diego, 992. [3].F. Cost nd R.M. Cesr, Shpe Anlysis nd Clssifiction, Theory nd Prctice, CRC Press, Boc Rton, Florid, 200. [4] A. Del Bimbo, Visul Informtion Retrievl, Morgn Kufmnn Publishers, Sn Frnsisco, Cliforni, 200. [5] S.R. Dubois nd F.H. Glnz, An Autoregressive Model Approch to Two-Dimensionl Shpe Clssifiction, IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence, Vol. 8, 986 pp. 55-66. [6] H. Freemn nd.s. Dvis, A Corner Finding Algorithm for Chin Coded Curves, IEEE Trnsctions on Computers, 26, 977, pp. 297-303. [7] R.C. Gonzlez nd R.E. Woods, Digitl Imge Processing, Addison Wesley, 993. [8] D. Hnd, H. Mnnil, nd P. Smyth: Principles of Dt Mining, MIT Press, Msschusetts, 200. [9] K.-C. Hung, The Generlized Uniqueness Wvelet Descriptor for Plnr Closed Curves, IEEE Trnsctions on Imge Processing, Vol. 9, o. 5, My 2000, pp. 834-845. [0] J. Iivrinen, J. Ruhm nd A. Vis, Unsupervised segmenttion of surfce defects, Proceedings of 3th Interntionl Conference on Pttern Recognition, Vol. 4, Wien, Austri, Aug. 25 30, 996, pp. 356 360. [] H. Kuppinen, T. Seppänen, nd M. Pietikäinen, An Experimentl Comprison of Autoregressive nd Fourier-Bsed Descriptors in 2D Shpe Clssifiction, IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence, Vol. 7, o. 2, Feb. 995, pp. 20-207. [2] R.. Kyshp nd Chellpp, Stochstic Models for Closed Boundry Anlysis: Representtion nd Reconstruction, IEEE Trnsction on Informtion Theory, Vol. 27, o. 5, 98, pp. 627-637. [3] B.M. Mehtre, M.S. Knknhlli, nd W.F. ee, Shpe Mesures for Content Bsed Imge Retrievl: A Comprison, Informtion Processing & Mngement, Vol. 33, o 3, 997, pp. 39-337. [4] F. Mokhtrin nd A.K. Mckworth, A Theory of Multiscle, Curvture-Bsed Shpe Representtion of Plnr Curves, IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence, Vol. 4, o. 8, Aug. 992, pp. 789-805 [5] E. Persoon nd K. Fu, Shpe Discrimintion Using Fourier Descriptors, IEEE Trnsctions on Systems, Mn, nd Cybernetics, Vol. 7, 977, pp. 70-79. [6] J. Ruhm nd R. Reinius, Pper Web Imging with Advnced Defect Clssifiction, Proceedings of the 2002 TAPPI Technology Summit, Atlnt, Georgi, Mrch 3-7, 2002. Proceedings of the 2th Interntionl Conference on Imge Anlysis nd Processing (ICIAP 03)