The Detection Techniques for Several Different Types of Fiducial Markers

Transcription

1 Vol. 1, No. 2, pp (2013) The Detection Techniques for Several Different Types of Fiducial Markers Chuen-Horng Lin 1,*,Yu-Ching Lin 1,and Hau-Wei Lee 2 1 Department of Computer Science and Information Engineering, National Taichung University of Science and Technology, Taichung, Taiwan, R.O.C. 2 Department of Software and Advanced Technology Research, Chiuan-Yan Tech. Co., Ltd., Changhua, Taiwan, R.O.C. * Corresponding Author / linch@nutc.edu.tw KEYWORDS:Reference fiducial mark; Automatic detection; Concentric circles mark; Straight lines intersection mark The detection techniques are proposed for several different types of fiducial mark (FM) in this paper. This study proposes "traditional fiducial mark positioning", "concentric circles fiducial mark detection" and "straight lines intersection mark positioning" methods according to different mark patterns. For traditional fiducial mark positioning method, two processing methods are proposed, including automatic detection of "reference fiducial mark" and fiducial mark positioning. The concentric circles fiducial mark detects the center and edge lines of concentric circles in the image. The straight lines intersection mark detects the intersection point of horizontal and vertical edges of the object. The experimental results showed that the proposed method can improve manual positioning of traditional fiducial marks, increase the effectiveness and accuracy, detect the concentric circles fiducial mark correctly and rapidly, and it can locate the straight lines intersection mark accurately. Manuscript received: September 23, 2013 / Accepted: October 23, Introduction With the development of manufacturing technology of semiconductor and optoelectronics industry, various electronic modules progress towards compactness and sophistication, and the product manufacturing technology aims at high speed, high precision and high stability, the requirement for manufacturing accuracy becomes strict increasingly. At present, the integration of micromodules into microchips almost challenges the process of precision positioning. The small size electronic parts depend on manual operation in industry, and they are aligned manually. In the process requiring for repeated positioning, the manual alignment cannot meet the circle's requirements anymore. Therefore, repeated positioning will be one of important topics of production automation. Using one to four industrial cameras (CCD camera, hereinafter referred to as CCD alignment system) is one of most effective methods. Even so, this technology requires the specific fiducial mark (FM) in the image to be taken out manually, called reference fiducial mark (RFM), and then the RFM is used as the standard of alignment [1-6]. The RFM is generally determined by the visual inspection of the operator. There are many types of FM in industrial application, as shown in Fig. 1. The object segmentation is a common method for extracting FM. In order to distinguish the object from the background accurately, the common segmentation techniques in the past were [7-13], including Gradient method [9], Sobel edge detection [9], Canny edge detection [7-8], Laplacian edge detection[9], threshold method [9], region growing [10], region splitting and merging [9] and clustering [10]. The correct FM must be found by matching processing. The matching methods in previous studies include local energy [14], fast corner detection [15], Hough transform[4], and matching cost computation [16]. In this paper, the images are classified into "with FM" and "without FM" types according to the FM type, and then they are classified into "traditional fiducial mark positioning", "concentric circles fiducial mark detection" and "straight lines intersection mark positioning" according to the mark pattern, as shown in Fig. 2, so as to analyze and research "with FM" and "without FM". 2. The proposed method Fig. 1 FMs This paper proposes three methods, including "traditional fiducial mark positioning", "concentric circles fiducial mark detection" and "straight lines intersection mark positioning". 2.1 Traditional fiducial mark positioning 86

2 Vol. 1, No. 2, pp (2013) Automatic positioning system With FM Without FM Traditional fiducial mark positioning Concentric circles fiducial mark detection Straight lines intersection mark positioning Fig. 2 Various alignment plate images Fig. 3 Binary image Fig. 4 Automatic fiducial mark detection In order to improve the way of obtaining the RFM and the effectiveness and accuracy of FM positioning, this paper will propose the RFM automatic detection and FM positioning methods respectively "RFM" automatic detection First, the RGB image is converted into YCbCr, and then the brightness Y is used as gray image of later processing. The RFM detection means to detect probable FM in the image, and then the user makes a choice. This FM is called RFM. The image is processed by Otsu [17], and then the image is binarized, as shown in Fig. 3. Afterwards, the rectangular region of object is taken out, and it is defined as candidate FM, as shown in Fig.4. In order to remove small noise, the minimum area of object is set as larger than 1/50 of the image size, and it is assumed not to occur in the edge of the image. Finally, the object selected by the user is called RFM FM positioning The FM positioning will use RFM as the standard of search. In order to accelerate the search speed, this paper processes the image by Haar Discrete Wavelet Transform (HDWT), and then selects lowfrequency band for FM positioning. This paper uses original image f 1 2 and low-frequency band of level = 1, 2, 3, expressed as f, f and f respectively. The order of search is f, f, f and f. The FM positioning means to scan the image from top to bottom and from left to right, and then the RFM size is used as the matching region, and to calculate the sum of absolute differences (SAD) among relative positions in the region. Afterwards, the sum is used as the basis of regional similarity (winner-take-all, WTA [18]). The image is searched in unit of two pixels, thus, the search time can be reduced to half of original time. When the image is processed by HDWT, the full image is searched by f 3, and then the position corresponds to 2 f. Afterwards, the adjacent region of f 2 is searched. When the positioning is completed, the aforesaid steps are repeated till 1 positioning of f and f. 2.2 Concentric circles fiducial mark detection Concentric circles fiducial mark detection, the concentric circles consist of inner circle and outer circle, as shown in Fig. 5, called concentric circles fiducial mark. First, the threshold is obtained by using Otsu', then the image is binarized, as shown in Fig. 6. In order to remove the noise, the morphological processing is used, as shown in Fig. 7. The object contour is obtained, as shown in Fig. 8. Finally, the contour at the edge of the image is eliminated to obtain the contour of inner circle. The hough transform is used to detect the inner circle contour, so as to find out the circle O passing most of edge points and its center O, as shown in Fig. 8. c Afterwards, the outer circle is detected. First, the average gray level L of all the pixels with image gray level higher than threshold T L is calculated. The gray level lower than T L is replaced by L, as shown in Fig. 9, so as to adjust the low gray level upwards. The contrast of the image is increased by using histogram equalization, as shown in Fig. 10. Finally, the region growing algorithm [19] is used, and the center point of inner circle is selected as the initial seed point. r o is the initial radius, r i is the radius of the primary expansion region of r o, r e is radius of the secondary expansion region of r o, as shown in Fig. (11). The conditions of region growing are expressed as follows. region growing = (1) 87

3 Vol. 1, No. 2, pp (2013) Fig. 5 Two-circle FM Fig. 6 Binary image Fig. 7 Noise processing Fig. 8 Inner circle detection Fig. 9 Low-level gray scale processing Fig. 10 Histogram equalization L 1 Locating point L 2 Fig. 11 Schematic diagram of geometrical relationship between concentric circles Fig. 12 Straight lines intersection mark image Fig. 13 Binary image Fig. 14 Edge image 88

4 Vol. 1, No. 2, pp (2013) in Fig Experimental results and Fig. 15 Straight lines intersection mark T T, and 1 and i and m e f i i fi ( x, y) Ae m i f i (2) 1 f i ( x, y) A i where T is the threshold of region growing. A A e is the region between radii ri ro and re ri. A i is the region between radii r o and r i. If the region growing is true, the region grows pixels to r o, otherwise the contour of r i as outer circle stops. 2.3Straight lines intersection mark detection This section aims at an object in unspecific shape, as shown in Fig. 12. The two edges of the object are indicated by straight lines (red straight lines). The intersection point of two straight lines O c is called locating point. First, the image is processed by Otsu's, then it is converted into binary image, as shown in Fig. 13. The edges are found by morphological processing, as shown in Fig. 14. The edge information is processed by hough transform for straight line detection, two edge lines L 1 and L 2 can be obtained, expressed as follows: L : a x b y c and L2 : a2x b2 y c (3) 2 and the intersection point O c of L 1 and L 2. The results are shown This experimental image database is provided by Chiuan Yan Technology Co. (hereinafter referred as CYT). There are 29 images, described in Table 1. This experiment was completed on Intel Core(TM)2Duo CPU 3.00 GHz 3.00 GHz, 2.00GB RAM computer and Microsoft Windows 7 platform. 3.1Traditional fiducial mark search In traditional fiducial mark image database, Case 1 is a multifiducial mark image, and Case 2 is a single fiducial mark image. There are different changes in mark position, light rays and contrast. The experimental program uses C++ language of Visual Studio "RFM" automatic detection The two images of Case 1 and Case 2 are used for the experiment on "RFM" automatic detection. The results are shown in Fig. 16, suggesting that the proposed method can detect the candidate RFM correctly Fiducial mark positioning The eight images of Case 2 are used for this experiment. The "RFM" will adopt the former result. Two "RFMs" are used to locate the fiducial mark of eight images respectively. Each experimental image is aligned after second on average. The experimental results are shown in Fig. 17 and 18. The red cross in the image is the center of the image. The yellow frame is the mark positioning region. The white point in this region represents the gravity center of the mark. According to the positioning results, the method proposed in this paper has quite accurate mark positioning results Table 1 Experimental image classification Image classification name Image type Image Qty Traditional fiducial mark positioning Case 1 Multiple fiducial marks Case 2 Single fiducial mark 8 1 Concentric circles fiducial mark detection Case 3 2 Straight lines intersection mark positioning Case

5 Vol. 1, No. 2, pp (2013) Image type Candidate "RFM" (a) (b) Case 1 Case 2 Fig. 16 RFM detection RFM Fig. 17 FM positioning result of the first group of RFM 90

6 Vol. 1, No. 2, pp (2013) RFM Fig. 18 FM positioning result of the second group of RFM Table 2 Comparison of computing time Method and language applied Image size 3648*2736 Image size 1280*1024 Traditional method Matlab program 6619 sec 124 sec The proposed method Matlab program sec 0.5 sec JAVA program sec sec C program sec sec (a) image (b) image (c) Fig. 19 Multi-FM positioning result (d) 91

7 Vol. 1, No. 2, pp (2013) In order to further validate the effectiveness of the proposed method, two images in size of and are used, as shown in Fig. 19a and 19b, and the result and computer computing time of "RFM" of (a) in Case 1 are shown in Fig. 19b, 19c and Table 2. The results show that the proposed method can improve traditional positioning method effectively, and it can shorten the computing time of computer. 3.2Concentric circles fiducial mark detection The experimental images detected are two images in Case 3. The result is shown in Fig. 20. Although the two images have different brightnesses, the proposed method can detect the contours of inner and outer circles correctly. Fig. 20 Concentric circles fiducial mark detection result Fig. 21 Straight lines intersection mark detection result 92

8 Vol. 1, No. 2, pp (2013) 3.3Straight lines intersection mark detection The images in Case 4 are used for this experiment, the images are taken from the objects in different positions. The detection result is shown in Fig. 21.The result shows no matter whether the intersection point of objects is close to the edge of the image or in the image, the proposed method can detect the intersection point position accurately 4. Conclusion This paper proposes three image detection and positioning methods, including "traditional fiducial mark", "concentric circles fiducial mark" and "straight lines intersection mark". According to the experiments in this paper, in terms of traditional fiducial mark positioning, the automatic detection of "RFM" actually can improve manual operation, and the FM positioning effectiveness and accuracy are very high. In terms of concentric circles fiducial mark detection, this paper provides a correct and rapid detection method. In terms of straight lines intersection mark positioning, the proposed method can locate the mark position accurately, so as to attain the goal for mark positioning. According to the processing mode and procedure of the proposed method, it is believed that diversified special FMs can be detected in the future. ACKNOWLEDGEMENTS This work was supported in part by Department of Software and Advanced Technology Research, Chiuan-Yan Tech. Co., Ltd., Changhua County, 522, Taiwan, R.O.C., under NO F of university and industry liaison system. REFERENCES [1] M. Tichem, M. S. Cohen, "Subμm Registration of Fiducial Marks Using Machine Vision" IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, (1994) [2] H. K. Nishihara, P. A. Crossley, "Measuring Photolithographic Overlay Accuracy and Critical Dimensions by Correlating Binarized Laplacian of Gaussian Convolutions" IEEE Transactions on Pattern Analysis and Machine Intelligence, 10, (1988) [3] X. Fernandez, J. Amat, "Research on Small Fiducial Mark Use for Robotic Manipulation and Alignment of Ophthalmic Lenses" Proc th IEEE International Conference on Emerging Technologies and Factory Automation, 2, pp (1999) [4] N. Guil, J. Villalba, and E. L. Zapata, "A Fast Hough Transform for Segment Detection" IEEE Transactions on Image Processing, 4, (1995) [5] S. K. Tsau, D. Y. Hong, H. W. Le, C. M. Chang, and C. H. Lin, Multiple Alignment Stage for the Automatic Precision Alignment System International Symposium on Computer, Consumer and Control, (2012) [6] Y. C. Lin, Y. Y. Chiu, H. W. Lee, B. Y. Jhan, and C. H. Lin, The Study of Automate Locate Special Fiducial Marks Proc. Sixth International Conference on Genetic and Evolutionary Computing, pp (2012) [7] J. Canny, "A Computational Approach to Edge Detection" IEEE Transaction on Pattern Analysis and Machine Intelligence, 8, (1986) [8] L. Ding, A. Goshtasby, "On the Canny Edge Detector" Pattern Recognition, 34, (2001) [9] R. C. Gonzalez, R. E. Woods, Digital Image Processing (Prentice-Hall, ed. 2, 2002) [10] F. A. Pellegrino, W. Vanzella, and V. Torre, "Edge Detection Revisited" IEEE transactions on systems, man, and cybernetics-part B: CYBERNETICS, 34, (2004) [11] Z. Hou, Q. Hu, and W. L. Nowinski, "On Minimum Variance Thresholding" Pattern Recognition Letters, 27, (2006) [12] F. Y. Shih, S. Cheng, "Automatic Seeded Region Growing for Color Image Segmentation" Image and Vision Computing, 23, (2005) [13] D. Mumford, J. Shah, Optimal Approximations by Piecewise Smooth Function and Associated Variational Problems, S.R.S. Varadhan, Ed. (Wiley, New York, 1989), vol. 42, pp [14] B. Robbins, R. Owens, "2D Feature Detection via Local Energy" Image and Vision Computing, 15, (1997) [15] T. Miroslav, H. Mark, "Fast Corner Detection" Image and Vision Computing, 16, (1998) [16] M. Z. Brown, D. Burschka, and G. D. Hager, "Advances in Computational Stereo" IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, (2003) [17] N. Otsu, "A threshold selection method from gray-level histograms" IEEE Transactions on Systems, Man, and Cybernetics, 9, (1979) [18] D. Scharstein, R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms" International Journal of Computer Vision, 47, 7-42 (2002) [19] S. A. Hojjatoleslami, J. Kittler, "Region Growing: A New Approach" IEEE Transaction on Image Processing, 7, (1998) 93