visual tasks. For example, a set of annular operators can be used to identify enclosed symmetry points, and then a grouping algorithm is applied to re

Transcription

1 Detection of regions of interest via the Pyramid Discrete Symmetry Transform Vito Di Gesu and Cesare Valenti 1. Introduction Pyramid computation has been introduced to design ecient vision algorithms [1], [2] based on both top-down and bottom-up strategies. It has been also suggested by biological arguments that show a correspondence between pyramids architecture and the mammalian visual pathway, starting from the retina and ending in the deepest layers of the visual cortex. This paradigm of computation can also be related to the work made by Pomerantz and Sager [3] in their study on visual perception; the authors describe the visual perception as a transition in which the attention goes from global to local features, (olystic phase), and from local to global features, (analytic phase). In [4] Navon synthesises in a phrase this mechanism of perception: Are we seeing the forest before trees or trees before the forest? the author supports the precedence of global features in visual perception (see gure 1.1). Pyramid computation has suggested both new data structures (quad-trees, multiresolution), and new machine vision architectures (PAPIA) [5]. The concept of irregular pyramid has been introduced in [6] to handle connectivity problems that can arise when spatial data are mapped through the pyramid layers. Fig Multiresolution and level of perception. The implementation of local operators in a pyramid structure allows to detect image feature at dierent level of details. Aim of this paper is to study the capability of symmetry operators to detect regions of interest in a pyramid environment. Symmetry plays a remarkable role in perception problems. For example, peaks of brain activity are measured in correspondence with visual patterns showing symmetries. Relevance of symmetry in vision was already noted by psychologists [7]. Symmetry operators have been included in vision systems to perform dierent

2 visual tasks. For example, a set of annular operators can be used to identify enclosed symmetry points, and then a grouping algorithm is applied to represent and to describe object-parts [8]; axial symmetry properties have been applied to perform image segmentation [9]. Here an algorithm, that computes the pyramid Discrete Symmetry Transform (P DST ) of a digital scene, is presented. It has been implemented on the regular pyramid, and applied to a multiresolution scene representation. The hierarchy of symmetries is stored by starting from the rst level on which signicant local symmetries are detected. The choice of the top layer is based on a theoretic result, discussed in the paper, which sets a criterion to maintain interesting image features. Several experiments on real complex scenes have been performed to show the performance of the proposed approach. In Section 2 general concepts and denitions of pyramid computation are given; Section 3 describes a new symmetry operator; Section 4 shows the pyramid algorithm; experimental results are shown in Section 5; conclusion are given in Section Pyramid computation Let D be a digital image, of dimension n n, dened on the set of gray values G. The pyramid representation of D, is a triple < P D; F; V >, where: a) P D is an ordered sequence of images (D 0 ; D 1 ; :::; D r ; D L?1 ), of decreasing sizes. The rule of decimation is determined by the spatial mapping dened in b). b) F : Dr k?! D r+1 is the spatial mapping between two consecutive layers, with 0 r < L? 1. The inverse mapping allows to map an element of D r+1 with a subset of elements of D r : F?1 : D r+1?! Dr k, with 0 < r L? 1. In the case of a 2 2 pyramid the function F generates the usual quadtree, and the dimension of D r is 2 L?r?1 2 L?r?1, for r = 0; 1; :::; L? 1. Fig The regular pyramid. Spatial mapping characterises the hierarchy and the topology of a pyramid. Figure 2.1 shows the case of the 2 2 pyramid also named regular pyramid.

3 However, regular pyramid performs a partition of the digital space that may cause edge and connectivity problems. Other pyramid topologies have been proposed in order to minimize such eects. For example, gure 2.2 shows the spatial mapping for the 3 3 and the DUAL pyramids. Fig Examples of F-functions. c) V : Dr k?! D r+1, with 0 r < L? 1, is the gray level mapping, such that if F?1 (x) = (x 1 ; x 2 ; :::; x k ) is the spatial mapping for a pixel x 2 D r+1 then its gray level g r+1 (x) = V (g r (x 1 ); g r (x 2 ); :::; g r (x k )). V?1 : D r+1?! Dr k is the inverse function, such that if F (y) = x then g r (y) = V?1 (g r+1 (x)). Examples of gray level mapping functions are: max, min, _, &, xor, average. For D D 0 the pyramid P D is computed by the recursive application of F and V. In the following, the whole transition from D r to D r+1 is indicated by D r+1 = }(D r ) then P D = } L?1 (D 0 ). 3. Symmetry An object is said to exhibit symmetry if the application of certain isometrics, called symmetry operators, leaves it unchanged while parts are permuted. The letter A, for instance, remains unchanged under reection, the letter Z under half-turn, and the letter H under both reection and half-turn, the circle has circular symmetry around its centre. Moreover, an object in a 2D space exhibits a symmetry with respect to an axis x, if x divides the object in two mirror-like components. In [10] a new symmetry transform, named the Discrete Symmetry Transform (DST ), has been introduced. The DST algorithm has been applied to guide the attention of a robot visual system in real scenes. A comparison with other techniques, currently used to compute local symmetry [Reisfeld95], has given coherent results and faster computation. The DST of a digital image D is computed as the product of two local operators: DST (D) = S(D) E(D). The rst operator is function of the axial moments computed in a circle, C R, of radius R and centred in each pixel x (i; j) of D: T k (i; j) = P +R r=?r ; P +R s=?r k k jr sin( N )? s cos( N )j g i;j

4 with k = 0; 1; 2; :::; N?1. In the discrete retina N is function of R. The denition of S depends on the kind of symmetry to be detected. For example, in case of circular symmetry: s P P 2 k S i;j = 1? (T k(i;j) 2 k n? (T k(i;j) n The second operator, E, weights S according to the local smoothness of the image, and it is dened as: E i;j = P (l;m)2c R;(r;s)2C R+1 jg l;m? g r;s j where C R and C R+1 are centred in (i; j). Moreover, the condition (l? p) 2 + (m? q) 2 = 1 (4? connectivity) must be satised. It is easy to see that E i;j = 0 i the image is locally at. The DST is invariant for image size and rotation. The choice of the kernel radius depends on the area of the objects in D. Note that, for a given a kernel size, the computation returns zero values only on uniform zones. The computation of the DST, for increasing values of R, is related to skew local symmetry operators based on the Medial Axis Transform. The evaluation of signicant zones depends on the probability distribution of the intensity levels in the transformed image, and it can be performed by using conventional tests of normality. 4. Pyramid symmetry In this section the pyramid DST, P DST, algorithm is described. The P DST can be performed following two paradigms of computation: Direct computation In this case the computation is done directly on the pyramid } L?1 (D 0 ), by using pyramid symmetry kernels: P S fs 0 ; S 1 ; :::; S L?1 g and P E fe 0 ; E 1 ; :::; E L?1 g: P DST (P D) = P S(P D) P E(P D) the operator indicates: DST (D r ) = S(D r ) E(D r ) for r = 0; 1; :::; L? 1. Indirect computation In this case the pyramid of the DST (D 0 ) is built: P DST (P D) = } L?1 (DST (D 0 )) The rst approach requires to set the layer where to stop the computation, in order to obtain meaningful results. On the contrary, the second one requires to set the layer where to start the computation. It must be noted that in general }(DST (D r )) 6= DST (}(D r )). In the following, a condition of approximate commutativity between direct and indirect computation is given. This result allows to choose the best layer, k, where to start the computation with the indirect PDST-algorithm. The time complexity is usually reduced. For example, in the case of a regular pyramid the heavy computation

5 is performed only at the layer k, size of which is 2 L?k?1, followed by the propagation of the result to the layer 0 which is of the order O(2 L )). Therefore the whole complexity becomes O(4 L?2k R 2 ) instead of O(4 L R 2 ). Theorem 1. Let DF T (D 0 ) be the Discrete Fourier Transform of image D 0, and max (0) = max = maxf x ; y g its highest signal frequency, then the DF T (D k ), maintain max i the sampling size, d, at layer 0 satises the following relation: d 1 4 k max. Proof. The proof derives from Shannon's sampling theorem. In fact, in case of a 2 2 pyramid the layer r + 1 is a sub sampling of layer r (see gure 4.1). Therefore the size of d must satisfy the inequality: d 1, where max (r) is the 4 max (r) maximum frequency value at the layer r. The application of this relation, starting from layer 0, k-times brings to the relation: d 1 4 k max. Fig Sampling rule in a 2 2 pyramid. This property indicates the layer k =?blog 4 (d max )c from which to start or where to stop the computation. In practical cases d can become too small (over sampling) and a good compromise can be found by taking the maximum of the most signicant frequencies in the DF T (D 0 ). Theorem 2. If the sampling condition of Theorem 1 holds, and the gray level mapping is the mean value then: } k?1 (S 0 (D 0 )) = P S(} k?1 (D 0 )) Proof. First of all let us show that: }(S r (D r )) = S r+1 (}(D r )). In fact if P +1 q=0 D r?1(2i + p; 2j + q) it follows that: D r (i; j) = 1 4 P +1 p=0

6 Direct Computation: P S r+1 (i; j) = 1 +1 P +R 4 p;q=0 l;m;?r S l;md r (2i + 2l + p; 2j + 2m + q) Indirect Computation: S r (i; j) = P +R l;m=?r S l;md r (2i + l; 2j + m) P S r+1 (i; j) = p;q=0 S r(2i + p; 2j + q) = P 1 +1 P +R 4 p;q=0 l;m=?r S l;md r (2i + l + p; 2j + m + q) But P +R l;m=?r S l;md r?1 (2i + 2l + p; 2j + 2m + q) is the sampled version of P +R l;m=?r S l;md r?1 (2i + l + p; 2j + m + q). Therefore, for a given k, satisfying the condition of Theorem 1, it can be stated that: } k?1 (S 0 (D 0 )) = P S(} k?1 (D 0 )). Unfortunately, because of the 4? connectivity condition, the operator P E do not satisfy fully the commutativity property, therefore direct and indirect pyramid computations of the P DST are approximately commutative: } k?1 (DST (D 0 )) P DST (} k?1 (D 0 )) 5. Experimental results The experiments have been performed considering a regular pyramid, and the P DST has been computed in the case of circular symmetry. The P DST has been applied to the identication of zones of interest in real images under natural conditions of illumination. The algorithm can be sketched as follows: 1) compute the DF T of D; 2) choose the frequency max ; 3) set the level k =?blog 4 (d max )c; 4) compute DST k ; 5) select areas of interest at level k; 6) propagate the result in the pyramid base D 0. The images used in the experiments have been collected by a camera or by a scanner. Signal frequencies, with power above 3 from the mean value, have been considered. The image in gure 6.1a has been considered in order to test the capability of the method to detect heads in a crouwd. In this case: d max = 0:027, and k = 2. Figure 6.1a,b show the input image D 0 and the layer D 2. Figure 6.1c,d show, the DST (D 0 ) and the } 2 (DST (D 2 )) respectively. Figure 6.1e,f show the areas of interest as detected by the direct and the indirect computations respectively. The results of a second experiment are shown in gure 6.2a,b. In this case: d max = 0:03, and k = 2. The selected zones are centered on the people and the books in the room. Both experiments show that a good agreement exists between the areas of interest found by the direct computation of the DST and its indirect computatation via the P DST. The CPU time on a MHz for an image of size 256 was of about 3:5sec for the direct computation of the DST and 0:21sec for the computation via the P DST.

7 6. Conclusion The pyramid version of the Discrete Symmetry Transform has been described; theoretic conditions, based on Shannon's sampling theorem, let us set the approximated equivalence between the direct and indirect computation of the P DST. This allows to determine the condition under which to use the indirect version of the P DST. Experimental results, performed on complex scenes, conrm the validity of the theoretic prevision. Further work will be done in order to compare the implementation of the proposed method on dierent kind of pyramid topologies. References 1. L.Uhr, L.: Layered Recognition Cone Networks that Preprocess, Classify and Describe. IEEE Trans.Comput., C-21, Pavlidis T.A., Tanimoto S.L.: A Hierarchical Data Structure for Picture Processing. Comp.Graphycs & Image Processing, Vol.4, Pomeranzt J.R. and Sager L.C.: Asymmetric integrality with dimensions of visual pattern. Perception & Psycophysics, Vol.18, , Navon D.: Forest befor trees: the precedence of global features in visual perception. Cognitive Psychology, Vol.9, , Cantoni V., Di Gesu V., Ferretti M., Levialdi S., Negrini R., Stefanelli R.: The Papia System. Journal of VLSI Signal Processing, Vol.2, , Kropatsch W.G., Building Irregular Pyramids by Dual Graph Contraction. Technical Report PRIP-TR-35, Institute f. Automation 183/2, Dept.for Pattern Recognition and Image Processing, TU Wien, Austria, Kholer W. and Wallach H.: Figural after-eects:an investigation of visual processes. Proc. Amer. Phil. Soc., Vol.88, , Kelly M.F. and Levine M.D.: From symmetry to representation. Technical Report, TR-CIM-94-12, Center for Intelligent Machines. McGill University, Montreal, Canada, Gauch J.M. and Pizer S.M.: The intensity axis of symmetry application to image segmentation. IEEE Trans. PAMI, Vol.15, N.8, , [Reisfeld95] Reisfeld D., Wolfson H., Yeshurun Y, Context Free Attentional Operators, the Generalized Symmetry Transform. Int. Journal of Computer Vision, Vol.14, , Di Gesu V. and Valenti C., Symmetry Operators in Computer Vision. Vistas in Astronomy, Elsevier Science, in press.

8 Fig a) The input image D 0; b) the DST (D 0); c) the image D 2; d) the }?2 DST (D 0; e) zones of interest obtained via direct computation; f) zones of interest obtained via indirect computation. Fig a) Zones of interest obtained via direct computation; b) zones of interest obtained via indirect computation.