Invertible Orientation Bundles on 2D Scalar Images

Size: px

Start display at page:

Download "Invertible Orientation Bundles on 2D Scalar Images"

Kerrie Harper
5 years ago
Views:

Invertible Orientation Bundles on 2D Scalar Images Stiliyan N. Kalitzin, Bart M ter Haar Romeny, Max A Viergever Image Science Institute, Utrecht University Abstract.

1 Invertible Orientation Bundles on 2D Scalar Images Stiliyan N. Kalitzin, Bart M ter Haar Romeny, Max A Viergever Image Science Institute, Utrecht University Abstract. A general approach for multiscale orientation analysis of 2D scalar images is proposed. A scale-dependent orientation bundle (map of the visual space into function of two arguments: position and orientation) is constructed from the local Gaussian-derivatives jet of a scalar image in 2D. It is shown that there exists a class of orientation filters exhibiting an invertible relation between the orientation bundle and the original image in space domain. This invertible transformation is used to regain the original acuity in the spatial domain after analyzing orientation features at any given scale. The approach turns out to be highly effective for the detection of elongated structures. Keywords" Integral Transformations; Orientation Analysis; Scale Space; 1 Introduction Local (differential) multi-scale analysis of image structure has now been well understood within the frameworks of the linear scale-space theory [2, 4]. Point neighborhoods can be studied at a given scale by the local jet of Gaussian derivatives of a scalar image. On the other hand, for the detection of extended, e.g. elongated structures the analysis can appropriately include orientation parameter [91. The goal of this paper is to link the local Gaussian jet with the concept of oriented filtering. Therefore, we construct an orientation bundle over the visual space of a scalar image. Such a bundle is a function of the couple (position,orientation) defined over the space of the original image field. At every spatial point this bundle carries information about the orientation features of the original image in the given neighborhood. Orientation is clearly a multi-scale concept, which we will fully exploit in this paper. Also, it is well known that the human visual front-end has a pronounced and precisely mapped orientation selectivity in the cortical column structure of the primary visual cortex. Recently, much attention has been given to steerable filters to include the orientation parameter in the analysis [6, 1, 8]. In these approaches, however, the field dependence on the local orientation angle is limited. The construction of the orientation bundle is achieved by a linear integral transformation with an oriented filter of finite aperture. It includes also a scale parameter which measures the linear size of the region around a given point that contributes to the orientation function. The larger the scale parameter, the larger are the "receptive fields" of the bundle points. Multi-scale orientation analysis has been introduced before by different authors [7, 5].

2 78 An important result is that a class of invertible orientation filters can be constructed. This means that the system of all position and orientation values can be used to reconstruct the original image, despite the amount of spaceblur caused by the aperture filters to construct the orientation bundle. In effect, the spatial information is encoded into the orientation dimension and can be retrieved with an inverse projection. This type of orientation "encoding" might contribute to understand both the role of orientation columns and the hyperacuity phenomenon in biological visual systems. If the visual system needs some amount of blurring in order to provide a mechanism for sensory grouping or object segmentation, it has to include a way to restore (at least approximately) the original acuity. The inverting property of our transformation puts it in a close association with the wavelet methods, and especially with the 2D Gabor wavelet model [3] (see also the extended reference list there). The last model addresses the same fundamental questions about the exchange between spatial and angular information. An essential difference between our approach and the wavelet one is that the last uses all scales to invert the filter, while in our case we can invert the local bundle for each scale separately. Multi-scale analysis provides a systematic way to explore global, i.e. nonlocal structures in images while still using local techniques applied at larger scale. The introduction of proper orientation degrees into the image provides a way to detect and possibly segment extended structures as curves, ends, curve bifurcations etc. Nonlinear differential invariants defined on both spatial and angular derivatives highlight information for the neighborhood around given image points at the desired scale, which may in cases turn out to be large. Invertibility, on the other hand, can be used to restore the original spatial configuration of the detected structures. In the paper we first give a general definition of linear orientation bundles and introduce a class of invertible orientation filters. Section 3 focuses on the special multi-scale orientation bundle as derived from the local Gaussian derivative jet. In section 4, the invertible Gaussian bundle is applied to the task of segmenting a thin Catheter in a noisy medical fluoroscopy image. In the last section we discuss some practical and conceptual issues concerning the use of our approach. The notations and conventions used in the paper are summarized in the Appendix. 2 Local orientation bundle. Invertibilityo 2.1 Orientation filtering. An orientation bundle defined on R d is a function (see also the Appendix) F(x,8) : R d S d-1 -~ R (1) where x indicates all spatial coordinates and 8 indicates all angular parameters defining a local orientation in R d. In addition, we will require that the bundle is a (infinitely dimensional) representation of the SO(d) rotational group in R d

3 79 acting both on the spatial arguments x --+ R(a)x, where R(a) is a d d matrix from the vector representation of SO(d), and on the angular arguments 0 -+ O-a when rotating over an angle a. In this work we consider only the two dimensional case. Let L(xi), i = 1, 2 be a gray scale image defined on R u, then we can define a general orientation linear bundle with the help of an oriented aperture function (filter) ~b(x, 0). Such functions, especially the so-called steerable filters [6, 1, 8], have been used for local orientation analysis. An oriented filter is constructed from a given scalar function r and its rotations over all angles 0. Explicitly, if we decompose r into SO(2) irreducible representations r 1 v~r n (2) we have for the rotated function r 1 ~r (3) r o) - v~ n We call this last construction (3) orientation covariance. Now we can define the linear orientation bundle F(x, 9) derived from the image L(x) as: F(x, O) = JR 2 d2x'r ', O)L(x + x'). (4) If we decompose the so constructed bundle F(x, O) into harmonic modes as (for notations and normalization conventions see the Appendix): F(x, ~9) - 1 F,,(x)d% (5) --j then Fn(x) = f d2x'+n(x')l(x + x'). (6) where (see also (3)) ~=(x) -- r -'=~ (7) are the angular momenta of the oriented filters. It is clear that all possible linear orientation bundles associated with an image L(x) are parameterized by the (in general infinite) set of of radial functions r ncz. Note that the covariance condition (7) implies in complex notations (see the Appendix) the following generic form of the filter harmonic momenta: where r~(z=)llzll ij<l --- r 9 r e) = e'~v,~(z~), r e) = znv_n(z~), n > 0 (8)

4 80 Alternatively, we can consider the integral transformation (4) in the frequency domain. Then for the Fourier amplitudes, using the convolution properties of the Fourier integrals we have (see the Appendix for the definitions and the normalization conventions): For the (Fourier transformed) angular momenta, we get: (9) ~,~(~) -- 2~,~,~(-~)/_,(~). (10) Here ~,,(w) are the angular momenta of the aperture functions in the frequency representation. It is easy to show that the covariance condition (7) holds also in the frequency domain: ~,~(~o) = ~n(llo, lt)e -~'~~ I1~oll ~ ~'Odl 2 -I- 0J22; 0 w ~--- Arg(wl, co2), (11) where $.(11~Ii) are arbitrary radial-frequency functions. The last can serve as an alternative set of filter-generating functions. 2.2 Restoration of the image from the bundle. Invertibility of the construction (4) means existence of a transformation of the orientation bundle F(x, 0) to the space of ordinary functions in R2: M(x) = / d2x'dor ', O)F(x + x', 0 + 7r). (12) such that it restores the original image M(x) =_ L(x). (13) Here ~b(x, 0) is an oriented aperture function defined in the same way as r 0) in (2). We can look at eq. (12) as an inverse transform of (4) and therefore we have introduced the +Tr angular shift in the last term in (12). This choice is purely conventional. From the harmonic expansions (3) and (5) we can express the field M(x) alternatively as M(x) = ~ / d2x'(-1)n~_,~(x')fn(x + x'), (14) where ~(x) - Cn(px)exp(-inO~) are the harmonic momenta of the aperture function r 0) as in (7). The factor (-1) n comes from the r replacement in (12). Invertibility condition (13) imposes a constraint on the set of functions r (p) and Cn (p). The simplest way to obtain this constraint is to consider a 2D Fourier transform of all 2D fields. Then formula (14) takes the form:

5 81 /~/(w) = 27rE(-1)n~_n(-w)Fn(W). (15) n From (15), (10) and the conventions from the Appendix we obtain the invertibility condition in the frequency domain: 1 = (27r) 2 E (-1)n~-~(-w)~n(-w)" (16) n This equation must hold for all w. Equation (16) contains the Fourier spectral components of the aperture harmonic functions ~bn(x) and k~(x). In terms of the "filter generators" Cn([IW]J) and (bn(ilwll) defined in (11) the condition (16) becomes: 1 = (2r 2 E (17') n 3 Gaussian orientation bundle 3.1 Definitions and construction. An important example of an orientation bundle with scaling properties can be obtained from (8) by taking rn(z, = an (18) where constants an = a:n are still arbitrary and the Gaussian kernel g(z, 2, a) is defined in (32). This choice provides a local orientation bundle constructed entirely from the components of the Gaussian derivative jet. Indeed, according to the definitions and notations of the Appendix, we have t Z z5 z~ ~n(x,g) = an(-~)ne -~--~ ~ an(--grc~z)ne -7 (19) for n > 0, and the complex conjugates for n < 0. We can substitute (19) in equation (6) and then introduce L(z, 2, G) as the convolution of the original image L(x) = L(z, 2) with the Gaussian kernel g(z, 2, cr). After partial integration of the derivatives we get: Fn(x, = an( bzp L(z, 2, (20) for n > 0, and the complex conjugates for n < 0. The n = 0 term in (20) is the original image blurred with a Gaussian kernel of scale G. Thus, for any choice of the numerical coefficients a,, for which the formal series make sense, we obtain a local orientation bundle that can be expressed entirely in terms of Gaussian derivatives of the original image at scale a.

6 82 We turn now to the question of inverting the transformation (20). In other words we want to reconstruct the original image L(x) from its orientation Gaussian bundle. Let us assume that the candidate for the inverse transform (12) is defined by functions ~n(x, ~) similar to (19): -- z2 z~ en(x,a) = bn(-~)ne ~ =- b~(-o'oz)ne -~ (21) Here bn are complex constants, in general different from an. Given the definition of the harmonic coefficients (19) and (21) as Gaussian derivatives, we can substitute (34) into the condition (16). Assuming that the forward and inverse transformation are taken with two different scale parameters (~1 and a2 correspondingly, we obtain: 1 = (aobo + ~ (anbn + gnbn)(ala2"~')n)e -vf'(~+~). (22) n>0 Let us now consider the choice for the parameters an and bn so that M(x) = L(x) when al = ~2. One (symmetric) option is a0=b0=l;an=bn=v 21hi-1 ~-Tn-~.T 'n<>~ (23) Indeed, with the chose (23) for the numbers an, bu we obtain by summing the powers of up 1 = e2~l~2"pe -'p(~+~) - e -(~1-~2)2"p (24) when ~1 = a2. Note that equation (24) implies: M(x, o'1, a2) = L(x, ),) (25) where ), = Itr - a211- Accordingly, for any or2 < ~1 a deblurring effect will be present. This effect is maximal when the two scales are equal and (if all harmonic frequencies are used) the initial image will be reconstructed. The symmetric choice of the coefficients (23) corresponds to a self-invertible bundle. From (25) we also see that )~ _> 0 for any choice of the two scales. Therefore, a "deblurring" beyond the original acuity is impossible which is in accord with the well-posedness arguments. 3.2 Examples. In figure 1 we present the aperture function r defined by (18) and (23). The orientation bundle (1) is obtained from a single bright spot (frame 1 in fig. 1). The resulting field (or the "impulse response function") r 45 ~ is shown in the second frame in fig.1 for angle 0 = 45 ~ In frames 3 and 4 we present the power spectra for the original image and the aperture function. The picture shows clearly the idea of spatial versus angular exchange of information that is realized. In terms of 2D frequency amplitudes, the oriented bundle taken for a fixed angle

7 83 performs a band-pass filtering in the spatial direction aligned with this angle. In other words, if we define "longitudinal" and "transversal" coordinates as xe = xlcos(o) + x2sin(o) and x~ = -xlsin(o) + x2cos(o) correspondingly, then for large scales we have O~eF(xe, x~, 0, ~) ~ O. In figure 2 the invertibility property is illustrated for four different scales (al = a2) taken for the Gaussian invertible bundle. We see that even when the zero-mode blur becomes significant with increasing scale (second row in figure 2), the invertibility of the orientation bundle ensures the proper restoration of the original image (third row in the same figure). Fig. 1. Frame 1:128x128 image with a bright spot of size of two pixels. Frame 2: Gaussian orientation filter response to the image of frame 1 and simulated for scale G = 2.0 and angle 0-45 ~ Frame 3,4: 2D power spectra for the images in frames 1 and 2 respectively. 4 Applications The main motivation for our analysis has been a specific class of visual tasks that are difficult to approach with other schemes. These tasks involve detection and segmentation of extended, i.e non-local sub-dimensional structures. Such structures are, for example, curves in two-dimensional images and curves and surfaces in 3D images. In some cases these objects are defined as borders between two (or more) volumes (or areas in 2D cases). Other images may contain thin, elongated structures that do not separate any distinctive volumes. Such a case is shown in figure 3, frame I where a catheter guide is visible as a thin (on places fuzzy) long structure. To solve the problem of finding extended objects, we must bring in topological knowledge in order to identify the features of these objects. Such feature identification can conveniently be accomplished by a nonlinear local detector (invariant) over the orientation bundle. We can proceed according to the following scheme. - Compute the field F(x, 0, or) from (4), (8),(18),(23) and for the selected scale O'.

84 Fig. 2. Frame 1: Image with two vertical bright bars of dimension 30xl pixels. Second row frames (2-5): zero harmonic mode (n=0) of the orientation bundle at scales 0.5, 1.0,2.0 and 3.

8 84 Fig. 2. Frame 1: Image with two vertical bright bars of dimension 30xl pixels. Second row frames (2-5): zero harmonic mode (n=0) of the orientation bundle at scales 0.5, 1.0,2.0 and 3.0 and the angular resolution (number of sectors taken on the circle) is 256. Third row of frames (6-9): restored images at the four scales. - Compute an appropriate non-linear local detector over the orientation bundle. For detecting long, string-like structures we used o I(x, 0, ~) = r(x, 0, ~)(-sin(0)btx~ + cos(0) )~F(x, O, ~)ggf(x, O, ~). (26) The two second derivatives terms in this product take high values near thin, elongated structures. - Project the bundle I(x, O, ~r) back by (12) with functions (21) and the same scale parameter. The back projection is necessary in order to restore the original spatial resolution. In fig. 3 we compare the results without inverting (second frame) and those after inverting (third frame). The non-inverted image is taken as integral of I(x, O, or) over all angles. In the last frame of fig.3 we superimpose the detected structures on the original image and we verify their exact location as a result of the inverse transform. One of the features of the above method is that it can be used at any scale. The invertability constraint (16,22) is valid for any (r and therefore we can use our scheme at several spatial scales independently. This enables us to detect structures of different thickness as shown in figure 4. o2

Frame 3: Inverse projection of the bundle defined by the invariant. Frame 4: Original image is superimposed with the result from frame 3. Fig. 4. Frame 1: Noisy (low dose) fluoroscopy image (512x512 pxls.

9 85 Fig. 3. Frame 1: Noisy (low dose) fluoroscopy image (128x256 pxls) of a thin catheter guide wire. In the lower left corner a broader vessel can be seen. Frame 2: Orientation analysis with the non-linear invariant (26) (scale a = 3.0, angular resolution of 256 sectors) and summed over all orientations (sectors). Frame 3: Inverse projection of the bundle defined by the invariant. Frame 4: Original image is superimposed with the result from frame 3. Fig. 4. Frame 1: Noisy (low dose) fluoroscopy image (512x512 pxls.) of a thin catheter guide wire. Frames 2 and 3: High-lightened elongated structures as in figure 1, frame 3 but for scales cr = 0.8 and cr = 3.0 correspondingly. 5 Discussion. Our construction provide s a geometrical interpretation of the local derivative jet of a scalar image by organizing its components into a local orientation bundle. This bundle contains a scale parameter which measures the extent to which we collect the neighboring information in a given direction around every point. It is clear that with increasing value of the scale parameter, the spatial resolution will decrease. In our construction this is compensated by an increase of the angular resolution. Indeed, Fn '~ a '~ so higher harmonics increase stronger with increasing scale. The example with the Gaussian orientation bundle involves an infinite num-

86 ber of angular momenta (the sum over n in (15)) in order for the inverse transformation to be exact. Truncation of the harmonic expansions, however, has a mild effect on the property (13).

10 86 ber of angular momenta (the sum over n in (15)) in order for the inverse transformation to be exact. Truncation of the harmonic expansions, however, has a mild effect on the property (13). Indeed, it is easy to show that the "departure" from the exact invertibility in the truncated case has a leading term: (-1) N+I M(x,(y,(y)- L(x) - (N + I)! (~r2a)n+il(x, V~a)-t -... (27) 1. where A _= _ z0~ is the Laplaeian operator and N is the highest harmonic component considered. To illustrate this truncation sensitivity, we consider in figure 4 the same example as in figure 2 and restore the original image for different values of the truncation parameter N. Fig. 5. For the same original image as the first frame in figure 2, the restored images for four different harmonic truncations with N=8, 16, 32 and 64 (corresponding to angular resolutions of 16, 32, 64 and 128 sectors) and fixed scale a = 2.0 are shown from left to right. A more practical approach can be based on a finite-dimensional representation of the rotational group, e.g. on steerable filters. Equation (16) can be solved in terms of a finite number of radial aperture functions. The price paid for this finite number is worse than the Gaussian low-pass filtering properties. Finally, we discuss the choice (26) for a non-linear detector of elongated structures. It is clear that if a test-point is positioned on a high-luminosity ridge, the oriented response will be maximal in the direction of the ridge. This, however, is not sufficient to filter only the elongated, or ridge-type of structures. What we need in addition is a term that suppresses "massive" objects and enhances structures with only sub-dimensional support. The two second derivatives in (26) do exactly the required function. They measure "ridgeness" in the angular and transversal direction and give therefore a proper multiplieative modulation to the orientation bundle. The non-linear character of the invariant (26) should be considered as a filtering capacity that is added to the linear transformation (4). 6 Appendix: Notations and conventions. In this Appendix we give some definitions and conventions concerning the formulas in the previous sections. We use below the terms "orientation" and "direction" indiscriminately.

11 87 - Euclidean rotational groups and direction parameters In a d-dimensional Euclidean space R d the (connected) orthogonal rotational group is SO(d). The unit sphere S d-1 represents the set of all spatial SO(d) directions and is isomorphic to the factor-space ~. Any function on the spatial directions is therefore a (infinite-dimensional) representation of SO(d) with stationary subgroup SO(d - 1). - Two-dimensional direction functions and harmonic decompositions. In two spatial dimensions, SO(l) ~- 1 and, therefore, a function on the local direction is just a function on the SO(2) -= U(1) group. As the last is an Abelian group parameterized with a single cyclic parameter, the following decomposition is valid: 1 ~,, in6 1 f~ f(o) ~ ~-- ~ ]ne,fn- j_ dof(~) -ine (28) V ATg n=--cx~ ~ ~: where fn = f-n are the complex harmonic coefficients. The sum over n in (28) is a sum over the irreducible U(1) representations contained in the function f(o). - Local orientation bundle By local orientation bundle we indicate in this paper a fiber bundle with a base manifold the space where the original image is defined and with a standard fiber the infinite-dimensional functional representation of the local rotation group as given in 2D by (28). In higher dimensional cases, the representation is defined as a function on the factor-space ~. The structural group of the fiber bundle is SO(d). - Fourier transformations and complex coordinate conventions. We assume the symmetric normalization conventions for Fourier transforms: 1 f f(x) - (27r)d/2 ] d f( )e 1 / dxf(x)e -~ (29) as well as for the complex vector notations: Z-~- -~(Xl "~ix2),1/' ~- ~(W 1 -Jr-i~2),0 z ~ 0z -- %,~(01 --i02) (30) and their complex conjugates. - Gaussian kernel and its derivatives. We assume the following parameterization for the Gaussian kernel in two spatial dimensions: 1 (~+~) 1 ~ -- g(x, 0") = 2~.o.2 e 2a 2 27ro.2 e = g(z, (31)

12 88 with the complex notations from (31). The n th complex Gaussian derivative is thus: g,~(x, o`) - Oz~g(x,o`) = (-k)~g(z,~,o`) (32) o-2 and an analogous formula holds for its complex conjugate. Finally, in the Fourier domain, the notations and conventions (30,31) lead to the following form of the Gaussian kernel and its derivatives (33): k ( tz/2r 2~ 2~T ~(~, p, o`) _ (ip)n~(~, ~, o`). (33) Acknowledgments: This work is carried out in the framework of the research program Imaging Science, supported by the industrial companies Phillips Medical Systems, KLMA, Shell International Exploration and Production, and ADAC Europe. References 1. William T. Freeman and Edward H. Adelson. The design and use of steerable filters. IEEE Trans. Pattern Analysis and Machine Intelligence, 13(9): , September J. J. Koenderink. The structure of images. Biol. Cybern., 50: , Tai Sing Lee. Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Analysis and Machine Intelligence, 18(10): , October T. Lindeberg. Scale-Space Theory in Computer Vision. The Kluwer International Series in Engineering and Computer Science. Kluwer Academic Publishers, Dordrecht, the Netherlands, M. Michalis. Low level image processing using steerable filters. Institute for Medical Informatics, Munich, PhD thesis. 6. P. Perona. Deformable kernels for early vision. In IEEE CVPR, pages , June E. P. Simoncelli. A rotation invariant pattern signature. In Proc. of the 3rd IEEE Int. Conf. on Image Processing. IEEE, to appear. 8. E. P. Simoncelli and H. Farid. Steerable wedge filters for local orientation analysis. IEEE Transactions on Image Processing, July to appear in. 9. Eero Simoncelli. A rotation-invariant pattern signature. In International Conference on Image Pproeessing, pages , 1996.

Invertible Apertured Orientation Filters in Image Analysis

International Journal of Computer Vision 31(2/3), 145 158 (1999) c 1999 Kluwer Academic Publishers. Manufactured in The Netherlands. Invertible Apertured Orientation Filters in Image Analysis STILIYAN