INSTITUT FÜR INFORMATIK UND PRAKTISCHE MATHEMATIK

Transcription

1 INSTITUT FÜR INFORMATIK UND PRAKTISCHE MATHEMATIK Structure Multivector for Local Analysis of Images Michael Felsberg and Gerald Sommer Bericht Nr Februar 2000 CHRISTIAN-ALBRECHTS-UNIVERSITÄT KIEL

2 Institut für Informatik und Praktische Mathematik der Christian-Albrechts-Universität zu Kiel Olshausenstr. 40 D 24098Kiel Structure Multivector for Local Analysis of Images Michael Felsberg and Gerald Sommer Bericht Nr Februar mfe@ks.informatik.uni-kiel.de Dieser Bericht ist als persönliche Mitteilung aufzufassen. This work has been supported by German National Merit Foundation and by DFG Graduiertenkolleg No. 357(M. Felsberg) and by DFG Grant So (G. Sommer).

3 Abstract The structure multivector is a new approach for analyzing the local properties of a two-dimensional signal(e.g. images). It combines the classical concepts of the structuretensorandtheanalyticsignalinanewway.thishasbeenmadepossible by using a representation in the algebra of quaternions. The resulting method is linear and of low complexity. The filter-response includes local phase, local amplitude and local orientation of intrinsically one-dimensional neighborhoods in the signal. As for the structure tensor, the structure multivector field can be used to apply special filters to it for detecting two-dimensional features like corners. Experiments and comparisons with other approaches have been made. 1

4 Contents 1 Introduction 3 2 A new approach for the 2D analytic signal Fundamentals The2Dphaseconcept Thesphericalanalyticsignal Properties of the Spherical Analytic Signal Thespatialrepresentation Thestructuremultivector Detectionofkeypoints Experiments and Discussion Implementationandtests Discussionofresults Futurework

5 1 Introduction In image and image sequence processing, different paradigms of interpreting the signals exist. Regardless of they are following a constructive or an appearance based strategy, they all need a capable low-level preprocessing scheme. One of the surely most discussed topics is the detection of key points, or points of higher intrinsic dimension (in the context of appearance based approaches one should better talk of representation than of detection). Key points are needed for solving the correspondence problem occuring in 3D reconstruction, for applying steerable filters(see e.g.[16, 20]), for all kinds of model based representations, etc.. Sincethepreprocessingisonlythefirstlinkinalongchainofoperations,itisuseful to have a linear approach, because otherwise it would be nearly impossible to design the higher-level processing steps. On the other hand, we need a rich representation if we want to treat as much as possible in the preprocessing. Furthermore, the representation of the signal during the different operations should be complete, in order to prevent a loss of information. These constraints enforce us to use the framework of geometric algebra which is also advantageous if we combine image processing with neural computing and robotics(see[18]). The representation of points with special intrinsic dimensionality needs a mathematical treatment of the relevant data, which means that structural information has to be separated from textures and noise. In the one-dimensional case, quadrature filters are a frequently used approach for processing data. But the standard extension to two dimensions is not very efficient because it closely depends on the preference direction of the filter. The structure tensor(see e.g.[13]) is a capable approach for detecting the existence and orientation of local, intrinsic one-dimensional neighborhoods. From the tensor field the orientation vector field can be constructed and by a normalized or differential convolution special symmetries can be detected(see e.g.[14]). The structure tensor itself can be computed with quadrature filters(see[10]) but the tensor itself does not possess the typical properties of a quadrature filter. Especially the linearity and the split of the identity is lost, because the phase is neglected. Inthisreport,weintroduceanewapproachforthe2Danalyticsignalwhichenables ustosubstitutethestructuretensorbyanentitywhichislinear,preservesthesplitofthe identity and has a geometrically meaningful representation: the structure multivector. 3

6 2 Anewapproachforthe2Danalyticsignal Inthefirstsection,werepeatsomefundamentals,inordertomakethenewconcepts easier to understand. In the following sections, we introduce the new approach for the analytic signal in two steps. Firstly, we define the coordinate system, in which we work in the sequel and which also follows directly from geometric constraints. Secondly, the spherical analytic signal is defined using the introduced coordinate system. 2.1 Fundamentals Sinceweworkonimages,whichcanbetreatedassampledintervalsof Ê ¾,weusethe geometricalgebra Ê ¼¾ whichisisomorphictothealgebraofquaternions À.Thewhole complex signal theory naturally embeds in the algebra of quaternions, e.g. complex numbers are considered as a subspace of quaternions here. The basis of the quaternions reads ½ whilethebasisofthecomplexnumbersreads ½. Normally,the basis vector ½ is omitted. Throughout this document, we use the following notations: incontrasttomultivectors,vectors(1dand2d)areboldface Ü Ü ½ Ü ¾, Ù Ù ½ Ù ¾, Ø Ø,etc.and2Dvectorshavethedirections ³ Ü, ³ Ù,etc. thefouriertransformofthe ÒDsignal ܵisdenoted ܵ Æ Ùµ Ê Ò Üµ ÜÔ ¾Ù ܵ Ü therealpart,the -part,the -part,andthe -partofaquaternion Õisobtainedby ÊÕ, ÁÕ,  Õ,and ÃÕ,respectively theautomorphismsofthequaternionsaredenotedby «Õµ Õ, Õµ Õ, and Õµ Õ,wheretheconjugationofthecomplexnumberscanbeexpressed eitherby Þµorby Þµ(Þ ÊÞ ÁÞ) The1Danalyticsignalcanbedefinedasfollows(seee.g.[13]).Thesignalwhichis obtainedfrom ܵbyaphaseshiftof ¾iscalledtheHilberttransform À ܵof ܵ. Since À ܵshallbereal-valued,thespectrummusthaveanoddsymmetry.Therefore, thetransferfunctionhastheform 1 À Ùµ Ò Ùµ (1) 1 Sinceweusevectornotationfor1Dfunctions,wehavetoredefinesomereal-valuedfunctionsaccordingto Ò Ùµ Ò Ùµ,where Ù Ù. 4

7 If we combine a signal and its Hilbert transform corresponding to ܵ ܵ À ܵ (2) wegetacomplex-valuedsignal,whichiscalledtheanalyticsignalof ܵ. Accordingto(1),theFouriertransformoftheanalyticsignal ܵislocatedinthe right half-space of the frequency domain: ܵ Æ Ùµ ¾ Ùµ Ò Ùµ Ùµ ¼µ ¼ if Ù ¼ if Ù ¼ if Ù ¼ Hence, we have recalled all facts of the one-dimensional analytic signal which are relevant for this report. More information can be found in any good textbook about signaltheory. Sinceweusethealgebraofquaternionsasageometricalgebra,wehavetointroduce some terms of geometric algebra(for a complete introduction, see e.g. [12]). If the algebraofquaternionsisidentifiedwiththegeometricalgebra Ê ¼¾,therealpartisalso calledthescalar,theentityconsistingof -and -partsiscalledthevectorandthe -part iscalledthebivector( ). Wehavealreadyusedtwoproducts:theinnerproduct,e.g. Ü Ù,andthegeometric product,e.g. Ù. Thegeometricproductisevaluatedbyapplyingtheassociativelaw and the multiplication table of the algebra. The inner product is well known for vectors, but it can be extended to elements of all grades(geometric algebras are graded algebras) by (3) Ö (4) where isthegradeoperatorand Ö and.theinnerproductwitha scalar is always zero. Theouterproductisdefinedby Ö (5) whichyieldstheidentity ÜÙ ÜÙ Ü Ùforvectors.Ifatleastonefactorisascalar,the outer product is identical to the geometric product respective the field multiplication. Arotationin2Dvectorspacecanbeexpressedbyaspinor,inthecaseofquaternions forexample,wehaveascalar,avector Ü Ü ½ Ü ¾ andabivector.together,they formthequaternion Õ Ü.Ifwewanttorotatethevector(andonlythevector) 5

8 by ³,thiscanbedonebythespinor ÜÔ ³¾µ: ÜÔ ³¾µÕ ÜÔ ³¾µ ÜÔ ³¾µÜ ÜÔ ³¾µ Ó ³¾µÜ ½ Ò ³¾µÜ ¾ µ Ó ³¾µÜ ¾ Ò ³¾µÜ ½ µµ ÜÔ ³¾µ Ó ¾ ³¾µÜ ½ ¾ Ó ³¾µ Ò ³¾µÜ ¾ Ò ¾ ³¾µÜ ½ µ ¾ Ó ³¾µ Ò ³¾µÜ ½ Ò ¾ ³¾µÜ ¾ Ó ¾ ³¾µÜ ¾ µ Ü ½ Ó ³µ Ü ¾ Ò ³µµ Ü ½ Ò ³µ Ü ¾ Ó ³µµ 2.2 The 2D phase concept Wewanttodevelopanew2Danalyticsignalforintrinsically1Dsignals(incontrastto the 2D analytic signal in[4] which is designed for intrinsically 2D signals), which shall include three properties: local amplitude, local phase and local orientation. Compared tothe1danalyticsignalweneedoneadditionalphase. Wecannotchoosethisphase withoutconstraints:ifthesignalisrotatedby,weobtainthesameanalyticsignal,but conjugated. Therefore, we have the following relationship: negation of the local phase isidenticaltoarotationof. One straightforward consequence is that points where the analytic signal has zero phase(i.e.itisreal-valued)haveorientation ³ Ù and ³ Ù atthesametime-therefore, these points have no orientation at all. Note the difference between direction and orientationinthiscontext;thedirectionisavaluein ¼ ¾µandtheorientationisavaluein ¼ µ(see[9]). Asecondconsequenceisthefollowing. Anyvalueofthe2Danalyticsignalcan beunderstoodasa3dvector. Theamplitudefixesthesphereonwhichthevalueis located.thelocalphasecorrespondstorotationsonagreatcircleonthissphere.tobe consistent,arotationofthesignalmustthencorrespondtoarotationonasmallcircle. An obvious parameterization is the following: assuming that we have a complexvalued function over a two-dimensional domain, we map the imaginary unit onto the normalized position vector(the direction vector). The resulting function is quaternionvalued and has the following properties: 6

9 Ifwechooseonefixedorientation ³ Ù intherighthalf-space,theimaginaryunitis changedinto Ó ³ Ù µ Ò ³ Ù µµ Ò Ù ½ µ. For every orientation, the function lies on a different complex plane(note that everyalgebraover ½ Õ ¼, Õ ¼ apureunitquaternion,isisomorphictothealgebra ofcomplexnumbers).thephaseinthisplaneisdenoted ³. The values of the function include information about the orientation of the function. Thecoordinatesystemdefinedinthiswayisdisplayedinfigure1. Itisthesame -part ³Ù -part ³ real part Figure 1: Coordinate system of the 2D phase approach asin[10],butgranlundandknutssonusethe2dphaseinthecontextoforientation adaptive filtering(see also[11]). Consider a given quaternion Õ, how can we extract the two phases, described just above? In fact, the answer of this question defines the introduced coordinate system. 7

10 Definition1Let Ö Þµbethefunctionwhichevaluatesthecomplexphaseof Þintheinterval ¼ ¾µandlet Õ beaquaternionwithzero part.then,wegettheorientationphase ³ Ù by ³ Ù Ö ÁÕ Â Õ µ ¾ µ¾ (6) andthephaseinthecomplexsubspacespannedby ½and Ó ³ Ù µ Ò ³ Ù µisobtainedby ³ Ö ÊÕ ÁÕ Â Õ µ ÜÔ ³ Ù µµ Ö ÊÕ Â Õ ÁÕ µ ÜÔ ³ Ù µµ (7) Note that these definitions of quaternionic phases are totally different from those of the quaternionic Fourier transform(see e.g. [4]). The reason why we use a different phasewillbeexplainedattheendofsection The spherical analytic signal Now,havingaphaseconceptwhichisrichenoughtocodealllocalpropertiesofintrinsically one-dimensional signals, we construct a generalized Hilbert transform and an analyticsignalforthe2dcase,whichmakeuseofthenewembedding. The following definition of the spherical Hilbert transform is motivated by theorem 1, which establishes a correspondence between the Hilbert transform and the spherical Hilbert transform. Definition2Let ÙµbetheFouriertransformof ܵ.ThenthesphericalHilberttransformof ܵisdefinedintheFourierdomainby À Ùµ À Ùµ Ùµ Ù Ùµ (8) Ù and À Ùµ Æ À ܵ. Notethattheoperator À Ùµhastobeappliedfromtheleft.Insection3.1,itwillbe shown that the components of the spherical Hilbert transform are identical to the Riesz transform. 8

11 Example 1: ܵ Ó ¾Ù ¼ ܵ À Ùµ À Ùµ Æ ¼ Ù Ù ¼ µ Æ ¼ Ù Ù ¼ µ ÙÆ ¼ Ù Ù ¼ µ ÙÆ ¼ Ù Ù ¼ µ ¾Ù Ù ¼Æ ¼ Ù Ù ¼ µ Ù ¼ Æ ¼ Ù Ù ¼ µ ¾Ù ¼ Ù Æ ¼ Ù ¼ ¼ Ù Ù ¼ µ Æ ¼ Ù Ù ¼ µ ¾ ¾ sothat À Ùµ Æ ÜÔ ³ ¼ µ Ò ¾Ù ¼ ܵ,where ³ ¼ Ö Ù ¼ µ. Example 2: ܵ Ò ¾Ù ¼ ܵ À Ùµ Ù Æ ¼ Ù ¼ ¼ Ù Ù ¼ µ Æ ¼ Ù Ù ¼ µ ¾ sothat À Ùµ Æ ÜÔ ³ ¼ µ Ó ¾Ù ¼ ܵ. Obviously, the spherical Hilbert transform yields functions which are identical to the 1D Hilbert transforms of the cosine and sine function, respectively, except for an additional exponential function. Up to now, we have only considered two special examples, but what about general signals? Whatkindofsignalscanbetreatedwiththisapproach? Theanswercanbe found easily: the orientation phase must be independent of the frequency coordinate. This sounds impossible, but in fact, the orientation phase is constant, if the spectrum is located on a line through the origin. Signals which have a spectrum of this form are intrinsically one-dimensional(i.e. theyareconstantinonedirection).thisisexactlytheclassoffunctionsforwhichthe structure tensor has been designed. For these functions we have the following theorem: Theorem1Let صbeaone-dimensionalfunctionwiththeHilberttransform À ص. Then, thesphericalhilberttransformofthetwo-dimensionalfunction ¼ ܵ Ü Òµµreads ¼ À ܵ Ò À Ü Òµµ,where Ò Ó µ Ò µisanarbitraryunitvector. 9

12 Proof: TheFouriertransformof ¼ ܵreads Ù ÒµµÆ ¼ ÙÒµif ص Æ µ.corresponding to(8)weobtain Ù Ù Ù ÒµµÆ ¼ Ù Òµ Ò Ò Ù Òµ Ù ÒµµÆ ¼ Ù Òµ andwecantaketheterm ÒintheinverseFouriertransformoutsidetheintegral: and therefore, Ò Ò Ù Òµ Ù ÒµµÆ ¼ Ù Òµ ÜÔ ¾Ù ܵ Ù Ê ¾ Ò Ê ¾ Ò Ò µ µ ÜÔ ¾Ò ܵ Ê Ò À Ò Üµµ Ò Ù Òµ Ù ÒµµÆ ¼ Ù Òµ ÜÔ ¾Ù ܵ Ù ¼ À ܵ Ò À Ü Òµµ ÜÔ µ À Ü Òµµ Nowwesimplyadapt(2)forthe2Dcaseandobtain: Definition3Thesphericalanalyticsignalofa2Dsignal ܵisobtainedby where À ܵisthesphericalHilberttransformof ܵ. ܵ ܵ À ܵ (9) Using this definition, we obtain for our two examples: Ó ¾Ù ¼ ܵ ÜÔ ³ ¼ µ Ò ¾Ù ¼ ܵ Ó ¾Ù ¼ ܵ Ù¼ Ù ¼ Ò ¾Ù ¼ ܵ ÜÔ Ù¼ Ù ¼ ¾Ù ¼ ܵ Ò ¾Ù ¼ ܵ ÜÔ ³ ¼ µ Ó ¾Ù ¼ ܵ Ò ¾Ù ¼ ܵ Ù ¼ and Ù ¼ Ó ¾Ù ¼ ܵ ÜÔ Ù¼ Ù ¼ ¾Ù ¼ Ü ¾µµ So the spherical analytic signal uses the phase concept, which has been defined in section 2.2. According to theorem 1, the spherical analytic signal of an intrinsically onedimensionalsignal ¼ ܵ Ü Òµµreads ¼ ܵ Ü Òµµ Ò À Ü Òµµ (10) 10

13 Of course, the spherical analytic signal can be computed for all functions which are Fourier transformable. However, for signals which do not have an intrinsic dimension ofone 2,thecorrespondencetothe1Danalyticsignalislost. Independently of the intrinsic dimensionality of the signal, the analytic signal can alsobecalculatedinadifferentway. The1Danalyticsignalcanbeobtainedinthe Fourierdomainbythetransferfunction ½ Ò Ùµ. Forthesphericalanalyticsignal wehavethesameresultifwemodifythefouriertransform. Definition4TheinversesphericalFouriertransformof Ùµisdefinedby ܵ ÜÔ ³ Ù ¾µ Ùµ ÜÔ ¾Ù ܵ ÜÔ ³ Ù ¾µ Ù (11) Ê ¾ Before we investigate the inverse of this transform and some of its properties, we establish a relation between the spherical analytic signal and the inverse spherical Fourier transform: Theorem2Thesphericalanalyticsignalof ܵistheinversesphericalFouriertransformof Ùµ. Proof: Ù TheinverseFouriertransformof ½ Ù µ Ùµcanbecalculatedbyanintegrationover Ù or Ù, the integrals are identical. Therefore, we have ܵ ¾ ½ Ù µ Ùµ ÜÔ ¾Ù ܵ Ù Ê Ù ½ ¾ ¾ ½ Ù µ Ùµ ÜÔ ¾Ù ܵ ½ Ù µ Ùµ ÜÔ ¾Ù ܵ Ù Ê Ù Ù Ù Ê Ùµ ÜÔ ¾Ù ܵ Á Ùµ ÜÔ ¾Ù ܵ Ù Ê ¾ Ù Ê ¾ ÜÔ ³ Ù ¾µ Ùµ ÜÔ ¾Ù ܵ ÜÔ ³ Ù ¾µ ٠ܵ 2 Thecaseofintrinsicdimensionzero(i.e.aconstantsignal)isirrelevant,becausetheHilberttransform iszeroinbothcases. 11

14 Above, we mentioned that the spherical analytic signal can be obtained by a similar transferfunctionasinthe1dcaseifthemodifiedfouriertransformisused.uptonow, we used the whole spectrum, but since ÜÔ ³ Ù ¾µ Ùµ ÜÔ ¾Ù ܵ ÜÔ ³ Ù ¾µ ÜÔ ³ Ù µ¾µ Ùµ ÜÔ ¾Ù ܵµ µ ÜÔ ³ Ù µ¾µ ÜÔ ³ Ù µ¾µ Ùµ ÜÔ ¾Ù ܵµ µ ÜÔ ³ Ù µ¾µ ÜÔ ³ Ù µ¾µ Ùµ ÜÔ ¾Ù ܵ ÜÔ ³ Ù µ¾µ i.e., the integrand is symmetric, we can also integrate over the half domain and multiply theintegralbytwo.therefore,wecanuseanytransferfunctionoftheform ½ Ò Ù Òµ withoutchangingtheintegral.themostworkablecasesarethosewhere Ò or Ò. By simply omitting half of the data, the redundancy in the representation is removed. Theinverseofthetransformdefinedby(11)isthetransformwhichtakesaspherical analytic signal and returns the classical complex spectrum. Since the original signal is simplytherealpartoftheanalyticsignal,wehave Ê Üµ Æ Ùµ (12) In order to calculate the energy of the spherical analytic signal, we firstly need the transferfunction,whichchanges Ùµto Ùµ:itisobtainedfrom(8)and(9)andreads Ù ½ Ù ½ Ó ³ Ùµ Ò ³ Ù µ.theenergyofthesphericalanalyticsignalis ½ Ó ³ Ù µ Ò ³ Ù µµ Ùµ ¾ Ù Ê ¾ Ê ¾ ½ Ó ³ Ù µµ ¾ Ùµ ¾ Ò ¾ ³ Ù µ Ùµ ¾ Ù ¾ ½ Ó ³ Ù µµ Ùµ ¾ Ù Ê ¾ andsince Ùµ Ùµ ¾ ¾ ½ Ó ³ Ù µµ Ùµ ¾ ½ Ê Ê ¾ Ùµ ¾ Ù i.e.itistwotimestheenergyoftheoriginalsignal 3. Ó ³ Ù µµ Ùµ ¾ Ù From the group of similarity transformations(i.e. shifts, rotations and dilations) only the rotation really affects the spherical analytic signal; the orientation phase is changed according to the rotation. If we interpret the spherical analytic signal as a vector field 3 ThisisonlyvalidforDCfreesignals.TheenergyoftheDCcomponentisnotdoubledasinthecase of the 1D analytic signal. 12

15 in3d(seealsosection3.2),thegroupof2dsimilaritytransforms 4 evencommuteswith the operator which yields the spherical analytic signal. The reader might ask, why do we use a quaternion-valued Fourier transform which differsfromtheqft(see[4,6,7]andforamoremathematicaldiscussion[2])? The reason is not obvious. Therefore, we will go into some detail. The QFT covers more symmetry concepts than the complex Fourier transform. The classical transform maps a reflection at the origin onto the conjugation operator. The QFTmapsareflectionatoneoftheaxesontooneofthealgebraautomorphisms.This canbeshownbytheisomorphismbetween Å and ¾ : InthecalculationoftheQFT,onlyproductsoftheform,,, and areused, theproducts,, and neverappear.therefore,wecanusethealgebra Å insteadof ÉtocalculatetheQFT.Moreover,wehave Å ¾ (seee.g.[5,8])and consequently, we can calculate the QFT of a real signal by two complex transformations using the formula Õ Ùµ Ùµ ½ ¾ Ù ½ Ù ¾ µ ½ ¾ (13) andthesymmetrywrt.theaxesisobvious. In this report, we want to present an isotropic approach which means that symmetry wrt. theaxesisnotsufficient. Therefore,wehadtodesignthenewtransform. The designofisotropicdiscretefiltersisaquiteoldtopic,seee.g.[3],butstillcurrent(e.g. [15]). 4 Notethatinthecontextofa3Dembedding,thegroupof2Dsimilaritytransformsisthesubgroup ofthe3dtransformsrestrictedtoshiftsin or direction,rotationsaroundtherealaxisandadilationof the and axes. 13

16 3 Properties of the Spherical Analytic Signal In this section we describe some global and local aspects of the spherical analytic signal. 3.1 The spatial representation The definition of the spherical Hilbert transform in the frequency domain can be transformed into a spatial integral transform. The transfer function(8) can be split into two functions: Ù ½ Ù and Ù ¾ Ù.TheonlythingleftistocalculatetheinverseFouriertransformof thesetwofunctions.in[19,17]thetransformpairscanbefound: Ü ½ ¾Ü Æ Ü ¾ ¾Ü Æ Ù ½ Ù and (14) Ù ¾ Ù (15) These Fourier correspondences can also be found by direct calculation. First we try tofindtheinversetransformof ½ Ù.ItcanbeobtainedusingtheHankeltransform,see [1]: ½ ÜÔ ¾Ù Ü Ù Ê ¾ Ù substitute Ü Ö Ó µ Ò µµand Ù Õ Ó ³µ Ò ³µµ ½ ¾ Õ ½ ÜÔ ¾ÖÕ Ó ³ µõ Õ ³ ¼ ¼ whichisthehankeltransformof Õ ½ Ö ½ Ü ½ Now, using the derivative theorem of the Fourier transform we obtain: and since we obtain(14). Ü ½ Ü ½ Æ ¾ Ù ½ Ù Ü ½ Ü ½ Ü The functions ½ ¾Ü and Ü ¾ ¾Ü arethekernelsofthe2driesztransform([19]).from a mathematician s point of view, the Riesz transform is the multidimensional generalization of the Hilbert transform. From(14) and(15) we obtain Ü ½ Ü Ü ½ ¾Ü Æ Ù½ Ù and (16) Ü ¾ ¾Ü Æ Ù¾ Ù (17) 14

17 by the linearity of the Fourier transform Ü ¾Ü Æ Ù Ù (18) Consequently,thesphericalHilberttransform(ortheRiesztransform)ofasignal ܵ isobtainedbyconvolvingthesignalwith Ü ¾Ü : À ܵ Ê ¾ Ø Ü Øµ Ø (19) ¾Ø The spherical analytic signal is directly obtained by the convolution ܵ Ê ¾ Æ ¼ ص Ø ¾Ø Ü Øµ Ø (20) Thegraphinfigure2sumsupallwaystocalculatethesphericalanalyticsignalfrom theprecedingsections.theinversesphericalfouriertransformisdenoted ½. Æ ¼ ܵ Ü ¾Ü µ ½ Ù Ù µ ½ ½ Figure 2: Three ways to calculate the spherical analytic signal 15

18 3.2 The structure multivector Normally, images are intrinsically two-dimensional, so the concepts described in section 2.3 cannot be applied globally. On the other hand, large areas of images are intrinsically one-dimensional, at least on a certain scale. Therefore, a local processing would take advance of the new approach. Compared to local properties of 1D signals, intrinsic one-dimensional signals only have one additional property: the local orientation. Therefore, it is quite reasonable to extend the local description by this parameter. The classical approach of the analytic signal has its local counterpart in the quadrature filters. A pair of quadrature filters(or a complex quadrature filter) is characterized bythefactthattheimpulseresponseisananalyticsignal. Ontheotherhand,both impulse responses can be bounded(i.e. they act on a local neighborhood), so that the problem of the unlimited impulse response of the Hilbert filter is circumvented. Anexampleoftheoutputofa1Dquadraturefiltercanbefoundinfigure3.While Figure 3: Upper left: impulse, upper right: magnitude of filter output, lower left: real part of filter output, lower right: imaginary part of filter output therepresentationinfigure3isverycommon,wewillintroducenowadifferentrep- 16

19 resentation. One-dimensional signals can be interpreted as surfaces in 2D space. If we assigntherealaxistothesignalvaluesandtheimaginaryaxistotheabscissa,weobtain a representation in the complex plane. The analytic signal can be embedded in the sameplane itcorrespondstoavectorfieldwhichisonlynon-zeroontheimaginary axis(see figure 4) Figure 4: Representation of the 1D analytic signal as a vector field The vector field representation of the analytic signal looks like particles which roll fromahill(impulse).therotationisdirectedawayfromthepeakandthespeed(amplitude of vectors) decreases with increasing distance. Based on the spherical analytic signal we now introduce the spherical quadrature filters. They are defined according to the 1D case. Definition 5 A hypercomplex filter whose impulse response is a spherical analytic signal is called a spherical quadrature filter. In practical applications, the convolution(20) cannot be applied directly either, because of the unlimited impulse response of the expression(20). Of course, one could create optimized filters for signals which are supposed to be band-limited, but it is better workable to use radial bandpass filters to band-limit the signal actively. The spatial representation of a radial bandpass can be used to design the spherical quadrature filter in the spatial domain. The spatial representation looks similar to a Bessel function(see figure 5). In fact, it is a linear combination of Bessel functions with different scales. It is remarkable that the spherical quadrature filters have isotropic energy and exactlychoosethefrequencybandstheyaredesignedfor.infigure6itcanbeseenthat theenergyisisotropicandthatitismaximalfortheradius,whichcorrespondsto afrequencyof ½.Theusedbandpasshasacenterfrequencyof ½. ¾ ½ 17

20 Figure 5: Bessel function Figure 6: Siemens-star convolved with a spherical quadrature filter(magnitude) 18

21 Infigure7theoutputofasphericalquadraturefilterappliedtoanimpulse-lineis displayed. The lower right image shows the combined imaginary parts which means Figure 7: Upper left: impulse-line, upper right: amplitude of filter output, lower left: real part of filter output, lower right: combined imaginary parts of filter output thatitissimplytheimaginarypartofthe correct imaginaryplane(thatonewhichis obtained from the orientation phase). Sameasfor1Dsignals,2Dsignalscanbeembeddedasasurfacein3Dspace.The signalvaluesareassignedtotherealaxisandthespatialcoordinatestothe -andthe -axis. Thesphericalanalyticsignalcanberepresentedinthesameembedding. It correspondstoavectorfield,whichisonlynon-zerointheplanespannedby and (see figure 8). Asinthe1Dcase,thevectorfieldlookslikeparticleswhichrollfromaridge(impulse-line). The rotation is directed orthogonal away from the ridge and the speed decreases with increasing distance. The result of filtering a signal with a spherical quadrature filter is a quaternionvaluedfield. Thoughthebivector(or )componentofthefieldisalwayszero,we denote this field as a multivector field or the structure multivector of the signal. As already the name induces, the structure multivector is closely related to the structure tensor. The structure tensor as defined in[10] includes the following information (in fact, only the orientation vector is considered): amplitude measurement for the existence of local structure 19

22 Figure 8: Representation of the spherical analytic signal as a vector field orientation orientation of the local structure Jähne[13] extracts an additional information: the coherence. The coherence is the relationship between the oriented gradients and all gradients, so it is a measurement for thedegreeoforientationinastructureanditiscloselyrelatedtothevarianceofthe orientation. Thevarianceisasecondorderproperty.Itincludesaproductoftheargumentsand therefore, it is not linear. Consequently, the coherence cannot be measured by a linear approach like the structure multivector. Two structures with different orientations simply yield the vector sum of both multivector fields. The structure multivector consists of three independent components(local phase, local orientation and local amplitude) and it codes three properties. Consequently, there is no additional information possible. The structure tensor possesses three degrees of freedom(it is a symmetric tensor) and codes only two properties. Therefore, one can additionally extract a third information, the coherence. 20

23 3.3 Detection of key points The current status of our approach of detecting key points is not totally satisfying, becauseitisstillnonlinear.thekeyideaistocalculatetheouterenergyofthemultivector field. Outer energy means being contrary to the inner energy, which is calculated by the inner product of the multivectors. Obviously, this cannot be done pointwise, because theouterproductofamultivectorwithitselfisalwayszero 5. Therefore, we consider a local neighborhood and in this local neighborhood we combine those areas which are assumed to have similar orientations. This is achieved as follows: the first vector for the outer product is obtained by convolving the signal with acosinemaskandthesecondoneisobtainedbyconvolvingwithasinemask: ܵ À ܵ Ó ³ Ü µö ܵµ (21) Ë Üµ À ܵ Ò ³ Ü µö ܵµ (22) where Ö Üµisafunctionwhichiszerofor Ü ¼andforlarge Ü(e.g. ashifted Gaussian function). Ifwehaveanintrinsically1Dstructure,bothsignalshavethesamephases. Ifwe have a rectangular corner, they are orthogonal. Consequently, the outer product(cross product 6 ) iszerointhefirstcaseandismaximizedinthesecondcase.  ܵ ܵ Ë Üµ (23) 5 Noteinthiscontext,theinnerproductstandsforthesquaredmagnitudeandtheouterproductisthe cross product between the 3D component vectors obtained from the multivectors. 6 Intheformula,weusethenotationofthecrossproductinordertodistinguishbetweenthealgebras weworkon(seealsofootnote5). 21

24 4 Experiments and Discussion This section shows some experiments and an implementation in MatLab. The results arecomparedtothosewhichcanbefoundin[10]forexample. 4.1 Implementation and tests Throughout this section we used the implementations which can be found in the appendix. The function which computes the structure multivector uses a multi-scale approach, becausetheshiftofthegaussianbandpassiscoupledwiththevarianceasforthegabor wavelets. Both functions are first implementations, which should only underline the practicability of the approach. For testing the implementations, we chose some synthetic examples with letters as gray level or textured images. Figure 9: Structure multivector applied to an image without texture. Upper left: original,upperright:amplitude,lowerleft: ³ -phase,lowerright: ³ Ù -phase Infigure9itcanbeseenthatthestructuremultivectorrespondsonlyattheedges. Therefore,theamplitudeisameasureforthepresenceofstructure. The ³ -phaseis 22

25 Figure 10: Structure multivector applied to an image with one texture. Upper left: original,upperright:amplitude,lowerleft: ³ -phase,lowerright: ³ Ù -phase Figure 11: Structure multivector applied to an image with two superposed textures. Upperleft:original,upperright:amplitude,lowerleft: ³ -phase,lowerright: ³ Ù -phase 23

26 Figure 12: Structure multivector applied to an image with two superposed textures and texturedbackground.upperleft:original,upperright:amplitude,lowerleft: ³ -phase, lowerright: ³ Ù -phase linear,whichcanonlybeguessedinthisrepresentation. Butitcanbeseenthatthe ³ -phaseismonotonicmoduloamaximalinterval(whichisinfact ¾).The ³ Ù -phase represents the orientation of the edge. Note that the highest and the lowest gray level (standingfor andzero,respectively)representthesameorientation. Infigure10itcanbeseenthatthestructuremultivectorrespondsalsoinsidethe object. The amplitude is nearly constant, which corresponds to a texture with constant energy.ofcourse,thispropertyislostifthewrongscaleisconsidered.the ³ -phase islinear,seenotesabove. The ³ Ù -phaserepresentstheorientationofthetexture. A constant gray level corresponds to a constant estimated orientation. The small spikes in the figure are produced by the extraction of the local orientation angle. The underlying quaternion-valued field does not show these artifacts. Infigure11itcanbeseenthatthestructuremultivectorrespondsonlywithrespect to the dominant texture. The magnitude of the response is modulated with that componentoftheweakertexturethatisnormaltothedominanttexture.thiseffectiseven moreobviousinfigure12. The ³ -phaseisalwaysdirectedparalleltothedominant texture.the ³ Ù -phaserepresentstheorientationofthedominanttextureineachcase. The described 2D detection algorithm works fine for images without texture. The amplitude of the detection filter response can be seen in figure 13. Obviously, the response increases with increasing sharpness of the corner. 24

27 Figure 13: Result of the 2D key-point detection applied to the multivector field from figure Discussion of results In contrast to the structure tensor which becomes zero for ideal 2D points(e.g. rectangular corner) the structure multivector responds at a high energy level. Therefore, it is easier to design a detector for those points, because the energy is higher. Furthermore, a concept like the outer energy does not produce artificial double lines beneath the original edges. These double lines makes it necessary to use a normalized convolution. Our approach does not need such tricks to obtain clear cut results. 4.3 Futurework Inourfuturework,wewilltrytolinearizethe2Dkeypointdetection.Thiswillbedone in an 8D algebra. Furthermore, we will try to separate texture information from edge information in a second filtering step, using the same algebra. Up to now, textures which are not parallel to the edge produce artificial 2D key points. The long-term objective will be the design of filters which covers the entire hierarchy of intrinsic dimensionality, consisting of(1d and 2D) textures, edges, corners and planes. Another extension of the presented theory will be the consideration of higher dimensional cases, especially the image sequence analysis. The approach of the spherical analytic signal can be extended to higher dimensions easily. The theoretical background will be further investigated with regard to the results from Clifford analysis, especially the monogenic function and other multidimensional generalizations of the analytic function. 25

28 Appendix MatLab function for computing the structure multivector ÙÒØÓÒÅ ØÖÑÛ Ê Ñ µ ± ÓÑÔÙØ Ø ØÖÙØÙÖ ÑÙÐØÚØÓÖ Ó ÖÐ ÒÐ Ê ÛØ ± ÔÖÐ ÕÙÖØÙÖ ÐØÖ ÛØ Ù Ò ÒÔ ± Ѿ ÚÖÒ ÖØÓ Ó ÛÚ ÒÖÒ ¼º Ò Þ Ê ½µ»¾ ¾¾ Ñ Ù Ú Ñ Ö ¹ ¹½ ¹ ¹½ µ ÕÖØ Ùº¾ Úº¾µ ÜÔ ¹¾ Ô» ѹµµº¾µ Ñ ÕÖØ ¾ Ôµ ½ ½µ¼ ÆØ Ø Ø¾ Ø Ø µµµ ÆÆ ½¹ ¾ ¾ ½¹ ¾ ¾µ ½ ½µ½ º» ÙÙº ÚÚº ÍØ Ø Ø¾ Ø Ø Ùµµµ ÎØ Ø Ø¾ Ø Ø Úµµµ ÍÍ ½¹ ¾ ¾ ½¹ ¾ ¾µ ÎÎ ½¹ ¾ ¾ ½¹ ¾ ¾µ Å ¾µÓÒÚ¾ Ê ¹Ñ ͵ ³ ѳµ Å µóòú¾ Ê ¹Ñ ε ³ ѳµ Å ½µÓÒÚ¾ Ê ÖРƵ ³ ѳµ 26

29 MatLab function for detecting intrinsically 2D structures ÙÒØÓÒ Øؾ Å Ñ Ûµ ± ØØÓÒ Ó ¾ ݹÔÓÒØ ÛØ ÖÐ ÐÓ¹ÒÓÖÑÐ ÛØÒ Ù Ú Ñ Ö ¹ ¹ µ Ñ ÕÖØ Ùº¾ Úº¾µ Ñ µ½ ÔÜÔ ¹Û ÐÓ Ñ»Ñµº¾»ÐÓ ¾µµº»Ñ Ô µ¼ ÙÙº Ô ÚÚº Ô ÓÒÚ¾ Å ½µ Ù ³ ѳµ ÓÒÚ¾ Å ½µ Ú ³ ѳµ ÓÒÚ¾ Å ¾µ Ù ³ ѳµ ÓÒÚ¾ Å ¾µ Ú ³ ѳµ ÓÒÚ¾ Å µ Ù ³ ѳµ ÓÒÚ¾ Å µ Ú ³ ѳµ º ¹ º º ¹º º ¹ º º¾ º¾ º¾ 27

30 References [1] BRACEWELL, R. N. Two-Dimensional Imaging. Prentice Hall signal processing series. Prentice Hall, Englewood Cliffs, [2] BRACKX, F., DELANGHE, R., AND SOMMEN, F. Clifford Analysis. Pitman, Boston, [3] BRADY, J. M., AND HORN, B. M. P. Rotationally symmetric operators for surface interpolation. Computer Vision, Graphics, and Image Processing 22, 1(April 1983), [4] BÜLOW, T. Global and Local Hypercomplex Spectral Signal Representations for Image Processing and Analysis. PhD thesis, Christian-Albrechts-University of Kiel, [5] DAVENPORT, C. M. A Commutative Hypercomplex Algebra with Associated Function Theory. In Clifford Algebras with Numeric and Symbolic Computations, R. Ablamowicz, Ed. Birkhäuser Bosten, 1996, pp [6] ELL, T. A. Hypercomplex Spectral Transformations. PhD thesis, University of Minnesota, [7] ERNST, R. R., AUE, W. P., BACHMANN, P., KARHAN, J., KUMAR, A., AND MÜLLER, L. Two-dimensional NMR spectroscopy. In Proc. 4th Ampère Int. Summer School, Pula, Yugoslavia(1976). [8] FELSBERG, M., BÜLOW, T., AND SOMMER, G. Commutative Hypercomplex Fourier Transforms of Multidimensional Signals. In Geometric Computing with Clifford Algebra, G. Sommer, Ed. Springer, Heidelberg, to appear. [9] GRANLUND, G. H. Hierarchical computer vision. In Proc. of EUSIPCO-90, Fifth European Signal Processing Conference, Barcelona(1990), L. Torres, E. Masgrau, and M. A. Lagunas, Eds., pp [10] GRANLUND, G. H., AND KNUTSSON, H. Signal Processing for Computer Vision. Kluwer Academic Publishers, Dordrecht, [11] HAGLUND, L. Adaptive Multidimensional Filtering. PhD thesis, Linköping University, [12] HESTENES, D., AND SOBCZYK, G. Clifford algebra to geometric calculus, A Unified Language for Mathematics and Physics. Reidel, Dordrecht, [13] JÄHNE, B. Digitale Bildverarbeitung. Springer, Berlin, [14] KARLHOLM, J. Local Signal Models for Image Sequence Analysis. PhD thesis, Linköping University,

31 [15] MERRON, J., AND BRADY, M. Isotropic gradient estimation. In IEEE Computer Vision and Pattern Recognition(1996), pp [16] MICHAELIS, M., AND SOMMER, G. A Lie group approach to steerable filters. Pattern Recognition Letters 16(1995), [17] NABIGHIAN, M. N. Toward a three-dimensional automatic interpretation of potential field data via generalized Hilbert transforms: Fundamental relations. Geophysics 49, 6(June 1984), [18] SOMMER, G. The global algebraic frame of the perception-action cycle. In Handbook of Computer Vision and Applications(1999), B. Jähne, H. Haußecker, and P. Geissler, Eds., vol. 3, Academic Press, San Diego, pp [19] STEIN, E., AND WEISS, G. Introduction to Fourier Analysis on Euclidean Spaces. Princeton University Press, New Jersey, [20] YU, W., DANIILIDIS, K., BEAUCHEMIN, S., AND SOMMER, G. Detection and characterization of multiple motion points. In 18th IEEE conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado(1999). 29