Scale Selection Properties of Generalized Scale-Space Interest Point Detectors

Transcription

1 J Mah Imaging Vis : DOI /s Scale Selecion Properies of Generalized Scale-Space Ineres Poin Deecors Tony Lindeberg Published online: 0 Sepember 01 The Auhors 01. This aricle is published wih open access a Springerlink.com Absrac Scale-invarian ineres poins have found several highly successful applicaions in compuer vision, in paricular for image-based maching and recogniion. This paper presens a heoreical analysis of he scale selecion properies of a generalized framework for deecing ineres poins from scale-space feaures presened in Lindeberg In. J. Compu. Vis. 010, under revision and comprising: an enriched se of differenial ineres operaors a a fixed scale including he Laplacian operaor, he deerminan of he Hessian, he new Hessian feaure srengh measures I and II and he rescaled level curve curvaure operaor, as well as an enriched se of scale selecion mechanisms including scale selecion based on local exrema over scale, complemenary pos-smoohing afer he compuaion of nonlinear differenial invarians and scale selecion based on weighed averaging of scale values along feaure rajecories over scale. I is shown how he seleced scales of differen linear and non-linear ineres poin deecors can be analyzed for Gaussian blob models. Specifically i is shown ha for a roaionally symmeric Gaussian blob model, he scale esimaes obained by weighed scale selecion will be similar o he scale esimaes obained from local exrema over scale of The suppor from he Swedish Research Council, Veenskapsråde conracs , and from he Royal Swedish Academy of Sciences as well as he Knu and Alice Wallenberg Foundaion is graefully acknowledged. T. Lindeberg School of Compuer Science and Communicaion, KTH Royal Insiue of Technology, Sockholm, Sweden ony@csc.kh.se scale normalized derivaives for each one of he pure secondorder operaors. In his respec, no scale compensaion is needed beween he wo ypes of scale selecion approaches. When using pos-smoohing, he scale esimaes may, however, be differen beween differen ypes of ineres poin operaors, and i is shown how relaive calibraion facors can be derived o enable comparable scale esimaes for each purely second-order operaor and for differen amouns of self-similar pos-smoohing. A heoreical analysis of he sensiiviy o affine image deformaions is presened, and i is shown ha he scale esimaes obained from he deerminan of he Hessian operaor are affine covarian for an anisoropic Gaussian blob model. Among he oher purely second-order operaors, he Hessian feaure srengh measure I has he lowes sensiiviy o nonuniform scaling ransformaions, followed by he Laplacian operaor and he Hessian feaure srengh measure II. The predicions from his heoreical analysis agree wih experimenal resuls of he repeaabiliy properies of he differen ineres poin deecors under affine and perspecive ransformaions of real image daa. A number of less complee resuls are derived for he level curve curvaure operaor. Keywords Feaure deecion Ineres poin Blob deecion Corner deecion Scale Scale-space Scale selecion Scale invariance Scale calibraion Scale linking Feaure rajecory Deep srucure Affine ransformaion Differenial invarian Gaussian derivaive Muli-scale represenaion Compuer vision 1 Inroducion The noion of scale selecion is essenial o adap he scale of processing o local image srucures. A compuer vision

2 178 J Mah Imaging Vis : sysem equipped wih an auomaic scale selecion mechanism will have he abiliy o compue scale-invarian image feaures and hereby handle he a priori unknown scale variaions ha may occur in image daa because of objecs and subsrucures of differen physical size in he world as well as objecs a differen disances o he camera. Compuing local image descripors a inegraion scales proporional o he deecion scales of scale-invarian image feaures, moreover makes i possible o compue scale-invarian image descripors Lindeberg [35]; Brezner and Lindeberg [4]; Mikolajczyk and Schmid [49]; Lowe [48]; Bay e al. []; Lindeberg [38, 43]. A general framework for performing scale selecion can be obained by deecing local exrema over scale of γ - normalized derivaive expressions Lindeberg [35]. This approach has been applied o a large variey of feaure deecion asks Lindeberg [34]; Brezner and Lindeberg [4]; Sao e al. [54]; Frangi e al. [11]; Krissian e al. []; Choma e al. [5]; Hall e al. [15]; Mikolajczyk and Schmid [49]; Lazebnik e al. [4]; Negre e al. [5]; Tuyelaars and Mikolajczyk [58]. Specifically, highly successful applicaions can be found in image-based recogniion Lowe [48]; Bay e al. []. Alernaive approaches for scale selecion have also been proposed in erms of he deecion of peaks over scale in weighed enropy measures Kadir and Brady [18] or Lyapunov funcionals Sporring e al. [56], minimizaion of normalized error measures over scale Lindeberg [36], deermining minimum reliable scales for feaure deecion according a noise suppression model Elder and Zucker [9], deermining opimal sopping imes in non-linear diffusionbased image resoraion mehods using similariy measuremens relaive o he original daa Mrázek and Navara [51], by applying saisical classifiers for exure analysis a differen scales Kang e al. [19] or by performing image segmenaion from he scales a which a supervised classifier delivers class labels wih he highes poserior Loog e al. [47]; Li e al. [5]. Recenly, a generalizaion of he differenial approach for scale selecion based on local exrema over scale of γ - normalized derivaives has been proposed by linking image feaures over scale ino feaure rajecories over scale in a generalized scale-space primal skech [39]. Specifically, wo novel scale selecion mechanisms have been proposed in erms of: pos-smoohing of differenial feaure responses by performing a second-sage scale-space smoohing sep afer he compuaion of non-linear differenial invarians, so as o simplify he ask of linking feaure responses over scale ino feaure rajecories, and weighed scale selecion where he scale esimaes are compued by weighed averaging of scale-normalized feaure responses along each feaure rajecory over scale, in conras o previous deecion of local exrema or global exrema over scale. The subjec of his aricle is o perform an in-deph heoreical analysis of properies of hese scale selecion mehods when applied o he ask of compuing scale-invarian ineres poins: i When using a se of differen ypes of ineres poin deecors ha are based on differen linear or non-linear combinaions of scale-space derivaives, a basic quesion arises of how o relae hresholds on he magniude values beween differen ypes of ineres poin deecors. By sudying he responses of he differen ineres poin deecors o uni conras Gaussian blobs, we will derive a way of expressing muually corresponding hresholds beween differen ypes of ineres poins deecors. Algorihmically, he resuling hreshold relaions lead o inuiively very reasonable resuls. ii The new scale selecion mehod based on weighed averaging along feaure rajecories over scale raises quesions of how he properies of his scale selecion mehod can be relaed o he previous scale selecion mehod based on local exrema over scale of scalenormalized derivaives. We will show ha for Gaussian blobs, he scale esimaes obained by weighed averaging over scale will be similar o he scale esimaes obained from local exrema over scale. If we assume ha scale calibraion can be performed based on he behaviour for Gaussian blobs, his resul herefore shows ha no relaive scale compensaion is needed beween he wo ypes of scale selecion approaches. In previous work on scale selecion based on γ -normalized derivaives [34, 35] a similar assumpion of scale calibraion based on Gaussian model signals has been demonsraed o lead o highly useful resuls for calibraing he value of he γ -parameer wih respec o he problems of blob deecion, corner deecion, edge deecion and ridge deecion, wih a large number of successful compuer vision applicaions building on he resuling feaure deecors. iii For he scale linking algorihm presened in [39], which is based on local gradien ascen or gradien decen saring from local exrema in he differenial responses a adjacen levels of scale, i urns ou ha a second pos-smoohing sage afer he compuaion of non-linear differenial invarians is highly useful for increasing he performance of he scale linking algorihm, by suppressing spurious responses of low relaive ampliude in he non-linear differenial responses ha are used for compuing ineres poins. This selfsimilar amoun of pos-smoohing is deermined as a consan imes he local scale for compuing he differenial expressions, and may affec he scale esimaes obained from local exrema over scale or weighed averaging over scale. We will analyze how large his effec will be for differen amouns of pos-smoohing

3 J Mah Imaging Vis : iv and also show how relaive scale normalizaion facors can be deermined for he differen differenial expressions o obain scale esimaes ha are unbiased wih respec o he effec of he pos-smoohing operaion, if we again assume ha scale calibraion can be performed based on he scale selecion properies for Gaussian blobs. Noably, differen scale compensaion facors for he influence of pos-smoohing will be obained for he differen differenial expressions ha are used for defining ineres poins. Wihou possmoohing, he scale esimaes obained from he differen differenial expressions are, however, all similar for Gaussian blobs, which indicaes he possibiliies of using differen ypes of differenial expressions for performing combined ineres poin deecion and scale selecion, so ha hey can be inerchangeably replaced in a modular fashion. When deecing ineres poins from images ha are aken of an objec from differen viewing direcions, he local image paern will be deformed by he perspecive projecion. If he ineres poin corresponds o a poin in he world ha is locaed a a smooh surface of an objec, his deformaion can o firs order of approximaion be modelled by a local affine ransformaion Gårding and Lindeberg [1]. While he noion of affine shape adapaion has been demonsraed o be a highly useful ool for compuing affine invarian ineres poins Lindeberg and Gårding [46]; Baumberg [1]; Mikolajczyk and Schmid [49]; Tuyelaars and van Gool [57], he success of such an affine shape adapaion process depends on he robusness of he underlying ineres poins ha are used for iniiaing he ieraive affine shape adapaion process. To invesigae he properies of he differen ineres poin deecors under affine ransformaions, we will perform a deailed analysis of he scale selecion properies for affine Gaussian blobs, for which closed form heoreical analysis is possible. The analysis shows ha he deerminan of he Hessian operaor and he new Hessian feaure srengh measure I do boh have significanly beer behaviour under affine ransformaions han he Laplacian operaor or he new Hessian feaure srengh measure II. In comparison wih experimenal resuls [39], he ineres poin deecors ha have he bes heoreical properies under affine ransformaions of Gaussian blob do also have significanly beer repeaabiliy properies under affine and perspecive ransformaions han he oher wo. These resuls herefore show how experimenal properies of ineres poins can be prediced by heoreical analysis, which conribues o an increased undersanding of he relaive properies of differen ypes of ineres poin deecors. In very recen work [4], hese generalized scale-space ineres poins have been inegraed wih local scale-invarian image descripors and been demonsraed o lead o highly compeiive resuls for image-based maching and recogniion. 1.1 Ouline of he Presenaion The paper is organized as follows. Secion reviews main componens of a generalized framework for deecing scaleinvarian ineres poins from scale-space feaures, including a richer se of ineres poin deecors a a fixed scale as well as new scale selecion mechanisms. In Sec. 3 he scale selecion properies of his framework are analyzed for scale selecion based on local exrema over scale of γ -normalized derivaives, when applied o roaionally symmeric as well as anisoropic Gaussian blob models. Secion 4 gives a corresponding analysis for scale selecion by weighed averaging over scale along feaure rajecories. Secion 5 summarizes and compares he resuls obained from he wo scale selecion approaches including complemenary heoreical argumens o highligh heir similariies in he roaionally symmeric case. I is also shown how scale calibraion facors can be deermined so as o obain comparable scale esimaes from ineres poin deecors ha have been compued from differen ypes of differenial expressions. Comparisons are also presened of he relaive sensiiviy of he scale esimaes o affine ransformaions ouside he similariy group, wih a brief comparison o experimenal resuls. Finally, Sec. 6 concludes wih an overall summary and discussion. Scale-Space Ineres Poins.1 Scale-Space Represenaion The concepual background we consider for feaure deecion is a scale-space represenaion Iijima [17]; Wikin [6]; Koenderink [0]; Koenderink and van Doorn [1]; Lindeberg [30, 31]; Florack [10]; Weicker e al. [60]; er Haar Romeny [14]; Lindeberg [38, 40] L: R R + R compued from a wo-dimensional signal f : R R according o Lx, y; fx u, y vgu,v; dudv 1 u,v R where g : R R + R denoes he roaionally symmeric Gaussian kernel gx,y; 1 π e x +y /

4 180 J Mah Imaging Vis : and he variance σ of his kernel is referred o as he scale parameer. Equivalenly, his scale-space family can be obained as he soluion of he linear diffusion equaion L 1 L 3 wih iniial condiion L, ; f. From his represenaion, scale-space derivaives or Gaussian derivaives a any scale can be compued eiher by differeniaing he scalespace represenaion or by convolving he original image wih Gaussian derivaive kernels: L x α y β, ; x α y β L, ; x α y β g, ; f, where α and β Z +.. Differenial Eniies for Deecing Scale-Space Ineres Poins A common approach o image maching and objec recogniion consiss of maching ineres poins wih associaed image descripors. Basic requiremens on he ineres poins on which he image maching is o be performed are ha hey should i have a clear, preferably mahemaically wellfounded, definiion, ii have a well-defined posiion in image space, iii have local image srucures around he ineres poin ha are rich in informaion conen such ha he ineres poins carry imporan informaion o laer sages and iv be sable under local and global deformaions of he image domain, including perspecive image deformaions and illuminaion variaions such ha he ineres poins can be reliably compued wih a high degree of repeaabiliy. The image descripors compued a he ineres poins should also v be sufficienly disinc, such ha ineres poins corresponding o physically differen poins can be kep separae. Preferably, he ineres poins should also have an aribue of scale, o make i possible o compue reliable ineres poins from real-world image daa, including scale changes in he image domain. Specifically, he ineres poins should preferably also be scale-invarian o make i possible o mach corresponding image paches under scale variaions. Wihin his scale-space framework, ineres poin deecors can be defined a any level of scale using i eiher of he following esablished differenial operaors [35]: helaplacian operaor 4 L L xx + L yy 5 hedeerminan of he Hessian de HL L xx L yy L xy 6 herescaled level curve curvaure κl L x L yy + L y L xx L x L y L xy 7 ii eiher of he following new differenial analogues and exensions of he Harris operaor [16] proposed in [39]: heunsigned Hessian feaure srengh measure I de HL k race HL D 1 L if de HL k race HL>0 8 0 oherwise hesigned Hessian feaure srengh measure I de HL k race HL if de HL k race HL>0 D 1 L de HL + k race HL if de HL + k race HL<0 0 oherwise where k ]0, 1 4 [ wih he preferred choice k 0.04, or iii eiher of he following new differenial analogues and exensions of he Shi and Tomasi operaor [55] proposed in [39]: heunsigned Hessian feaure srengh measure II 9 D L min λ 1, λ min L pp, L qq 10 hesigned Hessian feaure srengh measure II L pp if L pp < L qq D L L qq if L qq < L pp 11 L pp + L qq / oherwise where L pp and L qq denoe he eigenvalues of he Hessian marix he principal curvaures ordered such ha L pp L qq [34]: L pp 1 L xx + L yy L xx L yy + 4L xy 1 L qq 1 L xx + L yy + L xx L yy + 4L xy 13 Figure 1 shows examples of deecing differen ypes of ineres poins from a grey-level image. In his figure, he repeiive naure of he underlying image srucures in he row of similar books illusrae he abiliy of he ineres poin deecors o respond o approximaely similar srucures in he image domain by corresponding responses. Figure illusraes he repeaabiliy properies of such ineres poins

5 J Mah Imaging Vis : Fig. 1 Scale-invarian ineres poins deeced by linking op lef Laplacian norm L feaures, op righ deerminan of he Hessian de H norm L feaures, middle lef signed Hessian feaure srengh measure D1,norm L feaures, middle righ signed Hessian feaure srengh measure D,norm L feaures, boom lef rescaled level curve curvaure κ γ norm L feaures and boom righ scale-linked Harris- Laplace feaures over scale ino feaure rajecories and performing scale selecion by weighed averaging of scale values along each feaure rajecory. The 500 sronges ineres poins have been exraced and drawn as circles wih he radius reflecing he seleced scale measured in unis of σ. Posiive responses of he differenial expression DL are shown in red and negaive responses in blue. Image size: pixels. Scale range: [4, 51]

6 18 J Mah Imaging Vis : Fig. Illusraion of he repeaabiliy properies of he ineres poins by deecing signed Hessian feaure srengh D1,norm L ineres poins from wo images of a building aken from differen perspecive views, by linking image feaures over scale ino feaure rajecories and performing scale selecion by weighed averaging of scale values along each feaure rajecory. The 1000 sronges ineres poins have been exraced and drawn as circles wih he radius reflecing he seleced scale measured in unis of σ. Ineres poins ha have a posiive definie Hessian marix are shown in blue dark feaures, ineres poins wih negaive definie Hessian marix are shown in red brigh feaures whereas ineres poins wih an indefinie Hessian marix are marked in green saddle-like feaures. Image size: pixels. Scale range: [4, 56]

7 J Mah Imaging Vis : more explicily, by deecing signed Hessian feaure srengh D 1,norm L ineres poins from wo images of a building aken from differen perspecive views. A basic moivaion for defining he new differenial operaors D 1, D 1, D and D from he Hessian marix HL in a srucurally relaed way as he Harris and he Shi-and- Tomasi operaors are defined from he second-momen marix srucure ensor are ha: i under an affine ransformaion p Ap wih p x, y T and A denoing a non-singular marix i can be shown ha he Hessian marix Hf ransforms in a similar way Hf p A T Hf p A 1 as he second-momen marix μ p A T μp A 1 [31, 46] and ii provided ha he Hessian marix is eiher posiive or negaive definie, he Hessian marix HL compued a a poin p 0 defines an eiher posiive or negaive definie quadraic form Q HL p p p 0 T HLp p 0 in a similar way as he second-momen marix μ compued a p 0 does: Q μ p p p 0 T μp p 0. From hese wo analogies, we can conclude ha provided he Hessian marix is eiher posiive or negaive definie, hese wo ypes of descripors should have srong qualiaive similariies. Experimenally, he new differenial ineres poin deecors D 1, D 1, D and D can be shown o perform very well and o allow for image feaures wih beer repeaabiliy properies under affine and perspecive ransformaions han he more radiional Laplacian or Harris operaors [39]. The Laplacian L responds o brigh and dark blobs as formalized in erms of local minima or maxima of he Laplacian operaor. The deerminan of he Hessian de HL responds o brigh and dark blobs by posiive responses and in addiion o saddle-like image feaures by negaive responses as well as o corners. The unsigned Hessian feaure srengh D 1 L responds o brigh and dark blobs as well as o corners, wih he complemenary requiremen ha he raio of he eigenvalues λ 1 and λ of he Hessian marix wih λ 1 λ should be sufficienly close o one, as specified by he parameer k according o: k 1 k + 1 4k λ 1 λ 1 14 For his eniy o respond, i is herefore necessary ha here are srong inensiy variaions along wo differen direcions in he image domain. The signed Hessian feaure srengh measure D 1 L responds o similar image feaures as he unsigned eniy D 1 L, and in addiion o saddle-like image feaures wih a corresponding consrain on he raio beween he eigenvalues. The Hessian feaure srengh measures D L and D L respond srongly when boh of he principal curvaures are srong and he local image paern herefore conains srong inensiy variaions in wo orhogonal direcions. The unsigned eniy D L disregards he sign of he principal curvaures, whereas he signed eniy D L preserves he sign of he principal curvaure of he lowes magniude. Oher ways of defining image feaures from he secondorder differenial image srucure of images have been proposed by Danielsson e al. [7] and Griffin [13]..3 Scale Selecion Mechanisms Scale Selecion from γ -Normalized Derivaives In Lindeberg [9, 31, 35, 37] a general framework for auomaic scale selecion was proposed based on he idea of deecing local exrema over scale of γ -normalized derivaives defined according o ξ γ/ x, η γ/ y 15 where γ>0 is a free parameer 1 ha can be relaed o he dimensionaliy of he image feaures ha he feaure deecor is designed o respond o, e.g., in erms of he evoluion properies over scale in erms of i self-similar L p -norms of Gaussian derivaive operaors for differen dimensionaliies of he image space [35, Sec. 9.1], ii self-similar Fourier specra [35, Sec. 9.] or iii he fracal dimension of he image daa [53]; see also Appendix A.3 for an explici inerpreaion of he parameer γ in erms of he dimensionaliy D of second-order image feaures according o 13. Specifically, i was shown in [35] ha local exrema over scale of homogeneous polynomial differenial invarians D γ norm L expressed in erms of γ -normalized Gaussian derivaives are ransformed in a scale-covarian way: If some scale-normalized differenial invarian D γ norm L assumes a local exremum over scale a scale 0 in scale-space, hen under a uniform rescaling of he inpu paern by a facor s here will be a local exremum over scale in he scale-space of he ransformed signal a scale s 0. Furhermore, by performing simulaneous scale selecion and spaial selecion by deecing scale-space exrema, where he scale-normalized differenial expression D γ norm L assumes local exrema wih respec o boh space and scale, consiues a general framework for deecing scale-invarian ineres poins. Formally, such scale-space exrema are characerized by he firs-order derivaives wih respec o space and scale being zero D γ norm L 0 and D γ norm L Indeed, i can be shown ha he definiion of scale-normalized derivaives in his way capures he full degrees of freedom by which scale invariance can be obained from local exrema over scale of scalenormalized derivaives defined from a Gaussian scale-space, as formally proved by necessiy in [35, Appendix A.1].

8 184 J Mah Imaging Vis : and in addiion he composed Hessian marix compued over boh space and scale xx xy x H x,y; D γ norm L xy yy y D γ norm L x y being eiher posiive or negaive definie. 17 Generalized Scale Selecion Mechanisms In [39] his approach was exended in he following ways: by performing pos-smoohing of he differenial expression D γ norm L prior o he deecion of local exrema over space or scale D γ norm Lx, y; D γ norm Lx u, y v; u,v R g u, v; c dudv 18 wih an inegraion scale pos-smoohing scale pos c proporional o he differeniaion scale wih c>0 see Appendix A.1 for a brief descripion of he algorihmic moivaions for using such a pos-smoohing operaion when linking image feaures over scale ha have been compued from non-linear differenial eniies and by performing weighed averaging of scale values along any feaure rajecory T over scale in a scale-space primal skech according o τ T ˆτ T τψd γ normlxτ; τdτ τ T ψd 19 γ normlxτ; τdτ where ψ denoes some posiive and monoonically increasing ransformaion of he scale-normalized feaure srengh response D γ norm L and wih he scale parameer parameerized in erms of effecive scale [8] τ A log + B where A R + and B R 0 o obain a scale covarian consrucion of he corresponding scale esimaes ˆτT B ˆ T exp 1 A ha implies ha he resuling image feaures will be scaleinvarian. The moivaion for performing scale selecion by weighed averaging of scale-normalized differenial responses over scale is analogous o he moivaion for scale selecion from local exrema over scale in he sense ha ineresing characerisic scale levels for furher analysis should be obained from he scales a which he differenial operaor assumes is sronges scale-normalized magniude values over scale. Conrary o scale selecion based on local exrema over scale, however, scale selecion by weighed averaging over scale implies ha he scale esimae will no only be obained from he behaviour around he local exremum over scale, bu also including he responses from all scales along a feaure rajecory over scale. The inenion behind his choice is ha he scale esimaes should herefore be more robus and less sensiive o local image perurbaions. Experimenally, i can be shown ha scale-space ineres poins deeced by hese generalized scale selecion mechanisms lead o ineres poins wih beer repeaabiliy properies under affine and perspecive image deformaions compared o corresponding ineres poins deeced by regular scale-space exrema [39]. In his sense, hese generalized scale selecion mechanisms make i possible o deec more robus image feaures. Specifically, he use of scale selecion by weighed averaging over scale is made possible by linking image feaures over scale ino feaure rajecories, which ensures ha he scale esimaes should only be influenced by responses from scale levels ha correspond o qualiaively similar ypes of image srucures along a feaure rajecory over scale. The subjec of his aricle is o analyze properies of hese generalized scale selecion mechanisms heoreically when applied o he ineres poin deecors lised in Sec... 3 Scale Selecion Properies for Local Exrema over Scale For heoreical analysis, we will consider a Gaussian prooype model of blob-like image srucures. Wih such a prooype model, he semi-group propery of he Gaussian kernel makes i possible o direcly obain he scale-space represenaions a coarser scales in erms of Gaussian funcions, which simplifies heoreical analysis. Specifically, he resul of compuing polynomial differenial invarians a differen scales will be expressed in erms of Gaussian funcions muliplied by polynomials. Thereby, closed-form heoreical analysis becomes racable, which would oherwise By linking image feaures over scale ino feaure rajecories i also becomes possible o define a significance value by inegraing scalenormalized feaure responses over scale. Experimenally, i can be shown ha such ranking of image feaures leads o selecions of subses of ineres poins wih beer overall repeaabiliy properies han selecion of subses of ineres poins from he exremum responses of ineres poins deecors a scale-space exrema. An inuiive moivaion for his propery is a heurisic principle ha image feaures ha are sable over large ranges of scales should be more likely o be significan han image feaures ha only exis over a shorer life lengh in scale-space [7, Assumpion 1 in Sec. 3 on p. 96].

9 J Mah Imaging Vis : be much harder o carry ou regarding he applicaion of he non-linear operaions ha are used for defining he ineres poins o general image daa. The use of Gaussian prooype model can also be moivaed by concepual simpliciy. If we would like o model an image feaure a some scale, hen he Gaussian model is he model ha requires he minimum amoun of informaion in he sense ha he Gaussian disribuion is he disribuion wih maximum enropy 3 given a specificaion of he mean value m and he covariance marix Σ of he disribuion. Specifically, he Gaussian funcion wih scale parameer serves as an aperure funcion ha measures image srucures wih respec o an inner scale beyond which finer-scale srucures canno be resolved. In previous work [34, 35] i has been shown ha deerminaion of he γ -parameer in scale selecion for differen ypes of feaure deecion asks, such as blob deecion, corner deecion, edge deecion and ridge deecion, can be performed based on he behaviour of hese feaure deecors on Gaussian-based inensiy profiles. As will be shown laer, he heoreical resuls ha will be derived based on Gaussian blob models will lead o heoreical predicions ha agree wih he relaive repeaabiliy properies of differen ypes of ineres poin deecors under affine and perspecive ransformaions. Formally, however, furher applicaion of hese resuls will be based on an assumpion ha he scale selecion behaviour can be calibraed based on he behaviour for Gaussian prooype models. 3.1 Regular Scale Selecion from Local Exrema over Scale Two basic quesions in he relaion o he differen ineres poin deecors reviewed in Sec.. concern: How will he seleced scale levels be relaed beween differen ineres poin deecors? How will he scale-normalized magniude values be relaed beween differen ineres poin deecors ha respond o similar image srucures? Ideally, we would like similar scale esimaes o be obained for differen ineres poin deecors, so ha he ineres poin deecors could be modularly replaceable in he compuer vision algorihms hey are par of. Since he ineres poin deecors are expressed in erms of differen ypes of linear or non-linear combinaions of scale-space derivaives, a basic quesion concerns how o express comparable hresholds on he magniude values for he differen ineres poin deecors. In his secion, we will relae hese eniies by applying scale selecion from local exrema of scale-normalized 3 Maximum enropy soluions have been argued o be aken as preferred defaul soluions for underconsrained problems [3, 59] alhough he applicabiliy of hese argumens has also been quesioned [6, 8]. derivaives over scale o a single Gaussian blob: fx,y gx,y; 0 1 e x +y 0 π 0 Due o he semi-group propery of he Gaussian kernel g, ; 1 g, ; g, ; he scale-space represenaion of f obained by Gaussian smoohing is given by 1 Lx, y; gx,y; 0 + π 0 + e x +y The Pure Second-Order Ineres Poin Deecors By differeniaion, if follows ha he scale normalized signed or unsigned feaure srengh measure a he cener x, y 0, 0 of he blob will for he Laplacian 5, he deerminan of he Hessian 6 and he Hessian feaure srengh measures I 8 and II 10 be given by γ norm L γ 0, 0; π de H γ norm L0, 0; D 1,γ norm L0, 0; D,γ norm L0, 0; γ 4π kγ 4π γ π By differeniaing hese expressions wih respec o he scale parameer and seing he derivaive o zero, i follows ha he exremum value over scale will for all hese descripors be assumed a he same scale ˆ γ γ 0 9 For he specific choice of γ 1, he seleced scale ˆ will be equal o he scale of he Gaussian blob, i.e. ˆ 0, and he exremum value over scale for each one of he respecive feaure deecors is norm L max π 0 de H norm L max 1 64π k D 1,norm L max 64π 0 3 D,norm L max 1 8π 0 33

10 186 J Mah Imaging Vis : Table 1 Relaionships beween scale-normalized hresholds C DL for differen ypes of scale-invarian ineres poin deecors DL L,deHL, D 1 L, D1 L, D L and D L using scale-normalized derivaives wih γ 1. The complemenary expression for he Harris-Laplace operaor is based on he assumpion of a relaive inegraion scale of r 1 Feaure deecor DL C DL Laplacian L norm L xx + L yy C L C deerminan of he Hessian de H norm L L xx L yy L xy C de HL C /4 Hessian feaure srengh I D 1,norm L L xx L yy L xy kl xx + L yy C D1 L 1 4kC /4 Hessian feaure srengh Ĩ D1,norm L L xx L yy L xy ± kl xx + L yy C D 1 L 1 4kC /4 Hessian feaure srengh II D,norm min L pp, L qq C D L C/ Hessian feaure srengh ĨI D,norm L L pp or L qq C D L C/ Harris-Laplace H norm de μ k race μ C H 1 4kC 4 /56 These resuls are in full agreemen wih earlier resuls abou he scale selecion properies for Gaussian blobs concerning he scale-normalized Laplacian and he scale-normalized deerminan of he Hessian [31, Sec ] [35, Sec. 5.1] Scale Invarian Feaure Responses Afer Conras Normalizaion When applying differen ypes of ineres poin deecors in parallel, some approach is needed for expressing comparable hresholds beween differen ypes of ineres poin deecors. Le us assume ha such calibraion of corresponding hresholds beween differen ineres poin deecors can be performed based on he heir responses o Gaussian blobs. If we would like o presen a Gaussian blob on a screen and would like o make i possible o vary is size spaial exen wihou affecing is perceived brighness on he screen, le us assume ha his can be performed by keeping he conras beween he maximum and he minimum values consan. Le us herefore muliply he ampliude of he original Gaussian blob f by a facor π 0 so as o obain an inpu signal wih uni conras as measured by he range beween he minimum and maximum values. Then, he maximum value over scale of he conras normalized Gaussian blob will be given by Noe: For he Harris operaor [16], which is deermined from he second-momen marix μx, y;,s L x u, v; L xu, v; L y u, v; u,v R L x u, v; L y u, v; L y u, v; gx u, y v; sdudv 38 according o H norm de μ k race μ 39 for some k ]0, 1 4 [, a corresponding analysis shows ha he response a he cener x, y 0, 0 of a Gaussian blob is a scale 0 given by H max 1 4kr4 56r if we le he inegraion scale s be relaed o he local scale according o s r. This value herefore expresses he magniude value ha will obained by applying he Harris- Laplace operaor [49] o a Gaussian blob wih uni conras, provided ha scale selecion is performed using scalenormalized derivaives wih γ 1. In all oher respecs, he scale selecion properies of he Harris-Laplace operaor are similar o he scale selecion properies of he Laplacian operaor. norm L max 1 de H norm L max 1 16 D 1,norm L max 1 4k 16 D,norm L max The Rescaled Level Curve Curvaure Operaor When applying he rescaled level curve curvaure operaor κ γ norm L o a roaionally symmeric Gaussian blob we obain κ γ norm L γ x + y 8π x +y e These expressions provide a way o express muually relaed magniude hresholds for he differen ineres poin deecors as shown in Table 1. This expression assumes is spaial exremum on he circle x + y

11 J Mah Imaging Vis : where he exremum value is γ κ γ norm L 1eπ and his eniy assumes is exremum over scale a κγ norm L γ 5 γ In he special case when γ 7/8 [4] his corresponds o κl σ κl σ 0 45 wih he corresponding scale-normalized response κl max 5 γ γ eπ γ γ γ and he following approximae relaion for γ 7/8 ifhe Gaussian blob is normalized o uni conras κl max / Due o he use of a γ -value no equal o one, his magniude measure is no fully scale invarian. The scale dependency can, however, be compensaed for by muliplying he maximum feaure response over scale by a scale-dependen compensaion facor 1 γ. 3. Scale Selecion wih Complemenary Pos-smoohing When linking image feaures a differen scales ino feaure rajecories, he use of pos-smoohing of any differenial expression D norm L according o 18 was proposed in [39] o simplify he ask for he scale linking algorihm, by suppressing small local perurbaions in he responses of he differenial feaure deecors a any single scale. Since his complemenary pos-smoohing operaion will affec he magniude values of he scale-normalized differenial responses ha are used in he differen ineres poin deecors, one may ask how large effec his operaion will have on he resuling scale esimaes. In his secion, we shall analyze he influence of he possmoohing operaion for scale selecion based on local exrema over scale of scale-normalized derivaives The Laplacian and he Deerminan of he Hessian Operaors Consider again a roaionally symmeric Gaussian blob wih is scale-space represenaion of he form 4. Then, he scale-normalized Laplacian γ norml and he scalenormalized deerminan of he Hessian de H γ norm L are given by γ norm L γ x + y 0 + π de H γ norm L γ 0 + x y 4π e x +y e x +y Wih complemenary Gaussian pos-smoohing wih scale parameer pos c, he resuling differenial expressions assume he form γ norm L γ x + y c π c 3 de H γ norm L e x +y 0 +1+c 50 x +y γ c x y 4π c 3 e 0 +1+c 51 and assume heir exremal scale-normalized responses over scale a ˆ L γ c γ ˆ de HL 1 + c + c 4 1 γ 1 + c 1 γ 1 + c γ In he specific case when γ 1 and c 1/, hese local exrema over scale are given by ˆ L ˆσ L σ 0 54 ˆ de HL ˆσ de HL σ 0 55 In oher words, by comparison wih he resuls in Sec , we find ha he use of a pos-smoohing operaion wih inegraion scale deermined by c 1/, he scale esimaes will be abou 10 % lower when measured in unis of σ. To obain unbiased scale esimaes ha lead o ˆ 0 for a Gaussian blob, we can eiher muliply he scale esimaes by correcion facors from 5 and 53 or choose γ as funcion of c according o γ L 1 + c + c 56 γ de HL + 3c 1 + c 57 Wih c 1/, he laer seings correspond o he following values of γ : γ L

12 188 J Mah Imaging Vis : γ de HL The Hessian Feaure Srengh Measure I To analyze he effec of he pos-smoohing operaion for he Hessian feaure srengh measure I compued for a Gaussian blob, which is given by D 1,γ norm L γ x y + κ 0 + x y e x +y 0 + 4π provided ha his eniy is posiive, le us iniially disregard he effec of he local condiion de HL k race HL>0 in 8 and inegrae he closed-form expression 60 over he enire image plane R insead of over only he finie region where his eniy is posiive 1 4κ 1 4κ x + y κ Then, complemenary pos-smoohing wih inegraion scale pos c implies ha his approximaion of he possmoohed differenial eniy is given by D 1,γ norm L0, 0; γ 1 4κ 1 4κc 8κc c 1 4κ κ0 / 4π c 3 6 Corresponding inegraion wihin he finie suppor region 61 where D 1 > 0 gives an expression ha is oo complex o be wrien ou here. Unforunaely, i is hard o analyze he scales a which hese eniies assumes local exrema over scale, since differeniaion of he above menioned expression and solving for is roos leads o fourh-order equaions. In he case of γ 1, c 1/ and κ 0.04, we can, however, find he numerical soluion ˆ ˆσ 0.90 σ 0 63 For hese parameer seings, he use of a spaial possmoohing operaion does again lead o scale esimaes ha are abou 10 % lower. If we resric ourselves o he analysis of a single isolaed Gaussian blob, a similar approximaion holds for he signed Hessian feaure srengh measure D 1,γ norm L The Hessian Feaure Srengh Measure II For he Hessian feaure srengh measure II 10, we also have a corresponding siuaion wih a logical swiching beween wo differenial eniies L pp and L qq wih L pp and L qq deermined by 1 and 13. Solving for boundary beween hese domains, which is deermined by L pp + L qq 0, gives ha we should selec L qq wihin he circular region x + y and L pp ouside. Solving for he corresponding inegrals gives L qqinside γ c c e c / π c γ e + 0 c L ppouside 1 π c wih D L L qqinside + L pp ouside 67 Unforunaely, i is again hard o solve for he local exrema over scale of he pos-smoohed derivaive expressions in closed form. For his reason, le us approximae he composed expression D L by he conribuion from is firs erm 4 L qqinside and wih he inegral exended from he circular region 64 o he enire image plane γ c L qq π c 68 Then, he local exrema over scale are given by he soluions of he hird-order equaion c 4 1 γ c 3γ c 4 γ 1 + 3c 3γ + 4 γ where he special case wih γ 1 and c 1/ has he numerical soluion ˆ ˆσ σ 0 70 For he D,norm L operaor and hese parameer values, he use of a spaial pos-smoohing operaion does herefore lead 4 This approximaion may be reasonable for small values of c for which he major conribuion of he pos-smoohing inegraion originaes from values of D L near he ineres poin.

13 J Mah Imaging Vis : o scale esimaes ha are abou 16 % lower, and he influence is herefore sronger han for he Laplacian norm L, deerminan of he Hessian de H norm L or he Hessian feaure srengh D 1,norm L operaors. If we resric ourselves o he analysis of a single isolaed Gaussian blob, a similar approximaion holds for he signed Hessian feaure srengh measure D,γ norm L The Rescaled Level Curve Curvaure Operaor If we apply pos-smoohing o he rescaled level curve curvaure compued for a roaionally symmeric Gaussian blob 41 wih pos-smoohing scale pos c, we obain κ γ norm L γ 0 + x + y + c c 4 8π c 3 e 3x +y c 71 This eniy assumes i spaial exremum on he circle x + y 0 + 9c and he exremum value on his circle is κ γ norm L exr γ 1π c 7 e 3x +y c 73 By differeniaing his expression wih respec o he scale parameer, i follows ha he seleced scale level will be a soluion of he hird-order equaion 1 + 3c 5 γ c 4 6γ + 6c 3 γ c 3 6γ 6γ 0 γ Unforunaely, he closed form expression for he soluion is raher complex. Neverheless, we can noe ha due o he homogeneiy of his equaion, he soluion will always be proporional he scale 0 of he original Gaussian blob. In he specific case wih γ 7/8 and c 1/ we obain κγ norm L σ κγ norm L 0.70 σ 0 75 In oher words, compared o he case wihou pos-smoohing 45, he relaive difference beween he seleced scale levels is here less han 5 %, when measured in unis of σ. 3.3 Influence of Affine Image Deformaions To analyze he behaviour of he differen ineres poin deecors under image deformaions, le us nex consider an anisoropic Gaussian blob as a prooype model of a roaionally symmeric Gaussian blob ha has been subjeced o an affine image deformaion ha we can see as represening a local linearizaion of he perspecive mapping from a surface pach in he world o he image plane. Specifically, we can model he effec of foreshorening by differen spaial exens 1 and along he differen coordinae direcions 1 fx,y gx; 1 gy; π e 1 x 1 y 76 where he raio beween he scale parameer 1 and is relaed o he angle θ beween he normal direcions of he surface pach and he image plane according o σ σ 1 1 cos θ 77 if we wihou loss off generaliy assume ha 1. Since all he feaure deecors we consider are based on roaionally invarian differenial expressions, i is sufficien o sudy he case when he anisoropic Gaussian blob is aligned o one he coordinae direcions. Due o he semi-group propery of he one-dimensional Gaussian kernel, he scalespace represenaion of f is hen given by Lx, y; gx; 1 + gy; + 1 π e x 1 + y + 78 Noe on Relaion o Influence Under General Affine Transformaions A general argumen for sudying he influence of non-uniform scaling ransformaions can be obained by decomposing a general wo-dimensional affine ransformaion marix A ino [3] A R 1 diagσ 1,σ R 1 79 where R 1 and R can be forced o be roaion marices, if we relax he requiremen of non-negaive enries in he diagonal elemens σ 1 and σ of a regular singular value decomposiion. Wih his model, he geomeric average of he absolue values of he diagonal enries σ uniform σ 1 σ 80 corresponds o a uniform scaling ransformaion. We know ha he Gaussian scale-space is closed under uniform scaling ransformaions, roaions and reflecions. The differenial expressions we use for deecing ineres poins are based on roaionally invarian differenial invarians, which implies ha he scale esimaes will also be

14 190 J Mah Imaging Vis : roaionally invarian. Furhermore, our scale esimaes are ransformed in a scale covarian way under uniform scaling ransformaions. Hence, if we wihou essenial loss of generaliy disregard reflecions and assume ha σ 1 and σ are boh posiive, he degree of freedom ha remains o be sudied concerns non-uniform scaling ransformaions of he form 1 σ1 σ diagσ 1,σ diag σ1 σ, σ s, 1 diag 81 s whose influence on he scale esimaes will be invesigaed in his secion The Laplacian operaor For he Laplacian operaor, he γ -normalized response as funcion of space and scale is given by γ norm L x, y; γ e x / 1 + y / + π / x y y + x x 1 y 8 This eniy has criical poins a he origin x, y 0, 0 and a x, y ±, x, y 0, ± where he firs pair of roos corresponds o saddle poins if 1 >, while he oher pair of roos correspond o local exrema. Unforunaely, he criical poins ouside he origin lead o raher complex expressions. We shall herefore focus on he criical poin a he origin, for which he seleced scales will be he roos of he hird-order equaion γ norm L0, 0; γ 1 4π / 4γ 3 3γ γ1 + 8γ γ γ σ 1 For a general value of γ, he explici soluion is oo complex o be wrien ou here. In he specific case of γ 1, however, we obain for 1 L 1 /3 R R 4 86 where R R R R R R which in he special case of 1 0 reduces o L 0 91 If we on he oher hand reparameerize he scale parameers 1 and of he Gaussian blob as 1 s 0 and 0 /s, corresponding o a non-uniform scaling ransformaion wih relaive scaling facor s>1renormalized such he deerminan of he ransformaion marix is equal o one, hen a Taylor expansion of L around s 1gives L s s s 14 + O s From his resul we ge an approximae expression for how he Laplacian scale selecion mehod is affeced by affine ransformaions ouside he similariy group. Specifically, we can noe ha he scales seleced from local exrema over scale of he scale-normalized Laplacian operaor are no invarian under general affine ransformaions The Deerminan of he Hessian By differeniaion of 78 i follows ha he scale-normalized deerminan of he Hessian is given by de H γ norm Lx, y; γ x y + 1 x 1 y e x + 1 y + 4π

15 J Mah Imaging Vis : This expression does also have muliple criical poins. Again, however, we focus on he cenral poin x, y 0, 0, for which he derivaive wih respec o scale is of he form de H γ norm L 0, 0; γ 1 γ + γ γ 1 π This equaion has a posiive roo a de HL 1 4 γγ 1 + γ 1 γ γ which in he special case of γ 1 simplifies o he affine covarian expression de HL 1 96 Noably, if we again reparameerize he scale parameers according o 1 s 0 and 0 /s, hen for any non-uniform scaling ransformaion renormalized such ha he deerminan of he ransformaion marix is one, i holds ha de HL 0 97 which implies ha in his specific case and wih γ 1 scale selecion based on he scale-normalized deerminan of he Hessian leads o affine covarian scale esimaes for he Gaussian blob model. In his respec, here is a significan difference o scale selecion based on he scale-normalized Laplacian, for which he scale esimaes will be biased according o 86 and 9. For oher values of γ, a Taylor expansion of de HL around s 1gives de HL γ s γ s γ 4γ + γ 3 s O s 1 5 γ γ 0 98 implying a cerain dependency on he relaive scaling facor s. Provided ha γ 1 < 1/, his dependency will, however, be lower han for he Laplacian scale selecion mehod 9 wih γ The Hessian Feaure Srengh Measure I For he Hessian feaure srengh measure I, he behaviour of he scale-normalized response a he origin is given by D 1,γ norm 0, 0; γ 1 4k + 1 4k 1 + k / 4π provided ha his eniy is posiive. If we differeniae his expression wih respec o he scale parameer and se he derivaive o zero, we obain a fourh-order equaion, which in principle can be solved in closed form, bu leads o very complex expressions, even when resriced o γ 1. If we reparameerize he scale parameers according o 1 s 0 and 0 /s and hen resric he parameer k in D 1,γ norm o k 0.04, however, we can obain a manageable expression for he Taylor expansion of he seleced scale D1 L as funcion of he non-uniform scaling facor s in he specific case of γ 1 D1 L s 1 1 s s 14 + O s From his expression we can see ha he scales seleced from local exrema over scale of he scale-normalized Hessian feaure srengh measure I are no invarian under nonuniform scaling ransformaions. For values of s reasonably close o one, however, he deviaion from affine invariance is quie low, and significanly smaller han for he Laplacian operaor 9. This could also be expeced, since a major conribuion o he Hessian feaure srengh measure D 1,γ norm originaes from he affine covarian deerminan of he Hessian de H γ norm L The Hessian Feaure Srengh Measure II Wih 1 >, he Hessian feaure srengh measure II a he origin is given by D,γ norm L0, 0; L ξξ 0, 0; π γ 101 This eniy assumes is local exremum over scale a 1 D L γ γ + 16 γγ 1 + γ γ 10 which in he case when γ 1 reduces o D L

16 19 J Mah Imaging Vis : If we again reparameerize he scale parameers according o 1 s 0 and 0 /s and in order o obain more compac expressions resric ourselves o he case when γ 1, hen a Taylor expansion of D L around s 1gives D L s s s s 14 + O s From a comparison wih 9, 97 and 100 we can see ha he scales seleced from he scale-normalized Hessian feaure srengh measure II are more sensiive o non-uniform scaling ransformaions han he scales seleced by he scalenormalized Laplacian norm L, he deerminan of he Hessian de H norm L or he Hessian feaure srengh measure D 1,norm The Rescaled Level Curve Curvaure Operaor For he anisoropic Gaussian blob model, a compuaion of he rescaled level curve curvaure operaor κ γ norm L gives κ γ norm L γ x + y + 1 y + x 8π / e 3 x y This eniy assumes is spaial maximum on he ellipse x y + 3 and on his ellipse i holds ha γ 106 κ γ norm L 1eπ / 107 By differeniaing his expression wih respec o and seing he derivaive o zero, i follows ha he exremum over scale is assumed a 1 κl 5 4γ γ + 3γ5 γ γ which in he special case when γ 5/4 reduces o he affine covarian expression κl By again reparameerizing he scale parameers in he Gaussian blob model according o 1 s 0 and 0 /s and performing a Taylor expansion around s 1 for a general value of γ, i follows ha κl 1 5 4γ s γ s γ5 + 30γ 8γ s O s 1 5 γ 5 γ which in he case wih γ 7/8 assumes he form κl s s s 14 + O s wih second- and hird-order relaive bias erms abou hree imes he magniude compared o he Hessian feaure srengh measure I in Scale Selecion by Weighed Averaging Along Feaure Trajecories The reamen so far has been concerned wih scale selecion based on local exrema over scale of scale-normalized derivaives. Concerning he new scale selecion mehod based on weighed averaging over scale of scale-normalized derivaive responses, an imporan quesion concerns how he scale esimaes from his new scale selecion mehod are relaed o he scale esimaes from he previously esablished scale selecion based on local exrema over scale. In his secion, we will presen a corresponding heoreical analysis of weighed scale selecion concerning he basic quesions: How are he scale esimaes relaed beween he differen ineres poin deecors? Sec. 4.1 How much does he pos-smoohing operaion influence he scale esimaes? Sec. 4. How are he scale esimaes influenced by affine image deformaions? Sec. 4.3 Given ha image feaures x a differen scales have been linked ino a feaure rajecory T over scale T { x; : [ min, max ] } 11 scale selecion by weighed averaging over scale implies ha he scale esimae is compued as [39] τ T ˆτ T τψ D γ normlxτ; τ dτ τ T ψ D γ normlxτ; τ dτ max min log ψ D γ norm Lx; d max min ψ D γ norm Lx; d 113 for some posiive and monoonically increasing ransformaion funcion ψ of he magniude values D γ norm Lxτ; τ of he differenial feaure responses.

17 J Mah Imaging Vis : Specifically, he following family of scale invarian ransformaion funcions was considered ψ D γ norm L w DL D norm L a 114 where w DL [0, 1] is a so-called feaure weighing funcion ha measures he relaive srengh of he feaure deecor DL compared o oher possible compeing ypes of feaure deecors and a is he scalar parameer in he self-similar power law. In his secion, we shall analyze he scale selecion properies of his consrucion for he differenial feaure deecors defined in Sec.. under he simplifying assumpions of w DL 1 and a 1. Wih respec o he analysis a he cener of a Gaussian blob, he assumpion of w DL 1is paricularly relevan for he weighing funcions of he form w DL L ξξ + L ξη + L ηη AL ξ + L η + L ξξ + L ξη + L ηη + ε 115 considered in [39] if we make use of he fac ha L ξ L η 0 a any criical poin as a he cener of a Gaussian blob and disregard he influence of he noise suppression parameer ε. 4.1 The Pure Second-Order Ineres Poin Deecors From he explici expression for he magniude of he scalenormalized Laplacian response a he cener of roaionally symmeric Gaussian blob 5 i follows ha he weighed scale selecion esimae according o 113 will in he case of γ 1 5 and wih he effecive scale parameer τ defined as τ log be given by ˆτ L 0 log d π d π 0 + log 0 π 0 1 log π 0 Similarly, from he explici expression for he deerminan of he Hessian a he cener of he Gaussian blob 6, i follows ha he weighed scale esimae will be deermined by ˆτ de HL 0 0 log d 4π d 4π log 0 4π 0 1 4π 0 log Due o he similariy beween he explici expressions for he Hessian feaure srengh measure I 7 and he deerminan of he Hessian response 6 as well as he similariy 5 In his secion we will in many cases resric he analysis o he specific case of γ 1, since some of he resuls become significanly more complex for a general value of γ 1. In a few cases where he corresponding resuls become reasonably compac, we will, however, include hem. beween he Hessian feaure srengh measure II 8 and he Laplacian response 7 a he cener of a Gaussian blob, he scale esimaes for D 1,γ norm L and D,γ norm L will be analogous: ˆτ D1 L ˆτ D L 1 4k log d 4π k d 4π log d π 0 + d π k log 0 4π 0 1 4k 4π 0 log log 0 π 0 1 log π 0 When expressed in erms of he regular scale parameer ˆ e ˆτ 0 10 he weighed scale selecion mehod does hence for a roaionally symmeric Gaussian blob lead o similar scale esimaes as are obained from local exrema over scale of γ - normalized derivaives 9 when γ 1. Since hese scale esimaes are similar o he scale esimaes obained form local exrema over scale, i follows ha he scale-normalized magniude values will also be similar and he relaionships beween scale-normalized hresholds described in Table 1 will also hold for scale selecion based on weighed averaging over scale. Corresponding Scale Esimaes for General Values of γ For a general value of γ ]0, [, he corresponding scale esimaes become as follows in erms of effecive scale τ log : ˆτ L ˆτ D L log 0 ˆτ de HL ˆτ D1 L π1 γcoπγ γ π cscπγ 4γ 3γ + 11 γ log 0 Γ4 γγγ + πγ 1γ 3γ 1 coπγ log 0 1γ γ 11 1 where boh expressions have he limi value log 0 when γ 1. Noe ha coπγ when γ 1. By comparing hese scale esimaes o he corresponding scale esimae ˆτ log γ γ in 9 obained from local exrema over scale, we can compue a Taylor expansions of he

18 194 J Mah Imaging Vis : difference in he scale esimaes: γ ˆτ L log γ π 3 γ π 4 30 γ O γ γ γ O γ γ ˆτ de HL log γ π 8π γ γ O γ γ γ O γ Noably, he difference in scale esimaes beween he wo ypes of scale selecion approaches is smaller 6 for scale selecion using he deerminan of he Hessian de HL or he Hessian feaure srengh measure D 1,norm L compared o scale selecion based on he Laplacian norm L or he Hessian feaure srengh measure D,norm L. 4. Influence of he Pos-smoohing Operaion 4..1 The Laplacian and he Deerminan of he Hessian Operaors From he explici expressions for he pos-smoohed Laplacian 50 and he pos-smoohed deerminan of he Hessian 51, i follows ha he weighed scale esimaes are 6 A plausible explanaion why he difference beween he scale esimaed is smaller for he deerminan of he Hessian de HL and he Hessian feaure srengh measure D 1,norm L compared o difference in scale esimaes for he Laplacian norm L and he Hessian feaure srengh measure D,norm L is ha second-order derivaive responses are squared for he deerminan of he Hessian de HL and he Hessian feaure srengh measure D 1,norm L, whereas he Laplacian norm L and he Hessian feaure srengh measure D,norm L operaors depend on he second-order derivaive responses in a linear way. Thereby, he inegrals ha define he weighed scale selecion esimaes will ge a comparably higher relaive conribuion from scale levels near he maximum over scale, which in urn implies ha he influence due o skewness in he scale-space signaure caused by values of γ 1 will be lower compare wih Sec By varying he power a in he self-similar ransformaion funcion 114, i is more generally possible o modulae his effec. for γ 1 given by 0 ˆτ L 0 +1+c log d π 0 +1+c 3 0 log 0 1+c π 0 1+c 1 π 0 1+c 0 log ˆτ de HL 0 +1+c d π 0 +1+c c 0 log 0 +1+c 4π c c 4π c 3 c 1+c log1+c log 1+c 0 3c 6 π 0 c 1+c log1+c 16c 6 π 0 log c which agree wih he corresponding scale esimaes 5 and 53 for local exrema over scale of pos-smoohed γ - normalized derivaive expressions. In he specific case when c 1/ hese scale esimaes reduce o ˆ L ˆσ L σ 0 17 ˆ de HL ˆσ de HL σ 0 18 again agreeing wih our previous resuls 54 and 55 regarding scale selecion based on local exrema over scale of pos-smoohed scale-normalized derivaive expressions. 4.. The Hessian Feaure Srengh Measure I To analyze he effec of weighed scale selecion for he Hessian feaure srengh measure I, corresponding weighed inegraion over scale of he approximaion 6 of he possmoohed differenial eniy D 1 L for γ 1gives ˆτ D1 L 0 D 1,γ norml log d 0 D 1,γ norml d θ D1 L + log 0 19 where θ D1 L N D 1 L D D1 L and N D1 L 1 + 3c + c κlog 1 + c 130 c 1 + κ + c 1 + κ log 1 + c 8c 6 κ 131

19 J Mah Imaging Vis : D D1 L 1 + 3c + c κlog 1 + c c 1 + κ + c 1 + κ+ c 4 κ 13 In he specific case of c 1/ and κ 0.04, his scale esimae is given by ˆ D1 L ˆσ D1 L 0.90 σ in agreemen wih our previous resul 63 regarding scale selecion based on local exrema over scale of pos-smoohed scale-normalized derivaive expressions The Hessian Feaure Srengh Measure II To analyze he effec of pos-smoohing of he Hessian feaure srengh measure II in 10, le us again approximae he composed pos-smoohed differenial expression in 67 by he conribuion from is firs erm 68 wih he spaial inegraion exended o he enire plane. Then, he weighed scale esimae can be approximaed by ˆτ D L 0 L qq log d 0 L qq d θ D L + log where θ D L log1 + c 1 + c log1 + c 4c 1 + c log1 + c c 135 In he specific case when c 1/ his scale esimae reduces o ˆ D L ˆσ D1 L σ in agreemen wih our previous resul 70 regarding scale selecion based on local exrema over scale of pos-smoohed scale-normalized derivaive expressions. 4.3 Influence of Affine Image Deformaions To analyze how he scale esimaes ˆ obained by weighed averaging along feaure rajecories are affeced by affine image deformaions, le us again consider an anisoropic Gaussian blob 76 as a prooype model of a roaionally symmeric Gaussian blob ha has been subjeced o an affine image deformaion and wih is scale-space represenaion according o The Laplacian Operaor A he origin, he scale-normalized Laplacian response according o 8 reduces o norm L , 0; π 1 + 3/ + 3/ 137 and he scale esimae obained by weighed scale selecion is given by 0 ˆτ L log d π 1 + 3/ + 3/ d π 1 + 3/ + 3/ 4 log π 1 1 π 1 log Wih a reparameerizaion of he scale parameers 1 and of he Gaussian blob as 1 s 0 and 0 /s, corresponding o a non-uniform scaling ransformaion wih relaive scaling facor s>1 renormalized such he deerminan of he ransformaion marix is equal o one, he scale esimae in unis of can be wrien ˆ L e ˆτ L s 1 + s Noably, his scale esimae is no idenical o he scale esimae 86 obained from local exrema over scale. The Taylor expansion of L around s 1 is in urn given by L s s s 14 + O s and is, however, similar unil he hird-order erms o he Taylor expansion 9 of he corresponding scale esimae obained from local exrema over scale. In his respec, he behaviour of he wo scale selecion mehods is qualiaively raher similar when applied o he anisoropic Gaussian blob model The Deerminan of he Hessian A he origin, he response of he deerminan of he Hessian operaor 93 simplifies o de H norm L0, 0; 4π and he scale esimae obained by weighed scale selecion is ˆτ de HL 0 log d 4π d 4π 1 + +

20 196 J Mah Imaging Vis : log 1 log 1 log 1 8π log 1 log 1 4π 1 3 log 1 corresponding o he affine covarian scale esimae 14 ˆ de HL e ˆτ de HL and in agreemen wih our earlier resul 96 for scale selecion from local exrema over scale The Hessian Feaure Srengh Measure I Wih he Hessian feaure srengh measure I a he origin given by 99, he scale esimae obained by weighed scale selecion is deermined by ˆτ D1 L 0 D 1,γ norml log d 0 D 1,γ norml d where N D 1 L D D1 L N D1 L κ κ log 1 log 1 log 1 3κ 1 + κ log κ D D1 L 1 κ κ log 1 log 144 Wih a reparameerizaion of he scale parameers according o 1 s 0 and 0 /s, his expression simplifies o ˆτ D1 L N D 1 L D D 1 L wih 145 N D 1 L κ 1 s 1 + s log s + 1 s 146 D D 1 L 41 κ 1 + s s log s + 1 s κ 1 6s + s 4 + 4s 147 A Taylor expansion around s 1 of he scale esimae expressed in unis of exp gives D1 L 1 + κ 1 4κ s 1 κ s κ + 5κ 66κ 0 1 4κ s 14 + O s which simplifies o he following form for κ 0.04 D1 L s 1 1 s s 14 + O s and agreeing unil he hird-order erms wih he corresponding Taylor expansion 100 for he scale esimae obained from local exrema over scale. Specifically, a comparison wih he corresponding expression for he Laplacian operaor 140 shows ha scale selecion based on he Hessian feaure srengh measure I is less sensiive o affine image deformaions compared o scale selecion based on he Laplacian The Hessian Feaure Srengh Measure II Assuming ha 1, he Hessian feaure srengh measure II a he origin is given by D,norm L0, 0; L xx 0, 0; π and he weighed scale esimae ˆτ D L 0 π π log d d log log 16 log π π Wih he scale parameers reparameerized according o 1 s 0 and 0 /s, he corresponding scale esimae can be wrien ˆ D L e ˆτ D L s 4s 1 s s 15 for which a Taylor expansion around s 1gives D L s s s s 14 + O s and agreeing unil he second-order erms wih he corresponding Taylor expansion 104 for he scale esimae obained from local exrema over scale.

21 J Mah Imaging Vis : Again, he scale esimaes for scale selecion based on he Hessian feaure srengh measure II are more affeced by affine image deformaions compared o he scale esimaes obained by he deerminan of he Hessian, he Hessian feaure srengh measure I or he Laplacian. 5 Relaions Beween he Scale Selecion Mehods 5.1 Roaionally Symmeric Gaussian Blob From he above menioned resuls, we can firs noe ha for he specific case of a roaionally symmeric Gaussian blob, he scale esimaes obained from local exrema over scale vs. weighed averaging over scale are very similar. Table shows he scales ha are seleced for he Laplacian norm L and he deerminan of he Hessian de H norml in he presence of a general pos-smoohing operaion. Table 3 shows corresponding approximae esimaes for he Hessian feaure srengh measure D 1,norm L and he Hessian feaure srengh measure D,norm L for c 1/. Noably, he exac scale esimaes agree perfecly, whereas he approximae esimaes are very similar. In his sense, he wo scale selecion mehods have raher similar effecs when applied o a roaionally symmeric Gaussian blob Theoreical Symmery Properies Beween he Scale Esimaes The similariy beween he resuls of he wo scale selecion mehods can generally be undersood by sudying he scalespace signaures ha show how he Laplacian and he deerminan of he Hessian responses evolve as funcion of scale Table Exac scale esimaes obained from local exrema over scale vs. weighed averaging over scale for he Laplacian and deerminan operaors applied o a roaionally symmeric Gaussian blob wih scale parameer 0 and for a general amoun of pos-smoohing as deermined by he pos-smoohing parameer c Operaor Exrema over scale Weighed averaging norm L 0/1 + c 0 /1 + c de H norm 0 / 1 + c 0 / 1 + c Table 3 Approximae scale esimaes obained from local exrema over scale vs. weighed averaging over scale for he Hessian feaure srengh measures I and II applied o a roaionally symmeric Gaussian blob wih scale parameer 0 and for a specific amoun of possmoohing wih c 1/ Operaor Exrema over scale Weighed averaging D 1,norm L D,norm L a he cener of he blob below assuming no pos-smoohing corresponding o c 0: norm L 0, 0; π de H norm L0, 0; 4π The lef column in he upper and middle rows in Fig. 3 shows hese graphs wih a linear scaling of he regular scale parameer and he righ column shows corresponding graphs wih a logarihmic scaling of he scale parameer in erms of effecive scale τ. As can be seen from he laer graphs, he scale-space signaures assume a symmeric shape when expressed in erms of effecive scale, which implies ha he weighed scale esimaes, which correspond o he cener of graviy of he graphs, will be assumed a a similar posiion as he global exremum over scale. This propery can also be undersood algebraically, due o he funcional symmery of 154 and 155 under mappings of he form 0 vs corresponding o he symmery log 0 + log log 0 log 157 Since he response properies of he Hessian feaure srengh measures D 1,norm L and D,norm L are of similar forms 1 4k D 1,norm L0, 0; 4π D,norm L0, 0; π corresponding symmery properies follow also for hese operaors. These symmery properies do also exend o monoonically increasing ransformaions ψ of he differenial responses of he form ψ norm L, ψ de H norm L 160 ψ D 1,norm L, ψ D,norm L 161 These symmery properies do, however, no exend o general values of γ 1, since such values may lead o a skewness in he scale-space signaure see he boom row in Fig Calibraion Facors for Seing Scale-Invarian Inegraion Scales The scale esimaes may, however, differ depending on wha differenial expression he ineres poin deecor is based on.

22 198 J Mah Imaging Vis : Fig. 3 Scale-space signaures compued from a Gaussian blob wih scale parameer 0 10 for op row he Laplacian norm L for γ 1, middle row he deerminan of he Hessian de H norm L for γ 1and boom row he second-order derivaive L ξξ for a 1-D signal and γ 3/4soasogiveriseo he peak over scale a ˆ 0 using lef column a linear scaling of he scale parameer or righ column a logarihmic ransformaion in erms of effecive scale τ log Hence, if we would like o se an inegraion scale in for compuing a local image descripor from he scale esimae ˆ DL, in such a way ha he inegraion scale should be he same in r 0 16 for any ineres poin deecor DL applied o a roaionally symmeric Gaussian blob, irrespecive of wheher he ineres poins are compued from scale-space exrema or feaure rajecories in a scale-space primal skech, we can parameerize he inegraion scale according o in r ˆ DL 163 A DL wih he calibraion facor A DL deermined from he resuls in Table Anisoropic Gaussian Blob 5..1 Taylor Expansions for Non-uniform Scaling Facors Near s 1 From he analysis of he scale selecion properies of an anisoropic Gaussian blob wih scale parameers 1 and in Sec. 3.3 and Sec. 4., we found ha scale selecion based Table 4 Calibraion facors A DL o obain compensaed scale esimaes ˆ DL,comp ˆ DL /A DL ha lead o ˆ DL,comp 0 for a roaionally symmeric Gaussian blob irrespecive of he ineres poin operaor DL or he pos-smoohing parameer c Operaor Calibraion facor A DL norm L 1/1 + c de H norm 1/ 1 + c D 1,norm e θ D 1L wih θ D1 L according o 130 D,norm e θ D L wih θ D L according o 135 on local exrema over scale or weighed scale selecion lead o a similar and affine covarian scale esimae 1 for he deerminan of he Hessian operaor de H norm L. For he Laplacian norm L and he Hessian feaure srengh measures D 1,norm L and D,norm L, he scale esimaes are, however, no affine covarian. Moreover, he wo scale selecion mehods may lead o differen resuls. When performing a Taylor expansion of he scale esimae parameerized in erms of a non-uniform scaling facor s relaive o a base-line scale 0, he Taylor expansions around s 1 did, however, agree in heir lowes order erms. In his sense, he wo scale selecion approaches have approximaely similar properies for he Gaussian blob model for affine image deformaions near he similariy group.

23 J Mah Imaging Vis : From a comparison beween he Taylor expansions for he scale esimaes for he differen ineres poin deecors in Table 5, we can conclude ha afer he affine covarian deerminan of he Hessian de H norm L, he scale esimae obained from Hessian feaure srengh measure D 1,norm L has he lowes sensiive o affine image deformaions followed by he Laplacian norm L and he Hessian feaure srengh measure D,norm L. Corresponding resuls hold for he corresponding signed Hessian feaure srengh measures D 1,norm L and D,norm L. 5.. Graphs of Non-uniform Scaling Dependencies for General s 1 From he analysis in Sec. 4.3 i follows from 139, 14, 144 and 15 ha for an anisoropic Gaussian blob wih scale parameers 1 s 0 and 0 /s, he scale esimaes for weighed scale selecion using he Laplacian norm L, deerminan of he Hessian de H norm L and he Hessian feaure srengh measures D 1,norm L and D,norm L are in he absence Table 5 Taylor expansions for he scale esimaes obained for an anisoropic Gaussian blob wih scale parameers 1 s 0 and 0 /s around s 1 assuming s>1forhed,norm L operaor. The able shows he erms in he Taylor expansion ha are common for scale selecion based on local exrema over scale and scale selecion based on weighed averaging over scale Operaor Common erms in series expansion of scale esimae norm L s s 13 + Os de H norm L 0 D 1,norm L s 13 + Os D,norm L s s 1 + Os of pos-smoohing c 0 given by ˆ L 4s 1 + s ˆ de HL κ1 s 1+s log s+1 s ˆ D1 L e 41 κ1+s s log s+1 s κ1 6s +s 4 +4s s 4s 1 s ˆ D L s 167 Figure 4 shows graphs of how he scale esimaes depend on he non-uniform scaling parameer s for scale selecion by weighed averaging over scale. As can be seen from hese graphs, he behaviour is qualiaively somewha differen for he four differenial expressions. For he deerminan of he Hessian de H norm L, he scale esimae coincides wih he geomeric average of he scale parameers for any non-singular amoun of non-uniform scaling. For he Laplacian operaor norm L, he scale esimae ˆ L is lower han he geomeric average of he scale parameers in he wo direcions, whereas he scale esimaes are higher han he geomeric average for he Hessian feaure srengh measures D 1,norm L and D,norm L. For moderae values of s [1, 4], he scale esimaes from he Hessian feaure srengh measure D 1,norm L, are quie close o he affine covarian geomeric average. For he Hessian feaure srengh measure D,norm L on he oher hand, he scale esimae increases approximaely linearly wih he non-uniform scaling facor s. These graphs also show ha he qualiaive behaviour derived for Taylor expansions near s 1 Table 5 exend o Fig. 4 Dependency of he scale esimaes ˆ DL on he amoun of non-uniform scaling s [1, 4] when performing scale selecion by weighed averaging over scale for a non-uniform Gaussian blob wih scale parameers 1 s 0 and 0 /s for upper lefhe Laplacian norm L,upper righ he deerminan of he Hessian de H norm L,lower lefhe Hessian feaure srengh I D 1,norm L and lower righhe Hessian feaure srengh I D,norm L.Horizonal axis: Non-uniform scaling facor s. Verical axis: Scale esimae ˆ DL in unis of 0

24 00 J Mah Imaging Vis : Fig. 5 Sample images from a daase wih 14 images used for experimens wih repeaabiliy properies under affine image deformaions. Image size: pixels non-infiniesimal scaling facors up o a leas a facor of four. 5.3 Comparison wih Experimenal Repeaabiliy Properies In his secion, we shall compare he above menioned heoreical resuls wih experimenal resuls of he repeaabiliy properies of he differen ineres poin deecors under affine image ransformaions Experimenal Mehodology Figure 5 shows a few examples of images from an image daa se wih 14 images from naural environmens. Each such image was subjeced o 10 differen ypes of affine image ransformaions encompassing: a pure scaling Us wih scaling facor s, a pure roaion Rϕ wih roaion angle ϕ π/4, and non-uniform scalings Ns wih scaling facors s 4 and s, respecively, which are repeaed and averaged over four differen orienaions respecively N ϕ0 s Rϕ 0 NsRϕ wih relaive orienaions of ϕ 0 0, π/4, π/ and 3π/4. For a locally planar surface pach viewed by a scaled orhographic projecion model, he non-uniform rescalings correspond o he amoun of foreshorening ha arises wih slan angles equal o 3.8 and 45, respecively. In his respec, he chosen deformaions reflec reasonable requiremens of robusness o viewing variaions for image-based maching and recogniion. For each one of he resuling images, he 400 mos significan ineres poins were deeced. For ineres poins deeced based on scale-space exrema, he image feaures were ranked on he scale-normalized response of he differenial operaor a he scale-space exremum. For ineres poins deeced by scale linking, he image feaures were ranked on a significance measure obained by inegraing he scale-normalized responses of he differenial operaor along each feaure rajecory, using he mehodology described in [39]. To make a judgemen of wheher wo image feaures A and B deeced in wo differenly ransformed images f and f should be regarded as belonging o he same feaure or no, we associaed a scale dependen circle C A and C B wih each feaure, wih he radius of each circle equal o he deecion scale of he corresponding feaure measured in unis of he sandard deviaion σ of he Gaussian kernel used for scale-space smoohing o he seleced scale, in a similar way as he graphical illusraions of scale dependen image feaures in previous secions. Then, each such feaure was ransformed o he oher image domain, using he affine ransformaion applied o he image coordinaes of he cener of he circle and wih he scale value ransformed o be proporional o he deerminan of he affine ransformaion marix, de A, resuling in wo new circular feaures C A and C B. The relaive amoun of overlap beween any pair of circles was defined by forming he raio beween he inersecion and he union of he wo circles in a similar way as [50] define a corresponding raio for ellipses mc A,C B C A,C B C A,C B 169 Maching relaions were compued in boh direcions and a mach was hen permied only if a pair of image feaures maximize his raio over replacemens of eiher image feaure by oher image feaures in he same domain and, in addiion, he value of his raio was above a hreshold mc A,C B >m 0, where we have chosen m Furhermore, only one mach was permied for each image fea-

25 J Mah Imaging Vis : Table 6 Relaive ranking of 10 scale-invarian ineres poin deecors based on scale selecion from scale-space exrema wih regard o heir repeaabiliy scores under a se of 10 differen affine image deformaions applied o each one of he 14 images in he image daase illusraed in Fig. 5 and he exracion of he 400 mos significan ineres poins from each image Scale selecion from local exrema over scale Feaure deecor Type Complemenary p 400 κ γ norm L exr D 1,norm L exr de H norm L exr D 1 L> de H norm L exr D1 L > D 1,norm L exr norm L exr D,norm L exr D 1 L> D,norm L exr D 1 L> norm L exr D 1L> Harris-Laplace exr Table 7 Relaive ranking of 10 scale-invarian ineres poin deecors based on scale selecion by scale linking and weighed averaging over scale wih regard o heir repeaabiliy scores under a se of 10 differen affine image deformaions applied o each one of he 14 images in he image daase illusraed in Fig. 5 and he exracion of he 400 mos significan ineres poins from each image Scale selecion by weighed averaging over scale Feaure deecor Type Complemenary p 400 D 1,norm L link-w de H norm L link-w D 1 L> D,norm L link-w D 1 L> de H norm L link-w D1 L > κ γ norm L link-w de H norm L link-w D 1,norm L link-w D,norm L link-w D 1 L> norm L link-w D 1L> Harris-Laplace link-w ure, and maching candidaes were evaluaed in decreasing order of significance. Finally, given ha a oal number of N mached feaures maches have been found from N feaures deeced from he image f and N feaures from he ransformed image f,he maching performance was compued as p N mached maxn, N 170 The maching performance was compued in boh direcions from f o f as well as from f o f and he average value of hese performance measures was repored. The evaluaion of he maching score was only performed for image feaures ha are wihin he image domain for boh images before and afer he ransformaion. Moreover, only feaures wihin corresponding scale ranges were evaluaed. In oher words, if he scale range for he image f before he affine ransformaion was [ min, max ], hen image feaures were searched for in he ransformed image f wihin he scale range [ min, max ][de A min,de A max ]. In addiion, feaures in a narrow scale-dependen frame near he image boundaries were suppressed, o avoid boundary effecs from influencing he resuls. In hese experimens, we used min 4 and max Relaions Beween Experimenal Resuls and Theoreical Resuls Table 6 and Table 7 show he average repeaabiliy scores obained for he 7 differen ineres poin deecors we have sudied in his work. For he Laplacian norm L, deerminan of he Hessian de H norm L and he Hessian feaure srengh measures D,norm L and D,norm L,wehavealsoapplied complemenary hresholding on eiher D 1,norm L>0 or D 1,norm L>0, which increases he robusness of he image feaures and improves he repeaabiliy scores of he ineres poin deecors. For comparison, we do also show corresponding repeaabiliy scores obained wih he Harris- Laplace operaor. Wih hese variaions, a oal number 10 differenial ineres poin deecors are evaluaed. Separae evaluaions are also performed for scale selecion from local exrema over scale vs. scale selecion by scale linking and weighed averaging over scale. As can be seen from Table 6, he bes repeaabiliy properies for he ineres poin deecors based on scale selecion from local exrema over scale are obained for i he rescaled level curve curvaure κ γ norm L, ii he Hessian feaure srengh measure D 1,norm L and iii he deerminan of he Hessian de H norm L. From Table 7, we can see ha he bes repeaabiliy properies for he ineres poin deecors based on scale selecion using scale linking and weighed averaging over scale are obained for i he Hessian feaure srengh measure D 1,norm L, ii he deerminan of he Hessian de H norm L and iii he Hessian feaure srengh measure D,norm L. The repeaabiliy scores are furhermore generally beer for scale selecion based on weighed averaging over scale compared o scale selecion based on local exrema over scale. In comparison wih our heoreical analysis, we have previously shown ha he response of he deerminan of he Hessian de H norm L o an affine Gaussian blob is affine covarian, for boh scale selecion based on local exrema over scale 97 and scale selecion based on scale linking and weighed averaging over scale 143. For he Hessian feaure

26 0 J Mah Imaging Vis : srengh measure D 1,norm L, a major conribuion o his differenial expression comes from he affine covarian deerminan of he Hessian de H norm L, and he deviaions from affine covariance are small for boh scale selecion based on local exrema over scale 100 and scale selecion by weighed averaging over scale 148, provided ha he nonuniform image deformaions are no oo far from he similariy group in he sense ha he non-uniform scaling facor s used in he Taylor expansions is no oo far from 1. Specifically, he wo ineres poin deecors ha have he bes heoreical properies under affine image deformaions in he sense of having he smalles correcion erms in Table 5 are also among he op hree ineres poin deecors for boh scale selecion based on local exrema over scale and scale selecion based on scale linking and weighed averaging over scale. In his respec, he predicions from our heoreical analysis are in very good agreemen wih he experimenal resuls. Somewha more surprisingly he signed Hessian feaure srengh measure D,norm L performs very well when combined wih scale selecion based on weighed averaging over scale. The corresponding unsigned eniy D,norm L does no perform as well, and more comparable o he Laplacian operaor norm L. A possible explanaion for his is ha keeping he signs of he principal curvaures in he non-linear minimum operaion improves he abiliy of his operaor o disinguish beween nearby compeing image srucures, a propery ha is no capured by he analysis of isolaed Gaussian blobs. The repeaabiliy properies of he unsigned version D,norm L are herefore in closer agreemen wih he presened analysis. The rescaled level curve curvaure κ γ norm L performs comparably very well for scale selecion based on local exrema over scale, whereas i does no perform as well for scale selecion based on scale linking and weighed averaging over scale. For scale selecion based on local exrema over scale, our analysis showed ha he deviaion from affine covariance is comparably low 111 forhe value of γ 7/8 ha we used in our experimens. For his scale selecion mehod, he experimenal resuls are herefore in agreemen wih our heoreical resuls. Conrary o he oher ineres poin deecors, he repeaabiliy properies of he rescaled level curve curvaure operaor κ γ norm L are, however, no improved by scale linking. A possible algorihmic explanaion o his could be ha he rescaled level curve curvaure operaor κ γ norm L conains a differen ype of non-lineariy ha may cause difficulies for he scale linking algorihm. Calculaing closed-form expressions for he scale esimaes obained by weighed averaging over scale does also seem harder for his operaor. We herefore leave i as an open problem o invesigae if also his ineres poin deecor could be improved by scale linking and scale selecion from weighed averaging of possibly ransformed magniude values along he corresponding feaure rajecories. Experimenal resuls in [39] show ha he Hessian feaure srengh measure D 1,norm L and he deerminan of he Hessian de H norm L and are also he wo ineres poin deecors ha give he bes repeaabiliy properies under real calibraed perspecive image ransformaions. Thus, he wo bes ineres poin deecors according o our heoreical analysis are also he ineres poin deecors ha have he bes properies for real image daa. 6 Summary and Discussion We have analyzed he scale selecion properies of i he Laplacian operaor norm L, ii he deerminan of he Hessian de H norm L, iii iv he new Hessian feaure srengh measures D 1,norm L and D,norm L and iv he rescaled level curve curvaure operaor κ γ norm L when applied o a Gaussian prooype blob model and using scale selecion from eiher vi local exrema over scale of scalenormalized derivaives or vii weighed averaging of scale values along feaure rajecories over scale. We have also analyzed viii he influence of a secondary pos-smoohing sep afer he compuaion of possibly non-linear differenial invarians and ix he sensiiviy of he scale esimaes o affine image deformaions. The analysis shows ha he scale esimaes from he deerminan of he Hessian de H norm L are affine covarian for he Gaussian blob model for boh scale selecion based on local exrema over scale and scale selecion by weighed averaging over scale. The analysis also shows ha he scale esimaes from he Laplacian operaor norm L and he Hessian feaure srengh measures D 1,norm L and D,norm L are no affine covarian. Ou of he laer hree operaors, he Hessian feaure srengh measure D 1,norm L has he lowes sensiiviy o affine image deformaions ouside he similariy group, whereas he Hessian feaure srengh measure D,norm L has he highes sensiiviy. The sronger scale dependency of he Hessian feaure srengh measure D,norm L can be undersood from he fac ha i responds o he eigenvalue of he Hessian marix corresponding o he slowes spaial variaions. Experimenal resuls repored in Sec. 5.3 and [39], show ha he ineres poin deecors based on he new Hessian feaure srengh measure D 1,norm L and he deerminan of he Hessian de H norm L have significanly beer repeaabiliy properies under affine or perspecive image ransformaions han he Laplacian norm L or he Hessian feaure srengh measure D,norm L. Corresponding advanages hold relaive o he difference-of-gaussians DoG approximaion of he Laplacian operaor or he Harris-Laplace operaor. Hence, he ineres poin deecors ha have he bes

27 J Mah Imaging Vis : heoreical properies under affine deformaions of Gaussian blobs do also have he bes experimenal properies. In his respec, he predicions from his heoreical analysis agree wih corresponding experimenal resuls. When considering scale selecion for a roaionally symmeric Gaussian blob, i is shown ha he scale esimaes obained by scale selecion from local exrema over scale vs. weighed averaging over scale do for γ 1 in he -D case lead o similar resuls for each one of hese four operaors. This similariy can be explained from a symmery propery of he scale-space signaure under inversion ransformaions of he scale parameer, which correspond o reflecions along he scale axis afer a logarihmic ransformaion of he scale parameer in erms of effecive scale. Because of his similariy beween he scale esimaes obained from he wo ypes of scale selecion approaches, we may conclude ha no addiional scale compensaion or scale calibraion is needed beween scale esimaes ha are obained from weighed averaging over scale vs. local exrema over scale provided ha γ 1. Since he commonly used difference-of-gaussians operaor can be seen as a discree approximaion of he Laplacian operaor [41], he analysis of he scale selecion properies for he Laplacian operaor also provides a heoreical model for analyzing he scale selecion properies of he difference-of-gaussian keypoin deecor used in he SIFT operaor [48]. The above menioned resuls concerning he scale selecion properies of he Laplacian operaor norm L do also exend o he Harris-Laplace operaor [49]forwhich he spaial selecion is performed based on spaial exrema of he Harris measure H, whereas he scale selecion properies are solely deermined by he scale selecion properies of he Laplacian norm L. Incorporaing he scale selecion properies of he deerminan of he Hessian de H norm L,heresuls do also exend o he Harris-deHessian, demu-laplace and demu-dehessian operaors proposed in [39] aswellas oher possible ypes of hybrid approaches. For scale esimaes ha are compued algorihmically from real-world images in an acual implemenaion, he robusness of image feaures ha are obained by scale selecion from local exrema over scale or weighed scale selecion over scale may, however, differ subsanially. Experimenal resuls repored in Sec. 5.3 and [39] show ha weighed scale selecion leads o ineres poins ha have significanly beer repeaabiliy properies under perspecive image deformaions compared o ineres poins compued wih scale selecion from local exrema over scale. Theoreically, we have also seen ha in several cases, weighed scale selecion makes i easier o derive closedform expressions for he scale esimae han for scale selecion based on local exrema over scale. In hese respecs, scale selecion by weighed averaging over scale can have boh pracical and heoreical advanages. When making use of a complemenary pos-smoohing operaion o suppress spurious variaions in he non-linear feaure responses from he ineres poin deecors o simplify he ask of scale linking, he influence of his possmoohing operaion on he scale esimaes may, however, be differen for differen ineres poin deecors. If we assume ha scale calibraion can be performed based on he scale selecion properies for Gaussian blobs, we have derived a se of relaive calibraion or compensaion facors for each one of he five main ypes of ineres poin deecors sudied in his paper. To conclude, he analysis presened in his paper provides a heoreical basis for a defining a richer reperoire of mechanisms for compuing scale-invarian image feaures and image descripors for a wide range of possible applicaions in compuer vision. In very recen work [4], hese generalized scale-space ineres poins have been inegraed wih local scale-invarian image descripors and been demonsraed o lead o highly compeiive resuls for image-based maching and recogniion. As oulined in Appendix A., hese ineres poin deecors and he analysis of hese can be exended o higherdimensional image daa in a raher sraighforward manner. Acknowledgemens I would like o hank he anonymous reviewers for valuable commens and quesions ha improved he presenaion and Oskar Linde for valuable commens on an early version of he manuscrip. Open Access This aricle is disribued under he erms of he Creaive Commons Aribuion License which permis any use, disribuion, and reproducion in any medium, provided he original auhors and he source are credied. Appendix A.1 On he Algorihmic Advanages of Pos-smoohing for Scale Linking In [39] a generalized noion of a scale-space primal skech for differenial descripors was inroduced, where poinwise image feaures a differen scales are linked ino feaure rajecories over scale and bifurcaion evens beween hese feaure rajecories are explicily regisered. Specifically, i was experimenally shown ha: a significance measure obained by inegraing scalenormalized feaure responses along each feaure rajecory allows for a beer ranking of ineres poins han ranking on he magniude of he responses of scale-space exrema, and scale selecion by weighed averaging over scale allows for ineres poin deecion wih beer repeaabiliy properies under affine and perspecive image deformaions compared o scale selecion from local exrema over scale of scale-normalized derivaives.

28 04 J Mah Imaging Vis : Fig. 6 Illusraion of he effec of pos-smoohing: The lef figure shows exrema of he differenial expression D 1,norm L compued from an image deail a scale 4 wihou pos-smoohing. The righ figure shows exrema of he same differenial expression wih complemenary pos-smoohing using c 3/8. As can be seen from his figure, he use of pos-smoohing can reduce he number of muliple responses o similar srucures in he image domain. Observe ha his operaion is concepually differen from changing he scale a which derivaives are compued, since he pos-smoohing is performed afer he compuaion of he non-linear differenial expression In he original work on a scale-space primal skech for inensiy daa as obained from zero-order scale-space operaions wihou derivaive compuaions [7, 31] a scale linking algorihm was proposed based on i he deecion of greylevel blobs a any level of scale and ii adapive scale sampling refinemens. By heoreical analysis of singulariies in scale-space, he generic ypes of blob evens in scale-space were classified [6]. Specifically, he heoreical analysis showed ha alhough he local drif velociy in scale-space may momenarily end o infiniy near bifurcaion evens, i can be regarded as unlikely ha an exremum poin moves ouside he suppor region of he grey-level blob. For his reason, he maching of image srucures over scales used in he scale linking algorihm was based on deecing overlaps beween he suppor regions of he grey-level blobs a adjacen scales, and local refinemens were performed if he relaions beween parially overlapping grey-level blobs could no be decomposed ino he generic ypes of blob evens. Experimenally, i was shown ha his approach allowed for exracion of inuiively reasonable image srucures based on very few assumpions. Due o he adapive scale refinemens and he explici compuaion of grey-level blobs, his ype algorihm does, however, no appear suiable for realime implemenaion on a regular processor. In [39], a simplified and more efficien ype of scale linking algorihm was proposed, where scale refinemens and explici deecion of grey-level blobs are avoided. In order o sill be able o make use of he highly useful propery ha a local exremum can be expeced o be unlikely o drif ouside he suppor region of he grey-level blob, a local descen/ascen search is insead iniiaed a he posiion corresponding o he local exrema a he adjacen scale. If he descen/ascen search is iniiaed wihin he suppor region of a grey-level blob, his procedure should herefore proceed owards he exremum. This approach herefore avoids he complexiy problems ha oherwise would occur if maching a large number of image feaures beween adjacen scale levels. When compuing non-linear expressions from image daa a, in paricular fine or moderae scales, however, i urns ou ha one may ge several local exrema of low relaive ampliude in relaion o neighbouring local exrema, which do no correspond o percepually relevan image srucures. Since such local perurbaions canno be expeced o be sable under naural imaging condiions, hey canno be expeced o be percepually or algorihmically useful. By performing a small amoun of addiional Gaussian smoohing o he compued non-linear differenial expression, a large number of such spurious responses can be suppressed see Fig. 6. Thereby, he ask of scale linking will be simplified for he scale linking algorihm. Due o he suppression of a large number of irrelevan feaures, he scale linking

29 J Mah Imaging Vis : algorihm will also run significanly faser. Experimenally, he repeaabiliy properies of he resuling ineres poins do also become beer. From scale invariance argumens i is naural o le he relaion beween he pos-smoohing scale pos and he local scale level for compuing derivaives be given by pos c 171 for some fixed value of he relaive pos-smooh scale parameer c. In our firs work on generalized scale-space ineres poins [39], we used c 1/ when linking image srucures over scale ino feaure rajecories. Specifically, he experimens repored in Sec. 5.3 were performed using c 1/. In connecion wih our more recen work on inegraing he generalized scale-space ineres poins wih local image descripors [4], we found ha c 3/8 is a beer choice when maching image descripors under affine and perspecive image deformaions. Increasing c above c 1/ decreases he repeaabiliy properies for image deformaions ouside he similariy group, whereas decreasing c below c 1/4 leads o a lower suppression of irrelevan feaures which affecs he repeaabiliy properies of he resuling ineres poins. Beyond his, we have no ried o opimize he performance, bearing in mind ha one could also consider oher ypes of algorihms for scale linking, which could hen lead o differen rade-offs in erms of he complexiy of he local maching sep vs. efficiency or accuracy consideraions. A. 3-D Generalizaions of he Scale-Space Ineres Poin Deecors For image daa f : R 3 R ha are defined over a hreedimensional image domain indexed by he image coordinaes x,y,z, here are naural ways o exend he ineres poin deecors considered in his work based on he scalespace represenaion L: R 3 R + R of f generaed by convoluion wih he hree-dimensional Gaussian kernel gx,y,z; using helaplacian operaor 1 x+y+z e 17 π 3/ γ norm L γ race HL γ L xx + L yy + L zz 173 hedeerminan of he Hessian de H γ norm L 3γ L xx L yy L zz + L xy L xz L yz L xx L yz L yyl xz L zzl xy 174 herescaled Gaussian curvaure G γ norm L 4γ L z L xx L z L x L xz + L x L zz L z L yy L z L y L yz + L y L zz L z L x L yz + L xy L z L xz L y + L x L y L zz /L z 175 heunsigned Hessian feaure srengh measure I D 1,γ norm L 3γ de HL k race 3 HL if de HL k race 3 HL>0 0 oherwise heunsigned Hessian feaure srengh measure II 176 D,γ norm L γ min λ 1, λ, λ where λ 1, λ and λ 3 denoe he eigenvalues of he hreedimensional Hessian marix HL, whereas G γ norm L/ L x + L y + L z denoes he Gaussian curvaure 7 of a level surface. Relaed Work In relaion o hese 3-D exensions of he - D ineres poin deecors considered in his work, a hreedimensional exension of he Harris operaor H 3D de μ k race μ 178 defined from he hree-dimensional second-momen marix μx;,s Lx u; u R 3 Lx u; T gu; sdu 179 has been previously demonsraed o be effecive for deecing sparse spaio-emporal ineres poins in video daa Lapev and Lindeberg [3]. The hree-dimensional Hessian feaure srengh measure D 1,γ norm L can be seen as a differenial analogue o he spaio-emporal Harris operaor, defined from second-order derivaives of image inensiies only, which should allow for he compuaion of more dense ses of ineres poins, in analogy wih he denser ses 7 The moivaion for muliplying he Gaussian curvaure by a power of he gradien magniude in 175 is ha he resuling operaor should assume high values when he gradien magniude and he Gaussian curvaure are simulaneously high. More generally, also oher powers of he gradien magniude could be considered 04. The curren power of four is chosen because i leads o he simples calculaions, in analogy wih he muliplicaion by he gradien magniude raised o he power of hree for he -D rescaled level curve curvaure operaor 7.

30 06 J Mah Imaging Vis : of ineres poins ha are obained from he -D Hessian feaure srengh measure D 1,γ norm L compared o he ineres poins deeced by he -D Harris operaor. For 3-D image daa, he parameer k should be in he inerval ]0, 1 7 [. Also, he spaio-emporal deerminan of he Hessian has been used for deecing spaio-emporal ineres poins Willems e al. [61] and been demonsraed o allow for denser ses of spaio-emporal ineres poins han he spaioemporal Harris operaor. The new 3-D ineres poin deecors D 1,γ norm L, D,γ norm L and G γ norm L provide a way o exend his reperoire of 3-D ineres poin deecors. When applying hese operaors o spaio-emporal image daa, hey should be combined wih a spaio-emporal scale-space represenaion Lindeberg [33, 40]; Lindeberg and Fagersröm [45] ha allows for differen scale parameers over he space vs. ime. Moreover, he specific coupling beween space and ime should be explicily considered when expressing invariance properies over space-ime Lindeberg e al. [40, 44]. In he following, we shall analyze he scale-selecion properies of hese 3-D ineres poin deecors when applied o volumeric spaial image daa using he same amoun of scale-space smoohing over all spaial dimensions. A..1 Scale Selecion Properies for a 3-D Gaussian Blob Consider a single 3-D Gaussian blob 1 x +y +z fx,y,z gx,y,z; 0 e π 0 3/ Due o he semi-group propery of he Gaussian kernel g,, ; 1 g,, ; g,, ; he scale-space represenaion of f obained by Gaussian smoohing is given by Lx,y,z; gx,y,z; x +y +z e π 0 + 3/ A.. The Pure Second-Order Ineres Poin Deecors By differeniaion i follows he scale normalized feaure srengh measure a he cener x,y,z 0, 0, 0 of he Gaussian blob will for he Laplacian 173, he deerminan of he Hessian 174 and he Hessian feaure srengh measures I 176 and II 177 be given by γ norm L 3 γ 0, 0; 183 π 3/ + 0 5/ de H γ norm L0, 0; π 9/ / 3γ 1 7k 3γ D 1,γ norm L0, 0; π 9/ / γ D,γ norm L0, 0; 186 π 3/ + 0 5/ Differeniaing hese expressions wih respec o he scale parameer and seing he derivaive o zero gives ha he exremum over scale will for all hese ineres poin deecors be assumed a he same scale ˆ γ 5 γ If we would like hese ineres poin deecors o reurn a scale esimae corresponding o he scale parameer of a Gaussian blob, we should herefore choose γ 3D Using γ 1 would oherwise lead o a lower scale esimae ˆ 3 D,γ For scale selecion based on weighed averaging over scale ofhe scale-normalizedlaplacianresponse 183, he corresponding scale esimaes are for γ ]0, 5 [ given by ˆτ L 3 γ 0 π 3/ / log d 3 γ d π 3/ + 0 5/ log 0 + ψγ ψ 5 γ { if γ 5 } log where ψu denoes he digamma funcion, which is he logarihmic derivaive of he Gamma funcion ψu Γ u/γ u. Similarly, from he explici expression for he deerminan of he Hessian a he cener of he Gaussian blob 184, i follows ha he weighed scale esimae will for γ ]0, 5 [ be given by ˆτ de HL 3γ 0 16 π 9/ / log d 3γ 16 d π 9/ / log 0 + ψ3γ ψ 15 3γ { if γ 5 } log Due o he similariy beween he explici expressions for he Hessian feaure srengh measure I 185 and he deerminan of he Hessian response 184 aswellashesimilariy beween he Hessian feaure srengh measure II 186

31 J Mah Imaging Vis : and he Laplacian response 185 a he cener of a Gaussian blob, he scale esimaes for D 1,γ norm L and D,γ norm L will be analogous: 15 ˆτ D1 L log 0 + ψ3γ ψ 3γ { if γ 5 } log ˆτ D L log 0 + ψγ ψ γ { if γ 5 } log When expressed in erms of he regular scale parameer ˆ e ˆτ 194 he weighed scale selecion mehod does hence for a roaionally symmeric Gaussian blob lead o similar scale esimaes as are obained from local exrema over scale of γ - normalized derivaives 9 when γ 5/4. For oher values of γ, he scale esimaes are biased ˆ C by a scale facor equal o C 1 e ψγ ψ5 γ 196 for he Laplacian norm L and he Hessian feaure srengh measure D,norm L operaors, and a bias facor C e ψ3γ ψ15 3γ 197 for he deerminan of he Hessian de H norm L and he Hessian feaure srengh measure D 1,norm L operaors. By following he mehodology oulined in Secs.3., 3.3, 4. and 4.3, a corresponding more deailed analysis can be performed concerning he influence of he pos-smoohing operaion and affine image deformaions for hese 3-D ineres poin deecors. A..3 The Rescaled Gaussian Curvaure Operaor When compuing he rescaled Gaussian curvaure G γ norm L for he scale-space represenaion of a hreedimensional spherically symmeric Gaussian blob wih scale parameer 0, we obain G γ norm L 3γ x + y + z e x+y+z π which wih spherical coordinaes x + y + z R becomes G γ norm L R 3γ e + R 0 64 π This expression assumes is maximum value over R when 0 + R 00 for which he rescaled surface curvaure assumes he value G 3γ γ norm L 18 eπ Seing he derivaive of his expression wih respec o he scale parameer o zero gives ˆ γ 3 γ 0 0 for which he γ -normalized magniude response is 3 γ G 9 γ norm L eπ γ γ γ A corresponding analysis can be carried ou for he Gaussian curvaure modulaed by oher powers of he gradien magniude G a γ norm L G γ norm L L x + L y + 04 L z a where a<. Then, he corresponding scale esimae becomes ˆ 3 aγ 9 4a 3 aγ 0 05 For general values of a, he corresponding inermediae resuls are, however, more complex. A.3 Inerpreing he Parameer γ in Terms of he Dimensionaliy of he Image Feaures The value γ 5/4 obained from he analysis of a 3-D Gaussian blob in Appendix A.. Eq. 188 can be compared o he values obained by requiring he purely secondorder differenial eniies o respond o a -D Gaussian blob a 0 Eq. 9 γ D 1 06 or he value of he γ parameer obained by requiring he second-order derivaive operaor o respond o a 1-D Gaussian blob alernaively a 1-D Gaussian ridge embedded in a

32 08 J Mah Imaging Vis : higher-dimensional space a 0 [34] γ 1D More generally, for a D-dimensional Gaussian inensiy profile fx 1,...,x D gx 1,...,x D ; 0 Di1 x 1 i e 0 π 0 D/ 08 embedded in N D dimensions, he scale-space represenaion is given by Lx 1,...,x N ; gx 1,...,x D ; 0 + Di1 x 1 i e π 0 + D/ wih he corresponding γ -normalized Laplacian response γ norm L x 1,...,x N ; γ D i1 x i gx 1,...,x D ; which assumes he following value a he origin γ norm L D γ 0,...,0; π D/ 11 Differeniaing his expression wih respec o gives and seing he derivaive o zero gives γ ˆ D + 1 γ 0 1 Requiring his scale esimae o be equal o 0 implies ha he γ -value for a pure second-order operaor should herefore be relaed o he dimensionaliy D of he image feaures hey should respond o according o γ 1 + D 13 4 Noe ha he -D case is special in he sense ha only for his dimensionaliy D does scale selecion based on he mos scale-invarian choice γ 1 lead o scale esimaes ha are equal o he diffuseness parameer 0 of a Gaussian inensiy profile. I can also be noed ha only for wo-dimensional blobs will he corresponding γ -normalized magniude values of scale-space exrema be independen of he size of he blob, whereas in oher dimensionaliies he corresponding magniude values need o be normalized by a scale-dependen correcion facor ˆ 1 γ o lead o scale-invarian magniude values ha are independen of he diffuseness 0 of he D- dimensional Gaussian inensiy profile. References 1. Baumberg, A.: Reliable feaure maching across widely separaed views. In: Proc. CVPR, Hilon Head, SC, pp Bay, H., Ess, A., Tuyelaars, T., van Gool: Speeded up robus feaures SURF. Compu. Vis. Image Unders. 1103, Bevensee, R.: Maximum Enropy Soluions o Scienific Problems. Prenice Hall, New York Brezner, L., Lindeberg, T.: Feaure racking wih auomaic selecion of spaial scales. Compu. Vis. Image Unders.713, Choma, O., de Verdiere, V., Hall, D., Crowley, J.: Local scale selecion for Gaussian based descripion echniques. In: Proc. ECCV 00, Dublin, Ireland. Lecure Noes in Compuer Science, vol. 184, pp Springer, Berlin Consable, R.T., Henkelman, R.M.: Why MEM does no work in MR image reconsrucion. Magn. Reson. Med. 141, Danielsson, P.E., Lin, Q., Ye, Q.Z.: Efficien deecion of seconddegree variaions in D and 3D images. J. Vis. Commun. Image Represen. 13, Donoho, D.L., Johnsone, I.M., Hoch, J., Sern, A.S.: Maximum enropy and he nearly black objec. J. R. Sa. Soc., Ser. B Mehodological 54, Elder, J., Zucker, S.: Local scale conrol for edge deecion and blur esimaion. IEEE Trans. Paern Anal. Mach. Inell. 07, Florack, L.M.J.: Image Srucure. Series in Mahemaical Imaging and Vision. Springer, Berlin Frangi, A.F., Niessen, W.J., Hoogeveen, R.M., vanwalsum, T., Viergever, M.A.: Model-based quaniaion of 3D magneic resonance angiographic images. IEEE Trans. Med. Imaging 1810, Gårding, J., Lindeberg, T.: Direc compuaion of shape cues using scale-adaped spaial derivaive operaors. In. J. Compu. Vis. 17, Griffin, L.D.: The second order local-image-srucure solid. IEEE Trans. Paern Anal. Mach. Inell. 98, er Haar Romeny, B.: Fron-End Vision and Muli-Scale Image Analysis. Springer, Berlin Hall, D., de Verdiere, V., Crowley, J.: Objec recogniion using coloured recepive fields. In: Proc. ECCV 00, Dublin, Ireland. Lecure Noes in Compuer Science, vol. 184, pp Springer, Berlin Harris, C., Sephens, M.: A combined corner and edge deecor. In: Alvey Vision Conference, pp Iijima, T.: Observaion heory of wo-dimensional visual paerns. Tech. rep., Papers of Technical Group on Auomaa and Auomaic Conrol, IECE, Japan Kadir, T., Brady, M.: Saliency, scale and image descripion. In. J. Compu. Vis. 45, Kang, Y., Morooka, K., Nagahashi, H.: Scale invarian exure analysis using muli-scale local auocorrelaion feaures. In: Proc. Scale Space and PDE Mehods in Compuer Vision Scale- Space 05. Lecure Noes in Compuer Science, vol. 3459, pp Springer, Berlin Koenderink, J.J.: The srucure of images. Biol. Cybern. 50, Koenderink, J.J., van Doorn, A.J.: Generic neighborhood operaors. IEEE Trans. Paern Anal. Mach. Inell. 146, Krissian, K., Malandain, G., Ayache, N., Vaillan, R., Trousse, Y.: Model-based deecion of ubular srucures in 3D images. Compu. Vis. Image Unders. 80,

33 J Mah Imaging Vis : Lapev, I., Lindeberg, T.: Space-ime ineres poins. In: Proc. 9h In. Conf. on Compuer Vision, Nice, France, pp Lazebnik, S., Schmid, C., Ponce, J.: A sparse exure represenaion using local affine regions. IEEE Trans. Paern Anal. Mach. Inell. 78, Li, Y., Tax, D.M.J., Loog, M.: Supervised scale-invarian segmenaion and deecion. In: Proc. Scale Space and Variaional Mehods in Compuer Vision Scale-Space 11, Ein Gedi, Israel. Lecure Noes in Compuer Science, vol. 6667, pp Springer, Berlin Lindeberg, T.: Scale-space behaviour of local exrema and blobs. J. Mah. Imaging Vis. 11, Lindeberg, T.: Deecing salien blob-like image srucures and heir scales wih a scale-space primal skech: a mehod for focusof-aenion. In. J. Compu. Vis. 113, Lindeberg, T.: Effecive scale: a naural uni for measuring scalespace lifeime. IEEE Trans. Paern Anal. Mach. Inell. 1510, Lindeberg, T.: On scale selecion for differenial operaors. In: Høgdra, K.H.K.A., Braahen, B. eds. Proc. 8h Scandinavian Conf. on Image Analysis, pp Norwegian Sociey for Image Processing and Paern Recogniion, Tromsø Lindeberg, T.: Scale-space heory: a basic ool for analysing srucuresadifferenscales. J. Appl. Sa.1, Also available from. hp:// Lin94-SI-absrac.hml 31. Lindeberg, T.: Scale-Space Theory in Compuer Vision. The Kluwer Inernaional Series in Engineering and Compuer Science. Springer, Berlin Lindeberg, T.: Direc esimaion of affine deformaions of brighness paerns using visual fron-end operaors wih auomaic scale selecion. In: Proc. 5h In. Conf. on Compuer Vision, Cambridge, MA, pp Lindeberg, T.: Linear spaio-emporal scale-space. In: er Haar Romeny, B.M., Florack, L.M.J., Koenderink, J.J., Viergever, M.A. eds. Scale-Space Theory in Compuer Vision: Proc. Firs In. Conf. Scale-Space 97, Urech, The Neherlands. Lecure Noes in Compuer Science, vol. 15, pp Springer, Berlin Exended version available as echnical repor ISRN KTH NA/P 01/ SE from KTH 34. Lindeberg, T.: Edge deecion and ridge deecion wih auomaic scale selecion. In. J. Compu. Vis. 30, Lindeberg, T.: Feaure deecion wih auomaic scale selecion. In. J. Compu. Vis. 30, Lindeberg, T.: A scale selecion principle for esimaing image deformaions. Image Vis. Compu. 1614, Lindeberg, T.: Principles for auomaic scale selecion. In: Handbook on Compuer Vision and Applicaions, pp Academic Press, Boson Also available from hp:// 38. Lindeberg, T.: Scale-space. In: Wah, B. ed. Encyclopedia of Compuer Science and Engineering, pp Wiley, Hoboken 008. doi:10.100/ ecse609. Also available from hp:// EncCompSci.hml 39. Lindeberg, T.: Generalized scale-space ineres poins: scale-space primal skech for differenial descripors. In. J. Compu. Vis. 010 original version submied in June Lindeberg, T.: Generalized Gaussian scale-space axiomaics comprising linear scale-space, affine scale-space and spaio-emporal scale-space. J. Mah. Imaging Vis. 401, Lindeberg, T.: Scale invarian feaure ransform. Scholarpedia 75, Lindeberg, T.: Disinciveness and maching properies of generalized scale-space ineres poins 01. Unpublished manuscrip 43. Lindeberg, T.: Scale Selecion. Encyclopedia of Compuer Vision. Springer, Berlin 01, in press 44. Lindeberg, T., Akbarzadeh, A., Lapev, I.: Galilean-correced spaio-emporal ineres operaors. In: Inernaional Conference on Paern Recogniion, Cambridge, pp. I: Lindeberg, T., Fagersröm, D.: Scale-space wih causal ime direcion. In: Proc. ECCV 96, Cambridge, UK, vol. 1064, pp Springer, Berlin Lindeberg, T., Gårding, J.: Shape-adaped smoohing in esimaion of 3-D deph cues from affine disorions of local -D srucure. Image Vis. Compu. 15, Loog, M., Li, Y., Tax, D.: Maximum membership scale selecion. In: Muliple Classifier Sysems. Lecure Noes in Compuer Science, vol. 5519, pp Springer, Berlin Lowe, D.: Disincive image feaures from scale-invarian keypoins. In. J. Compu. Vis. 60, Mikolajczyk, K., Schmid, C.: Scale and affine invarian ineres poin deecors. In. J. Compu. Vis. 601, Mikolajczyk, K., Tuyelaars, T., Schmid, C., Zisserman, A., Maas, J., Schaffalizky, F., Kadir, T., van Gool, L.: A comparison of affine region deecors. In. J. Compu. Vis. 651, Mrázek, P., Navara, M.: Selecion of opimal sopping ime for nonlinear diffusion filering. In. J. Compu. Vis. 5 3, Negre, A., Braillon, C., Crowley, J.L., Laugier, C.: Real-ime imeo-collision from variaion of inrinsic scale. Exp. Robo. 39, Pedersen, K.S., Nielsen, M.: The Hausdorff dimension and scalespace normalisaion of naural images. J. Mah. Imaging Vis. 11, Sao, Y., Nakajima, S., Shiraga, N., Asumi, H., Yoshida, S., Koller, T., Gerig, G., Kikinis, R.: 3D muli-scale line filer for segmenaion and visualizaion of curvilinear srucures in medical images. Med. Image Anal., Shi, J., Tomasi, C.: Good feaures o rack. In: Proc. CVPR, pp Sporring, J., Colios, C.J., Trahanias, P.E.: Generalized scaleselecion. In: Proc. Inernaional Conference on Image Processing ICIP 00, Vancouver, Canada, pp Tuyelaars, T., van Gool, L.: Maching widely separaed views based on affine invarian regions. In. J. Compu. Vis. 591, Tuyelaars, T., Mikolajczyk, K.: A survey on local invarian feaures. In: Foundaions and Trends in Compuer Graphics and Vision, vol. 33. Now Publishers, Boson Uffink, J.: Can he maximum enropy principle be explained as a consisency requiremen? Sud. His. Philos. Mod. Phys. 63, Weicker, J., Ishikawa, S., Imiya, A.: Linear scale-space has firs been proposed in Japan. J. Mah. Imaging Vis. 103, Willems, G., Tuyelaars, T., van Gool, L.: An efficien dense and scale-invarian spaio-emporal ineres poin deecor. In: Proc. ECCV 08, Marseille, France. Lecure Noes in Compuer Science, vol. 5303, pp Springer, Berlin Wikin, A.P.: Scale-space filering. In: Proc. 8h In. Join Conf. Ar. Inell, Karlsruhe, Germany, pp

34 10 J Mah Imaging Vis : Tony Lindeberg is a professor in Compuer Science a KTH Royal Insiue of Technology in Sockholm, Sweden. He was born in Sockholm in 1964, received his M.S.c. degree in 1987, his Ph.D. degree in 1991, became docen in 1996, and was appoined professor in 000. He was a Research Fellow a he Royal Swedish Academy of Sciences beween 000 and 010. His research ineress in compuer vision relae o scale-space represenaion, image feaures, objec recogniion, spaio-emporal recogniion, focus-of-aenion and shape. He has developed heories and mehodologies for coninuous and discree scale-space represenaion, deecion of salien image srucures, auomaic scale selecion, scaleinvarian image feaures, affine invarian feaures, affine and Galilean normalizaion, emporal and spaio-emporal scale-space conceps as well as spaial and spaio-emporal image descripors for image-based recogniion. He does also work on compuaional modelling of biological vision and has previously worked on applicaions in medical image analysis and gesure recogniion. He is auhor of he book Scale-Space Theory in Compuer Vision.