Image Spaces. What might an image space be

Transcription

1 Image Spaces What might a image space be Map each image to a poit i a space Defie a distace betwee two poits i that space Mabe also a shortest path (morph) We have alread see a simple versio of this, i which each -piel image is mapped to a poit i R^, ad Euclidea distace is used Also, we ve cosidered liear subspaces (Eigefaces, Fisherfaces) Each poit is mapped to earest locatio o the subspace Shortest path i R^ is just liear iterpolatio of the images, which does t seem like the right morph What is shortest path with Eigefaces?

2 Riemaia Maifold More geeral otio of a image space Ca have differet topolog tha Euclidea space Ca represet o-liear subsets of images (more o this is a few weeks) Ca provide more geeral sese of distaces Maifold A smooth maifold of dimesio m is a locall compact Hausdorff space M together with the followig collectio of data (heceforth called atlas or smooth structure) cosistig of: A ope cover {U i } of M Cotiuous ijective maps F i : U i ->R m (called charts or local coordiates) such that: if U U 0 the trasitio map j i Ψ Ψ i is smooth j : Ψ i m ( U U ) R Ψ ( U U ) i j j i j R Lectures o the Geometr of Maifolds b Nicolaescu m

3 Maifold - iformal A maifold is: A set (collectio of images) with the usual topological properties (eg, ot discrete) A mappig from the eighborhood of each poit to Euclidea space of fied dimesio So that these mappigs fit together smoothl Eample: the surface of a sphere Eample: a surface that is topologicall a sphere The defiitio implies: At a poit o the maifold, we ca costruct a taget space Riema Maifold A Riema Maifold is a pair (M,g) cosistig of a smooth maifold M ad a metric g o the taget budle, ie a smooth smmetric positive defiite tesor field o M g is called a Riema metric o M The taget budle is (iformall) the collectio of taget spaces of M 3

4 Riema Maifold - ituitio We defie a local distace o the maifold At a poit, for a directio we move i, there is a local distace defied This distace is locall liear If we defie the distace for m orthogoal directios, the distace i a other directio is a liear combiatio of these Eample: a surface i Euclidea space that is topologicall a sphere with the usual Euclidea distace Geodesic Aalogous to lies, geodesics are shortest paths betwee poits Shortest paths locall, but ot globall For a two poits ver close together o the geodesic, it is the shortest path Eample: o a sphere, a two poits are coected b two geodesic paths, alog two directios of the great circle coectig them 4

5 Wh maifolds? Seems like correct mathematical otio of a image space with distaces Geeralit greater tha Euclidea space Image space ma be topologicall like a sphere, ot like R Offers much more fleible distaces Kedall s shape space Two sets of D poits Mostl we assume there eists a correct oeto-oe correspodece Ad this correspodece is give This is ver atural i morphometrics, where poits are measured ad labeled I visio we must solve for correspodece Net class we ll look at papers that do this 5

6 Shape Space What is shape? Qualities of poits that do t deped o traslatio, rotatio or scale So describe poits idepedet of similarit trasformatio Remove traslatio Simplest wa, traslate so poit is at origi, the remove poit oe More elegat, traslate ceter of mass to origi, remove a poit Scale so that sum Xi ^ = Resultig set of poits is called pre-shape Pre because we have t removed rotatio et Notatio:, U ad X deote sets of ormalized poits Poits called Xi ad Ui, with coordiates (i,i), (ui, vi) Pre-shape If we started with poits, we ow have - so that: sum i^ + i^ = So we ca thik of these coordiates as lig o a uit hpersphere i (-)- dimesioal space 6

7 Shape If we cosider all possible rotatios of a set of ormalized poits, these trace out a closed, D curve i pre-shape space Distaces betwee shapes ca be thought of as distaces betwee these curves Notice that to compute distace, without loss of geeralit we ca assume that oe set of poits (U) does ot rotate, sice rotatig both poit sets b the same amout does t chage distaces Procrustes Distaces Full Procrustes Distace D F mi(s, ) U sxr() That is, we fid a scalig ad rotatio of X that miimizes the euclidea distace to U (R() meas rotate b ) Partial Procrustes Distace D P mi() U XR() That is, rotate X to miimize the euclidea distace to U Procrustes Distace ρ Rotate X to miimize the geodesic distace o the sphere from X to U 7

8 8 Liear Pose Solvig We ca liearl fid optimal similarit trasformatio that matches X to U (ie, miimize sum AXi-Ui ^, where A is a similarit trasformatio This is asmmetric betwee X ad U I same wa we ca liearl compute Full Procrustes Distace This is smmetric Leads immediatel to other procrustes distaces Liear Pose: D rotatio, traslatio ad scale si, cos with cos si si cos s b s a t a b t b a t t s v v v u u u = = = Notice a ad b ca take o a values Equatios liear i a, b, traslatio Solve eactl with poits, or overcostraied sstem with more s a b a s = + = cos

9 Similarit Matchig Give poit sets X ad U, compare b fidig similarit trasformatio A that miimizes AX-U X = poits X, X U = poits U U Fid A to miimize sum AXi Ui ^ This is just a straightforward, liear problem Takig derivatives with respect to four ukows of A gives four liear equatios i four ukows Issues with this approach It is asmmetric Ok whe comparig a model to a image Not so sesible for comparig two shapes 9

10 0 Note that we ow also kow how to calculate the Full Procrustes Distace This is just a least-squares solutio to the overcostraied problem: = a b b a s v v v u u u cos si si cos It is ot obvious that Full Procrustes is smmetric Give two poits o the hpersphere, we ca draw the plae cotaiig these poits ad the origi D F ρ D P ρ Procrustes Distaces is ρ D P = si ( ρ/) D F = si ρ These are all mootoic i ρ So the same choice of rotatio miimizes all three D F is eas to compute, others are eas to compute from D F

11 Wh Procrustes Distace? Procrustes distace is most atural Our ituitio is that give two objects, we ca produce a sequece of itermediate objects o a straight lie betwee them, so the distace betwee the two objects is the sum of the distaces betwee itermediate objects This requires a geodesic Taget Space Ca compute a hperplae taget to the hpersphere at a poit i preshape space Project all poits oto that plae All distaces Euclidea Average shape eas to fid This is reasoable whe all shapes similar I this case, all distaces are similar too Note that whe ρ is small, ρ, si(ρ /), si(ρ) are all similar

12 Trouve ad Youes Image space is the space of all images To get a maifold, we must describe a taget space, that gives a cost to small image chages Normal Euclidea distace i this space allows itesit of a poit to chage, with a cost = square of the chage T&Y also allow image to deform, with a cost based o the smoothess of deformatio Ituitio A B With Euclidea distace, d(a,b) = d(a,c) But B seems much more similar to A, because the are related b a small deformatio C

13 Local Cost Let v be a deformatio This is a diffeomorphism (smooth, cotiuous oe-to-oe trasformatio) We might be iterested i o-smooth deformatios, but these are ot as ice mathematicall I(v()) is the deformed image We ca t epect two images to be idetical up to a deformatio So, add cost J()-I(v()) Use Euclidea orm here Total cost also requires a orm o v, a vector field (see paper) v ( ) I( v( ) ) v L g mi λ J + Taget Space This assigs a cost to a combiatio of deformatio ad itesit chage However, we eed a cost for chages i the image A image chage is cosistet with a deformatio + some itesit chage Ie, there are a ifiite umber of was to create a image chage We defie the cost of the image chage to be the ifium of all of these This defies a image maifold 3

14 Geodesics We have defied a local cost o ifiitesimal image chages For ti chages, we ca miimize over deformatios fairl easil, because everthig is liear To fid the distace betwee two images, we must compute a geodesic path betwee them This also gives us a morph betwee them Computig geodesics Gradiet descet Start with some path betwee images, discretizig time Path is represeted b discrete represetatio over deformatio at each time Durrlema describes use of kerels here Ad b chages i itesit at each time Compute derivatives ad miimize path Lots of variables, but this ca work i practice Geodesic shootig Geodesic path is etirel determied b iitial chage Pick some chage, calculate path, measure distace to target image, ad correct 4

15 Eample of Geodesic 5