Problem of conformal invarincy in vision.

Transcription

1 Institute for Information Transmission Problems, Moscow, Russia April 24, 2015

2 1.INTRODUCTION Aim of vision Visual system receives information from light (electro-magnetic radiation). The aim of vision is to obtain information about (Euclidean ) geometry of the external world from light which comes to retina and transform it into a finite set of invariants ( gestalts, words, emotions etc.) It must be objective, i.e. independent from position of observer ( that is invariant with respect to change of position of eyes, head, velocity etc.) Two questions arise : 1) Which information comes to eye (retina) and how does it change under movement of eye and head? 2) How do eyes and brain extract invariant information about the external geometry from the input subjective (dependent on position etc) information which comes to retina?

3 Eye

4 2. EYE AS AN OPTICAL DEVICE Eye is a transparent ball B 3 together with a lens L with center F B 3 near the boundary sphere S 2 which focuses light rays to the retina R S 2. We get a map π : L(F ) R S 2, l l R from a surface Σ to retina R which depends on the position of the eye ball B(OF ). The energy of light at a point  R is I R (Â) := I (AY )dσ Y D where D = {(AX ) S 2 (A), X L} is the intersection of the cone over lens L with vertex A with the unit sphere S 2 (A) with center A and dσ is the standard measure of this sphere. The function I R : R R is called the energy function.

5 What is the input function on the retina : the energy function I, 1-form di or 1-distribution D = [di ] = kerdi? Basic global objects of early vision Static and dynamics. Basic global objects of early vision are contours = curves on the retina R S 2 which are level set of the intensity function with big gradient (w.r.t. which metric?) It is the image of edge (boundary of the object of external world.) For simplicity we consider only immovable objects. More elaborate answer is that the basic objects are piece-wise smooth surfaces in the 3-cylinder R S 2 where R is the time. Locally we may approximate R S 2 by R R 2 = R 3.

6 Basic infinitesimal objects of early vision First order infinitesimal approximation of a non parametrized curve (contour) is a tangent line. The space of such infinitesimal contours is the contact bundle PTS 2 = PT S 2 = {(x, y, p = dy dx )} with the contact structure ker(dy pdx). An infinitesimal contour of order k is a k-jet of a contour. The space of such objects can be identified with J k (R, R) = {x, y, p = y,, y (k) }. A k-th order infinitesimal part of an input function I F(S 2 ) is the k-jet j k z (I ) J k (S 2, R). For k = 1, J 1 (S 2, R) = R T S 2. A better candidate for the space of first order infinitesimal functions is T S 2.

7 3. EYE AS A ROTATING RIGID BODY. Fixation eye movements Eye is a rigid ball BO 3 which can rotate around the center O w.r.t. three mutually orthogonal axes i, j, k. The center F 0 BO 3 of the eye crystal (lens) is near the boundary sphere SO 2 = B3 O and the retina region R S O 2 is a big part of SO 2. For a fixed position of head, there is a privilege initial position B(OF 0 ) of the eye ball corresponding to the standard (frontal) direction (OF 0 ) of the gaze.

8 Donder s and Listing s laws Donder s law (1846)(No twist). If the head is fixed, the result of a movement from position B(OF 0 ) to a new position B(OF ) is uniquely defined by the gaze OF and do not depend on previous movement. Mathematically, it defines a section s : S 2 SO 3 of the frame bundle SO 3 S 2 = SO 3 /SO 2 such that a curve γ(t) in S 2 has lift sγ(t) to the group of rotations SO 3. Due to this law, a movement of the eye is determined by a curve on the eye sphere. Listing s law (1845) The movement from B(OF 0 ) to B(OF ) is obtained by rotation with respect to the axe OF 0 OF. The curve in SO 3 is the parallel lift of the initial frame along the arc F 0 F S 2.

9 Eyes movements. Tremor, drift, microsaccades and macrosaccades Eyes participate in different involuntary types of movements which is divided into two types : fixation eye movements when the gaze is fixed and macrosaccades, very rapid (up to 900 /sec in humans ) rotation of the eyes with big amplitude.

10 Fixation eye movements Fixation eye movements include: tremor, drifts and microsaccades. Tremor is an aperiodic, wave-like motion of the eyes of high frequency but very small amplitude. Drifts occur simultaneously with tremor and are slow motions of eyes, in which the image of the fixation point for each eye remains within the fovea! Drifts occurs between the fast, jerk-like, linear microsaccades.

11 Characteristics of fixation eye movements Amplitude Duration Frequency Speed Tremor sec Hz Max 20 min/s Drift 1-9 min s 95-97% of time 1-30 min/s Micsac 1-50 min s Hz /s Per 1 s tremor moves on diameters of the fovea cone drift moves on diameters microsaccads moves on diameters. Under tremor the axis of eye draws a cone for 0.1 s. In 2-3 sec after compensation of fixation eye movement, a human lost ability to see an immobile object. (Yarbus)

12 Microdaccades, drift and tremor Drift and microsaccades

13 Model of eye movements by R.Engbert, K. Mergenthaler, P. Sinn, A. Pikovsky Self-avoiding Random Walk Involuntary eye movement described as a self-avoiding random walk on the square lattice Z 2 with quadratic potential ( Random walk in a swamp on a paraboloid ). Physiological aim of such movement (when gaze fix a point A): the images (Ā)(t) of A on retina must be homogeneously distributed between all receptors of the fovea.

14 Why we need fixation eye movements 1.(Geometry) For a fix gaze OF, the retina gets information only about the 2-dimensional Lagrangian submanifold L(F ) = RP 2 of the 4-dimensional space of lines L(E 3 ). When eye moves in a neighborhood of a fixed point OF, it gets information from a neighborhood of L(F ) in 4-manifold L(E 3 ). 2. To see immobile objects (Yarbus) 3. To determine direction of moving external objects (Roords et al., 2013) 4(Neuroscience) For better identification of contours in V1 cortex.

15 Central projection of a plane to sphere Let Σ = Π = Π ρ n = {A, n A = ρ} is the plane with normal vector n = (cos ϕ, sin ϕ, 0) where ρ = dist(π, O) and coordinates (y, z). Then Π ρ n = {A = ρn + (sin ϕy, cos ϕy, z) = The central projection is (ρ cos ϕ + sin ϕy, ρ sin ϕ cos ϕy, z)} π F : A Ā = F f (A)(A F ) = F 2F (A F ) (A F ) 2 (A F ).

16 When the central projection of a plane is a conformal map? The induces metric g S 2 of the sphere S 2 w.r.t. the local coordinates y, z s.t. Ā(y, z) = π F A(y, z) is g S 2 = dā2 = f 2 da 2 2r sin ϕdydf where da 2 = dy 2 + dz 2 is the metric of the plane Π n ρ f = 2F (A F ) (A F ) 2 = 2r(sin ϕy+β) R 2, R 2 = A F 2 := (y sin ϕ) 2 + z 2 + (ρ sin ϕ) 2, β = ρ cos ϕ r and df = 2r R 4 [{ρ2 2r 2 +y 2 +z 2 +2y(r 2ρ cos ϕ}dy 2z(β+sin ϕy)dz] It is a conformal map iff the plane is frontal ( i.e. orthogonal to the frontal direction, i.e. ϕ = 0. A small rotation R = RO α of eye (which is equivalent to a rotation R 1 of the external space in opposite direction) produces (approximately) a conformal transformation of the eye sphere.

17 Problem of conformal invariant perception of contours (problem of stability) as the main problem of conformal geometry of curves Importance of conformal group in vision (Hoffman, 1989 ) The main problem of differential geometry of curves in a homogeneous manifold M = G/H is to construct the full system of G-invariants of a curve C M which determines it up to a transformation from G. For Euclidean plane E 2 = SE(2)/SO 2, a solution given by Frenet associates with a curve γ the natural equation K = K(s) where s is the natural parameter (arc-length) and K(s) = z(s) (the curvature = acceleration of the path z(s)). For conformal geometry S 2 = SO 1,3 /Conf (E 2 ) of sphere similar solution is known ( A. Fialkov, J. Haantjes, R. Sharp, F.Brustall and D. Calderbank.) The natural equation of a curve γ S 2 is K = K(s) where s is a conformal parameter along a curve γ (defined up to a fractional linear transformations) and K is the conformal curvature which depends on 5-jet of γ.

18 5 CONFORMAL GEOMETRY OF SPHERE Let (V = R 1,3, g) be the Minkowski vector space, V 0 the light cone of isotropic vectors and S 2 := PV 0 = {[p] := Rp, p V 0 } S 2 the celestial sphere. The metric g induces a conformal structure [g 0 ] in S 2 and the connected Lorentz group G = SO(V ) SO 1,3 acts transitively on S 2 as the group of conformal transformations (the Möbius group).

19 Gauss decomposition of conformal Lie algebra and conformal group The gradation V = V 1 + V 0 + V 1 = Rp + E 2 + Rq, g(p, p) = g(q, q) = g(rp + Rq, E 2 ) = 0, g(p, q) = 1 defines a gradation of the Lie algebra g = so(v ) = g 1 + g 0 + g 1 = p E + so(e) + q E. It defines the Gauss decomposition of the conformal group G = SO(V ) = G 1 G 0 G +. S 2 = G/B = G/G 0 G +, G + Sim(E 2 ) = R + SO 2 R 2.

20 Sphere S 2 as Riemannian sphere C { } In terms of the holomorphic coordinate z S 2 C { }, ( ) a b SO(V ) = SL 2 (C)/(±1) ±A = ± : z az + b c d cz + d. The Lie algebra sl 2 (C) = {(b + az + cz 2 ) z }. The gradation is sl 2 (C) = g 1 + g 0 + g 1 = {b z } + {a z } + {cz 2 z }. The (local) Gauss decomposition is G = SL 2 (C) = ( ) G ( G 0 ) G + ( ) 1 b a =, d c 1 Note that S 2 = G/B = SL 2 (C)/G 0 G + where B = G 0 G 1 is the Borel subgroup of upper triangular matrices.

21 6. MULTISCALE APPROXIMATION OF DIFFERENTIAL GEOMETRY AND MODELS OF VISUAL CELLS. Points in Differential Geometry (DG) From quantum point of view, the main object of DG is the algebra C (M) of function on a manifold. Point z M is a special linear functional (called Dirac delta function ) δ z0 : f δ z0 (f ) = f (z 0 ). Tangent vector at z 0 is a linear functional V : C (M) R which satisfies the Leibnitz rule V (fg) = f (z 0 )V (g) + g(z 0 )V (f ). Moreover, such functional can be consider as a partial derivative of the delta function due to the formula ( x δ z0 )(f ) = ( x f ) where (x, y) are local coordinates of a point z M 2. Approximating the delta functional by functionals associated with smooth functions (e.g. Gauss functions G σ ) we get a sigma -approximation of DG.

22 Gauss filter as sigma-approximation of a point We assume that M = R 2 = {z = (x, y)}. The delta functional δ z0 associated with a point z 0 = (x 0, y 0 ) is approximated by Gauss functionals ( Gauss filter ) T G σ : I G σ (z)i (z)dxdy where G = G σ z 0 (z) = 1 2πσ exp( z z 0 2 2σ 2 ) is the Gauss function. Note that Gauss function and functional are isotropic,i.e. invariant under rotation w.r.t. z 0

23 Sigma-approximation of tangent vectors as functionals associated with derivative of Gauss function If X is a divergent free vector field, then the functional T X G associated with X G is a σ-approximation of the vector X z0 etc.

24 Visual neurons as functionals ( filters ) Many visual cells of early vision (in retina, LGN and V1 cortex) can be considered as functionals I F (I ) on the space of input functions, which associate with an input function I on retina the number, the degree of excitation of the cell. At some approximation, this functional can be considered as a linear functional ( generalized function or linear filter ) which measure the integral of the input function I in some small domain D with appropriate weight F (receptive profile (RP)): T F : I F (z)i (z)dxdy, D where F is a smooth function with support D ( receptive field (RF) of the cell). RP is ordinary constructed from the Gauss function.

25 Marr and Gabor filters Sometimes, more realistic assumption is that a cell with RP F acts as the convolution operator I F I, where the new function (F I )(z 0 ) := T F I z0 is obtained by integration of the shifted function I z0 (z) := I (z z 0 ) which is produced by movements of eye. A first model of cell with isotropic RP was proposed by D. Marr. It based on Kuffler description of ganglion cells (especially, P-cells) and is applied also to many types of cells in LGN and even in V1 cortex. Marr cell works as Marr filter (linear isotropic functional with RP Gz σ 0 ). Important anisotropic model the model of simple cells of V1 cortex. It is described as Gabor filters : linear functional with Gabor function as RP: Gab = kg0 σ (cos 2y + i sin 2y) w.r.t.some Cartesian coordinates (x, y) with center at 0.

26 On and Off Marr cell

27 Note that for small σ the first and the second directional derivatives of the Gauss functions are approximated by the odd and even Gabor functions. For example, x G σ 0 = x σ 2 G σ 0 sin (x/σ 2 )G σ 0 ( y ) 2 G σ 0 = 1 σ 2 (1 (y/σ) 2 )G σ 0 1 σ 2 cos( 2y/σ)G σ 0. So in sigma-approximation of DG the odd Gabor filters correspond to tangent vectors and even Gabor filters correspond to second order tangent vectors i.e. second jets of curves.

28 7. TRANSFORMATION OF INPUT FUNCTION IN RETINA AND LGN. Processing and data conversion in retina, LGN and V1 cortex 1. Retina produces a smoothing and contourization of the input function by Marr filters. 2. LGN organizes an additional isotropic preparation of the input data. Using feed back from the higher levels of the visual system, it modifies and correct parameters of visual cells (e.g. the scale ). 3.Recognition of local pieces of contours is started in V1 cortex. 4. There are two independent channel of data conversion: P-channel from P-cells of retina to 4 upper parvocellular layers of LGN ( which consists of parvocellular small cells) and then to 4β layer of V1 cortex.it is responsible for stable contours. M-channel from M-cells of retina to lower two layers of LGN (containing magnocellular big cells) and then to layer 4α of V1 cortex. It is responsible for detection and analysis of moving objects.

29 Retina Retina

30 Retinotopic map from retina to LGN and V1 cortex P cells project to upper 4 parvocellular layers of LGN and then to the layer 4β of V 1 M cells project to lower two magnocellular layers of LGN and then to the layer 4α of V1. Schwartz meromorphic dipole formula for retinotopic maps (E. Schwartz, 2002) z w = k z + a z + b Conformality.

31 Retinotopic map from LGN to V1 cortex

32 8. PRIMARY VISUAL CORTEX V1. Columnar structure and pinwheel field Local quantities in V1 ( RF, orientation, spatial frequency,temporal frequency, ocular dominance etc.) Columnar structure of V Cells with approximately the same RF are organized in vertical columns. Simple cells of a column at a point acts as Gabor filter with some orientation. For a regular point p V all simple cells of the column have the same orientation Γ p = RX p PT p V. Simple cells of a column at a singular point (called pinwheels) have all possible orientations and are parametrized by a circle. We get a fundamental 1-dimensional distribution with singularities (Pinwheel field) p Γ p

33 Pinwheel field of directions

34 J.Petitots model of V1 cortex. Parametrization of simple cells Petitot considers primary visual cortex VI as a surface V with a field of directions Γ. He observes that if we consider parametrization of simple cells according to their function (as Gabor filters), they will be parametrized by a surface Ṽ which is the blowing up of V at all centers of pinwheels. All simple cells of the column at a regular point acts as the same Gabor filter and define one point in Ṽ. Simple cells of columns at a singular point z (pinwheel) measure contours of any direction and parametrized by a circle (preimage of z in Ṽ ).

35 Petitot s model: Primary cortex as a contact bundle Under approximation that all points are centers of pinwheels, J.Petitot concludes that the space of simple cells can be identified with the contact bundle PTV = PT V V of directions with the natural contact structure.(before it was guessed by Hoffman). Simple cells of V detect not only points z of a contour C V, but also its direction T z C T. So they determine the horizontal lift of the contour to the horizontal curve C T PT (V ). If (x, y) are coordinates in V such that contours are described as y = y(x), then the contact manifold PTV can be locally identified with the manifold J 1 (R) of 1-jets of functions with coordinates (x, y, p = dy dx ) and the contact form η = dy pdx. The contact manifold J 1 (R) is identified with the Heisenberg group Heis 3 with a left invariant contact structure.

36 Generalized Petitot s model by Sarti-Citti-Petitot The simple cells of a singular column z 0 (i.e. Gabor filter with center at z 0 ) are parametrized by an angle θ PT z 0 V = S 1 = SO 2 and a scale σ that is with points of the group CO 2 = R + SO 2 or with the set of conformal frames in T z0 S 2. The set of simple cells locally, can be parametrized by the Borel group B = G 0 G = Sim(E 2 ) = CO 2 R 2 and the space of simple cells in V1 (in Petitot s approximation) can be considered as the principal CO 2 -bundle R + PT V = T V of conformal frames on S 2 with natural symplectic structure. This is the basic assumption of the generalized Petitot model.

37 Parametrization of a hypercolumn by stability group G + = G 0 G + = Sim(E 2 ) and V1 as the principal bundle of conformal frames of second order (Cartan connection) We conjecture that simple cells of a hypercolumn are parametrized locally by points of the stability subgroup B = G 0 G + CO 2 R 2 (or, equivalently) its Lie algebra. Two new parameters (a, b) R 2 + (the coordinates of an element from G + R 2 with respect to the basis Y 1, Y 2 ) corresponds to the differential dσ of the scale σ. They are second order objects, i.e. are defined by the second jet. Then in Petitot approximation (when we consider all points as center of pinwheels), we get the Tits model of the eye sphere where points are identified with the corresponding stability subgroups ( or subalgebras). One of the advantage of this generalized Petitot model is that it allows to explain partially the invariance of perception w.r.t. fixation eyes movements.

38 A neurophysiological approach to problem of conformal invariancy Let G be a transformation group of a manifold V (e.g the group SO 2 of rotation in V = R 2 ) and S is a G orbit. If observers are distributed along S and send information (say, about a curve in V ) to some center O anonymously, the information obtained by O will be G invariant. If simple cells of a hypercoloumn send information to a complex cell C it become invariant w.r.t. the stability subgroup. If C get information from other neighborhood hypercolumns, it become invariant w.r. t. the conformal group G.