Methods in Computer Vision: Introduction to Matrix Lie Groups

Methods in Computer Vision: Introduction to Matrix Lie Groups Oren Freifeld Computer Science, Ben-Gurion University June 14, 2017 June 14, 2017 1 / 46

Definition and Basic Properties Definition (Matrix Lie groups) Let n be a fixed positive integer. A matrix Lie group is a set G of n n matrices together with the binary operation of matrix product on G (that is, the domain is G G) such that: (G1) I n n G; (G2) A, B G AB G; (G3) A G A 1 exists, A 1 G. (note AA 1 = A 1 A = I) Matrix Lie groups are also called matrix groups, the terms being identical. It is possible to use a similar definition for matrix Lie groups whose elements take complex values; however, we restrict the discussion to real-valued matrix Lie groups. June 14, 2017 2 / 46

Definition and Basic Properties Let G be a matrix group. Basic linear algebra shows that: (G4) A G AI n n = I n n A = A (G5) A, B, C G (AB)C = A(BC) = ABC (G6) A G AA 1 = A 1 A = I n n. June 14, 2017 3 / 46

Definition and Basic Properties What (G1) through (G6) imply is that the set G, together with the binary operation of matrix product, is a group and that I n n is the identity element of the group. So unsurprisingly, every matrix group is a group. When n is understood from the context, we will sometimes denote the identity matrix by I instead of I n n. June 14, 2017 4 / 46

Definition and Basic Properties If G is a set of n n matrices, which may or may not form a matrix group once taken together with matrix product, it may be the case that there is some other binary operation, denoted by, say, : G G G, such that the pair (G, ) forms a group. This group, however, is not a matrix group, although it is a group of matrices. E.g., if G is n n matrices and is matrix addition, than the group (G, ) is not a matrix group. June 14, 2017 5 / 46

Definition and Basic Properties Henceforth, by a slight abuse of notation, we will write expressions such as G is a matrix Lie group, with the convention that we will mean that the set G, together with matrix product, is a matrix Lie group. Similarly, we will write the matrix Lie group G is given by the following set, meaning that the set, together with matrix product, defines the matrix Lie group of interest. June 14, 2017 6 / 46

Definition and Basic Properties Definition (The difference between two elements in a matrix Lie group) Let G be a matrix Lie group, and let A and B be in G. The group difference of A and B is given by A 1 B; it is not symmetric. Note that (G2) and (G3) imply the following: (G7) A, B G A 1 B G. Properties (G2), (G3), and (G7), are referred to as the group (algebraic) closure under its operations of composition, inversion, and difference, respectively. Suppose our data can be represented as elements of a matrix Lie group. These closure properties imply that the representation is consistent. Definition (Abelian matrix Lie group) If AB = BA for any A and B in a matrix Lie group G, then G is called Abelian. June 14, 2017 7 / 46

Definition and Basic Properties The class of matrix Lie groups is contained in the more general class of (finite-dimensional) Lie groups: every matrix Lie group is a Lie group while the converse is false. Matrix Lie groups are simpler to work with (and define) than the more general case of Lie groups. We will restrict discussion to matrix Lie groups. June 14, 2017 8 / 46

Several Important Matrix Lie Groups We start with the most general matrix Lie group. Definition (The general linear group of order n) The general linear group of order n, denoted by GL(n), is given by {Q Q R n n, det Q 0} We call it the most general as, by (G3), we see that every matrix Lie group of n n matrices is a matrix Lie subgroup 1 of GL(n). 1 The relation between a matrix Lie subgroup and a matrix Lie group is analogous to the relation between a subgroup and a group. June 14, 2017 9 / 46

Several Important Matrix Lie Groups GL(n) is not a connected space. To see that, pick a matrix A in GL(n) with a positive determinant and a matrix B in GL(n) with a negative determinant. Then, set p = A and q = B. As det : R n n R is continuous continuous curve between p and q must pass through a matrix of zero determinant; i.e., the curve must make an excursion outside of GL(n). In fact, it can be shown that GL(n) has exactly two connected components. Definition (The identity component) The identity component of a matrix Lie group is one that contains I n n. The identity component of a matrix Lie group is always a matrix Lie group by itself. Definition (GL + (n)) The identity component of GL(n), denoted by GL + (n), is given by {Q Q R n n, det Q > 0} June 14, 2017 10 / 46

Several Important Matrix Lie Groups Not every set of n n invertible matrices is a matrix group. E.g., consider SPD matrices. Even if A and B are SPD, C = AB is usually not SPD. The problem is not with the positive definiteness, but with the symmetry. June 14, 2017 11 / 46

Several Important Matrix Lie Groups Definition (US(n)) The uniform scale group, denoted by US(n), is given by {Q Q = SI n n, S R 0 }. June 14, 2017 12 / 46

Several Important Matrix Lie Groups Definition (The orthogonal group) The orthogonal group of degree n is given by O(n) {R R n n R T R = RR T = I n n } The distinction between O(n) and the Stiefel manifold of order (n, n), is that the Stiefel manifold is just the set, {R R n n R T R = RR T = I n n }, without the binary operation of matrix multiplication. June 14, 2017 13 / 46

Several Important Matrix Lie Groups Definition (The special orthogonal group) The special orthogonal group of degree n, also known as the rotation group and denoted by SO(n), is given by SO(n) {R R n n R T R = RR T = I n n & det R = +1} SO(n) is a subgroup of O(n) June 14, 2017 14 / 46

Several Important Matrix Lie Groups In particular, we single out the case of n = 3. Definition (SO(3)) The special orthogonal group of degree 3, also known as the rotation group (of order 3), is denoted by SO(3). June 14, 2017 15 / 46

Direct Products Matrix Lie groups Direct Products of Matrix Lie Groups Let G i be a matrix subgroup of GL(n i ), i = 1, 2,..., k. The standard group direct product of {G i } k i=1, denoted by G 1 G 2... G k, is not technically a matrix Lie group, as its elements are k-tuples of matrices (such as (g 1, g 2,..., g k )) rather than matrices. It is, however, easy to identify such a direct product with a matrix Lie subgroup of GL(n 1 + n 2 +... + n k ) using the correspondence between a k tuple (g 1, g 2,..., g k ) and a block-diagonal matrix: g 1 0 g (g 1, g 2,..., g k ) 2... GL(n 1 + n 2 +... + n k ). 0 g k If the G i s are k copies of the same matrix Lie group G, then we denote their direct product by G k. June 14, 2017 16 / 46

Matrix Lie Groups as Smooth Manifolds A matrix Lie group is a nonlinear space. Proof. If A is any invertible matrix (in fact, in GL(n)), then A A = 0 n n is not invertible. Thus, while a matrix Lie group is closed under the operations mentioned earlier, it is not closed under linear combinations. In particular, it does not make sense to talk about linear subspaces of such a group. It may make sense to talk about subsets, and these subsets may or may not be subgroups of the group. It may seem odd that an object called the general linear group is not linear, but the linearity in the name refers to the fact that every element A of the group is affiliated with a linear map R n R n, x Ax. June 14, 2017 17 / 46

Matrix Lie Groups as Smooth Manifolds While nonlinear, every matrix Lie group is a finite-dimensional smooth manifold. Thus, a matrix Lie group is a group on which one can do calculus. Alternatively, a matrix Lie group is a smooth manifold with a group structure. Groups are usually denoted by G, while manifolds are usually denoted by M. Since a matrix Lie group is both a group and a manifold, both notations may be used. June 14, 2017 18 / 46

Dimensionality Matrix Lie groups Matrix Lie Groups as Smooth Manifolds Every matrix group of n n matrices is a nonlinear subset of R n n, the latter being an n 2 -dimensional space. Since such a group is also a finite-dimensional manifold, it has its own dimension. Let D denote the dimension of the group; note D n 2. The dimension of both GL(n) and GL + (n), e.g., is D = n 2. Note this is the same dimension as that of R n n, in spite of the fact that GL + (n) is a proper subset of GL(n), which in turn is a proper subset of R n n. The point here is that once we deal with nonlinear manifolds, even if two spaces have the same dimension, one can still be strictly larger than the other. June 14, 2017 19 / 46

Dimensionality Matrix Lie groups Matrix Lie Groups as Smooth Manifolds The dimension of SO(2) is 1. The dimension of SO(3) is 3. More generally, the dimensionality of SO(n) is n 2 n 2 June 14, 2017 20 / 46

Smoothness and Tangent Spaces Matrix Lie Groups as Smooth Manifolds An important implication of the fact that a matrix Lie group G is a smooth manifold is that it is possible to define smooth curves on G. Note that if c : J G is a smooth curve from some open interval J in R into G, then c(t) is itself a matrix that belongs to G. If we differentiate c w.r.t. t (i.e., differentiate each entry of the matrix c(t) w.r.t. t), then we will get a new curve, ċ. However, this curve is a map ċ : J R n n, not ċ : J G. Regarding R n n as R n2 (which is hard to visualize: even for 2-by-2 matrices this is already a 4D space), we can imagine ċ(t) as an n 2 -dimensional vector attached to the curve at the n 2 -dimensional point c(t). June 14, 2017 21 / 46

Matrix Lie Groups as Smooth Manifolds In terms of standard differential geometry of curves and surfaces in Euclidean spaces, ċ(t) is a tangent vector to the curve. In fact, it is also a tangent vector to the manifold at the point p. For a given point p, and a given open interval J that contains 0, consider all smooth curves c : J G that pass though p and satisfy c(0) = p. Let us denote this class of curves by C J,p. The tangent vectors, at p, are given by the set of distinct 1 elements {ċ(0) : c C J,p }. 1 Many different curves in C J,p can have the same derivative at t equals zero. Thus, their derivatives form equivalent classes. The set-theoretic notation means we pick one representative from each class. June 14, 2017 22 / 46

Matrix Lie Groups as Smooth Manifolds It can be shown that the particular choice of J is immaterial. It turns out that the tangent vectors form a subspace of R n n. This subspace is denoted by T p G (had we used M instead of G we would have written T p M) and its dimension is identical to that of the matrix Lie group G: dim G = dim T p G. June 14, 2017 23 / 46

Matrix Lie Groups as Smooth Manifolds E.g., for G = GL(n), the dimension of the tangent space is n 2 (so T p G is a perfect copy of R n n, both being Euclidean spaces of the same dimension) while for G = SO(3) the dimension of the tangent space is 3. Let G be a D-dimensional matrix Lie group of n n matrices. Since at every point p in G we can attach a tangent space which is a copy of R D, all these tangent spaces are the same in some sense. Having said that, the tangent space at the identify (i.e., p = I) is rather special as we shall see. June 14, 2017 24 / 46

Matrix Lie Groups as Smooth Manifolds In spite of the fact that an element of T p G is always an n n matrix, it is often useful to regard it a vector in R n n. For this we use the following notation: VEC( ) : T p G R n2 MAT( ) : R n2 T p G (1) where VEC stands for concatenating the matrix columns (or rows) in a single long column, while MAT is the inverse of VEC. June 14, 2017 25 / 46

Matrix Lie Groups as Smooth Manifolds Characterization of Elements of a Tangent Space Let c C J,p. Define another smooth curve by c I : J G, c I : t p 1 c(t). We think of this operation as left-translation (of a curve) by p 1. It follows that c I (0) = I (as c(0) = p) and that c I C J,I. Likewise, for every c in C J,I there is an element in C J,p, denoted by c p, such that c p : J G, c p : t pc(t). Naturally, we think of this as left-translation (of a curve) by p. These relations establish an obvious bijection between C J,I and C J,p. Let c C J,I. Since d dt (pc(t)) = p d t=0 dt t=0 c(t), it follows that the tangent vectors that comprise T p G are given by T p G = {px : x T I G} Note: p G, x T I G, and their matrix product, px, is in T p G. June 14, 2017 26 / 46

Matrix Lie Groups as Smooth Manifolds Consequently, if we know how to characterize T I G, then we know how to characterize T p G. Similarly, We think of the map T I G = {p 1 x : x T p G} T I G T p G x px as left-translation (of a tangent vector) by p. Right-translation by p is defined similarly by T I G T p G x xp June 14, 2017 27 / 46

Matrix Lie Groups as Smooth Manifolds More generally, if p and g are in G then one way to map, bijectively, the tangent vectors in T p G into T q G is by left-multiplication of a tangent vector (which is in fact a matrix) from T p G by qp 1 to produce another tangent vector (again, a matrix) in the T q G Again, this is called left-translation, with right-translation defined in a similar way. June 14, 2017 28 / 46

Tangent Spaces to the Orthogonal Group Matrix Lie Groups as Smooth Manifolds Let R : (α, β) O(n) be a differentiable curve. Suppose that R t=0 is the n n identity matrix. Then: R T R = I n n d dt (RT R) = d dt I n n d dt (RT )R + R T d dt (R) = 0 n n Setting t = 0 and letting ω = d dt R t=0, we obtain: ω T + ω = 0 n n Thus, ω is skew-symmetric; namely, ω = ω T. June 14, 2017 29 / 46

Tangent Spaces to the Orthogonal Group Matrix Lie Groups as Smooth Manifolds Let R : (α, β) SO(n) be a differentiable curve. If R t=0 is the n n identity matrix and ω = d dt R t=0 then ω is skew-symmetric. In other words, SO(n) and O(n) have the same tangent space at the identity, and thus, at every p, the two groups share the same tangent space (see next slide) June 14, 2017 30 / 46

Tangent Spaces to the Orthogonal Group Matrix Lie Groups as Smooth Manifolds If p is a point in SO(n) (or SO(n) such that p I, we characterize the elements of T p SO(n) by T p SO(n) = {pa : A T I SO(n)}, i.e., T p SO(n) (or T p O(n)) is exactly the set of all n n matrices that can be written as the matrix product of the matrix group element p, and some n n skew-symmetric matrix. June 14, 2017 31 / 46

The Matrix Exponential and Matrix Logarithm Definition (The matrix exponential) where A k = exp : R n n R n n, exp : A k=0 k times {}}{ AA... A is a sequence of matrix products. For a real-valued square matrix A, we can define the matrix logarithm, denoted log(a) as a square matrix satisfying exp(b) = A. A k k! June 14, 2017 32 / 46

The Matrix Exponential and Matrix Logarithm The matrix exponential is generalization of the simple exponential function from the scalar case. There are however, some notable differences. E.g., let A and B be some two n n matrices. In general, exp(a) exp(b) exp(a + B), with equality if and only if AB BA = 0. Likewise, log(a 1 B) log(a) + log(b); however, as a first-order approximation, we have the following important result: log(a 1 B) log(b) log(a). June 14, 2017 33 / 46

The Matrix Exponential and Matrix Logarithm If A is block-diagonal, then exp(a) is also block diagonal and each block can be exponentiated independently. In the extreme case that the matrix is diagonal, then we can simply use scalar exponentiation for each diagonal entry. As a particular case, exp(0 n n ) = I n n and log(i n n ) = 0 n n. If A is nilpotent (so A k = 0 n n for some finite k), then the series has a finite number of terms. If A is diagonalizable, there is a simple way to exponentiate it using diagonalization and scalar exponentiation. Sometimes, if A has a specific structure (such as, but not limited to, the structures mentioned above) it is possible to compute exp(a) without having to deal with the infinite sum. If no structure on A can be utilized, then exp(a) can be efficiently approximated. June 14, 2017 34 / 46

Matrix Exponential Matrix Lie groups The Matrix Exponential and Matrix Logarithm Usually, the matrix exponential has no closed form so one resort to (good) numerical approximations. But if a matrix has some special structure, then sometimes we can derive a closed-form matrix exponential. This is the case, e.g., for SO(3). In which case, the closed-form solution is known as Rodrigues formula. In OpenCV: cv2.rodrigues June 14, 2017 35 / 46

The Matrix Exponential and Matrix Logarithm Ω = ω = 0 ω z ω y ω z 0 ω x ω y ω x 0 ω x ω y ω z SO(3) R = exp(ω) Rodrigues formula so(3) θ = ω r = r x r y r z = 1 θ ω = cos θi 3 3 + (1 cos θ)rr T + sin θ For inversion (computing ω from R), use: 0 r z r y sin θ r z 0 r x = R + RT 2 r y r x 0 0 r z r y r z 0 r x r y r x 0 June 14, 2017 36 / 46

The Lie Algebras of Matrix Lie Groups While Lie algebras are mathematical objects that are worth studying for their own merit, and while they can also be defined without any reference to Lie groups, our interest in them is due to the fact they provide an indispensable tool when working with Lie groups. In fact, one of the main reasons for the attractiveness of Lie groups (and not just matrix Lie groups) is their Lie algebras. We avoid giving the most general definition of Lie algebras and confine ourselves to Lie algebras of real-valued matrix Lie groups. June 14, 2017 37 / 46

The Lie Algebras of Matrix Lie Groups Definition (Lie algebra) Let G be a real-valued matrix Lie group. Its Lie algebra g is given by the vector space g = exp 1 (G), and [, ] : g g g, the Lie bracket of g, is given by (A, B) AB BA. g is closed under the Lie bracket. June 14, 2017 38 / 46

The Lie Algebras of Matrix Lie Groups Remark Note that the notation exp 1 (G) does not imply that the map exp : g G is invertible; rather, it is merely the standard set-theoretic notation for preimage of the set G under the map exp : g G, which means all those elements (of R n n ) such that when we exponentiate them, we end up in G. In other words, exp 1 (G) {A : exp(a) G}. In particular, depending on G, the map exp : g G might not be surjective. June 14, 2017 39 / 46

The Lie Algebras of Matrix Lie Groups Example (exp is not always a surjection) Since it can be shown that det(exp(a)) = e tr(a) (note the RHS is always positive), it follows that exp(a) always has a positive determinant. Consequently, if B has a negative determinant (hence B GL(n)), then there does not exist an A R n n such that exp(a) = B. Moreover, depending on G, the map exp : g G might not be injective. June 14, 2017 40 / 46

The Lie Algebras of Matrix Lie Groups As in the case of matrix Lie groups, we will usually avoid mentioning the Lie bracket, and simply refer to the vector space as the Lie algebra g. It turns out that the elements of g coincide with those of T I G, the tangent space at the identity. Thus, we will use g and T I G interchangeably. June 14, 2017 41 / 46

The Lie Algebras of Matrix Lie Groups Definition (The gl(n) Lie algebra) The Lie algebra of both GL(n) and GL + (n) is given by gl(n) exp 1 (GL(n)) = exp 1 (GL + (n)) = R n n. Definition (The us(n) Lie algebra) us(n) exp 1 (US(n)) = {Q : Q = si n n, s R}. June 14, 2017 42 / 46

The Lie Algebras of Matrix Lie Groups Definition (The so(n) Lie algebra) so(n) exp 1 (O(n)) = exp 1 (SO(n) = {Q : Q = Q T } gl(n). And again, will single out the 3D case: Definition (The so(3) Lie algebra) so(3) exp 1 (SO(3)) = {Q : Q = Q T } gl(3). June 14, 2017 43 / 46

The Lie Algebras of Matrix Lie Groups More generally, the matrix exponential maps linear subspaces of the Lie algebra to matrix Lie subgroups. This turns out to be very convenient for doing, e.g., statistics. Finally, the Lie algebra of a direct product of matrix Lie groups can be treated in terms that are direct analogs of the way we handle a direct product of matrix Lie groups. June 14, 2017 44 / 46

The Special Orthogonal Group The Lie Algebras of Matrix Lie Groups Recall this manifold is three dimensional. View it as a smooth nonlinear subset of R 9. Think of ω = d dt R t=0 as a tangent vector. Since ω has 3 degrees of freedom, it is also 3D in nature. Point: the tangent space (in this case, at the identity) has the same dimensionality as the manifold. In fact, this is a general result for smooth manifolds. The tangent space at the identity of a (matrix) Lie group is called a (matrix) Lie algebra. The Lie algebra of SO(3) is denoted so(3). Turns out, the matrix exponential maps so(3) to SO(3); i.e., if ω is a 3 3 skew-symmetric matrix, then R = exp(ω) is a rotation matrix. Again, this too is a general result for matrix Lie groups and matrix Lie algebras. June 14, 2017 45 / 46

The Lie Algebras of Matrix Lie Groups Utilizing the Tangent Space for Handling the Nonlinearity For optimization (e.g., during extrinsic calibration) For tracking For statistics June 14, 2017 46 / 46