1809, Carl Friedrich Gauss

Transcription

1 1809, Carl Friedrich Gauss

2 Singular Value Decomposition A = U D V T m x n m x n n x n n x n

3 Singular Value Decomposition A = U D V T m x n m x n n x n n x n Column orthogonal matrix U i T U i U j T T = =1 = UU =0 for i U i U i j i j

4 Singular Value Decomposition A = U D V T m x n m x n n x n n x n Column orthogonal matrix U i T U i T = =1 = UU =0 for i U i U j T U i j i j U T U =I m m V T V =I n n

5 Singular Value Decomposition A = U D V T m x n m x n n x n n x n Singular value matrix D = diag{ 1, 2,, n } where 1 2 n 0

6 Singular Value Decomposition A = U D V T m x n m x n n x n n x n Basis Scale Address U 1 U 2 U

7 SVD as basis + transformed Address

8 SVD of this?

9 U(:,1:4) D(1:4,1:4) V(:,1:4)

10 [u,d,v] = svd(i); semilogy(diag(d(1:20,1:20)),'x-') Im2 = u(:,1:20)*d(1:20,1:20)*v(:,1:20)';

11

12

13 Images as Vectors n = m n*m

14 Vector Mean n = = m I1 + I2 = mean image n*m n*m

15 Eigenfaces Eigenfaces look somewhat like generic faces.

16 Eigen-images of Berlin

17 Eigen-images Average of 16 individuals transformed via biometrical data of different ethnics

18 Average of 16 individuals transformed via biometrical data of different ages

19 Rank A = U D V T m x n m x n n x n n x n rank(a) = r min(m,n) D r Rank is the same as the number of nonzero singular values

20 Nullspace A = U D V T m x n m x n n x n n x n null(a) = V r:n A V r:n =0

21 Example III (Fundamental Matrix) F = 1.0e+003 * [u,d,v] = svd(f) u = d = 1.0e+004 * v = Rank(F) = 2 d(1,1) ans = e+004 d(2,2) ans = d(3,3) ans = e-016

22 Matrix Inversion with SVD A = U D V T m x n m x n n x n n x n A = V D 1 U T A A =I 1 D = 1, 2,, n diag{1/ 1/ 1/ } if 0, otherwise zero. i

23 Two types of Least Square Problem:

24 2D data x i y i

25 Line fitting x y i i y cx d

26 Line fitting Ei yi -cxi -d Vertical distance y cx d Line fitting error: E y -cx -d y -cx -d N i 1 1 y-cx-d i i 2 2 N N 2

27 Line fitting Ei yi -axi -b Vertical distance y ax b Line fitting error: E y1 x1 1 c d yn xn 1 2 b-ax 2

28 Line fitting Ei yi -axi -b Vertical distance y ax b Line fitting error: E y1 x1 1 c d yn xn 1 2 b-ax T T T T T b-ax b-ax b b-2b Axx A Ax 2

29 Line fitting Ei yi -axi -b Vertical distance y ax b Line fitting error: E y1 x1 1 c d yn xn 1 2 b-ax T T T T T b-ax b-ax b b-2b Axx A Ax 2 E T ba T T 2 2AAx=0 AAx x T ba

30 Line fitting Ei yi -axi -b Vertical distance y ax b x c A T A -1 d b T A Line fitting error: E y1 x1 1 c d yn xn 1 2 b-ax T T T T T b-ax b-ax b b-2b Axx A Ax 2 E T ba T T 2 2AAx=0 AAx x T ba

31 Linear Inhomogeneous Equations 1) rank(a) = r < n : infinite number of solutions = A x b 1 T x=vd Ub+ r+1 V r nvn Particular solution Homogeneous solution A=UDV T V= V1 V n where and. m x n n x 1 m x 1

32 Linear Inhomogeneous Equations = A x b m x n n x 1 m x 1 1) 2) rank(a) = r < n : infinite number of solutions 1 T x=vd Ub+ r+1 V r nvn Particular solution Homogeneous solution A=UDV T where and. rank(a) = n : exact solution x=a 1 b V= V1 V n

33 Linear Inhomogeneous Equations = A x b m x n n x 1 m x 1 1) 2) 3) rank(a) = r < n : infinite number of solutions 1 T x=vd Ub+ r+1 V r nvn Particular solution Homogeneous solution A=UDV T where and. rank(a) = n : exact solution x=a 1 b V= V1 V n n m : no exact solution in general (needs least squa min Ax - b 2 x or x=a\b -1 T T x= A A A b in MATLAB.

34 Two types of Least Square Problem:

35 Line fitting E ex + fy + g i i i Perpendicular distance ex fy g 0 Line fitting error: E ex -fy -g ex -fy -g N i 1 1 ex - fy - g i i 2 2 N N 2

36 Line fitting E ex + fy + g i i i Perpendicular distance ex fy g 0 Line fitting error: x1 y1 1e E f x N yn 1 g 2 Ax 2

37 Line fitting E ex + fy + g i i i Perpendicular distance ex fy g 0 Line fitting error: x1 y1 1e E f x N yn 1 g 2 Ax 2

38 Line fitting E ex + fy + g i i i Perpendicular distance ex fy g 0 Line fitting error: minimize x Ax 2 Trivial solution: x 0

39 Line fitting E ex + fy + g i i i Perpendicular distance ex fy g 0 Line fitting error: minimize x Ax 2 subject to x 1 x V 3 where A UDV T V V V V 1 2 3

40 Linear Homogeneous Equations Linear least square solve produces a trivial solution: -1 T T x= A A A b An additional constraint on x=0 x to avoid the trivial solution: x =1 A x = 0 m x n n x 1 m x 1

41 Linear Homogeneous Equations Linear least square solve produces a trivial solution: -1 T T x= A A A b x=0 An additional constraint on x to avoid the trivial solution: x =1 A x = 0 1) rank(a) = r < n-1 : infinite number of solutions n where 2 r+1 r+1 n n i i=r+1 x= V + + V 1 m x n n x 1 m x 1

42 Linear Homogeneous Equations Linear least square solve produces a trivial solution: -1 T T x= A A A b x=0 An additional constraint on x to avoid the trivial solution: x =1 A x = 0 1) rank(a) = r < n-1 : infinite number of solutions x= V + + V r+1 r+1 n n where n 2 i i=r+1 1 m x n n x 1 m x 1 2) rank(a) = n -1 x=v n : one exact solution

43 Linear Homogeneous Equations Linear least square solve produces a trivial solution: -1 T T x= A A A b x=0 An additional constraint on x to avoid the trivial solution: x =1 A x = 0 1) rank(a) = r < n-1 : infinite number of solutions x= V + + V r+1 r+1 n n where n 2 i i=r+1 1 m x n n x 1 m x 1 2) 3) rank(a) = n -1 x=v n : one exact solution n m : no exact solution in general (needs least squar min Ax 2 x subject to x =1 x=v n