Jacobi s Iterative Method for Solving Linear Equations and the Simulation of Linear CNN

Transcription

1 Jacobi s Iterative Method for Solving Linear Equations and the Simulation of Linear CNN Vedat Tavsanoglu Yildiz Technical University 9 August 006 1

2 Paper Outline Raster simulation is an image scanning-processing procedure for solving the system of difference equations of CNN. We prove that this iteration process is in fact the Jacobi iteration applied to the linear algebraic equations obtained from the state equations of a CNN with constant input by setting the derivatives of the states to zero. 9 August 006

3 Paper Outline Jacobi iteration equation and that obtained by setting the integration time-step to Ts=1/ 1/-a 00 in the discrete-time time state equations obtained by the use of Euler forward difference are the same. Depending on the value of the bandwith in a linear filter closed form equation may be preferred to the raster simulation. 9 August 006 3

4 A Jacobi s Iterative Method Suppose that the nonsingular square matrix split into three parts A a11 a 1.. a1n a a1 a.. a N 0 a a a.. a a N1 N NN NN a 1.. a1n a a N a31 a a N 1,N an1 a N. an,n A is A D-L-U 9 August 006 4

5 Jacobi s Iterative Method For a nonsingular matrix A the solution of the linear equation is given as: A x=b x= A -1 b 9 August 006 5

6 Jacobi s Iterative Method Now rearrange A x=b as Dx = (L + U)x + b where x=d -1-1 A x+d b A =L+U 9 August 006 6

7 Jacobi s Iterative Method Now assign arbitrary values to the vector x on the right-hand side and call it x(0) and assume that the x on the left-hand side is x(1): or in general x(1) = D x(k + 1) = D -1-1 A x(0) + D b -1-1 A x(k) + D b 9 August 006 7

8 Jacobi s Iterative Method The equation x(k + 1) = D -1-1 A x(k) + D b is the same as the state equation of a linear discrete-time time system with the constant input b where the state matrix and the input matrices are given as: A =D -1 A and D -1 9 August 006 8

9 Jacobi s Iterative Method Now defining -1 β =D b yields x(k + 1) A x(k) + β 9 August 006 9

10 Jacobi s Iterative Method Starting with an initial state the solution is obtained as: A k A k-1 A k- A x k = x I β Theorem 1: The steady-state value of x is given by xk = I-A -1 β 9 August

11 In Jacobi s Iterative Method xk = x(k)+ β k k-1 A I + A+...+ A define S I + A+...+ A k-1 We can write AS A+...+ A k S- S=I- k A A -1 k S= I-A I-A where we assumed that A does not possess unity eigenvalues 9 August

12 Jacobi s Iterative Method If the "spectral radius" of is less than, then for sufficiently large A k 0 and -1-1 A A k A S= I- I- I- 9 August 006 1

13 Jacobi s Iterative Method and consequently yields xk = x(k)+ β k k-1 A I + A+...+ A xk = k-1 I + A+...+ A xk I-A β -1 β 9 August

14 Jacobi s Iterative Method Now we will show that which is the solution of -1 x k I-A β = A -1 b A x=b 9 August

15 Jacobi s Iterative Method which can be proved as follows: A A A I- β = I- D b = I- -D -D D b A A Q.E.D. = D+ -D DD b = b 9 August

16 Jacobi s Iterative Method Result: Calculation of x(k) using xk = k-1 I + A+...+ A is Jacobi iteration. The same can be achieved using the closed-form equation: xk I-A β -1 β 9 August

17 Jacobi s Iterative Method A x0+β x1 = x = Ax 1 +β = A Ax 0 +β + β = A x 0 + A+I β k k-1 k- A A A A 3 x 3 = Ax +β = A A x 0 + A+I β + β = A x 0 + A + A+I β = xk = x Iβ X=zeros(M*N,1); for k=1:10000 X=A*X+beta; if abs((x-x_final)./x_final)<0.01 break end end 9 August

18 u ij x, y ij Application to CNN A space-invariant linear analog CNN is completely described by the cell state equation: dx ij dt AX BU ij ij 9 August

19 Application to CNN The template dot products are defined as : a 1, 1 a 1,0 a 1,1 xi 1, j 1 xi 1, j x i1, j1 1 1 AX ij a a a x x x a x a a a x x x b 1, 1 b 1,0 b 1,1 ui 1, j 1 ui 1, j u i1, j1 1 1 BUij b0, 1 b0,0 b0,1 u u u b u m1n1 b b b u u u 0, 1 0,0 0,1 i, j1 i, j i, j1 i, j im, jn m1n1 1, 1 1,0 1,1 i1, j1 i1, j i1, j1 i, j1 i, j i, j1 i, j im, jn 1, 1 1,0 1,1 i1, j1 i1, j i1, j1 where x ij is the state of pixel (i,j) u ij is the input of pixel (i,j) 9 August

20 Application to CNN In matrix form the state equations can be given as: where dx = dt A x+bu x is the row-wise temporal state-vector u is the row-wise temporal input-vector -A is the temporal state matrix B is the temporal input matrix 9 August 006 0

21 AA 0 Application to CNN Now using the centre and surround templates 0 A and defined as A A A a a a 1, 1 1,0 1,1 0 a0,0 0 a0, 1 0 a0, a1, 1 a1,0 a 1,1 the CNN cell state equation can be rewritten as: dx dt ij a x A X BU 00 ij ij ij 9 August 006 1

22 Application to CNN In the case of a constant input and a stable system all states will tend to constant final values, that is, for dx ij t 0 dt dx dt ij a a x A X BU 00 xij A Xij BU ij 00 ij ij ij 9 August 006

23 Application to CNN L In matrix form where 1 A X BU a x ij ij ij a a31 a an1 a N. an, N x=d (L+U)x+D b U 0 a.. a a a N N N1,N D= a 00 I 9 August 006 3

24 Application to CNN Defining yields Now using A -1 x=d L+U A x+b A results in -1 =D A -1 β =D b x= A x+β 9 August 006 4

25 Application to CNN Consider once again dx dt ij a x AX BU 00 ij ij ij Applying Euler approximation to the derivative we obtain ij 1 x n x n T s ij a x n A X n B U 00 ij ij ij 9 August 006 5

26 Application to CNN x ij n T s 1 1 x na x n A X nbu T s ij 00 ij ij ij Letting yields Ts 1 a 00 1 x n n 1 A X ( ) BU ij ij ij a00 9 August 006 6

27 Application to CNN Writing 1 x n n 1 AX ( ) BU ij ij ij a00 in matrix form or more compactly -1 x n+1 = D A x n +b xn+1= A xn+β 9 August 006 7

28 Application to CNN xn+1= A xn+β x= Ax+β is the state equation of the dynamical discretetime system obtained by applying Euler forward 1 difference with Ts a 00 to the state equations of the analog CNN. is the equation of the static system obtained by assuming that the dynamical system,i.e., the analog CNN has reached the steady state. 9 August 006 8

29 Application to CNN The iterative solution of the dynamical system xn+1= A x= Ax+β xn+β and the Jacobi iterative solution of the static system are exactly the same. 9 August 006 9

30 Application to CNN In xn+1= A xn+β n corresponds to the number of physical time-steps Inthecaseof x= A x+β such n represents the number of iterations in seeking the solution. 9 August

31 Ts 1 a 00 Application to CNN Theorem : The linear algebraic Jacobi iteration equations obtained from the state equations of a linear analog CNN with constant input by setting the derivatives of the states to zero and those obtained by setting the integration time-step to 1 Ts a 00 in the discrete-time time state equations obtained by the use of Euler forward difference are the same. 9 August

32 Application to CNN The solution of these equations are given by xk = I-A -1 β where for sufficiently large k, x(k) denotes the final pixel values of the output image. 9 August 006 3

33 Example on Gauss-type and Gabor type Filters Gauss-type Filter Circuit Symmetric feedback and input templates 0 G A G ( G 4 G ) G, B 0 G G For G, G A 1 ( 4) 1, 0 0 B August

34 A = Example on Gauss-type and Gabor-type Filters Temporal state matrix for an input image size of 4x4 with symmetric A in row-wise packing scheme August

35 j yo xom j n amn, e e Example on Gauss-type and Gabor-type Filters Replacing a mn, by jxom j yon amn, e e Symmetric template Hermitian template jxo e 0 jyo j yo A 1 ( 4) 1 e ( 4) e jxo e 0 Gauss - type Gabor - type 9 August

36 j yo xom j n amn, e e Example on Gauss-type A = and Gabor-type Filters Temporal state matrix for an input image size of 4x4 with symmetric A in row-wise packing scheme j yo j 4 e 0 0 e xo jyo jyo j xo e 4 e 0 0 e jyo jyo j xo 0 e 4 e 0 0 e j yo j 0 0 e xo e jxo j yo j e xo 4 e 0 0 e jxo jyo jyo j 0 e 0 0 e xo 4 e 0 0 e j xo jyo j yo 0 0 e 0 0 e 4 e 0 0 j e xo j xo j xo jyo j e 0 0 e e xo j xo j yo j xo e e 0 0 e j xo jyo jyo j e 0 0 e 4 e 0 0 e xo 0 0 j xo jyo jyo j xo e 0 0 e 4 e 0 0 e e jyo j xo jyo jyo j xo jyo jyo j 0 0 e e xo j xo j yo e e 0 0 j xo jyo e 0 0 e 4 e e 0 0 e 4 e e 0 0 e 4 9 August

37 Example on Gauss-type and Gabor-type Filters 9 August

38 Example on Gauss-type and Gabor-type Filters 9 August

39 Example on Gauss-type and Gabor-type Filters Theorem 3 : Every real symmetric (Hermitian( Hermitian) matrix is orthogonally (unitarily) similar to a diagonal matrix, i.e.: A -1 =TΛT Columns of T are the eigenvectors of A is diagonal whose diagonal elements are the eigenvalues of A 9 August

40 Example on Gauss-type and Gabor-type Filters Orthogonal similarity implies: -1 t T = T Unitary similarity implies: T = T -1 t where T denotes conjugate 9 August

41 Example on Gauss-type and Gabor-type Filters For a symmetric state matrix A (or A ) -1 xk TI-Λ T t β For a Hermitian state matrix A (or A ) x k T I-Λ T β -1 t 9 August

42 Example on Gauss-type and Gabor-type Filters For xo yo Λ = diag , , , , -0.5, -0.5, 0, 0, 0, 0, 0.5, 0.5, 0.309, 0.559, 0.559, August 006 4

43 Example on Gauss-type and Gabor-type Filters -1 xk = I-A β tinv -1 t xk TI-Λ T β -1 xk TI-Λ T t β t eig I + A+...+ A k-1β t jcb xk = 9 August

44 Example on Gauss-type and Gabor-type Filters Image size: : 3x3 tinv teig t jcb tinv (stored) Gauss Gabor Gauss Gabor Gauss Gabor Gauss Gabor t t t jcb inv eig t t t jcb inv eig t t t inv jcb eig t t t inv jcb eig t t t inv jcb eig t t t inv jcb eig t t t inv jcb eig t t t inv jcb eig August

45 Conclusions Jacobi iteration and raster simulation are the same. 1 The condition for the above is: Depending on the value of (bandwith) closed form equation may be preferred T s a 00 9 August