The Triangle Algorithm: A Geometric Approach to Systems of Linear Equations

Transcription

1 : A Geometric Approach to Systems of Linear Equations Thomas 1 Bahman Kalantari 2 1 Baylor University 2 Rutgers University July 19, 2013

2 Rough Outline Quick Refresher of Basics Convex Hull Problem Solving Systems of Linear Equations

3 Introduction (Brief) Definition The convex hull of a finite point set S R n is the set of all convex combinations of its points. The mathematical expression is: S conv(s) := { i=1 α i x i : ( i α i 0) S i=1 α i = 1}

4 Introduction (Brief) Definition The convex hull of a finite point set S R n is the set of all convex combinations of its points. The mathematical expression is: S conv(s) := { i=1 α i x i : ( i α i 0) S i=1 α i = 1} The convex hull of S R 2 forms a convex polygon. And more generally, the convex hull of S R n forms a convex polytope.

5 The Convex Hull Problem A quick refresher: Given a point p R m, we would like to determine whether or not the point lies within the convex hull of a finite set of m-dimensional points S.

6 The Convex Hull Problem A quick refresher: Given a point p R m, we would like to determine whether or not the point lies within the convex hull of a finite set of m-dimensional points S. Theorem (The Distance Duality Theorem) Let S = {v 1, v 2,..., v n } R m, p R m. 1 p conv(s) if and only if given any p conv(s), there exists a v j such that p v j p v j 2 p / conv(s) if and only if there exists a p conv(s) such that p v j < p v j, j.

7 The Convex Hull Problem A quick refresher: Given a point p R m, we would like to determine whether or not the point lies within the convex hull of a finite set of m-dimensional points S. Theorem (The Distance Duality Theorem) Let S = {v 1, v 2,..., v n } R m, p R m. 1 p conv(s) if and only if given any p conv(s), there exists a v j such that p v j p v j 2 p / conv(s) if and only if there exists a p conv(s) such that p v j < p v j, j. Definition We call p a witness if it satisfies p v i < p v i, i.

8 Skeleton Implementation Data: Given a vector p R m, a matrix S R mxn, and a fixed tolerance ɛ Result: A vector α such that p = S α, with p p < ɛ initialization of p 0 and α 0 ; while p p > ɛ do for i 1 to n total vertices do if p v i p v i then choose v i to be a pivot; end end calculate step size β = (p p k )T (v i p k ) v i p k ; update α k+1 = (1 β)α k + βα i ; update p k+1 = Sα k+1; end

9 Choosing the Pivot Naive Implementation for i 1 to n total vertices do if p v i p v i then choose first v i that satisfies condition to be a pivot; break; end end Algorithm 1: Blind Triangle Algorithm

10 Choosing the Pivot Naive Implementation Figure : 5 Point Convex Set in 2D Space (40 iterations) Figure : 8 Point Convex Set in 3D Space (245 iterations)

11 Choosing the Pivot A Better Implementation for i 1 to n total vertices do if p v i p v i then find angles δ i = pp k v i; end find δ j := inf{δ 1, δ 2,...}; choose pivot v j from corresponding angle δ j ; end Algorithm 2: Triangle Algorithm with Angle Checking

12 Choosing the Pivot A Better Implementation Figure : 5 Point Convex Set in 2D Space (35 iterations) Figure : 8 Point Convex Set in 3D Space (95 iterations)

13 Auxiliary Pivot Points Mid-Point Method for i 1 to n total vertices do if p v i p v i then find angles δ i = pp k v i; end find δ j := inf{δ 1, δ 2,...}; choose pivot v j from corresponding angle δ j ; end if two pivots v i and v ii are chosen in consecutive cycles then increment counter; if counter > γ then define new auxiliary point v n+1 := v i +v ii 2 ; S = [S v n+1 ]; end end Algorithm 3: Mid-Point Auxiliary Points

14 Auxiliary Pivot Points Mid-Point Method Figure : 5 Point Convex Set in 2D Space (8 iterations) Figure : 8 Point Convex Set in 3D Space (17 iterations)

15 Triangle Algorithm More Examples Figure : 100 Point Convex Set in 2D Space (7 iterations) Figure : 100 Point Convex Set in 3D Space (15 iterations)

16 Triangle Algorithm More Examples Figure : Half-Million Point Convex Set in 3D Space (5 iterations)

17 Solving a System of Equations Nonnegative Case

18 Solving a System of Equations Nonnegative Case Given A R n n and invertible, suppose Ax = b has a nonnegative solution x = A 1 b 0. Then we can generate an ɛ- approximate solution x 0 such that Ax 0 b < ɛ by approximating 0 conv([a -b]).

19 Solving a System of Equations Nonnegative Case Given A R n n and invertible, suppose Ax = b has a nonnegative solution x = A 1 b 0. Then we can generate an ɛ- approximate solution x 0 such that Ax 0 b < ɛ by approximating 0 conv([a -b]). a 1,1 a 1,2 a 1,n b 1 a 2,1 a 2,2 a 2,n b a n,1 a n,2 a n,n b n αi = 1, α i 0 α 1 α 2. α n α n =. 0 n

20 Solving a System of Equations Nonnegative Case Consider the following examples: 3x 1 + x 2 = 4 x x x 3 = 7 -x x 3 = x 1 x 2 x 3 4 = 7 2

21 Solving a System of Equations Nonnegative Case x x 2 = 7, x 3 2 Comparisons Method Iterations Triangle Algorithm 1 Jacobi 11 Gauss-Seidel 7 Successive Over-Relaxation (SOR) 7 Note: The initial guess used for Jacobi, Gauss-Seidel, and SOR was x 0 = ( 4 3, 7 3, 2 3 )T. The optimal relaxation factor for SOR was approximately ω = True solution is x true = (1, 1, 1) T.

22 Solving a System of Equations Nonnegative Case Note: x x x x 4 = , x 5 8 Comparisons Method Iterations Triangle Algorithm 1 Jacobi 46 Gauss-Seidel 26 Successive Over-Relaxation (SOR) 22 The initial guess used for Jacobi, Gauss-Seidel, and SOR was x 0 = (1, 2, 7 5, 1, 9 5 )T. The optimal relaxation factor for SOR was approximately ω = True solution is x true = (1, 2, 1, 1, 1) T.

23 Solving a System of Equations Issues and Performance Computational Complexity (Each Iteration) Method Complexity Triangle Algorithm O(nm) Gaussian Elimination O(n 3 ) Jacobi O(n 2 ) Gauss-Seidel O(n 2 ) Successive Over-Relaxation (SOR) O(n 2 ) Performance in high dimensions Comparison with Krylov Methods Auxilliary Point Methods

24 Further Work More General System Solver

25 Further Work More General System Solver Given A R n n and invertible, suppose Ax = b, but we do not know anything about x. Apply a change of variables: Let e = (1, 1,..., 1) T R m. Then t 0 such that if x is a solution to: A(x te) = b

26 Further Work More General System Solver Given A R n n and invertible, suppose Ax = b, but we do not know anything about x. Apply a change of variables: Let e = (1, 1,..., 1) T R m. Then t 0 such that if x is a solution to: A(x te) = b Then x 0. Define u := Ae, then is solvable. Ax = b + tu, x 0,

27 Further Work Incremental Triangle Algorithm Data: Given p = A α - α n+1 b conv(s) Result: A vector α such that p = S α, with p p < ɛ set x 0 = α α n+1. Define τ 0 according to: E(τ 0 ) = Ax 0 (b + τ 0 u) = min{ Ax 0 (b + tu) : t t 0 } replace t 0 with τ 0 ; if E(t 0 ) ɛ 0 then set x 0 = (x 0 t 0 e), stop; end Call Triangle Algorithm where S = {a 1, a 2,..., a n, (b + t 0 u)}; p (t 0 ) = p β n+1 t 0 u, where p = Aβ β n+1 b conv(s); Update p with p ; Update α with β; Update α n+1 with β n+1 ; Algorithm 4: Incremental Triangle Algorithm

28 Further Work Incremental Triangle Algorithm Auxiliary Pivot Methods to Improve Performance Eigenvalue Problems

29 Conclusion References and Acknowledgements B. Kalantari. A Characterization Theorem and an Algorithm for a Convex Hull Problem. B. Kalantari. Finding a Lost Treasure in Convex Hull of Points From Known Distances. 4th Canadian Conference on Computational Geometry, B. Kalantari. Solving Linear System of Equations Via A Convex Hull Algorithm. 29 Oct, Thank you to all those who made my research possible: Bahman Kalantari Meng Li Yan Wang DIMACS REU Program