15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm

Size: px

Start display at page:

Download "15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm"

Maximilian Henry
5 years ago
Views:

1 508J/65J Itroductio to Mathematical Programmig Lecture : Primal Barrier Iterior Poit Algorithm

2 Outlie Barrier Methods Slide The Cetral Path 3 Approximatig the Cetral Path 4 The Primal Barrier Algorithm 5 Correctess ad Complexity Barrier methods mi f(x) st g j (x) 0, h i (x) = 0, j =,,p i =,,m Slide S = {x g j (x) < 0, j =,,p, h i (x) = 0, i =,,m} Strategy A barrier fuctio G(x) is a cotious fuctio with the property that is approaches as oe of g j (x) approaches 0 from egative values Slide 3 Examples: G(x) = p p log( g j (x)), G(x) = j= j= g j (x) Slide 4 Cosider a sequece of : 0 < + < ad 0 Cosider the problem x { = argmi x S f(x) + } G(x) Theorem Every limit poit x geerated by a barrier method is a global miimum of the origial costraied problem

3 cetral path x(00) x* x() x(0) x(0) aalytic ceter c Primal path-followig IPMs for LO Barrier problem: (P) mi c x (D) max b p st Ax = b st A p + s = c x 0 s 0 mi B (x) = c x log x j st Ax = b j= Slide 5 Miimizer: x() 3 Cetral Path As varies, miimizers x() form the cetral path lim 0 x() exists ad is a optimal solutio x to the iitial LP For =, x( ) is called the aalytic ceter mi log x j st j= Ax = b Slide 6 Slide 7 3 Example mi x st x + x + x 3 = x, x, x 3 0 Slide 8

4 (/, 0, /) the aalytic ceter of Q Q x 3 P (/3, /3, /3) the aalytic ceter of P x x the cetral path { Q = x x = (x, 0, x 3 ), x + x 3 =, x 0}, set of optimal solutios to origial LP The aalytic ceter of Q is (/, 0, /) mi x log x log x log x 3 st x + x + x 3 = mi x log x log x log( x x ) x () = x () x () = x 3 () = x () The aalytic ceter: (/3, /3, /3) Slide 9 3 Solutio of Cetral Path Barrier problem for dual: max p b + log s j j= st p A + s = c Slide 0 Solutio (KKT): Ax() = b x() 0 A p() + s() = c s() 0 X()S()e = e 3

5 Theorem: If x, p, ad s satisfy optimality coditios, the they are optimal solutios to problems primal ad dual barrier problems Slide Goal: Solve barrier problem mi B (x) = c x log x j st Ax = b j= 4 Approximatig the cetral path B (x) = c i x i xi B (x) = x x i B (x) = 0, i = j x i x j i Slide Give a vector x > 0: Slide 3 B (x) B (x + d) B (x) + d i + x i i= B (x) di d j x i x i,j= j = B (x) + (c e X )d + d X d X = diag(x,,x ) Slide 4 Approximatig problem: Solutio (from Lagrage): mi (c e X )d + d X d st Ad = 0 c X e + X d A p = 0 Ad = 0 Slide 5 System of m+ liear equatios, with m+ uows (d j, j =,,, ad p i, i =,,m) 4

6 cetral path x(00) x* x() x(0) x(0) aalytic ceter c Solutio: ( )( ) d() = I X A (AX A ) A xe X c p() = (AX A ) A(X c xe) 4 The Newto coectio d() is the Newto directio; process of calculatig this directio is called a Newto step Slide 6 Startig with x, the ew primal solutio is x + d() ( ) The correspodig dual solutio becomes (p, s) = p(), c A p() We the decrease to = α, 0 < α < 4 Geometric Iterpretatio Tae oe Newto step so that x would be close to x() Slide 7 Measure of closeess XSe e β, 0 < β <, X = diag(x,, x ) S = diag(s,, s ) As 0, the complemetarity slacess coditio will be satisfied Slide 8 5

7 5 The Primal Barrier Algorithm Iput (a) (A, b, c); A has full row ra; (b) x 0 > 0, s 0 > 0, p 0 ; (c) optimality tolerace ǫ > 0; Slide 9 (d) 0, ad α, where 0 < α < Slide 0 (Iitializatio) Start with some primal ad dual feasible x 0 > 0, s 0 > 0, p 0, ad set = 0 (Optimality test) If (s ) x < ǫ stop; else go to Step 3 3 Let X = diag(x,,x ), + = α 4 (Computatio of directios) Solve the liear system Slide 5 (Update of solutios) Let 6 Let := + ad go to Step 6 Correctess + X + X d A p = e c Ad = 0 x + = x + d, + p = p, s + = c A p β β Slide Theorem Give α = β +, β <, (x 0, s 0, p 0 ), (x 0 > 0, s 0 > 0): X 0 S 0 e e β 0 The, after β + (s 0 ) x 0 ( + β) K = log β β ǫ( β) iteratios, (x K, s K, p K ) is foud: (s K ) x K ǫ 6

8 6 Proof Slide 3 Claim (by iductio): X S e e β For = 0 we have assumed it Assume it holds for ; X S e e = X α S e e + ( ) = X α S e e α X S e e + α β + α α α = β By left-multiplyig the first equatio by d ( ) + d X d + X d = e c + α α + We ext show that X d β <, where d = x x d solves + X + X d A p = e c, Ad = 0 e α e X d hece, X d β < = d X d ( ) X = e c d + ( ) = X e (s + A p ) d + ( ) X = e s d + ( ) = X S e e X d + e X d X S + e β X d 7

9 We ext show that x + ad (p +, s + ) are primal ad dual feasible Sice Ad = 0, we have + Ax = b because X d < by costructio ad x + = x + d = X (e + X d) > 0, A p + + s + = c, s + = c A p + = + X (e X d) > 0, because X d < ( ) + d j x j = x j +, + + x j ( ) d j s j = x x j j Therefore, ( ) ( ) + x + + d j d j + j s j = x + j + x j x j x j ( ) d j = x j D = diag(d,, d ), u = u i i Note that u u X + S + e e = X D e + X D e ad hece the iductio is complete Sice at every iteratio = d X d = X X S e e β = e X D e = e DX De ( β) = β, β x j s j β d ( β) (s ) x ( + β) 8

10 ( ) β β = α β β 0 = β + 0 e β+ 0 After 0 β + ( + β) β + (s 0 ) 0 x ( + β) log log = K β β ǫ β β ǫ( β) iteratios, the primal barrier algorithm fids primal ad dual solutios x K, (p K, s K ), that have duality gap (s K ) x K less tha or equal to ǫ 7 Complexity Wor per iteratio ivolves solvig a liear system with m + equatios i m + uows Give that m, the wor per iteratio is O( 3 ) Slide 4 ǫ 0 = (s 0 ) x 0 : iitial duality gap Algorithm eeds ( ǫ ) 0 O log ǫ iteratios to reduce the duality gap from ǫ 0 to ǫ, with O( 3 ) arithmetic operatios per iteratio 9

11 MIT OpeCourseWare 65J / 508J Itroductio to Mathematical Programmig Fall 009 For iformatio about citig these materials or our Terms of Use, visit:

15.093J Optimization Methods. Lecture 22: Barrier Interior Point Algorithms

15.093J Optimization Methods. Lecture 22: Barrier Interior Point Algorithms 1593J Otimizatio Methods Lecture : Barrier Iterior Poit Algorithms 1 Outlie 1 Barrier Methods Slide 1 The Cetral Path 3 Aroximatig the Cetral Path 4 The Primal Barrier Algorithm 5 The Primal-Dual Barrier