An Augmented Lagrangan Coordnaton-Decomposton Agorthm for Sovng Dstrbuted Non-Convex Programs Jean-Hubert Hours and Con N. Jones Abstract A nove augmented Lagrangan method for sovng non-convex programs wth nonnear cost and constrant coupngs n a dstrbuted framework s presented. The proposed decomposton agorthm s made of two ayers: The outer eve s a standard mutper method wth penaty on the nonnear equaty constrants, whe the nner eve conssts of a bockcoordnate descent BCD) scheme. Based on standard resuts on mutper methods and recent resuts on proxma reguarsed BCD technques, t s proven that the method converges to a KKT pont of the non-convex nonnear program under a sem-agebracty assumpton. Effcacy of the agorthm s demonstrated on a numerca exampe. I. INTRODUCTION When deang wth a arge-scae system, such as a power grd for nstance, sub-systems are couped n a compex manner through ther dynamcs, mpyng that oca contro actons may have a major mpact throughout the whoe network. Impementng NMPC controers for such systems may resut n arge-scae non-separabe non-convex nonnear programs NLP). Sovng such programs onne n a centrased fashon s a chaengng task from a computatona pont of vew. Moreover, autonomy of the agents s key to be hampered by such a centrased procedure. Therefore, dstrbuted MPC s currenty rasng much nterest, as an advanced contro strategy for decentrasng computatons, thus reducng the computatona burden and enabng autonomy of agents, whch then ony need to sove a oca NLP [8]. Potenta appcatons of dstrbuted NMPC abound, from networked systems contro nterconnected power pants) to coaboratve contro formaton fyng). Decomposng a convex NLP s generay performed by appyng Lagrangan decomposton methods [4]. A crtca requrement for ths strategy s strong duaty, whch s guaranteed by Sater s condton n the convex case. However, such an assumpton rarey hods n a non-convex settng. Nevertheess, n an augmented Lagrangan settng, t has been shown that the duaty gap can be ocay removed by usng approprate penaty functons [10]. Moreover, augmented Lagrangan methods turn out to be much more effcent than standard penaty approaches from a computatona pont of vew, as the rsk of -condtonng assocated wth arge penaty parameters s reduced by the fast convergence of the dua teraton. However, the resutng quadratc penaty term s not separabe even f the coupng constrants are. Severa approaches have been expored to remedy ths ssue [7]. For nstance, n [,1], Jean-Hubert Hours and Con N. Jones are wth the Laboratore d Automatque, Écoe Poytechnque Fédérae de Lausanne, Swtzerand. {jean-hubert.hours, con.jones}@epf.ch a oca convexfcaton procedure by means of proxma reguarsaton s anaysed. A more recent approach to the decomposton of non-convex programs s the sequenta programmng scheme presented n [9], whch conssts n teratve convex approxmatons and decompostons. Our decomposton strategy s not specfcay targeted at separabe NLPs, as objectve and constrants coupngs can be addressed by the proposed BCD scheme. It aso dffers from [9] n that ony oca convexfcaton s requred to guarantee convergence. Our approach s an augmented Lagrangan method wth parta constrant penasaton on the nonnear equaty constrants. It s a two-eve optmsaton scheme, whose outer ayer s a oop on the Lagrange mutpers estmates assocated wth the nonnear equaty constrants. The nner eve conssts n approxmatey sovng the prma probem to a crtca pont. Ths s performed va an nexact proxma reguarsed BCD scheme. By means of recent resuts on non-convex proxma reguarsed BCD methods [1], we prove convergence of the nner oop to a crtca pont of the non-convex nner probem under the assumpton that the NLP s sem-agebrac. Ths s the man novety of our approach, snce, unt recenty, BCD type methods were thought to ony have convergence guarantees under the very restrctve assumpton that the NLP s convex. Some studes have been conducted n the non-convex case, yet the convergence resuts are qute weak compared to [1]. The nner oop s the key step of the agorthm, whch enabes dstrbuted computatons. In genera, as the augmented Lagrangan s not separabe, nner teratons cannot be fuy paraesed, but some degree of paraesm can be acheved f the nterconnecton graph s sparse [4]. Fnay, convergence of our two-eve optmsaton technque to a KKT pont of the orgna NLP s proven under standard assumptons. Eventuay, the whoe agorthm conssts n teratvey sovng decouped Quadratc Programs QP) and updatng mutpers estmates n a coordnaton step. In Secton III, the genera framework of our agorthm s presented. In Sectons IV and V, the agorthm s descrbed and ts convergence anaysed. Fnay, a numerca exampe s presented n Secton VI. II. BACKGROUND Defnton 1 Norma cone to a convex set): Let be a convex set n R n and x. The norma cone to at x s the set n o N x) := v R n 8x, v > x x) appe 0. 1)
Lemma 1 Descent Lemma): Let f C, R) twce contnuousy dfferentabe n ), where s a convex compact set n R n. Let x, y. fy) appe fx)+rfx) > y x)+ r f 1 ky xk ) where n o r f 1 = max r fx) x. 3) Gven M R n n, kmk denotes the nduced matrx -norm. Defnton Reguar pont): Let h C 1 R n, R m ). A pont x R n such that hx) =0 s caed reguar f the gradents rh 1 x),...,rh m x) are neary ndependent. Defnton 3 Crtca pont): Let f be a proper ower semcontnuous functon. A necessary condton for x to be a mnmser of f s that 0 @fx ), 4) where @fx ) s the sub-dfferenta of f at x [11]. Ponts satsfyng 4) are caed crtca ponts. The ndcator functon of a cosed subset S of R n s denoted S and s defned as 0 f x S Sx) = 5) +1 f x/ S. Lemma Sub-dfferenta of ndcator functon [11]): Gven a convex set C, for a x C, @ C x) =N C x). 6) III. PROBLEM FORMULATION We consder NLPs of the foowng form: mnmse J z )+Qz 1,...,z N ) z 1,...,z N s.t. =1 F z )=0, {1,...,N}, Gz 1,...,z N )=0, z Z, {1,...,N}, 7) where z R n for {1,...,N} and Z are poytopes n R n. The term Qz 1,...,z N ) s a cost coupng term and Gz 1,...,z N ) R p a constrant coupng term. They mode the dfferent knds of systems nteractons, whch may appear n a dstrbuted NMPC context. For the remander of the paper, we aso defne the vector z := z 1 >,...,z N > > R n, wth dmenson n:= P N =1 n. For {1,...,N}, as Z s poytopc, there exsts A R q n and b R q such that Z := xr n A x appe b. One can thus ntroduce the poytope Z R n by Z := x R n Ax appe b, where AR q n, br q and q := P N =1 q. By posng m := P N =1 m, we aso defne Hz) := F 1 z 1 ) >,...,F N z N ) >,Gz 1,...,z N ) > > R r, 8) where r := m + p. The dstrbuted objectve s Jz) := J z )+Qz 1,...,z N ), 9) =1 so that NLP 7) can be rewrtten mnmse Jz) 10) z s.t. Hz) =0, z Z. Assumpton 1 Smoothness and sem-agebracty): N The functons J, Q, F N =1 and G are twce =1 contnuousy dfferentabe and sem-agebrac. Remark 1: The set of sem-agebrac functons s cosed wth respect to sum, product and composton. The ndcator functon of a sem-agebrac set s a sem-agebrac functon. Sem-agebracty s needed n order to appy the resuts of [1], whch are vad for functons satsfyng the Kurdyka- Lojasewcz KL) property, whch s the case for a rea sem-agebrac functons [5]. From a contro perspectve, ths means that our resuts are vad for poynoma systems subject to poynoma constrants and objectves. Assumpton : The NLP 10) admts an soated KKT pont z ) >, µ ) >, ) > ) > R n+r+q, whch satsfes z s reguar, z ) >, µ ) >, ) > ) > satsfes the second order optmaty condton, that s p > r Lz,µ, )p>0 for a p R n such that rhz ) > p =0, A I p =0, 11) where I {1,...,q} s the set of ndces of actve nequaty constrants at z. For a I, > 0. IV. OUTER LOOP: PARTIALLY AUGMENTED LAGRANGIAN Gven a penaty parameter >0, the augmented Lagrangan assocated wth probem 10) s defned as L z,µ) :=Jz)+µ > Hz)+ khz)k, 1) where µ R r. Ony equaty constrants Hz) =0are penased. The poytopc nequates are kept as constrants on the augmented Lagrangan. A. Agorthm descrpton The outer oop s smar to a dua ascent on the mutper estmates µ k, aong wth teratve updates of the penaty parameter k. At each outer teraton k, the prma probem mnmse L k z,µ k ) 13) zz s soved to a gven eve of accuracy k > 0. The outer oop s presented n Agorthm 1 beow. The notaton dx, S) stands for the dstance functon between a pont x and a set S n R n and s defned by dx, S) := nf zs kx zk. 14) At every outer teraton, a pont z s found, whch satsfes nexact optmaty condtons. Then, the dua estmate µ s updated, the penaty parameter ncreased and the toerance
Agorthm 1 Method of mutpers wth parta constrant penasaton Input: Objectve J, equaty constrant H, poytope Z, >1 and fna toerance on nonnear equaty constrants >0. Inta guess for optmser z 0 Z, nta guess for mutper estmates µ 0 R r, nta penaty parameter 0 > 1, nta toerance on optmaty condtons 0 > 0. Intazaton: z z 0, µ µ 0, 0, 0. repeat Fnd z Z such that d0, rl z,µ)+n Z z)) appe usng z as a warm-start z z µ µ + Hz),, unt khz)k appe on the frst order optmaty condtons s reduced. The man tunng varabes are the nta vaue of the penaty parameter 0 and the growth coeffcent. B. Convergence anayss We prove that the augmented Lagrangan teratons are ocay convergent to the soated KKT pont z ) >, µ ) >, ) > > satsfyng Assumpton. Our anayss s based on the resuts of [3]. We start by showng that the nner probem s we-defned, whch means that for any optmaty toerance >0, there exsts a pont z, as n Agorthm 1, satsfyng d0, rl z,µ)+n Z z)) appe, under approprate condtons on and µ. Lemma 3 Exstence of an nner crtca pont): There exsts >0, >0 and appe>0 such that 8 µ, S:= µ, R m+1 kµ µ k appe,, 15) the probem mnmse L z,µ) 16) s.t. z Z kz z k <appe. has a unque mnmser. Moreover, for a >0 and for a µ, S, there exsts z R n such that d0, rl z,µ)+n Z z )) appe. 17) Proof: The frst part of the statement s a drect consequence of Proposton.11 n [3], as the second order optmaty condton s satsfed Assumpton ). From Defnton 3, takng µ, S, ths mpes that we can fnd z Z\Bz,appe) such that d0, rl z,µ)+n Z\Bz,appe) z)) appe for a >0, where Bz,appe) s the open ba of radus appe centered at z. As r Bz,appe) \ r Z6= ;, where r stands for the reatve nteror, N Z\Bz,appe) z) =N Z z) \N Bz,appe) z), so that d0, rl z,µ)+n Z z)) appe d0, rl z,µ) + N Z\Bz,appe) z)) appe. 18) The end of the proof foows by takng z := z. The next Lemma s a reformuaton of the nexact optmaty condtons d0, rl z,µ)+n Z z)) appe. Lemma 4 Inexact optmaty condtons): Let z R n, f C 1 R n, R) and Z be a convex set n R n. The foowng equvaence hods: d0, rfz)+n Z z)) appe,9v R n such that kvk appe 19) rfz) N Z z)+v. Proof: Ths drecty foows from the defnton of the dstance as an nfmum. As z ) >, µ ) >, ) > > s an soated KKT pont, there exsts >0 such that z s the ony crtca pont of 10) n Bz, ). From the statement of Agorthm 1, the penaty parameter s guaranteed to ncrease to nfnty and the optmaty toerance to converge to zero. The foowng convergence theorem s vad f the terates z k stay n the ba Bz, mn {, appe}) for a arge enough k. As notced n [3], t s generay the case n practce, as warm-startng the nner probem 16) on the prevous souton eads the terates to stay around the same oca mnmum z. Thus one can reasonaby assume that z k Bz, mn {, appe}). Theorem 1 Loca convergence to a KKT pont): Let {z k } and {µ k } be sequences n R n such that zk Z\c B z, mn {appe, } 0) rl k z k,µ k ) N Z z k )+d k where { k } s ncreasng and k! + 1, {µ k } and { k }, assocated wth N Z z k ), are bounded, kd k k! 0. Assume that a mt ponts of {z k } are reguar. We then have that z k! z and µ k! µ, where µ k := µ k + k Hz k ), z and µ are defned n Assumpton. Proof: As Z\c B z, mn {appe, } s compact, by Weerstrass theorem, there exsts an ncreasng mappng : N! N and a pont z 0 Z\c B z, mn {appe, } such that z k) converges to z 0, whch s reguar by assumpton. We aso know by assumpton that 9v k) N Z z k) ), 0=rL k) z k),µ k) )+v k) + d k). 1) By defnton of the norma cone to Z at z k), ths means that rl z k) k),µ k) )+A > k) + d k) =0 9 k) 0, We thus have k) 0, > k) b Az k) )=0. ) rjz k) )+rhz k) ) > µ k) + A > k) = d k). 3) As z 0 s reguar, there exsts K N + such that rhz k) ) s fu-rank for a k K. Subsequenty, for a k K, µ k) = rhz k) )rhz k) ) > 1 rhz k) ) d k) rjz k) ) A > k). 4)
As z k)! z 0, by contnuty of > k) b Az k) =0, there exsts 0 0 such that k)! 0, 0 > b Az 0 )=0. 5) As d k)! 0, by contnuty of rh, ths mpes that µ k)! µ 0, where µ 0 := rhz 0 )rhz 0 ) > 1 rhz 0 ) rjz 0 ) A > 0. 6) By takng mt n 3), we obtan rjz 0 )+rhz 0 ) > µ 0 + A > 0 =0 0 0, 0 ) > b Az 0 )=0. 7) Feasbty of z 0 mmedatey foows from the convergence of µ k) and the fact that k)! +1. Thus we have Hz 0 ) = 0. Ths mpes that z 0 >, µ 0 > 0 > >, satsfes the KKT condtons. As z 0 Z \c B z, mn {appe, }, n whch z s the unque KKT pont by assumpton, one can cam that z 0 = z. Takng the KKT condtons on z and z 0, t foows that rhz ) > µ + A > I I = rhz0 ) > µ 0 + A > I 0 0 I 0, 8) where I and I 0 are the sets of ndces of actve constrants at z and z 0 respectvey. Obvousy, as z = z 0, I = I 0. Thus rhz ) > µ µ 0 )+A > I I I0)=0. 9) As z s reguar, we fnay obtan that µ 0 = µ. As a mt ponts of {z k } converge to z, one can concude that z k! z and µ k! µ. Remark : The convergence resut of Theorem 1 s oca. Yet convergence can be gobased meanng that the assumpton accordng to whch the terates e n a ba centered at z can be removed) by appyng the dua update scheme proposed n [6]. V. INNER LOOP: INEXACT PROXIMAL REGULARISED BCD The non-convex nner probem 13) s soved n a dstrbuted manner. An nexact BCD scheme s proposed, whch s based on the abstract resut of [1]. The man dea s to perform oca convex approxmatons of each agent s cost and compute descent updates from t. Contrary to the approach of [9], where spttng s, n some sense, apped after convexfcaton, we show that convexfcaton can be performed after spttng, under the assumpton that the KL property s satsfed. A. Agorthm descrpton The proposed agorthm s a proxma reguarsed nexact BCD, whch s based on Theorem 6. n [1], provdng genera propertes ensurng goba convergence to a crtca pont of the nonsmooth objectve, assumng that t satsfed the KL property. The nner mnmsaton method s presented n Agorthm beow. The postve rea numbers N =1 are the reguarsaton parameters. We assume that Matrces B 8 {1,...,N}, 9, + > 0 such that < + 8 N,, + 30). are postve defnte and need to be chosen Agorthm Inexact proxma reguarsed BCD Input: Augmented Lagrangan L., µ), poytopes {Z } N =1 and termnaton toerance >0. Intazaton: z1 0 Z 1,...,zN 0 Z N 0. whe z > do 1 for {1,...,N} do argmn zz r z L z1 +1,...,zN,µ)> z z ) + 1 z z )> B + I n )z z ) end for +1 end whe carefuy n order to guarantee convergence of Agorthm, as expaned n paragraph V-B. Thus, Agorthm conssts n sovng convex QPs sequentay. Under approprate assumptons [4], computatons can be party paraesed. Moreover, computatona effcacy can be mproved by warm-startng. B. Convergence anayss When consderng the mnmsaton of functons f : R n1... R n N! R [{+1} satsfyng the KL property [5] and havng the foowng structure fz) = f z )+Pz 1,...,z N ), 31) =1 where f are proper ower semcontnuous and P s twce contnuousy dfferentabe, two key assumptons need to be satsfed by the terates z of the nexact BCD scheme n order to ensure goba convergence [1]: a suffcent decrease property and the reatve error condton. The suffcent decrease property asserts that 8 {1,...,N}, 8 N, f )+Pz1 +1,...,,...,zN )+ appe f z )+P 1,..., 1,z,...,z N ). 3) The reatve error condton states that 8 1,...,N, 9b > 0 such that 8 N, 9 @f ), + r z P z1 +1,...,,z+1,...,z N ) appe b z. 33) Ths means that at every teraton one can fnd a vector n the sub-dfferenta of f computed at the current terate, whch s bounded by the momentum of the sequence z. It z
s actuay a key step n order to appy the KL nequaty, as done n [1]. For carty, we state Theorem 6. n [1]. Theorem Convergence of nexact BCD): Let f be a proper ower sem-contnuous functon wth structure 31) satsfyng the KL property and bounded from beow. Let > 0 such that appe for a {1,...,N} and N. Let z be a sequence satsfyng 3) and 33). If z s bounded, then z converges to a crtca pont of f. Remark 3: Note that Z + L., µ) has the structure 31). Our convergence anayss then many conssts n verfyng that the terates generated by Agorthm satsfy 3) and 33). The foowng defntons are necessary for the remander of the proof: 8 {1,...,N}, 8 N, Sz1 +1,..., 1,z,z+1,...,z N ):= µ > GGz1 +1,..., 1,z,z+1,...,z N ) + Gz+1 1,..., 1,z,z +1,...,z N ), Rz ):=Qz1 +1,..., 1,z,z+1,...,z N ) + Sz1 +1,..., 1,z,z+1,...,z N ), z ):=µ > F z )+ kf z )k, L z ):=J z )+ z )+R z ), z ):=J z )+ z )+ Z z ), z ):=L z )+ Z z ), 34) where µ G R r s the subvector of mutpers assocated wth the equaty constrant Gz 1,...,z N )=0. Lemma 5 Bound on hessan norm): where 8 {1,...,N}, 8 N, C := r J Z 1 + r r L Z 1 appe C, 35) Z 1 + max {1,...,N} r S + Q) Z 1. 36) Proof: The proof drecty foows from the defntons 34) and the fact that the functons nvoved n 34) are twce contnuousy dfferentabe over compact sets Z. Assumpton 3: The matrces B can be chosen so that B C I n 0 and C I n B 0, for a {1,...,N} and a N. Remark 4: Such an assumpton requres knowedge of the Lpschtz constant of the gradent of the augmented Lagrangan. However, a smpe backtrackng procedure may be apped n practce. Lemma 6 Suffcent decrease): For a {1,...,N} and a N, )+ z appe z ). 37) Proof: By defnton of n Agorthm, 8z Z, rl z ) > z ) + 1 z+1 z ) > B + I n ) z ) apperl z ) > z z )+ 1 z z ) > B + I n )z z ), whch by substtutng z wth z Z mpes that rl z ) > z ) 38) + 1 z+1 z ) > B + I n ) z ) appe 0. 39) However, by appyng the descent Lemma, one gets L ) appe L z)+rl z) > z )+ C z 40) Combnng ths ast nequaty wth nequaty 39), one obtans L ) appe L z ) + 1 z+1 z) > C )I n B)z +1 whch, by Assumpton 3, mpes that L )+ However, Z z )= Z z )=0. Thus, )+ Lemma 7 Reatve error condton): 8 {1,...,N}, 8 N, z z ), 41) appe L z ). 4) appe z ). 43) 9 @ ), + rq + S) 1,...,,z +1,...,z N ) appe b z, 44) where b := 3C + + and + s defned n 30). Proof: Wrtng the necessary optmaty condtons for the th QP of Agorthm, one obtans 0 rl z )+ B + I n z )+N Z ). 45) Thus there exsts w +1 N Z ) such that 0=w +1 + rl z)+b + I n ) z) = w +1 + rl )+rl z) rl ) +B + I n ) z). 46) The ast equaty yeds w +1 + rj )+rf ) > µ + rf ) > F ) + r Q + S)z1 +1,...,,z+1,...,z N ) = rl ) rl z)+b + I n )z ), 47).
from whch t mmedatey foows 9 @ ), + r Q + S)z1 +1,..., 1,z+1,z+1,...,z N ) = rl ) rl z)+b + I n )z ). 48) However, as L C Z, R), one gets + r Q + S) 1,...,,z +1,...,z N ) appe C + B + ) z appe 3C + + ) z+1 z. 49) Now that the two man ngredents are proven, one can state the theorem guaranteeng convergence of the terates of Agorthm to a crtca pont of Z + L., µ). Theorem 3 Convergence of Agorthm ): The sequence z generated by Agorthm converges to a pont z satsfyng 0 rl z,µ)+n Z z ) 50) or equvaenty d0, rl z,µ)+n Z z )) = 0. Proof: The sequence z s obvousy bounded, as the terates are constraned to stay n the poytope Z. The functon Z + L., µ) s bounded from beow. Moreover, by Assumpton 1, Z + L., µ) satsfes the KL property for a µ R m and >0. The sequence z satsfes the condtons 3) and 33). The parameter of Theorem can be taken as mn {1,...,N}. Convergence to a crtca pont z of Z + L., µ) then foows by appyng Theorem. We can then guarantee that, gven >0, by takng a suffcenty arge number of nner teratons, Agorthm converges to a pont z Z satsfyng d0, rl z,µ)+n Z z)) appe, as needed by Agorthm 1. VI. NUMERICAL EXAMPLE Our agorthm s tested on the foowng cass of nonconvex NLPs: 1 mnmse x > H x + x > H,+1 x +1 x 1,...,x N s.t. =1 kx k = a, 1,...,N =1 b appe x,j appe b, 1,...,N,j 1,...,d, 51) where N s the number of agents and d ther dmenson. Matrces H and H,+1 are ndefnte. When appyng Agorthm to mnmse the augmented Lagrangan assocated wth 51), the update of agent ony depends on agents 1 and +1, whch mpes that, after approprate re-orderng [4], every nner teraton actuay conssts of two sequenta update steps where N/ QPs are soved n parae. We fx d =3, N = 0, a = p R and b =0.6R wth R =. Matrces H and H,+1 are randomy generated. For a fxed number of outer teratons, we graduay ncrease the number of nner teratons and count the number of random probems, on whch the agorthm ends up wthn a fxed feasbty Percentage of postves 100 80 60 40 0 0 10 1 10 10 3 Tota number of teratons 10 4 Fg. 1. Percentage of postve probems for a fxed feasbty toerance on the nonnear equaty constrants. Feasbty margns 10 3 n bue, 10 4 n back and 10 6 n red. toerance wth respect to the nonnear equaty constrants kx k = a. Statstcs are reported for 500 random NLPs of the type 51). The nta penaty parameter s set to 0 =0.1 and the growth coeffcent = 100. The hessan matrces of Agorthm are set to 30 I d. The agorthm s ntased on random prma and dua nta guesses, feasbe wth respect to the nequaty constrants. Resuts for a feasbty toerances of 10 3, 10 4 and 10 6 are presented n Fgure 1. It ceary appears that obtanng an accurate feasbty margn requres a arge number of teratons, yet a reasonabe feasbty 10 3 ) can be obtaned wth approxmatey hundred teratons. REFERENCES [1] H. Attouch, J. Bote, and B.F. Svater. Convergence of descent methods for sem-agebrac and tame probems: proxma agorthms, forward-backward spttng and reguarsed Gauss-Sede methods. Mathematca Programmng, 137:91 19, 013. [] D.P. Bertsekas. Convexfcaton procedures and decomposton methods for nonconvex optmsaton probems. Journa of Optmzaton Theory and Appcatons, 9:169 197, 1979. [3] D.P. Bertsekas. Constraned optmsaton and Lagrange mutper methods. Athena Scentfc, 198. [4] D.P. Bertsekas and J.N. Tstsks. Parae and dstrbuted computaton: numerca methods. Athena Scentfc, Bemont, MA, 1997. [5] J. Bote, A. Dands, and A. Lews. The Lojasewcz nequaty for nonsmooth subanaytc functons wth appcatons to subgradent dynamca systems. SIAM Journa on Optmzaton, 17:105 13, 006. [6] A. R. Conn, G. I. M. Goud, and P. L. Tont. A gobay convergent augmented Lagrangan agorthm for optmsaton wth genera constrants and smpe bounds. SIAM Journa on Numerca Anayss, 8):545 57, 1991. [7] A. Hamd and S.K. Mshra. Decomposton methods based on augmented Lagrangan: a survey. In Topcs n nonconvex optmzaton. Mshra, S.K., 011. [8] I. Necoara, V. Nedecu, and I. Dumtrache. Parae and dstrbuted optmzaton methods for optmsaton and contro n networks. Journa of Process Contro, 1:756 766, 011. [9] I. Necoara, C. Savorgnan, Q. Tran Dnh, J. Suykens, and M. Deh. Dstrbuted nonnear optma contro usng sequenta convex programmng and smoothng technques. In Proceedngs of the 48 th Conference on Decson and Contro, 009. [10] R.T. Rockafear. Augmented Lagrangan mutper functons and duaty n nonconvex programmng. SIAM Journa on Contro and Optmzaton, 1):68 85, 1974. [11] R.T. Rockafear and R. Wets. Varatona Anayss. Sprnger, 1998. [1] A. Tankawa and H. Muka. A new technque for nonconvex prmadua decomposton of a arge-scae separabe optmsaton probem. IEEE Transactons on Automatc Contro, 1985.